TOP SECRET//COMINT//REL TO USA, FVEY (Report generated on: 4/11/2013 3:31:05PM) NewCross Program CrossProgram-1-13 New Active ECP Count: 1 ECP Lead: [REDACTED] Title of Change: Update Software on all Cisco ONS Nodes Submitter: [REDACTED] Approval Priority: C-Routine Project(s): No Project(s) Entered Site(s): APPLE1: CLEVERDEVICE : HOMEMAKER: DOGHUT : QUARTERPOUNDER: QUEENSLAND: SCALLION : SPORTCOAT: SUBSTRATUM: TITAN POINTE: SUBSTRATUM: BIRCHWOOD: MAYTAG: EAGLE: EDEN: SubSystem(s): No Subsystem(s) Entered System(s): Comms/Network: Comms/Network: Comms/Network: Comms/Network: Description of Change: Udate software on all Cisco Optical Network Switches. Reason for Change: All of our Cisco ONS SONET multiplexers are experiencing a software bug that causes them to intermittently drop out. Mission Impact: The mission impact is unknown. While the existing bug doesn't appear to affect traffic, applying the new software update could. Unfortunately, there is now way to be sure. We can't simulate the bug in our lab and so it's impossible to predict exactly what will happen when we apply the software update. We propose to update one of the nodes in NBP-320 first to determine if the update goes smoothly. Recently we tried to reset the standby manager card in the HOMEMAKER node. When that failed, we attempted to physically reseat it. Since it was the standby card, we did not expect that would cause any problems. However, upon reseating the card, the entire ONS crashed and we lost all traffic through the box. It took more than an hour to recover from this failure. The worst case scenario is that we have to blow away the entire configuration and start from scratch. Prior to starting our upgrade, we will save the configuration so that if we have to configure the box from scratch, we can simply uploade the saved configuration. We estimate that we will be down for no more than an hour for each node in the system. Additional Info: 3/26/2013 8:16:13 AM [REDACTED] We have tested the upgrade in our lab and it works well. However, we can't repeat the bug in our lab, so we don't know if we will encounter problems when we attempt to upgrade a node that is affected by the bug. Last CCB Entry: 04/10/13 16:08:11 [REDACTED] 09 Apr Blarney CCB - Blarney ECP board approved ECP lead: [REDACTED] Programs Affected: Blarney Fairview Oakstar Stormbrew No Related Work Tasks