Global Switch - SDC Interconnect failure

Here we will post network problems, Planned & Unplanned downtime as well as restoration times and other network issues.

Global Switch - SDC Interconnect failure

Postby matt » Tue Dec 10, 2013 2:31 pm

Global Switch - Sydney Data Center Interconnect failure

At 5:38 AM Spectrum automatic system detected flapping of both the primary and secondary 10 gig interconnect between The George Street data center and Global Switch Data center. These flaps interrupted multiple services on the Spectrum MPLS & Ethernet core's as both diverse circuits were interrupted at the same time.

These flaps continued until 6:18 when both circuits went hard down terminating both the primary and backup interconnect between the two data centers.

At approximately 7:23 AM spectrum staff identified that the primary and backup interconnects had been transposed and reversed the transposition restoring both the primary and backup network circuits to working capability.

Spectrum staff updated the fiber provider and requested information about the outage. The fiber provider advised that a mix up in scheduling interrupted both the backup and primary circuits at the same time during the same outage window. Human error resulted in the fiber pairs being transposed at the same time extending the outage until Spectrum Staff identified that the transposition had occurred.

Spectrum staff have begun a project to re-route the backup circuit via a different fiber provider. Spectrum have also asked the fiber provider to provide more feedback as to how they will improve there management processes.
User avatar
matt
Site Admin
 
Posts: 325
Joined: Thu Apr 09, 2009 11:44 am
Location: George Street Sydney

Return to Service Availability & Announcements

Who is online

Users browsing this forum: No registered users and 1 guest

cron