Best of the ISP-Lists

General

Disaster's Lessons About SONET Rings

Members of the ISP-Bandwidth list pore over the lessons the events of 9-11 taught the world about designing resilient urban fiber optic networks.

[November 5, 2001]

On the ISP-Bandwidth list in October, RB queried,

"I have heard that about half of the SONET rings cut as a result of the World Trade Center attack failed, and that in some cases improper configuration or collapsed rings played a role. Anyone have insight?"

A number of respondents noted that the failures were par for the course in such a disaster:

[AC observed] "All data SONET lines were add/dropped from the West Street CO at 160 West Street, which lost a good portion of its connectivity during the WTC disaster. I don't doubt that configuration issues played a part of this, but the large cell phone tower which fell from the north tower through one of 160 West Street's main cable vaults certainly served as the major contributor to the outage. I'm sure that companies will be demanding add/drops from separate facilities in the future to prevent issues like these from occurring."

[ER added] "Because of the physical size of the World Trade Center and the magnitude of the disaster, a SONET ring could be properly configured according to a carrier's specifications, and still fail. All carriers have different specs for a proper design of a SONET ring, some more robust than others. The amount of fiber that went though that entire area was huge: the towers falling took out major infrastructure in the entire downtown area. Subway tunnels that fiber ran through were also destroyed."

[FM agreed] "The SONET rings that failed could not possibly have been designed for such an eventuality, or they would have cost so much that they wouldn't be commercially viable."

RB suggested a preventative measure for the future:

"Should SONET rings loop through two end central offices as opposed to one? Many of the rings failed because they were collapsed at the Verizon end central office on West Street. If the SONET rings had looped through two end central offices, would more of them have survived? This may not be feasible from an economic point of view, but it is a fact that they failed because they were collapsed at West street. I am trying to assess the lessons of the 11th in terms of making networks more robust."

FM contended it's just not possible to prepare for a disaster of that magnitude:

"When something so unfortunate and dramatic happens, it is normal for certain things to stop functioning. Let us try to keep things in perspective. The SONET rings can be repaired or rebuilt; the lives that were lost shall not be."

Others argued that it's always important to ensure redundancy:

[RB noted] "Clearly, when an end central office is demolished, there are going to be huge outages. The question is, can the systems be made more reliable in an economical way? Some systems failed because of pure carrier negligence. Everyone talks about reliability, but their networks often have collapsed rings all over the place, and when a customer asks to see a map, they refuse. Clients pay for reliability. They have a right to see if the carrier is going to deliver. There has been considerable research on carrier outages, and in most cases it is due either to human error or to poor network design."

[ER agreed] "If there were two Verizon COs on the rings, then they would have had a greater probability of remaining operational. It is somewhat more expensive, but it should be required for total redundancy."

—End