ISPs Wary of Peering Crossfire

ISPs cannot control backbone disagreements, but understanding the cause of the problem empowers each ISP develop their own solution.

by Jim Thompson [December 16, 2005]

The recent conflict between Cogent Communications and Level 3 Communications over peering agreements (see The Cogent-Level 3 Dispute) disrupted service to tens of thousands of innocent customers. A peace settlement was reached but this may be only the first salvo in a fight that could bring federal regulation and escalate into a major industry war. When Titans clash, many are trampled.

Peering agreements have been at the heart of heated discussions and disputes since the emergence of the commercial Internet. Yet, aside from the occasional sword rattling there seemed little reason for worry . Most saw backtalk and veiled threats as simple part of the long-haul backbone business.

But all that changed on October 5, 2005 when Level 3 Communications cut off their link to Cogent Communications. The result was that tens of thousands and, perhaps, millions of customers suddenly lost access to significant portions of the Net.

Cease fire agreement
Fortunately, the shutdown was short-lived, but it did get the attention of other Tier 1 carriers, ISPs, and the federal government.

It should be noted that on October 28, 2005, both parties agreed to a "modified version of their original peering agreement." While the specifics were not disclosed, a press release from Cogent Communications stated that "the modified peering arrangement allows for the continued exchange of traffic between the two companies' networks, and includes commitments from each party with respect to the characteristics and volume of traffic to be exchanged."

Agreement or not, the situation begs the question: "how could such a thing happen?" After all, the decentralized and redundant Internet was supposedly created to prevent this very thing. In an interlocking network of networks, data traffic is supposed to be able to always find an alternate route to its final destination.

That's what is supposed to happen—at least in theory. But, in practical terms, blockage between backbone networks can still impact Internet traffic and site availability.

The problems are compounded as demand escalates for high speed connections, large data transfers, and real time applications such as live video on demand. You might be able to reroute a connection around a blockage, but that new route could make dozens of hops, many of them over small, slow, and technically questionable networks and machines. As a result, transferring a video or large file, could take hours or days even if you are connected via DSL, cable, or fiber.

The peering process
Peering is a process in which the big players like MCI, AT&T, AOL, Sprint, Level 3, Cogent, etc. agree to establish direct connections between their networks. With such peering agreements in place, if a customer of a company like Cogent wants to access data or visit a web site hosted by AOL or Level 3, the request is sent directly via the fastest pathway instead of being sent on a much longer journey around the broader Internet.

Historically, peering agreements consisted of a simple exchange of services between big backbone Internet companies with no payments or money exchanged. The basic understanding was that, since each company was a major player, they would be exchanging roughly the same amount of data. The smaller players pay fees (called Internet Transit Agreements) for the right to hand-off traffic to the big networks.

Once an agreement is reached, computer programs monitor traffic, allocated sufficient bandwidth, route traffic, and keep tabs on how much data is sent and received by each network.

Both Level 3 and Cogent use direct connections to other networks so there is often no alternate route for data to travel. If a cutoff occurs, data enters a transmission black hole or the electronic equivalent of a roach motel—data checks in, but it can't check out.

As long as everyone agrees to the basic rules, traffic flows efficiently, customers get what they want at the speeds they expect, and the networks have a steady flow of income. But, if instead of enjoying ride, someone decides to start tinkering with what's under the hood, problems can follow.

In some cases it's the tech staff complaining that they are receiving more traffic then they are sending resulting in strain on the hardware and the support personnel. Sometimes, it's the ever vigilant accountants worried about the company's bottom line who argue that incoming traffic should mean increased revenue.