DSL Prime: Build Your Own NSA Computer

Supercomputers are cheap, and every phone call record can be stored in much less than a petabyte.

by Dave Burstein of DSL Prime and Future of TV [May 19, 2006]

The computer that can track every call in the U.S. for a year fits in a large room. I could build it for less than $300,000 as a Linux cluster, mostly out of parts available at my neighborhood COMPUSA. "I'm shocked by the revelation about the National Security Agency," was a very silly comment by Sen. Patrick Leahy, who would probably also be shocked to discover gambling at Rick's Place in Casablanca. We all knew this kind of thing was going on. I wrote recently, "Telcos massively help governments to spy, whether in the U.S., France, or China." I quoted One Who Knows saying, "Since 9/11, everything we couldn't do before is permitted. Now, they always say yes, or we don't even have to ask."

Shocked or not, the obvious revulsion of the reporters covering this invasion of privacy is not echoed by the American people. Arshad Mohammed notes that in a "Washington Post-ABC News poll released yesterday, 63 percent of Americans said they found the NSA program to be an acceptable way to investigate terrorism, including 44 percent who strongly endorsed the effort. Another 35 percent said the program was unacceptable, including 24 percent who strongly objected to it." (ed. note: the poll may have had leading questions.)

I was however shocked when I heard that SBC's DSL coverage is only 76 percent, after hearing for years they were past 80 percent. Verizon is equally far behind claims. This led me to lie to the Chairman of the FCC, giving him an estimate of homes unserved about 25 percent too few. The latest FCC data are much more accurate, and make clear the key obstacle in achieving "affordable broadband for all in 2007" is the asset stripping of the telcos about to sell millions of lines. West Virginia is 43 percent unserved, New Hampshire 35 percent, Maine 30 percent—all states the Dionne Searcy and Peter Berman report Verizon is selling. SBC's capital spending is currently only 70 percent of depreciation. Unless you believe they have $15 billion in capital equipment that should be written off, that means they aren't even maintaining their network.

Stories to come:

• The Black Panther and the CEO: $1 million from SBC's Ed Whitacre to now Congressman Bobby Rush.
• Free SkypeOut and AIM phone numbers
• Alcatel: "We are transferring a lot of competence and (R&D-related) workload to China from Europe and, especially, the United States," as they hire 300 more engineers in Sichuan.
• DSL to Samoa
• Wireless: Watch Clearwire. Shelly Palmer at Columbia asserted, DSL and cable will die in the wireless era. I disagree, but it could be a contender. Think of the power of a combined wireless, satellite, + 250 gig PVR for video. BPL in Texas is also going to a million homes, with most of the cost to be covered by efficiencies in electric power.
• Why IP TV in France costs seven euros and less. Jennie's about to interview one of the key players.
• Betrayals A guest editorial from Professor Susan Crawford on rising phone prices.
• Reviews of Fundamentals of DSL Technology (all geeks should buy) and Dan Reingold's Confessions of a Wall Street Analyst (everyone should buy)

Build your own phone record computer
So cheap every country can do it
A record of who called whom, when, and for how long for a single phone call fits easily in 30 bytes. Assuming 200 million callers in the U.S. and ten calls a day, a year's file is 23 terabytes. Adding indexes and a duplicate for backup is still under 100 terabytes.

Google has constructed the equivalent of dozens of these around the world. A picture of a six year old 18 terabyte server fits in five racks. A similar unit with 2006 technology would hold 100 terabytes. 20 racks could hold every call since 2001 and fit in a large, well air-conditioned room.

More likely the actual unit cost much more, fitting a mainframe style that offers smoother operations and marginally faster operation. Because it is now so cheap and easy enough to store this information, expect many other countries to follow. If I were a cop, I'd love to have this info.

Copyright 2006 Dave Burstein.
The DSL Prime Newsletter is reprinted with permission.

"The power of the printing press belongs solely to those who own the presses"
—A.J. Leibling

The Internet is the cheapest printing press ever invented.