General

The DNS Vulnerability and the ISP

We called three experts at major internet security and DNS companies. We asked them what ISPs should know about the problem, and what they should do.

by Alex Goldman ISP-Planet Managing Editor [August 18, 2008]

By now you know about the DNS vulnerability. You know that the vulnerability was announced last month and Kaminsky made public the details on August 8 at the Black Hat conference (.ppt presentation, 107 pages).

The vulnerability itself was known about for some time, explains Patrick Peterson, director of technology at San Bruno, Calif.-based security appliance vendor IronPort (part of Cisco). What Kaminsky did, he says, is show how the vulnerability could be exploited.

"It's like Kryptonite locks. People knew there was a vulnerability, but the problem became more serious when someone showed how to open the lock with a Bic pen cap."

An invisible attack
It's a subtle attack. People don't have to click on an attachment if the cache is poisoned. Instead, they could just open a bookmark and find it resolves to a criminal's website, points out Bruce Van Nice, director of corporate marketing at DNS provider Nominum.

By the way, on slide 5 of his presentation to Black Hat, Kaminsky complimented Nominum's effective handling of the problem. He noted that Nominum had protected 120 million users, which he said was 42 percent of broadband subscribers (London-based Point Topic estimates there were 367.7 million broadband subscribers as of Q1 2008, including cellular broadband subscribers, but protecting a third of the internet is still a major achievement).

Kaminsky writes, "that was a lot of work. The IT guys stepped up."

You should step up too, if you haven't done so already.

Why? Precisely because the user doesn't have to do anything unusual, such as open an attachment, in order to get infected. "It's dangerous for even savvy, security-minded users," Van Nice notes.

Other reasons? DNS is a fundamental service. It effects everything else. It's a point we'll come back to repeatedly in this article.

If the DNS cache gets poisoned, any other service can be at risk. On slides 52-54 of his presentation, Kaminsky notes, "the internet is more than the Web; HTTP is more than the browser."

Internet telephony, music, instant messaging, and other services also use DNS.

Rick Yazwinski, principal engineer for Toronto-based Tucows, which is a registrar, certificate reseller, and application provider, is acutely aware of the risk to other applications. "Most people think this is just about the Web. It's also potentially about mail, and about any network service that has to look up where any other thing is on the internet."

That's business services such as VPNs and remote databases and customer services such as online games. To Kaminsky, this Flash call, AllowInsecureDomain(), is indicitive of the problem. Almost no website or service has complete control of every component. Everyone on the internet depends in part on somebody else, and therefore nobody is in full control of the level of security that they deliver.

We'll return to this point when we talk about DNS SEC, which is the long term solution to the problem, but for now, let's look at the patch, which is your immediate solution to the problem.

It's an important patch.

The patch
When Kryptonite realized its locks were vulnerable, the company updated its product—at a serious cost. Similarly, when Kaminsky notified the security community about the problem, researchers developed a patch.

Essentially, the flaw attacks a DNS server by asking for a nonexistent domain and then providing an answer from another computer. The DNS server then starts to trust the attacker as a source of DNS information. In the past, requests were protected by a request ID, usually between 0 and 65535 (2¹⁶ possibilities). In the past, all DNS traffic came and went from port 53. Now, companies are advising anyone running a DNS server to randomize the request source port. This should make attacks more difficult.

Flaws in randomization made some DNS servers more vulnerable than others, a point we'll come back to later. If you are an ISP and are uncertain about your DNS infrastructure, you can use OpenDNS, which is free but ad-supported.

"Source port randomization does not solve the problem," warns Peterson. "It just decreases the probability of a successful attack."

Making the attack more difficult should increase the number of packets in an attack, notes Yazwinski of Tucows. "Before, an attack required 65,000 packets. Now it requires 2 billion. 2 billion anomalous transactions are more easily detected than 65,000."

Want to test your DNS server to see if it's patched? There are tools that can do that. Petersen points to several that test the quality of the randomization on your server at DNS OARC and at DNSStuff. Kaminsky's blog, Dox Para, has a simpler test.

Monitoring your own DNS
What else should an ISP do, besides patch their DNS? Because the attack sends requests for URLs that your cache cannot have, you'll see a spike in what's known as "cache misses," requests for IP addresses that were not already stored.

"People can measure the cache miss to cache hit ratio," says Peterson. "In the event of a Kaminsky attack, they'll see a massive number of DNS cache misses."

Kaminsky notes that some DNS implementations were returning sequential query IDs instead of random ones. The DNS tests above measure the randomness of your DNS server. What must have seemed like a useful shortcut at the time became a security hole.

DNS SEC
Everyone I have spoken to agrees that DNS SEC is the long term solution, but points out that it doesn't work perfectly until everyone has implemented it.

"There's a political element," says Van Nice. "Who needs to go first? ICANN is taking actions that will, hopefully, ignite acitivity on this front."

Petersen points out that if a nice local ISP like Sonic.net implements it, but .com does not, there could be little immediate benefit to someone accessing ISP-Planet.com through Sonic.

Yazwinski says that the U.S. government now supports DNS SEC, and points out that "exploits like this push the entire net in the right direction."

In conclusion: DNS SEC is the answer, but it won't solve our problems today.

Long term consequences
The long term danger is not this specific vulnerability or this specific attack, but a whole new kind of attack inspired by the Kaminsky flaw.

"At Tucows, a few years ago we had a 6 Gbps DDoS attack," says Yazwinski. Such attacks, he notes, use brute force, and although it's difficult to defend against them, and no defense is perfect, everyone has built defenses that consist of redundant systems in multiple locations.

With this flaw, you might not even know you were under attack.

So are we not taking security seriously enough?

"Definitely. Somebody could put up a website that looks like a bank, with a self-signed cert. Anyone caught in their trap would think they were at the bank unless they look at the certificate, and I don't think the average user is that savvy."

Every app was threatened. "The level of trust that exists on the internet may be tarnished. People need to understand they're not as safe as they thought they were."

Similar attacks are possible against other parts of the internet, and against other services. In his Black Hat presentation, Kaminsky mentions SIP but decides "not to go there."

The DNS system will be attacked again too. "Everyone now recognizes that DNS touches a tremendous number of things on the internet," says Nominum's Van Nice. "It would be easier to count the things it doesn't touch."

"Until DNS SEC is adopted, expect regular fire drills," concludes IronPort's Peterson.

—End