Caching 101: Streamlining Usenet Service

For most ISPs, Usenet is the second or third most popular service in terms of Net traffic. It's a very expensive service to offer, however. Here's how caching can reduce the cost of Usenet and improve the quality of customers' experience at the same time.

by Amit Pandey
Director of NetCache, Network Appliance, Inc.

Q: What are the main issues ISPs have today with delivering Usenet News to their customers?

A: Unlike other Web content that lives on servers hosted by content providers, Usenet content is distributed to specialized news servers around the world via news feeds. The enormous amount of data distributed by these feeds requires extensive bandwidth. A quality Usenet feed could require two full T1 lines and 200GB storage for every week's worth of historical news traffic.

To cope with this problem, ISPs typically implement Usenet in one of two ways: Larger ISPs typically centralize Usenet service at their data centers with large clusters of news servers. Smaller ISPs outsource their Usenet service to other ISPs or news providers.

The trouble with the first model is that ISPs have to maintain a full Usenet service at tremendous server, storage, and maintenance expense. What's more, as articles have to be fetched from the ISP's data center, distributed users experience poor response time while the ISP's bandwidth is used up carrying redundant requests. For smaller ISPs the problems of response time and bandwidth glut are further exacerbated as their 'news data center' is actually at another company and location.

Q: How does caching solve these problems?

A: By caching Usenet traffic, both large and small ISPs can provide high-performance news services at a fraction of the costs of deploying traditional news servers or outsourcing the services. Caching provides these Usenet related benefits:

• Faster end user response times by putting news content closer to end users;
• Reduced bandwidth and storage requirements by storing popular news articles so they only need to be fetched once, and news feeds don't have to be sent to every POP.
• Reduced hardware costs because a single cache can replace multiple NNTP servers
• A more reliable architecture for deployments in remote POPs
  

Q: How and where would an ISP deploy caches to improve Usenet delivery and reduce its administration and management costs?

A: There are three deployments ISPs can use, depending on their current Usenet service and architecture.

• ISPs that run their own Usenet service can deploy caches at the front end of a Usenet service to reduce the number of news servers needed.
• Large ISPs can provide localized Usenet service by deploying a cache at each of their POPs to reduce bandwidth and improve response times.
• ISPs who outsource their Usenet can deploy caches at their data center to save bandwidth and improve user response time.

Q: What should ISPs look for when choosing a cache to use for Usenet traffic?

A: ISPs should consider a caching architecture that can not only handle their Usenet traffic, but also manage the other types of Web traffic such as HTTP and FTP—and be able to expand as future Web protocols and applications become mainstream. Caching appliances are a strong choice for ISPs because they are easily deployable at any point in the network and they offer the price/performance, reliability, security, and ease of management that ISPs need, to provide high quality Internet access to their customers.

—End

Comments? Questions?
Contact the Editors of ISP-Planet.