Executive Summary: Spam Index

The latest report from Brockmann & Company describes the anti-spam technologies currently in use, shows how to compare them, and issues a recommendation.

by Peter Brockmann Brockmann & Company [August 14, 2007]

False-Positive Epidemic
How many spam messages a day are too many? Five? Ten? Twenty? Fifty? A hundred? How about zero? Do e-mail users today really have to put up with lousy e-mail quality? We think not.

Brockmann & Company research shows that e-mail is consistently recognized as the most important communications service affecting job performance in all manner of organizations, all manner of industries and all manner of roles. E-mail is also 50 percent more important than mobile voice services, two times more important than desktop telephone service and 10 times more important than fax.

Yet, despite this very important role for the service, spam continues to detract from the users' experience, and risk commercial harm with False-Positives (good e-mail treated as spam) from over-zealous filters.

The Spam Index
is a new anti-spam measurement tool based on the actual experience of business e-mail users. First appearing in the May 2007 Brockmann & Company report, The Problem with E-mail, the Spam Index was the key to identify both Top Performers and Poor Performers and contrasted their business performance.

Here's how to calculate your Spam Index:

• Approximate number of spam delivered to inbox each day? A= Multiply this by 20 =
  
  
• Approximate minutes spent dealing with spam each day? B= Multiply this by 20 =
  
  
• Approximate number of message resend requests received or made last month? C =
  
  
• Approximate number of good messages trapped by spam filters last month? D =

Spam Index = A+B+C+D =

Applications
1) The Spam Index can measure the effectiveness of changes in the e-mail system. For example, before and after implementing or testing a new anti-spam solution, users should calculate their Spam Index. This way the e-mail administrator and the business users they serve can evaluate a solution's true effect on the e-mail experience and the costs of dealing with spam.

2) The Spam Index is an industry-wide comparative tool. Users can compare their Spam Index against a panel of over 500 other business people from around the world. Is their experience the same, better or worse than 75 percent, 90 percent of others? These comparisons can help justify the investment of time and resources to initiate, scope and implement a change in the e-mail system.

3)The Spam Index is useful to compare the effectiveness of eight anti-spam technologies. The Brockmann & Company study determined the average Spam Index performance of each technology, the variability of results for users of each technology and the level of user satisfaction with their e-mail experience.

Technologies reviewed include:

Challenge-Response: This technology allows e-mail from known senders to pass directly to protected users. In some cases, outbound e-mail is monitored to add those addressees to the known sender list. First time senders, which include virtually all spam, are challenged with a reply e-mail requesting that the sender reply to the message in order to assure the original e-mail is delivered. Vendors in this category include Sendio and SpamArrest.

Hosted Processors: Hosted e-mail filtering services use the readily available processing power of Internet data centers. Vendors in this category include AppRiver, MessageLabs, MXLogic, and Google-Postini, among others.

Filter Appliance: This class integrates software with a hardware appliance, tuning the package for optimal processing performance. Vendors in this category include Barracuda, Borderware, McAfee, and Secure Computing, among others.

Commercial Filters: This class tends to be virus filtering software with extended anti-spam functionality. Vendors that offer this class of anti-spam software include McAfee, Symantec, and TrendMicro, among others.

Real-time Black Lists: These reputation based systems collect feedback from users to manage a 'black list' of known spammer IP addresses and domains. Vendors in this category include Commtouch, IronPort, and Spamhaus, among others.

PC E-mail Client Filters: This spam filter is a feature of most e-mail clients. Users can train their client spam filters to move or delete mail meeting certain conditions. Vendors in this category include Apple, IBM Lotus, and Microsoft, among others.

Open Source Filters: This class of software includes projects such as ASSP, Mail Washer, and SpamAssassin, among others.

ISP-based Filtering: Offered by e-mail hosting service providers, this anti-spam technology provides anti-virus and anti-spam filtering for hosted domains.

Average Spam Index (lower numbers are better)

Recommendation
Challenge-Response technology Challenge-response technology earns the Brockmann & Company recommendation because of real-world end user performance:

• Lowest average Spam Index
• Most consistent Spam Index
• Highest user satisfaction with the e-mail experience

Key vendors to consider: Sendio, SpamArrest

Best Performing Technologies:

1. Challenge-Response
2. Hosted Processor Service
3. Filter Appliances

Learn More...
The Spam Index Report, The Problem with E-mail and other benchmark reports are available for free download at the Brockmann & Company website.

— End