Log Analysis Tools
On a busy cache, log files grow large very quickly. Access logs are usually rotated on a regular interval and transferred elsewhere for summarization, archival, and analysis. Post-processors that understand common log format, squid log format, or both can be used to summarize data and produce statistical reports.

Numerous analysis programs are freely available for use with squid logs. One index of squid log analyzers can be found at http://www.squid-cache.org/Scripts. A few examples are:
 

NLANR access_extract and summary.pl http://www.squid-cache.org/Scripts/NLANR	Basic squid scripts produce text reports
Calamaris http://calamaris.cord.de	Produces text or HTML reports
Squij http://www.pobox.com/~mnot/squij	Summarizes service time, hit rate and count, and vended object freshness
Squeezer http://www.geocities.com/maciej_kozinski/w3cache/squeezer.html	Analyzes cache traffic (transfer size and speed) and sibling efficiency

Squid log tools are obviously focused on cache performance analysis. But because the common log format has broader applicability, one can find many open source and commercial web server log analyzers. Such tools even have their own page at Yahoo. One comprehensive list of access log analyzers can be found at http://www.uu.se/Software/Analyzers/Access-analyzers.html. Some common log format analyzers:

Analog http://www.statslab.cam.ac.uk/~sret1/analog	Produces reports for several log formats
WebTrends Log Analyzer http://www.webtrends.com/products/Log	Produces reports in Word, Excel, and text format
Wusage http://www.boutell.com/wusage	Produces periodic log stats, offers GUI config dialog
Webalizer http://www.mrunix.net/webalizer	Produces usage stats in configurable HTML reports

Because the squid log format is (almost) a superset of the common log format, conversion from squid to common log format can be accomplished easily with a script like squid2common.pl. Conversion in the reverse direction is less useful because cache-specific information is missing. For this reason, admins may generate access logs in squid format, but convert them to common format when using web server log analysis tools. Some products support custom log formats that allow you to establish your own balance between resource usage and information gathering.

 

Introduction/Common Log Format

Squid Log Format

HTTP Status Codes

Log Analysis Tools