[Omaha.pm] IIS log hackery

["Jay Hannah" <jhannah@omnihotels.com>]

IIS logs look like this:

> zcat 443.access_log-20050609.gz | head
66.45.77.200 - - [08/Jun/2005:04:15:20 -0500] "GET /Omni?prop=STLDTN&pagedst=AvailReq&pagesrc=Hotels HTTP/1.1" 200 23293
10.0.37.2 - - [08/Jun/2005:04:15:22 -0500] "GET /Omni HTTP/1.0" 200 22159
10.0.37.2 - - [08/Jun/2005:04:15:28 -0500] "GET /Carlton HTTP/1.0" 200 6097
66.45.77.200 - - [08/Jun/2005:04:15:32 -0500] "POST /Omni?prop=STLDTN&pagedst=AvailReq&pagesrc=Hotels HTTP/1.1" 200 21303
66.45.77.200 - - [08/Jun/2005:04:15:41 -0500] "GET /Omni?prop=SFODTN&pagedst=AvailReq&pagesrc=Hotels HTTP/1.1" 200 23304
10.0.37.3 - - [08/Jun/2005:04:15:46 -0500] "GET /Omni HTTP/1.0" 200 22159
10.0.37.3 - - [08/Jun/2005:04:15:47 -0500] "GET /Carlton HTTP/1.0" 200 6097
204.94.250.10 - - [08/Jun/2005:04:15:48 -0500] "GET /images/calendar.gif HTTP/1.1" 200 145
10.0.37.2 - - [08/Jun/2005:04:15:50 -0500] "GET /Omni HTTP/1.0" 200 22159
10.0.37.2 - - [08/Jun/2005:04:15:50 -0500] "GET /Carlton HTTP/1.0" 200 6097

Here's a couple quick deals I just did where I found myself piping to Perl:


Count all the prop=XXXXX thingies to report count per XXXXX

> zcat 443.access_log-20050609.gz | grep 209.73.169. | perl -ne 'print /prop=(\w+)/, "\n";' | sort | uniq -c
      5 ATLCNN
      3 AUSSTH
     28 BOSPAR
     25 CHIAMB
     24 CHIDTN
...etc...


Count hits by class C

> zcat 443.access_log-20050609.gz | cut -d' ' -f1 | perl -pe 's/\.\d+$//' | sort | uniq -c | sort -nr | head
  10740 10.0.37
   4789 66.45.77
   3454 209.73.169
   1820 68.142.230
   1248 12.129.73
    223 10.0.33
    187 67.79.134
...etc...


Cheers,

j