WWW Access Log

• Analyze and count httpd access_log
• Make a graph
  • Count lines at a fixed time
  • Real-time counting

Make a graph

Count lines in a log-file at a fixed time

Firstly we write a perl program which counts the number of access
to a web-site, and makes a figure automatically in an image format
like GIF or PNG. This program is executed regularly (once a week or
month) to update the figure. Now we extend the
webplot.pl program in the previous section.

$logfile='/path/to/access_log';
$gnuplot='/path/to/gnuplot';
$imgfile='/path/to/access.png';

We firstly need to know the place (PATH) of some files and gnuplot
itself. Those are defined at the top of the program. The name of image
file which is made by gnuplot is arbitrary but you’d better to give an
appropriate extension. The example above is ‘png’.

open(LOG,$logfile);
while(<LOG>){
if(/.html/){
split;
$day = substr($_[3],1,2);
$mon = substr($_[3],4,3);
$year= substr($_[3],8,4);
$count[$day]++;
}
}
close(LOG);

$day=$#count;
$count[$day+1]=$count[$day];

This part is almost the same as the previous one, but the input is
not STDIN but the log file. I also added two variables, $mon,
$year, in order to make a title of graph. I hope your log-file
is renewed (rotated) every month.

In order to draw the last day of a month (31th) with the histgram,
we added the last element of $count, and that is the same as
one before.

& make_gnuplot;
open(GNUPLOT,"| ".$gnuplot);
foreach $i (0..$#plot){ print GNUPLOT $plot[$i]; }
for($i=1;$i<=$#count;$i++){
printf(GNUPLOT "%10d%10dn",$i,$count[$i]);
}
print GNUPLOT "endn";
close(GNUPLOT);
exit 0;

This part make a figure of the number of access. Commands given to
gnuplot are stored in the array $plot, which is prepared
in the next subroutine.

sub make_gnuplot{
$i=0;
$plot[$i++]=sprintf("set term png colorn");
$plot[$i++]=sprintf("set output '%s'n",$imgfile);
$plot[$i++]=sprintf("set size 0.7,0.7n");
$plot[$i++]=sprintf("set xrange [0:31]n");
$plot[$i++]=sprintf("set yrange [0:*]n");
$plot[$i++]=sprintf("set xtics 1,7,31n");
$plot[$i++]=sprintf("set mxtics 7n");
$plot[$i++]=sprintf("set nokeyn");
$plot[$i++]=sprintf("set gridn");
$plot[$i++]=sprintf("set title '%s %s'n",$mon,$year);
$plot[$i++]=sprintf("plot '-' with stepn");
}

Here our image format is PNG. The lines set size and
set xtics change its appearance. The default size of PNG is
640×480 pixels, and this is reduced by 70%. The last line means that
numerical data to be plotted come from STDIN. With this technique you
can make a graph of $count without any temporary files.

This script is invoked by ‘cron’ once a week, for example, to make a
graph of access.