Introduction
Assuming you have varnish installed on your server and you started to catch the pages on it.Your webserver is now missing the hits to your pages.
Well, that is great for your webserver load, but not too good for your statistics analysis, because /var/log/apache/access_log
will miss the hit.
Configure varnishncsa:
We’ll use varnishncsa
to get the logs that awstats
will be able to analyse like:
varnishncsa [?a] [?b] [?C] [?c] [?D] [?d] [?f] [?I regex] [?i tag] [?n varnish_name] [?P file] [?r file] [?V] [?w file] [?X regex] [?x tag]
Add this line in the /etc/rc.local
file:
varnishncsa -a -w /var/log/varnish/access.log -D -P /var/run/varnishncsa.pid
-a: To append the logs to an already existing file
-w: To write the logs to the/var/log/varnish/access.log
file
-D: To runvarnishncsa
as a daemon
-P: To write the PID file in the/var/run/
folder
Configure logrotate:
Create the following file /etc/logrotate.d/varnish
and put this contents on it:
/var/log/varnish/*log { create 640 http log compress postrotate /bin/kill -USR1 `cat /var/run/varnishncsa.pid 2>/dev/null` 2> /dev/null || true endscript }
If you need more info or options run: man logrotate
We now have varnish writing logs to a file, and logrotate will rotate them everyday, we only need now to analyse them.
Configure awstats with varnish:
Download and install awstats
Get the lastest version from here, install it basically all you need to do is to copy the contents of the .tar.gz file in /usr/local/awstats/
folder.
Next run the awstats_configure.pl
tool. Skip (write none) in the webserver config file question, then follow the questions and you will end up with a config file like: /etc/awstats/awstats.your-domanin.conf
and should look more or less like this:
LogFile="/var/log/varnish/access.log" LogType=W LogFormat=1 LogSeparator=" " SiteDomain="www.debian-tutorials.com" HostAliases="debian-tutorials.com www.debian-tutorials.com 127.0.0.1 localhost" # Possible values: # 0 - No DNS Lookup # 1 - DNS Lookup is fully enabled # 2 - DNS Lookup is made only from static DNS cache file (if it exists) # Default: 2 DNSLookup=1 DirData="/var/lib/awstats" DirIcons="/icon" AllowToUpdateStatsFromBrowser=0 AllowFullYearView=2
Be sure the read the example located at /usr/share/doc/awstats/examples/awstats.model.conf.gz to fine tune your configuration.
Make awstats update hourly:
Create the file /etc/cron.hourly/awstats
and copy these lines inside the file:
#!/bin/sh /usr/lib/cgi-bin/awstats.pl -update -config=www.your-domain.com
That will update your awstats database hourly, let’s now make if visible, better only for you:
Copy this folder to your www root directory:
cp -R /usr/share/awstats/ /var/www-root/
Now create
etc/apache2/awstats.conf file and add this lines:
ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ ScriptAlias /awstats/ /usr/lib/cgi-bin/ Options ExecCGI -MultiViews +SymLinksIfOwnerMatchAllowOverride None Options None AuthType Basic AuthName ‘Private scripts’ AuthUserFile ‘/www-root/.htpasswd’ Require valid-user Order allow,deny Allow from all
Now and add the following line at the end of the /etc/apache2/apache2.conf file :
Include /etc/apache2/awstats.conf
Create a user that will be able to read the stats:
httpasswd
/www-root/.htpasswdusername
NOTE: Replace /www-root/.htpasswd with the file and folder where you want the password file to be in, but be sure the file and upwards folders are owned by the user that runs your Apache server.
Final Step:
Restart Apache, and start varnishnsca.
/etc/init.d/apache2 restart varnishncsa -a -w /var/log/varnish/access.log -D -P /var/run/varnishncsa.pid
Debian (and ubunty) by default will drop with a version of varnishncsa with an init script. Just edit /etc/default/varnish to set enabled to ‘yes’ or ‘1’, start it (/etc/init.d/varnishncsa start), and then you can grab your logs from /var/log/varnish/varnishncsa.log