As per the title, this is for IIS7 and rather than manually trawling through the IIS Manager to find all the site ID numbers so that I can get to the log files, this command exports them to a spreadsheet / text file.
At the command prompt on the Windows Server – noting that you will probably need to run the Command Prompt As Administrator
[bash]
%windir%system32inetsrvappcmd list site > c:sites.xls
[/bash]
Then get the file by copying or whatever to a machine with Excel on it and open it up.
There are a number of websites and servers that I want to track stats for.
Each individual CPanel includes Awstats by default, but I have a number of dedicated servers that don’t have cpanel, and clients that want stats access who I don’t want to give access to the CPanel too (yep, I don’t trust them!)
So this is about installing a stand-alone Awstats with all features and a login process for the various clients to view the relevant areas or servers and domains that will be monitored.
This will include:
First up the Awstats server is a Linux Debian Squeeze (6.0.6) setup and installing Awstats is the starting point.
I referenced a number of sites including:
and all of them have helpful information but not quite aligned to what I found on my server.
The simple command for getting Awstats is:
[bash]
apt-get install awstats
[/bash]
Next there are some fix ups and notes:
To start with the apt-get does not unzip the model file or create the appropriate directories. (Considering that this has been documented for around 5 years I am wondering why someone has not fixed it. I might even give it a try myself, but the tweaks are minor)
So create the correct directories in /usr/share/doc/awstats/examples#
[bash]
/usr/share/doc/awstats/examples# mkdir -p wwwroot/cgi-bin
[/bash]
Next copy the .gz file to the new directory
[bash]
/usr/share/doc/awstats/examples# cp awstats.model.conf.gz wwwroot/cgi-bin
[/bash]
Next unzip the .gz file
[bash]
/usr/share/doc/awstats/examples# cd wwwroot/cgi-bin
/usr/share/doc/awstats/examples/wwwroot/cgi-bin/#gunzip awstats.model.conf.gz
[/bash]
Running the configure script for Awstats is next but check that we have appropriate answers for the questions:
The Apache default config file path is /etc/apache2/httpd.conf but, I prefer to use the sites-available construct and have a distinct config for the sites on the server, so.. /etc/apache2/sites-available/001-awstats will be the config file that I use
The site name is whatever your site is and I’ll use the default web site for this server just to get it going
[bash]
/usr/share/doc/awstats/examples# perl awstats_configure.pl
—– AWStats awstats_configure 1.0 (build 1.8) (c) Laurent Destailleur
—– This tool will help you to configure AWStats to analyze statistics for one web server.
You can try to use it to let it do all that is possible in AWStats setup, however following
the step by step manual setup documentation (docs/index.html) is often a better idea.
Above all if:
– You are not an administrator user,
– You want to analyze downloaded log files without web server,
– You want to analyze mail or ftp log files instead of web log files,
– You need to analyze load balanced servers log files,
– You want to ‘understand’ all possible ways to use AWStats…
Read the AWStats documentation (docs/index.html).
—–> Running OS detected: Linux, BSD or Unix Warning: AWStats standard
directory on Linux OS is ‘/usr/local/awstats’.
If you want to use standard directory, you should first move all content
of AWStats distribution from current directory:
/usr/share/doc/awstats/examples to standard directory: /usr/local/awstats
And then, run configure.pl from this location.
Do you want to continue setup from this NON standard directory [yN] ?
ANSWER = y
—–> Check for web server install
Enter full config file path of your Web server. Example: /etc/httpd/httpd.conf
Example: /usr/local/apache2/conf/httpd.conf
Example: c:Program filesapache groupapacheconfhttpd.conf
Config file path (‘none’ to skip web server setup): >
ANSWER = /etc/apache2/sites-available/001-awstats
—–> Check and complete web server config file ‘/etc/apache2/sites-available/001-awstats’
All AWStats directives are already present.
—–> Update model config file ‘/usr/share/doc/awstats/examples/wwwroot/cgi-bin/awstats.model.conf’
File awstats.model.conf updated.
—–> Need to create a new config file ?
Do you want me to build a new AWStats config/profile file (required if first install) [y/N] ?
ANSWER = y
—–> Define config file name to create What is the name of your web site or profile analysis ?
Example: <a href="http://www.mysite.com">www.mysite.com</a>
Example: demo Your web site, virtual server or profile name: >
ANSWER = default-web-site.your-domain.tld.somewhere
—–> Define config file path In which directory do you plan to store your config file(s) ?
Default: /etc/awstats Directory path to store config file(s) (Enter for default): >
ANSWER = [Enter for Default]
—–> Create config file ‘/etc/awstats/awstats.default-web-site.your-domain.tld.somewhere.conf’
Config file /etc/awstats/awstats.default-web-site.your-domain.tld.somewhere.conf created.
—–> Add update process inside a scheduler
Sorry, configure.pl does not support automatic add to cron yet.
You can do it manually by adding the following command to your cron:
/usr/share/doc/awstats/examples/wwwroot/cgi-bin/awstats.pl -update -config=default-web-site.your-domain.tld.somewhere
Or if you have several config files and prefer having only one command:
/usr/share/doc/awstats/examples/tools/awstats_updateall.pl now
Press ENTER to continue…
A SIMPLE config file has been created:
/etc/awstats/awstats.default-web-site.your-domain.tld.somewhere.conf
You should have a look inside to check and change manually main parameters.
You can then manually update your statistics for ‘default-web-site.your-domain.tld.somewhere’ with command:
> perl awstats.pl -update -config=default-web-site.your-domain.tld.somewhere
You can also read your statistics for ‘default-web-site.your-domain.tld.somewhere’ with URL:
> <a href="http://localhost/awstats/awstats.pl?config=default-web-site.your-domain.tld.somewhere">http://localhost/awstats/awstats.pl?config=default-web-site.your-domain.tld.somewhere</a>
Press ENTER to finish…
[/bash]
Next edit the apache config file that was created by the script, in my example it is /etc/apache2/sites-available/001-awstats
Add this section for the awstats-icon alias. I suggest near the top of the file after the CGI script directives rather than the bottom.
[text]
Alias /awstatsicons/ /usr/share/awstats/icon/
<Directory /usr/share/awstats/icon>
Options None
AllowOverride None
Order allow,deny
Allow from all
</Directory>
[/text]
There may be a pointer in the file already for Alias
[text]
Alias /awstatsicon "/usr/share/awstats/examples/wwwroot/icon/"
[/text]
If there is then it is incorrect. Just use a # at the start of the line to disable it.
To complete the Apache config enable the new 001-awstats site and restart apache2
[bash]
/etc/apache2/sites-enabled# a2ensite
/etc/apache2/sites-enabled# apachectl graceful
[/bash]
You can also use “service apache2 graceful” with same outcome
Next in the /etc/awstats/ directory find the awstats.your-domain-name.conf file that was created earlier.
Edit the file around line 51 MAIN SETUP SECTION and change the log file location for the default web site
from
[text]LogFile="/var/log/httpd/mylog.log"[/text]
to
[text]LogFile="/var/log/apache2/access.log"[/text]
everything else should be ok as a default.
So now we have a configured Awstats for a basic test. Try running this command
[bash]/etc/awstats# /usr/lib/cgi-bin/awstats.pl -config=default-web-site.your-domain.tld.somewhere[/bash]
Remember that the -config option here must match the awstats.default-web-site.your-domain.tld.somewhere.conf of the file created earlier. Except do not include the awstats. or the .conf as this prefix and suffix are automatically assumed by the application.
The response should look something like this:
[bash]
Create/Update database for config "/etc/awstats/awstats.default-web-site.your-domain.tld.somewhere.conf" by AWStats version 6.95 (build 1.943)
From data in log file "/var/log/apache2/access.log"…
Phase 1 : First bypass old records, searching new record…
Searching new records from beginning of log file…
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)…
Jumped lines in file: 0
Parsed lines in file: 28
Found 0 dropped records,
Found 0 corrupted records,
Found 0 old records,
Found 28 new qualified records.
[/bash]
Finally the test is to check that it is alive ! In a web browser try the following substituting the example domain in the URL and as the config to check.
[text]
http://default-web-site.your-domain.tld.somewhere/cgi-bin/awstats.pl?config=default-web-site.your-domain.tld.somewhere
[/text]
Ok, so that works.
Now to configure a process to update the stats from the logs
awstats_updateall.pl is the perl script that will do this, but it is sitting in an odd place under examples. I moved it just to have it in a more logical place alongside the awstats.pl script. This is not essential.
[bash]
#cd /usr/share/doc/awstats/examples/
/usr/share/doc/awstats/examples# cp awstats_updateall.pl /usr/lib/cgi-bin/
[/bash]
There is a whole commentary on the sequencing of rotating logs and stats updates at http://awstats.sourceforge.net/docs/awstats_faq.html#ROTATE
For what I need using the Apache Logrotate process will suit. Modify the /etc/logrotate.d/apache2 file
[bash]
# cd /etc/logrotate.d/
/etc/logrotate.d/# mcedit apache2
[/bash]
Insert the following 3 new lines just after the sharedscripts line and before the postrotate line.
[text]
prerotate
/usr/lib/cgi-bin/awstats_updateall.pl -awstatsprog=/usr/lib/cgi-bin/awstats.pl now
endscript
[/text]
I did not include the config option “-configdir=/etc/awstats/” as that is the default and it is redundant
The command “/usr/lib/cgi-bin/awstats_updateall.pl -awstatsprog=/usr/lib/cgi-bin/awstats.pl now” can be run from the command line to confirm that it is correct.
So now the awstats is running and the logrotate will update the stats every morning
Security of access to the files is the next topic and I’ve recorded that as a separate post.
Next add another site with logs. Adding another site is simply a case of identifying the log files and adding a conf file for them.
Secure access with .htacess
The default setup for awstats is really aimed at a minimal number of web site or other logs and the ability to setup specific config files for each site is good but seems a little top heavy.
The awstats.conf and awstats.conf.local files provide for the single site type setup. Configure the main .conf settings or alternatively add the specific overrides of the defaults into the awstats.conf.local file and all is good.
Using the awstats.model.conf for multiple web sites and copying it to a site specific file also is a good plan for a nominal number of sites.
But once we get to having 37 sites it is getting a bit out of hand.
So how to improve the process and automate it?
Roughly:
I think the order should be:
awstats looks in /etc/awstats when the updateall process is executed
in the /etc/awstats directory are the header files which look like:
#include main config file from somewhere else (it cannot be in this directory as it would be read as a conf file directly)
#include the required log file format – parsing both apache and windows logs
#have only the site specific instructions in the file so that the file size and chance of overriding config standards is reduced
I’ll experiment with the above and update this with the result.