{"id":841,"date":"2013-01-11T13:58:02","date_gmt":"2013-01-11T02:58:02","guid":{"rendered":"http:\/\/howden.net.au\/thowden\/?p=841"},"modified":"2013-01-11T13:58:02","modified_gmt":"2013-01-11T02:58:02","slug":"awstats-for-many-users-from-a-single-server-debian-squeeze","status":"publish","type":"post","link":"https:\/\/howden.net.au\/thowden\/2013\/01\/awstats-for-many-users-from-a-single-server-debian-squeeze\/","title":{"rendered":"Awstats for many users from a single server Debian Squeeze"},"content":{"rendered":"<p>There are a number of websites and servers that I want to track stats for.<\/p>\n<p>Each individual CPanel includes Awstats by default, but I have a number of dedicated servers that don&#8217;t\u00a0 have cpanel, and clients that want stats access who I don&#8217;t want to give access to the CPanel too (yep, I don&#8217;t trust them!)<\/p>\n<p>So this is about installing a stand-alone Awstats with all features and a login process for the various clients to view the relevant areas or servers and domains that will be monitored.<\/p>\n<p>This will include:<\/p>\n<ul>\n<li>a mix of Windows and Linux based systems<\/li>\n<li>transfer of logs from the source servers to the Awstats server<\/li>\n<\/ul>\n<p>First up the Awstats server is a Linux Debian Squeeze (6.0.6) setup and installing Awstats is the starting point.<\/p>\n<p>I referenced a number of sites including:<\/p>\n<ul>\n<li><a href=\"http:\/\/awstats.sourceforge.net\/\">http:\/\/awstats.sourceforge.net\/<\/a><\/li>\n<li><a href=\"http:\/\/www.debian-administration.org\/articles\/277\">http:\/\/www.debian-administration.org\/articles\/277<\/a><\/li>\n<li><a href=\"http:\/\/sebs-blog.de\/awstats-on-debian-lenny.html\">http:\/\/sebs-blog.de\/awstats-on-debian-lenny.html<\/a><\/li>\n<li><a href=\"http:\/\/www.abludo.com\/guides-tutorials\/awstats-website-statistics-on-debian-installation-guide.aspx\">http:\/\/www.abludo.com\/guides-tutorials\/awstats-website-statistics-on-debian-installation-guide.aspx<\/a><\/li>\n<\/ul>\n<p>and all of them have helpful information but not quite aligned to what I found on my server.<\/p>\n<p>The simple command for getting Awstats is:<\/p>\n<p>[bash]<br \/>\napt-get install awstats<br \/>\n[\/bash]<\/p>\n<p>Next there are some fix ups and notes:<\/p>\n<p>To start with the apt-get does not unzip the model file or create the appropriate directories. (Considering that this has been documented for around 5 years I am wondering why someone has not fixed it. I might even give it a try myself, but the tweaks are minor)<\/p>\n<p>So create the correct directories in \/usr\/share\/doc\/awstats\/examples#<\/p>\n<p>[bash]<br \/>\n\/usr\/share\/doc\/awstats\/examples# mkdir -p wwwroot\/cgi-bin<br \/>\n[\/bash]<\/p>\n<p>Next copy the .gz file to the new directory<\/p>\n<p>[bash]<br \/>\n\/usr\/share\/doc\/awstats\/examples#\u00a0cp awstats.model.conf.gz\u00a0wwwroot\/cgi-bin<br \/>\n[\/bash]<\/p>\n<p>Next unzip the .gz file<\/p>\n<p>[bash]<br \/>\n\/usr\/share\/doc\/awstats\/examples#\u00a0cd \u00a0wwwroot\/cgi-bin<\/p>\n<p>\/usr\/share\/doc\/awstats\/examples\/wwwroot\/cgi-bin\/#gunzip\u00a0\u00a0awstats.model.conf.gz<br \/>\n[\/bash]<\/p>\n<p>Running the configure script for Awstats is next but check that we have appropriate answers for the questions:<\/p>\n<p>The Apache default config file path is \/etc\/apache2\/httpd.conf but, I prefer to use the sites-available construct and have a distinct config for the sites on the server, so.. \/etc\/apache2\/sites-available\/001-awstats will be the config file that I use<\/p>\n<p>The site name is whatever your site is and I&#8217;ll use the default web site for this server just to get it going<\/p>\n<p>[bash]<br \/>\n\/usr\/share\/doc\/awstats\/examples# perl awstats_configure.pl<br \/>\n&#8212;&#8211; AWStats awstats_configure 1.0 (build 1.8) (c) Laurent Destailleur<br \/>\n&#8212;&#8211; This tool will help you to configure AWStats to analyze statistics for one web server.<br \/>\nYou can try to use it to let it do all that is possible in AWStats setup, however following<br \/>\nthe step by step manual setup documentation (docs\/index.html) is often a better idea.<br \/>\nAbove all if:<br \/>\n&#8211; You are not an administrator user,<br \/>\n&#8211; You want to analyze downloaded log files without web server,<br \/>\n&#8211; You want to analyze mail or ftp log files instead of web log files,<br \/>\n&#8211; You need to analyze load balanced servers log files,<br \/>\n&#8211; You want to &#8216;understand&#8217; all possible ways to use AWStats&#8230;<br \/>\nRead the AWStats documentation (docs\/index.html).<\/p>\n<p>&#8212;&#8211;&gt; Running OS detected: Linux, BSD or Unix Warning: AWStats standard<br \/>\ndirectory on Linux OS is &#8216;\/usr\/local\/awstats&#8217;.<br \/>\nIf you want to use standard directory, you should first move all content<br \/>\n of AWStats distribution from current directory:<br \/>\n\/usr\/share\/doc\/awstats\/examples to standard directory: \/usr\/local\/awstats<br \/>\nAnd then, run configure.pl from this location.<br \/>\nDo you want to continue setup from this NON standard directory [yN] ?<br \/>\nANSWER = y<\/p>\n<p>&#8212;&#8211;&gt; Check for web server install<br \/>\nEnter full config file path of your Web server. Example: \/etc\/httpd\/httpd.conf<br \/>\nExample: \/usr\/local\/apache2\/conf\/httpd.conf<br \/>\nExample: c:Program filesapache groupapacheconfhttpd.conf<br \/>\nConfig file path (&#8216;none&#8217; to skip web server setup): &gt;<br \/>\nANSWER = \/etc\/apache2\/sites-available\/001-awstats<\/p>\n<p>&#8212;&#8211;&gt; Check and complete web server config file &#8216;\/etc\/apache2\/sites-available\/001-awstats&#8217;<br \/>\n \u00a0 All AWStats directives are already present.<\/p>\n<p>&#8212;&#8211;&gt; Update model config file &#8216;\/usr\/share\/doc\/awstats\/examples\/wwwroot\/cgi-bin\/awstats.model.conf&#8217;<br \/>\nFile awstats.model.conf updated.<\/p>\n<p>&#8212;&#8211;&gt; Need to create a new config file ?<br \/>\nDo you want me to build a new AWStats config\/profile file (required if first install) [y\/N] ?<br \/>\nANSWER = y<\/p>\n<p>&#8212;&#8211;&gt; Define config file name to create What is the name of your web site or profile analysis ?<br \/>\nExample: &lt;a href=&quot;http:\/\/www.mysite.com&quot;&gt;www.mysite.com&lt;\/a&gt;<br \/>\nExample: demo Your web site, virtual server or profile name: &gt;<br \/>\nANSWER = default-web-site.your-domain.tld.somewhere<\/p>\n<p>&#8212;&#8211;&gt; Define config file path In which directory do you plan to store your config file(s) ?<br \/>\nDefault: \/etc\/awstats Directory path to store config file(s) (Enter for default): &gt;<br \/>\nANSWER = [Enter for Default]<\/p>\n<p>&#8212;&#8211;&gt; Create config file &#8216;\/etc\/awstats\/awstats.default-web-site.your-domain.tld.somewhere.conf&#8217;<br \/>\nConfig file \/etc\/awstats\/awstats.default-web-site.your-domain.tld.somewhere.conf created.<\/p>\n<p>&#8212;&#8211;&gt; Add update process inside a scheduler<br \/>\nSorry, configure.pl does not support automatic add to cron yet.<br \/>\nYou can do it manually by adding the following command to your cron:<br \/>\n \/usr\/share\/doc\/awstats\/examples\/wwwroot\/cgi-bin\/awstats.pl -update -config=default-web-site.your-domain.tld.somewhere<br \/>\nOr if you have several config files and prefer having only one command:<br \/>\n\/usr\/share\/doc\/awstats\/examples\/tools\/awstats_updateall.pl now<br \/>\nPress ENTER to continue&#8230;<\/p>\n<p>A SIMPLE config file has been created:<br \/>\n \/etc\/awstats\/awstats.default-web-site.your-domain.tld.somewhere.conf<br \/>\nYou should have a look inside to check and change manually main parameters.<br \/>\nYou can then manually update your statistics for &#8216;default-web-site.your-domain.tld.somewhere&#8217; with command:<br \/>\n &gt; perl awstats.pl -update -config=default-web-site.your-domain.tld.somewhere<br \/>\n You can also read your statistics for &#8216;default-web-site.your-domain.tld.somewhere&#8217; with URL:<br \/>\n &gt; &lt;a href=&quot;http:\/\/localhost\/awstats\/awstats.pl?config=default-web-site.your-domain.tld.somewhere&quot;&gt;http:\/\/localhost\/awstats\/awstats.pl?config=default-web-site.your-domain.tld.somewhere&lt;\/a&gt;<\/p>\n<p>Press ENTER to finish&#8230;<\/p>\n<p>[\/bash]<\/p>\n<p>Next edit the apache config file that was created by the script, in my example it is \/etc\/apache2\/sites-available\/001-awstats<br \/>\nAdd this section for the awstats-icon alias. I suggest near the top of the file after the CGI script directives rather than the bottom.<\/p>\n<p>[text]<br \/>\nAlias \/awstatsicons\/ \/usr\/share\/awstats\/icon\/<br \/>\n&lt;Directory \/usr\/share\/awstats\/icon&gt;<br \/>\n \u00a0 Options None<br \/>\n \u00a0 AllowOverride None<br \/>\n \u00a0 Order allow,deny<br \/>\n \u00a0 Allow from all<br \/>\n&lt;\/Directory&gt;<br \/>\n[\/text]<\/p>\n<p>There may be a pointer in the file already for Alias<\/p>\n<p>[text]<br \/>\nAlias \/awstatsicon &quot;\/usr\/share\/awstats\/examples\/wwwroot\/icon\/&quot;<br \/>\n[\/text]<\/p>\n<p>If there is then it is incorrect. Just use a # at the start of the line to disable it.<\/p>\n<p>To complete the Apache config enable the new 001-awstats site and restart apache2<\/p>\n<p>[bash]<br \/>\n\/etc\/apache2\/sites-enabled# a2ensite<br \/>\n\/etc\/apache2\/sites-enabled# apachectl graceful<br \/>\n[\/bash]<\/p>\n<p>You can also use &#8220;service apache2 graceful&#8221; with same outcome<br \/>\nNext in the \/etc\/awstats\/ directory find the awstats.your-domain-name.conf file that was created earlier.<br \/>\nEdit the file around line 51 MAIN SETUP SECTION and change the log file location for the default web site<br \/>\nfrom<\/p>\n<p>[text]LogFile=&quot;\/var\/log\/httpd\/mylog.log&quot;[\/text]<\/p>\n<p>to<\/p>\n<p>[text]LogFile=&quot;\/var\/log\/apache2\/access.log&quot;[\/text]<\/p>\n<p>everything else should be ok as a default.<\/p>\n<p>So now we have a configured Awstats for a basic test. Try running this command<\/p>\n<p>[bash]\/etc\/awstats# \/usr\/lib\/cgi-bin\/awstats.pl -config=default-web-site.your-domain.tld.somewhere[\/bash]<\/p>\n<p>Remember that the -config option here must match the awstats.default-web-site.your-domain.tld.somewhere.conf of the file created earlier. Except do not include the awstats. or the .conf as this prefix and suffix are automatically assumed by the application.<\/p>\n<p>The response should look something like this:<\/p>\n<p>[bash]<\/p>\n<p>Create\/Update database for config &quot;\/etc\/awstats\/awstats.default-web-site.your-domain.tld.somewhere.conf&quot; by AWStats version 6.95 (build 1.943)<br \/>\nFrom data in log file &quot;\/var\/log\/apache2\/access.log&quot;&#8230;<br \/>\nPhase 1 : First bypass old records, searching new record&#8230;<br \/>\nSearching new records from beginning of log file&#8230;<br \/>\nPhase 2 : Now process new records (Flush history on disk after 20000 hosts)&#8230;<br \/>\nJumped lines in file: 0<br \/>\nParsed lines in file: 28<br \/>\nFound 0 dropped records,<br \/>\nFound 0 corrupted records,<br \/>\nFound 0 old records,<br \/>\nFound 28 new qualified records.<\/p>\n<p>[\/bash]<\/p>\n<p>Finally the test is to check that it is alive ! In a web browser try the following substituting the example domain in the URL and as the config to check.<\/p>\n<p>[text]<br \/>\nhttp:\/\/default-web-site.your-domain.tld.somewhere\/cgi-bin\/awstats.pl?config=default-web-site.your-domain.tld.somewhere<br \/>\n[\/text]<\/p>\n<p>Ok, so that works.<\/p>\n<p>Now to\u00a0configure a\u00a0process to update the stats from the logs<\/p>\n<p>awstats_updateall.pl\u00a0 is the perl script that will do this, but it is sitting in an odd place under examples. I moved it just to have it in a more logical place alongside the awstats.pl script.\u00a0 This is not essential.<\/p>\n<p>[bash]<br \/>\n#cd \/usr\/share\/doc\/awstats\/examples\/<br \/>\n\/usr\/share\/doc\/awstats\/examples# cp awstats_updateall.pl \/usr\/lib\/cgi-bin\/<br \/>\n[\/bash]<\/p>\n<p>There is a whole commentary on the sequencing of\u00a0rotating logs and stats\u00a0updates at <a href=\"http:\/\/awstats.sourceforge.net\/docs\/awstats_faq.html#ROTATE\">http:\/\/awstats.sourceforge.net\/docs\/awstats_faq.html#ROTATE<\/a><\/p>\n<p>For what I need using the Apache Logrotate process will suit.\u00a0 Modify the \/etc\/logrotate.d\/apache2 file<\/p>\n<p>[bash]<br \/>\n# cd \/etc\/logrotate.d\/<br \/>\n\/etc\/logrotate.d\/# mcedit apache2<br \/>\n[\/bash]<\/p>\n<p>Insert\u00a0the following 3 new lines just after the sharedscripts line and before the postrotate line.<\/p>\n<p>[text]<br \/>\nprerotate<br \/>\n\/usr\/lib\/cgi-bin\/awstats_updateall.pl -awstatsprog=\/usr\/lib\/cgi-bin\/awstats.pl now<br \/>\nendscript<br \/>\n[\/text]<\/p>\n<p>I did not include the config option &#8220;-configdir=\/etc\/awstats\/&#8221; as that is the default and it is redundant<br \/>\nThe command &#8220;\/usr\/lib\/cgi-bin\/awstats_updateall.pl -awstatsprog=\/usr\/lib\/cgi-bin\/awstats.pl now&#8221; can be run from the command line to confirm that it is correct.<\/p>\n<p>So now the awstats is running and the logrotate will update the stats every morning<\/p>\n<p>Security of access to the files is the next topic and I&#8217;ve recorded that as a separate post.<\/p>\n<p>Next add another site with logs.\u00a0Adding another site is simply a case of identifying the log files and adding a conf file for them.<\/p>\n<p>Secure access with .htacess<\/p>\n","protected":false},"excerpt":{"rendered":"<p>There are a number of websites and servers that I want to track stats for. Each individual CPanel includes Awstats by default, but I have a number of dedicated servers that don&#8217;t\u00a0 have cpanel, and clients that want stats access who I don&#8217;t want to give access to the CPanel too (yep, I don&#8217;t trust [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[26,28],"tags":[],"class_list":["post-841","post","type-post","status-publish","format-standard","hentry","category-apache","category-awstats"],"_links":{"self":[{"href":"https:\/\/howden.net.au\/thowden\/wp-json\/wp\/v2\/posts\/841","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/howden.net.au\/thowden\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/howden.net.au\/thowden\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/howden.net.au\/thowden\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/howden.net.au\/thowden\/wp-json\/wp\/v2\/comments?post=841"}],"version-history":[{"count":0,"href":"https:\/\/howden.net.au\/thowden\/wp-json\/wp\/v2\/posts\/841\/revisions"}],"wp:attachment":[{"href":"https:\/\/howden.net.au\/thowden\/wp-json\/wp\/v2\/media?parent=841"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/howden.net.au\/thowden\/wp-json\/wp\/v2\/categories?post=841"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/howden.net.au\/thowden\/wp-json\/wp\/v2\/tags?post=841"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}