mod_watch - virtual website statistics

[zman818]

mod_watch/Cacti v0.9

- implemented by zman818 (zman818@hotmail.com)
- based on original work by Anthony Howe (snert.com)


mod_watch/Cacti (mwc) is a Cacti implementation of Anthony Howe's most excellent mod_watch module for Apache/MRTG. Mod_watch will watch and collect the bytes, requests, and documents in & out per virtual host, file owner, remote-ip address, directory or location, and the web server as a whole (configured through the use of httpd.conf directives).

A more detailed overview of mod_watch itself can be viewed at http://www.snert.com/Software/mod_watch/

The example below shows simple traffic statistics for 3 individual, virtual sites running under the same apache server. Other statistics are also available (hits, documents, etc). It's a simple adaptation of the stock bandwidth graph. Obviously such things as total traffic could be added to dress it up.

Please note that this is my first attempt at packaging and distributing a template. If I've borked it, I apologize in advance.

[jmeyerdo]

Hi!

Great script/module.
But unfortunately website of mod_watch is not available these days (no longer available?).

Does anyone know if the module (as RPM for Fedora Core or as Source) is available anywhere else?

Is there any other solution to graph the traffic for different virtual hosts?

Kind regards, Jens

[unclecameron]

Yeah, I found the same problem with snert not serving up pretty much anything these days, for no apparent reason. Anyway, I found this version of mod_watch lurking around somewhere, YMMV though as I haven't been able to install it yet, since I don't want to recompile and apxs wasn't loaded originally on the server I'm testing it on. I'm building a new server this weekend that I'll try to build this module into, I'll let you know if that works.

Anyway, here it is, let me know if you get it working

Cameron
San Diego

[WimanX]

Does a similar template exist for IIS?

[drewhka]

Corrected version of mod_watch_4.3 for Apache 2.2 (old sources contain deprecated macro & functions). Successfuly tested on Apache/2.2.4 (FreeBSD)

[Indiana]

Hi all,

I use mod_watch4.03 with apache 2.0 (on debian sarge) but /watch-list didn't show all virtual hosts.
Only 18 of 21
The missed three are equal the other vhosts entries (all are automatic generated out of a webfrontend).

[Indiana]

Hmm .. still get some crazy results .. some vhosts are listed some not
Moved now from apache2.0 to 2.2 (and mod watch 4.03 to 4.3).

Edit: Ok, now I got it .. It there is no bit of the vhost requestet it isn't in the watch-hash.

[firehole]

Indiana -

How did you fix your problem with missing vhosts? I have the same issue.

[ziggyz27]

Ok i hope this helps. Unlike normal Apache 2 my setup uses Includes to include Virtual Hosts for different domains. So the apache2mrtg.pl script that comes with mod_watch-4.3 did not work for me, it was just unable to find my virtual_hosts.

So modified the script to parse Includes.
NOTE: This is a quick and dirty modification but works for me.
I have also added few print statements commented out here to debug.

Code: Select all

#!/usr/bin/perl
#
# apache2mrtg.pl
#
# Copyright 2001, 2003 by Anthony Howe.  All rights reserved.
#

# Where the scripts should live
$SCRIPTDIR = "/usr/local/sbin";

##########################################################################
#       Nothing to be configured below this point.                       #
##########################################################################
$AUTHOR = 'achowe@snert.com';
$VERSION = '2.0';


unless (@ARGV == 1) {
        print STDERR "usage: apache2mrtg.pl path/to/httpd.conf\n";
        exit 2;
}

%seen = ();

sub target {
        my $name = shift;

        if ($seen{$name}) {
                print STDERR "DUPLICATE entry \"$name\"\n";
                return;
        }

        $seen{$name} = 1;

        print <<EOT;
Title[$name]: $name Data Traffic
Target[$name]: `${SCRIPTDIR}/mod_watch.pl http://$name/watch-info`
MaxBytes[$name]: 1250000
PageTop[$name]: <h2><a href="http://$name/">$name</a> Data Traffic</h2>

EOT
}

#########################################################################

$x=0;
while (<ARGV>) {
        # Skip commented blocks
        next if m!\#.*</?virtualhost!i;

        if (m!<virtualhost\s!i .. m!</virtualhost>!i) {
                $name = $1 if m!ServerName\s+(\S+)!;
        }

        if (m!</virtualhost>!i) {
                target($name);
        }

#############################################################################
#     Added to make script compliant to httpd.conf that uses Includes to    #
#     include virtual hosts. For example: Direct Admin type environments.   #
#############################################################################
        if(m!Include\s!i) {

                                my($text) = $_;
        #                       print "$_\n";
                                my($directory, $filename)= $text  =~ m/(.*\/)(.*)$/;
                                $directory =~ s/^\s+|\s+$//g ;
                                $directory =~ s/Include //;
        #                       print "D=$directory, F=$filename\n";
                                $mypath = $directory . "/" . $filename;
                                if($filename =~ m/httpd.conf/){

        #                               print "Processing $mypath we need it\n===============================================\n";
                                        $ALLHTTPDCONFS = $mypath;
                                        open(ALLHTTPDCONFS) or die("Could not open log file.");

                                        while(<ALLHTTPDCONFS>){

                                                 # Skip commented blocks
                                                        next if m!\#.*</?virtualhost!i;

                                                        if (m!<virtualhost\s!i .. m!</virtualhost>!i) {
                                                                $name = $1 if m!ServerName\s+(\S+)!;
                                                        }

                                                        if (m!</virtualhost>!i) {
        #                                                        print "Processing for: $name \n***************************************\n";
                                                                target($name);
                                                        }



                                        }

                                                $x++;
                                } else {
        #                               print "Skipping $filename we dont need it\n==============================================\n";
                                }

        }
        #       print "Processed total $x domains from includes!\n";
}

[deksar]

I would like to limit monthly bandwidth usage of some of my (apache) virtualhost accounts (shared IP - domain based) and check their current bandwidth usage to via a URL.
I mean a limit like: 20 GB/Month. Is this possible with mod_watch/Cacti?

(for Apache 2.2.x.)

By the way, thanks a lot for the corrections, drewhka and ziggyz27.
Thank you all!

[ziggyz27]

You should use an apache add-on like mod_cband

Checkout
http://www.freshports.org/www/mod_cband/

[deksar]

You should use an apache add-on like mod_cband
Checkout
http://www.freshports.org/www/mod_cband/

I tried it already a while ago. Too many errors..
Unfortunately, it is not getting updated anymore. It seems mod_cband died long time ago. Dozens of bugs are waiting:
http://sourceforge.net/tracker/?group_i ... tid=791368

[guilgot]

I need a solution to view in realtime the bandwidth used by virtualhosts...
This should be the soluion and I am happy to see that I am not the only one trying to use this great module !!!

The instalation with apache2 looks ok (well, no error message at restart...)

BUT...
I still have nothing when I check http://mydomain/watch-info
and ./mod_watch.pl -f ifRequests,ifDocuments http://mydomain/watch-list returns 0 0 0...

Secondly, do you know how to install it as a plugin on Munin (as explain here : http://www.freshnet.org/wordpress/2007/ ... ith-munin/) ?


Regards,

Guillaume

[guilgot]

I think I have read all web pages about mod_watch without any success...

I try to make it run with cacti and munin but the problem seems to come from mod_watch it self which don't want to give me any response to watch-info... (watch-table and watch-list does now...)

And more I have found a recursive error in in my error.log :
shVerifyString(b7fa6c58, b4ac3b18) failed!
[Sun Mar 21 14:55:05 2010] [crit] (20014)Internal error: shGetLockedEntry(b7fa6c58, "SERVER") failed in watchLogHash()

I use Apache/2.2.12 (Ubuntu) PHP/5.2.10-2ubuntu6.4 with Suhosin-Patch with mod_watch-4.3_apache22_mod...

I use several access log files... Do you think it could come from that ?
Do I need to use apache2mrtg.pl ?

Please, could you tell if, finaly, it works on your servers because I become crasy !`

Thanks in advanced,

Guilgot

[hbokh]

Regarding the error log / watchLogHash I found a fix here:
http://www.ovcharov.me/2009/09/01/kak-p ... #more-2295
It's in Russian but it comes to adding /altering a line in "SharedHash.c" at line 102 and should look like this:

Code: Select all

shVerifyString(struct shTable *tp, char *str)
{
    /* if ((char *) tp->shared <= str && str < (char *) tp->eshared) { */
    if ((char *) apr_shm_baseaddr_get(tp->shared) <= str && str < (char *) tp->eshared) {
        if (str + strlen(str) < (char *) tp->eshared)
            return str;
    }

Recompile the whole bunch and it should work somewhat better (read: no more errors).

On this side of the line getting info from "watch-list" DOES work, "watch-info" however gives a 404...