[XML] ApacheStats version 0.4

Templates, scripts for templates, scripts and requests for templates.

Moderators: Developers, Moderators

mcgrof
Posts: 1
Joined: Thu Nov 18, 2004 3:40 pm
Contact:

ApcheStats 0.5

Post by mcgrof »

Here is a version which automatically detects which method is the best to use. It also outputs more variables in apache2 is in extended mode.

Code: Select all

#!/usr/bin/perl -w
#
#  ApacheStats 0.5
#
#  Copyright (C) 2004 Rolf Poser <rolf@stasolutions.co.za>
#  Copyright (C) 2004 Luis R. Rodriguez <mcgrof@ruslug.rutgers.edu>
#
#  This program is free software; you can redistribute it and/or modify
#  it under the terms of the GNU General Public License as published by
#  the Free Software Foundation; either version 2 of the License
#
#  This program is distributed in the hope that it will be useful,
#  but WITHOUT ANY WARRANTY; without even the implied warranty of
#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#  GNU General Public License for more details.
#
#  You should have received a copy of the GNU General Public License
#  along with this program; if not, write to the Free Software
#  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA

use strict;
use Switch;

my $output = "";
my $response = "";
my $bin  = "/usr/bin/";
my $wget = $bin . "wget";
my $lynx = $bin . "lynx";
my $hostname = ($#ARGV <= 0) ? "localhost" : chomp $ARGV[0];
my $url = "http:\/\/".$hostname."\/server-status?auto";

# Change the following if you have an environment variable of "http_proxy"
# that interferes with your direct access to the web server in question.
# The symptoms are that the ACL on the server disallows you access because
# you are connecting via a proxy server:
delete $ENV{'HTTP_PROXY'};
delete $ENV{'http_proxy'};

my $method = ( -f $wget ) ? $wget : (-f $lynx) ? $lynx : "perl";

switch($method) {
        case "$wget" {
                $method .= " --user-agent=\"ApacheStats\/0.4 wget\/1.8\" -q -O -";
                $output = `$method $url`;
        }
        case "$lynx" {
                $method .= " -useragent=\"ApacheStats\/0.4 Lynx\/2.8\" -dump";
                $output = `$method $url`;
        }
        case "perl" {
                use LWP::UserAgent;
                use HTTP::Request;
                use HTTP::Response;
                my $userAgent = new LWP::UserAgent;
                $userAgent->agent("ApacheStats/0.4 ");
                my $request = new HTTP::Request('GET', $url);
                my $response = $userAgent->request($request);
                print $response->error_as_HTML and exit() if (!$response->is_success);
                $output = $response->content;
        }
}

print "No method usable" and exit() if ($output eq "");

$output =~ /Total Accesses: (.*)/;
print "apache_total_hits:$1 ";
$output =~ /Total kBytes: (.*)/;
print "apache_total_kbytes:".$1." ";
$output =~ /CPULoad: (.*)/;
print "apache_cpu_load:".$1." ";
$output =~ /ReqPerSec: (.*)/;
print "apache_req_per_sec:".$1." ";
$output =~ /BytesPerSec: (.*)/;
print "apache_bytes_per_sec:".$1." ";
$output =~ /BytesPerReq: (.*)/;
print "apache_bytes_per_req:".$1." ";
$output =~ /Busy.*: (.*)/;
print "apache_busy_workers:$1 ";
$output =~ /Idle.*: (.*)/;
print "apache_idle_workers:$1 ";

Trancos
Posts: 1
Joined: Fri Apr 30, 2004 7:58 am
Contact:

Problem with authentication

Post by Trancos »

Hi all!!!!!!!

I have been a problems with the ApacheStats, because my apache have an authentication with mysql.

Could you post what i need to add in order to get the stats.

Regards,


Trancos
bilbo
Posts: 7
Joined: Sat Oct 23, 2004 10:03 am

too many rrd's for apachestats 0.4?

Post by bilbo »

Just installed apachestats 0.4. The script works well! And it's really nice to import all templates; saves a lot of time.
But here comes my problem. I implemented some host (device in cacti wording) with reference to the Host Template "WebServer - Apache". Then I created all five graphs e.g. bytes/request, hits/sec. etc. Even this works fine. But now I have 5 new rrd's, all of them holding the same complete information. And the poller executes ws_apachestats.pl 5 times to populate the 5 rrd's.

Isn't it possible to only create 1 rrd with alle the information? And to have the poller only execute the ws_apachestats script once a run for each host?

Is there a possibility to reduce the overhead?

Thnx for any help
bytebanditkiller

graph is empty

Post by bytebanditkiller »

i am quite a newbi... so i have a problem with the script:

"THE GRAPHS ARE CREATED BUT EMPTY"

if i run it from the shell i have the following error:

Can't locate LWP/UserAgent.pm in @INC
(@INC contains: /usr/local/lib/perl5/site_perl/5.6.1/mach /usr/local/lib/perl5/site_perl/5.6.1 /usr/local/lib/perl5/site_perl /usr/local/lib/perl5/5.6.1/BSDPAN /usr/local/lib/perl5/5.6.1/mach /usr/local/lib/perl5/5.6.1 .) at ws_apachestats.pl line 24.
BEGIN failed--compilation aborted at ws_apachestats.pl line 24

i have perl5.6.1 on a freebsd system.

Do i have to reinstall perl with some variables to make your script working?

thx for your help :wink:
bytebanditkiller

perl module installed :-)

Post by bytebanditkiller »

after a little bit of googleing i found out quickly i was missing the module for perl.

i maybe installed too much but with the installation of following bundle the needed module came with. :D

perl -MCPAN -e 'install bundle::LWP'

cool it seems to the http bundle i already have got or is standard.

...at least i just have a different error now, telling me cannot connect to localhost:80 nor to 127.0.0.1:80

ether when i have configured virtualhost nor without... :cry:
User avatar
jshanley
Posts: 11
Joined: Fri Jul 30, 2004 7:39 pm

Post by jshanley »

Your best option is to add 127.0.0.1 to the IPs that your apache listens to, For example, you could add a line like this in your httpd.conf, near the other "Listen" directives...

Code: Select all

Listen 127.0.0.1:80
Of course, make sure you have appropriately configured server status reports in your httpd.conf ... like

Code: Select all

ExtendedStatus On
<Location /server-status>
    SetHandler server-status
    Order allow,deny
    Allow from 127.0.0.1
    Allow from localhost
    Allow from 123.45.67.89 # Replace with a real IP 
    Deny from all
</Location>
Hope it helps.
zye

Post by zye »

great script :-)

works like a charm

thx a lot
bytebanditkiller

Listen 127.0.0.1:80

Post by bytebanditkiller »

hy jshanley

thank you very much....

i forgot to put the

Listen 127.0.0.1:80

the scripts just works fine... what a great thing thank you!
User avatar
gninja
Cacti User
Posts: 371
Joined: Tue Aug 24, 2004 5:02 pm
Location: San Francisco, CA
Contact:

Alternate ports

Post by gninja »

I'm in the process of adding support for checking something other than port 80.

I'm partway done, but it'll take me at least a week with my workload to get it working, tested, and release a patch.

Has anyone else done this already?
FreeBSD/RHEL
cacti-0.8.7i, spine 0.8.7i, PIA 3.1+boost 5.1
MySQL 5.5/InnoDB
RRDtool 1.2.27, PHP 5.1.6
User avatar
gninja
Cacti User
Posts: 371
Joined: Tue Aug 24, 2004 5:02 pm
Location: San Francisco, CA
Contact:

Post by gninja »

Ok, I've got a modification that allows you to query ports other than 80.

You can also query multiple servers per host, but that requires either 0.8.6b, or 0.8.6c with the graphs_new.php from 0.8.6b, since the new version of that file removes templates from the create graph pulldown if you have one created already. No other functionality seems to depend on the change in that file.

I'll leave it to the wonderful Rolf Poser to implement this (or not) into the main ApacheStats package.

In the query script, change:

Code: Select all

my $hostname = $ARGV[0];

chomp $hostname;

if ($hostname eq '') {
        $hostname = "localhost";
}

# Set the query URL:

$URL = "http:\/\/".$hostname."\/server-status?auto";
To:

Code: Select all

my ($hostname, $port);

if ($ARGV[0]) {
  $hostname = $ARGV[0];
  chomp $hostname;
} else {
  $hostname = 'localhost';
}

if ($ARGV[1]) {
  $port = $ARGV[1];
  chomp $port;
} else {
  $port = 80;
}

# Set the query URL:

$URL = "http:\/\/".$hostname.":".$port."\/ampserver-status?auto";
That will prevent a warning message if hostname or port isn't passed to the scripts, and query on the right port.

Now go to the Webserver - Apache Statistics Data Input Method, change the input string from:

Code: Select all

perl <path_cacti>/scripts/ws_apachestats.pl <hostname>
To:

Code: Select all

perl <path_cacti>/scripts/ws_apachestats.pl <hostname> <httpport>
Save, then add an input field named 'httpport'...

Now edit the Webserver - Apache Statistics Data Template, scroll down to the bottom, click the checkbox for Use Per-Data Source Value for httpport, drop '80' in the text box for a default, hit save.

Now when you create a new graph, you'll have the option to change the port to something other than port 80. All your existing graphs should continue to work.
FreeBSD/RHEL
cacti-0.8.7i, spine 0.8.7i, PIA 3.1+boost 5.1
MySQL 5.5/InnoDB
RRDtool 1.2.27, PHP 5.1.6
Hypatia

Threads Graph Incorrect?

Post by Hypatia »

I'm seeing some Apache Thread graphs that I don't understand. I expect that when my Apache server suddenly get a lot of hits, it will spawn some extra threads that will eventually time out and die when the server goes idle again. The graph I've attached seems to be plotted backwards!

Idle threads ramp up be for the busy threads do. This is happening pretty consistantly.

I've either configured things incorrectly, there's some kind of averaging bug in the template, or I've got a magic Apache server that can predict the future. I like the last possibility, but ir doesn't seem likely.

Has anyone seen this? Or explain to me where I'm being stupid?
Attachments
Thread_details.png
Thread_details.png (5.25 KiB) Viewed 13677 times
Rolf Poser
Posts: 42
Joined: Wed Dec 10, 2003 7:19 am

ApacheStats gets its own homepage:

Post by Rolf Poser »

Hello All:

First off, I just want to say to everyone that has asked, answered or contributed - a BIG THANK YOU :D !

This has been a collaborative work, and it's been quite amazing how this little script has grown from being actually something very simple.

In order to make this scale in the future, and to have a single point of reference for this project, I've put up a rudimentary home page for it at:

http://www.stasolutions.co.za/opensource/apachestats/

I will be expanding the functionality over the next while and polish the source and documentation using questions and contributions received here.

Kind Regards,
Rolf.
Rolf Poser
Posts: 42
Joined: Wed Dec 10, 2003 7:19 am

Preview of 0.6

Post by Rolf Poser »

Hello folks:

For those that are interested, I've been rebuilding ApacheStats for the next release 0.6.

Major Changes are:

- Two new graphs - CPU Utilisation and the Thread Scoreboard (see below)
- Results caching - It re-uses data from first query - truly only a single query to the target webserver
- Incorporated some ideas mentioned here - automation of finding the best retriever and specifying the port numbers.

Still need to package it with updated documentation, so it'll take me another few days.

Attached is a preview of the Thread Scoreboard graph for a large Apache site:

Regards,
Rolf.[/img]
Attachments
Scoreboard.png
Scoreboard.png (8.06 KiB) Viewed 13299 times
pbulteel
Cacti User
Posts: 150
Joined: Fri Sep 05, 2003 9:20 am
Location: London
Contact:

Post by pbulteel »

Awesome! Can't wait for you to get the next one out. Have you played with the apache-snmp module at all to see if there's anything work polling from there? I've only "seen" it exists, I haven't tried installing it and testing it.

It's come along nicely since I last tried it - and added to it...

=)
uname -a
Rolf Poser
Posts: 42
Joined: Wed Dec 10, 2003 7:19 am

mod_snmp

Post by Rolf Poser »

Hello Paul:

I've taken a look at the mod_snmp module. Looks good, except that it doesn't work for Apache 2 and up. I contacted the author and he wasn't really enthusiastic to work on updating it to a newer version. That was more than a year ago, so I don't know whether Harrie has changed his mind.... Will ask again.

I've been busy compiling a listing/index of various apache monitoring mechanisms that I'll put up on my website at some point. There is some really great stuff out there, and I want to try to incorporate as much of it into ApacheStats if I can.

BTW, Thanks for the kind words.

You'll probably not recognise the code - it's been quite a rewrite, and it's helped that I've actually learnt more about perl scripting :wink:

You'll still be able to use LWP if you prefer (setting).

Regards,
Rolf.
Post Reply

Who is online

Users browsing this forum: No registered users and 3 guests