[XML] ApacheStats version 0.2

Rolf Poser · Post by **Rolf Poser** » Mon Dec 08, 2003 1:46 pm

Howdy folks!

After a first attempt and getting the exports/imports wrong, I feel I have to apologise for all you brave people out there that gave it a spin!

Anyway - the good news is that I've made a number of improvements, changed some things, improved my use of naming and added a graph.

It now plots Bytes/Request - something that I'm particularly interested in (don't ask).

I have also TESTED the imports and exports and in the process uncovered a bug in lib/import.php . I've included a temporary fix with the hope that Ian will be able to incorporate some sort of more permanent solution in the long term. This applies to version cacti-0.8.4.

For more information you should refer to the README.txt file that I include.

Please let me know what else you think one should monitor in Apache. I have my own wishlist, but why not add your thoughts -- particularly if this important for you?

Enjoy!!!

Regards,
Rolf.

Rolf Poser · Post by **Rolf Poser** » Wed Dec 10, 2003 4:41 pm

Here are some thoughts that I have in terms of what additional stats would be useful:

1. Stack graph by response code per hit (e.g. red for 404's or 500's and so on).
2. Service time per hit (will require enabling of sorts, is available in some implementations??)
3. Stack graph by HTTP version per hit
4. Stack graph by HTTP request type

What other stats would you like to see?

michiel · Post by **michiel** » Thu Jan 08, 2004 6:24 pm

I can't seem to pass the hostname on to the script. When I set it all up as you describe and add the graphs to my hosts. they're all showing the stats of "localhost" ie the machine it's running on. Somehow the <hostname> is not passed on. Any idea why?

I have 7 machines I'm trying to create graphs for.

(bit later:)

Now I found the "Input Fields" in the Data Input Methods.

sorry
I'll explore that a bit more ...

fletch · Post by **fletch** » Tue Jan 13, 2004 11:46 pm

Hi,
I added some logic to unix_apachestats.pl to handle both apache 1.x and 2.x thread info properly:

Code: Select all

$VURL = "http:\/\/".$hostname."\/server-status";

my $apacheversion = `$LYNX -dump $VURL | grep "Server Version" | cut -c27`; 
$output = `$BASH -c '$LYNX -dump $URL | $HEAD -9'`; 

$output =~ /Total Accesses: (.*)/; 
print "apache_total_hits:".$1." "; 
$output =~ /Total kBytes: (.*)/; 
print "apache_total_kbytes:".$1." ";
if ($apacheversion < 2) {
	$output =~ /BusyServers: (.*)/;
	print "apache_busy_workers:".$1." ";
	$output =~ /IdleServers: (.*)/;
	print "apache_idle_workers:".$1; 
}
else {
        $output =~ /BusyWorkers: (.*)/;
        print "apache_busy_workers:".$1." ";
        $output =~ /IdleWorkers: (.*)/;
        print "apache_idle_workers:".$1;
}

For those of us who aren't allowed to upgrade all their Apache servers

pbulteel · Post by **pbulteel** » Mon Feb 16, 2004 6:59 pm

I redid the code so it's completely written in perl. It requires LWP which should be already installed. Thanks for the original script and the modification! And although I didn't put it in there... it's GPL of course!

Code: Select all

#!/usr/bin/perl -w
#
# Taken from ftp://ftp.ora.com/published/oreilly/nutshell/web-client/ 
# 
# Modified by: Patrick Bulteel
# Date: 02/16/2004 - (I love you hunny! It's my anniversary!)
#
# Modified so that it uses perl more extensively instead of running lynx.
# Also modified so that it uses a wildcard for Busy and Idle servers instead
# of having to hit the server twice, we can do it all in one swell swoop.
#

use strict;
use LWP::UserAgent;
use HTTP::Request;
use HTTP::Response;

my $ua = new LWP::UserAgent;
# I added this to the original O'reilly code...
my $fullurl = "http://" . $ARGV[0] . "/server-status?auto";

# I changed this from the original O'reilly code...
my $request = new HTTP::Request('GET', $fullurl);
my $response = $ua->request($request);
if ($response->is_success) {
    # I grabbed this from the original unix_apachestats.pl
     $response->content =~ /Total Accesses: (.*)/; 
    print "apache_total_hits:".$1." "; 
    $response->content =~ /Total kBytes: (.*)/; 
    print "apache_total_kbytes:".$1." ";
    # I wildcarded the Busy and Idle stats... =)
    $response->content =~ /Busy.*: (.*)/;
    print "apache_busy_workers:".$1." ";
    $response->content =~ /Idle.*: (.*)/;
    print "apache_idle_workers:".$1;
} else {
     print $response->error_as_HTML;
}

Rolf Poser · Post by **Rolf Poser** » Fri Feb 27, 2004 9:25 am

Patrick:

Thanks for the modifications - I like your work-around on the Busy* and Idle* wildcards - why didn't I think of that !! (My next release was going to parse the command line for a "1" or "2" flag, which would have been set from within cacti... Not as slick as your solution.)

Interestingly, I tried the LWP approach as with your post, but found that it is way slower on my system than using lynx. In fact, in the end I decided on using wget, which worked most efficient on my system.

What is the main reason why you would want to use LWP? If there is a good one (speed doesn't seem to be one -- at least not on my system - vanilla Redhat 9), I'll consider posting my new version of ApacheStats with your script and with the most recent one I wrote.?

Regards,
Rolf.

pbulteel · Post by **pbulteel** » Fri Feb 27, 2004 12:08 pm

I didn't really notice any difference in speed between using lynx/wget or LPW, but I prefer having everything in perl (lynx, wget I'd use for shell scipt). I guess I don't like mixing if I have the functionality in the language I'm using. I'm running RedHat 7.3/9.0 and Mandrake 8.2/9.2 - some with custom apache builds.

What I noticed from the 2nd script is that it wouldn't parse the Server Version string on my install of Apache so I look at the output and noticed that they both said Busy and Idle and it just came to me! =)

Maybe I'll time the scripts to see just how long they take. If there's a long enough delay, then we can decide which one to use... or you could make it optional in the code... include both ways in the code and just have them comment/uncomment the one they want to use? Also, not all my systems had lynx installed...

Rolf Poser · Post by **Rolf Poser** » Fri Feb 27, 2004 3:01 pm

Patrick:

I also tend to like having things uniform, but in my case I was really worried about performance and timing, so I would prefer sticking to wget.

I think therefore the best will be to allow for all three options in the script, so that everyone can choose what they want.

The unreleased version already caters for the lynx/wget options, but I'll freshen up my previous LWP script to check whether there would be any reason for it being slower than yours and will then include the new LWP calls in some way.

Thanks for the suggestions and the ideas.

Rolf.

pbulteel · Post by **pbulteel** » Mon Mar 01, 2004 1:37 am

Thank you for putting the templates together! And thanks to Ian for creating such a wonderful tool (and all the people that help) that keeps on getting better and better...

SCaRaBaeuS · Post by **SCaRaBaeuS** » Fri Apr 16, 2004 3:06 am

looks great and all but doesn't work.. the first version did tho

do i have to install something for perl?

bcs i get:

Code: Select all

[root@www scripts]# perl unix_apachestats.pl localhost
<HTML>
<HEAD><TITLE>An Error Occurred</TITLE></HEAD>
<BODY>
<H1>An Error Occurred</H1>
403 Forbidden
</BODY>
</HTML>

and my httpd.conf has:

Code: Select all

ExtendedStatus        on 

<Location /server-status> 
    SetHandler server-status 
    Order deny,allow 
    Deny from all 
    Allow from localhost
</Location>

even tho it worked with the first version.. now it doesn't and i got perl installed :S

alanyoung1979 · Post by **alanyoung1979** » Thu May 13, 2004 6:17 pm

I run this command from the command line as root:

Code: Select all

[root@foobar /var/www/html/cacti]# perl /var/www/html/cacti/scripts/unix_apachestats.pl host.foobar.com
apache_total_hits:5603 apache_total_kbytes:23513 apache_busy_workers:23513 apache_idle_workers:23513
[root@foobar /var/www/html/cacti]#

I then run the command as the user cacti:

Code: Select all

[root@foobar/var/www/html/cacti]# su - cacti
[cacti@foobar cacti]$ perl /var/www/html/cacti/scripts/unix_apachestats.pl host.foobar.com
apache_total_hits:5605 apache_total_kbytes:23515 apache_busy_workers:23515 apache_idle_workers:23515
[cacti@foobar cacti]$

However when I use this to monitor the same host (ie host.foobar.com) i get no data in my graphs.

This is the command (there are 4 graphs) that cacti runs (taken from Utilities -> View Poller Cache):

Code: Select all

 Data Source: host.foobar.com - Apache Statistics
RRD: /var/www/html/cacti/rra/host.foobar.com_apache_idle_workers_747.rrd
Action: 2, Script: perl /var/www/html/cacti/scripts/unix_apachestats.pl host.foobar.com

This is what gets returned in the log:

Code: Select all

05/13/2004 3:55 PM - CMD: /usr/local/rrdtool-1.0.48/bin/rrdtool update /var/www/html/cacti/rra/host.foobar.com_apache_idle_workers_747.rrd --template apache_total_hits:apache_total_kbytes:apache_busy_workers:apache_idle_workers N::::

So it works fine from the command line, but it seems that no data is graphed. Any ideas?

Please note that host.foobar.com is not the real host name.

alanyoung1979 · Post by **alanyoung1979** » Thu May 13, 2004 6:19 pm

Last graph, due to 3 file attachment limit

Rolf Poser · Post by **Rolf Poser** » Mon May 17, 2004 4:43 pm

Dear Alan:

Could you confirm for me that you're using v0.85a of cacti with this, and which version of perl. I know that this sounds arbitrary, but I just want to check before I try to figure out what the problem might be?

Thanks,
Rolf.

alanyoung1979 · Post by **alanyoung1979** » Mon May 17, 2004 4:48 pm

Rolf,

I am running cacti 0.8.5a with perl 5.6.1. The server OS is Redhat ES 2.1.

Thanks for looking into this,

Alan

Rolf Poser · Post by **Rolf Poser** » Mon May 17, 2004 4:54 pm

ScArAbEuS:

Have you checked running the script against your webserver manually:

perl <cacti_home>/scripts/....pl hostname

You shouldn't need anything special for perl, but do check whether the script points to the correct locations for lynx and/or wget.

Let me know whether any of this helped.

Thanks,
Rolf

Cacti