[XML] successor to apachestats 0.4

Post by **gandalf** » Sun Oct 16, 2005 4:58 am

Script was updated due to initialization errors. Please replace old one
Templates have been updated due to errors with newer cacti versions
This post is based on http://forums.cacti.net/viewtopic.php?t=4028 which contains the well know apachestats XML of Rolf Poser. Due to several questions arising (and some fun to have yet another simple perl script), I extended both two of the existing Graph Templates as well as implementing two new ones.
I did not include the docs from Rolf, as they hold for this version as well.
The script is based on the ws_apachestats_hs.pl, so it uses wget. This may as well be adjusted at your own. It has been enhanced in two ways:
- it includes CPULoad and Thread Scoreboard output
- output is performed in one single "print" statement to overcome possible problems using cactid

With later cacti versions, problems were reported with two of the graphs, the "Bytes/Requets" and the "Thread Details (%)" one. These have been corrected as they were due to the fact, that LINEx statements without color were defined.

Explanation of the Output for the "Thread Scoreboard" is best done using the "server-status" URL at your Webserver (e.g. http://<your webserber>/server-status)

ATTENTION
This script extends the Apache Webserver Data Template. So be careful, if you are using the apachestats-0.4 already! As well, the existing Apachestats Graph Templates will be modified!
At the moment, I do not know any way, to modify existing RRDs to include the additional Data Sources needed for CPULoad and Thread Scoreboard (perhaps export/manually modify/import will do?)
SO IT IS WISE NOT TO IMPORT DIRECTLY TO A PRODUCTION SYSTEM
If anyone knows a better way to do this, I'll appreciate to here from you.

Multiple RRDs
Due to the lack of my knowledge of the cacti system, I was not able to suppress the generation of one RRD for each of the Graph Templates. So, if you are going to define all 7 Graph Templates, you will notice 7 RRDs as well. All of them will hold "the same" data. The webserver is inquired 7 times for the same set of data. This is not a desired behaviour. There is a chance to get rid of 6 of them manually (Well, I did this yesterday, but took me "some time". It seems not that easy to describe this here)

As always, use this at your own risk.
Comments, enhancements and the like are appreciated

happy cactiing
Reinhard

Post by **gandalf** » Sun Oct 16, 2005 2:31 pm

Just did some tweaking with the "old" and the "new" templates to check any oddities in the mixture of both.
First, I cleared all my "old" resources: Existing Graphs and RRDs as well as all ApacheStats Graph, Data and Host Templates; even the CDEFs.
The I imported the ApacheStats 0.4 Templates and build all 5 Graphs (well, I had to tweak the both corrupted ones as described above to get them to work). This all went fine, as desired.
Now I imported the new ApacheStats Templates. Import went well

. Not to forget the new ws_apachestats.pl script!
Checked all Templates, they look as desired.
At last I build both new Graph Templates (CPULoad and Thread Scoreboard). No probs up to this.
I seems, that "overloading" existing Data Templates in this way does not confuse cacti!

The last thing remains: Both new templates will create their own RRDs.

I hope this will ease upgrading ...
Reinhard

JJX · Post by **JJX** » Mon Oct 17, 2005 6:11 am

i am facing a strange(?) error:

debian:/var/www/cacti/scripts# su www-data
debian:~/cacti/scripts$ ./ws_apachestats.pl 127.0.0.1
apache_total_hits:165 apache_total_kbytes:227 apache_busy_workers:1 apache_idle_workers:8 thread_O:247 threadC:0 threadD:0 threadG:0 threadI:0 threadK:0 threadL:0 threadR:0 threadS:0 threadW:1 thread_W:8 apache_cpuload:3.78073

works fine

when i run the poller:

debian:~/cacti/scripts$ ../poller.php
Waiting on 1/1 pollers.
Waiting on 1/1 pollers.
Use of uninitialized value in concatenation (.) or string at /var/www/cacti/scripts/ws_apachestats.pl line 63.
Use of uninitialized value in concatenation (.) or string at /var/www/cacti/scripts/ws_apachestats.pl line 65.
Use of uninitialized value in concatenation (.) or string at /var/www/cacti/scripts/ws_apachestats.pl line 67.
Use of uninitialized value in concatenation (.) or string at /var/www/cacti/scripts/ws_apachestats.pl line 69.
Use of uninitialized value in split at /var/www/cacti/scripts/ws_apachestats.pl line 89.
Use of uninitialized value in concatenation (.) or string at /var/www/cacti/scripts/ws_apachestats.pl line 103.
Use of uninitialized value in concatenation (.) or string at /var/www/cacti/scripts/ws_apachestats.pl line 63.
Use of uninitialized value in concatenation (.) or string at /var/www/cacti/scripts/ws_apachestats.pl line 65.
Use of uninitialized value in concatenation (.) or string at /var/www/cacti/scripts/ws_apachestats.pl line 67.
Use of uninitialized value in concatenation (.) or string at /var/www/cacti/scripts/ws_apachestats.pl line 69.
Use of uninitialized value in split at /var/www/cacti/scripts/ws_apachestats.pl line 89.
Use of uninitialized value in concatenation (.) or string at /var/www/cacti/scripts/ws_apachestats.pl line 103.
Use of uninitialized value in concatenation (.) or string at /var/www/cacti/scripts/ws_apachestats.pl line 63.
Use of uninitialized value in concatenation (.) or string at /var/www/cacti/scripts/ws_apachestats.pl line 65.
Use of uninitialized value in concatenation (.) or string at /var/www/cacti/scripts/ws_apachestats.pl line 67.
Use of uninitialized value in concatenation (.) or string at /var/www/cacti/scripts/ws_apachestats.pl line 69.
Use of uninitialized value in split at /var/www/cacti/scripts/ws_apachestats.pl line 89.
Use of uninitialized value in concatenation (.) or string at /var/www/cacti/scripts/ws_apachestats.pl line 103.

any idea?

i think this problem is because ws_apachestats.pl doesnt get correct parameters....
any idea?

Post by **gandalf** » Tue Oct 18, 2005 9:33 am

You'll have to feed the hostname to the script. This should be done via the Data Input Method. It should have one Input Parameter, <hostname>. This should, of course, be done by the Host Template you have imported. Please post the screen of your Data Input Method for Apache Webstats.
Reinhard

JJX · Post by **JJX** » Tue Oct 18, 2005 11:40 am

here is a screenshot - it looks fine

Post by **gandalf** » Wed Oct 19, 2005 9:00 am

Yes, indeed, looks as expected. So I'll have to have a more detailed view at it.
Reinhard

pyoung · Post by **pyoung** » Mon Nov 07, 2005 2:57 pm

How did you pare down the script so it's not storing duplicated RRD data?

Post by **gandalf** » Tue Nov 08, 2005 5:30 pm

pyoung wrote:How did you pare down the script so it's not storing duplicated RRD data?

Well, to be honest, it DOES create all that duplicate rrd's if you create all graphs from the device screen.
You may do so and AFTERWARDS change the references to these rrd's from the Data Source list. Then you may delete the obsolete rrd's.
OR, you create only ONE of the graphs from the device screen. The others must be created from the Graph Management screen (not remembering exactly, no cacti installation here at the moment). There you may refer to the Graph Template AND define the rrd file manually (if I'm mistaken, it must have been the [/b]Data Source screen instead).
AFAIK there's no option to do this automatically
Reinhard

Ingimar · Post by **Ingimar** » Thu Nov 10, 2005 11:19 am

Hi there.

I just installed the Apache status. I have extended status on in httpd.conf and access to /server-status works fine. I run the ws_apachestats.pl script and the output seems fine:

ws_apachestats.pl output wrote:apache_total_hits:6338 apache_total_kbytes:214646 apache_busy_workers:4 apache_idle_workers:9 thread_O:243 threadC:0 threadD:0 threadG:0 threadI:0 threadK:0 threadL:0 threadR:1 threadS:0 threadW:3 thread_W:9 apache_cpuload:12.1027

There are no errors in cacti.log and all graphs look fine to me except the Apache Statistics - CPU Load graph which does not display (it's not empty, it just does not display at all). When I check the properties for the graph they are:

Apache Statistics - CPU Load - Graph properties wrote:/usr/bin/rrdtool graph - \
--imgformat=PNG \
--start=-86400 \
--end=-300 \
--title="Baula - Apache Statistics - CPU Load" \
--base=1000 \
--height=120 \
--width=500 \
--alt-autoscale-max \
--lower-limit=0 \
--vertical-label="Apache CPU Load" \
DEF:a="/var/www/cacti/rra/baula_apache_cpuload_26.rrd":apache_cpuload:AVERAGE \
DEF:b="/var/www/cacti/rra/baula_apache_cpuload_26.rrd":apache_cpuload:MAX \
DEF:c="/var/www/cacti/rra/baula_apache_cpuload_26.rrd":apache_cpuload:AVERAGE \
CDEF:cdeff=c,1800,TREND \
AREA:c#FF5F00:"Apache CPU Load" \
GPRINT:c:LAST:"Current\:%8.2lf %s" \
GPRINT:c:AVERAGE:"Average\:%8.2lf %s" \
GPRINTMAX:"Maximum\:%8.2lf %s" \
LINE1:b#FF0000:"MAX" \
LINE1:cdeff#000000:"30min moving average\n"

And when I try running this rrdtool command from the command line I get the following error from rrdtool:

Output from the above rrdtool command wrote:ERROR: invalid rpn expression 'c,1800,TREND'

Could this be causing the graph not to be displayed?

I'm running on Cacti 0.8.6g on CentOS 4.2 with RRDTool 1.0.50. Both Cacti and RRDTool are fetched from Dag's RPM repository.

I hope someone can help me with this one.

Post by **gandalf** » Sat Nov 12, 2005 4:58 am

This may be related to rrdtool 1.0.x. I used the TREND function. You may delete the "30 min moving average" items out of the graph template to get rid of this
Reinhard

Ingimar · Post by **Ingimar** » Sat Nov 12, 2005 7:15 am

lvm wrote:This may be related to rrdtool 1.0.x. I used the TREND function. You may delete the "30 min moving average" items out of the graph template to get rid of this
Reinhard

That did the trick! Thanks alot!

suse · Post by **suse** » Tue Nov 15, 2005 8:30 am

one problem
I found
CACTID: Host[3] ERROR: Empty result [hostname]: 'perl /home/mrtg/cacti/scripts/ws_apachestats.pl hostname'
CACTID: Host[3] DS[84] WARNING: Result from SCRIPT not valid. Partial Result: ...

but if I ran perl /home/mrtg/cacti/scripts/ws_apachestats.pl hostname
I can get this
apache_total_hits:3424769 apache_total_kbytes:4463443333 apache_busy_workers:179 apache_idle_workers:31 thread_O:290 threadC:5 threadD:0 threadG:0 threadI:0 threadK:66 threadL:0 threadR:5 threadS:0 threadW:103 thread_W:31 apache_cpuload:.0389393

so why my data can not be polled?

Post by **gandalf** » Wed Nov 16, 2005 3:31 pm

suse wrote:one problem
I found
CACTID: Host[3] ERROR: Empty result [hostname]: 'perl /home/mrtg/cacti/scripts/ws_apachestats.pl hostname'
CACTID: Host[3] DS[84] WARNING: Result from SCRIPT not valid. Partial Result: ...

Please replace "perl" by the "/full/path/to/perl" (from the Data Source Screen). Remember to "Clear Poller Cache" from Settings to regenerate all commands used by the poller.
Reinhard

stevewest15 · Post by **stevewest15** » Mon Dec 05, 2005 4:06 pm

Hi,

Great job on this script! I'm a newbie and trying to figure out why 4 of the graphs don't plot (Bytes/Request, CPU Load, Hits/s, & Kbits/s). Not sure where to look at trouble shooting. Here is what I'm seeing in my cacti/log/cacti.log:

12/05/2005 03:40:01 PM - CMDPHP: Poller[0] Host[1] DS[79] CMD: perl /home/apache/htdocs/cacti/scripts/ws_apachestats.pl localhost, output: apache_total_hits: apache_total_kbytes: apache_busy_workers:5 apache_idle_workers:5 thread_O:1014 threadC:0 threadD:0 threadG:0 threadI:0 threadK:3 threadL:0 threadR:0 threadS:0 threadW:2 thread_W:5

I can access wget localhost/server-status which provides me with various apache 1.3 server status.

Can anyone help me look at what's causing not getting these graphs?

thx,

SW

Post by **gandalf** » Wed Dec 07, 2005 3:42 pm

The Script uses the /server-status?auto URL and parses the output. Please post the output of your webserver when pointing to that url
Reinhard

[XML] successor to apachestats 0.4

[XML] successor to apachestats 0.4

Who is online