apache/tomcat monitoring

[rombert]

As a note, the script works better ( just tried it ) if you would replace
`GET $xml` with direct usage of LWP::Simple, i.e.

Thanks!

I tried using LWP initially but had issues getting it to work (easily) with the HTTP auth... The GET is definitely a cheap and dirty hack. Were you able to make it work with a http://user:pass@host:port/uri type URL?

Yes, I'm using it for monitoring using cacti as we speak.

[jehan.procaccia]

I realy appreciate TomcatStats-0.1.zip to monitor tomcat.
unfortunalty 2 of the 4 graphs don't work. for example "connection rate doen't" and cacti graph debug shows:

RRDTool Command:

/usr/bin/rrdtool graph - \
--imgformat=PNG \
--start=-86400 \
--end=-300 \
--title="cas1 - Connection Rate" \
--base=1000 \
--height=120 \
--width=500 \
--alt-autoscale-max \
--lower-limit=0 \
--y-grid=1 \
--vertical-label="Requests / Second" \
--slope-mode \
DEF:a="/usr/share/cacti/rra/cas1_jvm_memory_free_777.rrd":error_count:AVERAGE \
DEF:b="/usr/share/cacti/rra/cas1_jvm_memory_free_777.rrd":request_count:AVERAGE \
CDEF:cdefe=b,a,- \
AREA:a#EE5019:"Errors" \
GPRINTLAST:" Current\:%8.2lf %s" \
GPRINTAVERAGE:"Average\:%8.2lf %s" \
GPRINTMAX:"Maximum\:%8.2lf %s\n" \
AREA:cdefe#A150AA:"Requests":STACK \
GPRINT:cdefe:LAST:"Current\:%8.2lf %s" \
GPRINT:cdefe:AVERAGE:"Average%8.2lf %s" \
GPRINT:cdefe:MAX:"Maximum\:%8.2lf %s\n" \
COMMENT:"TomcatStats v0.1 - Timothy Denike"

RRDTool Says:

ERROR: invalid y-grid format

Cacti.log does show the sript running correctly apprently:

03/04/2008 10:14:09 PM - CMDPHP: Poller[0] Host[32] DS[778] CMD: perl /usr/share/cacti/scripts/tomcatstats.pl cas.it-sudparis.eu:8080 admin secret http-8080, output: jvm_memory_free:15038392 jvm_memory_max:66650112 jvm_memory_total:66650112 connector_max_time:737 connector_error_count:74 connector_bytes_sent:11421258 connector_processing_time:15620 connector_request_count:2766 connector_bytes_received:8126 connector_current_thread_count:25 connector_min_spare_threads:25 connector_max_threads:150 connector_max_spare_threads:75 connector_current_threads_busy:2

And rra file does has values

[root@supervis /usr/share/cacti/rra]
$ rrdtool fetch cas1_jvm_memory_free_777.rrd AVERAGE | tail -2
1204665300: 1.4968382080e+07 6.6650112000e+07 6.6650112000e+07 0.0000000000e+00 4.9133333333e+00 7.6693333333e+01 9.8181818182e-03 1.3333333333e-02 0.0000000000e+00 2.5000000000e+01 2.5000000000e+01 1.5000000000e+02 7.5000000000e+01 2.0000000000e+00

do you know what is that "ERROR: invalid y-grid format
" ?
Heap statitics has the same error
but throughput and threads statitics do work fine .

Thanks .

[gandalf]

do you know what is that "ERROR: invalid y-grid format

See teh Graph Template and remove the number "1" from the y-grid
Reinhard

[jehan.procaccia]

OK, setting 0 instead of 1 to
Unit Grid Value -> Use Per-Graph Value (Ignore this Value)
resolve my "y-grid" problem for the "connexion rate" graph, but not for "heap statistics"

RRDTool Command:

/usr/bin/rrdtool graph - \
--imgformat=PNG \
--start=-86400 \
--end=-300 \
--title="cas1 - Heap Statistics" \
--base=1024 \
--height=120 \
--width=500 \
--alt-autoscale-max \
--lower-limit=0 \
--y-grid=1048576 \
--vertical-label="Megabytes" \
--slope-mode \
DEF:a="/usr/share/cacti/rra/cas1_jvm_memory_free_777.rrd":jvm_memory_free:AVERAGE \
DEF:b="/usr/share/cacti/rra/cas1_jvm_memory_free_777.rrd":jvm_memory_total:AVERAGE \
DEF:c="/usr/share/cacti/rra/cas1_jvm_memory_free_777.rrd":jvm_memory_max:AVERAGE \
CDEF:cdefa=b,a,- \
CDEF:cdefbc=c,0,* \
AREA:cdefa#FAFD9E:"Used" \
GPRINT:cdefa:LAST:" Current\:%8.2lf %s" \
GPRINT:cdefa:AVERAGE:"Average%8.2lf %s" \
GPRINT:cdefa:MAX:"Maximum\:%8.2lf %s\n" \
AREA:a#2175D9:"Free":STACK \
GPRINTLAST:" Current\:%8.2lf %s" \
GPRINTAVERAGE:"Average\:%8.2lf %s" \
GPRINTMAX:"Maximum\:%8.2lf %s\n" \
LINE1:b#002A8F:"Total\:" \
GPRINTLAST:"Current\:%8.2lf %s" \
GPRINTAVERAGE:"Average\:%8.2lf %s" \
GPRINTMAX:"Maximum\:%8.2lf %s\n" \
LINE1:cdefbc#ED7600:"Max\:" \
GPRINT:c:LAST:" Current\:%8.2lf %s" \
GPRINT:c:AVERAGE:"Average\:%8.2lf %s" \
GPRINT:c:MAX:"Maximum\:%8.2lf %s\n" \
COMMENT:"TomcatStats v0.1 - Timothy Denike"

RRDTool Says:

ERROR: invalid y-grid format

What surprise me is that there's only one graph template -> "Tomcat - Connexion Rate". How comes that "thread statitics and throughput" graphs works !?

I also see in creating graphs, that there's graph item data sources that are never used: "processing-time, max-spare-threads, min-spare-threads, max-thread, max-time" !?

Finally, I would like to monitor https stats, I suppose I have to copy the "Tomcat Status" data input method with a modified input method set to "perl <path_cacti>/scripts/tomcatstats.pl <hostname>:8080 s2ia glopglop http-8443" . Then create a second data template and graph templates ... If ever you have an xml file doing that I'all appreciate.

Thanks .

[jehan.procaccia]

Forget it, I found how to run heap-statistics and found the other graph templates (they were on the "Next" page !)
for heap stat graph template I had to set "0" to "unit-grid-value" instead of the initial 1048576 (don't know why there was that value !?) .
thanks anyway .

[BMDan]

wget is far more widely deployed; consider replacing:
`GET $url`

with:

`/usr/bin/wget -qO - $url`

--BMDan

[BMDan]

Probably a silly question, but why doesn't this work:

Data Input Methods config

No matter how I try, I cannot get Cacti to prompt me for those input fields. I can't even find a place to enter these parameters on a per-graph, per-data source, or even per-host basis (and yes, I recognize only one of those would make sense). If I can beat this, that would mean I can monitor multiple Tomcat connectors on a single box, and it means I wouldn't have to make the script smart enough to figure out usernames and passwords for each monitored box.

~BMDan

[deckw]

I just started to use the templates and for some reason the scripts are giving the output of U. Yet when I run the script on my Linux box it works perfectly? Anyone have any clues? Thanks in advance.

[gandalf]

I suppose the script uses some commands without providing /full/path/to/command. From cli, your $PATH may interfere
Reinhard

[jdictionary]

wget is far more widely deployed; consider replacing:
`GET $url`

with:

`/usr/bin/wget -qO - $url`

--BMDan

Also, it is a good idea to make wget time-out quickly in the event that Tomcat is not responding (crashed, down due to maintenance, etc.) I have similar scripts to monitor Tomcat with Cacti and my wget to the Tomcat manager looks like this:

wget --tries=1 --timeout=5 -q -O - $url

[eschoeller]

It seems that with the way this template is designed each graph that is created ends up with it's own RRD files / data sources. Thus for each device that uses this template you end up with 4 host_error_count_XXX.rrd files. Each of these RRDs contain the same information. Furthermore, 4 queries are done per host to get the same data.

Is there anyway to restructure this template to use only ONE data source ... or data sources that only contain the information that is needed?

In the later, I would imagine that 4 Data templates would need to be written
that all use the same Data Input Method ... but then you're still running the query 4 times per host/device ....

[eschoeller]

I have another question about this template.

In regards to the "Connection Rate" Graph template, Does anyone know why the CDEF chosen for the request_count graph items are "B minus A" ? This doesn't make any sense to me at all, and my graph is blank. Switching this CDEF to "None" populates the graph with requests (I don't have any errors yet)

Code: Select all

/usr/local/rrdtool/bin/rrdtool graph - \
--imgformat=PNG \
--start=-86400 \
--end=-60 \
--title="cuconnect0 - Connection Rate" \
--base=1000 \
--height=120 \
--width=500 \
--alt-autoscale-max \
--lower-limit=0 \
--vertical-label="Requests / Second" \
--slope-mode \
--font TITLE:10: \
--font AXIS:8: \
--font LEGEND:8: \
--font UNIT:8: \
DEF:a="/usr/local/cacti-0.8.7b-dev/rra/HOST_error_count_2080.rrd":error_count:AVERAGE \
DEF:b="/usr/local/cacti-0.8.7b-dev/rra/HOST_error_count_2080.rrd":error_count:MAX \
DEF:c="/usr/local/cacti-0.8.7b-dev/rra/HOST_error_count_2080.rrd":request_count:AVERAGE \
DEF:d="/usr/local/cacti-0.8.7b-dev/rra/HOST_error_count_2080.rrd":request_count:MAX \
CDEF:cdeff=b,a,- \
CDEF:cdefh=b,a,- \
AREA:a#EE5019FF:"Errors"  \
GPRINT:a:LAST:"  Current\:%8.2lf %s"  \
GPRINT:a:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:b:MAX:"Maximum\:%8.2lf %s\n"  \
AREA:c#A150AAFF:"Requests":STACK \
GPRINT:cdeff:LAST:"Current\:%8.2lf %s"  \
GPRINT:cdeff:AVERAGE:"Average%8.2lf %s"  \
GPRINT:cdefh:MAX:"Maximum\:%8.2lf %s\n"  \
COMMENT:"TomcatStats v0.1 - Timothy Denike" 

The CDEF "B-A", in this situation subtracts the AVERAGE error_count from the MAX error_count - but it's applied to the request_count Graph Items!!

What's the purpose of that?

I apologize in advance if something within my configuration has changed the behavior of this ... There is a chance the author was not storing MAX values in the RRDs, in which case B-A would then equate to (AVERAGE requests) - (AVERAGE errors) which would make more sense - You'd only be graphing the requests without errors then. Given my limited understanding of how tomcat works ... I'm not sure if the request counter encapsulates all requests, even ones that have an error ...

Thanks!

[eschoeller]

I have yet another question about these templates. Whenever I restart tomcat I get some severe spikes in my data. I can't figure out what is causing this at all.

Any ideas? (we restart daily at 5AM)

[eschoeller]

I solved this problem by setting a maximum value for the following Data Source Items within the Tomcat Status Data Template:

error_count
bytes_sent
request_count
processing_time

I set the maximum of all these items to 1,000,000 using the command:

Code: Select all

find *max_time*.rrd -exec /usr/local/rrdtool/bin/rrdtool tune {} -a processing_time:1000000

And then I made sure to set the maximum values in the Data Template itself once I found that my graphs were rendering correctly after a morning restart.

I REALLY don't think this is the best way to solve this problem, it's more of a band-aid. I still don't understand why I see this type of behavior. [/code]

[trevorj]

Fixed. Great Script.