ApacheStats 0.8.2 (multiple RRD, Low hdd, Script Server)

RCK · Post by **RCK** » Thu Jan 10, 2008 5:45 pm

Dear community,

I released a new version of ApacheStats with one goal in mind: reduce disk usage.
The previous versions were using only one "Data Template", the results was each graph was using one 733K RRD file with same data inside.
In order to change that, I have updated the php script to filter each graph data, and rebuild the whole templates to use it

The result is 6 small RRD files (24k) to graph one host, it's a total of 350K instead of 5400K!

Warning:
This template will only work with ONE consolidation function (AVERAGE) into your RRA definition.
(Cacti default is using TWO consolidation functions).
To check how many consolidation function your have, go to "Management" -> "Data Sources" -> "RRAs", then edit the 4 archives and see how many lines are selected. You only need the "AVERAGE" line here.
note: you can also create alternate RRA functions with only "AVERAGE" line selected, then link those alternate RRA from ApacheStats08 data template. ("Templates" -> "Data Templates")

Upgrade step

update to cacti 0.8.7e
upload / update your file /scripts/ss_apache_stats.php
Delete all Apache08 CDEF in "Management" -> "Graph Management" -> "CDEFs"
Delete all Apache08 graph templates in "Templates" -> "Graph Templates"
Import the XML "cacti_host_template_webserver_-_apache.xml" into your cacti website

Install step

update to cacti 0.8.7e
upload the "ss_apache_stats.php" file to your cacti/scripts/ directory
import the XML "cacti_host_template_webserver_-_apache.xml" into your cacti website
If you don't care about 1min polling system, go into (Data Templates), edit the 5 "ApacheStats08" and remove "Hourly (1 Minute Average)" associated RRA's
add one device with the "WebServer - Apache" host template, the 7 graphs templates will show up in the (Associated Graph Templates) list
click (Create Graphs for this Host) and choose all graphs EXCEPT number (D) and (F) (to share common data source later), click create
choose the last two graphs (D) and (F) and click create

If you want to save 100K, you can remove the duplicate RRD by sharing common data source (*)

go to (Graph Management) and edit "<host> ApacheStats08 - D - Bytes / Hit - same data source..."
- change the data source "<host> ApacheStats08 (apache_total_hits)" to the other "<host> ApacheStats08 (apache_total_hits)"
- change the data source "<host> ApacheStats08 (apache_total_kbytes)" to the other "<host> ApacheStats08 (apache_total_kbytes)", and save
go to (Graph Management) and edit "<host> ApacheStats08 - F - Thread Details (%) - same data source..."
- change the data source "<host> ApacheStats08 (apache_busy_workers)" to the other "<host> ApacheStats08 (apache_busy_workers)"
- change the data source "<host> ApacheStats08 (apache_idle_workers)" to the other "<host> ApacheStats08 (apache_idle_workers)", and save
go to (Data Source) and look at (<host> ApacheStats08 - Total Hits), (<host> ApacheStats08 - Total kBytes), (<host> ApacheStats08 - Workers)
you will have to locate the 3 biggest <ID> and delete them. (look at the second column)
If you have correctly migrated the date source at step 2,3,5,6, Cacti won't propose you to delete one associated graph.
if cacti had time to create them, delete the 3 RRD files that have those <ID>, or use the RRDCLEANER plugin to perform the job.

(*) I don't think it is currently possible to automatically create multiple graphs with common data source in cacti, dear cacti masters, please add this option in futur !

Best regards,
RCK

---------------
Changelog 0.8.2
---------------
- export was done with cacti 0.8.7e
- ss_apache_stats.php: add support for apache 1.x - BusyServers,IdleServers (thanks helzerr)
- ss_apache_stats.php: no more floor() on CPU query to have better values (thanks helzerr)
- fixed the CDEFs function for graph D and F (thanks kilos / spkane)
- fixed some multiple graph comments after import.

---------------
Changelog 0.8.1
---------------
- export was done with cacti 0.8.7b
- CDEF were updated to correctly works with one consolidation function (AVERAGE)
- fixed some graph template that were still using the (MAX) consolidation.

---------------
Changelog 0.8.0
---------------
- data splitted into multiple RRD files to save 87% disk space (708K instead of 5400K)
- no more thread_O useless RRA
- no more redondant RRD data
- fix some useless "Graph Item Inputs"
- cacti 0.8.7 support
- this version of ss_apache_stats.php is compatible with ApacheStats 0.64

previous version credits

Rolf Poser for apachestats 0.4 - http://forums.cacti.net/viewtopic.php?t=9861
Mahuani for apachestats 0.6 SS - http://forums.cacti.net/viewtopic.php?t=17995

RCK · Post by **RCK** » Thu Jan 10, 2008 5:48 pm

Graph output

RCK · Post by **RCK** » Thu Jan 10, 2008 5:49 pm

Graph output

fmangeant · Post by **fmangeant** » Fri Jan 11, 2008 2:51 am

Many many thanks

phi00611 · Post by **phi00611** » Fri Jan 11, 2008 3:36 am

Hello,
This is exactly what I was looking for.

Thanks.

phi00611 · Post by **phi00611** » Fri Jan 11, 2008 5:18 am

Hello,

I have the following error when I try to import the host_template "cacti_host_template_webserver_-_apache.xml" :

Error: XML: Hash version does not exist.

Could you help me?[/b]

RCK · Post by **RCK** » Fri Jan 11, 2008 5:42 am

phi00611 wrote:Hello,

I have the following error when I try to import the host_template "cacti_host_template_webserver_-_apache.xml" :

Error: XML: Hash version does not exist.

Could you help me?[/b]

Be sure to have at least cacti version 0.8.7a
If this is the case, update your php version to 5.2.5
I tested my package under a fresh install without problem, can you also do the same if the cacti & php version are up-to-date ?

phi00611 · Post by **phi00611** » Fri Jan 11, 2008 6:00 am

I updated cacti to 0.8.7a, and now the import works fine.

Thanks for you help.

phi00611 · Post by **phi00611** » Fri Jan 11, 2008 8:02 am

I updated my version of php 5.2.0-8 to 5.2.5. But all the graphes remain empty?

RCK · Post by **RCK** » Fri Jan 11, 2008 8:57 am

phi00611 wrote:I updated my version of php 5.2.0-8 to 5.2.5. But all the graphes remain empty?

1- import the templates again
2- delete the data source and graph that are empty
3- be sure to have uploaded ss_apache_stats.php to scripts folder
4- test the script with "php ss_apache_stats.php <host>" command
5- create your graph again

fmangeant · Post by **fmangeant** » Mon Jan 21, 2008 9:52 am

Hi

the only change that I'd make would be to graph bits/s instead of kbits/s, and let Cacti do the maths.

RCK · Post by **RCK** » Mon Jan 21, 2008 1:03 pm

It's already like this no ?
I thought only graph title was displaying "kbit/s"

What would you exactly change ?

fmangeant · Post by **fmangeant** » Mon Jan 21, 2008 3:54 pm

RCK wrote:It's already like this no ?
I thought only graph title was displaying "kbit/s"

What would you exactly change ?

So maybe the graph title & legend should be renamed to "bits/s" ?

I'm also wondering something : do you know how the stats for a "big" web server will be handled ?

http://www.apache.org/server-status returns that :

Code: Select all

Server uptime: 6 hours 38 minutes 44 seconds
Total accesses: 2239679 - Total Traffic: 206.5 GB

RCK · Post by **RCK** » Mon Jan 21, 2008 4:35 pm

You know, I only take previous template and update the rrd processing

* Rolf Poser for apachestats 0.4 - http://forums.cacti.net/viewtopic.php?t=9861
* Mahuani for apachestats 0.6 SS - http://forums.cacti.net/viewtopic.php?t=17995

kbit/s title is from there, they should have a good reason for that :p

LOL !
I will try to monitor apache.org, just to see

Weedy · Post by **Weedy** » Wed Jan 23, 2008 10:04 am

I dont know what the kbit/s graph is ACTUALLY plotting but I'm guessing it is supposed to plot apache traffic per second.

Code: Select all

 # lynx --dump http://127.0.0.1/server-status?auto
Total Accesses: 692641
Total kBytes: 10627606
CPULoad: .173695
Uptime: 49754
ReqPerSec: 13.9213
BytesPerSec: 218730
BytesPerReq: 15711.8
BusyWorkers: 18
IdleWorkers: 57
Scoreboard: ................................................................___W
_K____K___K____W_____.......................................___CK_____K__W_KW___
_K___...........................................................................
............................_______KK___W__K____K___K...........................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
............................................................................

So why do we use "Total kBytes" when we have "BytesPerSec"? right now "Total kBytes" shows about 10 gigs and "BytesPerSec" something like 213.6 kB/second. But cacti is graphing like "1.3 kBits/s" (no way in hell thats right), anyway I'm going to make the edit and see how it works.

ApacheStats 0.8.2 (multiple RRD, Low hdd, Script Server)

ApacheStats 0.8.2 (multiple RRD, Low hdd, Script Server)

Who is online