Extended system stats graphing- vmstat, iostat, mpstat (sar)

[fazil]

I installed the plugin. I have graphs but hachurés.les other graphs I outside of this plugin are not hatched. (sorry for my english:( )

any id ?

[neocontrol]

Okay I fixed my one graph that was complaining about the CF's. I had to go in and change 'Limit' to 'Average'. And bam, worked as expected.

I have chopped up this script some, so you can run it with the nagios plugin, check_by_ssh. That way you can use passwordless DSA keys.

#!/usr/bin/perl -w

@input=`sar -Brcquwv -n SOCK 1 1|grep Average`;
$i=0;
$f=0;
$v=0;
foreach $line (@input) {
@line = split(' ', $line);
$fc=$#line;
for ($x=1; $x<=$fc; $x++) {
if ($x == $fc) { $end = 1; } else { $end = 0; }
if ($i == 0) {
$line[$x]=~s/\///;
$line[$x]=~s/\%//;
$line[$x]=~s/-/_/;
$fields[$f]=$line[$x];
#print "settings fields $f to $line[$x] - ";
$f++;
if ($end) {
$i=1;
}
next;
}
if ($i == 1) {
$vals[$v]=$line[$x];
#print "setting vals $v to $line[$x]\n";
$v++;
if ($end) {
$i=0;
}
next;
}
}
}
$output='';
for ($x=0; $x<=$#fields; $x++) {
$output=$output."$fields[$x]:$vals[$x] ";
}
print $output;

Now my only issue is getting the load average graphs working. It's missing a RRA file it seems. Not sure why it hasn't been created yet.

[emrebilmuh]

Hello guys,

I hope to find somebody here from past

Can anybody provide me the tarball of these templates? Because the address (http://frylab.com/~adamb/sarparse/ ) is not available anymore.

These templates are absolutely what i need.


Thank you all

[neocontrol]

Take a look on one of the earlier pages, it's stored there. Page 2 or 3 I think.

[emrebilmuh]

Take a look on one of the earlier pages, it's stored there. Page 2 or 3 I think.

Thank you my friend. I couldn't imagine that the one thought to include ball in the doc

[jug]

The download link for the php and pl files are not working, would it be possible that someone uploads a zip file with this files do this post so that it could be downloaded directly from here?

[nissim]

Hello,

You scripts are exactly what I need to monitor my unix machines through sar and generate nice graphs.

I tried to download the tarball from the url you provided but the website is unavailable.

Can you please send an up2date link or tell how can I download it??

Maybe I can send you my private e-mail so you can send me the tar?

Many thanks.

[ahhdem]

sorry guys, I haven't worked for that company in a while and just managed to collect all the data I'll need to package up a proper release, including the scripts and templates for multicore/proc and disks

hang tight! I should be getting the last bits tonight, but will be out of town for the next week. I want to test a little bit and hopefully fix a bug in the disk graphing before I release so expect a proper posting in the next couple weeks. In the meantime, here are the old files needed for the listed graphs!

Enj0y!

[dreamnid]

Hi,

I was wondering if you can give us any status updates on the other scripts for monitoring the multiproc and disks.

Thanks!

[ahhdem]

Keep an eye here for an update on multiproc/disk templates and scripts:

https://sourceforge.net/projects/sarparse/files/

[ahhdem]

I just posted a tar file release with an INSTALL file and some brief instructions, and all the templates and files for multicore cpu graphing and detailed disk graphing.

I appreciate any feedback esp. regarding installation/setup process

[stucky101]

EXCELLENT WORK !
This is the first time someone posted something really useful and complete regarding linux perfstats not relying on awful snmp.
Still not quite a full sar report but THIS much closer !
I absolutely agree that the out-of-the box stuff is pretty useless except the FS and NIC stats which are sufficient.

1 thing though.

When you add a host using this host template it says that there is a problem with both Sarparse::IOstat and Sarparse::Multicore_CPU in that they don't return any data.
Afterwards you can see that eventhough there aren't any errors at all (not even in DEBUG mode) they always return 0 rows.
Amazingly the CPU graph works anyway. I'd expect empty or broken graphs but no. The following graphs exist and look fine
at first glance :

paging activity
context switches
dir cache
load average
open handles
open sockets
process creation
processes
run queue
cpu usage
memory hits
memory usage

I believe some graphs are missing ?
Keep up this awesome project - its the best by far I've seen here !

I have cacti-0.8.7e

[stucky101]

I also noticed that the cpu graph scales to 120% eventhough the percent idle never exceeds 100.
Is that desired ?
I also saw that the graph uses the STACK Item Type for most stuff. Doesn't this mean that graphs are
stacked on top of each other thereby giving totally wrong values ? Inever understood why anybody would wanna see it displayed like this.
The out of the box load average is like that too and I changed it all. Please confirm that I have to replace all STACKs with
something else if I don't like that view.

[stucky101]

BTW the error when adding a host is this

This data query returned 0 rows, perhaps there was a problem executing this data query. You can run this data query in debug mode to get more information.

The debug looks fine

+ Running data query [10].
+ Found type = '4 '[script query].
+ Found data query XML file at '/var/www/html/cacti-0.8.7e/resource/script_queries/host_disk_stats.xml'
+ XML file parsed ok.
+ Executing script for list of indexes '/usr/bin/php -q /var/www/html/cacti-0.8.7e/scripts/sarparse_iostat.php pscom-prd-web4 index'
+ Executing script query '/usr/bin/php -q /var/www/html/cacti-0.8.7e/scripts/sarparse_iostat.php pscom-prd-web4 query index'
+ Executing script query '/usr/bin/php -q /var/www/html/cacti-0.8.7e/scripts/sarparse_iostat.php pscom-prd-web4 query name'
+ Found data query XML file at '/var/www/html/cacti-0.8.7e/resource/script_queries/host_disk_stats.xml'
+ Found data query XML file at '/var/www/html/cacti-0.8.7e/resource/script_queries/host_disk_stats.xml'
+ Found data query XML file at '/var/www/html/cacti-0.8.7e/resource/script_queries/host_disk_stats.xml'

[ahhdem]

BTW the error when adding a host is this

This data query returned 0 rows, perhaps there was a problem executing this data query. You can run this data query in debug mode to get more information.

The debug looks fine

+ Running data query [10].
+ Found type = '4 '[script query].
+ Found data query XML file at '/var/www/html/cacti-0.8.7e/resource/script_queries/host_disk_stats.xml'
+ XML file parsed ok.
+ Executing script for list of indexes '/usr/bin/php -q /var/www/html/cacti-0.8.7e/scripts/sarparse_iostat.php pscom-prd-web4 index'
+ Executing script query '/usr/bin/php -q /var/www/html/cacti-0.8.7e/scripts/sarparse_iostat.php pscom-prd-web4 query index'
+ Executing script query '/usr/bin/php -q /var/www/html/cacti-0.8.7e/scripts/sarparse_iostat.php pscom-prd-web4 query name'
+ Found data query XML file at '/var/www/html/cacti-0.8.7e/resource/script_queries/host_disk_stats.xml'
+ Found data query XML file at '/var/www/html/cacti-0.8.7e/resource/script_queries/host_disk_stats.xml'
+ Found data query XML file at '/var/www/html/cacti-0.8.7e/resource/script_queries/host_disk_stats.xml'

Thanks for this.. The disk graphing had a bug in a previous cacti version where the graphs were graphing the same values, despite the queries returning different results. My approach to this was basically walk away and try again later as I could not track down what the issue was and didnt get much support, though I didnt try too hard for that support It appears to be working now as far as graphing accurate values go but thatportion of the code is the least mature. In the coming weeks I anticipate using these templates on a wide array of hosts and hopefully will be able to pound out some of these outlying corner cases.

The following graphs exist and look fine
at first glance :

paging activity
context switches
dir cache
load average
open handles
open sockets
process creation
processes
run queue
cpu usage
memory hits
memory usage

I believe some graphs are missing?

I have cacti-0.8.7e

You will notice many of these graphs are utilizing multiple metrics returned via sar, for example the memory hits/usage graphs, open sockets, etc. There are some varying versions of sysstat out there that return the same values with different formatting for the metric names. One sure way to circumvent this is to use the latest release built from source vs. the distro packaged version. In many cases I have tried to add simple regex replaces to the sarparse.pl script, which handles fetching and formatting most of the data. If you feel you are missing some data, verify that the name (i.e. pgpgin pgpgout etc) matches what you see int he Data Query or Data Input Method sections. I currently dont have access to a live installation so this is all from memory, please bear with me

I also noticed that the cpu graph scales to 120% eventhough the percent idle never exceeds 100.
Is that desired ?
I also saw that the graph uses the STACK Item Type for most stuff. Doesn't this mean that graphs are
stacked on top of each other thereby giving totally wrong values ? Inever understood why anybody would wanna see it displayed like this.
The out of the box load average is like that too and I changed it all. Please confirm that I have to replace all STACKs with
something else if I don't like that view.

Here is the reasoning behind STACK and 120%: The values returned by sar for free, user, system, idle, etc, are in amounts that will accumulate to 100. Without a stack, some usage may get lost behind other layers. I.E. if the user amount is 9% and the System amount is 5%, unless the system is presented in front of the user, user will overlap it. Since you cant predict which will be more vs. less, I went with a stack. Because I was doing a stack, I chose to also paint IDLE/Free % in Green to give a good easy visualisation of aggregate usage relative to the upper limit. I -have- experienced situations, however rare, where the total aggregate usage managed to creep -beyond- 100, which is the reason for some extra scale headroom. Feel free to customize the graph template and re-create it. You should be able to do it without clobbering your rrd, but if you do, a simple dump and import into the newly created rrd can transfer your collected data.

Also, bear in mind, this is an aggregate usage across all cpus, to view breakdown by core (and potentially 'processes', depending on what you are running.. (java?)), the multi-core cpu graphs are to be leveraged. In most cases I have found these to be primarily useful as a confirmation of the machines ability to distribute the load and utilize all cores effectively.

Feel free to change whatever suits your needs, just try to be aware of the potential pitfalls, the cpu %age being a great example.
Cheers
-Adam