Linux extended iostat template and scripts

[valleed]

Good Job!

Thanks for the templates/scripts!

[buznakka]

Hello, all.

Mark, thanks for your work on this.

I have a question for anyone about this part of the README.
The OID used by this script (.1.3.6.1.3.1) is marked as being reserved for
"experimental" usage. While it will work out of the box, this range may not be
unique across your organisation. If you decide to change this, you will need to
adjust the instructions below to reflect this.

How do I figure out which OID to use instead of the "experimental" one? Is it generally recommended to use another OID?

Thanks,
SOW

[itonohito]

Will these templates fit into latest version 0.8.7d ? I currently run 0.8.7a and can't install them (Error: XML: Hash version does not exist.) So I need upgrade. What version of Cacti should I upgrade to?

[toe_cutter]

Where are the templates? :s

[MarkRound]

Where are the templates? :s

Re-added to the original post. The forums have been playing up for me over the last few days - I tried to post an amended version, but it ate it...

[toe_cutter]

The attached file contains all the DS, graph and Data Query templates you need. The only down-side is that you can't create the full suite of graphs in on fell swoop. You have to create 1 type of graph at at time.

I'm sorry lints, but your template didnt quite work out (even after importing Marks templates):

Code: Select all

Graph Template

[success] Linux - Disk - Throughput [update]
[success] Linux - Disk - Queues [update]
[success] Linux - Disk - Request size [update]
[success] Linux - Disk - IOs [update]
   + Found Dependency: (Data Template Item) iostat_reads
   + Found Dependency: (GPRINT Preset) Normal
   + Unmet Dependency: (Data Template Item) 
   + Unmet Dependency: (Data Template Item) 
   + Unmet Dependency: (Data Template Item) 
   + Unmet Dependency: (Data Template Item) 
   + Unmet Dependency: (Data Template Item) 
   + Unmet Dependency: (Data Template Item) 
   + Unmet Dependency: (Data Template Item) 
   + Unmet Dependency: (Data Template Item) 
   + Unmet Dependency: (Data Template Item) 
   + Unmet Dependency: (Data Template Item) 
   + Unmet Dependency: (Data Template Item) 
   + Unmet Dependency: (Data Template Item) 
   + Unmet Dependency: (Data Template Item) 
   + Unmet Dependency: (Data Template Item) 
   + Unmet Dependency: (Data Template Item) 
[success] Linux - Disk - Times [update]
   + Found Dependency: (Data Template Item) iostat_await
   + Found Dependency: (GPRINT Preset) Normal
   + Unmet Dependency: (Data Template Item) 
   + Unmet Dependency: (Data Template Item) 
   + Unmet Dependency: (Data Template Item) 
   + Unmet Dependency: (Data Template Item) 
   + Unmet Dependency: (Data Template Item) 
[success] Linux - Disk - Utilisation [update]

Data Query

[success] Linux - Disk Stats [update]
   + Found Dependency: (Data Input Method) Get SNMP Data (Indexed)
   + Found Dependency: (Graph Template) Linux - Disk - Throughput
   + Found Dependency: (Data Template) Linux - Disk - Throughput
   + Found Dependency: (Data Template Item) iostat_wkbs
   + Found Dependency: (Data Template Item) iostat_rkbs
   + Found Dependency: (Graph Template) Linux - Disk - Queues
   + Found Dependency: (Data Template) Linux - Disk - Queue
   + Found Dependency: (Data Template Item) iostat_qsz
   + Found Dependency: (Graph Template) Linux - Disk - Request size
   + Found Dependency: (Data Template) Linux - Disk - Request size
   + Found Dependency: (Data Template Item) iostat_rqsz
   + Found Dependency: (Graph Template) Linux - Disk - IOs
   + Found Dependency: (Data Template) Linux - Disk - IOs
   + Unmet Dependency: (Data Template Item) 
   + Found Dependency: (Data Template Item) iostat_reads
   + Unmet Dependency: (Data Template Item) 
   + Unmet Dependency: (Data Template Item) 
   + Found Dependency: (Graph Template) Linux - Disk - Times
   + Found Dependency: (Data Template) Linux - Disk - Times
   + Found Dependency: (Data Template Item) iostat_await
   + Unmet Dependency: (Data Template Item) 
   + Found Dependency: (Graph Template) Linux - Disk - Utilisation
   + Found Dependency: (Data Template) Linux - Disk - Utilisation
   + Found Dependency: (Data Template Item) iostat_util

@MarkRound
Aah, ok! Thanks for the file

[DarkMan]

Very nice script, thanks!

I corrected lints template, there was a couple of errors with the Data Source among other little things....should work now, it does for me with cacti 0.8.7d

Having a bunch of dm-0, dm-1, dm-2, ... wasn't very interresting so I did a very crappy but working change to iostat.pl. Basically, i use devmap_name to find out what target correspond to the device-mapper (dm). I now got the LV name which is quite more usefull.

Code: Select all

sub getoid {
    my $oid = shift(@_);
    print "Fetching oid : $oid\n" if (debug);
    if ( $oid =~ /^$base_oid\.(\d+)\.(\d+).*/ && exists( $stats{$oid} ) ) {
        print $oid. "\n";
        if ( $1 == 1 ) {
            print "integer\n";
        }
        else {
            print "string\n";
        }

        if ( $stats{$oid} =~ m/dm-/ ) {
                my $dev_minor = $stats{$oid} ;
                $dev_minor =~ s/dm-//;
                my $devname = `devmap_name 253 $dev_minor`;
                print $devname . "\n";
        }
        else {
                print $stats{$oid} . "\n";
        }
    }
}

[toe_cutter]

I corrected lints template, there was a couple of errors with the Data Source among other little things....should work now, it does for me with cacti 0.8.7d

Thanks! lints template works better now
All i have left is to ask the serveradmin to update iostat.pl

[DarkMan]

Hmmm, looked at the graph produced this morning by iostat and there seems to be a large discrepancy between the io mesured by iostat, and the io mesured off the network (by snmp) during a network backup (using netbackup).

Network throughput averages about 20MB/s while the disk averages about 40MB/s. I did some testing and the network speed seems right, so why would iostat by twice the value?

[DarkMan]

Did some disk testing and this is was I got :

Running on SLES 10 SP1
sysstat version 6.0.2

dd if=dm_vfrud_dsp.dbf of=/dev/null bs=512
2091944+0 records in
2091944+0 records out
8568602624 bytes (8.6 GB) copied, 1049.49 seconds, 8.2 MB/s

From iostat.cache :
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
cciss/c0d0 0.00 15.83 0.03 7.73 0.53 377.60 0.27 188.80 48.69 0.05 6.18 0.58 0.45
cciss/c0d1 0.17 1903.37 2.47 114.77 487.47 32290.13 243.73 16145.07 279.59 1.92 16.35 0.22 2.57

From Cacti graph : Average write 33.50MB/s

Why is iostat 2x over dd? This is very odd. Ran a test on my laptop and iostat and dd gives the same result.

And why is the graph 2x over iostat?

[DarkMan]

Yay it's a kernel bug in SLES 10...even with SP2. Tested with Debian Lenny, iostat report far more sane results. Time to hack iostat.pl again since I won't be able to change the kernel on our production servers.

Well, turns out iostat for the cciss/c0dx is wrong but not the dm-X device.

Now for why the graph are wrong....

[DarkMan]

Oh joy, there are 2 more field returned by iostat under SLES (rsec/s and wsec/s). That made everything after $base_oid.6 wrong. Added 2 more OID for that extra data but keep the order intact so I wouldn't have to modify the snmp_queries :

Code: Select all

$stats{"$base_oid.6.$devices"}  = $5;           # w/s
$stats{"$base_oid.14.$devices"}  = $6;          # rsec/s ###SLES
$stats{"$base_oid.15.$devices"}  = $7;          # wsec/s ###SLES
$stats{"$base_oid.7.$devices"}  = $8;           # w/s

You have to add \s+(\d+[\.,]\d+) to the regexp for each extra OID you create (or remove if you have less).

Then did another hack because for some reason under SLES, rsec/s wsec/s rkB/s wkB/s are double the real value for cciss/cXdY devices (not dm devices).

Code: Select all

 if ( $1 =~ m/cciss/ ) {
                $stats{"$base_oid.7.$devices"}  =  $stats{"$base_oid.7.$devices"}/2;            # rkB/s
                $stats{"$base_oid.8.$devices"}  =  $stats{"$base_oid.8.$devices"}/2;            # wkB/s
                $stats{"$base_oid.14.$devices"}  =  $stats{"$base_oid.14.$devices"}/2;          # rsec/s ###SLES###
                $stats{"$base_oid.15.$devices"}  =  $stats{"$base_oid.15.$devices"}/2;          # wsec/s ###SLES###

        }

And now, a bit less ugly hack to change those dm-X to lvol name :

Code: Select all

        if ( $1 =~ m/dm-/ ) {
                $stats{"$base_oid.2.$devices"} =~ s/dm-//;
                $stats{"$base_oid.2.$devices"} = `devmap_name 253 $stats{"$base_oid.2.$devices"}`;
        }

Ignore my previous hack to the sub getoid. All of this goes into sub process.

Enjoy!

PS : Damn you Novell!!!

[eschoeller]

DarkMan:

Did you consider using iostat flag:

-N Display the registered device mapper names for any device mapper devices. Useful for viewing LVM2 statistics.

Instead of changing iostat.pl ?

[eschoeller]

@DarkMan

What version of sysstat installs with SLES10?

RHEL5 has: sysstat-7.0.2-3.el5.x86_64
Debian Etch: 7.0.0-4
Ubuntu 8.10: 8.1.2-2ubuntu1

The output of 'iostat -xkd' is identical on all 3 of these platforms.

I am currently working on modifying iostat.pl to work with Solaris, which has a much different output. Instead of modifying the existing OID tables and regex, I am adding some functionality to detect which O/S is running (Solaris or Linux) and parse/report on the data correctly. In order to make iostat.pl portable between Distros and O/S's it sounds like we'll need another conditional to handle SLES 10 ...? Seems messy, I wonder why your output is different than other linux distros ....

[DarkMan]

SLES9 SP3 : 5.0.1-35.7
SLES10 SP1 : 6.0.2-16.19
SLES10 SP2 : 8.0.something

Even downloaded, compiled and installed sysstats 9.0.1. Still get a different output under SLES.

I did consider using -N...until I realised it doesn't exist with the version we have here currently.