Script to check mem paging with sar - no data in 1 field

[guido90210]

Hi folks,

Using cacti 0.8.6h with MySQL 3.23.58 (rpm) and rrdtool 1.2.12 on RHEL3 HP bl20p blade server...

I just started using cacti last week, and was very impressed with the graphs it came up with 'out of the box'.

Anyway, I want to graph memory paging activity using the output from 'sar -B', initially on the cacti host itself. I wrote this petite shell script:

#!/bin/sh
sar -B|tail -2|head -1|awk '{printf "paged_in:" $3 " paged_out:" $4 " page_faults:" $5 " major_faults:" $6}'

Running it from the command line works - gives this:

[root@oragrid scripts]# sh ./memory_paging.sh
paged_in:1.32 paged_out:76.58 page_faults:0.00 major_faults:0.00[root@oragrid scripts]#

(note that there is no carriage return at the end of the output - hope that's OK.)

However, cacti seems to only get 3 of the 4 fields - it never gets the value for 'major_faults'. It does, however, set up the RRA with the four fields.

When I turn debugging on in cacti, delete the RRA for this service and look at cacti.log, it says this:

05/08/2006 02:00:07 PM - POLLER: Poller[0] Parsed MULTI output field 'paged_in:70.76' [map paged_in->paged_in]
05/08/2006 02:00:07 PM - POLLER: Poller[0] Parsed MULTI output field 'paged_out:0.00' [map paged_out->paged_out]
05/08/2006 02:00:07 PM - POLLER: Poller[0] Parsed MULTI output field 'page_faults:0.00' [map page_faults->page_faults]

Then this a few lines further down:

05/08/2006 02:00:07 PM - POLLER: Poller[0] CACTI2RRD: /usr/local/inst/rrdtool-prod/bin/rrdtool create /usr/local/inst/cacti-0.8.6h/rra/oragrid_mem_pag_stats_22.rrd --step 300 DS:paged_in:GAUGE:600:0:10000 DS:major_faults:GAUGE:600:0:10000 DS:paged_out:GAUGE:600:0:10000 DS:page_faults:GAUGE:600:0:100 RRA:AVERAGE:0.5:1:600 RRA:AVERAGE:0.5:6:700 RRA:AVERAGE:0.5:24:775 RRA:AVERAGE:0.5:288:797 RRA:MAX:0.5:1:600 RRA:MAX:0.5:6:700 RRA:MAX:0.5:24:775 RRA:MAX:0.5:288:797
05/08/2006 02:00:07 PM - POLLER: Poller[0] CACTI2RRD: /usr/local/inst/rrdtool-prod/bin/rrdtool update /usr/local/inst/cacti-0.8.6h/rra/oragrid_mem_pag_stats_22.rrd --template paged_in:paged_out:page_faults N:70.76:0.00:0.00

This has got me confoozed... anyone got any ideas?

Thanks,
G.

[gandalf]

Please post a screenshot of the Data Input Methods definition screen
Reinhard

[guido90210]

Hi Reinhard,

Thanks... here's the Data Input Methods screen:

[guido90210]

May as well post the other screens too... here's the Data Template screen:

[guido90210]

Here's the graph template screen:

[gandalf]

Those resources seem to be ok. Which poller are you running?
Reinhard

[guido90210]

I'm running the standard php poller (poller.php or whatever)... this is in my cactiuser crontab file:

*/5 * * * * /usr/bin/php /usr/local/inst/cacti-prod/poller.php > /dev/null 2>&1

Thanks,
G.

[gandalf]

Ok, should have asked more precise. Which poller is configured in Settings ? There's a dropdown for either cactid or cmd.php
Reinhard

[guido90210]

Aah, sorry, it's cmd.php.

Thanks,
G.

[gandalf]

Then I'm clueless. Will ask a dev
Reinhard

[guido90210]

Thanks Reinhard... I've been pretty busy with other things the past few days, but when I get some time I'll try to exclude some variables (OS, software versions etc.) and see where I get to. I'll post back to the list if/when I find out anything.

Ciao,
G.

[TheWitness]

Are you all patched? This reminds me of a patch that was placed into Cacti a few point releases ago. I would also add the "\n".

Larry

[guido90210]

Howdy,

I'm using 0.8.6h, and haven't applied any patches to that... is that OK? Would I need to apply the patches listed for 0.8.6h?

I looked back at the patches for previous versions, and the patch entitled "Multiple Output Field Scripts with the Script Server" in the 0.8.6f patch section seems to possibly relate to this issue, but I checked, and my files do seem to contain that code.

I've been playing around with this a bit, and have noticed the following:

* When I replace my memory_paging.sh script with this:
#!/bin/sh
echo paged_in:1.23 paged_out:4.56 page_faults:7.89 major_faults:0.12

, delete the rra file and let the poller run again, it works fine, i.e. it gets values for all four fields OK.

* When I re-order the printing of the fields from the original script, like this:
#!/bin/sh
sar -B|tail -2|head -1|awk '{printf "major_faults:" $6 " paged_in:" $3 " paged_out:" $4 " page_faults:" $5}'

, it still doesn't get a value for major_faults (but the other 3 fields are OK).

* When I print the data fields in the same order but change the names, like this:
#!/bin/sh
sar -B|tail -2|head -1|awk '{printf "major_faults:" $3 " paged_in:" $4 " paged_out:" $5 " page_faults:" $6}'

, the first three fields are fine, but the last field ('page_faults') becomes the one that contains 'NaN' in the rra file.

Therefore, the problem seems to lie with the last field printed in my script.

It happens without a \n on the end of the line and with a \n on the end of the line.

Hmmmmmmm.............

Looks like I might have to cron my 'sar' script to run just before poller.php, and have the 'sar' script write the output to a file, which another script simply reads and echoes, and have Cacti use the second ('echo') script to get the data.

Unless anyone else has any other ideas...?

Ciao,
G.

[TheWitness]

Doesn't make sense. I will have to test this week sometime.

TheWitness

[guido90210]

Hello,

I looked in my cacti log with debugging turned on, and saw this:

05/18/2006 04:25:05 PM - CMDPHP: Poller[0] DEBUG: SQL Exec: "insert into poller_output (local_data_id,rrd_name,time,output) values (22,'','2006-05-18 16:25:02','paged_in:134.82 paged_out:0.00 page_faults:0.00')"

The problem does appear to be when the poller runs the script - it only gets 3 of the 4 values back. I noticed a (possibly) similar issue in the thread entitled "Cacti2RRD Only Updating One DS" in this forum.

Thanks,
G.