Monitor the iowait in a server

[shingouki]

hi Team,

anyone have successfully monitored and graphed "iowait" percentage stats(like the iowait figures in "top" command) using cacti and net-snmp? Kindly let me know which OIDs are involved. Many thanks.

Shingouki

[gandalf]

Yes, I have. But "unfortunately" I'm on vacation this week. Will check next week at my company and probably release some template on this.
Reinhard

[fmangeant]

Hi

you can use the .1.3.6.1.4.1.2021.11 OID (.iso.org.dod.internet.private.enterprises.ucdavis.systemStats) :

Code: Select all

$ snmpwalk -v 1 -c public localhost .1.3.6.1.4.1.2021.11
UCD-SNMP-MIB::ssIndex.0 = INTEGER: 1
UCD-SNMP-MIB::ssErrorName.0 = STRING: systemStats
UCD-SNMP-MIB::ssSwapIn.0 = INTEGER: 0
UCD-SNMP-MIB::ssSwapOut.0 = INTEGER: 0
UCD-SNMP-MIB::ssIOSent.0 = INTEGER: 0
UCD-SNMP-MIB::ssIOReceive.0 = INTEGER: 0
UCD-SNMP-MIB::ssSysInterrupts.0 = INTEGER: 2
UCD-SNMP-MIB::ssSysContext.0 = INTEGER: 2
UCD-SNMP-MIB::ssCpuUser.0 = INTEGER: 19
UCD-SNMP-MIB::ssCpuSystem.0 = INTEGER: 5
UCD-SNMP-MIB::ssCpuIdle.0 = INTEGER: 75
UCD-SNMP-MIB::ssCpuRawUser.0 = Counter32: 526125149
UCD-SNMP-MIB::ssCpuRawNice.0 = Counter32: 495988
UCD-SNMP-MIB::ssCpuRawSystem.0 = Counter32: 145828721
UCD-SNMP-MIB::ssCpuRawIdle.0 = Counter32: 2070525039
UCD-SNMP-MIB::ssCpuRawWait.0 = Counter32: 437165708
UCD-SNMP-MIB::ssCpuRawKernel.0 = Counter32: 138137451
UCD-SNMP-MIB::ssCpuRawInterrupt.0 = Counter32: 3567108
UCD-SNMP-MIB::ssIORawSent.0 = Counter32: 3192209518
UCD-SNMP-MIB::ssIORawReceived.0 = Counter32: 2851248440
UCD-SNMP-MIB::ssRawInterrupts.0 = Counter32: 4294967295
UCD-SNMP-MIB::ssRawContexts.0 = Counter32: 4294967295
UCD-SNMP-MIB::ssCpuRawSoftIRQ.0 = Counter32: 4124162
UCD-SNMP-MIB::ssRawSwapIn.0 = Counter32: 1594923
UCD-SNMP-MIB::ssRawSwapOut.0 = Counter32: 11776445

[shingouki]

thanks, also are there any ways I can calculate the percentage in IOwait like an output in top command from the remote host among thse OIDs?

[gandalf]

Well, I had a look at my templates. But the one I've written uses the iostat command at cli, transport via http and local parsing of the output at the cacti server. So that's not a thing to be proud of; will not release this to public
Reinhard

PS: will think about implementing it based on the host mib, but my case belongs to some AIX hosts without net-snmp ...

[shingouki]

hi,

you mean from remote host do "iostat" and then send the result using "http" to another host or to the cacti server? What tools are required, that one made me interested?

Regards,

Shingouki

Well, I had a look at my templates. But the one I've written uses the iostat command at cli, transport via http and local parsing of the output at the cacti server. So that's not a thing to be proud of; will not release this to public
Reinhard

PS: will think about implementing it based on the host mib, but my case belongs to some AIX hosts without net-snmp ...

[gandalf]

Well, in my case (AIX box running IBM HTTP Sever which is kinda Apache), I didn't have the chance for snmp-pass or snmp-exec scripts and there is not HOST MIB AFAIK. So I wrote some cgi wrapper for "iostat" to parse the cpu data. The wrapper returns some html/text that in turn is "parsed" by a cacti-perl-script. This cacti-perl-script retrieves the data via wget-ting the cgi-url.
So, this is kinda home-brewn snmp-exec replacement by http-cgi. Nothing to be fond of. And of course you may use this for wget-ting other data as well. And it's only feasible if you're already running an http server on the target host. Remember all those security stuff ...
If someone is interested in this, I may post the target-host-cgi-script, the cacti-prtl-script and the related templates as well.
Reinhard

[shingouki]

sure dude those scripts and templates of yours will come very useful, could you post them here and share with us pls, many thanks.

Shingouki

Well, in my case (AIX box running IBM HTTP Sever which is kinda Apache), I didn't have the chance for snmp-pass or snmp-exec scripts and there is not HOST MIB AFAIK. So I wrote some cgi wrapper for "iostat" to parse the cpu data. The wrapper returns some html/text that in turn is "parsed" by a cacti-perl-script. This cacti-perl-script retrieves the data via wget-ting the cgi-url.
So, this is kinda home-brewn snmp-exec replacement by http-cgi. Nothing to be fond of. And of course you may use this for wget-ting other data as well. And it's only feasible if you're already running an http server on the target host. Remember all those security stuff ...
If someone is interested in this, I may post the target-host-cgi-script, the cacti-prtl-script and the related templates as well.
Reinhard

[gandalf]

Now, here's the result. But it's kinda complicated, because iostat/vmstat, when called once, will report strange data for CPU usage. So I decided to do some averaging out; the examples use 16 cycles, 1 seconds each, and takes the average of the last 15 values. You may decide to change this to your own needs.
To shed some light on this, please try the following

Code: Select all

iostat
iostat -t 1 2
iostat -t 1 16

If your installation does not include iostat bit vmstat, you may use

Code: Select all

vmstat
vmstat 1 2
vmstat 1 16

instead. You will notice, that the first line holds some strange results.
On the remote host, you will need a webserver (this webserver will be queried for the results). Please install attached file iostat_gen.sh or vmstat_gen.sh (only one of them needed) to a directory, where it can be run from crontab. Please modify the crontab of the remote host to run this script every 5 min. It's output is written to /tmp/iostat.html or /tmp/vmstat.html and must be readable from your webserver. You may of course change this to your needs.
Now, how does the polling cacti server access this data? In this case, snmp is not used. Instead, we will use http (due to this, you need a webserver on the remote host), access method is wget. Please find attached script lvm_wget.pl. Put this one into <path_cacti>/scripts. It's a perl script and takes three parameters
- hostname (of the remote host)
- port (the webserver's listen port, defaults to 80)
- url on that very host
From this, the script forms some kind of wget against http://<hostname>:<port>/url.
At least, you need the data input method, data template and graph template stuff. Please import the attached XML to get this.

There's another way to use this lvm_wget.pl. If your webserver supplies other interesting data via cgi scripts, you may simply define the url accordingly. Please pay attention to the correct formatting in that cgi script (must be "cacti compatible"). If it is not, you may add some parsing code to a copy of lvm_wget.pl.

[gandalf]

Here's the rest of it (cannot have more than three attachments to a post )
Reinhard

[cyberconte]

Those 'strange results' in the first set are averages since system boot.

[gandalf]

Those 'strange results' in the first set are averages since system boot.

????
What are you going to say?
Reinhard

[dinfiesta]

Runing this on my redhat (I do have iostat installed)

iostat -t 1 16|tail -n 15| awk '{n++;user+=$(NF-3);sys+=$(NF-2);idle+=$(NF-1);iowait+=$(NF)} END{print "user:"user/n " sys:"sys/n " idle:"idle/n " iowait:"iowait/n}' > /tmp/iostat.html

I get this error:

awk: cmd. line:1: (FILENAME=- FNR=3) fatal: attempt to access field -3

Any ideas?

[gandalf]

Sorry, but I'm not that awk guy. Did only copy that by myself....
Reinhard

[dinfiesta]

Sorry, but I'm not that awk guy. Did only copy that by myself....
Reinhard

Thank you!
Anyone that has my same problem?