Drive Space graphs blank

[libsys]

I've had a weird error happen for a RHEL AS 3 box I'm graphing with cacti. We recently ran an up2date update on the server, which patched various programs. After this happened, cacti stopped graphing anything on the box. I eventually discovered that the update had run the RPM'd version of snmpd over the top of the version I built from source. I run my own build instead of the RPM packaged version. Once I restarted the correct snmpd everything seemed.

That was about 2 weeks ago, and I've just now realised that the drive space graphs for this server aren't showing anything - all the other graphs are. I'm graphing interface traffic, cpu, memory and processes as well as mounted partitions, and they are graphing ok.

Not being a linux guru, I'm having some issues working out which OID cacti is using to get the partition information. I'm using the "Host MIB - Hard Drive Space" data template as the graph's data template.

Can anyone help me out with this? Even if someone can tell me which OID is used for this snmp query, maybe I can see if this has been changed by the upgrade.

[libsys]

In addition to the above, here is the debug output of the "SNMP - Get Mounted Partitions" data query for the server that's at fault:

+ Running data query [8].
+ Found type = '6 '[script query].
+ Found data query XML file at '/www/cacti.lib.tils.qut.edu.au/resource/script_server/host_disk.xml'
+ XML file parsed ok.
+ Executing script for list of indexes '/usr/local/php/bin/php -q /www/cacti.lib.tils.qut.edu.au/scripts/ss_host_disk.php jasper.lib.tils.qut.edu.au 3 3:161:500:libsys:libsys:JFD8cV6J:MD5::[None] index'
+ Executing script query '/usr/local/php/bin/php -q /www/cacti.lib.tils.qut.edu.au/scripts/ss_host_disk.php jasper.lib.tils.qut.edu.au 3 3:161:500:libsys:libsys:JFD8cV6J:MD5::[None] query index'
+ Executing script query '/usr/local/php/bin/php -q /www/cacti.lib.tils.qut.edu.au/scripts/ss_host_disk.php jasper.lib.tils.qut.edu.au 3 3:161:500:libsys:libsys:JFD8cV6J:MD5::[None] query description'
+ Executing script query '/usr/local/php/bin/php -q /www/cacti.lib.tils.qut.edu.au/scripts/ss_host_disk.php jasper.lib.tils.qut.edu.au 3 3:161:500:libsys:libsys:JFD8cV6J:MD5::[None] query sau'
+ Found data query XML file at '/www/cacti.lib.tils.qut.edu.au/resource/script_server/host_disk.xml'
+ Found data query XML file at '/www/cacti.lib.tils.qut.edu.au/resource/script_server/host_disk.xml'
+ Found data query XML file at '/www/cacti.lib.tils.qut.edu.au/resource/script_server/host_disk.xml'
+ Found data query XML file at '/www/cacti.lib.tils.qut.edu.au/resource/script_server/host_disk.xml'
+ Found data query XML file at '/www/cacti.lib.tils.qut.edu.au/resource/script_server/host_disk.xml'
+ Found data query XML file at '/www/cacti.lib.tils.qut.edu.au/resource/script_server/host_disk.xml'
+ Found data query XML file at '/www/cacti.lib.tils.qut.edu.au/resource/script_server/host_disk.xml'
+ Found data query XML file at '/www/cacti.lib.tils.qut.edu.au/resource/script_server/host_disk.xml'
+ Found data query XML file at '/www/cacti.lib.tils.qut.edu.au/resource/script_server/host_disk.xml'
+ Found data query XML file at '/www/cacti.lib.tils.qut.edu.au/resource/script_server/host_disk.xml'

Compare this to one that is working (both servers are Redhat linux, using the same net-snmp client):

+ Running data query [8].
+ Found type = '6 '[script query].
+ Found data query XML file at '/www/cacti.lib.tils.qut.edu.au/resource/script_server/host_disk.xml'
+ XML file parsed ok.
+ Executing script for list of indexes '/usr/local/php/bin/php -q /www/cacti.lib.tils.qut.edu.au/scripts/ss_host_disk.php localhost 6 3:161:500:libsys:libsys:JFD8cV6J:MD5::[None] index'
+ Executing script query '/usr/local/php/bin/php -q /www/cacti.lib.tils.qut.edu.au/scripts/ss_host_disk.php localhost 6 3:161:500:libsys:libsys:JFD8cV6J:MD5::[None] query index'
+ Found item [hrStorageIndex='1'] index: 1
+ Found item [hrStorageIndex='2'] index: 2
+ Found item [hrStorageIndex='3'] index: 3
+ Found item [hrStorageIndex='4'] index: 4
+ Found item [hrStorageIndex='5'] index: 5
+ Found item [hrStorageIndex='6'] index: 6
+ Found item [hrStorageIndex='7'] index: 7
+ Found item [hrStorageIndex='8'] index: 8
+ Found item [hrStorageIndex='9'] index: 9
+ Found item [hrStorageIndex='10'] index: 10
+ Found item [hrStorageIndex='11'] index: 11
+ Found item [hrStorageIndex='12'] index: 12
+ Found item [hrStorageIndex='13'] index: 13
+ Found item [hrStorageIndex='14'] index: 14
+ Executing script query '/usr/local/php/bin/php -q /www/cacti.lib.tils.qut.edu.au/scripts/ss_host_disk.php localhost 6 3:161:500:libsys:libsys:JFD8cV6J:MD5::[None] query description'
+ Found item [hrStorageDescr='Memory Buffers'] index: 1
+ Found item [hrStorageDescr='Real Memory'] index: 2
+ Found item [hrStorageDescr='Swap Space'] index: 3
+ Found item [hrStorageDescr='/'] index: 4
+ Found item [hrStorageDescr='/sys'] index: 5
+ Found item [hrStorageDescr='/archive'] index: 6
+ Found item [hrStorageDescr='/boot'] index: 7
+ Found item [hrStorageDescr='/home'] index: 8
+ Found item [hrStorageDescr='/usr'] index: 9
+ Found item [hrStorageDescr='/var'] index: 10
+ Found item [hrStorageDescr='/www'] index: 11
+ Found item [hrStorageDescr='/proc/sys/fs/binfmt_misc'] index: 12
+ Found item [hrStorageDescr='/var/lib/nfs/rpc_pipefs'] index: 13
+ Found item [hrStorageDescr='/proc/fs/nfsd'] index: 14
+ Executing script query '/usr/local/php/bin/php -q /www/cacti.lib.tils.qut.edu.au/scripts/ss_host_disk.php localhost 6 3:161:500:libsys:libsys:JFD8cV6J:MD5::[None] query sau'
+ Found item [hrStorageAllocationUnits='1024'] index: 1
+ Found item [hrStorageAllocationUnits='1024'] index: 2
+ Found item [hrStorageAllocationUnits='1024'] index: 3
+ Found item [hrStorageAllocationUnits='1024'] index: 4
+ Found item [hrStorageAllocationUnits='4096'] index: 5
+ Found item [hrStorageAllocationUnits='4096'] index: 6
+ Found item [hrStorageAllocationUnits='1024'] index: 7
+ Found item [hrStorageAllocationUnits='4096'] index: 8
+ Found item [hrStorageAllocationUnits='4096'] index: 9
+ Found item [hrStorageAllocationUnits='4096'] index: 10
+ Found item [hrStorageAllocationUnits='4096'] index: 11
+ Found item [hrStorageAllocationUnits='4096'] index: 12
+ Found item [hrStorageAllocationUnits='4096'] index: 13
+ Found item [hrStorageAllocationUnits='4096'] index: 14
+ Found data query XML file at '/www/cacti.lib.tils.qut.edu.au/resource/script_server/host_disk.xml'
+ Found data query XML file at '/www/cacti.lib.tils.qut.edu.au/resource/script_server/host_disk.xml'
+ Found data query XML file at '/www/cacti.lib.tils.qut.edu.au/resource/script_server/host_disk.xml'
+ Found data query XML file at '/www/cacti.lib.tils.qut.edu.au/resource/script_server/host_disk.xml'
+ Found data query XML file at '/www/cacti.lib.tils.qut.edu.au/resource/script_server/host_disk.xml'
+ Found data query XML file at '/www/cacti.lib.tils.qut.edu.au/resource/script_server/host_disk.xml'
+ Found data query XML file at '/www/cacti.lib.tils.qut.edu.au/resource/script_server/host_disk.xml'

Any help would be greatly appreciated.

[gandalf]

This seems to be a snmpd configuration problem fpor those target devices. Please try to snmpwalk the OIDs found in the script manually against both kind of targets
Reinhard

[pbrubaker]

Have you made sure that it's not an SELinux problem?

[libsys]

This seems to be a snmpd configuration problem fpor those target devices. Please try to snmpwalk the OIDs found in the script manually against both kind of targets

I'm not sure how to find this out. I'm finding it difficult to determine exactly which script is being run. The graphs in question are using the "ucd 90 Filesystems" graph template. The data source for this graph uses the "Host MIB - Hard Drive Space" data template. This data template uses the "Get Script Server Data (Indexed)" data input method.

It's at this point I'm not sure how to continue further and actually work out which script is being run, and therefore how to determine the OID of the component in question. If anyone can offer any help or advice in regards to this, I'm all ears

Have you made sure that it's not an SELinux problem?

I'm not sure where to start looking. I did a bit of a Google search but didn't turn up anything helpful. Unfortunately, I don't know that much about selinux.

Thanks,
Guy

[gandalf]

I'm not sure how to find this out. I'm finding it difficult to determine exactly which script is being run. The graphs in question are using the "ucd 90 Filesystems" graph template. The data source for this graph uses the "Host MIB - Hard Drive Space" data template. This data template uses the "Get Script Server Data (Indexed)" data input method.

Try

Code: Select all

snmpwalk -c <community string> -v 1 <target> host

Perhaps you'll have to substitute "host" by "hrstorage" or "storage" (sorry, I'm currently on a non-SNMP device, so I can't verify)
Reinhard

[libsys]

Perhaps you'll have to substitute "host" by "hrstorage" or "storage" (sorry, I'm currently on a non-SNMP device, so I can't verify)

Ok, well as we're using SNMP v3, I used the following command:

./snmpwalk -v 3 -c #### -u #### -l authNoPriv -a MD5 -A #### localhost hrStorage

The username, community string and password were definitely all correct. When I run the command, I get this error:

HOST-RESOURCES-MIB::hrStorage = No Such Object available on this agent at this OID

The same thing happened when I tried using just "storage" as the OID.

[gandalf]

Just checked it. Please query against "host", e.g.

Code: Select all

snmpwalk .... <target> host

You should get the Storage Table (see e.g. last link of my signature containg some examples on that)
Reinhard

[libsys]

Just checked it. Please query against "host"

Thanks for your continued help. Unfortunately, I tried the above and got the same error message.

I've done an snmpwalk on the whole MIB and can't see any HOST-RESOURCES-MIB in the output. Is it possible that this branch was wiped by the redhat update? If so, how do I go about getting it back?

[gandalf]

The host mib is not mandatory. Please try

Code: Select all

snmpwalk -c <community string> -v 1  <target> version
UCD-SNMP-MIB::versionIndex.0 = INTEGER: 1
UCD-SNMP-MIB::versionTag.0 = STRING: 5.1.2
UCD-SNMP-MIB::versionDate.0 = STRING: $Date: 2004/02/04 11:25:20 $
UCD-SNMP-MIB::versionCDate.0 = STRING: Mon May 22 17:58:21 2006
UCD-SNMP-MIB::versionIdent.0 = STRING: $Id: versioninfo.c,v 5.1.4.1 2004/02/04 11:25:20 slif Exp $
UCD-SNMP-MIB::versionConfigureOptions.0 = STRING: "'--build=i386-redhat-linux' '--host=i386-redhat-linux' '--target=i386-redhat-linux-gnu' '--program-prefix=' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--datadir=/usr/share' '--includedir=/usr/include/net-snmp' '--libdir=/usr/lib' '--libexecdir=/usr/libexec' '--localstatedir=/var' '--sharedstatedir=/usr/com' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--enable-static' '--enable-shared' '--with-cflags=-O2 -g -pipe -m32 -march=i386 -mtune=pentium4' '--with-sys-location=Unknown' '--with-logfile=/var/log/snmpd.log' '--with-persistent-directory=/var/net-snmp' '--with-mib-modules=host agentx smux ucd-snmp/lmSensors' '--with-libwrap=/usr/lib' '--sysconfdir=/etc' '--enable-ipv6' '--enable-ucd-snmp-compatibility' '--with-pic' '--with-sys-contact=root@localhost' 'CFLAGS=-O2 -g -pipe -m32 -march=i386 -mtune=pentium4' 'build_alias=i386-redhat-linux' 'host_alias=i386-redhat-linux' 'target_alias=i386-redhat-linux-gnu'"

and find host as one of the --with-mib-modules directives
Reinhard

[libsys]

Thanks for the help - I finally got it working again. I had to recompile net-snmp with the host mib module. I'm not sure why it was working before and then stopped, as I hadn't changed anything. I certainly hadn't reinstalled net-snmp at any time since it was originally installed. The weird thing is that I never specified the host mib module when I originally installed net-snmp so I fail to understand why it worked before and then stopped.

In any case, this fixed it and it's now working. I'm very happy