CPU utilization OID on Foundry switch graphed as -1.#J

[hammond1]

Foundry gave me the OID of 1.3.6.1.4.1.1991.1.1.2.1.35 for snAgGblCpuUtilData as I wanted to graph CPU utilization on a Foundry switch that I had long been getting interface stats for using Cacti. Version info below:

Cacti 0.8.7b
MySQL 5.0.51a
PHP 5.2.5
Net-SNMP 5.4.1.3
CygWin 2.573.2.3
Apache 2.2.8
OS: Win2K Pro

I created a new SNMP-Generic OID Template data source for this device/Foundry switch using the above OID, left the Data Source Type at GAUGE, and I started seeing graphed data immediately after the next polling cycle.

That only lasted a few days - I have only seen the "-1.#J" as output for this particular graph since, while the interface stats for this device continue to graph data successfully.

I performed an smnpwalk:

snmpwalk -v 1 -c Community_Name x.x.x.x 1.3.6.1.4.1.1991.1.1.2.1.35

The output from the command:

SNMPv2-SMI::enterprises.1991.1.1.2.1.35.0 = Gauge32: 1

(yes, I tried setting the device configuration to SNMP version 2, retrying the command with "-v 2c", same output and graph still shows -1.#J)

cacti.log file has entries stating (the numbers in the brackets for host and DS change):

"CMDPHP: Poller [0] HOST[50] DS[1907] WARNING: Result from SNMP not valid. Partial Result: No Such Instance"

I also tried to rebuild the Poller Cache, not sure how to interpret the results for the Data Source Name associated with the SNMP-Generic OID template:

" SNMP Version: 2, Community: Beatter, OID: 1.3.6.1.4.1.1991.1.1.2.1.35
RRD: C:\Apache2\htdocs\cacti\rra\rack_4_bigiron_snmp_oid_1934.rrd "

In any event I see no change afterwards.

Thanks for any assistance you can provide.

[BSOD2600]

1) Not a windows specific problem...

2) you're not using the correct OID. Try 1.3.6.1.4.1.1991.1.1.2.1.35.0 in your data template. Rebuild your poller cache after doing so.

[hammond1]

Tried appending the ".0" to the OID, then rebuilt the poller cache, and no change, i.e. graph still shows no data while interface stat queries against this Foundry switch work just fine.

Sorry, guess I posted in the wrong category.

Thanks.

[BSOD2600]

1) go back to the poller cache for that query, what is the OID which it's using?

2) change the cacti logging level to medium for a polling cycle. What OID is it using for the Foundry switch and did it retrieve any data?

[hammond1]

The OID per the View Poller Cache>Host = Foundry switch name>Data Source name for this query:

1.3.6.1.4.1.1991.1.1.2.1.35.0

I left the log level at Medium for several polling instances (was distracted by a request after I made the level change), not sure how to correlate log entries to specific queries.

However when I View SNMP Cache and go to this Host I have 673 rows and all of them appear to be related to interface stat queries, I see none Generic SNMP query.

[BSOD2600]

I left the log level at Medium for several polling instances (was distracted by a request after I made the level change), not sure how to correlate log entries to specific queries.

You should be able to see the OID in question being polled. At very least, look for the host[##] entries in the log file which relate to your Foundry switch.

However when I View SNMP Cache and go to this Host I have 673 rows and all of them appear to be related to interface stat queries, I see none Generic SNMP query.

The SNMP cache will only contain things queries from 'Data Input Methods'. Look in the poller cache and filter by the Host. You should see that OID in question.

[hammond1]

I see the following entries related to the OID in the log file (I just did a text search on the OID):

12/16/2008 10:23:02 AM - CMDPHP: Poller[0] Host[3] DS[1934] SNMP: v2: 10.1.1.251, dsname: snmp_oid, oid: 1.3.5.1.4.1.1991.1.1.2.1.35.0, output: U

If I click on View Poller Cache and choose this Foundry switch as the host I see the Data Source for this Generic SNMP query under the "Name* *" column, with ID = 1934, Data Input Method = Get SNMP Data, Poller Interval = 5 minutes, Active = Yes, and Template Name = SNMP - Generic OID Template.

If I click on the Data Source name it takes me to the Data Source config page for this query, and the OID in that config page IS different than the one above: 1.3.6.1.4.1.1991.1.1.2.1.35.0.

It looks like Cacti is polling the OID 1.3.5..... rather than 1.3.6.....

[hammond1]

Oops, misspoke (sorry). I can't cut and paste from the log file because the Cacti host is a VM - can't cut and paste from the VM interface. So I committed a typo when typing out the log entry, i.e. Cacti, per the log file entries, IS querying the OID as configured in the Data Source configuration page:

1.3.6.1.4.1.1991.1.1.2.1.35.0

[BSOD2600]

Alright, well the output: U means it didn't get any data.

When you snmpget 1.3.6.1.4.1.1991.1.1.2.1.35.0, you get data back right? If not, what about an snmpwalk of 1.3.6.1.4.1.1991.1.1.2.1.35.

[hammond1]

snmpget on 1.3.6.1.4.1.1991.1.1.2.1.35.0 returns:

SNMPv2-SMI::enterprises.1991.1.1.2.1.35.0 = Gauge32: 1

snmpwalk on 1.3.6.1.4.1.1991.1.1.2.1.35 returns the same:

SNMPv2-SMI::enterprises.1991.1.1.2.1.35.0 = Gauge32: 1

[BSOD2600]

Alright, then something just isn't making sense here. If you truly put 1.3.6.1.4.1.1991.1.1.2.1.35.0 in the SNMP Generic template OID field and have rebuilt the poller cache, there isn't any reason why this query should not work.

What is the SNMP timeout for the device in cacti? Maybe try increasing it some.

[hammond1]

Yes that is the OID in the Data Source for this query. I have old eyes so I just double-checked, character-by-character, that the OID is correct. I also performed another poller rebuild and still don't see data being graphed.

And....Foundry reports (they went to their lab and backline engineers in addition to the documentation for this switch) and there is no configurable timeout for SNMP queries.

[BSOD2600]

Yes that is the OID in the Data Source for this query. I have old eyes so I just double-checked, character-by-character, that the OID is correct. I also performed another poller rebuild and still don't see data being graphed.

Well then I'm at a total loss why a manual snmpget works and cacti's does not. They do the same thing.

And....Foundry reports (they went to their lab and backline engineers in addition to the documentation for this switch) and there is no configurable timeout for SNMP queries.

I was referring to the snmp timeout in CACTI for that device.

[hammond1]

I increased the timeout (Config>Settings>SNMP Timeout) from 500 to 1000, and increased the number of SNMP retries from 1 to 5. I then waited for a couple of polling cycles to happen and no change.

Sorry to provide a stumper. Perhaps some packet captures, one taken whilest performing the manual snmpwalk/get, another during the polling cycle, would help. Of course the latter capture would have a ton of packets between this Foundry switch and the host.

I'll give that a shot tomorrow, almost out of here for the day.

Thanks.

[hammond1]

Performed the packet capture using Wireshark, filtering for the switch IP with "host x.x.x.x". Once started I performed the following command:

snmpwalk -v 2c -c Community_Name x.x.x.x 1.3.6.1.4.1.1991.1.1.2.1.35

Received the response:

SNMPv2-SMI::enterprises.1991.1.1.2.1.35.0 = Gauge32: 91

And let the capture run until after the next polling cycle.

There are four packets in the capture related to the snmpwalk manual request:

1) From the Cacti host to the switch, an SNMP packet with Info "get-next-request SNMPv2-SMI::enterprises.1991.1.1.2.1.35"
2) From the switch to the Cacti host, an SNMP packet with Info "get-response SNMPv2-SMI::enterprises.1991.1.1.2.1.35.0"
3) From the Cacti host to the switch, an SNMP packet with Info "get-next-response SNMPv2-SMI::enterprises.1991.1.1.2.1.35.0
4) From the switch to the Cacti host, an SNMP packet with Info "get-response SNMPv2-SMI::enterprises.1991.1.1.2.1.36.0

During the polling cycle part of the capture there were two packets related to this OID query:

1) From the Cacti host to the switch, an SNMP packet with Info "get-request SNMPv2-MIB::sysUpTime.6.1.4.1.1991.1.1.1.2.1.35.0"
2) From the switch to the Cacti host, an SNMP packet with Info "get-response SNMPv2-MIB::sysUpTime.6.1.4.1.1991.1.1.2.1.35.0"

Nowhere else in the this packet capture do I see "sysUpTime" in the Info field as part of the OID - instead I see as the text part of the OID ifInOctets, ifOutOctets, or enterprises but then again the CPU Utilization OID is the only Generic SNMP query I am performing against this switch.

Don't know if this will help or not.

Thanks,