Cacti thinks device is down, but snmpwalk works fine

[MaffooClock]

I don't think this is a windows-specific question, so I hope I'm not posting this in the wrong place.

I have a Linux-based Cacti box collecting data from three Linux-based boxes and three Windows 2003 boxes.

One of the Windows 2003 hosts (Melbourne) suddenly shows "Down" and no data is being graphed anymore. However, Melbourne is online and working perfectly. When clicking the name of the device to view its settings, the heading at the top does produce SNMP data:

Code: Select all

System: Hardware: x86 Family 15 Model 0 Stepping 7 AT/AT COMPATIBLE - Software: Windows Version 5.2 (Build 3790 Uniprocessor Free)
Uptime: 85134 (0 days, 0 hours, 14 minutes)
Hostname: MELBOURNE
Location:
Contact: Matthew Clark

Also, performing an snmpwalk on Melbourne from the Cacti PC produces thousands of lines of data:

Code: Select all

root@Woolhara:~# snmpwalk -v 2c -c public melbourne
SNMPv2-MIB::sysDescr.0 = STRING: Hardware: x86 Family 15 Model 0 Stepping 7 AT/AT COMPATIBLE - Software: Windows Version 5.2 (Build 3790 Uniprocessor Free)
SNMPv2-MIB::sysObjectID.0 = OID: SNMPv2-SMI::enterprises.311.1.1.3.1.2
DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (136295) 0:22:42.95
SNMPv2-MIB::sysContact.0 = STRING: Matthew Clark
SNMPv2-MIB::sysName.0 = STRING: MELBOURNE
SNMPv2-MIB::sysLocation.0 = STRING:
SNMPv2-MIB::sysServices.0 = INTEGER: 79
<snip>

Nothing has changed as far as I can think of. It should go without saying, but I have ensured the SNMP service is running on Melbourne. I have rebooted Melbourne. I have removed the SNMP service and re-installed on Melbourne. Baed on the conditions, I didn't think these actions would help, but you always have to start at the bottom and work your way up...

I even deleted the entire device and all associated graphs from Cacti and re-created from scratch. Still down. This would seem to isolate the problem to Melbourne itself, but that's contradictory to the fact that snmpwalk returns (lots of) data.

Yet, all other devices in Cacti are working -- no issues at all.

So why would Cacti think a host is down, and what could I possibly be missing? What checks can I execute that might narrow down the possibilities? I am highly proficient with PHP, Linux, and Cacti, but I am not knowledgable enough about the internals of Cacti to go rooting around in the source without a flashlight...

[BSOD2600]

What Downed Host Detection are you using? If ping, I suggest you switch to ICMP instead of UDP.

[MaffooClock]

"Ping and SNMP" is my current Downed Host Detection method.
I suppose I can have cacti run as root and change the UDP ping to ICMP...

[fmangeant]

Hi

or you can use "SNMP" as downed detection method, and run the poller as "cactiuser".

[MaffooClock]

Hot dog. Running the poller as root did the trick (I left the Down Host Detection at "Ping and SNMP", but changed ping type to ICMP).

I'll try fmangeant's suggestion next...

[torstentfk]

HI,

I have now the same problem:
the proxy-server is reachable and respones to the snmp queries; snmpget and snmpgetnext to get OIDs. .1.3.6.1.2.1.1.3.0 und .1.3.6.1.2.1.1.1.0. is working; server is pingable.

The poller (poller.php) logs that the host is not reachable.
If I use the method "ping" cacti/poller.php is recaching the server each polling cycle.

I use php5 with snmp support, debian. The server is very restrictive to snmp polls. Which OIDs and trees must I alow that cacti assumes that the host is up?

Torsten

[gandalf]

In latest 0.8.6j it is .1 and .1.3. Cacti uses snmpgetnext on those OIDs
Reinhard

[torstentfk]

In latest 0.8.6j it is .1 and .1.3. Cacti uses snmpgetnext on those OIDs
Reinhard

Hi Reinhard, the new guru...

on the CLI I get fetch these OIDs and get a correct answer. But the server are still down in cacti.

Torsten

[gandalf]

Seems, that a tcpdump/wireshark trace is required ...
Reinhard

[torstentfk]

Seems, that a tcpdump/wireshark trace is required ...
Reinhard

OK:

Code: Select all

>#tcpdump -s 1500 host XXXX
10:27:17.684592 IP cacti-srv 32791 > proxy.3401: UDP, length 38
10:27:17.685234 IP proxy > cacti-srv: ICMP proxy udp port 3401 unreachable, length 74

Settings for the poller: Method:SNMP-only, Ping:UDP.
If I change these settings, there is no change at the tcpdump.
I do not see any snmp-send request as the poller starts.

Torsten

[gandalf]

If you are polling a squid proxy, this IS the snmp request! Squid receives SNMP requests at port 3401 by default. But I'm waiting for the UDP ping request to come up: nothing more found in the trace? The UDP ping should come first!
Reinhard

[torstentfk]

Hi,

aeh --- you are right; I was searching the data templates and found this port (3401) . This was not accessable through the firewall. So cacti checks first and then polls - if it can! I thought that the proxy query is a snmp:161 poll, that was wrong.

Sorry - my fault.

Torsten

[TheWitness]

So, no bug then? If it is, please open one at bugs.cacti.net. Be as specific as possible. Also, please note, I have versioned ping.php. Chech that first before placing your bug report.

Thanks,

TheWitness

[Criggie]

...you can use "SNMP" as downed detection method, and run the poller as "cactiuser".

That worked perfectly for me -

Downed Host Detection SNMP Reliable
Ping ICMP ping

and poller runs as cactiuser.

Now, the two hosts in question were freeBSD and windows XP. The linux boxes and printers on the network were fine the whole time.

How can I modify the records to note that neither machien was really down in that time? My freeBSD box had a 100% uptime, now its down to 99.5%