Broken Graphs on Motorola Canopy

[debloxie]

hi there

just implemented cacti for our canopies APs and SMs. They polled jus fine but i started noticing broken graphs along the way. Is there an explanation for this?

Am running on Compaq Proliant server is P3 with 1G RAM and cmd.php is still below 10secs.

Here is an excerpt of the cacti log file:

12/18/2006 05:50:04 PM - SYSTEM STATS: Time:2.0720 Method:cmd.php Processes:1 Threads:N/A Hosts:18 HostsPerProcess:18 DataSources:30 RRDsProcessed:14
12/18/2006 05:45:14 PM - RECACHE STATS: RecacheTime:0.2156 HostsRecached:1
12/18/2006 05:45:14 PM - PCOMMAND: Poller[0] Host[32] WARNING: Recache Event Detected for Host
12/18/2006 05:45:14 PM - SYSTEM STATS: Time:12.0799 Method:cmd.php Processes:1 Threads:N/A Hosts:18 HostsPerProcess:18 DataSources:30 RRDsProcessed:9
12/18/2006 05:45:08 PM - CMDPHP: Poller[0] Host[32] DS[29] WARNING: Result from SNMP not valid. Partial Result:
12/18/2006 05:45:06 PM - CMDPHP: Poller[0] ASSERT: '379700<' failed. Recaching host '192.168.10.243', data query #1
12/18/2006 05:40:06 PM - SYSTEM STATS: Time:3.0655 Method:cmd.php Processes:1 Threads:N/A Hosts:17 HostsPerProcess:17 DataSources:30 RRDsProcessed:14
12/18/2006 05:35:16 PM - RECACHE STATS: RecacheTime:0.1963 HostsRecached:1
12/18/2006 05:35:15 PM - PCOMMAND: Poller[0] Host[32] WARNING: Recache Event Detected for Host
12/18/2006 05:35:15 PM - SYSTEM STATS: Time:13.0949 Method:cmd.php Processes:1 Threads:N/A Hosts:16 HostsPerProcess:16 DataSources:28 RRDsProcessed:8
12/18/2006 05:35:11 PM - CMDPHP: Poller[0] Host[33] NOTICE: HOST EVENT: Host Returned from DOWN State:
12/18/2006 05:35:10 PM - CMDPHP: Poller[0] Host[32] DS[29] WARNING: Result from SNMP not valid. Partial Result:
12/18/2006 05:35:08 PM - CMDPHP: Poller[0] Host[32] DS[29] WARNING: Result from SNMP not valid. Partial Result:
12/18/2006 05:35:06 PM - CMDPHP: Poller[0] ASSERT: '319600<' failed. Recaching host '192.168.10.243', data query #1
12/18/2006 05:30:05 PM - SYSTEM STATS: Time:4.0719 Method:cmd.php Processes:1 Threads:N/A Hosts:16 HostsPerProcess:16 DataSources:28 RRDsProcessed:11
12/18/2006 05:30:04 PM - CMDPHP: Poller[0] Host[29] ERROR: HOST EVENT: Host is DOWN Message: Host did not respond to SNMP
12/18/2006 05:30:01 PM - CMDPHP: Poller[0] Host[25] NOTICE: HOST EVENT: Host Returned from DOWN State:
12/18/2006 05:25:13 PM - RECACHE STATS: RecacheTime:0.5209 HostsRecached:2
12/18/2006 05:25:13 PM - PCOMMAND: Poller[0] Host[33] WARNING: Recache Event Detected for Host
12/18/2006 05:25:13 PM - PCOMMAND: Poller[0] Host[32] WARNING: Recache Event Detected for Host
12/18/2006 05:25:12 PM - SYSTEM STATS: Time:11.0909 Method:cmd.php Processes:1 Threads:N/A Hosts:16 HostsPerProcess:16 DataSources:28 RRDsProcessed:9
12/18/2006 05:25:09 PM - CMDPHP: Poller[0] ASSERT: '45600<6600' failed. Recaching host '192.168.10.88', data query #1
12/18/2006 05:25:08 PM - CMDPHP: Poller[0] Host[32] DS[29] WARNING: Result from SNMP not valid. Partial Result:
12/18/2006 05:25:06 PM - CMDPHP: Poller[0] ASSERT: '259500<' failed. Recaching host '192.168.10.243', data query #1
12/18/2006 05:20:04 PM - SYSTEM STATS: Time:3.0739 Method:cmd.php Processes:1 Threads:N/A Hosts:16 HostsPerProcess:16 DataSources:28 RRDsProcessed:12
12/18/2006 05:15:15 PM - RECACHE STATS: RecacheTime:0.2400 HostsRecached:1
12/18/2006 05:15:15 PM - PCOMMAND: Poller[0] Host[32] WARNING: Recache Event Detected for Host
12/18/2006 05:15:14 PM - SYSTEM STATS: Time:13.1117 Method:cmd.php Processes:1 Threads:N/A Hosts:16 HostsPerProcess:16 DataSources:28 RRDsProcessed:7
12/18/2006 05:15:09 PM - CMDPHP: Poller[0] Host[32] DS[29] WARNING: Result from SNMP not valid. Partial Result:
12/18/2006 05:15:07 PM - CMDPHP: Poller[0] ASSERT: '199600<' failed. Recaching host '192.168.10.243', data query #1
12/18/2006 05:10:05 PM - SYSTEM STATS: Time:4.0740 Method:cmd.php Processes:1 Threads:N/A Hosts:16 HostsPerProcess:16 DataSources:28 RRDsProcessed:11
12/18/2006 05:05:15 PM - RECACHE STATS: RecacheTime:0.1910 HostsRecached:1
12/18/2006 05:05:15 PM - PCOMMAND: Poller[0] Host[32] WARNING: Recache Event Detected for Host
12/18/2006 05:05:14 PM - SYSTEM STATS: Time:13.0997 Method:cmd.php Processes:1 Threads:N/A Hosts:16 HostsPerProcess:16 DataSources:28 RRDsProcessed:7
12/18/2006 05:05:09 PM - CMDPHP: Poller[0] Host[32] DS[29] WARNING: Result from SNMP not valid. Partial Result:
12/18/2006 05:05:07 PM - CMDPHP: Poller[0] ASSERT: '139700<' failed. Recaching host '192.168.10.243', data query #1
12/18/2006 05:00:07 PM - SYSTEM STATS: Time:5.0768 Method:cmd.php Processes:1 Threads:N/A Hosts:16 HostsPerProcess:16 DataSources:28 RRDsProcessed:10
12/18/2006 04:55:18 PM - RECACHE STATS: RecacheTime:3.1768 HostsRecached:1
12/18/2006 04:55:15 PM - PCOMMAND: Poller[0] Host[32] WARNING: Recache Event Detected for Host
12/18/2006 04:55:15 PM - SYSTEM STATS: Time:13.0778 Method:cmd.php Processes:1 Threads:N/A Hosts:16 HostsPerProcess:16 DataSources:28 RRDsProcessed:7
12/18/2006 04:55:09 PM - CMDPHP: Poller[0] Host[32] DS[29] WARNING: Result from SNMP not valid. Partial Result:
12/18/2006 04:55:07 PM - CMDPHP: Poller[0] ASSERT: '79800<' failed. Recaching host '192.168.10.243', data query #1
12/18/2006 04:50:09 PM - SYSTEM STATS: Time:7.0588 Method:cmd.php Processes:1 Threads:N/A Hosts:16 HostsPerProcess:16 DataSources:28 RRDsProcessed:8
12/18/2006 04:50:09 PM - CMDPHP: Poller[0] Host[41] ERROR: HOST EVENT: Host is DOWN Message: Host did not respond to SNMP
12/18/2006 04:45:17 PM - RECACHE STATS: RecacheTime:0.2557 HostsRecached:1
12/18/2006 04:45:16 PM - PCOMMAND: Poller[0] Host[32] WARNING: Recache Event Detected for Host
12/18/2006 04:45:16 PM - SYSTEM STATS: Time:14.0858 Method:cmd.php Processes:1 Threads:N/A Hosts:16 HostsPerProcess:16 DataSources:28 RRDsProcessed:7
12/18/2006 04:45:11 PM - CMDPHP: Poller[0] Host[32] DS[29] WARNING: Result from SNMP not valid. Partial Result:
12/18/2006 04:45:09 PM - CMDPHP: Poller[0] Host[32] DS[29] WARNING: Result from SNMP not valid. Partial Result:
12/18/2006 04:45:07 PM - CMDPHP: Poller[0] ASSERT: '19900<' failed. Recaching host '192.168.10.243', data query #1


Meanwhile my other CPUs and radios poll fine with good graphs so i think this is a canopy specific problem

The site is http://41.223.65.5/cacti: login as canopy and password is canopy also.

Does anyone hav any clues for teletronics radios ?

D.

[debloxie]

Just to add this to the story

i checked the cacti log files and found this about device 41 and 40 wch are the canopy AP and one SM respectively

12/18/2006 06:10:04 PM - CMDPHP: Poller[0] Host[41] NOTICE: HOST EVENT: Host Returned from DOWN State:
12/18/2006 06:10:04 PM - CMDPHP: Poller[0] Host[40] NOTICE: HOST EVENT: Host Returned from DOWN State:

If u can make anything of this

Cheers

[gandalf]

Prior to polling any host, cacti checks host availability. If this fails, cacti won't poll this host. Check Settings->Downed Host Detection. On SNMP capable devices, SNMP test should be fine.
Reinhard

[debloxie]

hi

all the devices are snmp capable and my option is snmp wch was descriped as reliable.

i found from the logs that anytime in the cacti log that this occurs:

12/18/2006 06:10:04 PM - CMDPHP: Poller[0] Host[41] NOTICE: HOST EVENT: Host Returned from DOWN State:
12/18/2006 06:10:04 PM - CMDPHP: Poller[0] Host[40] NOTICE: HOST EVENT: Host Returned from DOWN State:

thats when the graphs break. is it that the hosts are alwasy going down, i think probably it affects the AP wch affects the SMs

Cheers

[gandalf]

You may want to lower the maximum snmp get requests settings. Perhaps that device can't respond for that many of SNMP requests in time
Reinhard

[debloxie]

i hav increased the maximum snmp get oids to 30 wch is half the total max of 60. i believe this is just peculiar to canopy whereas others as cpu ethernet cards, cisco routers, terabeam radios all work ok and poll fine.

lets c if it works well.

cheers and thanks

D

[debloxie]

Hi gandalf,

still bringing out the same old broken graphs, shud i increase the snmp get max oids to 60 or some other values. i can reach the devices with less 4ms latency wch is still very ok.

what do u think

D

[gandalf]

No, not increase, decrease. Start at 1, post your findings.
Reinhard

[debloxie]

ok i ll do that and then i ll give u feedback

cheers

D.