Intermittent missed polls or missed processing

[lkwong]

I have been noticing the following consistent problem. It seems that every few hours, CACTI will miss processing or polling for all of my data sources. Since this is occurring consistently, I do not believe it is due to loss of network connectivity to the devices (from my server). Every time this occurs, I get a small spike on my graphs. The CACTI log for these occurrences always looks like the following:

01/16/2007 08:33:12 AM - SYSTEM STATS: Time:11.1490 Method:cmd.php Processes:1 Threads:N/A Hosts:9 HostsPerProcess:9 DataSources:85 RRDsProcessed:46
01/16/2007 08:34:12 AM - SYSTEM STATS: Time:11.1404 Method:cmd.php Processes:1 Threads:N/A Hosts:9 HostsPerProcess:9 DataSources:85 RRDsProcessed:46
01/16/2007 08:37:41 AM - SYSTEM STATS: Time:100.0329 Method:cmd.php Processes:1 Threads:N/A Hosts:9 HostsPerProcess:9 DataSources:85 RRDsProcessed:31
01/16/2007 08:37:41 AM - SYSTEM STATS: Time:40.4232 Method:cmd.php Processes:1 Threads:N/A Hosts:9 HostsPerProcess:9 DataSources:85 RRDsProcessed:38
01/16/2007 08:37:41 AM - SYSTEM STATS: Time:159.6887 Method:cmd.php Processes:1 Threads:N/A Hosts:9 HostsPerProcess:9 DataSources:85 RRDsProcessed:48
01/16/2007 08:38:13 AM - SYSTEM STATS: Time:11.1914 Method:cmd.php Processes:1 Threads:N/A Hosts:9 HostsPerProcess:9 DataSources:85 RRDsProcessed:61
01/16/2007 08:39:12 AM - SYSTEM STATS: Time:11.1484 Method:cmd.php Processes:1 Threads:N/A Hosts:9 HostsPerProcess:9 DataSources:85 RRDsProcessed:46

The above example shows missed polls/processing at 0835h and 0836h. Could this be due to the server not having enough resources or
processing power? I also use CACTI to monitor the load average on the server itself (every 5 minutes) and there doesn't seem to be a direct correlation between high load average and the missed polls/processing.

Any advice is appreciated.

Thanks.

[TheWitness]

IF you look at your post, you will see poller "start" time's all over the place. Sometimes you have two instances of the poller starting at the same time. You obviously have something buggered up to be doing that and it is likely the cause of your problems.

TheWitness