[solved] Holes in graphs

[montecristo]

Hi,

I'm presently graphing around 20 Cisco Pix / ASA devices using Cacti 0.8.7e with NET-SNMP version 5.5

Some of the graphs have regular "holes" in them. I increased the SNMP timeout to 1500ms/2000ms for most of them but I'm still having the same issue.

I'm suspecting that it might have something to do with the double satellite hops that the SNMP requests have to go through to the reach the polled devices (round trip delay of around 1100ms).

But these double VSAT hops are applicable to all polled firewalls and I'm getting graph holes on only half of them.

Any similar cases or advice ?

Thx

K.

[BSOD2600]

1) http://forums.cacti.net/viewtopic.php?t=11747
2) consult the cacti.log with the logging level increased to get an idea what is going on.
3) you might need an even higher snmp timeout

[montecristo]

Thanks for your answer.

- I increased all timeouts to 3000ms to see if it makes a difference
- Looking at the logs, I only see two errors. They are displayed every minute (my polling cycle) and I'm not sure what it means:

07/29/2010 10:59:00 AM - POLLER: Poller[0] WARNING: Poller Output Table not Empty. Issues Found: 85, Data Sources: traffic_in(DS[54]), traffic_out(DS[54]), traffic_in(DS[55]), traffic_out(DS[55]), traffic_in(DS[56]), traffic_out(DS[56]), traffic_in(DS[57]), traffic_out(DS[57]), traffic_in(DS[63]), traffic_out(DS[63]), traffic_in(DS[64]), traffic_out(DS[64]), traffic_in(DS[65]), traffic_out(DS[65]), traffic_in(DS[66]), traffic_out(DS[66]), traffic_in(DS[67]), traffic_out(DS[67]), traffic_in(DS[68]), traffic_out(DS[68]), traffic_in(DS[69]), Additional Issues Remain. Only showing first 20
07/29/2010 10:59:00 AM - POLLER: Poller[0] WARNING: Scheduled Task is out of sync with the Poller Interval! The Poller Interval is '60' seconds, with a maximum of a '300' second Scheduled Task, but 120 seconds have passed since the last poll!

[BSOD2600]

The Poller Interval is '60' seconds, with a maximum of a '300' second Scheduled Task, but 120 seconds have passed since the last poll!

Is an issue.

You sure the scheduled task runs every 1 min? What is the cacti cron/polling intervals set to?

[montecristo]

I double-checked everything. The Cacti Cron/Poll is set to 1 min. The Widows scheduled task is set to 1 min as well.
I checked also under "Data Sources" and everything is set to 1 minute. What's the issue ? Could this be causing the problem ?

[TheWitness]

Your poller is over running and a new feature by MS is to not allow the same scheduled task to run more than 1 at a time. You have to get control of the poller timing first.

TheWitness

[montecristo]

I've been reading most of the other posts on the subject and it seems indeed that it's a poller overrunning issue.

However, I'm not seing the scheduled tasks running multiple times. It is set to run every minute just like the Cacti cron/poller timer.

So I'm not sure exactly what to do to "get control of the poller timing". What should I check ?

[BSOD2600]

Check the scheduled task log file -- validate it's only running every 1 min. how long does it normally run for?

Post a picture of a few graphs with holes.

[TheWitness]

Increasing timeouts is one way of overrunning a poller. If you simply goto the Cacti Log and filter on "SYSTEM STATS", you should get a good idea of how long it's typically running.

I don't recommend 1 minute polling due to scalability concerns. However, I know of several installations nearing or in excess of 10k hosts that run just fine with a 5 minute interval.

TheWitness

[montecristo]

Hi guys,

You were both right. From the logs, I saw that the poller would run sometimes for more than 1 minute.
As per your advice, I change the polling to 5 minutes. The graphs look perfect now.

Thanks for your help

Cheers