Poller Timed Out

doublek · Post by **doublek** » Tue Oct 17, 2006 12:35 pm

Were currently pulling ~ 31000 differnt requests from 598 different hosts and were starting to see the poller timeout. I beleive our current settings are optimal but if someone could verify it would be appreciated.

10/17/2006 10:39:54 AM - POLLER: Poller[0] Maximum runtime of 292 seconds exceeded. Exiting.
10/17/2006 10:39:54 AM - SYSTEM STATS: Time:292.8391 Method:cactid Processes:4 Threads:20 Hosts:598 HostsPerProcess:150 DataSources:75563 RRDsProcessed:0

10/17/2006 10:40:00 AM - CACTID: Poller[0] ERROR: Cactid Timed Out While Processing Hosts Internal

Post by **TheWitness** » Tue Oct 17, 2006 12:47 pm

What version of Cactid? It looks as though it crashed, or you have run out of PHP memory. What is your PHP memory set to in php.ini? Set it much higher if it's below 32mbytes.

I would recommend Cactid 0.8.6i ASAP.

TheWitness

doublek · Post by **doublek** » Thu Oct 19, 2006 1:50 pm

We are currently running with Cactid 0.8.6g and in the process of upgrading to Cactid 0.8.6i. Yet we had a hardware failuare Saturday on one of our Cacti servers, so the last few days have been troubleshooting the poller on our Apache Server until we can rebuild the Poller server.

Setup = 3 servers
1 - Poller
1 - Apache Web Server
1 - MySQL db - turning into a cluster

When we started the poller up on our Apache Web Server we have run into nothting but troubles. For some reason our polls have become way incosistant and now one poller seems to be timing out. We also have XML templates that exist / verified permissions as well, yet Cacti won't recognize them. Were working on tracking down the erros and I've attached what we've found so far.

Also our PHP memory is set to 1024M

doublek · Post by **doublek** » Thu Oct 19, 2006 1:52 pm

XML Errors

doublek · Post by **doublek** » Thu Oct 19, 2006 1:57 pm

Sorry about the images. I forgot the alt key on my print screen. If it needs removed I can repost the image.

Post by **TheWitness** » Thu Oct 19, 2006 9:10 pm

Whatever your system writer is, it's hosing your system up. Also, it looks like you did not bring your resources directory over from the other server.

It also looks like you have some hokey scripts or somehing causing one of your cactid processes to languish.

TheWitness

Post by **TheWitness** » Thu Oct 19, 2006 9:15 pm

Also, the ping timeout values are in Milliseconds. 3 is rather low.

TheWitness

doublek · Post by **doublek** » Thu Oct 19, 2006 10:57 pm

I've adjusted the ping timout and installed CACTID 0.8.6i. Yet were still seeing the same issues. For some reason one poller always seems to exceed the maximum timeout.

See Attached Logs

Is their a table inside the DB that dictactes which poller to use for which device / host?

I've noticed the errors with some of the scripts aswell and were working on correcting them.

Side note:
To install cactid on 6.1-RELEASE-p6 FreeBSD 6.1-RELEASE-p6 we had to chage the following line in the configure file as well as the cactid.conf.dist instances

Orig:
CFLAGS="-I$SNMP_INCDIR -I$SNMP_INCDIR $CFLAGS"

Changed:
CFLAGS="-I$SNMP_INCDIR -I$SNMP_INCDIR/.. $CFLAGS"

doublek · Post by **doublek** » Thu Oct 19, 2006 11:05 pm

We've verified all the resource files and have even tried 777 access on the whole dir.

Post by **TheWitness** » Fri Oct 20, 2006 4:27 pm

Also, verify that your RRDtool path is accurate. None of the rrdfiles are updating for some reason.

TheWitness

doublek · Post by **doublek** » Fri Oct 20, 2006 5:48 pm

Will verify the path tonight and also the version.

One other item we thought about was some of our current cacti directories have simlinks so were going to verify the permissions on them as well.

Cacti

Poller Timed Out

Poller Timed Out

Cactid Upgrades

Could not Loacte XML file.

Sorry about the Image

Ping Timout

Resources Files

RRDtool Path

Who is online