Graphing suddenly stopped dead. :o

[chris-tv2]

We monitor about 130 servers with cacti and it has run smoothly for about 5 months. Then suddenly at 3 in the morning every graph went blank.

I have read everything I could get my hands on trying to debug the problem, but have so far been unsuccessfull. I have made at debug log dump, and the problem so far as I can see, is the total lack of rrdtool update statements in the log. There are no Error: tags in the logfile at all, the only thing is a few servers which do not respond to snmp.

The snmp data is collected properly from the servers according to the log. The cron script is running every 5 minutes and seems to be in order.

To begin with i started by moving the rrd files to force cacti to recreate them. That didn't work. They weren't recreated. So i tried removing the cacti installation and dropping in a fresh copy from the cacti website. Didn't work either.

I have tried both cmd.php and cactid with equal lack of success. I tried cleaning out the data portion of the database, only keeping the settings but that didn't work either. I suppose it could be corrupt data in the database, but everything that i do in cacti seems to work with no errors in the log.
I tried clearing the poller cache more than once, also didn't make any difference.

I must admit that I am at my wits end, and looking at a complete reconfiguration which would take days. Anybody have anything similiar happen to them and fixed it ?

[gandalf]

Please check the size of the table poller_output. If it is huge, truncate it (=clear). You will want to consider php.ini's memory settings. Recommend settings is 64 MB instead of 8MB default.
Reinhard

[chris-tv2]

Thanx a lot !! that fixed it

Wonder why I didn't think of that