Not all rrd files updating

[cooldude919]

Heres some info about the server.

Basically its bsod's windows install kit with cactid updated to the "k" version running on windows 2003 server. If you need any other info let me know.

Basically i saw server emails from yesturday saying hosts where down, but never got ones saying they came back up. It seems since yesturday around 5pm the majorty of my hosts have not been getting updated. Heres some error message from around that time.

ERROR: Cactid Timed Out While Processing Hosts Internal

SS[5] An invalid file descriptor was given in one of the sets.
ERROR: Input Expected, Script Server Terminating
PHP Script Server communications lost.
01/19/2008 05:19:57 PM - CACTID: Poller[0] ERROR: Cactid Timed Out While Processing Hosts Internal
01/19/2008 05:19:57 PM - CACTID: Poller[0] ERROR: Cactid Timed Out While Processing Hosts Internal
01/19/2008 05:19:57 PM - CACTID: Poller[0] ERROR: Cactid Timed Out While Processing Hosts Internal
01/19/2008 05:19:57 PM - CACTID: Poller[0] ERROR: Cactid Timed Out While Processing Hosts Internal
01/19/2008 05:20:00 PM - CACTID: Poller[0] ERROR: Cactid Timed Out While Processing Hosts Internal
01/19/2008 05:20:00 PM - CACTID: Poller[0] ERROR: Cactid Timed Out While Processing Hosts Internal
01/19/2008 05:20:00 PM - CACTID: Poller[0] ERROR: Cactid Timed Out While Processing Hosts Internal
01/19/2008 05:20:00 PM - CACTID: Poller[0] ERROR: Cactid Timed Out While Processing Hosts Internal
01/19/2008 05:20:01 PM - CACTID: Poller[0] ERROR: SS[0] An invalid file descriptor was given in one of the sets.
01/19/2008 05:20:01 PM - CACTID: Poller[0] ERROR: SS[1] An invalid file descriptor was given in one of the sets.

Looking in the log, i havent got a "Green" poller completed message since yesturday, just warnings that i sometimes get about ss scripts that seems to be due to network congestion(im polling over WAN links)

Now i put it in debug mode, and it seems the poller is still running fine from this,
CACTID: Poller[0] Time: 98.6870 s, Threads: 15, Hosts: 357

There are serveral of those that add up to my total number of hosts, so that seems fine.

Looking in the rrd folder, there are some with the last polling time as being updated, but the majortiy havent been updated since yesturday at 5pm.

Ive tried restarting the server, rebuilding poller cache, replacing cactid exe files and rrdtool exe files. No errors in mysqladmin, so either its something funky in a table, or a problem with rrdtool?

Whats throwing me off is that i dont see anything in the log abouty rrdtool. Is it just taking too long to update?

please help!, thanks

*edit* sorry, guess i should mention this is an older build of the installer with cacti 8.6j.

[cooldude919]

Well i did some more reading, and changed php memory from 128mb to 256mb and it seems to be able to complete a polling session now. I will keep an eye on it and see what happens.

[cooldude919]

well at least some good came out of this, my polling times are now the best they have ever been. Before they where anywhere from 120-250sec.

Now i am seeing this.
01/20/2008 01:56:38 PM - MANAGE: Poller[0] Checking Manage
01/20/2008 01:56:24 PM - SYSTEM STATS: Time:84.1952 Method:cactid Processes:5 Threads:14 Hosts:2146 HostsPerProcess:430 DataSources:6605 RRDsProcessed:4375
01/20/2008 01:51:33 PM - MANAGE: Poller[0] Checking Manage
01/20/2008 01:51:19 PM - SYSTEM STATS: Time:78.7428 Method:cactid Processes:5 Threads:14 Hosts:2146 HostsPerProcess:430 DataSources:6605 RRDsProcessed:4375
01/20/2008 01:46:28 PM - MANAGE: Poller[0] Checking Manage
01/20/2008 01:46:24 PM - SYSTEM STATS: Time:84.2480 Method:cactid Processes:5 Threads:14 Hosts:2146 HostsPerProcess:430 DataSources:6605 RRDsProcessed:4375
01/20/2008 01:41:32 PM - MANAGE: Poller[0] Checking Manage
01/20/2008 01:41:13 PM - SYSTEM STATS: Time:72.6209 Method:cactid Processes:5 Threads:14 Hosts:2146 HostsPerProcess:430 DataSources:6605 RRDsProcessed:4375
01/20/2008 01:36:30 PM - MANAGE: Poller[0] Checking Manage
01/20/2008 01:36:24 PM - SYSTEM STATS: Time:84.0168 Method:cactid Processes:5 Threads:14 Hosts:2146 HostsPerProcess:430 DataSources:6605 RRDsProcessed:4375


and this is just on a p4 3.2ghz with HT 1.5gb ram and 80gb sata hd.