Cacti poller sometimes gives errors

[hid3nax]

Thanks, I'll give it a try.

[hid3nax]

Unfortunately, that did not help. It seems this made tings worse. I got much random massive errors. It ended up in gaps in almost all graphs on all hosts I have. I've set the number from 2 to 26 now.

Any help is appreciated. If needed, I can give access to my Cacti installation.

Here's the log:

Code: Select all

01/22/2012 01:12:14 AM - SYSTEM STATS: Time:12.6667 Method:cmd.php Processes:2 Threads:N/A Hosts:13 HostsPerProcess:7 DataSources:243 RRDsProcessed:198  
01/22/2012 01:07:15 AM - SYSTEM STATS: Time:13.6573 Method:cmd.php Processes:2 Threads:N/A Hosts:13 HostsPerProcess:7 DataSources:243 RRDsProcessed:198  
01/22/2012 01:02:13 AM - SYSTEM STATS: Time:11.9235 Method:cmd.php Processes:2 Threads:N/A Hosts:13 HostsPerProcess:7 DataSources:243 RRDsProcessed:198  
01/22/2012 12:57:14 AM - SYSTEM STATS: Time:12.0611 Method:cmd.php Processes:2 Threads:N/A Hosts:13 HostsPerProcess:7 DataSources:243 RRDsProcessed:198  
01/22/2012 12:52:14 AM - SYSTEM STATS: Time:12.3038 Method:cmd.php Processes:2 Threads:N/A Hosts:13 HostsPerProcess:7 DataSources:243 RRDsProcessed:198  
01/22/2012 12:47:14 AM - SYSTEM STATS: Time:12.1931 Method:cmd.php Processes:2 Threads:N/A Hosts:13 HostsPerProcess:7 DataSources:243 RRDsProcessed:198  
01/22/2012 12:47:01 AM - POLLER: Poller[0] WARNING: Poller Output Table not Empty. Issues Found: 21, Data Sources: (DS[121]), users(DS[122]), cpu_user(DS[126]), load_1min(DS[127]), load_15min(DS[128]), load_5min(DS[129]), ping(DS[131]), hdd_total(DS[132]), hdd_used(DS[132]), (DS[137]), users(DS[138]), proc(DS[139]), cpu_nice(DS[140]), cpu_system(DS[141]), cpu_user(DS[142]), load_1min(DS[143]), load_15min(DS[144]), load_5min(DS[145]), ping(DS[147]), hdd_total(DS[148]), hdd_used(DS[148]), Additional Issues Remain. Only showing first 20  
01/22/2012 12:42:53 AM - CMDPHP: Poller[0] Host[13] DS[257] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:42:53 AM - CMDPHP: Poller[0] Host[13] DS[246] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:42:53 AM - CMDPHP: Poller[0] Host[13] DS[257] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:42:53 AM - CMDPHP: Poller[0] Host[12] DS[244] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:42:53 AM - CMDPHP: Poller[0] Host[12] DS[244] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:42:53 AM - CMDPHP: Poller[0] Host[12] DS[243] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:42:53 AM - CMDPHP: Poller[0] Host[12] DS[243] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:42:53 AM - CMDPHP: Poller[0] Host[12] DS[232] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:42:52 AM - CMDPHP: Poller[0] Host[11] DS[220] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:42:52 AM - CMDPHP: Poller[0] Host[11] DS[220] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:42:52 AM - CMDPHP: Poller[0] Host[11] DS[219] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:42:52 AM - CMDPHP: Poller[0] Host[11] DS[219] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:42:52 AM - CMDPHP: Poller[0] Host[11] DS[208] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:42:52 AM - CMDPHP: Poller[0] Host[10] DS[168] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:42:52 AM - CMDPHP: Poller[0] Host[10] DS[167] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:42:52 AM - CMDPHP: Poller[0] Host[10] DS[167] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:42:52 AM - CMDPHP: Poller[0] Host[10] DS[166] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:42:52 AM - CMDPHP: Poller[0] Host[10] DS[165] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:42:52 AM - CMDPHP: Poller[0] Host[10] DS[165] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:42:52 AM - CMDPHP: Poller[0] Host[10] DS[166] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:42:52 AM - CMDPHP: Poller[0] Host[9] DS[170] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:42:52 AM - CMDPHP: Poller[0] Host[9] DS[170] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:42:52 AM - CMDPHP: Poller[0] Host[9] DS[149] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:42:52 AM - CMDPHP: Poller[0] Host[9] DS[149] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:42:52 AM - CMDPHP: Poller[0] Host[9] DS[148] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:42:52 AM - CMDPHP: Poller[0] Host[9] DS[148] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:42:52 AM - CMDPHP: Poller[0] Host[9] DS[137] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:42:52 AM - CMDPHP: Poller[0] Host[8] DS[132] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:42:52 AM - CMDPHP: Poller[0] Host[8] DS[132] WARNING: Result from SERVER not valid. Partial Result: 01/22/2012 12:42:52  
01/22/2012 12:42:52 AM - PHPSVR: Poller[0] Maximum runtime of 300 seconds exceeded for the Script Server. Exiting.  
01/22/2012 12:42:14 AM - SYSTEM STATS: Time:12.8580 Method:cmd.php Processes:2 Threads:N/A Hosts:13 HostsPerProcess:7 DataSources:243 RRDsProcessed:198  
01/22/2012 12:42:02 AM - POLLER: Poller[0] WARNING: Poller Output Table not Empty. Issues Found: 21, Data Sources: (DS[66]), users(DS[67]), proc(DS[68]), cpu_nice(DS[69]), cpu_system(DS[70]), cpu_user(DS[71]), load_1min(DS[72]), load_15min(DS[73]), load_5min(DS[74]), ping(DS[76]), (DS[77]), (DS[78]), hdd_total(DS[79]), hdd_used(DS[79]), (DS[83]), users(DS[84]), proc(DS[85]), cpu_nice(DS[86]), cpu_system(DS[87]), cpu_user(DS[88]), load_1min(DS[89]), Additional Issues Remain. Only showing first 20  
01/22/2012 12:37:59 AM - CMDPHP: Poller[0] Host[7] DS[104] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:59 AM - CMDPHP: Poller[0] Host[6] DS[102] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:59 AM - CMDPHP: Poller[0] Host[6] DS[102] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:59 AM - CMDPHP: Poller[0] Host[6] DS[101] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:59 AM - CMDPHP: Poller[0] Host[6] DS[101] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:59 AM - CMDPHP: Poller[0] Host[6] DS[99] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:59 AM - CMDPHP: Poller[0] Host[6] DS[99] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:59 AM - CMDPHP: Poller[0] Host[6] DS[98] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:59 AM - CMDPHP: Poller[0] Host[6] DS[98] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:59 AM - CMDPHP: Poller[0] Host[6] DS[96] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:59 AM - CMDPHP: Poller[0] Host[6] DS[96] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:58 AM - CMDPHP: Poller[0] Host[6] DS[83] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:57 AM - CMDPHP: Poller[0] Host[5] DS[79] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:57 AM - CMDPHP: Poller[0] Host[5] DS[79] WARNING: Result from SERVER not valid. Partial Result: 01/22/2012 12:37:55  
01/22/2012 12:37:55 AM - PHPSVR: Poller[0] Maximum runtime of 300 seconds exceeded for the Script Server. Exiting.  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[13] DS[257] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[13] DS[246] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[13] DS[257] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[12] DS[244] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[12] DS[244] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[12] DS[243] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[12] DS[243] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[12] DS[232] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[11] DS[220] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[11] DS[220] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[11] DS[219] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[11] DS[219] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[11] DS[208] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[10] DS[168] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[10] DS[167] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[10] DS[167] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[10] DS[166] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[10] DS[165] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[10] DS[165] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[10] DS[166] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[9] DS[170] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[9] DS[170] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[9] DS[149] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[9] DS[149] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[9] DS[148] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[9] DS[148] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[9] DS[137] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[8] DS[132] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:53 AM - CMDPHP: Poller[0] Host[8] DS[132] WARNING: Result from SERVER not valid. Partial Result: 01/22/2012 12:37:53  
01/22/2012 12:37:53 AM - PHPSVR: Poller[0] Maximum runtime of 300 seconds exceeded for the Script Server. Exiting.  
01/22/2012 12:37:18 AM - SYSTEM STATS: Time:16.8220 Method:cmd.php Processes:2 Threads:N/A Hosts:13 HostsPerProcess:7 DataSources:243 RRDsProcessed:276  
01/22/2012 12:37:11 AM - CMDPHP: Poller[0] Host[7] DS[104] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:11 AM - CMDPHP: Poller[0] Host[6] DS[102] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:11 AM - CMDPHP: Poller[0] Host[6] DS[102] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:11 AM - CMDPHP: Poller[0] Host[6] DS[101] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:11 AM - CMDPHP: Poller[0] Host[6] DS[101] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:11 AM - CMDPHP: Poller[0] Host[6] DS[99] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:11 AM - CMDPHP: Poller[0] Host[6] DS[99] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:11 AM - CMDPHP: Poller[0] Host[6] DS[98] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:11 AM - CMDPHP: Poller[0] Host[6] DS[98] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:11 AM - CMDPHP: Poller[0] Host[6] DS[96] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:11 AM - CMDPHP: Poller[0] Host[6] DS[96] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:10 AM - CMDPHP: Poller[0] Host[6] DS[83] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:07 AM - CMDPHP: Poller[0] Host[5] DS[79] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:07 AM - CMDPHP: Poller[0] Host[5] DS[79] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:05 AM - CMDPHP: Poller[0] Host[5] DS[66] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:05 AM - CMDPHP: Poller[0] Host[4] DS[60] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:37:05 AM - CMDPHP: Poller[0] Host[4] DS[60] WARNING: Result from SERVER not valid. Partial Result: 01/22/2012 12:37:04  
01/22/2012 12:37:04 AM - PHPSVR: Poller[0] Maximum runtime of 300 seconds exceeded for the Script Server. Exiting.  
01/22/2012 12:37:01 AM - POLLER: Poller[0] WARNING: Poller Output Table not Empty. Issues Found: 2, Data Sources: hdd_total(DS[219]), hdd_used(DS[219])  
01/22/2012 12:37:01 AM - POLLER: Poller[0] WARNING: There are '1' detected as overrunning a polling process, please investigate  
01/22/2012 12:36:59 AM - SYSTEM STATS: Time:297.5389 Method:cmd.php Processes:2 Threads:N/A Hosts:13 HostsPerProcess:7 DataSources:243 RRDsProcessed:161  
01/22/2012 12:36:59 AM - POLLER: Poller[0] Maximum runtime of 297 seconds exceeded. Exiting.  
01/22/2012 12:32:54 AM - CMDPHP: Poller[0] Host[13] DS[257] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:32:54 AM - CMDPHP: Poller[0] Host[13] DS[246] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:32:54 AM - CMDPHP: Poller[0] Host[13] DS[257] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:32:53 AM - CMDPHP: Poller[0] Host[12] DS[244] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:32:53 AM - CMDPHP: Poller[0] Host[12] DS[244] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:32:53 AM - CMDPHP: Poller[0] Host[12] DS[243] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:32:53 AM - CMDPHP: Poller[0] Host[12] DS[243] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:32:53 AM - CMDPHP: Poller[0] Host[12] DS[232] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:32:53 AM - CMDPHP: Poller[0] Host[11] DS[220] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:32:53 AM - CMDPHP: Poller[0] Host[11] DS[220] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:32:53 AM - CMDPHP: Poller[0] Host[11] DS[219] WARNING: Result from SERVER not valid. Partial Result: U  
01/22/2012 12:32:53 AM - CMDPHP: Poller[0] Host[11] DS[219] WARNING: Result from SERVER not valid. Partial Result: 01/22/2012 12:32:53  
01/22/2012 12:32:53 AM - PHPSVR: Poller[0] Maximum runtime of 300 seconds exceeded for the Script Server. Exiting.  
01/22/2012 12:32:01 AM - POLLER: Poller[0] WARNING: There are '2' detected as overrunning a polling process, please investigate  
01/22/2012 12:31:59 AM - SYSTEM STATS: Time:297.5366 Method:cmd.php Processes:2 Threads:N/A Hosts:13 HostsPerProcess:7 DataSources:243 RRDsProcessed:52  
01/22/2012 12:31:59 AM - POLLER: Poller[0] Maximum runtime of 297 seconds exceeded. Exiting.  
01/22/2012 12:27:01 AM - POLLER: Poller[0] WARNING: There are '2' detected as overrunning a polling process, please investigate  
01/22/2012 12:26:59 AM - SYSTEM STATS: Time:297.2807 Method:cmd.php Processes:2 Threads:N/A Hosts:13 HostsPerProcess:7 DataSources:243 RRDsProcessed:74  
01/22/2012 12:26:59 AM - POLLER: Poller[0] Maximum runtime of 297 seconds exceeded. Exiting.  
01/22/2012 12:17:16 AM - SYSTEM STATS: Time:14.7111 Method:cmd.php Processes:2 Threads:N/A Hosts:13 HostsPerProcess:7 DataSources:243 RRDsProcessed:198  
01/22/2012 12:12:14 AM - SYSTEM STATS: Time:12.3764 Method:cmd.php Processes:2 Threads:N/A Hosts:13 HostsPerProcess:7 DataSources:243 RRDsProcessed:198  
01/22/2012 12:07:14 AM - SYSTEM STATS: Time:12.0094 Method:cmd.php Processes:2 Threads:N/A Hosts:13 HostsPerProcess:7 DataSources:243 RRDsProcessed:198  
01/22/2012 12:02:14 AM - SYSTEM STATS: Time:12.2925 Method:cmd.php Processes:2 Threads:N/A Hosts:13 HostsPerProcess:7 DataSources:243 RRDsProcessed:198  
01/21/2012 11:57:14 PM - SYSTEM STATS: Time:12.3790 Method:cmd.php Processes:2 Threads:N/A Hosts:13 HostsPerProcess:7 DataSources:243 RRDsProcessed:198  
01/21/2012 11:52:14 PM - SYSTEM STATS: Time:12.4554 Method:cmd.php Processes:2 Threads:N/A Hosts:13 HostsPerProcess:7 DataSources:243 RRDsProcessed:198  
01/21/2012 11:47:14 PM - SYSTEM STATS: Time:12.1404 Method:cmd.php Processes:2 Threads:N/A Hosts:13 HostsPerProcess:7 DataSources:243 RRDsProcessed:198  
01/21/2012 11:47:02 PM - POLLER: Poller[0] WARNING: Poller Output Table not Empty. Issues Found: 21, Data Sources: (DS[29]), proc(DS[31]), load_1min(DS[35]), load_15min(DS[36]), load_5min(DS[37]), ping(DS[39]), (DS[40]), (DS[41]), hdd_total(DS[42]), hdd_used(DS[42]), cpu_nice(DS[44]), cpu_system(DS[45]), cpu_user(DS[46]), (DS[47]), users(DS[48]), proc(DS[49]), load_1min(DS[53]), load_15min(DS[54]), load_5min(DS[55]), ping(DS[57]), (DS[58]), Additional Issues Remain. Only showing first 20  
01/21/2012 11:42:57 PM - CMDPHP: Poller[0] Host[7] DS[104] WARNING: Result from SERVER not valid. Partial Result: U  
01/21/2012 11:42:57 PM - CMDPHP: Poller[0] Host[6] DS[102] WARNING: Result from SERVER not valid. Partial Result: U  
01/21/2012 11:42:57 PM - CMDPHP: Poller[0] Host[6] DS[102] WARNING: Result from SERVER not valid. Partial Result: U  
01/21/2012 11:42:57 PM - CMDPHP: Poller[0] Host[6] DS[101] WARNING: Result from SERVER not valid. Partial Result: U  
01/21/2012 11:42:57 PM - CMDPHP: Poller[0] Host[6] DS[101] WARNING: Result from SERVER not valid. Partial Result: U  
01/21/2012 11:42:57 PM - CMDPHP: Poller[0] Host[6] DS[99] WARNING: Result from SERVER not valid. Partial Result: U  
01/21/2012 11:42:57 PM - CMDPHP: Poller[0] Host[6] DS[99] WARNING: Result from SERVER not valid. Partial Result: U  
01/21/2012 11:42:57 PM - CMDPHP: Poller[0] Host[6] DS[98] WARNING: Result from SERVER not valid. Partial Result: U  
01/21/2012 11:42:57 PM - CMDPHP: Poller[0] Host[6] DS[98] WARNING: Result from SERVER not valid. Partial Result: U  
01/21/2012 11:42:57 PM - CMDPHP: Poller[0] Host[6] DS[96] WARNING: Result from SERVER not valid. Partial Result: U  
01/21/2012 11:42:57 PM - CMDPHP: Poller[0] Host[6] DS[96] WARNING: Result from SERVER not valid. Partial Result: U  
01/21/2012 11:42:57 PM - CMDPHP: Poller[0] Host[6] DS[83] WARNING: Result from SERVER not valid. Partial Result: U  
01/21/2012 11:42:55 PM - CMDPHP: Poller[0] Host[5] DS[79] WARNING: Result from SERVER not valid. Partial Result: U  
01/21/2012 11:42:55 PM - CMDPHP: Poller[0] Host[5] DS[79] WARNING: Result from SERVER not valid. Partial Result: U  
01/21/2012 11:42:54 PM - CMDPHP: Poller[0] Host[5] DS[66] WARNING: Result from SERVER not valid. Partial Result: U  
01/21/2012 11:42:54 PM - CMDPHP: Poller[0] Host[4] DS[60] WARNING: Result from SERVER not valid. Partial Result: U  
01/21/2012 11:42:54 PM - CMDPHP: Poller[0] Host[4] DS[60] WARNING: Result from SERVER not valid. Partial Result: U  
01/21/2012 11:42:53 PM - CMDPHP: Poller[0] Host[4] DS[47] WARNING: Result from SERVER not valid. Partial Result: U  
01/21/2012 11:42:53 PM - CMDPHP: Poller[0] Host[3] DS[42] WARNING: Result from SERVER not valid. Partial Result: U  
01/21/2012 11:42:53 PM - CMDPHP: Poller[0] Host[3] DS[42] WARNING: Result from SERVER not valid. Partial Result: 01/21/2012 11:42:53  
01/21/2012 11:42:53 PM - PHPSVR: Poller[0] Maximum runtime of 300 seconds exceeded for the Script Server. Exiting.  
01/21/2012 11:42:18 PM - AUTH LOGIN: User 'huglester' Authenticated  
01/21/2012 11:42:14 PM - SYSTEM STATS: Time:12.8117 Method:cmd.php Processes:2 Threads:N/A Hosts:13 HostsPerProcess:7 DataSources:243 RRDsProcessed:198  
01/21/2012 11:37:13 PM - SYSTEM STATS: Time:12.3740 Method:cmd.php Processes:2 Threads:N/A Hosts:13 HostsPerProcess:7 DataSources:243 RRDsProcessed:198  
01/21/2012 11:37:01 PM - POLLER: Poller[0] WARNING: There are '1' detected as overrunning a polling process, please investigate  
01/21/2012 11:36:59 PM - SYSTEM STATS: Time:297.4708 Method:cmd.php Processes:2 Threads:N/A Hosts:13 HostsPerProcess:7 DataSources:243 RRDsProcessed:99  
01/21/2012 11:36:59 PM - POLLER: Poller[0] Maximum runtime of 297 seconds exceeded. Exiting.  
01/21/2012 11:27:14 PM - SYSTEM STATS: Time:12.4308 Method:cmd.php Processes:2 Threads:N/A Hosts:13 HostsPerProcess:7 DataSources:243 RRDsProcessed:198  
01/21/2012 11:22:14 PM - SYSTEM STATS: Time:12.8926 Method:cmd.php Processes:2 Threads:N/A Hosts:13 HostsPerProcess:7 DataSources:243 RRDsProcessed:198  
01/21/2012 11:17:14 PM - SYSTEM STATS: Time:12.5835 Method:cmd.php Processes:2 Threads:N/A Hosts:13 HostsPerProcess:7 DataSources:243 RRDsProcessed:198  
01/21/2012 11:12:14 PM - SYSTEM STATS: Time:12.7435 Method:cmd.php Processes:2 Threads:N/A Hosts:13 HostsPerProcess:7 DataSources:243 RRDsProcessed:198  
01/21/2012 11:07:14 PM - SYSTEM STATS: Time:12.5088 Method:cmd.php Processes:2 Threads:N/A Hosts:13 HostsPerProcess:7 DataSources:243 RRDsProcessed:198  
01/21/2012 11:02:15 PM - SYSTEM STATS: Time:13.1406 Method:cmd.php Processes:2 Threads:N/A Hosts:13 HostsPerProcess:7 DataSources:243 RRDsProcessed:198  

[gandalf]

Those weird stuff seems to stem from disk usage Script Server Data Query.
Please use http://docs.cacti.net/usertemplate:data ... disk_usage as a faster, SNMP only replacement.
It is recommended to use "Index Count Changed" as the default reindex option for this (indeed, I've chosen that as the default reindex option for all data queries).
Easiest option is to stop all existing Disk Usage data queries and setup the new ones instead. You may want to use AUTOM8 in order to reduce trouble. And you may want to change the Host Templates accordingly.

It is not 100% clear to me, why those strange results are retrieved when this query fails. So my fast analysis may go wrong ...
R.

[hid3nax]

gandalf,
Thank you for the advice. Would it be a good practice just to delete all 'old' disk space usage graps+data sources and re-create them with the new template? I don't care much about the history.

[gandalf]

gandalf,
Thank you for the advice. Would it be a good practice just to delete all 'old' disk space usage graps+data sources and re-create them with the new template? I don't care much about the history.

That would be the easiest move
R.

[hid3nax]

Great. I re-created all graphs with the new template. I'll give some time and see if this has cured the problem. Thanks!

[hid3nax]

Unfortunately, the problem is still there. It seems that it was not a problem with the disk space monitoring script, but with the 'script server' machanism itself. Now it reports errors for other graphs using script server scripts.

Now this is a bad luck. I guess I'll just live with it. Thanks for help anyway!

[hid3nax]

I just wanted to bump this thread and give some update.

In short, I have upgraded to 0.8.8 and the annoying issue is still there.

In logs, I see messages like these:

Code: Select all

04/06/2012 05:50:19 PM - CMDPHP: Poller[0] Host[13] DS[256] WARNING: Result from CMD not valid. Partial Result: U
04/06/2012 05:50:02 PM - POLLER: Poller[0] WARNING: Poller Output Table not Empty. Issues Found: 16, Data Sources: (DS[232]), users(DS[233]), proc(DS[234]), load_5min(DS[240]), uptime(DS[241]), ping(DS[242]), traffic_in(DS[245]), traffic_out(DS[245]), ssCpuRawIdle(DS[339]), ssCpuRawInterrupt(DS[339]), ssCpuRawKernel(DS[339]), ssCpuRawNice(DS[339]), ssCpuRawSoftIRQ(DS[339]), ssCpuRawSystem(DS[339]), ssCpuRawUser(DS[339]), ssCpuRawWait(DS[339])

Code: Select all

04/06/2012 02:50:15 PM - SYSTEM STATS: Time:13.5224 Method:cmd.php Processes:18 Threads:N/A Hosts:18 HostsPerProcess:1 DataSources:396 RRDsProcessed:240
04/06/2012 02:50:01 PM - POLLER: Poller[0] WARNING: Poller Output Table not Empty. Issues Found: 16, Data Sources: (DS[232]), users(DS[233]), proc(DS[234]), load_5min(DS[240]), uptime(DS[241]), ping(DS[242]), traffic_in(DS[245]), traffic_out(DS[245]), ssCpuRawIdle(DS[339]), ssCpuRawInterrupt(DS[339]), ssCpuRawKernel(DS[339]), ssCpuRawNice(DS[339]), ssCpuRawSoftIRQ(DS[339]), ssCpuRawSystem(DS[339]), ssCpuRawUser(DS[339]), ssCpuRawWait(DS[339])
04/06/2012 02:47:40 PM - PHPSVR: Poller[0] Maximum runtime of 300 seconds exceeded for the Script Server. Exiting.

Code: Select all

04/06/2012 12:50:14 PM - SYSTEM STATS: Time:12.6318 Method:cmd.php Processes:17 Threads:N/A Hosts:17 HostsPerProcess:1 DataSources:372 RRDsProcessed:226
04/06/2012 12:50:02 PM - POLLER: Poller[0] WARNING: Poller Output Table not Empty. Issues Found: 25, Data Sources: (DS[208]), users(DS[209]), cpu_nice(DS[211]), cpu_system(DS[212]), load_1min(DS[214]), load_15min(DS[215]), load_5min(DS[216]), ping(DS[218]), traffic_in(DS[221]), traffic_out(DS[221]), traffic_in(DS[222]), traffic_out(DS[222]), uptime(DS[223]), (DS[246]), traffic_in(DS[258]), traffic_out(DS[258]), traffic_in(DS[259]), traffic_out(DS[259]), ssCpuRawIdle(DS[379]), ssCpuRawInterrupt(DS[379]), Additional Issues Remain. Only showing first 20
04/06/2012 12:46:49 PM - PHPSVR: Poller[0] Maximum runtime of 300 seconds exceeded for the Script Server. Exiting.
04/06/2012 12:45:59 PM - PHPSVR: Poller[0] Maximum runtime of 300 seconds exceeded for the Script Server. Exiting.
04/06/2012 12:45:14 PM - SYSTEM STATS: Time:12.5328 Method:cmd.php Processes:17 Threads:N/A Hosts:17 HostsPerProcess:1 DataSources:372 RRDsProcessed:226
04/06/2012 12:40:14 PM - SYSTEM STATS: Time:12.6157 Method:cmd.php Processes:17 Threads:N/A Hosts:17 HostsPerProcess:1 DataSources:372 RRDsProcessed:226
04/06/2012 12:40:01 PM - POLLER: Poller[0] WARNING: Poller Output Table not Empty. Issues Found: 1, Data Sources: ssCpuRawSystem(DS[379])
04/06/2012 12:40:01 PM - POLLER: Poller[0] WARNING: There are '2' detected as overrunning a polling process, please investigate
04/06/2012 12:40:01 PM - SYSTEM STATS: Time:298.8768 Method:cmd.php Processes:17 Threads:N/A Hosts:17 HostsPerProcess:1 DataSources:372 RRDsProcessed:211
04/06/2012 12:40:01 PM - POLLER: Poller[0] Maximum runtime of 298 seconds exceeded. Exiting.
04/06/2012 12:30:14 PM - SYSTEM STATS: Time:12.5098 Method:cmd.php Processes:17 Threads:N/A Hosts:17 HostsPerProcess:1 DataSources:372 RRDsProcessed:226

Code: Select all

04/06/2012 06:50:14 AM - SYSTEM STATS: Time:12.7088 Method:cmd.php Processes:17 Threads:N/A Hosts:17 HostsPerProcess:1 DataSources:372 RRDsProcessed:226
04/06/2012 06:50:01 AM - POLLER: Poller[0] WARNING: Poller Output Table not Empty. Issues Found: 29, Data Sources: (DS[208]), users(DS[209]), cpu_nice(DS[211]), cpu_system(DS[212]), load_1min(DS[214]), load_15min(DS[215]), load_5min(DS[216]), ping(DS[218]), traffic_in(DS[221]), traffic_out(DS[221]), traffic_in(DS[222]), traffic_out(DS[222]), uptime(DS[223]), (DS[232]), users(DS[233]), proc(DS[234]), load_5min(DS[240]), uptime(DS[241]), ping(DS[242]), traffic_in(DS[245]), Additional Issues Remain. Only showing first 20
04/06/2012 06:47:39 AM - PHPSVR: Poller[0] Maximum runtime of 300 seconds exceeded for the Script Server. Exiting.
04/06/2012 06:45:59 AM - PHPSVR: Poller[0] Maximum runtime of 300 seconds exceeded for the Script Server. Exiting.
04/06/2012 06:45:14 AM - SYSTEM STATS: Time:12.7518 Method:cmd.php Processes:17 Threads:N/A Hosts:17 HostsPerProcess:1 DataSources:372 RRDsProcessed:226
04/06/2012 06:40:14 AM - SYSTEM STATS: Time:12.8949 Method:cmd.php Processes:17 Threads:N/A Hosts:17 HostsPerProcess:1 DataSources:372 RRDsProcessed:226

Any clues? Any comments, suggestions..?

[stefanbrudny]

Any clues? Any comments, suggestions..?

I'm suffering the same issue since moving from Gentoo notebook to CentOS VM. No clue with this unfortunately, as possibly many things changed and are untraceable. Just for reference:
* CentOS 6.2,
* custom rrdtool as stock one segfaults (compiled manually also segfaults, but less often), anyway those and graph breaks doesn't overlap, so I'm assuming they're not connected,
* Virtual Machine with e1000 (paravirtualized not tested so far, but as a last resort its possible),
* machine is somewhat loaded, approx. 2-3 lavg/1min.
* I'm using snmp query only for gathering data,
* php limits are already set too high.

None more interesting things, just plain boring VM with graphs.

[gandalf]

From the log, I assume that two pollers are running in parallel on your system. See 2nd link of my sig to find out where to search and how to cure
R.

[stefanbrudny]

Must... have...been... blind!

Two admins, one set poller in /etc/cron.d/ , second in /etc/crontab.

Thanks!