[SOLVED] Weather map causes poller warning and missing data

[lafiro]

I noticed that with even 1 map enabled, I see missing information on my graphs, not all, but usually the same graphs.
This is a piece of my logs just as I enabled a map:

Code: Select all

05/15/2014 10:29:03 AM - SYSTEM STATS: Time:1.6434 Method:spine Processes:1 Threads:20 Hosts:5 HostsPerProcess:5 DataSources:396 RRDsProcessed:298  
05/15/2014 10:29:01 AM - SYSTEM STATS: Time:0.1323 Method:spine Processes:1 Threads:20 Hosts:5 HostsPerProcess:5 DataSources:396 RRDsProcessed:0  
05/15/2014 10:29:01 AM - POLLER: Poller[0] Maximum runtime of 58 seconds exceeded. Exiting.  
05/15/2014 10:28:43 AM - SYSTEM STATS: Time:1.1750 Method:spine Processes:1 Threads:20 Hosts:5 HostsPerProcess:5 DataSources:396 RRDsProcessed:198  
05/15/2014 10:28:25 AM - SYSTEM STATS: Time:3.8068 Method:spine Processes:1 Threads:20 Hosts:5 HostsPerProcess:5 DataSources:396 RRDsProcessed:198  
05/15/2014 10:28:22 AM - POLLER: Poller[0] WARNING: Poller Output Table not Empty. Issues Found: 24, Data Sources: traffic_in(DS[115]), traffic_out(DS[115]), traffic_in(DS[116]), traffic_out(DS[116]), traffic_in(DS[117]), traffic_out(DS[117]), traffic_in(DS[118]), traffic_out(DS[118]), traffic_in(DS[119]), traffic_out(DS[119]), traffic_in(DS[120]), traffic_out(DS[120]), traffic_in(DS[121]), traffic_out(DS[121]), traffic_in(DS[122]), traffic_out(DS[122]), traffic_in(DS[123]), traffic_out(DS[123]), traffic_in(DS[124]), traffic_out(DS[124]), Additional Issues Remain. Only showing first 20  
05/15/2014 10:28:02 AM - SYSTEM STATS: Time:0.6790 Method:spine Processes:1 Threads:20 Hosts:5 HostsPerProcess:5 DataSources:396 RRDsProcessed:384  
05/15/2014 10:28:01 AM - SYSTEM STATS: Time:0.1332 Method:spine Processes:1 Threads:20 Hosts:5 HostsPerProcess:5 DataSources:396 RRDsProcessed:0  
05/15/2014 10:28:01 AM - POLLER: Poller[0] Maximum runtime of 58 seconds exceeded. Exiting.  
05/15/2014 10:27:43 AM - SYSTEM STATS: Time:2.4343 Method:spine Processes:1 Threads:20 Hosts:5 HostsPerProcess:5 DataSources:396 RRDsProcessed:198  
05/15/2014 10:27:23 AM - SYSTEM STATS: Time:1.5137 Method:spine Processes:1 Threads:20 Hosts:5 HostsPerProcess:5 DataSources:396 RRDsProcessed:198  
05/15/2014 10:27:05 AM - SYSTEM STATS: Time:3.5919 Method:spine Processes:1 Threads:20 Hosts:5 HostsPerProcess:5 DataSources:396 RRDsProcessed:248  
05/15/2014 10:27:01 AM - SYSTEM STATS: Time:0.1351 Method:spine Processes:1 Threads:20 Hosts:5 HostsPerProcess:5 DataSources:396 RRDsProcessed:0  
05/15/2014 10:27:01 AM - POLLER: Poller[0] Maximum runtime of 58 seconds exceeded. Exiting.  
05/15/2014 10:26:42 AM - SYSTEM STATS: Time:0.8548 Method:spine Processes:1 Threads:20 Hosts:5 HostsPerProcess:5 DataSources:396 RRDsProcessed:198  
05/15/2014 10:26:25 AM - SYSTEM STATS: Time:3.1500 Method:spine Processes:1 Threads:20 Hosts:5 HostsPerProcess:5 DataSources:396 RRDsProcessed:198  
05/15/2014 10:26:22 AM - POLLER: Poller[0] WARNING: Poller Output Table not Empty. Issues Found: 1, Data Sources: traffic_out(DS[162])  
05/15/2014 10:26:02 AM - SYSTEM STATS: Time:0.9306 Method:spine Processes:1 Threads:20 Hosts:5 HostsPerProcess:5 DataSources:396 RRDsProcessed:309  
05/15/2014 10:26:01 AM - SYSTEM STATS: Time:0.1600 Method:spine Processes:1 Threads:20 Hosts:5 HostsPerProcess:5 DataSources:396 RRDsProcessed:0  
05/15/2014 10:26:01 AM - POLLER: Poller[0] Maximum runtime of 58 seconds exceeded. Exiting.  
05/15/2014 10:25:43 AM - SYSTEM STATS: Time:1.8881 Method:spine Processes:1 Threads:20 Hosts:5 HostsPerProcess:5 DataSources:396 RRDsProcessed:198  
05/15/2014 10:25:23 AM - SYSTEM STATS: Time:1.3422 Method:spine Processes:1 Threads:20 Hosts:5 HostsPerProcess:5 DataSources:396 RRDsProcessed:198  
05/15/2014 10:25:02 AM - SYSTEM STATS: Time:1.4092 Method:spine Processes:1 Threads:20 Hosts:5 HostsPerProcess:5 DataSources:396 RRDsProcessed:297  
05/15/2014 10:25:01 AM - SYSTEM STATS: Time:0.1439 Method:spine Processes:1 Threads:20 Hosts:5 HostsPerProcess:5 DataSources:396 RRDsProcessed:0  
05/15/2014 10:25:01 AM - POLLER: Poller[0] Maximum runtime of 58 seconds exceeded. Exiting.  
05/15/2014 10:24:42 AM - SYSTEM STATS: Time:0.9237 Method:spine Processes:1 Threads:20 Hosts:5 HostsPerProcess:5 DataSources:396 RRDsProcessed:198  
05/15/2014 10:24:22 AM - SYSTEM STATS: Time:0.7650 Method:spine Processes:1 Threads:20 Hosts:5 HostsPerProcess:5 DataSources:396 RRDsProcessed:198  
05/15/2014 10:24:02 AM - SYSTEM STATS: Time:1.3419 Method:spine Processes:1 Threads:20 Hosts:5 HostsPerProcess:5 DataSources:396 RRDsProcessed:347  
05/15/2014 10:24:01 AM - POLLER: Poller[0] WARNING: There are '1' detected as overrunning a polling process, please investigate  
05/15/2014 10:24:00 AM - SYSTEM STATS: Time:0.1366 Method:spine Processes:1 Threads:20 Hosts:5 HostsPerProcess:5 DataSources:396 RRDsProcessed:0  
05/15/2014 10:24:00 AM - POLLER: Poller[0] Maximum runtime of 58 seconds exceeded. Exiting.  
05/15/2014 10:23:42 AM - SYSTEM STATS: Time:1.6134 Method:spine Processes:1 Threads:20 Hosts:5 HostsPerProcess:5 DataSources:396 RRDsProcessed:198  
05/15/2014 10:23:22 AM - SYSTEM STATS: Time:1.1639 Method:spine Processes:1 Threads:20 Hosts:5 HostsPerProcess:5 DataSources:396 RRDsProcessed:198  
05/15/2014 10:23:02 AM - SYSTEM STATS: Time:0.5812 Method:spine Processes:1 Threads:20 Hosts:5 HostsPerProcess:5 DataSources:396 RRDsProcessed:198  
05/15/2014 10:22:48 AM - SYSTEM STATS: Time:1.6232 Method:spine Processes:1 Threads:20 Hosts:5 HostsPerProcess:5 DataSources:396 RRDsProcessed:198  

As soon as the map is disabled, the logs go back to normal and there is no missing information in any of my graphs(tested this a few times for hours and not a single piece of missed information when weathermap is not running)

Anything I can change to fix this? [i5 laptop with 4GB memory dedicated just to Cacti on Windows 7 Ultimate 64bit]

[Howie]

Well... the next release is significantly faster, but one map shouldn't take almost a minute to run unless it's very complex, very large (libpng takes a while to compress a PNG file), or doing something odd.

You have made life a little harder for yourself by choosing 1 minute polling, so the available time to generate the map is less.

Could you post some of the map config?

My first suggestion would be to try switching to poller_output for data collection. Usually that's quicker than running rrdtool for every link. (add "SET rrd_use_poller_output 1" in the top section of the map config file, and give it a few polls to start picking up data).

[lafiro]

Edit, see new post below.

[Howie]

Strange...

05/15/2014 10:24:00 AM - POLLER: Poller[0] Maximum runtime of 58 seconds exceeded. Exiting.

This suggests that you are running 1 minute poll intervals.

[lafiro]

Edit:

I added that one line to the top of the config's and that did the trick!

Thank you so much!!!!!

Code: Select all

05/15/2014 04:43:02 PM - SYSTEM STATS: Time:0.8314 Method:spine Processes:1 Threads:20 Hosts:5 HostsPerProcess:5 DataSources:396 RRDsProcessed:198  
05/15/2014 04:42:57 PM - WEATHERMAP: Poller[0] STATS: Weathermap 0.97c run complete - Thu, 15 May 14 16:42:57 -0400: 2 maps were run in 3 seconds with 0 warnings.  
05/15/2014 04:42:57 PM - WEATHERMAP: Poller[0] [Map 5] 23.conf: Wrote map to C:\inetpub\wwwroot\cacti\plugins\weathermap\output\a79c5e67dc6db35be53c.png and C:\inetpub\wwwroot\cacti\plugins\weathermap\output\a79c5e67dc6db35be53c.thumb.png  
05/15/2014 04:42:56 PM - WEATHERMAP: Poller[0] [Map 5] 23.conf: About to write image file. If this is the last message in your log, increase memory_limit in php.ini [WMPOLL01]  
05/15/2014 04:42:56 PM - WEATHERMAP: Poller[0] [Map 5] 23.conf: Map: C:\inetpub\wwwroot\cacti\plugins\weathermap\configs\Grassi23.conf -> C:\inetpub\wwwroot\cacti\plugins\weathermap\output\a79c5e67dc6db35be53c.html & C:\inetpub\wwwroot\cacti\plugins\weathermap\output\a79c5e67dc6db35be53c.png  
05/15/2014 04:42:56 PM - WEATHERMAP: Poller[0] [Map 3] 21.conf: Wrote map to C:\inetpub\wwwroot\cacti\plugins\weathermap\output\c3980ee153ecb902718e.png and C:\inetpub\wwwroot\cacti\plugins\weathermap\output\c3980ee153ecb902718e.thumb.png  
05/15/2014 04:42:55 PM - WEATHERMAP: Poller[0] [Map 3] 21.conf: About to write image file. If this is the last message in your log, increase memory_limit in php.ini [WMPOLL01]  
05/15/2014 04:42:54 PM - WEATHERMAP: Poller[0] [Map 3] 21.conf: Map: C:\inetpub\wwwroot\cacti\plugins\weathermap\configs\Grassi21.conf -> C:\inetpub\wwwroot\cacti\plugins\weathermap\output\c3980ee153ecb902718e.html & C:\inetpub\wwwroot\cacti\plugins\weathermap\output\c3980ee153ecb902718e.png  
 
05/15/2014 04:42:54 PM - WEATHERMAP: Poller[0] Weathermap 0.97c starting - Normal logging mode. Turn on DEBUG in Cacti for more information  
05/15/2014 04:42:54 PM - SYSTEM STATS: Time:2.2085 Method:spine Processes:1 Threads:20 Hosts:5 HostsPerProcess:5 DataSources:396 RRDsProcessed:198  
05/15/2014 04:42:38 PM - WEATHERMAP: Poller[0] STATS: Weathermap 0.97c run complete - Thu, 15 May 14 16:42:38 -0400: 2 maps were run in 2 seconds with 0 warnings.