Cacti Master Poller process terminated by user

[elbuit]

Hi.
I've found an issue when cacti poller runs during more than 60 seconds.
I'm using debian 6.0 and cacti 8.8a.

fgrep -B 1 terminated /var/log/cacti/poller-error.log

Usually, the "poller is terminated by user" when more than 60 seconds elapsed.

Code: Select all

OK u:0.08 s:0.20 r:59.85
01/31/2013 10:01:04 PM - CMDPHP: Poller[0] WARNING: Cacti Master Poller process terminated by user
--
[...]
--
01/31/2013 10:36:02 PM - SYSTEM STATS: Time:60.2986 Method:spine Processes:8 Threads:20 Hosts:118 HostsPerProcess:15 DataSources:18110 RRDsProcessed:4443
01/31/2013 10:36:03 PM - CMDPHP: Poller[0] WARNING: Cacti Master Poller process terminated by user
--
OK u:0.04 s:0.03 r:56.10
01/31/2013 10:41:05 PM - CMDPHP: Poller[0] WARNING: Cacti Master Poller process terminated by user
--
OK u:0.12 s:0.23 r:61.03
01/31/2013 10:46:03 PM - CMDPHP: Poller[0] WARNING: Cacti Master Poller process terminated by user
--
OK u:0.09 s:0.19 r:60.25
01/31/2013 10:51:02 PM - CMDPHP: Poller[0] WARNING: Cacti Master Poller process terminated by user
--
OK u:0.18 s:0.33 r:59.04
01/31/2013 10:56:02 PM - CMDPHP: Poller[0] WARNING: Cacti Master Poller process terminated by user
--
[...]
--
OK u:0.10 s:0.78 r:60.26
02/01/2013 01:31:03 AM - CMDPHP: Poller[0] WARNING: Cacti Master Poller process terminated by user
--

It happens when I do some maintenance deletes on mysql. This causes a slowdown in cacti poller.php operations.
When there are no manteinance deletes, I can reproduce the same problem by inserting a sleep(56) on poller.php at line 400 before rrd_close($rrdtool_pipe);
if I add the same sleep at spine.c, spine and poller.php finish OK, but not when sleep is added to poller.php


Any suggestions??

Thanks.

[paulgevers]

I believe you are running Cacti with 1 minute interval. I believe the poller is killed if it is still running when the next poll starts. I think this is designed behaviour to prevent lock up of the machine for no good reason at all. What you want to do is make sure that the poller finishes in time. Please explain in more detail what problem do you exactly want to solve?

[elbuit]

I believe you are running Cacti with 1 minute interval. I believe the poller is killed if it is still running when the next poll starts. I think this is designed behaviour to prevent lock up of the machine for no good reason at all. What you want to do is make sure that the poller finishes in time. Please explain in more detail what problem do you exactly want to solve?

Hi Paul.
The poller runs on 5 min interval.
This is my cron.d entry:

Code: Select all

#MAILTO=root
*/5 * * * * www-data php /var/www/cacti/poller.php 2>&1 >> /var/log/cacti/poller-error.log ;cp -u -v /var/www/cacti/plugins/weathermap/output/*.png /var/www/cacti/maps/

And a poller settings screenshot :



I noticed that if I put a 55 secons sleep at begining of poller.php I don't have a timeout "cacti terminated by user".

Or if I force spine to run more than one minute adding a usleep(), the poller.php finishes ok.
But if I put the same sleep before rrd_close($rrdtool_pipe); things changes.

I suppose that there will be a timeout in rrdtool pipe.

thanks in advance

[elbuit]

Hi again.

I'm using network-weathermap, and I found that if I disable this plugin I can't reproduce "poller terminatef by user".

I'll investigate more about this.