poller TIMEOUT with crontab - not if run by hand

[Peach]

Hi all,
after reading an interesting post I decided to install cacti on my production server.

everything works fine and I have added this line to /etc/crontab

Code: Select all

*/5 * * * * apache /usr/lib64/php5/bin/php /var/www/localhost/htdocs/cacti/poller.php > /dev/null 2>&1

after that I started seeing this in the logs:

Code: Select all

Sep 24 15:55:01 tenten suhosin[7976]: ALERT - script tried to increase memory_limit to 268435456 bytes which is above the allowed value (attacker 'REMOTE_ADDR not set', file '/var/www/localhost/htdocs/cacti/poller.php', line 150)

so I raised the memory_limit to 256M for php5-cli
after this I don't get any more errors in the logs, but:

Code: Select all

# tail -n  2 /var/www/localhost/htdocs/cacti/log/cacti.log  
09/24/2008 04:10:01 PM - POLLER: Poller[0] Maximum runtime of 298 seconds exceeded. Exiting.
09/24/2008 04:10:01 PM - SYSTEM STATS: Time:299.1241 Method:cmd.php Processes:1 Threads:N/A Hosts:2 HostsPerProcess:2 DataSources:5 RRDsProcessed:0

at this point I'm clueless... what's the problem?

any help is greatly appreciated.

thanks

[edit] I forgot: I'm under gentoo and cacti version is 0.8.7b
and max_execution_time = 30 (from php.ini)

[gandalf]

Please see 2nd link of my sig for more debugging hints
Reinhard

[Peach]

Please see 2nd link of my sig for more debugging hints
Reinhard

thanks a lot gandalf, your post was very useful. I've fount that the two scripts I've uploaded for UPS monitoring had a header copied from somewhere, that it should have been deleted before uploading.

Now solved this I'm struggling to make cacti work correctly:
I'm getting a timeout of cmd.php whenever the poller tries to run it:

Code: Select all

11/15/2008 05:00:00 PM - POLLER: Poller[0] Maximum runtime of 298 seconds exceeded. Exiting.
11/15/2008 05:00:00 PM - SYSTEM STATS: Time:298.8942 Method:cmd.php Processes:1 Threads:N/A Hosts:2 HostsPerProcess:2 DataSources:7 RRDsProcessed:0

instead if I do a:

Code: Select all

/usr/lib64/php5/bin/php -q cacti/cmd.php 1 1

there's no problem (apart from the graphs with no data... that's another problem maybe)
and in cacti.log I get

Code: Select all

11/15/2008 05:00:41 PM - SYSTEM STATS: Time:39.5524 Method:cmd.php Processes:1 Threads:N/A Hosts:2 HostsPerProcess:2 DataSources:7 RRDsProcessed:4

so what's the point?
I've got one host (localhost) with default diskspace graphs and UPS monitoring scripts. that's all.

I forgot to mention:
crontab contains this:

Code: Select all

*/5 * * * * apache /usr/lib64/php5/bin/php /var/www/localhost/htdocs/cacti/poller.php > /dev/null 2>&1

all files installed are owned by apache:

Code: Select all

# ls -l cacti/
totale 872
-rw-r--r-- 1 apache apache   6387 22 set 12:38 about.php
-rw-r--r-- 1 apache apache   4612 22 set 12:38 auth_changepassword.php
-rw-r--r-- 1 apache apache  11498 22 set 12:38 auth_login.php
-rw-r--r-- 1 apache apache 124294 22 set 12:38 cacti.sql
-rw-r--r-- 1 apache apache  12960 22 set 12:38 cdef.php
drwxr-xr-x 2 apache apache   4096 22 set 12:38 cli
-rw-r--r-- 1 apache apache  18369 22 set 12:38 cmd.php
-rw-r--r-- 1 apache apache   5765 22 set 12:38 color.php
-rw-r--r-- 1 apache apache  23279 22 set 12:38 data_input.php
-rw-r--r-- 1 apache apache  33923 22 set 12:38 data_queries.php
-rw-r--r-- 1 apache apache  49265 22 set 12:38 data_sources.php
-rw-r--r-- 1 apache apache  31304 22 set 12:38 data_templates.php
-rw-r--r-- 1 apache apache   5958 22 set 12:38 gprint_presets.php
-rw-r--r-- 1 apache apache   3404 22 set 12:38 graph_image.php
-rw-r--r-- 1 apache apache  11550 22 set 12:38 graph.php
-rw-r--r-- 1 apache apache   7716 22 set 12:38 graph_settings.php
-rw-r--r-- 1 apache apache  13828 22 set 12:38 graphs_items.php
-rw-r--r-- 1 apache apache  37125 22 set 12:38 graphs_new.php
-rw-r--r-- 1 apache apache  50544 22 set 12:38 graphs.php
-rw-r--r-- 1 apache apache   9945 22 set 12:38 graph_templates_inputs.php
-rw-r--r-- 1 apache apache  18482 22 set 12:38 graph_templates_items.php
-rw-r--r-- 1 apache apache  25042 22 set 12:38 graph_templates.php
-rw-r--r-- 1 apache apache  29052 22 set 12:38 graph_view.php
-rw-r--r-- 1 apache apache   5758 22 set 12:38 graph_xport.php
-rw-r--r-- 1 apache apache  50228 22 set 12:38 host.php
-rw-r--r-- 1 apache apache  18045 22 set 12:38 host_templates.php
drwxr-xr-x 2 apache apache   4096 22 set 12:38 images
drwxr-xr-x 5 apache apache   4096 24 set 23:16 include
-rw-r--r-- 1 apache apache   2242 22 set 12:38 index.php
drwxr-xr-x 2 apache apache   4096 22 set 12:38 install
drwxr-xr-x 2 apache apache   4096 22 set 12:38 lib
drwxr-xr-x 2 apache apache     38 22 set 12:38 log
-rw-r--r-- 1 apache apache   2451 22 set 12:38 logout.php
-rw-r--r-- 1 apache apache   4306 22 set 12:38 poller_commands.php
-rw-r--r-- 1 apache apache   2243 22 set 12:38 poller_export.php
-rw-r--r-- 1 apache apache  14960 22 set 12:38 poller.php
drwxr-xr-x 5 apache apache     66 22 set 12:38 resource
drwxr-xr-x 2 apache apache   4096 15 nov 16:50 rra
-rw-r--r-- 1 apache apache   7084 22 set 12:38 rra.php
drwxr-xr-x 2 apache apache   4096 15 nov 16:42 scripts
-rw-r--r-- 1 apache apache   7304 22 set 12:38 script_server.php
-rw-r--r-- 1 apache apache    353 22 set 12:38 script_server.pl
-rw-r--r-- 1 apache apache   4996 22 set 12:38 settings.php
-rw-r--r-- 1 apache apache   6667 22 set 12:38 templates_export.php
-rw-r--r-- 1 apache apache   6524 22 set 12:38 templates_import.php
-rw-r--r-- 1 apache apache  17835 22 set 12:38 tree.php
-rw-r--r-- 1 apache apache  41776 22 set 12:38 user_admin.php
-rw-r--r-- 1 apache apache  48852 22 set 12:38 utilities.php

crontab is running correctly

rras seems to be ok:

Code: Select all

# ls -l cacti/rra
totale 668
-rw-r--r-- 1 apache apache  94816 15 nov 17:00 tenten_hdd_free_10.rrd
-rw-r--r-- 1 apache apache  94816 15 nov 17:00 tenten_hdd_free_11.rrd
-rw-r--r-- 1 apache apache  94816 15 nov 17:00 tenten_hdd_free_12.rrd
-rw-r--r-- 1 apache apache 386312 15 nov 17:00 tenten_line_freq_9.rrd

now I'm enabling some DEBUG modes to see if something new will popup

EDIT nothing interesting in the logs with DEBUG statement enabled. still I don't get why by hand works and via crontab not... what's wrong???

[gandalf]

Please change

Code: Select all

*/5 * * * * apache /usr/lib64/php5/bin/php /var/www/localhost/htdocs/cacti/poller.php > /dev/null 2>&1

to

Code: Select all

*/5 * * * * apache /usr/lib64/php5/bin/php /var/www/localhost/htdocs/cacti/poller.php > /tmp/poller.log 2>&1

and rerun. See /tmp/poller.log, then
Reinhard

[Peach]

Please change

Code: Select all

*/5 * * * * apache /usr/lib64/php5/bin/php /var/www/localhost/htdocs/cacti/poller.php > /dev/null 2>&1

to

Code: Select all

*/5 * * * * apache /usr/lib64/php5/bin/php /var/www/localhost/htdocs/cacti/poller.php > /tmp/poller.log 2>&1

and rerun. See /tmp/poller.log, then
Reinhard

here it is:

Code: Select all

# cat /tmp/poller.log 
11/16/2008 08:40:01 PM - POLLER: Poller[0] NOTE: Poller Int: '300', Cron Int: '300', Time Since Last: '300', Max Runtime '298', Poller Runs: '1'
11/16/2008 08:40:01 PM - POLLER: Poller[0] DEBUG: About to Spawn a Remote Process [CMD: /usr/bin/php, ARGS: -q /var/www/localhost/htdocs/cacti/cmd.php 0 1]
Waiting on 1/1 pollers.
Waiting on 1/1 pollers
....
Waiting on 1/1 pollers.
Waiting on 1/1 pollers.

[gandalf]

If this lasts forever, it is very likely a script waiting for eternity
Reinhard

[Peach]

If this lasts forever, it is very likely a script waiting for eternity
Reinhard

it is not.
there are about 254 lines of

Code: Select all

Waiting on 1/1 pollers.

secondly running it by hand works, I can see rddtool gaining the results in cacti.log but actually through crontab it goes anywhere.