poller not working under Solaris 10

[cactard]

I've got cacti installed, and it's accessible through apache, and appears to work ok (I can log into it and browse around things). The poller process run through cron however is timing out and giving the following error:

cacti@newhost$ /usr/local/php/bin/php /usr/local/apache2/htdocs/cacti/poller.php
11/15/2006 11:46:54 AM - POLLER: Poller[0] Maximum runtime of 292 seconds exceeded. Exiting.
11/15/2006 11:46:54 AM - SYSTEM STATS: Time:292.8986 Method:cmd.php Processes:1 Threads:N/A Hosts:4 HostsPerProcess:4 DataSources:28 RRDsProcessed:0
PHP Warning: pclose(): 47 is not a valid stream resource in /usr/local/apache2/htdocs/cacti/lib/rrd.php on line 47

I can run an snmpget manually and get a valid response:

newhost 12:37:06 (/)$ /usr/local/net-snmp/bin/snmpget -v 1 -c collamer localhost sysUpTime.0
DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (26713562) 3 days, 2:12:15.62


Here's a rundown of the setup that I have.

OS is Solaris 10 x86 release 6/06
Perl from Sun - SUNWperl584core 5.8.4

The following software from Sunfreeware pkgs:
apache 2.0.59
php 5.1.6
mysql 5.0.24
rrdtool 1.0.49
bison 1.875d
flex 2.5.33
libiconv 1.9.2
libtool 1.5
make 3.80

Anyone have an idea of what I can do to resolve the error? I'm not sure if this is a cacti problem, or a php problem.

[cactard]

Forgot to mention that the version of cacti is 0.8.6i

[cactard]

I just did a quick install of 0.8.6h with a new database, and tried running the poller again, same problem occurs:

cacti@newhost$ php ./poller.php
sh: -: not found
11/15/2006 03:03:39 PM - POLLER: Poller[0] Maximum runtime of 292 seconds exceeded. Exiting.
11/15/2006 03:03:39 PM - SYSTEM STATS: Time:292.9102 Method:cmd.php Processes:1 Threads:N/A Hosts:2 HostsPerProcess:2 DataSources:0 RRDsProcessed:0
PHP Warning: pclose(): 43 is not a valid stream resource in /usr/local/apache2/htdocs/cacti-0.8.6h/lib/rrd.php on line 47

[2stupid4this]

Hi, sry. i can't help you but i've the same problem with the CACTID Poller ...

Perhaps someone finds a solution ...

Here's the thread > http://forums.cacti.net/viewtopic.php?t=17680

[cactard]

When I put the graph into debug mode, I see the following:

RRDTool Says:
ERROR: opening '/usr/local/apache2/htdocs/cacti/rra/3660_main_router_5min_cpu_31.rrd': No such file or directory

Now when I go into that directory, there are no files - indeed that file does not exist. I setup a linux box using vmware on my laptop, and had cacti running in no time. When I look into the cacti/rra directory on that box, there are several .rrd files for each graph. Looks like RRDtool is not making graphs where it is supposed to.

[cactard]

When I do a verbose query on the Associated Data Queries, this is what comes out:

+ Running data query [1].
+ Found type = '3' [snmp query].
+ Found data query XML file at '/usr/local/apache2/htdocs/cacti/resource/snmp_queries/interface.xml'
+ XML file parsed ok.
+ Executing SNMP walk for list of indexes @ '.1.3.6.1.2.1.2.2.1.1'
+ No SNMP data returned
+ Found data query XML file at '/usr/local/apache2/htdocs/cacti/resource/snmp_queries/interface.xml'
+ Found data query XML file at '/usr/local/apache2/htdocs/cacti/resource/snmp_queries/interface.xml'
+ Found data query XML file at '/usr/local/apache2/htdocs/cacti/resource/snmp_queries/interface.xml'

Does this mean that it cannot find snmpwalk? If I run the snmpwalk manually, I get back valid information.

Also, when I look under the edit item for the Associated Graph Templates, I see this when in debug mode:

RRDTool Command:

/usr/local/rrdtool-1.0.49/bin/rrdtool graph - \
--imgformat=PNG \
--start=-86400 \
--end=-300 \
--title="router - CPU Usage" \
--rigid \
--base=1000 \
--height=120 \
--width=500 \
--alt-autoscale-max \
--lower-limit=0 \
--vertical-label="percent" \
DEF:a="/usr/local/apache2/htdocs/cacti/rra/router_5min_cpu_32.rrd":5min_cpu:AVERAGE \
AREA:a#FF0000:"CPU Usage" \
GPRINTLAST:"Current\:%8.0lf" \
GPRINTAVERAGE:"Average\:%8.0lf" \
GPRINTMAX:"Maximum\:%8.0lf"

RRDTool Says:

ERROR: opening '/usr/local/apache2/htdocs/cacti/rra/router_5min_cpu_32.rrd': No such file or directory


I see 80 people have read this thread, and not a single reply yet. I can't be the only person having trouble with this...

[TheWitness]

Check your paths and make sure they are pointing to valid net-snmp binaries.

TheWitness

[cactard]

I did re-check the paths, and things seem to be ok in that regard. I've got the binaries in /usr/local/net-snmp/bin, and that is in the path:

cacti@newbox$ which snmpwalk
/usr/local/net-snmp/bin/snmpwalk

I just removed the php 5.x install and installed 4.4.4 in it's place. Now things look a little better, at least the call to rrdtool from within the cacti web interface looks like it succeeds. Here is the output from the graph in debug mode:


RRDTool Command:

/usr/local/rrdtool-1.0.49/bin/rrdtool graph - \
--imgformat=PNG \
--start=-86400 \
--end=-300 \
--title="router - CPU Usage" \
--rigid \
--base=1000 \
--height=120 \
--width=500 \
--alt-autoscale-max \
--lower-limit=0 \
--vertical-label="percent" \
DEF:a="/usr/local/apache2/htdocs/cacti/rra/router_5min_cpu_32.rrd":5min_cpu:AVERAGE \
AREA:a#FF0000:"CPU Usage" \
GPRINTLAST:"Current\:%8.0lf" \
GPRINTAVERAGE:"Average\:%8.0lf" \
GPRINTMAX:"Maximum\:%8.0lf"

RRDTool Says:

OK

[cactard]

Graphs are now being generated, although they are empty. The poller problem still persists:

11/21/2006 09:05:37 AM - POLLER: Poller[0] Maximum runtime of 292 seconds exceeded. Exiting.
11/21/2006 09:05:37 AM - SYSTEM STATS: Time:293.9237 Method:cmd.php Processes:1 Threads:N/A Hosts:5 HostsPerProcess:5 DataSources:30 RRDsProcessed:0
PHP Warning: pclose(): 47 is not a valid stream resource in /usr/local/apache2/htdocs/cacti/lib/rrd.php on line 48

[cactard]

If I look at the data query debug information, it looks like it's not getting back any snmp data:

Data Query Debug Information
+ Running data query [1].
+ Found type = '3' [snmp query].
+ Found data query XML file at '/usr/local/apache2/htdocs/cacti/resource/snmp_queries/interface.xml'
+ XML file parsed ok.
+ Executing SNMP walk for list of indexes @ '.1.3.6.1.2.1.2.2.1.1'
+ No SNMP data returned
+ Found data query XML file at '/usr/local/apache2/htdocs/cacti/resource/snmp_queries/interface.xml'
+ Found data query XML file at '/usr/local/apache2/htdocs/cacti/resource/snmp_queries/interface.xml'
+ Found data query XML file at '/usr/local/apache2/htdocs/cacti/resource/snmp_queries/interface.xml'


I can query the router manually using snmpget, but not using the oid shown above.

cacti@newbox$ snmpget -v 2c -c 'string' router .1.3.6.1.2.1.2.2.1
IF-MIB::ifEntry = No Such Object available on this agent at this OID

cacti@newbox$ snmpget -v 2c -c 'string' router .1.3.6.1.2.1.1.1.0 SNMPv2-MIB::sysDescr.0 = STRING: Cisco Internetwork Operating System Software
IOS (tm) 3600 Software (C3660-I-M), Version 12.2(11)YT2, EARLY DEPLOYMENT RELEASE SOFTWARE (fc1)
TAC Support: http://www.cisco.com/tac
Copyright (c) 1986-2003 by cisco Systems, Inc.
Compiled Thu 27-Feb-03

[cactard]

How can I get more info on the actual data query that cacti is doing?

[TheWitness]

I'm going to be a bit scarce for a while. Gandolf will be back in a day or so.

TheWitness

[gandalf]

Back again. Please have a look at my "NaN Debugging" HowTo. Please report your findings, step by step.
Reinhard

[CaliMonk]

I'm having a very similar problem.

Information:

- CentOS 4.4
- Apache 2.0
- PHP 4.3.9
- RRDTool 1.2.15
- net-snmp 5.1.2

As with several other reports I've seen on the forums, my poller isn't working:

11/24/2006 11:40:27 AM - POLLER: Poller[0] Maximum runtime of 292 seconds exceeded. Exiting.
11/24/2006 11:40:27 AM - SYSTEM STATS: Time:292.8139 Method:cmd.php Processes:1 Threads:N/A Hosts:2 HostsPerProcess:2 DataSources:2 RRDsProcessed:0

I've debugged this, the cmd checks the hosts but keeps getting 'false' as a reply back from $ping->ping().

Because it doesn't get a ping reply, it keeps looping waiting for data, which is logically never gets. Due to the ping reply being FALSE the cmd loop doesn't break with a "HOST DOWN" error.

The bit I'm referring too:

Code: Select all

if ($ping->ping($ping_availability, read_config_option("ping_method"), read_config_option("ping_timeout"), read_config_option("ping_retries"))) {
           $host_down = false;
           update_host_status(HOST_UP, $host_id, $hosts, $ping, $ping_availability, $print_data_to_stdout);
}else{
         $host_down = true;
          update_host_status(HOST_DOWN, $host_id, $hosts, $ping, $ping_availability, $print_data_to_stdout);
}

There should be another check to see if the reply is false (otherwise you shouldn't reply false), or make sure it never returns false

Another odd bit:

Code: Select all

function ping($avail_method = AVAIL_SNMP_AND_PING, $ping_type = ICMP_PING, $timeout=500, $retries=3)

The default is $ping_type = ICMP_PING, even though the code checks for "PING_ICMP". This should be backwards.

Tracking the issue even further, i notice that cacti_snmp_get is the one not returning any data.

Current code:

Code: Select all

if ($version == "1") {
   $snmp_value = @snmpget($hostname:$port, $community, $oid, ($timeout * 1000), $retries);
}

Working code:

Code: Select all

if ($version == "1") {
  $snmp_value = @snmpget("$hostname:$port", $community, "$oid", ($timeout * 1000), $retries);
}

As you can see, $oid (the object ID) and $hostname:$port need to be placed within 2 ""'s.

This is what works for me, reference: http://uk2.php.net/manual/en/function.snmpget.php

I thought this was the complete fix, unfortunately this only gives me a reply from SNMP. My Poller still seems to 'hang', running cmd manually gives me data. Invoking it from poller.php doesn't.

So, more debugging...

In lib/poller.php there is a check for /usr/bin/php. This returns false for me (See http://uk2.php.net/file_exists), i believe this is another reason for a few of the errors people get.

The check:

Code: Select all

//      if (file_exists($filename)) {
                if ($config["cacti_server_os"] == "win32") {
                        pclose(popen("start \"Cactiplus\" /I \"" . $filename . "\" " . $args, "r"));
                }else{
                        exec($filename . " " . $args . " > /dev/null &");
                }
//      }

Commenting those lines like i did, makes the poller get the data. I no longer have to call cmd.php by hand.

Now everything is working rather nicely

- Ivo

[gandalf]

Wow, this would have taken a few minutes to find out, I suppose. But I'd like you to report this (use a simple screen copy) to http://bugs.cacti.net. All bugs are tracked there. Thank you for your debugging
Reinhard