rrd file shows NaN for data values, but snmpget gathers data

networkfupa · Post by **networkfupa** » Tue Oct 21, 2008 6:18 pm

I have successfully created some graphs for a custom application and I
am polling without any problem.
Except for a few hosts.

Let's take a single graph for example, it is free memory and total
memory on two independant machines, machine1 and machine2.

Machine1 only shows data for free memory, and machine2 only shows data
for total memory.
I did an snmpwalk/snmpget on the OID's present in my data templates,
and it works fine!:

[admin1@machine1 ~]$ snmpget -v1 -c public machine1
1.3.6.1.4.1.2008.2800.100.1.1.101.1
RFC1155-SMI::enterprises.2008.2800.100.1.1.101.1 = STRING: "1016"

So it wasn't snmp that was a problem.
Then I checked the rrd file which shows a bunch of NaN's. I can't
figure out why it is returning as NaN on one graph but works on the
other.
Can you please offer some insights into this?

Thank you,
Matt

[admin1@machine1 ~]$ rrdtool dump
/usr/local/cacti/cacti_current/rra/memmax7001_22912.rrd

Code: Select all

<!-- Round Robin Database Dump --><rrd> <version> 0003 </version>
       <step> 300 </step> <!-- Seconds -->
       <lastupdate> 1224630312 </lastupdate> <!-- 2008-10-21 16:05:12 PDT -->

       <ds>
               <name> ChMemMax7001 </name>
               <type> GAUGE </type>
               <minimal_heartbeat> 5000 </minimal_heartbeat>
               <min> 0.0000000000e+00 </min>
               <max> NaN </max>

               <!-- PDP Status -->
               <last_ds> UNKN </last_ds>
               <value> 0.0000000000e+00 </value>
               <unknown_sec> 12 </unknown_sec>
       </ds>

<!-- Round Robin Archives -->   <rra>
               <cf> AVERAGE </cf>
               <pdp_per_row> 1 </pdp_per_row> <!-- 300 seconds -->

               <params>
               <xff> 5.0000000000e-01 </xff>
               </params>
               <cdp_prep>
                       <ds>
                       <primary_value> NaN </primary_value>
                       <secondary_value> 0.0000000000e+00 </secondary_value>
                       <value> NaN </value>
                       <unknown_datapoints> 0 </unknown_datapoints>
                       </ds>
               </cdp_prep>
               <database>
                       <!-- 2008-10-19 14:10:00 PDT / 1224450600 -->
<row><v> NaN </v></row>
                       <!-- 2008-10-19 14:15:00 PDT / 1224450900 -->
<row><v> NaN </v></row>
                       <!-- 2008-10-19 14:20:00 PDT / 1224451200 -->
<row><v> NaN </v></row>
                       <!-- 2008-10-19 14:25:00 PDT / 1224451500 -->
<row><v> NaN </v></row>
                       <!-- 2008-10-19 14:30:00 PDT / 1224451800 -->
<row><v> NaN </v></row>
                       <!-- 2008-10-19 14:35:00 PDT / 1224452100 -->
<row><v> NaN </v></row>
                       <!-- 2008-10-19 14:40:00 PDT / 1224452400 -->
<row><v> NaN </v></row>
                       <!-- 2008-10-19 14:45:00 PDT / 1224452700 -->
<row><v> NaN </v></row>
                       <!-- 2008-10-19 14:50:00 PDT / 1224453000 -->
<row><v> NaN </v></row>
                       <!-- 2008-10-19 14:55:00 PDT / 1224453300 -->
<row><v> NaN </v></row>
                       <!-- 2008-10-19 15:00:00 PDT / 1224453600 -->
<row><v> NaN </v></row>
                       <!-- 2008-10-19 15:05:00 PDT / 1224453900 -->
<row><v> NaN </v></row>
                       <!-- 2008-10-19 15:10:00 PDT / 1224454200 -->
<row><v> NaN </v></row>
                       <!-- 2008-10-19 15:15:00 PDT / 1224454500 -->
<row><v> NaN </v></row>
                       <!-- 2008-10-19 15:20:00 PDT / 1224454800 -->
<row><v> NaN </v></row>
                       <!-- 2008-10-19 15:25:00 PDT / 1224455100 -->
<row><v> NaN </v></row>
                       <!-- 2008-10-19 15:30:00 PDT / 1224455400 -->
<row><v> NaN </v></row>
                       <!-- 2008-10-19 15:35:00 PDT / 1224455700 -->
<row><v> NaN </v></row>
                       <!-- 2008-10-19 15:40:00 PDT / 1224456000 -->
<row><v> NaN </v></row>
                       <!-- 2008-10-19 15:45:00 PDT / 1224456300 -->
<row><v> NaN </v></row>
                       <!-- 2008-10-19 15:50:00 PDT / 1224456600 -->
<row><v> NaN </v></row>
                       <!-- 2008-10-19 15:55:00 PDT / 1224456900 -->
<row><v> NaN </v></row>
                       <!-- 2008-10-19 16:00:00 PDT / 1224457200 -->
<row><v> NaN </v></row>
                       <!-- 2008-10-19 16:05:00 PDT / 1224457500 -->
<row><v> NaN </v></row>
                       <!-- 2008-10-19 16:10:00 PDT / 1224457800 -->
<row><v> NaN </v></row>
                       <!-- 2008-10-19 16:15:00 PDT / 1224458100 -->
<row><v> NaN </v></row>
                       <!-- 2008-10-19 16:20:00 PDT / 1224458400 -->
<row><v> NaN </v></row>
                       <!-- 2008-10-19 16:25:00 PDT / 1224458700 -->
<row><v> NaN </v></row>
                       <!-- 2008-10-19 16:30:00 PDT / 1224459000 -->
<row><v> NaN </v></row>

Any suggestions are appreciated.

Thanks,

Post by **TheWitness** » Tue Oct 21, 2008 8:01 pm

You need to provide a little more info. Like what version of Cacti, what poller? Also, show what is appearing in your Cacti Log.

TheWitness

networkfupa · Post by **networkfupa** » Tue Oct 21, 2008 8:12 pm

I will provide this shortly (I dont have full access to the machine it is on)

networkfupa · Post by **networkfupa** » Tue Oct 21, 2008 8:13 pm

TheWitness wrote:You need to provide a little more info. Like what version of Cacti, what poller? Also, show what is appearing in your Cacti Log.

TheWitness

We are running cacti-0.8.6k and cactid-0.8.8l

I upped the logging level for one polling cycle. Here’s the log from that (the portion that pertains to the OIDs)

Code: Select all

10/21/2008 05:40:20 PM - CACTID: Poller[0] Host[803] ERROR: Problems parsing Multi SNMP OID! (oid: 1.3.6.1.4.1.2008.2800.100.1.2.101.1 )
10/21/2008 05:40:21 PM - CACTID: Poller[0] Host[803] DS[22912] SNMP: v2: pl1rap904.media.purewebcraft.com, dsname: ChMemMax7001, oid: 1.3.6.1.4.1.2008.2800.100.1.2.101.1 , value: U
10/21/2008 05:40:21 PM - CACTID: Poller[0] Host[803] DS[22911] SNMP: v2: pl1rap904.media.purewebcraft.com, dsname: ChMemAvail7001, oid: 1.3.6.1.4.1.2008.2800.100.1.1.101.1, value: 1016
10/21/2008 05:40:21 PM - CACTID: Poller[0] Host[803] ERROR: Problems parsing Multi SNMP OID! (oid: 1.3.6.1.4.1.2008.2800.100.2.1.101.1 )
10/21/2008 05:40:21 PM - CACTID: Poller[0] Host[803] ERROR: Problems parsing Multi SNMP OID! (oid: 1.3.6.1.4.1.2008.2800.100.2.1.101.1 )
10/21/2008 05:40:21 PM - CACTID: Poller[0] Host[803] ERROR: Problems parsing Multi SNMP OID! (oid: 1.3.6.1.4.1.2008.2800.100.2.6.101.1 )
10/21/2008 05:40:21 PM - CACTID: Poller[0] Host[803] DS[22914] SNMP: v2: pl1rap904.media.purewebcraft.com, dsname: ChMemMax8001, oid: 1.3.6.1.4.1.2008.2800.100.2.2.101.1, value: 2048
10/21/2008 05:40:21 PM - CACTID: Poller[0] Host[803] DS[22913] SNMP: v2: pl1rap904.media.purewebcraft.com, dsname: ChMemAvail8001, oid: 1.3.6.1.4.1.2008.2800.100.2.1.101.1 , value: U
10/21/2008 05:40:21 PM - CACTID: Poller[0] Host[803] DS[22918] SNMP: v2: pl1rap904.media.purewebcraft.com, dsname: CTcpRing7001, oid: 1.3.6.1.4.1.2008.2800.100.1.6.101.1, value: 8287
10/21/2008 05:40:21 PM - CACTID: Poller[0] Host[803] DS[22917] SNMP: v2: pl1rap904.media.purewebcraft.com, dsname: CRecvRate7001, oid: 1.3.6.1.4.1.2008.2800.100.1.5.101.1, value: 0.99834526
10/21/2008 05:40:21 PM - CACTID: Poller[0] Host[803] DS[22916] SNMP: v2: pl1rap904.media.purewebcraft.com, dsname: CPacketPub7001, oid: 1.3.6.1.4.1.2008.2800.100.1.4.101.1, value: 0.023172788
10/21/2008 05:40:21 PM - CACTID: Poller[0] Host[803] DS[22915] SNMP: v2: pl1rap904.media.purewebcraft.com, dsname: CPacketDelivery7001, oid: 1.3.6.1.4.1.2008.2800.100.2.1.101.1 , value: U
10/21/2008 05:40:21 PM - CACTID: Poller[0] Host[803] DS[22922] SNMP: v2: pl1rap904.media.purewebcraft.com, dsname: CTcpRing8001, oid: 1.3.6.1.4.1.2008.2800.100.2.6.101.1 , value: U
10/21/2008 05:40:21 PM - CACTID: Poller[0] Host[803] DS[22921] SNMP: v2: pl1rap904.media.purewebcraft.com, dsname: CRecvRate8001, oid: 1.3.6.1.4.1.2008.2800.100.2.5.101.1, value: 0.9988621
10/21/2008 05:40:21 PM - CACTID: Poller[0] Host[803] DS[22920] SNMP: v2: pl1rap904.media.purewebcraft.com, dsname: CPacketPub8001, oid: 1.3.6.1.4.1.2008.2800.100.2.4.101.1, value: 0.023172598
10/21/2008 05:40:21 PM - CACTID: Poller[0] Host[803] DS[22919] SNMP: v2: pl1rap904.media.purewebcraft.com, dsname: CPacketDelivery8001, oid: 1.3.6.1.4.1.2008.2800.100.2.3.101.1, value: 0.99919015
10/21/2008 05:40:21 PM - CACTID: Poller[0] Host[803] ERROR: Problems parsing Multi SNMP OID! (oid: 1.3.6.1.4.1.2008.2800.100.1.2.101.1 )
10/21/2008 05:40:21 PM - CACTID: Poller[0] Host[803] DS[22923] SNMP: v2: pl1rap904.media.purewebcraft.com, dsname: ChMemMax7001, oid: 1.3.6.1.4.1.2008.2800.100.1.2.101.1 , value: U

Post by **TheWitness** » Tue Oct 21, 2008 8:25 pm

Ah, I see, it is a string output. Well, that would likely be a bug in cactid handling strings. What I would like to encourage is that you consider moving to Cacti 0.8.7c, once released and maybe participate in the beta, as I was not planning on releasing another cactid version. If you move to 0.8.7x, you can use spine, which is under active development.

The other workaround would be to gather this in a script, but I don't think that that technique scales.

Also, a quick hack to cactid pre-validation logic might fix things too.

TheWitness

networkfupa · Post by **networkfupa** » Tue Oct 21, 2008 11:55 pm

TheWitness wrote:Ah, I see, it is a string output. Well, that would likely be a bug in cactid handling strings. What I would like to encourage is that you consider moving to Cacti 0.8.7c, once released and maybe participate in the beta, as I was not planning on releasing another cactid version. If you move to 0.8.7x, you can use spine, which is under active development.

The other workaround would be to gather this in a script, but I don't think that that technique scales.

Also, a quick hack to cactid pre-validation logic might fix things too.

TheWitness

Hm, upgrading doesn't seem like a viable option at the moment. The strange thing is that the string works and is graphed for other data templates, but others not.

How could I change the cactid pre-validation logic?
Or is it possible to convert the snmp data type from a string to an int or double?

The way I am exporting the snmp information is by using a simple

Code: Select all

exec OID command

Post by **TheWitness** » Wed Oct 22, 2008 5:29 am

There is a strip_quotes command in Cactid. In spine, I use a trim command which executes an rtrim(ltrim(string)). You can alter the trim command to remove single and double quotes as well.

So the fix would include bringing those three utilities from util.c into cactid and modifying the validation logic in cactid. Actually the validation code in spine 0.8.7c beta2 has improved quite a bit from cactid. I would likely copy those whole sections.

TheWitness

networkfupa · Post by **networkfupa** » Wed Oct 22, 2008 11:52 am

TheWitness wrote:There is a strip_quotes command in Cactid. In spine, I use a trim command which executes an rtrim(ltrim(string)). You can alter the trim command to remove single and double quotes as well.

So the fix would include bringing those three utilities from util.c into cactid and modifying the validation logic in cactid. Actually the validation code in spine 0.8.7c beta2 has improved quite a bit from cactid. I would likely copy those whole sections.

TheWitness

Do you happen to know of any ways I can change the data format on the snmpd.conf side?

networkfupa · Post by **networkfupa** » Wed Oct 22, 2008 6:10 pm

TheWitness wrote:There is a strip_quotes command in Cactid. In spine, I use a trim command which executes an rtrim(ltrim(string)). You can alter the trim command to remove single and double quotes as well.

So the fix would include bringing those three utilities from util.c into cactid and modifying the validation logic in cactid. Actually the validation code in spine 0.8.7c beta2 has improved quite a bit from cactid. I would likely copy those whole sections.

TheWitness

How would I go about installing the beta version? I grabbed the lastest 0.8.7 branch from the subversion repository. Is this 0.8.7c beta2, or should I get it from the main?

Post by **TheWitness** » Wed Oct 22, 2008 7:51 pm

branches/0.8.7.

networkfupa · Post by **networkfupa** » Thu Oct 23, 2008 12:55 am

TheWitness wrote:Ah, I see, it is a string output. Well, that would likely be a bug in cactid handling strings. What I would like to encourage is that you consider moving to Cacti 0.8.7c, once released and maybe participate in the beta, as I was not planning on releasing another cactid version. If you move to 0.8.7x, you can use spine, which is under active development.

The other workaround would be to gather this in a script, but I don't think that that technique scales.

Also, a quick hack to cactid pre-validation logic might fix things too.

TheWitness

Hi TheWitness,
I have tried installing the latest 0.8.7c to try and resolve my problem, but it seems like I am getting some errors when just navigating around the cacti admin console. For example, when I go to graph management, I get a blank page. I see the following in the httpd error log:

[client 10.67.44.44] PHP Parse error: parse error, unexpected '=', expecting ')' in /var/www/html/cacti/lib/rrd.php on line 68, referer: http://10.67.33.44/cacti/graph_view.php ... &leaf_id=9
[client 10.67.44.44] PHP Parse error: parse error, unexpected '=', expecting ')' in /var/www/html/cacti/lib/rrd.php on line 68, referer: http://10.67.44.44/cacti/settings.php?tab=general
[client 10.67.44.44] PHP Parse error: parse error, unexpected '=', expecting ')' in /var/www/html/cacti/lib/rrd.php on line 68, referer: http://10.67.44.44/cacti/settings.php?tab=general

I'm not sure if this would prevent me from graphing, because I am able to navigate around in other areas of the admin console.
I have created some hosts, data templates and graph templates, but it seems as if no data is being polled. I have followed all of the instructions in the documentation to build this out thus far.

Nothing is showing in cacti.log and it is set to debug.

Here is my version of rrdtool just in case:
[root@localhost ~]# rrdtool -v
RRDtool 1.0.50 Copyright 1997-2004 by Tobias Oetiker <tobi@oetiker.ch>

Usage: rrdtool [options] command command_options

Valid commands: create, update, graph, dump, restore,
last, first, info, fetch, tune, resize, xport

RRDtool is distributed under the Terms of the GNU General
Public License Version 2. (www.gnu.org/copyleft/gpl.html)

For more information read the RRD manpages

I tried to run the poller manually as well and that seemed to not work:
[cactiuser@localhost ~]$ php /var/www/htmlcacti/poller.php
Status: 404
Content-type: text/html
X-Powered-By: PHP/4.3.9

No input file specified.

I am using php4, should I be using 5?

Post by **TheWitness** » Thu Oct 23, 2008 6:27 am

First, can you log a bug at http://bugs.cacti.net relative to the rrd.php issue. The PHP version is entirely up to you, but PHP5 offers many advantages. If you are using CentOS, and you enable centosplus (yum.conf), you will automatically get PHP5 and MYSQL5.

It's hidden, but it makes running CentOS 4.x very convenient. Of course, CentOS5 includes these things out of the gate.

Regards,

Larry

networkfupa · Post by **networkfupa** » Fri Nov 21, 2008 2:36 pm

Ok for now I am being forced to use a non-beta version of cacti.
How can I go about finding a solution without using version 0.8.7c?

networkfupa · Post by **networkfupa** » Mon Jan 05, 2009 5:53 pm

I noticed this following error when running the poller on the command line:01/05/2009 03:23:08 PM - POLLER: Poller[0] WARNING: Cron is out of sync with the Poller Interval! The Poller Interval is '300' seconds, with a maximum of a '300' second Cron, but 652 seconds have passed since the last poll!

I have re-started with a fresh installation.
I only added ONE single host.

I created a data template and a new graph template for an OID.
I extended the snmpd.conf with the following line and restarted snmpd:

exec 1.3.6.1.4.1.2008.2600.103.2.1 hibernate_ConnectCount_vm2 /bin/cat /usr/local/logs/stats/hibernate_ConnectCount_vm2

I can poll the OID on the server using snmpwalk:[msacks@tl1rad718 ~]$ snmpwalk -v2c -c public myServer 1.3.6.1.4.1.2008.2600.103
SNMPv2-SMI::enterprises.2008.2600.103.2.1.1.1 = INTEGER: 1
SNMPv2-SMI::enterprises.2008.2600.103.2.1.2.1 = STRING: "hibernate_ConnectCount_vm2"
SNMPv2-SMI::enterprises.2008.2600.103.2.1.3.1 = STRING: "/bin/cat /usr/local/logs/stats/hibernate_ConnectCount_vm2"
SNMPv2-SMI::enterprises.2008.2600.103.2.1.100.1 = INTEGER: 0
SNMPv2-SMI::enterprises.2008.2600.103.2.1.101.1 = STRING: "2357756"
SNMPv2-SMI::enterprises.2008.2600.103.2.1.102.1 = INTEGER: 0
SNMPv2-SMI::enterprises.2008.2600.103.2.1.103.1 = ""

This is the OID I am putting into the data template as an snmp query:
1.3.6.1.4.1.2008.2600.103.2.1.101.1

and yet the rra STILL gets populated with NaN's.

I then associate a graph template with the device and it creates a graph, but the graph is blank and the rra is full of NaN's.

What is cacti incapable of doing which snmpwalk can?

Where is the disconnect?
How might I go about solving this issue?

networkfupa · Post by **networkfupa** » Mon Jan 05, 2009 5:58 pm

Bug ID http://bugs.cacti.net/view.php?id=1360 opened for this issue

rrd file shows NaN for data values, but snmpget gathers data

rrd file shows NaN for data values, but snmpget gathers data

Who is online