Cacti seems to loose values between poller and rrd

[NLCoredump]

I encountered a strange problem.

I have a Script/Command datasource which looks to work ok. This datasource gives me 7 values.

01/27/2010 08:40:16 PM - SPINE: Poller[0] Host[648] DS[15768] SCRIPT: /usr/bin/snmpget -v2c -Ovqa -c ***** hostname.domain .1.3.6.1.3.1983.1.4.4.1.2.18.97.110.115.119.101.114.115.95.116.111.95.113.117.101.114.105.101.115.1,
output: "num_a_NOER:4733706 num_a_NXDOMAIN:511780 num_a_SERVFAULT:0 num_a_nodata:120512 num_a_secure:0 num_a_bogus:0 num_r_bogus:0"

This looks ok, but now the comes the problem.

01/27/2010 08:40:17 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/cacti/rra/hostname_unbound_dns_num_r_bogus_15768.rrd
--template num_a_NXDOMAIN:num_a_SERVFAULT:num_a_nodata:num_a_secure:num_a_bogus 1264621214:511780:0:120512:0:0

Somehow the first and the last value of the original string got lost.

This is just an example, i have other datatemplates which use an similar data input methode that also have this problem, somehow the first and the last value disapear. I suspect the problem to be in the quotes.

I searched the forum for similar problems but didn't find any. But it is possible that this is an old problem, if so, please post relevant links.

The problem did occure 6 months ago, but was only recently noticed by the viewers of the graphs. It must have started after an upgrade. The problem is the i have no idea what they did, the people that did it are no longer available, and after 6 months the logging of that periode is no longer available. So what they have done will stay a mistery, but i still have to get it running again.

[gandalf]

Please verify that the scripts spits out all results using a single "print" call. No multiple prints. No prints in a loop, please.
R.

[NLCoredump]

The script/command is actualy an snmp-get on one oid. This gives one string with the values as shown in the logging.

The matching data template has all values defined the same and initial rrd-file is also created with 7 datasources.

But some how when the poller parses the String supplied by the snmp-get command it drops the first and the last value.

[gandalf]

Is it possible, that you added the two missing data source to the data template after having created the rrd file? In this case, delete the file and wait 10 minutes.
R.

[NLCoredump]

I added the same template again to the host. rrd-file was newly created with the rigth data sources.

rrdtool info shows following:

ds[num_a_NOER].type = "ABSOLUTE"
ds[num_a_NOER].minimal_heartbeat = 600
ds[num_a_NOER].min = 0.0000000000e+00
ds[num_a_NOER].max = NaN
ds[num_a_NOER].last_ds = "U"
ds[num_a_NOER].value = NaN
ds[num_a_NOER].unknown_sec = 15
ds[num_a_NXDOMAIN].type = "ABSOLUTE"
ds[num_a_NXDOMAIN].minimal_heartbeat = 600
ds[num_a_NXDOMAIN].min = 0.0000000000e+00
ds[num_a_NXDOMAIN].max = NaN
ds[num_a_NXDOMAIN].last_ds = "413248"
ds[num_a_NXDOMAIN].value = 2.0593754153e+04
ds[num_a_NXDOMAIN].unknown_sec = 0
ds[num_a_SERVFAULT].type = "ABSOLUTE"
ds[num_a_SERVFAULT].minimal_heartbeat = 600
ds[num_a_SERVFAULT].min = 0.0000000000e+00
ds[num_a_SERVFAULT].max = NaN
ds[num_a_SERVFAULT].last_ds = "0"
ds[num_a_SERVFAULT].value = 0.0000000000e+00
ds[num_a_SERVFAULT].unknown_sec = 0
ds[num_a_nodata].type = "ABSOLUTE"
ds[num_a_nodata].minimal_heartbeat = 600
ds[num_a_nodata].min = 0.0000000000e+00
ds[num_a_nodata].max = NaN
ds[num_a_nodata].last_ds = "90786"
ds[num_a_nodata].value = 4.5242192691e+03
ds[num_a_nodata].unknown_sec = 0
ds[num_a_secure].type = "ABSOLUTE"
ds[num_a_secure].minimal_heartbeat = 600
ds[num_a_secure].min = 0.0000000000e+00
ds[num_a_secure].max = NaN
ds[num_a_secure].last_ds = "0"
ds[num_a_secure].value = 0.0000000000e+00
ds[num_a_secure].unknown_sec = 0
ds[num_a_bogus].type = "ABSOLUTE"
ds[num_a_bogus].minimal_heartbeat = 600
ds[num_a_bogus].min = 0.0000000000e+00
ds[num_a_bogus].max = NaN
ds[num_a_bogus].last_ds = "0"
ds[num_a_bogus].value = 0.0000000000e+00
ds[num_a_bogus].unknown_sec = 0
ds[num_r_bogus].type = "ABSOLUTE"
ds[num_r_bogus].minimal_heartbeat = 600
ds[num_r_bogus].min = 0.0000000000e+00
ds[num_r_bogus].max = NaN
ds[num_r_bogus].last_ds = "U"
ds[num_r_bogus].value = NaN
ds[num_r_bogus].unknown_sec = 15


This is the same rrd-file which belongs the the log entries posted earlier. The thing is that the whole setup worked until 6 months ago. And than stopped after some people did changes which where not recorded. It could be that they upgrade to 0.8.7d but it could also be that they activated the spine poller. Or both.

If i take a look at the original rrd file, than i can see historical data for the datasources that are dropped now.

Doesn't mather how many datasources are supplied in the String. The first and the last are always missing. I tested it with similar data-templates which use a similar data input methode. Same problem. That makes me think that the problem could be in the quotes. I found 2 strip_quotes functions in the php-code which i isolated and tested, they work fine if i feed it the string supplied by the snmp-get. But i have no clue if the are actually used in this process, probably not as we use spine.

[NLCoredump]

The command that is run outputs following when run form cli

[root@*]# /usr/bin/snmpget -v2c -Ovqa -c ***** host.domain .1.3.6.1.3.1983.1.4.4.1.2.18.97.110.115.119.101.114.115.95.116.111.95.113.117.101.114.105.101.115.1
"num_a_NOER:2722023 num_a_NXDOMAIN:353447 num_a_SERVFAULT:0 num_a_nodata:74885 num_a_secure:0 num_a_bogus:1 num_r_bogus:0"

This mathes with what the logging shows. The thing i don't get is that when it goes to the rrd-tool, the values are just missing from the template line and from the data.

rrdtool update some.rrd --template num_a_NXDOMAIN:num_a_SERVFAULT:num_a_nodata:num_a_secure:num_a_bogus 1264621214:511780:0:120512:0:0

[gandalf]

Doesn't mather how many datasources are supplied in the String. The first and the last are always missing. I tested it with similar data-templates which use a similar data input methode. Same problem. That makes me think that the problem could be in the quotes. I found 2 strip_quotes functions in the php-code which i isolated and tested, they work fine if i feed it the string supplied by the snmp-get. But i have no clue if the are actually used in this process, probably not as we use spine.

Please approach TheWitness with this information. He's the spine guy
R.