Getting NaN in RRD & Graph, Even when Poller shows a Value

[skynetcm]

Cacti Version: 0.8.8
CentOS
Linux 2.6.18-194.26.1.el5 #1 SMP Tue Nov 9 12:54:40 EST 2010 i686 i686 i386 GNU/Linux

I currently have graphs that are not working at all , After alot of digging I suspect that there is an issue between the poller and the creation of the rrd file.

i can snmp get the Device and get a result, I can see in the logs that the poller is also getting a result (See below) , but when i

From the Poller:

SPINE: Poller[0] Host[172] TH[1] DS[3207] SNMP: v2: 192.168.101.2, dsname: gigeEth1CollisionEr, oid: .1.3.6.1.4.1.5454.1.80.4.1.2.10.0, value: 0

The Value of 0 is correct.

From the rrdtool info:

filename = "tacoma_trango_-_baldi_gigeeth1collisioner_3207.rrd"
rrd_version = "0003"
step = 60
last_update = 1348177822
header_size = 3428
ds[gigeEth1CollisionEr].index = 0
ds[gigeEth1CollisionEr].type = "COUNTER"
ds[gigeEth1CollisionEr].minimal_heartbeat = 120
ds[gigeEth1CollisionEr].min = 0.0000000000e+00
ds[gigeEth1CollisionEr].max = 1.0000000000e+01
ds[gigeEth1CollisionEr].last_ds = "0"
ds[gigeEth1CollisionEr].value = NaN
ds[gigeEth1CollisionEr].unknown_sec = 22
rra[0].cf = "AVERAGE"
rra[0].rows = 600
rra[0].cur_row = 142
rra[0].pdp_per_row = 1
rra[0].xff = 5.0000000000e-01
rra[0].cdp_prep[0].value = NaN
rra[0].cdp_prep[0].unknown_datapoints = 0
rra[1].cf = "AVERAGE"
rra[1].rows = 700
rra[1].cur_row = 59
rra[1].pdp_per_row = 6
rra[1].xff = 5.0000000000e-01
rra[1].cdp_prep[0].value = 0.0000000000e+00
rra[1].cdp_prep[0].unknown_datapoints = 2
rra[2].cf = "AVERAGE"
rra[2].rows = 775
rra[2].cur_row = 582
rra[2].pdp_per_row = 24
rra[2].xff = 5.0000000000e-01
rra[2].cdp_prep[0].value = 0.0000000000e+00
rra[2].cdp_prep[0].unknown_datapoints = 14
rra[3].cf = "AVERAGE"
rra[3].rows = 797
rra[3].cur_row = 666
rra[3].pdp_per_row = 288
rra[3].xff = 5.0000000000e-01
rra[3].cdp_prep[0].value = NaN
rra[3].cdp_prep[0].unknown_datapoints = 158
rra[4].cf = "MIN"
rra[4].rows = 600
rra[4].cur_row = 569
rra[4].pdp_per_row = 1
rra[4].xff = 5.0000000000e-01
rra[4].cdp_prep[0].value = NaN
rra[4].cdp_prep[0].unknown_datapoints = 0
rra[5].cf = "MIN"
rra[5].rows = 700
rra[5].cur_row = 40
rra[5].pdp_per_row = 6
rra[5].xff = 5.0000000000e-01
rra[5].cdp_prep[0].value = inf
rra[5].cdp_prep[0].unknown_datapoints = 2
rra[6].cf = "MIN"
rra[6].rows = 775
rra[6].cur_row = 692
rra[6].pdp_per_row = 24
rra[6].xff = 5.0000000000e-01
rra[6].cdp_prep[0].value = inf
rra[6].cdp_prep[0].unknown_datapoints = 14
rra[7].cf = "MIN"
rra[7].rows = 797
rra[7].cur_row = 354
rra[7].pdp_per_row = 288
rra[7].xff = 5.0000000000e-01
rra[7].cdp_prep[0].value = NaN
rra[7].cdp_prep[0].unknown_datapoints = 158
rra[8].cf = "MAX"
rra[8].rows = 600
rra[8].cur_row = 564
rra[8].pdp_per_row = 1
rra[8].xff = 5.0000000000e-01
rra[8].cdp_prep[0].value = NaN
rra[8].cdp_prep[0].unknown_datapoints = 0
rra[9].cf = "MAX"
rra[9].rows = 700
rra[9].cur_row = 384
rra[9].pdp_per_row = 6
rra[9].xff = 5.0000000000e-01
rra[9].cdp_prep[0].value = -inf
rra[9].cdp_prep[0].unknown_datapoints = 2
rra[10].cf = "MAX"
rra[10].rows = 775
rra[10].cur_row = 409
rra[10].pdp_per_row = 24
rra[10].xff = 5.0000000000e-01
rra[10].cdp_prep[0].value = -inf
rra[10].cdp_prep[0].unknown_datapoints = 14
rra[11].cf = "MAX"
rra[11].rows = 797
rra[11].cur_row = 305
rra[11].pdp_per_row = 288
rra[11].xff = 5.0000000000e-01
rra[11].cdp_prep[0].value = NaN
rra[11].cdp_prep[0].unknown_datapoints = 158
rra[12].cf = "LAST"
rra[12].rows = 600
rra[12].cur_row = 447
rra[12].pdp_per_row = 1
rra[12].xff = 5.0000000000e-01
rra[12].cdp_prep[0].value = NaN
rra[12].cdp_prep[0].unknown_datapoints = 0
rra[13].cf = "LAST"
rra[13].rows = 700
rra[13].cur_row = 15
rra[13].pdp_per_row = 6
rra[13].xff = 5.0000000000e-01
rra[13].cdp_prep[0].value = NaN
rra[13].cdp_prep[0].unknown_datapoints = 2
rra[14].cf = "LAST"
rra[14].rows = 775
rra[14].cur_row = 161
rra[14].pdp_per_row = 24
rra[14].xff = 5.0000000000e-01
rra[14].cdp_prep[0].value = NaN
rra[14].cdp_prep[0].unknown_datapoints = 14
rra[15].cf = "LAST"
rra[15].rows = 797
rra[15].cur_row = 196
rra[15].pdp_per_row = 288
rra[15].xff = 5.0000000000e-01
rra[15].cdp_prep[0].value = NaN
rra[15].cdp_prep[0].unknown_datapoints = 158


and also on a fetch:

1348091280: nan
1348091640: nan
1348092000: nan
1348092360: nan


From Cacti the Graph Template Debug:

RRDTool Command:
/usr/bin/rrdtool graph - \
--imgformat=PNG \
--start=-86400 \
--end=-60 \
--title='Baldi Trango -> Tacoma - GigE - GE1' \
--base=1000 \
--height=120 \
--width=500 \
--alt-autoscale-max \
--lower-limit=0 \
--vertical-label='' \
--slope-mode \
--font TITLE:10: \
--font AXIS:7: \
--font LEGEND:8: \
--font UNIT:7: \
DEF:a="/var/www/cacti/rra/baldi_trango_-_tacoma_gigeeth1crcerror_3235.rrd":"gigeEth1CRCError":AVERAGE \
DEF:b="/var/www/cacti/rra/baldi_trango_-_tacoma_gigeeth1collisioner_3234.rrd":"gigeEth1CollisionEr":AVERAGE \
LINE2:a#FF0000FF:"CRC Errors" \
GPRINTLAST:"Last\:%8.3lf %s" \
GPRINTAVERAGE:"Avg\:%8.3lf %s" \
GPRINTMAX:"Max\:%8.3lf %s\n" \
LINE2:b#FFAB00FF:"Collisions" \
GPRINTLAST:"Last\:%8.3lf %s" \
GPRINTAVERAGE:"Avg\:%8.3lf %s" \
GPRINTMAX:"Max\:%8.3lf %s\n"
RRDTool Says:
OK

I am at a loss trying to figure out why even tho i get a value , why the rrd is show Nan:


Any help is appreciated.

[gandalf]

What is your polling interval? 300 sec or 60 sec?
R.

[skynetcm]

Yea after alot more digging that i found that alot of the rrd files had a step of 60 , and a heartbeat of 120 from when were were running everyone min.

Thus causing cacti to ignore the data .

That has since been changed to poll every 5 min. and i have updated the templates to now use 300 step and 600 heartbeat.
But i have yet to figure out how to update the --step to 300 via command line with out running the rrdtool create and loosing current data.

i have search around and re-read the rrdtool commands, and dont find one that can update the --step to 300. Any thoughts?

[gandalf]

You can't update the rrd file's "step" value. You will have to drop the old files. Cacti will recreate the new ones on next polling interval automatically using the current settings.
R.