NaN and gaps in Cacti graph

[sawkal28]

Hi,

I'm new to Cacti and saw that some of the graphs for one of the monitored device/router had NaN values. My poller interval is 1 minute and only the graphs for that particular device is not updated in /var/www/htm/rra and therefore no data in the graphs. Not a permission issue because the other graphs are created although there seem to be gaps in the other graphs of the other 3 devices. There is no rrdtool update when I checked under Cacti Log File (below are the logs for the device) but the data source are in the Poller Cache. Downed Host Detection is SNMP. Tried to do what others had done, but still does not work. I could have missed some other things or might not understand. So appreciate if someone can help.

04/18/2011 05:10:58 PM - SPINE: Poller[0] Host[32] DEBUG: The POPEN returned the following File Descriptor 9
04/18/2011 05:10:58 PM - SPINE: Poller[0] Host[32] DS[637] SCRIPT: /usr/bin/perl /var/www/html/scripts/qospol.pl 172.30.2.40 nar1Sa4 2 get QoSbitrate 2.BULKDATA.2, output: 819000
04/18/2011 05:10:47 PM - SPINE: Poller[0] Host[32] DEBUG: The POPEN returned the following File Descriptor 9
04/18/2011 05:10:47 PM - SPINE: Poller[0] Host[32] DS[636] SCRIPT: /usr/bin/perl /var/www/html/scripts/qospol.pl 172.30.2.40 nar1Sa4 2 get QoSbitrate 1.VOIP&SIG.2, output: U
04/18/2011 05:10:36 PM - SPINE: Poller[0] Host[32] DEBUG: The POPEN returned the following File Descriptor 9
04/18/2011 05:10:36 PM - SPINE: Poller[0] Host[32] DS[635] SCRIPT: /usr/bin/perl /var/www/html/scripts/qospol.pl 172.30.2.40 nar1Sa4 2 get QoSbitrate 1.class-default.2, output: 427000
04/18/2011 05:10:26 PM - SPINE: Poller[0] Host[32] DEBUG: The POPEN returned the following File Descriptor 9
04/18/2011 05:10:26 PM - SPINE: Poller[0] Host[32] RECACHE: Processing 2 items in the auto reindex cache for '172.30.2.40'
04/18/2011 05:10:26 PM - SPINE: Poller[0] Host[32] SNMP Result: Host responded to SNMP


Thanks,
Diana

[sawkal28]

Also when trying to execute RRDTool Command, still NaN value:

/usr/bin/rrdtool graph - \
--imgformat=PNG \
--start=-86400 \
--end=-60 \
--title="Fa0/0 class class-default - traffic" \
--rigid \
--base=1000 \
--height=120 \
--width=500 \
--alt-autoscale-max \
--lower-limit=0 \
--vertical-label="bit/sec" \
--slope-mode \
--font TITLE:10: \
--font AXIS:8: \
--font LEGEND:8: \
--font UNIT:8: \
DEF:a="/var/www/html/rra/xxxx1841_ds_635.rrd":QoStrafficperclass:AVERAGE \
AREA:a#FFD660FF:"Traffic per class" \
GPRINTLAST:"Current%8.0lf" \
GPRINTAVERAGE:"Average%8.0lf" \
GPRINTAVERAGE:"Average%8.0lf" \
GPRINTMAX:"Maximum%8.0lf" \
GPRINTMAX:"Maximum%8.0lf"
RRDTool Says:

OK

[gandalf]

Please show us "rrdtool info" for the failing data source's rrd file
R.

[sawkal28]

Also I've noticed that if I did not put my snmp community name for my router, then I will see the rrdtool update in DEBUG but the result is still NaN :

04/19/2011 10:24:23 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/rra/XX_ds_635.rrd --template QoStrafficperclass 1303223058:U
04/19/2011 10:24:23 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/rra/XX_ds_636.rrd --template QoStrafficperclass 1303223058:U
04/19/2011 10:24:23 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/rra/XX_ds_637.rrd --template QoStrafficperclass 1303223058:U
04/19/2011 10:24:23 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/rra/XX_ds_638.rrd --template QoStrafficperclass 1303223058:U
04/19/2011 10:24:23 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/rra/XX_ds_639.rrd --template QoStrafficperclass 1303223058:U
04/19/2011 10:24:23 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/rra/XX_ds_641.rrd --template QoStrafficperclass 1303223058:U
04/19/2011 10:24:23 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/rra/XX_ds_640.rrd --template QoStrafficperclass 1303223058:U

04/19/2011 10:25:22 PM - SPINE: Poller[0] Host[32] ERROR: Empty result [172.30.2.40]: '/usr/bin/perl /var/www/html/scripts/qospol.pl 172.30.2.40 2 get QoSbitrate 2.VOIP-RTP.2'
04/19/2011 10:25:22 PM - SPINE: Poller[0] Host[32] DEBUG: The POPEN returned the following File Descriptor 9
04/19/2011 10:25:22 PM - SPINE: Poller[0] Host[32] DS[640] SCRIPT: /usr/bin/perl /var/www/html/scripts/qospol.pl 172.30.2.40 2 get QoSbitrate 2.MISSIONCRITICAL.2, output: U
04/19/2011 10:25:22 PM - SPINE: Poller[0] Host[32] ERROR: Empty result [172.30.2.40]: '/usr/bin/perl /var/www/html/scripts/qospol.pl 172.30.2.40 2 get QoSbitrate 2.MISSIONCRITICAL.2'
04/19/2011 10:25:21 PM - SPINE: Poller[0] Host[32] DEBUG: The POPEN returned the following File Descriptor 9
04/19/2011 10:25:21 PM - SPINE: Poller[0] Host[32] DS[639] SCRIPT: /usr/bin/perl /var/www/html/scripts/qospol.pl 172.30.2.40 2 get QoSbitrate 2.class-default.2, output: U
04/19/2011 10:25:21 PM - SPINE: Poller[0] Host[32] ERROR: Empty result [172.30.2.40]: '/usr/bin/perl /var/www/html/scripts/qospol.pl 172.30.2.40 2 get QoSbitrate 2.class-default.2'
04/19/2011 10:25:21 PM - SPINE: Poller[0] Host[32] DEBUG: The POPEN returned the following File Descriptor 9
04/19/2011 10:25:21 PM - SPINE: Poller[0] Host[32] DS[638] SCRIPT: /usr/bin/perl /var/www/html/scripts/qospol.pl 172.30.2.40 2 get QoSbitrate 2.BUSINESSAPPS.2, output: U
04/19/2011 10:25:21 PM - SPINE: Poller[0] Host[32] ERROR: Empty result [172.30.2.40]: '/usr/bin/perl /var/www/html/scripts/qospol.pl 172.30.2.40 2 get QoSbitrate 2.BUSINESSAPPS.2'
04/19/2011 10:25:20 PM - SPINE: Poller[0] Host[32] DEBUG: The POPEN returned the following File Descriptor 9
04/19/2011 10:25:20 PM - SPINE: Poller[0] Host[32] DS[637] SCRIPT: /usr/bin/perl /var/www/html/scripts/qospol.pl 172.30.2.40 2 get QoSbitrate 2.BULKDATA.2, output: U
04/19/2011 10:25:20 PM - SPINE: Poller[0] Host[32] ERROR: Empty result [172.30.2.40]: '/usr/bin/perl /var/www/html/scripts/qospol.pl 172.30.2.40 2 get QoSbitrate 2.BULKDATA.2'
04/19/2011 10:25:20 PM - SPINE: Poller[0] Host[32] DEBUG: The POPEN returned the following File Descriptor 9
04/19/2011 10:25:20 PM - SPINE: Poller[0] Host[32] DS[636] SCRIPT: /usr/bin/perl /var/www/html/scripts/qospol.pl 172.30.2.40 2 get QoSbitrate 1.VOIP&SIG.2, output: U
04/19/2011 10:25:20 PM - SPINE: Poller[0] Host[32] ERROR: Empty result [172.30.2.40]: '/usr/bin/perl /var/www/html/scripts/qospol.pl 172.30.2.40 2 get QoSbitrate 1.VOIP&SIG.2'

And when I manually enter the below, I get a value
[root@localhost rra]# /usr/bin/perl /var/www/html/scripts/qospol.pl 172.30.2.40 nar1Sa4 2 get QoSbitrate 2.BULKDATA.2
5000

Anything wrong with qospol.pl ?

Appreciate if anyone can help,

Thanks,
Diana

[sawkal28]

Hi gandalf,

Here's the rrdtool info

[root@localhost rra]# /usr/bin/rrdtool info /var/www/html/rra/XX_ds_635.rrd | tail
rra[18].cdp_prep[1].value = NaN
rra[18].cdp_prep[1].unknown_datapoints = 5
rra[19].cf = "LAST"
rra[19].rows = 797
rra[19].pdp_per_row = 288
rra[19].xff = 5.0000000000e-01
rra[19].cdp_prep[0].value = NaN
rra[19].cdp_prep[0].unknown_datapoints = 11
rra[19].cdp_prep[1].value = NaN
rra[19].cdp_prep[1].unknown_datapoints = 11
[root@localhost rra]# /usr/bin/rrdtool info /var/www/html/rra/XX_ds_636.rrd | tail
rra[18].cdp_prep[1].value = NaN
rra[18].cdp_prep[1].unknown_datapoints = 5
rra[19].cf = "LAST"
rra[19].rows = 797
rra[19].pdp_per_row = 288
rra[19].xff = 5.0000000000e-01
rra[19].cdp_prep[0].value = NaN
rra[19].cdp_prep[0].unknown_datapoints = 11
rra[19].cdp_prep[1].value = NaN
rra[19].cdp_prep[1].unknown_datapoints = 11
[root@localhost rra]# /usr/bin/rrdtool info /var/www/html/rra/XX_ds_637.rrd | tail
rra[18].cdp_prep[1].value = NaN
rra[18].cdp_prep[1].unknown_datapoints = 11
rra[19].cf = "LAST"
rra[19].rows = 797
rra[19].pdp_per_row = 288
rra[19].xff = 5.0000000000e-01
rra[19].cdp_prep[0].value = NaN
rra[19].cdp_prep[0].unknown_datapoints = 11
rra[19].cdp_prep[1].value = NaN
rra[19].cdp_prep[1].unknown_datapoints = 11
[root@localhost rra]# /usr/bin/rrdtool info /var/www/html/rra/XX_ds_638.rrd | tail
rra[18].cdp_prep[1].value = NaN
rra[18].cdp_prep[1].unknown_datapoints = 5
rra[19].cf = "LAST"
rra[19].rows = 797
rra[19].pdp_per_row = 288
rra[19].xff = 5.0000000000e-01
rra[19].cdp_prep[0].value = NaN
rra[19].cdp_prep[0].unknown_datapoints = 11
rra[19].cdp_prep[1].value = NaN
rra[19].cdp_prep[1].unknown_datapoints = 11
[root@localhost rra]# /usr/bin/rrdtool info /var/www/html/rra/XX_ds_639.rrd | tail
rra[18].cdp_prep[1].value = NaN
rra[18].cdp_prep[1].unknown_datapoints = 5
rra[19].cf = "LAST"
rra[19].rows = 797
rra[19].pdp_per_row = 288
rra[19].xff = 5.0000000000e-01
rra[19].cdp_prep[0].value = NaN
rra[19].cdp_prep[0].unknown_datapoints = 11
rra[19].cdp_prep[1].value = NaN
rra[19].cdp_prep[1].unknown_datapoints = 11
[root@localhost rra]# /usr/bin/rrdtool info /var/www/html/rra/XX_ds_640.rrd | tail
rra[18].cdp_prep[1].value = NaN
rra[18].cdp_prep[1].unknown_datapoints = 0
rra[19].cf = "LAST"
rra[19].rows = 797
rra[19].pdp_per_row = 288
rra[19].xff = 5.0000000000e-01
rra[19].cdp_prep[0].value = NaN
rra[19].cdp_prep[0].unknown_datapoints = 12
rra[19].cdp_prep[1].value = NaN
rra[19].cdp_prep[1].unknown_datapoints = 12
[root@localhost rra]# /usr/bin/rrdtool info /var/www/html/rra/XX_ds_641.rrd | tail
rra[18].cdp_prep[1].value = NaN
rra[18].cdp_prep[1].unknown_datapoints = 0
rra[19].cf = "LAST"
rra[19].rows = 797
rra[19].pdp_per_row = 288
rra[19].xff = 5.0000000000e-01
rra[19].cdp_prep[0].value = NaN
rra[19].cdp_prep[0].unknown_datapoints = 12
rra[19].cdp_prep[1].value = NaN
rra[19].cdp_prep[1].unknown_datapoints = 12

[gandalf]

Unfortunately, you've chopped off the important part of the output
R.