Graph Stopped graphing, but realtime plugin works

[jkukis]

I have a problem I've never seen before, I made up a template for one device we have that measures different power levels. The template was working, graphing beautifully, then stopped. The odd part is if I click the 'Realtime Plugin' icon next to the graph, the realtime graphs graph without any problem.

The cacti logs show 'Result from SNMP not valid', yet if I do a snmpwalk from the same box that cacti runs on I get a valid output below:

Code: Select all

snmpwalk -v2c -c <removed> .1.3.6.1.4.1.3815.1.2.2.1.1.2.1.1.2.6
SNMPv2-SMI::enterprises.3815.1.2.2.1.1.2.1.1.2.6 = STRING: "1218514.875000"

If I do a data source debug:

Code: Select all

/bin/rrdtool create \
/var/www/html/cacti/rra/babelx2_cc_bb_pow_p1_19037.rrd \
--step 300  \
DS:bb_pow_p1:GAUGE:600:0:1000000000 \
RRA:AVERAGE:0.5:1:500 \
RRA:AVERAGE:0.5:1:600 \
RRA:AVERAGE:0.5:6:700 \
RRA:AVERAGE:0.5:24:775 \
RRA:AVERAGE:0.5:288:797 \
RRA:MIN:0.5:1:600 \
RRA:MIN:0.5:6:700 \
RRA:MIN:0.5:24:775 \
RRA:MIN:0.5:288:797 \
RRA:MAX:0.5:1:500 \
RRA:MAX:0.5:1:600 \
RRA:MAX:0.5:6:700 \
RRA:MAX:0.5:24:775 \
RRA:MAX:0.5:288:797 \
RRA:LAST:0.5:1:600 \
RRA:LAST:0.5:6:700 \
RRA:LAST:0.5:24:775 \
RRA:LAST:0.5:288:797 \

and a graph debug looks good:

Code: Select all

/bin/rrdtool graph - \
--imgformat=PNG \
--start=-86400 \
--end=-300 \
--title="emusl0d - Phase Power" \
--base=1000 \
--height=120 \
--width=600 \
--alt-autoscale-max \
--lower-limit=0 \
--vertical-label="Watts" \
--slope-mode \
--font TITLE:10: \
--font AXIS:8: \
--font LEGEND:8: \
--font UNIT:8: \
DEF:a="/var/www/html/cacti/rra/babelx2_cc_bb_pow_p1_19037.rrd":bb_pow_p1:LAST \
DEF:b="/var/www/html/cacti/rra/babelx2_cc_bb_pow_p1_19037.rrd":bb_pow_p1:MIN \
DEF:c="/var/www/html/cacti/rra/babelx2_cc_bb_pow_p1_19037.rrd":bb_pow_p1:MAX \
DEF:d="/var/www/html/cacti/rra/babelx2_cc_bb_pow_p2_19038.rrd":bb_pow_p2:LAST \
DEF:e="/var/www/html/cacti/rra/babelx2_cc_bb_pow_p2_19038.rrd":bb_pow_p2:MIN \
DEF:f="/var/www/html/cacti/rra/babelx2_cc_bb_pow_p2_19038.rrd":bb_pow_p2:MAX \
DEF:g="/var/www/html/cacti/rra/babelx2_cc_bb_pow_p3_19039.rrd":bb_pow_p3:LAST \
DEF:h="/var/www/html/cacti/rra/babelx2_cc_bb_pow_p3_19039.rrd":bb_pow_p3:MIN \
DEF:i="/var/www/html/cacti/rra/babelx2_cc_bb_pow_p3_19039.rrd":bb_pow_p3:MAX \
DEF:j="/var/www/html/cacti/rra/babelx2_cc_bb_ppow_p1_19036.rrd":bb_ppow_p1:LAST \
DEF:ba="/var/www/html/cacti/rra/babelx2_cc_bb_esum_19035.rrd":bb_esum:MIN \
DEF:bb="/var/www/html/cacti/rra/babelx2_cc_bb_esum_19035.rrd":bb_esum:MAX \
DEF:bc="/var/www/html/cacti/rra/babelx2_cc_bb_pow_tot_19040.rrd":bb_pow_tot:LAST \
DEF:bd="/var/www/html/cacti/rra/babelx2_cc_bb_pow_tot_19040.rrd":bb_pow_tot:MIN \
DEF:be="/var/www/html/cacti/rra/babelx2_cc_bb_pow_tot_19040.rrd":bb_pow_tot:MAX \
LINE1:a#FF0000FF:"Phase A"  \
GPRINT:a:LAST:"          Now\:%8.2lf %s"  \
GPRINT:c:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:c:MAX:"Maximum\:%8.2lf %s\n"  \
LINE1:d#0000FFFF:"Phase B"  \
GPRINT:d:LAST:"          Now\:%8.2lf %s"  \
GPRINT:f:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:f:MAX:"Maximum\:%8.2lf %s\n"  \
LINE1:g#00BD27FF:"Phase C"  \
GPRINT:g:LAST:"          Now\:%8.2lf %s"  \
GPRINT:i:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:i:MAX:"Maximum\:%8.2lf %s\n"  \
LINE1:j#750F7DFF:"Peak Demand Power"  \
GPRINT:j:LAST:"Now\:%8.2lf %s"  \
GPRINT:bb:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:bb:MAX:"Maximum\:%8.2lf %s\n"  \
LINE1:bc#FFAB00FF:"Total"  \
GPRINT:bc:LAST:"            Now\:%8.2lf %s"  \
GPRINT:be:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:be:MAX:"Maximum\:%8.2lf %s\n" 

Does anyone have any ideas or troubleshooting I could try? Is there a limit to decimal places that Cacti can handle, with a value like '1218514.875000' I am getting, and if so is there a way to limit that?
I tried creating the device again, applying the templates from scratch and get the same results.

[gandalf]

Please follow debugging steps as of 2nd link of my sig. Pay attention to the "rrdtool update" step and post findings
Reinhard

[jkukis]

I finally got time to look at this again and figured out the issue, although I'm not sure why it caused it to break completely. One of my OID's for a data source got entered in incorrectly(the first one cacti tried to poll), when I snmpwalked it manually it came back with 'NULL' because it was an invalid OID.

After cacti failed on that first OID it reported invalid results for all the following SNMP OID's it was trying, even though the rest of them were valid OID's. I fixed the first incorrect OID, reran the poller and it worked without any issues.

Any ideas why this caused the whole thing to break?