Specific graph off by factor of 100x

[Johnny Lingo]

I'm redoing the Cisco ISR 4000 series Memory Template to differentiate between processor & platform memory. I began by graphing them separately. Fairly straightforward, but the one gotcha is OIDs for Platform memory are returned in KB (not Bytes), so I use the CDEF "multiply by 1024" function to make it more consistent. This is working nicely.

isr4000_processor_memory.png (72.75 KiB) Viewed 786 times

isr4000_platform_memory.png (85.39 KiB) Viewed 786 times

But when I take those graphs and combine them to a single, "Platform Memory Used" shows up 100X higher than it should

isr4000_combined_memory.png (112.61 KiB) Viewed 786 times

Have double-checked the graph settings and "Multiply by 1024" is enabled on the Platform ones. Yet it displays 300-400 GB in use when it should be 3-4 GB.

Any ideas how to troubleshoot this one? Very stumped here.

[netniV]

If you click on the cog next to a graph, what do the data values show?

[Johnny Lingo]

The data values match what's showing up in the graph.

I've run the data sources via CLI SNMPGets and can confirm they're accurate.

Code: Select all

# Check Platform Used
$ snmpwalk -c public -v 2c isr4351 .1.3.6.1.4.1.9.9.109.1.1.1.1.12.7
SNMPv2-SMI::enterprises.9.9.109.1.1.1.1.12.7 = Gauge32: 3397312

# Check Platform Free
$ snmpwalk -c public -v 2c isr4351 .1.3.6.1.4.1.9.9.109.1.1.1.1.13.7
SNMPv2-SMI::enterprises.9.9.109.1.1.1.1.13.7 = Gauge32: 481856

[Johnny Lingo]

Another funny thing - if I take the exact same template applied to a Cisco ASR1001x (similar router, same SNMP OIDs) the graph values are correct.

This makes me think it might be some type of auto-scaling feature causing the 100X bump.

[netniV]

Did you mean you apply this to another device type? As it reads like you're applying it to the same type unless I'm missing something there

What I would check is the graph creation code for both graphs too

[Johnny Lingo]

Did you mean you apply this to another device type? As it reads like you're applying it to the same type unless I'm missing something there

Different device templates (Cisco ISR 4000 vs. Cisco ASR 1000) but same data template. Both use the exact same SNMP OIDs.

What I would check is the graph creation code for both graphs too

I had looked at this before but after a second look, do see something odd. There should be 5 data sources, but instead there's only 4. The one with the short name "platmemused" is missing.

Code: Select all

RRDtool Command:
/usr/bin/rrdtool graph - \
--imgformat=PNG \
--start='1517246160' \
--end='1517332291' \
--pango-markup  \
--title='isr4351 - Memory' \
--vertical-label='Memory (Bytes)' \
--slope-mode \
--base=1024 \
--height=120 \
--width=500 \
--tabwidth '30' \
--alt-autoscale-max \
--lower-limit='0' \
COMMENT:"From 2018/01/29 09\:16\:00 To 2018/01/30 09\:11\:31\c" \
COMMENT:"  \n" \
--slope-mode \
--watermark 'Generated by Cacti®' \
DEF:a='/home/cacti-1.1.28/rra/isr4351_procmemused_7202.rrd':'procMemUsed':AVERAGE \
DEF:b='/home/cacti-1.1.28/rra/isr4351_procmemfree_7201.rrd':'procMemFree':AVERAGE \
DEF:c='/home/cacti-1.1.28/rra/isr4351_platmemfree_7200.rrd':'platMemFree':AVERAGE \
DEF:d='/home/cacti-1.1.28/rra/isr4351_platmemcommit_7199.rrd':'platMemCommit':AVERAGE \
CDEF:cdefba='a,1024,*' \
CDEF:cdefbf='c,1024,*' \
CDEF:cdefca='d,1024,*' \
LINE2:a#990012B2:'Proc Used     '  \
GPRINT:a:LAST:'Last\:%8.2lf %s'  \
GPRINT:a:MIN:'Min\:%8.2lf %s'  \
GPRINT:a:AVERAGE:'Avg\:%8.2lf %s'  \
GPRINT:a:MAX:'Max\:%8.2lf %s\n'  \
LINE2:b#157419B2:'Proc Free     '  \
GPRINT:b:LAST:'Last\:%8.2lf %s'  \
GPRINT:b:MIN:'Min\:%8.2lf %s'  \
GPRINT:b:AVERAGE:'Avg\:%8.2lf %s'  \
GPRINT:b:MAX:'Max\:%8.2lf %s\n'  \
LINE2:cdefba#E67451B2:'Plat Used     '  \
GPRINT:cdefba:LAST:'Last\:%8.2lf %s'  \
GPRINT:cdefba:MIN:'Min\:%8.2lf %s'  \
GPRINT:cdefba:AVERAGE:'Avg\:%8.2lf %s'  \
GPRINT:cdefba:MAX:'Max\:%8.2lf %s\n'  \
LINE2:cdefbf#9CB071B2:'Plat Free     '  \
GPRINT:cdefbf:LAST:'Last\:%8.2lf %s'  \
GPRINT:cdefbf:MIN:'Min\:%8.2lf %s'  \
GPRINT:cdefbf:AVERAGE:'Avg\:%8.2lf %s'  \
GPRINT:cdefbf:MAX:'Max\:%8.2lf %s\n'  \
LINE2:cdefca#800080B2:'Plat Committed'  \
GPRINT:cdefca:LAST:'Last\:%8.2lf %s'  \
GPRINT:cdefca:MIN:'Min\:%8.2lf %s'  \
GPRINT:cdefca:AVERAGE:'Avg\:%8.2lf %s'  \
GPRINT:cdefca:MAX:'Max\:%8.2lf %s\n' 
RRDtool Says:
OK

Also in the graph, the Description is missing for PlatMemUsed: