cached memory not graphing

[kamina]

I have a RHEL 4 host with 32GB of memory, and it's being graphed using the UCD/NET template. Everything seems fine apart from the cached memory (granted, the maximum free memory I've had till now is 2GB).

I was looking on the host yesterday, and there was over 12GB cached memory, cacti was showing NAN. Now Cacti is showing 0.0 cached, but the host shows over 14GB:

Code: Select all

$ free -m
             total       used       free     shared    buffers     cached
Mem:         31984      31957         27          0         76      14766
-/+ buffers/cache:      17114      14870
Swap:        12001         87      11914

Is this a limitation with 32bit counters, as in they can't show memory over some limit?

edit:

No errors in the log

edit2:

This is the debug for the data source:

Code: Select all

Data Source Debug

/usr/bin/rrdtool create \
/var/www/cacti/rra/goa_dw_database_mem_cache_1140.rrd \
--step 300  \
DS:mem_cache:GAUGE:600:0:10000000 \
RRA:AVERAGE:0.5:1:600 \
RRA:AVERAGE:0.5:6:700 \
RRA:AVERAGE:0.5:24:775 \
RRA:AVERAGE:0.5:288:797 \
RRA:MIN:0.5:1:600 \
RRA:MIN:0.5:6:700 \
RRA:MIN:0.5:24:775 \
RRA:MIN:0.5:288:797 \
RRA:MAX:0.5:1:600 \
RRA:MAX:0.5:6:700 \
RRA:MAX:0.5:24:775 \
RRA:MAX:0.5:288:797 \
RRA:LAST:0.5:1:600 \
RRA:LAST:0.5:6:700 \
RRA:LAST:0.5:24:775 \
RRA:LAST:0.5:288:797 \

[fmangeant]

Hi

can you check this please :

Code: Select all

$ rpm -qa | grep net-snmp
$ snmpwalk -v 1 -c yourcommunity xx.xx.xx.xx memory
$ rrdtool info /var/www/cacti/rra/goa_dw_database_mem_cache_1140.rrd

[kamina]

On the database machine:

(installed yesterday so should be up to date...)

Code: Select all

$ rpm -q net-snmp
net-snmp-5.1.2-11.EL4.10

On cacti:

Code: Select all

# rpm -q net-snmp
net-snmp-5.1.2-11.EL4.7



# rrdtool info /var/www/cacti/rra/goa_dw_database_mem_cache_1140.rrd 
filename = "/var/www/cacti/rra/goa_dw_database_mem_cache_1140.rrd"
rrd_version = "0003"
step = 300
last_update = 1178272295
ds[mem_cache].type = "GAUGE"
ds[mem_cache].minimal_heartbeat = 600
ds[mem_cache].min = 0.0000000000e+00
ds[mem_cache].max = 1.0000000000e+07
ds[mem_cache].last_ds = "UNKN"
ds[mem_cache].value = NaN
ds[mem_cache].unknown_sec = 95
rra[0].cf = "AVERAGE"
rra[0].rows = 600
rra[0].pdp_per_row = 1
rra[0].xff = 5.0000000000e-01
rra[0].cdp_prep[0].value = NaN
rra[0].cdp_prep[0].unknown_datapoints = 0
rra[1].cf = "AVERAGE"
rra[1].rows = 700
rra[1].pdp_per_row = 6
rra[1].xff = 5.0000000000e-01
rra[1].cdp_prep[0].value = NaN
rra[1].cdp_prep[0].unknown_datapoints = 4
rra[2].cf = "AVERAGE"
rra[2].rows = 775
rra[2].pdp_per_row = 24
rra[2].xff = 5.0000000000e-01
rra[2].cdp_prep[0].value = NaN
rra[2].cdp_prep[0].unknown_datapoints = 22
rra[3].cf = "AVERAGE"
rra[3].rows = 797
rra[3].pdp_per_row = 288
rra[3].xff = 5.0000000000e-01
rra[3].cdp_prep[0].value = NaN
rra[3].cdp_prep[0].unknown_datapoints = 118
rra[4].cf = "MIN"
rra[4].rows = 600
rra[4].pdp_per_row = 1
rra[4].xff = 5.0000000000e-01
rra[4].cdp_prep[0].value = NaN
rra[4].cdp_prep[0].unknown_datapoints = 0
rra[5].cf = "MIN"
rra[5].rows = 700
rra[5].pdp_per_row = 6
rra[5].xff = 5.0000000000e-01
rra[5].cdp_prep[0].value = NaN
rra[5].cdp_prep[0].unknown_datapoints = 4
rra[6].cf = "MIN"
rra[6].rows = 775
rra[6].pdp_per_row = 24
rra[6].xff = 5.0000000000e-01
rra[6].cdp_prep[0].value = NaN
rra[6].cdp_prep[0].unknown_datapoints = 22
rra[7].cf = "MIN"
rra[7].rows = 797
rra[7].pdp_per_row = 288
rra[7].xff = 5.0000000000e-01
rra[7].cdp_prep[0].value = NaN
rra[7].cdp_prep[0].unknown_datapoints = 118
rra[8].cf = "MAX"
rra[8].rows = 600
rra[8].pdp_per_row = 1
rra[8].xff = 5.0000000000e-01
rra[8].cdp_prep[0].value = NaN
rra[8].cdp_prep[0].unknown_datapoints = 0
rra[9].cf = "MAX"
rra[9].rows = 700
rra[9].pdp_per_row = 6
rra[9].xff = 5.0000000000e-01
rra[9].cdp_prep[0].value = NaN
rra[9].cdp_prep[0].unknown_datapoints = 4
rra[10].cf = "MAX"
rra[10].rows = 775
rra[10].pdp_per_row = 24
rra[10].xff = 5.0000000000e-01
rra[10].cdp_prep[0].value = NaN
rra[10].cdp_prep[0].unknown_datapoints = 22
rra[11].cf = "MAX"
rra[11].rows = 797
rra[11].pdp_per_row = 288
rra[11].xff = 5.0000000000e-01
rra[11].cdp_prep[0].value = NaN
rra[11].cdp_prep[0].unknown_datapoints = 118
rra[12].cf = "LAST"
rra[12].rows = 600
rra[12].pdp_per_row = 1
rra[12].xff = 5.0000000000e-01
rra[12].cdp_prep[0].value = NaN
rra[12].cdp_prep[0].unknown_datapoints = 0
rra[13].cf = "LAST"
rra[13].rows = 700
rra[13].pdp_per_row = 6
rra[13].xff = 5.0000000000e-01
rra[13].cdp_prep[0].value = NaN
rra[13].cdp_prep[0].unknown_datapoints = 4
rra[14].cf = "LAST"
rra[14].rows = 775
rra[14].pdp_per_row = 24
rra[14].xff = 5.0000000000e-01
rra[14].cdp_prep[0].value = NaN
rra[14].cdp_prep[0].unknown_datapoints = 22
rra[15].cf = "LAST"
rra[15].rows = 797
rra[15].pdp_per_row = 288
rra[15].xff = 5.0000000000e-01
rra[15].cdp_prep[0].value = NaN
rra[15].cdp_prep[0].unknown_datapoints = 118

Now for the snmpwalk, I was unsure what OID to use. I checked that the template being used is UCD/NET - Memory - Cache so I just grabbed the OID mentioned:

Code: Select all

# snmpwalk -v 2c -c xxxxx goa .1.3.6.1.4.1.2021.4.15.0
UCD-SNMP-MIB::memCached.0 = INTEGER: 14920316

edit:

Code: Select all

# snmpwalk -v 2c -c xxxxxx goa .1.3.6.1.4.1.2021.4 memory
UCD-SNMP-MIB::memIndex.0 = INTEGER: 0
UCD-SNMP-MIB::memErrorName.0 = STRING: swap
UCD-SNMP-MIB::memTotalSwap.0 = INTEGER: 12289716
UCD-SNMP-MIB::memAvailSwap.0 = INTEGER: 12198900
UCD-SNMP-MIB::memTotalReal.0 = INTEGER: 32752184
UCD-SNMP-MIB::memAvailReal.0 = INTEGER: 143360
UCD-SNMP-MIB::memTotalFree.0 = INTEGER: 12342260
UCD-SNMP-MIB::memMinimumSwap.0 = INTEGER: 16000
UCD-SNMP-MIB::memShared.0 = INTEGER: 0
UCD-SNMP-MIB::memBuffer.0 = INTEGER: 93400
UCD-SNMP-MIB::memCached.0 = INTEGER: 14982180
UCD-SNMP-MIB::memSwapError.0 = INTEGER: 0
UCD-SNMP-MIB::memSwapErrorMsg.0 = STRING: 

edit2:

Does this from the rrdinfo mean the max value is set at 10GB?

Code: Select all

ds[mem_cache].max = 1.0000000000e+07 

[fmangeant]

edit2:

Does this from the rrdinfo mean the max value is set at 10GB?

Code: Select all

ds[mem_cache].max = 1.0000000000e+07 

Yes

You can modify the max value using this command :

Code: Select all

$ rrdtool tune /var/www/cacti/rra/goa_dw_database_mem_cache_1140.rrd -a mem_cache:20000000

for a maximum of 20 Gb.

Edit : you should also modify the data template...

[kamina]

Great, thanks a lot. Is it safe to modify it to 40GB, then I will have headroom to graph all free memory after a reboot?

[gandalf]

Does this from the rrdinfo mean the max value is set at 10GB?

Code: Select all

ds[mem_cache].max = 1.0000000000e+07 

Good find!
Reinhard

[kamina]

Thanks, it's working perfectly now