Physical memory doesn't show up correctly

[relkai]

I'm using cacti in our company to monitor about 20 servers at the moment.
I've got several linux machines with more than 2GB physical memory, but when using the data query "SNMP Get Mounted Partitions" the graph never shows more than 2GB "real memory".
I know it's not exactly what this query is made for, but when using "ucd/net Memory usage" the graph only shows the free space, but I want a full overview.
Has anybody an idea how to fix it?

[TomekN]

Use my memory template. Find it in Scripts.

[relkai]

Very nice template - it looks awesome and is exactly what I was searching for!

But I've got a problem importing the template into cacti. I only get the following error message:

Code: Select all

Error: XML: Hash version does not exist.

I'm using cacti version v0.8.6c. I know, it's by far not the newest version and this is most likely my problem, but this is the only version I'm getting in Debian (sarge) by typing "apt-get install cacti".

Is there a way to import your script into my old version of cacti?

[TomekN]

Is there a way to import your script into my old version of cacti?

I'm not sure what is that hash error about..
Percheps one of developers will help us here.

[relkai]

I just tried to import the template on my gentoo box (cacti v0.8.6g) and here I'm getting a different error:

Code: Select all

Error: XML: Generated with a newer version of Cacti.

So the version is definitely my problem.

But the problem on Debian still exists - how could I modify the newer template for my old version of cacti?

[gandalf]

Very nice template - it looks awesome and is exactly what I was searching for!

But I've got a problem importing the template into cacti. I only get the following error message:

Code: Select all

Error: XML: Hash version does not exist.

I'm using cacti version v0.8.6c. I know, it's by far not the newest version and this is most likely my problem, but this is the only version I'm getting in Debian (sarge) by typing "apt-get install cacti".

Is there a way to import your script into my old version of cacti?

You will need at least the same version of cacti the templates were exported with
Reinhard

[relkai]

Thanks Reinhard!
I just updated my home gentoo box (768MB RAM) to the newest cacti version for testing purposes and were able to successfully import Tomek's nice graph template.

But now I'm a bit messed up. I created the following graphs just to be able to compare them:
- Linux - Memory Usage
- ucd/net - Memory Usage
- ucd/net - Memory Usage (Netventure)

Since I imported the custom graph template from Tomek I'm getting the following error messages in my cacti.log:

Code: Select all

03/09/2006 03:15:02 PM - CMDPHP: Poller[0] Host[1] DS[24] WARNING: Result from SNMP not valid.  Partial Result:
03/09/2006 03:15:03 PM - CMDPHP: Poller[0] Host[1] DS[26] WARNING: Result from SNMP not valid.  Partial Result:
03/09/2006 03:15:03 PM - CMDPHP: Poller[0] Host[1] DS[25] WARNING: Result from SNMP not valid.  Partial Result:
03/09/2006 03:15:03 PM - SYSTEM STATS: Time:1.0860 Method:cmd.php Processes:1 Threads:N/A Hosts:2 HostsPerProcess:2 DataSources:20 RRDsProcessed:17

The data sources 24,25,26 are the following which are used by the custom graph template:
24: Memory - Total
25: Swap - Free
26: Swap - Total

Therefore the graph output isn't correct (negative values).

But the really confusing part is that the three graphs are showing different values (aspecially for "Memory free") at the same time:

Linux - Memory Usage


ucd/net - Memory Usage


ucd/net - Memory Usage (Netventure)


To make things even more confusing, here's my "top" output regarding the memory usage at the same time the graphs were created:

Code: Select all

Mem:    774920k total,   764668k used,    10252k free,    40236k buffers
Swap:  2008116k total,      228k used,  2007888k free,   476452k cached

I'm a real SNMP noob so I've got no idea how to fix it or how to do a manual snmpwalk.

Can anybody help?

relkai

[TomekN]

My template calls standard, buildin ucd values (.4 branch):

Code: Select all

[root@verso rra]# snmpwalk -v 1 -c public localhost .1.3.6.1.4.1.2021.4
UCD-SNMP-MIB::memIndex.0 = INTEGER: 0
UCD-SNMP-MIB::memErrorName.0 = STRING: swap
UCD-SNMP-MIB::memTotalSwap.0 = INTEGER: 1004020
UCD-SNMP-MIB::memAvailSwap.0 = INTEGER: 1000204
UCD-SNMP-MIB::memTotalReal.0 = INTEGER: 385948
UCD-SNMP-MIB::memAvailReal.0 = INTEGER: 101124
UCD-SNMP-MIB::memTotalFree.0 = INTEGER: 1101328
UCD-SNMP-MIB::memMinimumSwap.0 = INTEGER: 16000
UCD-SNMP-MIB::memShared.0 = INTEGER: 0
UCD-SNMP-MIB::memBuffer.0 = INTEGER: 82896
UCD-SNMP-MIB::memCached.0 = INTEGER: 46976
UCD-SNMP-MIB::memSwapError.0 = INTEGER: 0
UCD-SNMP-MIB::memSwapErrorMsg.0 = STRING:

[root@verso rra]# snmpwalk -v 1 -c public localhost .1.3.6.1.4.1.2021.4 -On
.1.3.6.1.4.1.2021.4.1.0 = INTEGER: 0
.1.3.6.1.4.1.2021.4.2.0 = STRING: swap
.1.3.6.1.4.1.2021.4.3.0 = INTEGER: 1004020
.1.3.6.1.4.1.2021.4.4.0 = INTEGER: 1000204
.1.3.6.1.4.1.2021.4.5.0 = INTEGER: 385948
.1.3.6.1.4.1.2021.4.6.0 = INTEGER: 101124
.1.3.6.1.4.1.2021.4.11.0 = INTEGER: 1101328
.1.3.6.1.4.1.2021.4.12.0 = INTEGER: 16000
.1.3.6.1.4.1.2021.4.13.0 = INTEGER: 0
.1.3.6.1.4.1.2021.4.14.0 = INTEGER: 82896
.1.3.6.1.4.1.2021.4.15.0 = INTEGER: 46976
.1.3.6.1.4.1.2021.4.100.0 = INTEGER: 0
.1.3.6.1.4.1.2021.4.101.0 = STRING:

These are the only values I use (not all of them)
Check your snmpd response.

[relkai]

Thank you Tomek.

Here's my snmpwalk output:

Code: Select all

cheffe rra # snmpwalk -v 1 -c public localhost .1.3.6.1.4.1.2021.4
UCD-SNMP-MIB::memIndex.0 = INTEGER: 0
UCD-SNMP-MIB::memErrorName.0 = STRING: swap
UCD-SNMP-MIB::memTotalSwap.0 = INTEGER: 2008116
UCD-SNMP-MIB::memAvailSwap.0 = INTEGER: 2007888
UCD-SNMP-MIB::memTotalReal.0 = INTEGER: 774920
UCD-SNMP-MIB::memAvailReal.0 = INTEGER: 28160
UCD-SNMP-MIB::memTotalFree.0 = INTEGER: 2036048
UCD-SNMP-MIB::memMinimumSwap.0 = INTEGER: 16000
UCD-SNMP-MIB::memShared.0 = INTEGER: 0
UCD-SNMP-MIB::memBuffer.0 = INTEGER: 40644
UCD-SNMP-MIB::memCached.0 = INTEGER: 459128
UCD-SNMP-MIB::memSwapError.0 = INTEGER: 0
UCD-SNMP-MIB::memSwapErrorMsg.0 = STRING:

cheffe rra # snmpwalk -v 1 -c public localhost .1.3.6.1.4.1.2021.4 -On
.1.3.6.1.4.1.2021.4.1.0 = INTEGER: 0
.1.3.6.1.4.1.2021.4.2.0 = STRING: swap
.1.3.6.1.4.1.2021.4.3.0 = INTEGER: 2008116
.1.3.6.1.4.1.2021.4.4.0 = INTEGER: 2007888
.1.3.6.1.4.1.2021.4.5.0 = INTEGER: 774920
.1.3.6.1.4.1.2021.4.6.0 = INTEGER: 27512
.1.3.6.1.4.1.2021.4.11.0 = INTEGER: 2035400
.1.3.6.1.4.1.2021.4.12.0 = INTEGER: 16000
.1.3.6.1.4.1.2021.4.13.0 = INTEGER: 0
.1.3.6.1.4.1.2021.4.14.0 = INTEGER: 40648
.1.3.6.1.4.1.2021.4.15.0 = INTEGER: 460192
.1.3.6.1.4.1.2021.4.100.0 = INTEGER: 0
.1.3.6.1.4.1.2021.4.101.0 = STRING:

The output seems to be correct or did I oversee something?

[TomekN]

Please paste your RRDTool Command (Graph Management, Turn On Graph Debug Mode).
Here is mine:

Code: Select all

/usr/bin/rrdtool graph - \
--imgformat=PNG \
--start=-86400 \
--end=-300 \
--title="verso - Memory Usage" \
--base=1024 \
--height=120 \
--width=500 \
--alt-autoscale-max \
--lower-limit=0 \
--vertical-label="bytes" \
--slope-mode \
DEF:a="/home/httpd/root/cacti/rra/verso_mem_buffers_63.rrd":mem_buffers:AVERAGE \
DEF:b="/home/httpd/root/cacti/rra/verso_mem_cache_64.rrd":mem_cache:AVERAGE \
DEF:c="/home/httpd/root/cacti/rra/verso_mem_free_65.rrd":mem_free:AVERAGE \
DEF:d="/home/httpd/root/cacti/rra/verso_mem_total_66.rrd":mem_total:AVERAGE \
DEF:e="/home/httpd/root/cacti/rra/verso_swap_free_67.rrd":swap_free:AVERAGE \
DEF:f="/home/httpd/root/cacti/rra/verso_swap_total_68.rrd":swap_total:AVERAGE \
CDEF:cdefa=d,c,-,b,-,a,-,1024,* \
CDEF:cdeff=a,1024,* \
CDEF:cdefba=b,1024,* \
CDEF:cdefbf=c,1024,* \
CDEF:cdefca=d,0,* \
CDEF:cdefcb=d,1024,* \
CDEF:cdefcd=f,e,-,1024,* \
CDEF:cdefci=e,0,* \
CDEF:cdefcj=e,1024,* \
CDEF:cdefdd=f,0,* \
CDEF:cdefde=f,1024,* \
AREA:cdefa#FFF200:"Memory Used"  \
GPRINT:cdefa:LAST:"   Current\:%8.2lf %s"  \
GPRINT:cdefa:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:cdefa:MIN:"Min\:%8.2lf %s"  \
GPRINT:cdefa:MAX:"Max\:%8.2lf %s\n"  \
AREA:cdeff#6EA100:"Memory Buffers":STACK \
GPRINT:cdeff:LAST:"Current\:%8.2lf %s"  \
GPRINT:cdeff:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:cdeff:MIN:"Min\:%8.2lf %s"  \
GPRINT:cdeff:MAX:"Max\:%8.2lf %s\n"  \
AREA:cdefba#00CF00:"Memory Cached":STACK \
GPRINT:cdefba:LAST:" Current\:%8.2lf %s"  \
GPRINT:cdefba:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:cdefba:MIN:"Min\:%8.2lf %s"  \
GPRINT:cdefba:MAX:"Max\:%8.2lf %s\n"  \
AREA:cdefbf#00FF00:"Memory Free":STACK \
GPRINT:cdefbf:LAST:"   Current\:%8.2lf %s"  \
GPRINT:cdefbf:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:cdefbf:MIN:"Min\:%8.2lf %s"  \
GPRINT:cdefbf:MAX:"Max\:%8.2lf %s\n"  \
AREA:cdefca#000000:"Memory Total":STACK \
GPRINT:cdefcb:LAST:"  Current\:%8.2lf %s\n"  \
COMMENT:"\n"  \
AREA:cdefcd#FF0000:"Swap Used":STACK \
GPRINT:cdefcd:LAST:"     Current\:%8.2lf %s"  \
GPRINT:cdefcd:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:cdefcd:MIN:"Min\:%8.2lf %s"  \
GPRINT:cdefcd:MAX:"Max\:%8.2lf %s\n"  \
LINE1:cdefci#FFFFFF:"Swap Free"  \
GPRINT:cdefcj:LAST:"     Current\:%8.2lf %s"  \
GPRINT:cdefcj:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:cdefcj:MIN:"Min\:%8.2lf %s"  \
GPRINT:cdefcj:MAX:"Max\:%8.2lf %s\n"  \
LINE1:cdefdd#000000:"Swap Total"  \
GPRINT:cdefde:LAST:"    Current\:%8.2lf %s\n"

[relkai]

Here it is:

Code: Select all

/usr/bin/rrdtool graph - \
--imgformat=PNG \
--start=-86400 \
--end=-300 \
--title="cheffe - Memory Usage" \
--base=1024 \
--height=120 \
--width=500 \
--alt-autoscale-max \
--lower-limit=0 \
--vertical-label="bytes" \
--slope-mode \
DEF:a="/var/www/localhost/htdocs/cacti/rra/cheffe_mem_buffers_21.rrd":mem_buffers:AVERAGE \
DEF:b="/var/www/localhost/htdocs/cacti/rra/cheffe_mem_cache_22.rrd":mem_cache:AVERAGE \
DEF:c="/var/www/localhost/htdocs/cacti/rra/cheffe_mem_free_23.rrd":mem_free:AVERAGE \
DEF:d="/var/www/localhost/htdocs/cacti/rra/cheffe_mem_total_24.rrd":mem_total:AVERAGE \
DEF:e="/var/www/localhost/htdocs/cacti/rra/cheffe_swap_free_25.rrd":swap_free:AVERAGE \
DEF:f="/var/www/localhost/htdocs/cacti/rra/cheffe_swap_total_26.rrd":swap_total:AVERAGE \
CDEF:cdefa=d,c,-,b,-,a,-,1024,* \
CDEF:cdeff=a,1024,* \
CDEF:cdefba=b,1024,* \
CDEF:cdefbf=c,1024,* \
CDEF:cdefca=d,0,* \
CDEF:cdefcb=d,1024,* \
CDEF:cdefcd=f,e,-,1024,* \
CDEF:cdefci=e,0,* \
CDEF:cdefcj=e,1024,* \
CDEF:cdefdd=f,0,* \
CDEF:cdefde=f,1024,* \
AREA:cdefa#FFF200:"Memory Used"  \
GPRINT:cdefa:LAST:"   Current\:%8.2lf %s"  \
GPRINT:cdefa:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:cdefa:MIN:"Min\:%8.2lf %s"  \
GPRINT:cdefa:MAX:"Max\:%8.2lf %s\n"  \
AREA:cdeff#6EA100:"Memory Buffers":STACK \
GPRINT:cdeff:LAST:"Current\:%8.2lf %s"  \
GPRINT:cdeff:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:cdeff:MIN:"Min\:%8.2lf %s"  \
GPRINT:cdeff:MAX:"Max\:%8.2lf %s\n"  \
AREA:cdefba#00CF00:"Memory Cached":STACK \
GPRINT:cdefba:LAST:" Current\:%8.2lf %s"  \
GPRINT:cdefba:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:cdefba:MIN:"Min\:%8.2lf %s"  \
GPRINT:cdefba:MAX:"Max\:%8.2lf %s\n"  \
AREA:cdefbf#00FF00:"Memory Free":STACK \
GPRINT:cdefbf:LAST:"   Current\:%8.2lf %s"  \
GPRINT:cdefbf:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:cdefbf:MIN:"Min\:%8.2lf %s"  \
GPRINT:cdefbf:MAX:"Max\:%8.2lf %s\n"  \
AREA:cdefca#000000:"Memory Total":STACK \
GPRINT:cdefcb:LAST:"  Current\:%8.2lf %s\n"  \
COMMENT:"\n"  \
AREA:cdefcd#FF0000:"Swap Used":STACK \
GPRINT:cdefcd:LAST:"     Current\:%8.2lf %s"  \
GPRINT:cdefcd:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:cdefcd:MIN:"Min\:%8.2lf %s"  \
GPRINT:cdefcd:MAX:"Max\:%8.2lf %s\n"  \
LINE1:cdefci#FFFFFF:"Swap Free"  \
GPRINT:cdefcj:LAST:"     Current\:%8.2lf %s"  \
GPRINT:cdefcj:AVERAGE:"Average\:%8.2lf %s"  \
GPRINT:cdefcj:MIN:"Min\:%8.2lf %s"  \
GPRINT:cdefcj:MAX:"Max\:%8.2lf %s\n"  \
LINE1:cdefdd#000000:"Swap Total"  \
GPRINT:cdefde:LAST:"    Current\:%8.2lf %s\n" 

RRDTool Says:

OK

Looks identical to me...

[TomekN]

Your're right. Maybe that will help:
http://forums.cacti.net/about11373.html

[relkai]

Ok, I turned on debug mode for the logfile and here are the warning lines:

Code: Select all

03/09/2006 04:40:02 PM - CMDPHP: Poller[0] Host[1] DS[24] WARNING: Result from SNMP not valid.  Partial Result:
03/09/2006 04:40:02 PM - CMDPHP: Poller[0] Host[1] DS[24] SNMP: v1: 127.0.0.1, dsname: mem_total, oid: , output: U
03/09/2006 04:40:02 PM - CMDPHP: Poller[0] DEBUG: SQL Exec: "insert into poller_output (local_data_id,rrd_name,time,output) values (24,'mem_total','2006-03-0
9 16:40:02','U')"
03/09/2006 04:40:02 PM - CMDPHP: Poller[0] Host[1] DS[26] WARNING: Result from SNMP not valid.  Partial Result:
03/09/2006 04:40:02 PM - CMDPHP: Poller[0] Host[1] DS[26] SNMP: v1: 127.0.0.1, dsname: swap_total, oid: , output: U
03/09/2006 04:40:02 PM - CMDPHP: Poller[0] DEBUG: SQL Exec: "insert into poller_output (local_data_id,rrd_name,time,output) values (26,'swap_total','2006-03-
09 16:40:02','U')"
03/09/2006 04:40:02 PM - CMDPHP: Poller[0] Host[1] DS[25] WARNING: Result from SNMP not valid.  Partial Result:
03/09/2006 04:40:02 PM - CMDPHP: Poller[0] Host[1] DS[25] SNMP: v1: 127.0.0.1, dsname: swap_free, oid: , output: U
03/09/2006 04:40:02 PM - CMDPHP: Poller[0] DEBUG: SQL Exec: "insert into poller_output (local_data_id,rrd_name,time,output) values (25,'swap_free','2006-03-0
9 16:40:02','U')"

I don't understand why it doesn't show any OID for these data sources. The data sources that were queried correctly contain an OID:

Code: Select all

03/09/2006 04:40:02 PM - CMDPHP: Poller[0] Host[1] DS[21] SNMP: v1: 127.0.0.1, dsname: mem_buffers, oid: .1.3.6.1.4.1.2021.4.14.0, output: 38452
03/09/2006 04:40:02 PM - CMDPHP: Poller[0] DEBUG: SQL Exec: "insert into poller_output (local_data_id,rrd_name,time,output) values (21,'mem_buffers','2006-03
-09 16:40:02','38452')"

So my problem seems to be another one - maybe simpler because the OID is just missing...

[relkai]

I got it!

I looked into the corresponding data templates and the OID was indeed missing.
I don't know why this happened - maybe these templates were incompletely created when I tried to import your graph template into my old version of cacti.
I added the OID manually and now everything works perfect.

Thanks a lot, Tomek.
Your help was really great!