odd disk graphs (san volumes)

[giana]

i would like to start monitoring our SAN space but the graphs looks a bit crazy to me. ( /san1 and /san2 )

/san1 Graph displays just displays the free disk space
/san2 Graph displays unknown things, ok it looks like a diskspace graph but this is not a continuous report of the correct data.
it never reports the correct data.

here what cacti should display in the graphs

Code: Select all

# df -h |grep san
/dev/cvfsctl1_san1    3.5T  2.8T  622G  83% /san1
/dev/cvfsctl2_san2     13T  2.4T   11T  19% /san2

here a localy snmpwalk for /san1

Code: Select all

[root@ethbridge ~]# cat  snmpwalk |grep hrStorage |grep '.14 ='
HOST-RESOURCES-MIB::hrStorageIndex.14 = INTEGER: 14
HOST-RESOURCES-MIB::hrStorageType.14 = OID: HOST-RESOURCES-TYPES::hrStorageFixedDisk
HOST-RESOURCES-MIB::hrStorageDescr.14 = STRING: /san1
HOST-RESOURCES-MIB::hrStorageAllocationUnits.14 = INTEGER: 8192 Bytes
HOST-RESOURCES-MIB::hrStorageSize.14 = INTEGER: 457157952
HOST-RESOURCES-MIB::hrStorageUsed.14 = INTEGER: 372840630

and here for /san2

Code: Select all

HOST-RESOURCES-MIB::hrStorageIndex.16 = INTEGER: 16
HOST-RESOURCES-MIB::hrStorageType.16 = OID: HOST-RESOURCES-TYPES::hrStorageFixedDisk
HOST-RESOURCES-MIB::hrStorageDescr.16 = STRING: /san2
HOST-RESOURCES-MIB::hrStorageAllocationUnits.16 = INTEGER: 8192 Bytes
HOST-RESOURCES-MIB::hrStorageSize.16 = INTEGER: 1708706048
HOST-RESOURCES-MIB::hrStorageUsed.16 = INTEGER: 317162913

mountpoint:

Code: Select all

]# mount |grep san
/dev/cvfsctl1_san1 on /san1 type cvfs (rw,sparse=yes)
/dev/cvfsctl2_san2 on /san2 type cvfs (rw,sparse=yes)

im using cacti-0.8.6j-1 on a FC5 i386 box

the monitored system:
Linux 2.6.9-55.0.6.ELsmp #1 SMP Thu Aug 23 11:13:21 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux

Cactilog is emty about that
someone has a clue how to get continuously the correct data ?

[gandalf]

Which Graph/Data Template are you using?
Reinhard

[giana]

Hi Reinhard,

the default templates are untouched

Graph Template is 'Host MIB - Available Disk Space'
Data Templates is 'Host MIB - Hard Drive Space' ucd/net Hard Drive Disk Space

[TheWitness]

Verify, if using spine, that you are using the 0.8.7a in the Announcements forum. Some systems have a problem with Max Values in the data template as well as potentially other problems. Best to run the poller on just that host. Gandolf has more details on how to do this

TheWitness

[giana]

i removed the data-sources and the graphes.
then i have restarted snmpd on the monitored system, to recreate the graphs.
- and again i've the odd graphs -
looks like the graphs could not display more than 1,69TByte Total Data.
i try to upgrade to 0.8.7a. - there is no rpm for FC5 so i will take a minute to
figure out what i need to backup before i unpack a tar file.

i never have heard about spine before - need to understand what it does.
the information site is my frind.
by the way i have run "php poller.php" to get the time.

Code: Select all

11/09/2007 05:59:54 PM - SYSTEM STATS: Time:43.0844 Method:cmd.php Processes:1 Threads:N/A Hosts:50 HostsPerProcess:50 DataSources:900 RRDsProcessed:496

[giana]

Verify, if using spine, that you are using the 0.8.7a in the Announcements forum.

i've set up a FC8 in a VMware. yum installed cacti-0.8.6j-10.fc8 (rpm) for me. a quick setup shows me the DiskGraphs from a RHEL4 system.
i'll try to monitor this volume:

Code: Select all

#mount |grep FAST
/dev/cvfsctl1_san2 on /mnt/san2_FAST type cvfs (rw,sparse=yes)

#ls -al |grep cvf /dev/
lrwxrwxrwx  1 root          root       13 Nov 16 16:38 cvfsctl -> /dev/cvfsctl0
brw-------  1 root          root 252,   0 Nov 16 16:38 cvfsctl0
brw-------  1 root          root 252,   1 Nov 16 16:38 cvfsctl1
brw-r--r--  1 root          root 252,  10 Nov 16 16:38 cvfsctl10
[...]

#snmpwalk -v2c -c public localhost |grep hrStorage |grep .15
HOST-RESOURCES-MIB::hrStorageIndex.15 = INTEGER: 15
HOST-RESOURCES-MIB::hrStorageType.15 = OID: HOST-RESOURCES-TYPES::hrStorageFixedDisk
HOST-RESOURCES-MIB::hrStorageDescr.15 = STRING: /mnt/san2_FAST
HOST-RESOURCES-MIB::hrStorageAllocationUnits.15 = INTEGER: 8192 Bytes
HOST-RESOURCES-MIB::hrStorageSize.15 = INTEGER: 1708706048
HOST-RESOURCES-MIB::hrStorageUsed.15 = INTEGER: 404563017

it is a adic filesystem which need licensing. on the RHEL4 system it displays the right values

Code: Select all

# df -h |grep FAST
/dev/cvfsctl1_san2     13T  3.1T  9.8T  24% /mnt/san2_FAST

the upgrade from cacti-0.8.6j-10.fc8 (rpm) to cacti-0.8.7a (tar) was successful. i dont use spine or cactid.
im not using it because my network query does not take more than 300sec. it takes about 50sec. (php poller.php)
do i need spine or cactid anymore?

Some systems have a problem with Max Values in the data template as well as potentially other problems.

Is there someting to play around with to alter that?

Best to run the poller on just that host. Gandolf has more details on how to do this ;)

yes i still would like to graph my 13TByte Partition. - if i need to run the poller on just that host i will do that.
BTW can someone poke me i a direction how to do that?

giana

[TheWitness]

Ignore my spine comments. This is all about max values in the RRDfile and possibly integer overflow, but offhand it does not look like that is a problem here.

TheWitness

[giana]

is there another handy way to monitor big partitions like mine ? if the value is in GB it its okay. i dont need exactly the free bytes.
do i need a script local on the host ? or is there a way to remove the max value limit for rrd files ?
- giana

[giana]

nobody has a clue?

[TheWitness]

I think that you need to develop a template. This is somewhat fundamental in Cacti. So, if you plan on using it, you should be cutting your teath on templating. If you installation is large, develop it offline on a test box and then migrate the template to your production environment.

So, the reason you don't get a response is likely due to the fact that:

1) This problem has been seen before and fixed
2) The ball is in your court IMHO.

If you do develop your own template. Please publish it.

TheWitness