Host Templates for HP LeftHand Storage System (P4500, DL320)

[jftuga]

I just wanted to mention that I was successful getting this up & running on my P4500 with CMC 8.5.00.0319. Do these work OK with CMC 9?
Eric, thanks for the great template.

-John

[ottow]

John,

I use them sucessfully under CMC 9, all the latest updates applied.

Otto

[eschoeller]

@NilsCant

I am aware that some of the scaling on some of the graphs may be incorrect. I spent a good deal of time making sure the throughput graphs were right. But things change and it's possible they've been broken (or fixed <g>) by something (an update). Your method of double checking the template is certainly sound, so I understand where you are coming from. I can't offer any specific guidance at this time, I've got a myriad of other problems to solve in a short time-line

The good news is that even if the actual values being reported are incorrect, they'll be consistently incorrect all the time .. which means the graph will still present a valid "trend" of what's happening, albeit with distorted values.

Have you compared the graphs in the CMC to the graphs in Cacti?

If you come up with anything please let us know. Hopefully I'll have some more time in the future to dive into this issue.

[sirhopcount]

Hi,

First of all thanks for the templates, this saves me allot of work. I do have a small question about the templates and there destinations (devices/ip's). On the documentation page you specify that "HP LeftHand Cluster” are templates for the cluster and the “HP LeftHand System” are for the system. The system part speaks for itself but when I look at the Graph Template I also see other templates:

HP LeftHand - RAID
HP LeftHand - Volume

I presume that these are also only for the "cluster" and I should make a "device" with the ip address of the VIP ?

Regards,

SirHopcount.

[WiredWorm]

Hi,

I'm very new to Cacti and i'm having a few issues getting the templates for the HP LeftHand SAN to work. I was hoping someone on here might be able to assist.

Graphs are being generated but there's no data on the actual graph itself. Checking in the Cacti.log file I can see some entries like this:

06/02/2011 01:10:00 PM - CMDPHP: Poller[0] WARNING: SNMP Get Timeout for Host:'xx.xx.xx.xxx', and OID:'.1.3.6.1.4.1.9804.3.1.1.2.12.48.1.17.1'
06/02/2011 01:10:00 PM - CMDPHP: Poller[0] Host[3] DS[11] WARNING: Result from SNMP not valid. Partial Result: U

I'm guessing this is the cause of my problem?

Is anyone able to offer any pointers on this and what I might need to do to fix the issue? Our SAN implementation is quite recent so we're running the latest version of the CMC (v9.0.01 Build 0125). All of the storage nodes have also had the latest updates applied.

Thanks

Pete

Edit: I think I found the problem. My SNMP version was set to v1 instead of v2. I diagnosed this using the snmpget command within net-snmp and noticed that it worked fine with v2. I've changed the setting and it looks like it's fixed the issue.

[eschoeller]

@sirhopcount:

Yep, that's right. The cluster VIP should be added to Cacti as an additional device and the "HP LeftHand Cluster" host template should be used ... which will include "HP - LeftHand - Cluster" and "HP - LeftHand - Volumes" data queries, and all of the graphs associated with those data queries. The "HP - LeftHand - Cluster" query includes 8 graphs per cluster, and the "HP - LeftHand - Volume" query includes 6 graphs per volume. Does this make sense?

@WiredWorm:

Glad you figured out the problem. Yes, it's always a good idea to troubleshoot with snmpwalk/get at the commandline.

[mh90291]

I just installed the templates yesterday and so far it seems pretty cool. The one problem I am having is volume usage is showing 9 GIGs used where it should show 900 GIGs used. I correctly shows the provisioned correctly. It is a thin provisioned LUN.

[mh90291]

Related to my post about the used size reporting 100th of the actual, I'm running 9.0.01 build 0163 of the management console. I've now believe the P4500 SNMP is broken or I'm missing something really important.

When I query clusVolumeUsedSpace.48 it returns Counter64: 89506304 kB which no matter what I do to the number comes even close to the actual use of the thin provisioned volume of about 1.2T. The volume itself is 2T and even a query of clusVolumeSize.48 is a little too far off at Counter64: 2147483648 kB (though much closer than the used space query).

It would seem that Cacti is reporting correctly what it's getting.

[eschoeller]

Sorry I haven't been able to reply. It does look like there could be an issue in SanIQ. We are in the process of upgrading from 8.5.00.0319.0 to 9-something. I am going to ask someone on our development team to verify this behavior. If that is the case, we will have a big problem and I'm glad we know about it ahead of time. If you're certain there is an issue I would strongly recommend opening a case with HP about it.

I am curious, is clusVolumeClusterUsedPercent being reported correctly?

[rybal1212]

I am completely new to SNMP, but I absolutely love this project, and cacti now!

I was able to run this:
snmpwalk -m ALL -t 90 -v2c -c 'communityname' ipaddress .1.3.6.1.4.1.9804.3.1.1.2.12.97.1.2
and get a list of all my volumes.

I changed the OID to be .1.3.6.1.4.1.9804.3.1.1.2.12.97.1.37, and got a counter of IO write operations. .1.3.6.1.4.1.9804.3.1.1.2.12.97.1.36 returns the IO Read operations per volume.

Pardon my ignorance, how would I even begin to associate the IO read\write counters with the volume name, and then turn the IO read\write counters into IOPS and graph it, much like how these templates are graphing cluster IOPS?

Many Thanks!
Ryan

[eschoeller]

So it sounds like you're trying to graph per-volume IOPs correct? The template set already does that!!

There are two host templates included. One is 'HP LeftHand System' and the other is 'HP LeftHand Cluster'. My recommendation is to setup each node using the 'HP LeftHand System' template. Then, using the VIP for your cluster, create a device using the 'HP LeftHand Cluster' template.

Within the HP LeftHand Cluster host template there is a Data Query called HP - LeftHand - Volumes. When you're on the 'New Graphs' page for the HP LeftHand Cluster you created, you'll see a list of your volumes. Select the volumes you'd like to graph and then select the graph type. There is a drop-down box with 6 different graphs that you can create per volume: Cache Hits, IOPS, Latency, QDepth, Throughput, Usage.

You could graph volume stats directly from each node, but due to the way LeftHand clustering works you'd end up with the same data N times (where N is the number of nodes in your cluster). It doesn't matter which node is the 'VIP holder' they all share this information globally.

One of these days I'll get around to developing per-initiator statistics, but for our installation this would be somewhat overkill, we have hundreds of initiators. Smaller installs would certainly benefit from this. You'd be able to zero in on which hosts drive up your throughput or latency for instance.

Hope this helps!

[rybal1212]

Thank you very much, I found that a few minutes after I made the post, and it is awesome. One more quick question- is it possible to show IOPS per storage system somehow?

When I add a storage node, and choose LeftHand RAID IOPS, it gives an snmp error. When I press debug, this shows up:

+ Found data query XML file at '/usr/share/cacti/site/resource/snmp_queries/lefthand-raid.xml'
+ XML file parsed ok.
+ Executing SNMP walk for list of indexes @ '.1.3.6.1.4.1.9804.3.1.1.2.4.12'
+ No SNMP data returned

The OID '1.3.6.1.4.1.9804.3.1.1.2.4.16' when I run it from a command line shows a counter64 value, so it would seem it is accessible somehow.

[rybal1212]

Sorry, my stupid mistake again, my nic got unplugged on my cacti system for the storage network.. Got it to work. Thanks so much for your hard work!

[eschoeller]

No problem, I'm glad you're finding the template set useful!

[rybal1212]

I am also noticing what you describe- the cluster statistics are not accurate. The IOPS and queue depth are much lower than reported in the CMC, and the latency is all over the place.

Would you suggest I also open a case with HP about this? Any tips on how to explain this behavior?

I am half tempted to install PRTG traffic grapher, since they have instructions on how to configure that software to monitor the SNMP data, and then opening a case with this erratic data. http://h10032.www1.hp.com/ctg/Manual/c01750818.pdf