Graphing Clariion statistics from EMC Navisphere CLI output

[cjlove72]

Hi all,

I'm a recent Cacti user and I've seen several posts on trying to use Cacti to graph EMC Clariion stats, so I'm well aware that SNMP doesn't provide any useful information.

However, the Navisphere CLI *does* provide information on RAID Group usage, SP cache hit usage, LUN read/write cache hit ratios etc etc.

So, IN THEORY, it should be possible to install Navisphere CLI on the Cacti server and create scripts using navicli to pull this information out of the Clariions, and then create all the necessary definitions within Cacti to populate RRDs with this information and then graph these statistics.

Has anyone attempted this yet? I'd love to hear from someone (anyone!) who has tried getting this up and running before I embark on what I expect to be a long and painful (but ultimately very useful!) learning experience.

Cheers,

Chris

[gandalf]

If you go for it, please use walkthroughs provided at 1st link of my sig
Reinhard

[hdaube]

Hi,

i am also working on this.
But i cannot manage these problems:
- writing in perl to run the NAVICLI Commands
- Importing more than one field with an unknown number of values

I know, whats needs to be done:
- CLARiiON - Navisphere Analyzer needs to be running on the CLARiiON
- Data Logging needs to be set up (maybe with the default intervalls = 60sec for real time, 120sec for archive)
- CLARiiON Flare Code Version does not affect
- it makes sense to collect the files each 10min or later, otherwise there is not much data in
- Command to retrieve the Logging Files from CLARiiON :
"naviseccli -user admin -password password -scope 0 -h 1.2.3.4 analyzer -archive -all
-then the files need to be converted / dumped:
naviseccli analyzer -archivedump -data <Filename>.nar -out <New Filename>.csv -object s,l,d -format on,pt,u,ql,rt,tb,tt,rb,rs,rio,wb,ws,wio,up,ff,abql,dc,st,asd
--> Objects can be selected on request - check Navisphere Analyzer Documentation for details

Maybe this information is usefull. For me, the major thins is to load the content of the .csv file in CACTI.

Holger

[shadow15g]

I'm also start working on that.
Anybody have any luck ?

[hdaube]

I'm also start working on that.
Anybody have any luck ?

Still working on it.
My perl script is running, collect all NAR files from the CLARiiON, converts them to CSV and prints out each line from each CSV file.

But i still have no idea how to bring this output into CACTI.

A helpfull thing could be, that one field of the csv-file contains date & time.

[TheWitness]

This is easy, if it's in a csv file, and you have direct access to it, then...

Something like this. Say, the value is field 10. Then you write a nice little script that does the following:

Code: Select all

#!/bin/bash
myVar=`tail -1 <myfile> | awk -F"," '{print $10}'`
echo $myVar

If you want to go fancy, you would write a script_server script that read the file once into a global variable as an array of all parts and then share it between the various data sources with the first call loading the global array. That should have the least overhead since it will only have the file system overhead once.

If the file is on another host, you could rcp/scp it or run the script using ssh/rsh.

TheWitness

[Rael]

Any updates on this? I'm looking for this as well.

[TheWitness]

Any updates on this? I'm looking for this as well.

I recommend, you learn to use your keyboard for something other than asking. How about contributing. Let's see how well you do. Sincerely, it's like that commercial that shows someone missing the garbage can with a piece of garbage and it just lies there on the side walk, instead of being the one complaining it's there on the sitewalk, why don't you simply pick it up an throw it in the garbage... (aka write it, and then contribute your writing).

TheWitness

[hdaube]

This is easy, if it's in a csv file, and you have direct access to it, then...

Something like this. Say, the value is field 10. Then you write a nice little script that does the following:

Code: Select all

#!/bin/bash
myVar=`tail -1 <myfile> | awk -F"," '{print $10}'`
echo $myVar

If you want to go fancy, you would write a script_server script that read the file once into a global variable as an array of all parts and then share it between the various data sources with the first call loading the global array. That should have the least overhead since it will only have the file system overhead once.

If the file is on another host, you could rcp/scp it or run the script using ssh/rsh.

TheWitness

Hi,

nice idea with the AWK. I circle around the CSV File, because it looks like this:
---------------
Object Name,Poll Time,Utilization [%],Queue Length,Response Time [ms],Total Bandwidth [MB/s],Total Throughput [IO/s],Read Bandwidth [MB/s],Read Size [KB],Read Throughput [IO/s],Write Bandwidth [MB/s],Write Size [KB],Write Throughput [IO/s],Used Prefetches [%],Forced Flushes/s,Average Busy Queue Length,Disk Crossings/s,Service Time [ms],Average Seek Distance [GB]
SP A,08/03/2008 05:02:47,17.876896,0.414778,0.428274,32.428543,968.487950,26.475349,63.897130,424.287561,5.953194,11.201886,544.200389,,,2.320192,,0.184586,
SP A,08/03/2008 05:12:47,21.215028,0.271324,0.241639,30.961737,1122.849532,21.057491,61.150413,352.620207,9.904245,13.167438,770.229324,,,1.278922,,0.188939,
SP A,08/03/2008 05:22:47,20.834043,0.333600,0.289540,35.893853,1152.172478,23.451633,61.877592,388.096427,12.442220,16.674823,764.076052,,,1.601225,,0.180824,
SP A,08/03/2008 05:32:47,19.498747,0.266029,0.219095,29.296765,1214.217975,18.721839,44.117921,434.543676,10.574925,13.888778,779.674299,,,1.364339,,0.160587,
SP A,08/03/2008 05:42:47,19.726563,0.254216,0.220238,28.294289,1154.275769,18.147817,50.888619,365.177215,10.146472,13.166907,789.098555,,,1.288696,,0.170900,
SP A,08/03/2008 05:52:47,17.584881,0.285602,0.346549,26.858213,824.130644,16.936456,69.083181,251.044183,9.921757,17.728353,573.086461,,,1.624131,,0.213375,
SP A,08/03/2008 06:02:47,18.011758,0.312748,0.257192,33.689580,1216.010398,26.787748,39.560532,693.384350,6.901832,13.523007,522.626047,,,1.736357,,0.148122,
SP A,08/03/2008 06:12:47,23.223957,0.380700,0.211709,37.110734,1798.226192,27.596846,29.448189,959.623367,9.513888,11.617205,838.602825,,,1.639257,,0.129149,
----------------------
.... and may has another 50000 lines. Each field is a value.

Maybe i'm on the wrong way. When polling my Perl-Script, it gets a file from the CLARiiON and converts it to this .csv file. So each polling intervall generates so much data.
For example: if the CLARiiON has 100 LUNs, the csv file contains each 10 minutes aprox. 1000 lines with each line containing 20 values.

The file is local on the CACTI Server, so no need to SCP it.

[TheWitness]

Well, since you are keeping multiple intervals, cacti is only interested in the "last" one. So, instead of scp, do the following:

Code: Select all

ssh -l <mylogin> <myclariion> cat <myfile> | grep <myhost> | tail -1

Parse the data using script server script into a global array. Then if you are reading the values in many script servers, say some for storage, others for cpu and or network. If all the values are pumped into the global array by the first call, then all subsequent calls can simply use the global array instead of using the ssh call a second time. Since, on a per host basis, all script server calls are serial, you will only have 1 SSH session per polling interval.

If you are not familiar with the script server, then readup. I would declare in your function, for a host named "myhost", I would create a global array in the function as follows:

Code: Select all

<pre-requisite script_server_stuff>

function get_my_clariion_cpu($myhostname) {
  global $<myhostname>_clariion_stats;

  if (isset($<myhostname>_clariion_stats)) {
    get the value from here;
  } else {
    ssh the data into the global array;
  }

  return $my_cpu_stats;
}

TheWitness

[rony]

The following script:

Code: Select all

#!/bin/bash
myVar=`tail -1 <myfile> | awk -F"," '{print $10}'`
echo $myVar

should be like this for effeciency:

Code: Select all

#!/bin/bash
tail -1 $1 | awk -F"," '{print $10}'

[spike100]

So, IN THEORY, it should be possible to install Navisphere CLI on the Cacti server and create scripts using navicli to pull this information out of the Clariions, and then create all the necessary definitions within Cacti to populate RRDs with this information and then graph these statistics.

Hiya, I've just come from the Clariion Performance workshop and am looking to generate some 'baselline' stats for my Clariions. This thread looks to be a bit cold now, but I'm working on producing some graphs. Did you guys get anywhere?

I've got a bit of a different plan in mind.. To keep the load off the SPs I'm looking to populate a RRD file retrospectively. I don't need 'live' figures so was looking at writing a dump of yesterdays figures directly into a RRD from a single daily download of a .nar file. This could be done more frequently if finer grained graphs were needed.

There's a guy in the department doing something similar already so I can pick his brains....

Maybe we could put our heads together on this...

Spike

[TheWitness]

Spike,

I was going to do this for one of my customers. However, they invested in another tool to perform this. My plan was to use either rsh or ssh to access the node with the cli and return results, and do the graphing of those results.

For the number of Clariions that they had, it would take a second or two to get all the volume usage, cpu usage, memory usage, and IOPS.

Some times you win and sometimes you loose. I still believe it's the right thing to do. If security is a concern, you can setup a crontab on the management node to write this data to a file somewhere, then your Cacti scripts just need to grep the files. That's likely a much faster solution than several SSH calls.

TheWitness

[hdaube]

I've got a bit of a different plan in mind.. To keep the load off the SPs I'm looking to populate a RRD file retrospectively. I don't need 'live' figures so was looking at writing a dump of yesterdays figures directly into a RRD from a single daily download of a .nar file. This could be done more frequently if finer grained graphs were needed.

Spike,

load on the SPs is not an issue. Navisphere Analyzer + Data Collection creates around 10% overhead. Depending on your setup, you will have 3-6 files per SP (@all: each CLARiiON has 2 SPs, each one has it's own IP and creates different data).

I finished a perl script to collect all .nar (Navisphere Data) files from one CLARiiON and dump them into one csv file. Now i still need some help to bring this data into CACTI or RRD.

My script is attached. To run it ou need to have perl installed (Cacti on Linux already has.....) and you need Navisphere Secure CLI installed on the system. If anybody does not have a live environment, i'll provide some of my Navisphere Data Files.

Sorry, i'm on a dead-end street with my knowledge.

[TheWitness]

Can you provide the output and the field names associated with the output?

To do this more correctly, you need a few functions similar to the host disk stuff for Disk Utilization. Each disk would report things like:

Input Fields for the Data Query Would be like
volumeName
fileSystemType
blockSize
totalBlocks

Output Fields would include:
IOSperSecond
usedMBytes
freeMBytes
IOWait
etc.

If the CSV file has those details, then that's great. You can create a data query. Then for things like CPU, Memory, Cache, you might be able to simply use a graph template.

Network utilization again requires a Data Query. What I didn't like was the EMC choose not to support 64bit counters for their SNMP agent (I have no freeking clue why). So again back to the Nav utility.

Good luck.

TheWitness