How-to ? graph output of a .csv file

[wandererjs]

All:

I have a question for the scripting gurus of the forums.

Short version: how do I graph the output of a csv file generated by a script, knowing the script will return from 50 to 1000 rows of data?

First few rows of .csv are below for reference

IfDescr,counter1,queue0-DropPackets,queue0-TxBytes,queue1-DropPackets,queue1-TxBytes,queue2-DropPackets,queue2-TxBytes,queue3
-DropPackets,queue3-TxBytes,retransmit-counter,signalling-counter,user-be-counter,user-cmcs-counter,user-speech-counter,user-spe
ech-rate-limit-user-speech,user-video-counter,
ge-0/1/0.40,28841,83,9698099408,0,84736015,0,0,0,1693252792,215342943,3200147888,20675755064,0,47472,0,0,
ge-0/1/0.86,5612008,123,10781112912,0,103780711,0,0,0,1937553490,461212572,3830835519,48844263173,0,31216146,0,0,
ge-0/1/0.19,7829,26,5513335360,0,33097821,0,0,0,2193558755,55719676,3618554369,9332238835,0,0,0,0,
ge-0/1/0.2,0,0,5692453,0,26268,0,0,0,2401117,37363,2412635,5703091,0,0,0,0,
ge-0/1/0.4,17779,1754,150427117571,0,699765687,0,0,0,5852821007,1417580274,10170544314,226382448141,0,118378,0,0,
ge-0/1/0.10,2276187,86014,466434610033,121,9352286821,0,0,0,26327423645,26282596732,44680491021,938465545817,0,11733320,0,0,
ge-0/1/0.13,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
ge-0/1/0.14,2147136,37749,425880603810,24,5748110722,0,0,0,20467141487,11385752665,37623727271,710748841558,0,3582757,0,0,
ge-0/1/0.155,5193,336,78414931446,0,311080756,0,0,0,3087967938,787215820,5797181635,175271324590,0,193100,0,0,
ge-0/1/0.168,31908,136,24688232888,0,136005863,0,0,0,3725587062,271008434,6088532476,39880323456,0,62439,0,0,
ge-0/1/0.202,115842,652,95391523516,0,596054571,0,0,0,7358139735,2086092094,15398514663,221756663676,0,1238967,0,0,

Longer version, and why I'm asking:

I'm trying to graph some QoS based stats on a Juniper router. I wrote a perl script do pull the stats, even though they are all polled via SNMP, as 1) I need to filter the thousands of interfaces down to a few hundred, 2) to keep SNMP traffic down to a minimum, I'm polling only a handful of tables, and 3 (most importantly), one of the tables is not directly tied back to the interface: I have to pull the counter name, parse the interface name, locate the IfIndex based on the ifDescr name, and update *that* entry.

Anyway: I'm certain this is trivial for Cacti; it does pretty much all this behind the scenes when doing an SNMP indexed query. The problem is, Cacti does all this behind the scenes so I can't figure out what I need to do when I've already done half the work manually: gathered the data, organized it, processed it, and spit it out.

Inputs to the script (perl) would be the IP and SNMP community; exports would be the CSV file given as an example above (with or without the column headers).

I thought this would be built as a Script (indexed), but the template seems to want different arguments to pass back to rebuild the index (and refresh the index, and query the index, et cetra), which is something the script will not be easily able to do. I *could* add command line options for this, but as it takes 2-3 minutes to pull all these tables for the real stats I need, and it would take just as long to verify the index.

The other script templates seem to expect a single value or a single row returned, which is not what this script does.

The target is to graph all the stats on the first line on a per-router, per interface basis. I'm now familiar enough with RRDTOOL to where I could use the perl plugin and do this directly, but that defeats the purpose of using Cacti for graph, data source, and template management.

Thoughts?

[gandalf]

Best I can do is to direct you to the HowTo http://docs.cacti.net/node/225
Reinhard

[wandererjs]

Ok, I think that gave me the last few things I need. I *will* need to add a few functions for index and reindex, but can use the native Cacti check for when to re-index.

Thanks for posting that link: I spent hours reading through the docs and forums; must have missed that part in the How-To when I was focusing on a pure SNMP solution.

[wandererjs]

Ok, I think that gave me the last few things I need. I *will* need to add a few functions for index and reindex, but can use the native Cacti check for when to re-index.

Thanks for posting that link: I spent hours reading through the docs and forums; must have missed that part in the How-To when I was focusing on a pure SNMP solution.

Gandalf:

After reading through the how-to/docs a few more times and working the example step-by-step, I think I figured out why I was having trouble:

Cacti currently does not handle receiving multiple columns of data for a single index (i.e, like a CSV file). The required functions of query and get also require a column name, which (unless I'm reading this wrong) means Cacti can pull only a single column at a time.

Is this correct? Are there any plans to allow this? I guess I see why Cacti was built like this: its background is SNMP, which will always return one column at a time.

Thanks much,

Wanderer

[gandalf]

You're quite right.
There would be a last solution: Update rrd's outside cacti with a script of your own. This works as described in http://docs.cacti.net/node/358
Reinhard