Best/Recommended way to bulk export traffic data

[davermenator]

Greetings! First off I would like to thank the developers for this wonderful tool and the community for helping them support it. You all rock!!

I am starting to branch out and do some more powerful functions with my cacti server. What I am seeking to accomplish is a way to have a directory either on the local Cacti server or on a remote machine (via [s]FTP) that has the csv export file for every link (or selected links if that is possible) on my server. Is this best done by a script? Is there a magic button I have not found? Or maybe I would be better off directly querying the database directly? After reading the documentation out there my head is swimming and I am not sure what the best approach would be.

My intent is to use the data to feed an excel document so that I can manipulate it in there to whatever form tickles my fancy - 95th percentile, time analysis, all sorts of fun stuff like that. Thoughts on the best way to do this will be greatly appreciated!!

I am running 0.8.7e with PA 2.6 if that makes a difference.

Thanks for your time!

[victorantunes]

Just a few thoughts first: you want to collect the exported .csv files into your custom server. That's fine. But how often are you going to export them? As you know, data gets updated every poller run, and the .csv file would reflect that. Hence, you gotta ask yourself how badly do you need your .csv files to be updated in relation to your data sources.

Yeah, just thought I'd point out the obvious there, because there is the issue of performance.

Now, I'm on the process of making a little plugin of my own, which internally calls the CSV Export function.

It's not pretty, and that's mostly because I'm not some great programmer. I'm currently using PHP's cURL library. Again, it's not pretty AT ALL. It's very much insecure, even, since I have to insert user login and password inside the code itself.

That's mostly due to Cacti's authentication (I'm using Builtin Authentication, btw) that forces your code to have credentials. Though, I haven't bothered with that in a while, since it's only a local test scenario.

Also, be aware that data inside your .csv won't be equal to the data in your graphs. There's the issue of SI vs Binary units (http://en.wikipedia.org/wiki/Mebibyte), which I've struggled a little to treat, and the scales themselves, since data on the .csv is expressed in scientific notation.



Anyway, sorry for writing so much and not offering much actual help, but the bottom line is: if you're doing it by code, which is the only way I know how, you need (at least from my experience) a way to send your credentials so Cacti can authenticate you. And yes, I've only managed to do it via cURL, but IIRC, I've read somewhere that you can do it with JSON and etc.

From there, it's basically about dealing with standard file functions for read and write operations.