Storing the RRD data into DB

manickkarthik · Post by **manickkarthik** » Mon Feb 26, 2007 1:18 am

Hello,

If you can then send the procedure for storing the RRD File data into Database.

Post by **rony** » Mon Feb 26, 2007 9:51 am

Nope, Cacti is built around RRDTool for a reason.

marnues · Post by **marnues** » Fri Mar 02, 2007 4:39 pm

so there is no chance of moving away from rrdtool?
it would really save my company a lot of hassle
we're probably going to do it at some point if you guys don't

Post by **gandalf** » Sat Mar 03, 2007 4:24 am

Please read the stuff at the first link of my signature. It explains cacti's architecture. You will find, that rrdtool is used for storing and for graphing. If you plan to store the data in SQL tables, you won't be able to graph them with cacti.
Reinhard

Post by **TheWitness** » Sat Mar 03, 2007 10:19 am

I don't suggest that you do that for various reasons. But more important to me is to know "what" is wrong with RRDtool? The reason that I ask this is that even if you "don't" use RRDtool as your graphing tool, it still provides the "BEST IN THE WORLD" data storage engine.

With things like RRDtools "fetch" command, or "export" commands, you can easily pass this to any graphing package you want.

So, your opinion counts. Please share it.

TheWitness

marnues · Post by **marnues** » Sat Mar 03, 2007 2:15 pm

there's an issue with rrdtool and nfs that routinely loses data
its not a huge loss, but it is certainly noticable
we don't have a useful replacement for nfs (replacing nfs would be such a huge task anyway, affecting much more than just Cacti) so we are hoping to eventually replace rrdtool with a DB and another graphing solution (I'm going to be looking into the speed of gnuplot soon)

we understand the principles behind rrdtool and would essentially emulate them as they do fit perfectly into Cacti
however, using a DB ensures that we are not losing data because the writer software encounters an nfs lock like it does with rrdtool

Exo7 · Post by **Exo7** » Mon Mar 05, 2007 6:01 am

Maybe the server feature of rrdtool 1.2 is a solution ? it can send data to a remote rrd server via a TCP port.
Since it is now available to Cacti via the Boost plugin, it should be less work than rewriting everything.

Post by **TheWitness** » Mon Mar 05, 2007 7:48 am

Yes, boost is quite nice I think. Reduced poller times by a factor of 10 in some cases. I am planning a 1.2 release to help with the Multi-process boost lockup issues. So, keep your eyes open for it.

I now see what you are speaking of with data loss. What is it about NFS that causes these problems. Is it that there are too many files in one directory or is it that nfs can't keep up with the write activity or something else?

TheWitness

Post by **TheWitness** » Mon Mar 05, 2007 7:49 am

Oh, you are mounting your RRA directory. Yuk. I don't recommend that, unless your HTTP server is different from the poller. I alway recommend that the poller have local storage, but HTML can symlink. However, boost WILL help this issue.

TheWitness

marnues · Post by **marnues** » Mon Mar 05, 2007 1:12 pm

We actually have 6 boxes nearly dedicated to Cacti right now.
So we have to mount things remotely.

Someone else on my team has spoken with the rrdtool guys and apparently this is a known issue and it doesn't sound like it will be resolved with them anytime soon.
However, it looks like we're using rrdtool 1.0.49, so I will request an upgrade before we do anything drastic.
Thanks for the suggestion!

marnues · Post by **marnues** » Mon Mar 05, 2007 1:18 pm

Well, I guess I checked the wrong box.
We're using rrdtool 1.2.15

Post by **TheWitness** » Mon Mar 05, 2007 1:36 pm

You should find that boost helps here. Again the important thing, more so than anything else, is that put your disc on the poller if at all possible.

Boost will allow you to have multiple web sites accessing that data without problem and will update your RRD files on demand using the boost_server from any of the X web sites.

However, if you are using Multi-process boost, we should be talking (some sort of engagement) before you begin.

TheWitness

marnues · Post by **marnues** » Mon Mar 05, 2007 6:59 pm

Our rrd files are actually stored on an industrial strength file server farm with all kinds of bells and whistles, so no chance of them being on the same box

however, any chance of not completely redoing Cacti's internals would be something sweet, so any info on this boost stuff (the only boost I know of are c/c++ libraries) would be greatly appreciated

Post by **TheWitness** » Mon Mar 05, 2007 7:18 pm

Do you have the Plugin Architecture installed? Please advise. Then boost can be found under the plugin section of the forum. You can look at it, but the svn version is closer to what I would have you deploy:

svn checkout svn://svn.cacti.net/var/svnroot/cacti_plugins

TheWitness

marnues · Post by **marnues** » Tue Mar 06, 2007 7:01 pm

we are not using the plugin architecture
I'll have to discuss this with my boss and others involved in the project

Storing the RRD data into DB

Storing the RRD data into DB

Who is online