Storing the RRD data into DB

[marnues]

we're using a custom, distributed writer, so I'm not sure that boost will help

[msw1970]

Do you have the Plugin Architecture installed? Please advise. Then boost can be found under the plugin section of the forum. You can look at it, but the svn version is closer to what I would have you deploy:

svn checkout svn://svn.cacti.net/var/svnroot/cacti_plugins

TheWitness

Hey

I've noticed that there's no modified directory in the SVN for boost... Can I use the same modified files from the version I'm currently running??

[TheWitness]

I would use the Plugin Architecture 1.1 and not the modified files. That was only a patch until PIA 1.1 was out. That's why you do not find it there now.

Also, I believe we need a PIA 1.2 to fix some important bugs as well as BOOST_1.2 needs to be released somewhat concurrently.

TheWitness

[TheWitness]

Mention that he "should" sign up for a few hours of consulting and I will get it going for you. I'm pretty cheap

TheWitness

[marnues]

haha
I'll actually keep that in mind, btw

[disirk]

My particular setup is as follows:

I have a server with the rra directory exported via NFS. This server is responsible for all RRD updates.

The "poller" mounts the rra directory via NFS as does the web server. Since for some reason my system just isn't speedy enough to keep up with the size of my polled objects I had to split the polling from the actual rrd updating.

07/27/2007 02:57:16 PM - SYSTEM STATS: Time:136.2194 Method:cactid Processes:4 Threads:15 Hosts:563 HostsPerProcess:141 DataSources:98039 RRDsProcessed:0

I have "On Demand" enabled in Boost, but I have commented out the portion that executes the poller_boost.php because the load on the box was killing the poller, and the system still was unable to keep up with the rrd updates.

The RRD updater is performing as such:
07/27/2007 02:20:14 PM - SYSTEM BOOST STATS: Time:1273.6349 RRDUpdates:499966

Still not the best I've seen in the forums. but it ALMOST keeps up...

Now, my problem occurs when I access the graphs from the webpage...using boost and the boost server I lose data. I'm assuming because the boost server is attempting to retrieve data from poller_output_boost and input it into the rrd file before creating the graph. The problem here is, if the poller_boost process is running and its about to update the rrd...then I get something like:

ERROR: illegal attempt to update using time 1185544824 when last update time is 1185545724 (minimum one second step)
ERROR: illegal attempt to update using time 1185544824 when last update time is 1185545724 (minimum one second step)
ERROR: illegal attempt to update using time 1185544824 when last update time is 1185545724 (minimum one second step)
ERROR: illegal attempt to update using time 1185544824 when last update time is 1185545724 (minimum one second step)
ERROR: illegal attempt to update using time 1185544824 when last update time is 1185545724 (minimum one second step)
ERROR: illegal attempt to update using time 1185544824 when last update time is 1185545724 (minimum one second step)
ERROR: illegal attempt to update using time 1185544824 when last update time is 1185545724 (minimum one second step)
ERROR: illegal attempt to update using time 1185544824 when last update time is 1185545724 (minimum one second step)

I tried removing the boost server and let the web server just draw the graphs, but apparently it does the same thing because I see the same errors and gaps in my graphs.

Anyone have any ideas?

-Dave

[TheWitness]

Update to boost v1.5 right away. But you will have lost some data.

Larry

[TheWitness]

You might also want to conduct a GotoMeeting with me to isolate the issue behind updates.

Larry

[disirk]

Updated to 1.5 yesterday.

I'll let you know what I find, I am going to start the boost_server again and see what happens.

[TheWitness]

I may work on boost v1.6 today. It's purpose will be to speed the rate of database deletes.

Larry

[disirk]

It looks like this may be working a bit better. I don't see gaps in my graphs at the time I "accessed" them via the web.

Next step is to move my rra's to a faster hard drive...this 7200 RPM thing just isn't cutting it

tty ad4 ad6 cpu
tin tout KB/t tps MB/s KB/t tps MB/s us ni sy in id
0 30 0.00 0 0.00 52.07 248 12.63 23 0 8 2 67
0 30 2.00 1 0.00 58.30 491 27.97 11 0 10 2 77
0 30 0.00 0 0.00 44.69 301 13.15 3 0 3 1 92
0 30 0.00 0 0.00 46.06 268 12.07 12 0 6 2 80
0 30 0.00 0 0.00 56.56 497 27.43 1 0 6 0 92
0 30 16.00 1 0.02 49.23 335 16.12 18 0 11 2 69
0 30 0.00 0 0.00 40.67 288 11.43 4 0 5 1 91


Thats the output from my iostat with a 2 second wait. ad6 obviously is where my rra's are stored. Hopefully a 10k+ RPM drive will be better.