rrd replication

[oxo-oxo]

This is on a small afterburner.

What if ...
A poller (rrdtool) was on a "far away site": detroit
- but if another poller "polls" the pollers SQL table and did it's own rrdtool, basically replicating the rrd's "locally": cologne

The data isn't lost due to network outrage/maintenance as the data is being collected/stored locally: detroit

The poller poller saves the poller table data and creates a copy of the rrd: cologne
- which may get gaps due to network problems.

The far site users look at their data, the enterprise users look at all data: just locally.
- performance enhancement ..?

The detroit poller isn't able to connect to cologne's MySQL
- does it matter ... the poller could have cached what it has to poll and refresh from central MySQL when connection restored ...
No data lost as it is in the rrd


(TheWitness will say "core code" so don't hold your breath for this feature, but it is doable)

[bud]

been working on something that may just be able to accommodate this, it is not going to be as feature rich as boost, but i currently have other selfish interests. Sill working on the process control, but the mechanics of it have turned out to be pretty simple so far, just a couple of additional tables, a DB trigger to automate the queue id generation, and some modification to a some Cacti functions. Short story is that rather than the poller updating the RRA directly, the update/create command gets intercepted, and after some reformatting which includes stripping off the RRA path it gets placed in a DB table queue. This way a path can be changed and prepended to the rrd file prior to the actual update, and would sit there until deleted, so updates could occur on as many instances of an RRD file as you want.

id: 1
control_id: 62
brp_stream: 1
process_state: 2
insert_time: 2008-01-03 17:15:03
hostname: 127.0.0.1
rra: localhost_physical_memory_62.rrd
command: update
command_options: --template physical_memory:free_memory 1199398502:523184:104712

id: 8
control_id: 84
brp_stream: 2
process_state: 2
insert_time: 2008-01-03 17:15:13
hostname: 127.0.0.1
rra: localhost_lhd_size_84.rrd
command: update
command_options: --template ldh_free:lhd_size 1199398502:8787357184:40007729152

Above are a couple of DB records from the queue. The brp_stream and process_state get used by an external script that assembles and processes the rrdtool commands for each queue or stream so you can have multiple local or remote rrd updaters. There would be a couple of different ways of accomplishing this, but in theory if one or more of these scripts are started on a remote system, based on a configuration variable an rra could be created/updated on what ever file system you want. I'm trying to restrict my focus on this functionality for the initial version, but as you can see by the DB records it wouldn't be that difficult to incorporate the hostname into the path resulting in more structured rrd paths.

if i piqued your interest and unless the development team want's to incorporate this into the Cacti core, God willing look for something called Backgrounded RRD Processing or brp in the coming weeks.

Cheers, bud (aka sidewinder/pyuska)

[gandalf]

Bud, please see discussion at http://forums.cacti.net/viewtopic.php?t=24869 to add your thoughts there. Most appreciated.
Reinhard