Transporter, now at v0.2

[johnrembo]

Hi All,

Transporter plugin allows you to simply save poller-output data in cacti's database, so that other applications could access output data without touching rrd-files.

table fragment:


transporter settings:


In current release I've added some sample functions, so you could see what we can do with the archived data (truth, sample calculations currently works only on interface-traffic data templates):



some statistical data:


and charts:

[dononeil]

I downloaded and isntalled the plugin and it's putting the data in the DB ok, but when I click on the "Chart without RRD" I just get a blank window. Do I need to wait until enough data is stored, or is there something else going on?

[johnrembo]

I downloaded and isntalled the plugin and it's putting the data in the DB ok, but when I click on the "Chart without RRD" I just get a blank window. Do I need to wait until enough data is stored, or is there something else going on?

well,hmm....for data to be displayed on the chart it is enough to have 3 records for the same data-source in transporter output table (poller should run at least 3 times).
If required data is not yet available - it should display error message.
If you don't see any errors - it's time to check your web-server's logs for errors.

ps: have you enabled "Transporter statistics" under "User management"?

[FGor]

This is plugin work only with update interval = 5min ??

[johnrembo]

This is plugin work only with update interval = 5min ??

nope...it should work with any poller interval

Code: Select all

$pi=read_config_option("poller_interval");

[dononeil]

How do I 'unravel' the 'local_data_id' to the host table 'id' value? I don't see where the local_data_id matches any other table values that can eventually get me to where I can figure out what host the data is associated with.

[johnrembo]

How do I 'unravel' the 'local_data_id' to the host table 'id' value? I don't see where the local_data_id matches any other table values that can eventually get me to where I can figure out what host the data is associated with.

Hi,
first part of an answer is here:
http://forums.cacti.net/viewtopic.php?t=24959

and second part is at graph_local table (graph_local.id is actualy local_graph_id from 1st part).

[TheWitness]

johnrembo,

Very nice, I have been wanting to do something like this for a while. However, you may want to implement some form of partitioning to allow the data to be pruned rapidly. Since MySQL does not currently support partitioning (5.0 anyway), you might want to consider a workaround, or we should start to look at MySQL 5.1 to resolve table size and management issues.

I will likely be adding this type of functionality to the base, but for now, this is your domain

Regards,

Larry

[johnrembo]

speaking of sql partitioning - i've seen "handmade" solutions in mysql versions prior to 5.1 (watch4net APG - EMC SMARTS frontend), but I had no interest in it, because when you run "transporter" on 8 cpu machine with 16Gb's or ram and 250 days uptime - you simply do not care of how big your heap-table is and how many "slow queries" it produces

adopting the plugin for "mortal" users requires some time, currently I don't have it (at least until end of 2008). And, if we'll proceed with complete rrdtool replacement (otherwise i do not see any reasons why we should need endless partitioned sql storage) - this plugin should contain sub-plugin - "rrdtool emulator" - that's a challenge you know

ps: we had some tests a while ago - ~2500 DSL customers whose snr's, iptv/internet traffic and errors/discards were monitored during the weekend. 3 days of pooling every 1min resulted in ~60Gb "transporter" sql table. Of course plugin is far not perfect and there are many ways to optimize it, but stillt's more complicated that it might seem.

johnrembo,

Very nice, I have been wanting to do something like this for a while. However, you may want to implement some form of partitioning to allow the data to be pruned rapidly. Since MySQL does not currently support partitioning (5.0 anyway), you might want to consider a workaround, or we should start to look at MySQL 5.1 to resolve table size and management issues.

I will likely be adding this type of functionality to the base, but for now, this is your domain

Regards,

Larry

[TheWitness]

My goal is not to replace RRDtool, but to use something like www.fusionio.com, combined with something like the CSV exported data to feed a Flash Chart type application like www.fusioncharts.com (no relationship to FusionIO). Then, you can build any Flash type graph you want from the CSV data. However, I need to change that CSV function not to use the PHP XML functions as they are absurd in their use of memory.

TheWitness

[dononeil]

I'm still having a hard time figuring out what device in my cacti setup is being referenced by the "local_data_id" field in the transporter table. The post before telling me to look at someone elses code didn't get me far, it wasn't what I was looking for.

I need to basically be able to re-map the "local_data_id" field in the transporter table to the "id' in the host table.

Any suggestions on how to get from point a to point b, even if multiple queries, would be appreciated.

Thanks!

[dononeil]

I've run into a new 'bug' of sorts with the transporter plugin... when I try to use the CLI commands I get the following:

php /www/cacti/cli/add_graphs.php --list-graph-templates | grep -i "Interface - Traffic (bits/sec, 64 bit counters)"

PHP Warning: Call-time pass-by-reference has been deprecated; If you would like to pass it by reference, modify the declaration of [runtime function name](). If you would like to enable call-time pass-by-reference, you can set allow_call_time_pass_reference to true in your INI file in /data/cacti/plugins/transporter/setup.php on line 119

83 Interface - Traffic (bits/sec, 64 bit counters)

Now, you can see that I get the proper output on the last line, but of course this is causing my perl script to bomb, because I'm expecting just the last line of output. If I disable the transporter plugin I don't get the Warning.

Looking at the setup.php in the transporter plugin yields:

transporter_process($dbtable,$dbhistcycles,&$rrd_update_array);

I have no idea on how to fix this, to get the warnings to go away so my perl code works again. Any ideas?

[TheWitness]

Edit the php.ini file and locate:

allow_call_time_pass_reference

Set it to "Yes" or "On" (not sure which it is) and restart Apache. Although the restart is not required for the CLI.

TheWitness

[zorrosam]

Hi,

i can't obtain any data in the window after that i clicked on the transporte icon on the right of the graph.

The output is:

no traffic_in or traffic_out data on current graph. Transporter must run at least three times(3 polling cycles) before calculations are available.

Why?

The plugin is enable .... and i think the works with the cacti poller. is correct?

any help is appreciate.

zorro

[sangayya.alagundimath]

Hi All,

i have downloaded the plugin and extracted it, but can any one tell me how to install this.