Add an RRA to an existing RRD file

[gbigras1]

Hi there,

I'd like to add a new RRA to an RRD file. Can't figure out how to do it.

Here's some background information:

I'm using Cacti 0.8.7e with rrdtool version 1.3.8-1 on FC 11.

I've setup my RRDs to use four custom RRAs instead of the default ones. The major difference is that there is no data point averaging being done.
Here is my RRA setup (5 minute polling interval):

RRA1: Daily
Consolidation: average
steps: 1
rows: 300
Timespan: 86400

RRA2: Weekly
Consolidation: average
steps: 1
rows: 2304
Timespan: 604800

RRA3: Monthly
Consolidation: average
steps: 1
rows: 9216
Timespan: 2592000

RRA4: Yearly
Consolidation: average
steps: 1
rows: 109440
Timespan: 31536000

The reason there is no averaging is because the powers from above do not want to loose any data point. They want to be able to zoom into any "peak" on the graph anytime in the past and be able to look at the real data as opposed to an average.

However now they want to be able to keep more than a year's worth. Hard drive space is not an issue so I've opted for a ten year RRA. I just don't know how to modify an existing RRD file to incorporate these changes. I've tried some scripts to export data and import back but it doesn't seem to work.

thanks in advance for your help
Guy

[gandalf]

Hi
this is quite easy. But I need exact specifications.

Do you want to store "un-consolidated" data for 10 years? Or do you want to store them for e.g. 1 year and have a different consolidation for 10 years all-in-all?

Second question?
Which "timespans" do you want to have displayed when clicking on the graph
- daily
- weekly
- monthly
- yearly?
Please be aware, that when using unconsolidated data ONLY, at least the latter will cause to read An AWFUL LOT from disk and hence will slow down significantly. But there's a workaround
R.

[gbigras1]

What I want is to have an RRD file with an RRA that contains up to 10 years worth of data (no loss of data points) (I guess that would be unconsolidated) and then 5 graphs per device with the following timespans: daily, weekly, monthly, yearly and 10 years.

I know that one needs one RRA per graph but I thought that each RRA must be unconsolidated if I want to be able to zoom into past data and see the 'real' data points.

Is it possible that I could create one RRA, say the one called 'daily", with 10 years worth of unconsolidated data but only graph 24hrs worth, then the other timespans would be graphed using RRAs with consolidated data ?

thanks for the quick reply.
Guy

[gandalf]

You're getting it right, man!
Let's do some math, assuming 300 sec poller timespan.

To cover a day, you need
(1h / 5min) * 24 = 12*24 = 288 data points
To cover a year, you need
288 * 365 = 105,120 data points
To cover 10 years, you need
105120 * 10 = 1,051,200 data points

The numbers will differ, if you choose a different poller timespan. This number guarantees, that you have "unconsolidated" data for 10 years. Create your first RRA from that calculation.

Now let's turn to the optimization. You may not want to read 5 min raw data for a graph covering a week. Why? A week spans
288 * 7 = 2016 data points
It is unlikely, that your screen is that wide. So in fact, for e.g. an image of a width of 500 points, about 4 data points will be "averaged" into one pixel on your display. Things are getting worse for e.g. a month or a year.

So it makes less sense to use unconsolidated data when not zooming into the graph. In other words, you will make graph display much faster when storing consolidated data for graphing an coarse timespan. You still have your unconsolidated data in place when zooming! That's magic!

As a rule of thumb, you may want to use the standard RRAs for weekly, monthly. Increase the number of data points for the yearly RRA (it stores in fact two years worth of data) by a factor of 5 to get 10 years and you're done.

You will receive a very huge rrd file, though.

Reinhard

[gandalf]

One more word. If you're heading for peaks, it makes MUCH sense to use MAX consolidation function for graphing. Cacti unfortunately uses AVERAGE in most places.
This way, the peaks will not decrease when timespan increases (e.g. on a weekly, monthly, yearly graph)
R.

[gbigras1]

Thanks alot for your help Gandalf. I wish I would've posted to the forum much earlier.

Now I have some RRD files to modify or re-create. What should my approach be ? Is it possible to create the new RRD files with the modified RRAs, export data from the old RRDs and import back into the new RRDs ?

Can I use the existing export and import scripts or rrdtool functions ? Or will I need to make some custom scripts ?

thanks again.

[gandalf]

You will have to specify, which exact properties need a change. See "rrdtool tune" for the parameters that are changeable by rrdtool itself.
Then, please see contrib section of rrdtool download site to find some user contributed scripts that allow changing other stuff as well.
R.

[gbigras1]

Okay, will do.
I'm not ready to start working on this at the moment but will surely look at the rrdtool utilities and the contrib section. Plus I'll have more precise help requirements when (or if) I re-post later on.

thanks alot for your help Gandalf.
ciao for now.

Guy

[Sirup]

Hi Gandalf,

I also have a question to this topic. The monthy graph is consolidated to 2hours. I want this consolidated to 1h.

I just do not know where to start.
Or is there actually a good reason why it is 2h?

Thanks,

[gandalf]

Hi Gandalf,

I also have a question to this topic. The monthy graph is consolidated to 2hours. I want this consolidated to 1h.

This requires a change of "pdp_per_row". No support to do this is provided by rrdtool, sorry.

Or is there actually a good reason why it is 2h?

Using Tobi's defaults
R.

[coolest]

Wow this is helpful... This gives me the idea to consolidate my 5 years data.

Gandalf, you never ceases to amaze me... Thanks.

[Sirup]

Okay thanks. I just thought this is possible to change. Read somewhere somebody wanted to have e.g. Weekly - 15 Minutes.

But thanks anyways

[torstentfk]

Hello,

search for the correct syntax to simply add new rra to an existing rrd-file - but found nothing.
Here my rrd:

Code: Select all

rd_version = "0003"
step = 300
ds[pls_HostsPerProcess].type = "GAUGE"
ds[pls_HostsPerProcess].minimal_heartbeat = 600
ds[pls_HostsPerProcess].min = 0.0000000000e+00
ds[pls_HostsPerProcess].max = NaN
ds[pls_HostsPerProcess].last_ds = "1"
ds[pls_HostsPerProcess].value = 6.7000000000e+01
ds[pls_HostsPerProcess].unknown_sec = 0
rra[0].cf = "AVERAGE"
rra[0].rows = 600
rra[0].pdp_per_row = 1
rra[0].xff = 5.0000000000e-01
...

How can I add the same type of RRA again?

Torsten

[gandalf]

How can I add the same type of RRA again?

You can't. See above. You may search the contrib section of rrdtool, though
R.