[HOWTO] RESIZE existing RRAs of existing RRDs

[maxclark]

Thanks for posting this.

[fmangeant]

Thanks Reinhard !

I used to modify RRAs juste after installing Cacti, but I forgot to do so on my secondary server. A few minutes after running the script, RRD files were at the desired size

BTW, this thread should be made sticky...

[mokeybrains]

I resized a bunch or RRAs. my test one (which I did first and verified) I accidentally resized twice and 'grew' it twice. Is there an easy way to shrink the RRA? Feature request for next resize.pl version!

Also, RE: wanting to see fine granularity a year ago: that's absurd, no one should be looking that far back. Move on to the future!

[gandalf]

I resized a bunch or RRAs. my test one (which I did first and verified) I accidentally resized twice and 'grew' it twice. Is there an easy way to shrink the RRA? Feature request for next resize.pl version!

To be honest, that should be a question of only a few lines of code. But I never though this would be of any use. Cactitians only like to blow their filesystems
If you need a quick shot: replace GROW by SHRINK in the code and this should do ...
Reinhard

[mokeybrains]

Ha! You would think that Cactitians would care about system performance... but then again, load and cpu graphs near 0 are boring.

Attached is a little patch to allow you to us resize to grow or unGrow your RRD files.

The '-G' flag will shrink, while the -g flag will still grow. You can, in a fit of confusion, use them both and the difference of the two will be applied to your RRD file.

[Ahuber]

I just wanted to mention, ... My rrd's are 32mb, some are 80mbs (depending on how many DS). And they only hold 1 minute resolution for the entire year (365 days)
After reading this post, that's not right, is it?

[mokeybrains]

I find it hard to believe that anyone would really want to look at any data on a minute granularity from half a year ago. No, I am not trying to bait any one, but really, I monitor 51 devices, and the last thing I need is granularity from a long time ago: it just doesn't help in debugging machines or billing customers...

[Ahuber]

Unfortunatley I agree with you monkey brains. But I'm trying to work out that mistake, by saving my graphs, becuase if it's more than 2 months old data, I only want to see what it averaged over 2 or 4 hours. We do need some granularity to detect trends over the course of 8 to 12 or more months, but not at a minute. I am reading many posts to consolidate, it's kinda hard right now, but I am determined. I don't know how large an rra should be at a per minute over 12 months, does anyone know?

[randy2009]

Hi, i resized about 100 of my rrd files. They are filled with about 9 months of data. I use them for pnp4nagios. This is the structure (default pnp4nagios):

Old values:

RRA:AVERAGE:0.5:1:2880 (0) (1 min interval for 48 hours)
RRA:AVERAGE:0.5:5:2880 (1) (5 min interval for 10 days)
RRA:AVERAGE:0.5:30:4320 (2) (30 min interval for 90 days)
RRA:AVERAGE:0.5:360:5840 (3) (6 hour interval for 4 years)

RRA:MAX:0.5:1:2880 (4) "
RRA:MAX:0.5:5:2880 (5) "
RRA:MAX:0.5:30:4320 (6) "
RRA:MAX:0.5:360:5840 (7) "

RRA:MIN:0.5:1:2880 (8) "
RRA:MIN:0.5:5:2880 (9) "
RRA:MIN:0.5:30:4320 (10) "
RRA:MIN:0.5:360:5840 (11) "

I did it like this: (resize.pl is downloaded from this thread)

Code: Select all

    sudo /home/nagios/resize.pl -f "/rrd/*.rrd" -r '0 4 8' -o "/rrd/" -g 17280 >> log_omzetting.log
    sudo /home/nagios/resize.pl -f "/rrd/*.rrd" -r '1 5 9' -o "/rrd/" -g 102240 >> log_omzetting.log
    sudo /home/nagios/resize.pl -f "/rrd/*.rrd" -r '2 6 10' -o "/rrd/" -g 30720 >> log_omzetting.log

New values:

RRA:AVERAGE:0.5:1:20160 (0) (1 min interval for 14 days)
RRA:AVERAGE:0.5:5:105120 (1) (5 min interval for 365 days)
RRA:AVERAGE:0.5:30:35040 (2) (30 min interval for 2 years)
RRA:AVERAGE:0.5:360:5840 (3) (6 hour interval for 4 years)

RRA:MAX:0.5:1:20160 (4) "
RRA:MAX:0.5:5:105120 (5) "
RRA:MAX:0.5:30:35040 (6) "
RRA:MAX:0.5:360:5840 (7) "

RRA:MIN:0.5:1:20160 (8) "
RRA:MIN:0.5:5:105120 (9) "
RRA:MIN:0.5:30:35040 (10) "
RRA:MIN:0.5:360:5840 (11) "


If i check the rrd's with rrdtool -info i see that the rrd databases are ok.

But now i have a pretty big problem. In the rrd files there are about 9 month of data. If i graph these rrd's (pnp4nagios) i see that all is ok, but if i zoom in to the past all data is being lost/gone. If i then zoom out (still in the past) i see the data. So it seems that rrdtool wants to retreive data out of the wrong RRA (9 months ago my resolution was diffrent) when zooming to close.

If i check a not resized rrd i can zoom-in as much as i want. So probally due to the resize i can't graph my data correctly.

I think it has to do with the fact that rrdtool want's data from let's say 2 month ago, in the old situation it goes to the 3rd RRA (2). In the new situation the data is still in the 3rd RRA (2) but rrdtool now looks at the structure and think it can take the data out of the second RRA (3). But it can't, it can only take data out of the 2nd RRA after the resize (data from 2 months ago).

Can someone help me fix this problem? I've been searching google for a while, but can't find anything

[gandalf]

When resizing an rrd file, it takes time for the new "slots" to be filled up with data.

RRDTool uses some technique to fetch the data from the "optimal" rra file. But when doing so, it does not check if data was filled into that (in your case just extended) rra already.

If the display interval exceeds that single (not yet filled) rra, rrdtool will dynamically select a different rra that already holds the (consolidated) data.

R.

[randy2009]

Thank's for replying, I'll try to understand it.

If the display interval exceeds that single (not yet filled) rra, rrdtool will dynamically select a different rra that already holds the (consolidated) data.

But why can't i get data if i zoom in the past? It looks like rrd doesn't select a different rra, because it selects the same rra (which doesn't hold historical data (few months ago)), or am i wrong?

And is there a way to resolve this problem? (i use the graphs to make management reports)

Randy

[gandalf]

And is there a way to resolve this problem? (i use the graphs to make management reports)

Wait until RRA gets filled.

Or don't zoom that deep, that the "unfilled" rra is selected. Post some screenshots of fille/unfilled graphs so I can explain using those examples
R.

[randy2009]

Here you see my complete (1year) graph:


If i zoom in to the date 11.04.10 - 16.04.10 i get this graph


If i then zoom out again (date 04.04.10 - 26.04.10) i get the missing data:


Here i zoom in more:


If i zoom to let's say last week i get all the data:


More data:

If i zoom in on the beginning of the graph (week 31 last year (about 22.07.2010)).


If i zoom a little futher:




The data is primary used to report to customers, that's why i need the data.

[gandalf]

Please retry now that some more days have passed by. The graphs of April should be filled by now
R.

[randy2009]

If i look at the 2nd graph, i get the same image (11.04.10 - 16.04.10), nothing has changed.

Same for the data 9 months ago