[HOWTO] RESIZE existing RRAs of existing RRDs

[gandalf]

Please find a new post at http://forums.cacti.net/viewtopic.php?t=11425
Reinhard

Preface
This tutorial is based on rrdtool, written by Tobias Oetiker. It is written by Reinhard Scheck to help you figuring out, how to resize an existing RRA of a given rrd file.

Be warned!
BACKUP ALL YOUR RRDs!
There's a good chance, that you will destroy all of your rrd files. I'm not joking! At the time of writing, rrdtool 1.2.11 is stable. It contains a bug in the rrdtool resize function when resizing an existing rrd filetype 0001 (e.g. created with rrdtool 1.0.49) using the actual rrdtool 1.2.11 (which creates filetype 0003 itself). This bug has been fixed due to the work of Alex van den Bogaerdt. But tis fix was not yet incorporated to the stable release!
SO BACKUP ALL YOUR RRDs!
As always: Use this information at your own risk.

Here we go!
As an attachment to the forum entry you will find a tiny shell script resize.sh (well, at least it was tiny at the beginning).

Help!
Put resize.sh wherever you want. There's no need to put it into the rrd working directory. But you will need some scratch space here for all rrds to be resized (due to the way rrdtool resize works). The user that runs this script must have

• write permissions to the current directory used for scratch
• read permissions on the original rrds to be resized
• write permissions to the target directory to store the resized rrds in

The script does not care about space provided. To get help, simply type

Code: Select all

./resize.sh -h

you will receive

Code: Select all

resize.sh Version 0.3 - resize an existing rrd
Usage:    resize.sh -f <filemask> -r rra -o <output dir> -g <growth> -i [-d <debug>]
Requires: standard GNU commands
Author:   Reinhard Scheck
Date:     2005-10-03; last update: 2005-10-03
Options:
          -f, filemask of the source rrds
          -r, rra to be changed (first rra denotes as -r 0)
          -o, output directory for resized rrds
          -g, growth (number of data points to be ADDED to those already defined)
          -i, invoke rrdtool info instead of resizing
          -d, debug level (0=standard, 1=function trace, 2=verbose)
          -h, usage and options (this help)

Dry run
You may want to have a look at your rrds before resizing them. Specially for the required parameter -r (denoting the rra to be resized), you will want to have a look at those rras, that are defined in the rrd in question. Example (linefeeds only for ease of reading):

Code: Select all

./resize.sh -f "/var/www/html/cacti/rra/localhost_uptime_57.rrd" /
            -r 0 /
            -o /var/www/html/cacti/rra/resized/ 
            -g 8000 /
            -i

will result in:

Code: Select all

-- RRDTOOL INFO localhost_uptime_57.rrd ...
ds[uptime].type = "GAUGE"
rra[0].cf = "AVERAGE"
rra[0].rows = 600
rra[1].cf = "AVERAGE"
rra[1].rows = 700
rra[2].cf = "AVERAGE"
rra[2].rows = 775
rra[3].cf = "AVERAGE"
rra[3].rows = 797
rra[4].cf = "MIN"
rra[4].rows = 600
rra[5].cf = "MIN"
rra[5].rows = 700
rra[6].cf = "MIN"
rra[6].rows = 775
rra[7].cf = "MIN"
rra[7].rows = 797
rra[8].cf = "MAX"
rra[8].rows = 600
rra[9].cf = "MAX"
rra[9].rows = 700
rra[10].cf = "MAX"
rra[10].rows = 775
rra[11].cf = "MAX"
rra[11].rows = 797
rra[12].cf = "LAST"
rra[12].rows = 600
rra[13].cf = "LAST"
rra[13].rows = 700
rra[14].cf = "LAST"
rra[14].rows = 775
rra[15].cf = "LAST"
rra[15].rows = 797

You may notice

• a single data source (uptime)
• four consolidation functions (AVERAGE, MIN, MAX, LAST)
• four rra's for each of the consolidation functions

Of course, you may also enter a partly qualified dataset name. But it makes sense to take only those rrd's, that belong to the same datasource (e.g. with the same rrd file structure).

Resizing a single RRA of a single RRD
For ease of use, you may simply omit the trailing parameter -i. But pay attention to the parameter -r! In this example, only the first RRA of the consolidation function AVERAGE shall be resized. It depends on your needs, whether this will result in a correct RRD!

Code: Select all

./resize.sh -f "/var/www/html/cacti/rra/localhost_uptime_57.rrd" /
            -r 0 /
            -o /var/www/html/cacti/rra/resized/ 
            -g 8000

The output will look like:

Code: Select all

-- RRDTOOL RESIZE localhost_uptime_57.rrd RRA (0) growing 8000.. (95328).. RRA#0.. (159328).. Done.

The first parenthesis contain the file size before resizing, the second one after resizing.

Resizing multiple RRA of a single RRD
Simply enter all RRAs to be resized in quotes:

Code: Select all

./resize.sh -f "/var/www/html/cacti/rra/localhost_uptime_57.rrd" /
            -r “0 4 8 12” /
            -o /var/www/html/cacti/rra/resized/ 
            -g 8000

to result in

Code: Select all

-- RRDTOOL RESIZE localhost_uptime_57.rrd RRA (0 4 8 12) growing 8000.. (95328).. RRA#0#4#8#12.. (351328).. Done.

Resizing multiple RRAs of multiple RRDs
Please enter all RRAs to be resized in quotes and partly qualify alll RRDs:

Code: Select all

./resize.sh -f "/var/www/html/cacti/rra/*_uptime_*.rrd" /
            -r “0 4 8 12” /
            -o /var/www/html/cacti/rra/resized/ 
            -g 8000

to result in

Code: Select all

-- RRDTOOL RESIZE router_uptime_59.rrd RRA (0 4 8 12) growing 8000.. (95328).. RRA#0#4#8#12.. (351328).. Done.
-- RRDTOOL RESIZE gandalf_uptime_58.rrd RRA (0 4 8 12) growing 8000.. (95328).. RRA#0#4#8#12.. (351328).. Done.
-- RRDTOOL RESIZE localhost_uptime_57.rrd RRA (0 4 8 12) growing 8000.. (95328).. RRA#0#4#8#12.. (351328).. Done.

Something to keep in mind

Be warned!
You may even enter -o to resolve to the current RRD directory. This will result in overwriting your existing RRDs. YOU DON'T WANT TO DO THAT. Always look at the output after resizing. Try to generate graphs from them. Verify, that everything runs fine.
BACKUP YOUR ORIGINAL RRDs.
I appreciate any feedback to improve this document.
Reinhard Scheck aka lvm

[adrianmarsh]

Thanks for this!! At some point soon I'll be trying it out.

I take its linux based, and chance it'll run in cygwin? Any dependancies?

[Grand Master Wierdo]

Just gave this a go. Excellent work.

FYI works no probs on Cygwin, but watch out for resize bug. I am having to wait for next release of RRDTool to complete my resizing, but I managed to resize all of my version 3 RRA's no problem.

GMW

[gandalf]

AFAIK, the current bug only hits, when resizing file 0001 (rrdtool 1.0.49 or the like) with rrdtool 1.2.x. So, if you're running a pure 1.0.x system, this should work. BUT ALWAYS SAVE YOUR RRDs BEFORE.
Reinhard

[adrianmarsh]

Damn.. still haven't got to this yet....

Is there a way of identifing all the (eg) "600s" RRAs in an RRD? For each RRD there could be multiple DS' right? so I'd have to run the script on each "collective type" of RRD right? (using filename wildcards I guess), but how could I be sure its updating the right RRA within the RRD?

I was thinking about writing something that would "rrdtool info" the rrd file first, grep for "600" to identify the RRA and then run the script on that file for that RRA, but haven't looked any more into it as I think my understanding of the RRD format is still bad (even though I've read and re-read those tutoritals.. just can't get my head around it).

Luckily I'm fairly new to all this RRD/Cacti/SNMP stuff (can you tell?) so all mine have been built with 1.2.11 from the start.

[gandalf]

This is already done, see my script above
hth
Reinhard

[gandalf]

Please find a new post at http://forums.cacti.net/viewtopic.php?t=11425
Reinhard