Errors with resize.pl

[joanypony]

Hi,
I'm currently in the process of increasing the scale of my yearly RRA. Consequently I'll have to resize all of my existing RRDs. This process is very well documented on this forum and has been a fantastic help to me. Unfortunately I am getting an error with the resize.pl script and am stuck.

I run resize.pl against an rrd. It creates the scratch files, but then dies when it tries to put the resized file into the target directory. I have kglob commented out in resize.pl but I don't think that could be causing the issue.

I'm running it as root to bypass any permissions issues. I've tried it on two different cacti servers, and on different RRD files. The result is always the same.

Command with Info/Dry run is:

Code: Select all

./resize.pl -f "/cacti/rra/resized_95thcisco_traffic_in_4356.rrd" -s 797 -o /home/joan/resized/ -g 3883 -i
-- RRDTOOL INFO resized_95thcisco_traffic_in_4356.rrd ...
ds[traffic_in].type = "COUNTER"
ds[traffic_out].type = "COUNTER"
rra[0].cf = "AVERAGE"
rra[0].rows = 500
rra[1].cf = "AVERAGE"
rra[1].rows = 600
rra[2].cf = "AVERAGE"
rra[2].rows = 700
rra[3].cf = "AVERAGE"
rra[3].rows = 775
rra[4].cf = "AVERAGE"
rra[4].rows = 797
rra[5].cf = "MAX"
rra[5].rows = 500
rra[6].cf = "MAX"
rra[6].rows = 600
rra[7].cf = "MAX"
rra[7].rows = 700
rra[8].cf = "MAX"
rra[8].rows = 775
rra[9].cf = "MAX"
rra[9].rows = 797
user time: 0.00 system time: 0.00 real time: 0.00

Command Output with debug is:

Code: Select all

./resize.pl -f "/cacti/rra/resized_95thcisco_traffic_in_4356.rrd" -s 797 -o /home/joan/resized/ -g 3883 -d 2
function make_output_dir called, parms: /home/joan/resized/
function make_output_dir for /home/joan/resized/ returns: File exists
function get_rras called, parms: /cacti/rra/resized_95thcisco_traffic_in_4356.rrd 797
function resize called, parms: /cacti/rra/resized_95thcisco_traffic_in_4356.rrd 4 9  3883 /home/joan/resized/
-- RRDTOOL RESIZE resized_95thcisco_traffic_in_4356.rrd RRA (4 9 ) growing 3883function file_size called, parms: orig.rrd
ERROR: attempting to write beyond end of file at ./resize.pl line 160

The scratch files of orig.rrd and resize.rrd remain.

-rw-r--r-- 1 root root 173340 Jul 30 12:04 orig.rrd
-rw-r--r-- 1 root root 235468 Jul 30 12:04 resize.rrd

I'm running cacti 0.8.7e and rrdtool 1.4.1-1.

I've found only one other post online with this error -
http://osdir.com/ml/debian-bugs-dist/20 ... 05757.html

and noone ever replied to that dude
My OS is CentOS 5.4 and I have perl v5.8.8. Has anyone any ideas? This script looks perfect for what I need to do..

Regards,
Joan

[TheWitness]

RRDtool has the option to add rows without all the hoopla. So, my question is why do you need the resize binary at all? I would a nice simple compact bash script with about a dozen lines would do it.

Here is the pseudo code:

get all rrdfiles of interest
foreach rrd file do
check current rows with rrdtool info
if more rows needed
calculate rows needed
adjust rows with rrdtool tune
endif
done

TheWitness

[joanypony]

Hi,
Thank you, I'll give it a go.
Another question: I am increasing my "yearly" RRA to have a 2 hour average rather than a 1 day average. I'm assuming to update existing RRDs I will change my ROWs using the proposed script in the previous post. For any future RRDs, I edit my current yearly RRA in use in most data templates to reflect the steps and rows I want. Does this sound ok?

By the way, support on this forum is fantastic. Thanks!

regards,
Joan

[TheWitness]

Number of rows applies to the RRA and not the data template. Otherwise, yes.

[joanypony]

Hi,
I had to leave this project alone for a couple of months and now I've come back to it. I've managed to get resize.pl working (i gave up on coding it myself, as I'm crap at coding) and I have resized some existing RRDs on a test system. My goal is a 2 hour average on yearly data instead of a 1 day average. My question is - how do I know if I have succeeded? I can see from rrdtool info that the row size has changed in the RRD file and the new RRD is producing a graph. How can I see immediately that rrdtool is storing the extra data points for me?

Thanks,
Joan

[gandalf]

My goal is a 2 hour average on yearly data instead of a 1 day average.

That's not resizing. Resizing is based on existing "averaging intervals" and resizes the interval count.
But you want to CHANGE the "averaging interval". And honestly, that is not supported by rrdtool.

A workaround would be to use the "existing 2 h averaging interval" for "monthly data". In fact. there is no such thing as "monthly data".

So what your goal might be is to resize the RRA having a "2h averaging interval" to a size that is sufficient for 1 year. This will even work when zooming into the "1 year average graph", as rrdtool auto-selects the best matching RRA.

To suppress the "current 1 year display", you may remove the related RRA from that data template, but that's quite a bit of work.

R.

[joanypony]

Hi,
I hear what you're saying and I think I understand. But doesn't using resize.pl to increase the Rows of the "yearly" RRA of my existing RRD files have the effect of reducing the average? I was doing this combined with editing the "yearly" RRA listed within Cacti.
I understand that increasing timespan of "monthly" will collect the data for me, but I need to have the 1 year display showing the correct and more detailed data. (95th % billing and so on..) Thanks again!

Regards,
Joan

[gandalf]

Hi,
I hear what you're saying and I think I understand. But doesn't using resize.pl to increase the Rows of the "yearly" RRA of my existing RRD files have the effect of reducing the average?

No. RRDTool chooses the optimal RRA. And if you want to see MAX instead of AVERAGE (which will be reduced as timespan increases), you should instruct CACTI/RRDTool to do so!

I was doing this combined with editing the "yearly" RRA listed within Cacti.

Editing the RRAs will have effect only for newly created graphs.

I understand that increasing timespan of "monthly" will collect the data for me, but I need to have the 1 year display showing the correct and more detailed data. (95th % billing and so on..)

Again, RRDTool does not know a "yearly display" or "yearly RRA". I already explained this somewhere in ore detail.
When defining an RRA in Cacti, we unfortunately combine different items: the RRA settings and the timespan.
The RRA settings go directly to RRDTool.
Cacti timespan defines a "default view" when selecting any graph. In this case, we deduce, which "timespans" where defined along with the rra. E.g. you may define 4 different timespans by default. Then we create those 4 graphs, the last of them being the "yearly graph". But in fact, RRDTool chooses the best matching RRA settings for that graph on it's own. It does not necessarily pick the set that was defined in Cacti along with that timespan.
And this again happens, when zooming into a graph. So you may view a "yearly" graph, that was created from RRA setting "A" and when zooming into it switch automagically to the best fitting RRA setting "B", showing all the details you're interested in.
You may want to see my explanations in the rrdtool chapter at link#1 of my sig for more.
It's a bit hard to understand, I agree
R.

[joanypony]

Hi,
Thanks, explaining source of timespans is a big help. I already tried displaying the MAX but it was averaging out our traffic peaks too much in the older data. With my new (and unfeasible) method I was suspicious that it wasn't working as there were no blank gaps in my graphs when I zoomed in.. so, now what I need to know is, how can I edit the RRAs on my old RRDs to make them keep the "monthly" data for longer? I hope that makes sense..
I know how to edit the RRA within Cacti for all new files, but how can I make my existing RRD files keep 2 hour averaging data for at least a year? Apologies if this is a stupid question, RRDs and RRAs tend to melt my brain slightly..

Regards,
Joan

[gandalf]

so, now what I need to know is, how can I edit the RRAs on my old RRDs to make them keep the "monthly" data for longer?

? That's exactly what resize.pl is for!
You only assumed, that you will have to change the 4th rra (the one "related" to the one year timespan). But as you want to keep 2h averages, you will have to resize that very rra (should be the 3rd one, "related" to monthly timespan).

When doing so on a test RRD file, you will notice, that zooming into the "yearly" graph is fine until the granularity printed requires data from the 3rd RRA. Then you will notice blank data. Those will get filled up in time until the resized 3rd rra got filled up.
R.

[joanypony]

Hi,
Back again Resizing the RRA appropriate to monthly data has given me the result I want on my existing RRDs, thank you for all your help with that.
I have another question though. Which RRA should I be editing in my cacti web interface to ensure that all new RRDs created keep detailed (2 hourly average) data for a year? Do I leave the "monthly" one as it is, but increase the number of rows of the "yearly" one? Or do I increase the rows and timespan of the "monthly" one? I'd still like to have the 5 different default timespan graphs displayed for Cacti.

Regards,
Joan

[TheWitness]

Increase the number of rows in the monthly. Done. Keep in mind that if you import templates moving forward, that you need to apply the system defaults, and then verify that the Data Templates for those imported templates have the correct RRA associated with them.

This is only a problem when you create numerous 'custom' RRAs on top of the default of course.

TheWitness

[joanypony]

Great, thanks so much for all your help.
Regards,
Joan