verify 95th percentile value

[sys0p]

Hello ,

I am having a dispute with someone who is saying that my 95th percentile graph value is not correct.

So right now I ended up trying to manually calculate the 95th percentile for the last month.
In order to do this I have dumped the data from the RRD with rrdump and extracted with awk the all the values, 5 minute average,30 minute and 2 hours.
I have did this the old dirty way, I throwed the values for iN/OUT in an xls and took out the 95th percentile values and choosed the biggest one.

Still the calculations are not correct, can someone please enlighten me what am I doing wrong ? maybe there is another way to fetch the data from the rrd file ? the formula I am sure is correct( I have tested it for one day only and the values match).

Thank You !

[BSOD2600]

If you're using standard rrd row settings, then you're likely experiencing data consolidation, which is throwing off your calculations.

Read http://docs.cacti.net/node/275 and http://docs.cacti.net/node/54

[rony]

Is this a single interface graph (IN and OUT) traffic?

What is the specific dispute? To low, to high? Usually to high..

Are your RRDTool files (and RRA settings) configured to store more than 1 month of 5 minute averages? BSOD linked some information above.

Check my ISP billing documentation for guideline on RRA definitions at http://forums.cacti.net/viewtopic.php?t ... highlight=

[sys0p]

Is this a single interface graph (IN and OUT) traffic?

What is the specific dispute? To low, to high? Usually to high..

Are your RRDTool files (and RRA settings) configured to store more than 1 month of 5 minute averages? BSOD linked some information above.

Check my ISP billing documentation for guideline on RRA definitions at http://forums.cacti.net/viewtopic.php?t ... highlight=

Thank You both for your feedback , rony: yes it is a single interface IN/OUT and indeed they say it is too high.. (:

Billing interval is 1 month that means I should increase the number of rows for the daily (5 min average) RRA to 26280 ? Currently I have the defaults.


Thats another thing that I want to do is actually have 95th Monthly 5 minute average this would be done when modifying the daily 5 minute average rra ?

Also the billing script looks nice Rony, I will see how we could implement this.

[rony]

Well, if you still have the defaults, then the values are probably wrong. They are typically low, not high after consolidation.

[sys0p]

Well, if you still have the defaults, then the values are probably wrong. They are typically low, not high after consolidation.

So what should I change them for ?

[rony]

Read the section on RRA definitions in the ISP Billing Document...

I will eventually write a section like this on the documentation site.

[sys0p]

Read the section on RRA definitions in the ISP Billing Document...

I will eventually write a section like this on the documentation site.

I have read it and updated this thread with :

Billing interval is 1 month that means I should increase the number of rows for the daily (5 min average) RRA to 26280 ? Currently I have the defaults.


am I correct ?

Thank You Rony !

[rony]

Yes, that is correct. Sorry don't recall the numbers off the top of my head, had to reference the document..

[sys0p]

Yes, that is correct. Sorry don't recall the numbers off the top of my head, had to reference the document..

Hello Rony ,

I have modified my RRA definitions to read :
Daily (5 Minute Average) 1 26280 86400 Delete
Weekly (30 Minute Average) 6 700 604800 Delete
Monthly (2 Hour Average) 24 775 2678400 Delete
Yearly (1 Day Average) 288 797 33053184

and in one day I see more than 288 datapoints ( 5minute average- 12 datapoints per hour X 24 )

Do you know what am I doing wrong ? I am just trying to get Monthly 95th percentile figures on 5 minute average.(not 30 minutes)

Thank You !

[rony]

If these are existing RRDtool files, you need to either tune them to the new parameters, or delete them and let Cacti recreate them. Cacti will not modify existing RRDtool files structures.

[sys0p]

If these are existing RRDtool files, you need to either tune them to the new parameters, or delete them and let Cacti recreate them. Cacti will not modify existing RRDtool files structures.

I have recreated all of them using those rra definitions, my rrd's look like this(this will be a long paste) :

rrd_version = "0001"
step = 300
last_update = 1181585401
ds[traffic_in].type = "COUNTER"
ds[traffic_in].minimal_heartbeat = 600
ds[traffic_in].min = 0.0000000000e+00
ds[traffic_in].max = 1.0000000000e+10
ds[traffic_in].last_ds = "222644973357355"
ds[traffic_in].value = 4.1254068053e+07
ds[traffic_in].unknown_sec = 0
ds[traffic_out].type = "COUNTER"
ds[traffic_out].minimal_heartbeat = 600
ds[traffic_out].min = 0.0000000000e+00
ds[traffic_out].max = 1.0000000000e+10
ds[traffic_out].last_ds = "1225641184107749"
ds[traffic_out].value = 1.7303946837e+08
ds[traffic_out].unknown_sec = 0
rra[0].cf = "AVERAGE"
rra[0].rows = 700
rra[0].pdp_per_row = 6
rra[0].xff = 5.0000000000e-01
rra[0].cdp_prep[0].value = 8.2833425082e+07
rra[0].cdp_prep[0].unknown_datapoints = 0
rra[0].cdp_prep[1].value = 3.5107360881e+08
rra[0].cdp_prep[1].unknown_datapoints = 0
rra[1].cf = "AVERAGE"
rra[1].rows = 775
rra[1].pdp_per_row = 24
rra[1].xff = 5.0000000000e-01
rra[1].cdp_prep[0].value = 8.2833425082e+07
rra[1].cdp_prep[0].unknown_datapoints = 0
rra[1].cdp_prep[1].value = 3.5107360881e+08
rra[1].cdp_prep[1].unknown_datapoints = 0
rra[2].cf = "AVERAGE"
rra[2].rows = 26280
rra[2].pdp_per_row = 1
rra[2].xff = 5.0000000000e-01
rra[2].cdp_prep[0].value = NaN
rra[2].cdp_prep[0].unknown_datapoints = 0
rra[2].cdp_prep[1].value = NaN
rra[2].cdp_prep[1].unknown_datapoints = 0
rra[3].cf = "AVERAGE"
rra[3].rows = 797
rra[3].pdp_per_row = 288
rra[3].xff = 5.0000000000e-01
rra[3].cdp_prep[0].value = 5.5604264612e+09
rra[3].cdp_prep[0].unknown_datapoints = 0
rra[3].cdp_prep[1].value = 2.5009818923e+10
rra[3].cdp_prep[1].unknown_datapoints = 0
rra[4].cf = "MIN"
rra[4].rows = 700
rra[4].pdp_per_row = 6
rra[4].xff = 5.0000000000e-01
rra[4].cdp_prep[0].value = 4.1255157309e+07
rra[4].cdp_prep[0].unknown_datapoints = 0
rra[4].cdp_prep[1].value = 1.7305611821e+08
rra[4].cdp_prep[1].unknown_datapoints = 0
rra[5].cf = "MIN"
rra[5].rows = 775
rra[5].pdp_per_row = 24
rra[5].xff = 5.0000000000e-01
rra[5].cdp_prep[0].value = 4.1255157309e+07
rra[5].cdp_prep[0].unknown_datapoints = 0
rra[5].cdp_prep[1].value = 1.7305611821e+08
rra[5].cdp_prep[1].unknown_datapoints = 0
rra[6].cf = "MIN"
rra[6].rows = 26280
rra[6].pdp_per_row = 1
rra[6].xff = 5.0000000000e-01
rra[6].cdp_prep[0].value = NaN
rra[6].cdp_prep[0].unknown_datapoints = 0
rra[6].cdp_prep[1].value = NaN
rra[6].cdp_prep[1].unknown_datapoints = 0
rra[7].cf = "MIN"
rra[7].rows = 797
rra[7].pdp_per_row = 288
rra[7].xff = 5.0000000000e-01
rra[7].cdp_prep[0].value = 4.2423734943e+06
rra[7].cdp_prep[0].unknown_datapoints = 0
rra[7].cdp_prep[1].value = 1.5743475801e+07
rra[7].cdp_prep[1].unknown_datapoints = 0
rra[8].cf = "MAX"
rra[8].rows = 700
rra[8].pdp_per_row = 6
rra[8].xff = 5.0000000000e-01
rra[8].cdp_prep[0].value = 4.1578267773e+07
rra[8].cdp_prep[0].unknown_datapoints = 0
rra[8].cdp_prep[1].value = 1.7801749060e+08
rra[8].cdp_prep[1].unknown_datapoints = 0
rra[9].cf = "MAX"
rra[9].rows = 775
rra[9].pdp_per_row = 24
rra[9].xff = 5.0000000000e-01
rra[9].cdp_prep[0].value = 4.1578267773e+07
rra[9].cdp_prep[0].unknown_datapoints = 0
rra[9].cdp_prep[1].value = 1.7801749060e+08
rra[9].cdp_prep[1].unknown_datapoints = 0
rra[10].cf = "MAX"
rra[10].rows = 26280
rra[10].pdp_per_row = 1
rra[10].xff = 5.0000000000e-01
rra[10].cdp_prep[0].value = NaN
rra[10].cdp_prep[0].unknown_datapoints = 0
rra[10].cdp_prep[1].value = NaN
rra[10].cdp_prep[1].unknown_datapoints = 0
rra[11].cf = "MAX"
rra[11].rows = 797
rra[11].pdp_per_row = 288
rra[11].xff = 5.0000000000e-01
rra[11].cdp_prep[0].value = 4.9256313141e+07
rra[11].cdp_prep[0].unknown_datapoints = 0
rra[11].cdp_prep[1].value = 1.9986930796e+08
rra[11].cdp_prep[1].unknown_datapoints = 0
rra[12].cf = "LAST"
rra[12].rows = 700
rra[12].pdp_per_row = 6
rra[12].xff = 5.0000000000e-01
rra[12].cdp_prep[0].value = 4.1255157309e+07
rra[12].cdp_prep[0].unknown_datapoints = 0
rra[12].cdp_prep[1].value = 1.7305611821e+08
rra[12].cdp_prep[1].unknown_datapoints = 0
rra[13].cf = "LAST"
rra[13].rows = 775
rra[13].pdp_per_row = 24
rra[13].xff = 5.0000000000e-01
rra[13].cdp_prep[0].value = 4.1255157309e+07
rra[13].cdp_prep[0].unknown_datapoints = 0
rra[13].cdp_prep[1].value = 1.7305611821e+08
rra[13].cdp_prep[1].unknown_datapoints = 0
rra[14].cf = "LAST"
rra[14].rows = 26280
rra[14].pdp_per_row = 1
rra[14].xff = 5.0000000000e-01
rra[14].cdp_prep[0].value = NaN
rra[14].cdp_prep[0].unknown_datapoints = 0
rra[14].cdp_prep[1].value = NaN
rra[14].cdp_prep[1].unknown_datapoints = 0
rra[15].cf = "LAST"
rra[15].rows = 797
rra[15].pdp_per_row = 288
rra[15].xff = 5.0000000000e-01
rra[15].cdp_prep[0].value = 4.1255157309e+07
rra[15].cdp_prep[0].unknown_datapoints = 0
rra[15].cdp_prep[1].value = 1.7305611821e+08
rra[15].cdp_prep[1].unknown_datapoints = 0

What am I doing wrong ? i see alot more datapoints than what I need 288/day to get monthly 95th percentile 5 minute average.
or am I missing something ?

[rony]

Rows for the 5 minute average should be 26,280 for 3 months of 5 minute data retention.

Please check out Chapter 2 Sub Section II in the ISP Billing documentation located: http://forums.cacti.net/viewtopic.php?t=19887

[sys0p]

Rows for the 5 minute average should be 26,280 for 3 months of 5 minute data retention.

Please check out Chapter 2 Sub Section II in the ISP Billing documentation located: http://forums.cacti.net/viewtopic.php?t=19887

So is there anything I did wrong looking at my rra definitions and rrd output ?

[rony]

Can you post a screen shot of your rra screen?