verify 95th percentile value

Post general support questions here that do not specifically fall into the Linux or Windows categories.

Moderators: Developers, Moderators

sys0p
Posts: 48
Joined: Sun Jan 14, 2007 3:46 pm
Location: Europe

verify 95th percentile value

Post by sys0p »

Hello ,

I am having a dispute with someone who is saying that my 95th percentile graph value is not correct.

So right now I ended up trying to manually calculate the 95th percentile for the last month.
In order to do this I have dumped the data from the RRD with rrdump and extracted with awk the all the values, 5 minute average,30 minute and 2 hours.
I have did this the old dirty way, I throwed the values for iN/OUT in an xls and took out the 95th percentile values and choosed the biggest one.

Still the calculations are not correct, can someone please enlighten me what am I doing wrong ? maybe there is another way to fetch the data from the rrd file ? the formula I am sure is correct( I have tested it for one day only and the values match).

Thank You !
User avatar
BSOD2600
Cacti Moderator
Posts: 12171
Joined: Sat May 08, 2004 12:44 pm
Location: USA

Post by BSOD2600 »

If you're using standard rrd row settings, then you're likely experiencing data consolidation, which is throwing off your calculations.

Read http://docs.cacti.net/node/275 and http://docs.cacti.net/node/54
User avatar
rony
Developer/Forum Admin
Posts: 6022
Joined: Mon Nov 17, 2003 6:35 pm
Location: Michigan, USA
Contact:

Post by rony »

Is this a single interface graph (IN and OUT) traffic?

What is the specific dispute? To low, to high? Usually to high.. :)

Are your RRDTool files (and RRA settings) configured to store more than 1 month of 5 minute averages? BSOD linked some information above.

Check my ISP billing documentation for guideline on RRA definitions at http://forums.cacti.net/viewtopic.php?t ... highlight=
[size=117][i][b]Tony Roman[/b][/i][/size]
[size=84][i]Experience is what causes a person to make new mistakes instead of old ones.[/i][/size]
[size=84][i]There are only 3 way to complete a project: Good, Fast or Cheap, pick two.[/i][/size]
[size=84][i]With age comes wisdom, what you choose to do with it determines whether or not you are wise.[/i][/size]
sys0p
Posts: 48
Joined: Sun Jan 14, 2007 3:46 pm
Location: Europe

Post by sys0p »

rony wrote:Is this a single interface graph (IN and OUT) traffic?

What is the specific dispute? To low, to high? Usually to high.. :)

Are your RRDTool files (and RRA settings) configured to store more than 1 month of 5 minute averages? BSOD linked some information above.

Check my ISP billing documentation for guideline on RRA definitions at http://forums.cacti.net/viewtopic.php?t ... highlight=
Thank You both for your feedback , rony: yes it is a single interface IN/OUT and indeed they say it is too high.. (:

Billing interval is 1 month that means I should increase the number of rows for the daily (5 min average) RRA to 26280 ? Currently I have the defaults.


Thats another thing that I want to do is actually have 95th Monthly 5 minute average this would be done when modifying the daily 5 minute average rra ?

Also the billing script looks nice Rony, I will see how we could implement this.
User avatar
rony
Developer/Forum Admin
Posts: 6022
Joined: Mon Nov 17, 2003 6:35 pm
Location: Michigan, USA
Contact:

Post by rony »

Well, if you still have the defaults, then the values are probably wrong. They are typically low, not high after consolidation.
[size=117][i][b]Tony Roman[/b][/i][/size]
[size=84][i]Experience is what causes a person to make new mistakes instead of old ones.[/i][/size]
[size=84][i]There are only 3 way to complete a project: Good, Fast or Cheap, pick two.[/i][/size]
[size=84][i]With age comes wisdom, what you choose to do with it determines whether or not you are wise.[/i][/size]
sys0p
Posts: 48
Joined: Sun Jan 14, 2007 3:46 pm
Location: Europe

Post by sys0p »

rony wrote:Well, if you still have the defaults, then the values are probably wrong. They are typically low, not high after consolidation.
So what should I change them for ?
User avatar
rony
Developer/Forum Admin
Posts: 6022
Joined: Mon Nov 17, 2003 6:35 pm
Location: Michigan, USA
Contact:

Post by rony »

Read the section on RRA definitions in the ISP Billing Document...

I will eventually write a section like this on the documentation site.
[size=117][i][b]Tony Roman[/b][/i][/size]
[size=84][i]Experience is what causes a person to make new mistakes instead of old ones.[/i][/size]
[size=84][i]There are only 3 way to complete a project: Good, Fast or Cheap, pick two.[/i][/size]
[size=84][i]With age comes wisdom, what you choose to do with it determines whether or not you are wise.[/i][/size]
sys0p
Posts: 48
Joined: Sun Jan 14, 2007 3:46 pm
Location: Europe

Post by sys0p »

rony wrote:Read the section on RRA definitions in the ISP Billing Document...

I will eventually write a section like this on the documentation site.
I have read it and updated this thread with :

Billing interval is 1 month that means I should increase the number of rows for the daily (5 min average) RRA to 26280 ? Currently I have the defaults.


am I correct ?

Thank You Rony !
User avatar
rony
Developer/Forum Admin
Posts: 6022
Joined: Mon Nov 17, 2003 6:35 pm
Location: Michigan, USA
Contact:

Post by rony »

Yes, that is correct. Sorry don't recall the numbers off the top of my head, had to reference the document.. ;)
[size=117][i][b]Tony Roman[/b][/i][/size]
[size=84][i]Experience is what causes a person to make new mistakes instead of old ones.[/i][/size]
[size=84][i]There are only 3 way to complete a project: Good, Fast or Cheap, pick two.[/i][/size]
[size=84][i]With age comes wisdom, what you choose to do with it determines whether or not you are wise.[/i][/size]
sys0p
Posts: 48
Joined: Sun Jan 14, 2007 3:46 pm
Location: Europe

Post by sys0p »

rony wrote:Yes, that is correct. Sorry don't recall the numbers off the top of my head, had to reference the document.. ;)
Hello Rony ,

I have modified my RRA definitions to read :
Daily (5 Minute Average) 1 26280 86400 Delete
Weekly (30 Minute Average) 6 700 604800 Delete
Monthly (2 Hour Average) 24 775 2678400 Delete
Yearly (1 Day Average) 288 797 33053184

and in one day I see more than 288 datapoints ( 5minute average- 12 datapoints per hour X 24 )

Do you know what am I doing wrong ? I am just trying to get Monthly 95th percentile figures on 5 minute average.(not 30 minutes)

Thank You !
User avatar
rony
Developer/Forum Admin
Posts: 6022
Joined: Mon Nov 17, 2003 6:35 pm
Location: Michigan, USA
Contact:

Post by rony »

If these are existing RRDtool files, you need to either tune them to the new parameters, or delete them and let Cacti recreate them. Cacti will not modify existing RRDtool files structures.
[size=117][i][b]Tony Roman[/b][/i][/size]
[size=84][i]Experience is what causes a person to make new mistakes instead of old ones.[/i][/size]
[size=84][i]There are only 3 way to complete a project: Good, Fast or Cheap, pick two.[/i][/size]
[size=84][i]With age comes wisdom, what you choose to do with it determines whether or not you are wise.[/i][/size]
sys0p
Posts: 48
Joined: Sun Jan 14, 2007 3:46 pm
Location: Europe

Post by sys0p »

rony wrote:If these are existing RRDtool files, you need to either tune them to the new parameters, or delete them and let Cacti recreate them. Cacti will not modify existing RRDtool files structures.
I have recreated all of them using those rra definitions, my rrd's look like this(this will be a long paste) :

rrd_version = "0001"
step = 300
last_update = 1181585401
ds[traffic_in].type = "COUNTER"
ds[traffic_in].minimal_heartbeat = 600
ds[traffic_in].min = 0.0000000000e+00
ds[traffic_in].max = 1.0000000000e+10
ds[traffic_in].last_ds = "222644973357355"
ds[traffic_in].value = 4.1254068053e+07
ds[traffic_in].unknown_sec = 0
ds[traffic_out].type = "COUNTER"
ds[traffic_out].minimal_heartbeat = 600
ds[traffic_out].min = 0.0000000000e+00
ds[traffic_out].max = 1.0000000000e+10
ds[traffic_out].last_ds = "1225641184107749"
ds[traffic_out].value = 1.7303946837e+08
ds[traffic_out].unknown_sec = 0
rra[0].cf = "AVERAGE"
rra[0].rows = 700
rra[0].pdp_per_row = 6
rra[0].xff = 5.0000000000e-01
rra[0].cdp_prep[0].value = 8.2833425082e+07
rra[0].cdp_prep[0].unknown_datapoints = 0
rra[0].cdp_prep[1].value = 3.5107360881e+08
rra[0].cdp_prep[1].unknown_datapoints = 0
rra[1].cf = "AVERAGE"
rra[1].rows = 775
rra[1].pdp_per_row = 24
rra[1].xff = 5.0000000000e-01
rra[1].cdp_prep[0].value = 8.2833425082e+07
rra[1].cdp_prep[0].unknown_datapoints = 0
rra[1].cdp_prep[1].value = 3.5107360881e+08
rra[1].cdp_prep[1].unknown_datapoints = 0
rra[2].cf = "AVERAGE"
rra[2].rows = 26280
rra[2].pdp_per_row = 1
rra[2].xff = 5.0000000000e-01
rra[2].cdp_prep[0].value = NaN
rra[2].cdp_prep[0].unknown_datapoints = 0
rra[2].cdp_prep[1].value = NaN
rra[2].cdp_prep[1].unknown_datapoints = 0
rra[3].cf = "AVERAGE"
rra[3].rows = 797
rra[3].pdp_per_row = 288
rra[3].xff = 5.0000000000e-01
rra[3].cdp_prep[0].value = 5.5604264612e+09
rra[3].cdp_prep[0].unknown_datapoints = 0
rra[3].cdp_prep[1].value = 2.5009818923e+10
rra[3].cdp_prep[1].unknown_datapoints = 0
rra[4].cf = "MIN"
rra[4].rows = 700
rra[4].pdp_per_row = 6
rra[4].xff = 5.0000000000e-01
rra[4].cdp_prep[0].value = 4.1255157309e+07
rra[4].cdp_prep[0].unknown_datapoints = 0
rra[4].cdp_prep[1].value = 1.7305611821e+08
rra[4].cdp_prep[1].unknown_datapoints = 0
rra[5].cf = "MIN"
rra[5].rows = 775
rra[5].pdp_per_row = 24
rra[5].xff = 5.0000000000e-01
rra[5].cdp_prep[0].value = 4.1255157309e+07
rra[5].cdp_prep[0].unknown_datapoints = 0
rra[5].cdp_prep[1].value = 1.7305611821e+08
rra[5].cdp_prep[1].unknown_datapoints = 0
rra[6].cf = "MIN"
rra[6].rows = 26280
rra[6].pdp_per_row = 1
rra[6].xff = 5.0000000000e-01
rra[6].cdp_prep[0].value = NaN
rra[6].cdp_prep[0].unknown_datapoints = 0
rra[6].cdp_prep[1].value = NaN
rra[6].cdp_prep[1].unknown_datapoints = 0
rra[7].cf = "MIN"
rra[7].rows = 797
rra[7].pdp_per_row = 288
rra[7].xff = 5.0000000000e-01
rra[7].cdp_prep[0].value = 4.2423734943e+06
rra[7].cdp_prep[0].unknown_datapoints = 0
rra[7].cdp_prep[1].value = 1.5743475801e+07
rra[7].cdp_prep[1].unknown_datapoints = 0
rra[8].cf = "MAX"
rra[8].rows = 700
rra[8].pdp_per_row = 6
rra[8].xff = 5.0000000000e-01
rra[8].cdp_prep[0].value = 4.1578267773e+07
rra[8].cdp_prep[0].unknown_datapoints = 0
rra[8].cdp_prep[1].value = 1.7801749060e+08
rra[8].cdp_prep[1].unknown_datapoints = 0
rra[9].cf = "MAX"
rra[9].rows = 775
rra[9].pdp_per_row = 24
rra[9].xff = 5.0000000000e-01
rra[9].cdp_prep[0].value = 4.1578267773e+07
rra[9].cdp_prep[0].unknown_datapoints = 0
rra[9].cdp_prep[1].value = 1.7801749060e+08
rra[9].cdp_prep[1].unknown_datapoints = 0
rra[10].cf = "MAX"
rra[10].rows = 26280
rra[10].pdp_per_row = 1
rra[10].xff = 5.0000000000e-01
rra[10].cdp_prep[0].value = NaN
rra[10].cdp_prep[0].unknown_datapoints = 0
rra[10].cdp_prep[1].value = NaN
rra[10].cdp_prep[1].unknown_datapoints = 0
rra[11].cf = "MAX"
rra[11].rows = 797
rra[11].pdp_per_row = 288
rra[11].xff = 5.0000000000e-01
rra[11].cdp_prep[0].value = 4.9256313141e+07
rra[11].cdp_prep[0].unknown_datapoints = 0
rra[11].cdp_prep[1].value = 1.9986930796e+08
rra[11].cdp_prep[1].unknown_datapoints = 0
rra[12].cf = "LAST"
rra[12].rows = 700
rra[12].pdp_per_row = 6
rra[12].xff = 5.0000000000e-01
rra[12].cdp_prep[0].value = 4.1255157309e+07
rra[12].cdp_prep[0].unknown_datapoints = 0
rra[12].cdp_prep[1].value = 1.7305611821e+08
rra[12].cdp_prep[1].unknown_datapoints = 0
rra[13].cf = "LAST"
rra[13].rows = 775
rra[13].pdp_per_row = 24
rra[13].xff = 5.0000000000e-01
rra[13].cdp_prep[0].value = 4.1255157309e+07
rra[13].cdp_prep[0].unknown_datapoints = 0
rra[13].cdp_prep[1].value = 1.7305611821e+08
rra[13].cdp_prep[1].unknown_datapoints = 0
rra[14].cf = "LAST"
rra[14].rows = 26280
rra[14].pdp_per_row = 1
rra[14].xff = 5.0000000000e-01
rra[14].cdp_prep[0].value = NaN
rra[14].cdp_prep[0].unknown_datapoints = 0
rra[14].cdp_prep[1].value = NaN
rra[14].cdp_prep[1].unknown_datapoints = 0
rra[15].cf = "LAST"
rra[15].rows = 797
rra[15].pdp_per_row = 288
rra[15].xff = 5.0000000000e-01
rra[15].cdp_prep[0].value = 4.1255157309e+07
rra[15].cdp_prep[0].unknown_datapoints = 0
rra[15].cdp_prep[1].value = 1.7305611821e+08
rra[15].cdp_prep[1].unknown_datapoints = 0

What am I doing wrong ? i see alot more datapoints than what I need 288/day to get monthly 95th percentile 5 minute average.
or am I missing something ?
User avatar
rony
Developer/Forum Admin
Posts: 6022
Joined: Mon Nov 17, 2003 6:35 pm
Location: Michigan, USA
Contact:

Post by rony »

Rows for the 5 minute average should be 26,280 for 3 months of 5 minute data retention.

Please check out Chapter 2 Sub Section II in the ISP Billing documentation located: http://forums.cacti.net/viewtopic.php?t=19887
[size=117][i][b]Tony Roman[/b][/i][/size]
[size=84][i]Experience is what causes a person to make new mistakes instead of old ones.[/i][/size]
[size=84][i]There are only 3 way to complete a project: Good, Fast or Cheap, pick two.[/i][/size]
[size=84][i]With age comes wisdom, what you choose to do with it determines whether or not you are wise.[/i][/size]
sys0p
Posts: 48
Joined: Sun Jan 14, 2007 3:46 pm
Location: Europe

Post by sys0p »

rony wrote:Rows for the 5 minute average should be 26,280 for 3 months of 5 minute data retention.

Please check out Chapter 2 Sub Section II in the ISP Billing documentation located: http://forums.cacti.net/viewtopic.php?t=19887
So is there anything I did wrong looking at my rra definitions and rrd output ?
User avatar
rony
Developer/Forum Admin
Posts: 6022
Joined: Mon Nov 17, 2003 6:35 pm
Location: Michigan, USA
Contact:

Post by rony »

Can you post a screen shot of your rra screen?
[size=117][i][b]Tony Roman[/b][/i][/size]
[size=84][i]Experience is what causes a person to make new mistakes instead of old ones.[/i][/size]
[size=84][i]There are only 3 way to complete a project: Good, Fast or Cheap, pick two.[/i][/size]
[size=84][i]With age comes wisdom, what you choose to do with it determines whether or not you are wise.[/i][/size]
Post Reply

Who is online

Users browsing this forum: No registered users and 3 guests