verify 95th percentile value

[sys0p]

Ok clearly I dont know much about this rrdtool, could you explain me a bit about this, how can I get these results the way I want ? or point me to the right place in the documentation ?

You may want to consider my RRDTool Chapter at http://docs.cacti.net
Reinhard

Ok I think I got that right. I have another issue, I deleted some rra definitions and after I configured them back as they were .

Now when I go to :

Graphs -> List Mode -> <aggr02 - Traffic>


This means I use the preview view, I can only see the Daily (5 Minute Average) nothing about Monthly or Yearly.

For sure this Graph rrd have these rra defs inside.

What else do I have to change to get that working back ?



Thanks!

I solved this , I had to modify in data templates/Traffic :

Associated RRA's
This field is always templated.

[sys0p]

Hello ,

I have extracted the data from one rrd file in order to manually check it against what cacti shows .
So I did something like :

perl -e '$ctime = time; $rrdres = 300; \
system "rrdtool fetch name_traffic_in_964.rrd MAX -e @{[int($ctime/$rrdres)*$rrdres]} -s e-31d"
'>outfile

then I looked at this interval on 01 Aug between 01:00:00-01:30:00 AM




Wed 01 Aug 2007 01:00:00 AM PDT 2.8702160331e+07 1.0714571746e+06
Wed 01 Aug 2007 01:05:00 AM PDT 2.5528005545e+07 9.4028848055e+05
Wed 01 Aug 2007 01:10:00 AM PDT 2.4052456847e+07 1.0092209013e+06
Wed 01 Aug 2007 01:15:00 AM PDT 2.5298063967e+07 9.5917174977e+05
Wed 01 Aug 2007 01:20:00 AM PDT 2.5781276103e+07 8.6416381436e+05
Wed 01 Aug 2007 01:25:00 AM PDT 2.5215195558e+07 8.7841117900e+05
Wed 01 Aug 2007 01:30:00 AM PDT 2.5790253485e+07 8.6856973776e+05

Now I am attaching also the graph for that period of time




I verified the 95th percentile and its maybe off by 5 mbit but the odd thing is that by taking any time interval and the numbers do not match at all for that period of time , can someone explain why if there is anything I am doing wrong and what should I do in order to be able to verify that the figures are actually ok ?

[gandalf]

First thing that came to my mind: 1024 vs 1000 when calculating kilo's and mega's
Reinhard

[sys0p]

First thing that came to my mind: 1024 vs 1000 when calculating kilo's and mega's
Reinhard

I know that cacti's way is to do it with 1000, but even with that the numbers are far off ..

Can you explain why they are so far off and if there is anything I am missing or doing wrong ? the problem is that customers are disputing this and I cant proove it with raw data .. probably I am not the only one in this positions and others are quite interesting in howto do this manually when disputing a graph.

Thank You !

[gandalf]

I did not follow your math to the letter. So please explain, where the difference occurs
Reinhard

[sys0p]

I did not follow your math to the letter. So please explain, where the difference occurs
Reinhard

Sorry ,

at 01:00:00 AM we have 28702160.331 bits IN and 1071457.1746 OUT

at 01:05:00 AM we have 25528005.545 bits IN and 940288.48055 OUT

If you do that /1000*1000 you should get Mbits right ?

Do you see the difference now by looking at the graph ?

[gandalf]

at 01:00:00 AM we have 28702160.331 bits IN and 1071457.1746 OUT
at 01:05:00 AM we have 25528005.545 bits IN and 940288.48055 OUT

You deduced those numbers from rrdtool fetch, I suppose? And the data was grabbed using the standard traffic templates? Did you take into account, that standard IF MIB uses units of Bytes instead of Bits? This make a factor of 8. And, by default, rrdtool will report COUNTERS as rates, that is, calculated based on seconds. If polling interval is at 300 sec., COUNTER differences will be divided by 300 to get a "per second" value
Reinhard

[sys0p]

at 01:00:00 AM we have 28702160.331 bits IN and 1071457.1746 OUT
at 01:05:00 AM we have 25528005.545 bits IN and 940288.48055 OUT

You deduced those numbers from rrdtool fetch, I suppose? And the data was grabbed using the standard traffic templates? Did you take into account, that standard IF MIB uses units of Bytes instead of Bits? This make a factor of 8. And, by default, rrdtool will report COUNTERS as rates, that is, calculated based on seconds. If polling interval is at 300 sec., COUNTER differences will be divided by 300 to get a "per second" value
Reinhard

This is how I got the data :

perl -e '$ctime = time; $rrdres = 300; \
system "rrdtool fetch name_traffic_in_964.rrd MAX -e @{[int($ctime/$rrdres)*$rrdres]} -s e-31d"
'>outfile

yes 95th percentile template

so what am I doing wrong or what should I be doing to be able to get the proper values as they are on the graph from the rrd ?

[gandalf]

So you fetched the raw data from rrdtool as I supposed. For this discussion, it is of no importance, that you've fetched data from last 31 days. We only need some few data points.

What will rrdtool do, when rrdtool graph ist called?
It will compute rates from the raw data of the rrd file. To do so, it takes two consecutive pairs of data and computes the difference. This is divided by the interval (rrdtool lingo: step size) to form a rate (in this case: Bytes per second). This is done for each pair of data that lies between start and end time.
Finally, rrdtool will apply all CDEFs given (in this case, "Bytes to Bits" is used. This one will multiply by 8).
The result is printed in the graph
Reinhard

[sys0p]

So you fetched the raw data from rrdtool as I supposed. For this discussion, it is of no importance, that you've fetched data from last 31 days. We only need some few data points.

What will rrdtool do, when rrdtool graph ist called?
It will compute rates from the raw data of the rrd file. To do so, it takes two consecutive pairs of data and computes the difference. This is divided by the interval (rrdtool lingo: step size) to form a rate (in this case: Bytes per second). This is done for each pair of data that lies between start and end time.
Finally, rrdtool will apply all CDEFs given (in this case, "Bytes to Bits" is used. This one will multiply by .
The result is printed in the graph
Reinhard

Reinhard ,

Thank you for your explanation .. so for the time interval that I put before, can you explain me directly on the numbers all the steps please ? that would really make it clear for me and for others (:

For example I am not sure what you refere to when you say pair of data..

Basically in this case :

at 01:00:00 AM we have 28702160.331 bits IN and 1071457.1746 OUT

at 01:05:00 AM we have 25528005.545 bits IN and 940288.48055 OUT

and on the graph at 01:00 and 01:05 it shows 200mbit flat line, how do I make the math to match that figure ? because however I do it it seems it doesnt match ..

Thanks !



Thanks !

[gandalf]

Hmm, just thinking ...
The value is right above 114 MBps, know as the limit for 300 sec polling and use of 32 bit COUNTERs for traffic.
In this case, COUNTER wraps have to be taken into account ...
Well, it's late now, and I'm too lazy to calculate all these numbers ...
Reinhard

[sys0p]

Hmm, just thinking ...
The value is right above 114 MBps, know as the limit for 300 sec polling and use of 32 bit COUNTERs for traffic.
In this case, COUNTER wraps have to be taken into account ...
Well, it's late now, and I'm too lazy to calculate all these numbers ...
Reinhard

Maybe we can follow it up today please (:

[rony]

So you fetched the raw data from rrdtool as I supposed. For this discussion, it is of no importance, that you've fetched data from last 31 days. We only need some few data points.

What will rrdtool do, when rrdtool graph ist called?
It will compute rates from the raw data of the rrd file. To do so, it takes two consecutive pairs of data and computes the difference. This is divided by the interval (rrdtool lingo: step size) to form a rate (in this case: Bytes per second). This is done for each pair of data that lies between start and end time.
Finally, rrdtool will apply all CDEFs given (in this case, "Bytes to Bits" is used. This one will multiply by .
The result is printed in the graph
Reinhard

Reinhard ,

Thank you for your explanation .. so for the time interval that I put before, can you explain me directly on the numbers all the steps please ? that would really make it clear for me and for others (:

For example I am not sure what you refere to when you say pair of data..

Basically in this case :

at 01:00:00 AM we have 28702160.331 bits IN and 1071457.1746 OUT

at 01:05:00 AM we have 25528005.545 bits IN and 940288.48055 OUT

and on the graph at 01:00 and 01:05 it shows 200mbit flat line, how do I make the math to match that figure ? because however I do it it seems it doesnt match ..

Thanks !



Thanks !

I have seen this before... This is normal, based on the principle behind Nth Percentile calulations. As more data points are updated in the set with smaller than the top 5% highest values, the highest value drop out (Round Robin Database). This in turn lowers the Nth Percentile value.

[sys0p]

So you fetched the raw data from rrdtool as I supposed. For this discussion, it is of no importance, that you've fetched data from last 31 days. We only need some few data points.

What will rrdtool do, when rrdtool graph ist called?
It will compute rates from the raw data of the rrd file. To do so, it takes two consecutive pairs of data and computes the difference. This is divided by the interval (rrdtool lingo: step size) to form a rate (in this case: Bytes per second). This is done for each pair of data that lies between start and end time.
Finally, rrdtool will apply all CDEFs given (in this case, "Bytes to Bits" is used. This one will multiply by .
The result is printed in the graph
Reinhard

Reinhard ,

Thank you for your explanation .. so for the time interval that I put before, can you explain me directly on the numbers all the steps please ? that would really make it clear for me and for others (:

For example I am not sure what you refere to when you say pair of data..

Basically in this case :

at 01:00:00 AM we have 28702160.331 bits IN and 1071457.1746 OUT

at 01:05:00 AM we have 25528005.545 bits IN and 940288.48055 OUT

and on the graph at 01:00 and 01:05 it shows 200mbit flat line, how do I make the math to match that figure ? because however I do it it seems it doesnt match ..

Thanks !



Thanks !

I have seen this before... This is normal, based on the principle behind Nth Percentile calulations. As more data points are updated in the set with smaller than the top 5% highest values, the highest value drop out (Round Robin Database). This in turn lowers the Nth Percentile value.

Rony ,

If untill now I thought it makes some sense.. now you completely ruined all that .

Can you please be more clear how can I exactly proove the customer that the 95th percentile figure is right and values match the one's on the graph ?

Btw , where I can contact you regarding your billing script ?

[sys0p]

Can anyone please shed some light on this ? we really need to be able to calculate this also by hand following a clear rule.

Up untill now I have not seen a clear rule on how to calculate the 95th percentile figure from the rrd dump to match the one on the graph (also to match the traffic figures at a certain time on the graph with the one's from the rrd dump ) , a great ideea would be to actually put it down in a written demonstration based on real numbers.

I hope Cacti developers and Moderators give more attention to this as I think I am not the only one who is having disputes.


Thanks !