values stored in RRD database

acceleratebiz · Post by **acceleratebiz** » Sat Jan 21, 2006 1:27 am

I'm needing to calculate my 95 percentile by hand for now (discussed in another thread). So I did an xml export of a particular RRD file, but just need clarification on what the values are.

In the xml:
<rrd>
<rra>
<database>

<row>
<v> 7.2373984163e+06 </v>
<v> 1.3080255702e+06 </v>
</row>

<row>
<v> 8.0848807969e+06 </v>
<v> 1.2078303745e+06 </v>
</row>

<row>
<v> 8.4168070302e+06 </v>
<v> 1.1894149480e+06 </v>
</row>

<row>
<v> 8.3684057149e+06 </v>
<v> 1.7527794035e+06 </v>
</row>

<row>
<v> 7.2132144539e+06 </v>
<v> 1.5155669575e+06 </v>
</row>

now that appears simple. I imagine, for example, the last number (1.5155669575e+06) is 1,515,566 bytes or octets/sec. Can't be bits/sec because that number is too low then (customer was using more). However, according to cacti's graph, this customer did not use 60mbps incoming traffic (from the interface's perspective) or more as these numbers would indicate. The graph shows the max for the month was 50.55mbps and for that time period, visually it looks like it should be 40mbps or possibly close to 45mbps. So what are these numbers in the <v> element?

Post by **gandalf** » Sat Jan 21, 2006 5:09 am

This is more or less a rrdtool related question. So you should have a look at http://www.rrdtool.org to get some ideas
Reinhard

acceleratebiz · Post by **acceleratebiz** » Sat Jan 21, 2006 7:13 am

It's not cacti that stores the data in the RRD database? How does cacti graph these numbers?

I'll also try looking at rrdtool.org, thanks.

Post by **gandalf** » Sat Jan 21, 2006 7:44 am

Sorry, I did not want to offend you.
Yes, cacti retrieves all data and feeds them into rrdtool "databases" by means of rrdtool update. You can see this working: Goto Settings and select Logging Level DEBUG. Then, please have a look at log/cacti.log. You will see all the value gathering and some line below, you'll find the rrdtool update of the raw values.

Due to principles of operation of rrdtool, data storage is somewhat "special". To understand, please see the discussion of COUNTER and GAUGE at rrd-beginners. And there's another specialty, the consolidation. For this, see rrdtutorial and some of the links in my signature. Additional resource: Tutorials and explanations of Alex van den Bogaerdt.
Your example show data from the monthly consolidated rra as can be deduced from the timestamps. If you want to achieve some results concerning the 95th percentile, it's important to know, how this rra is defined (e.g. rrdtool info <filename>), e.g. is it an AVERAGE cf or a MAXIMUM cf ...
If you're running a rrdtool 1.2.x version, you may want to play around with the VDEF function, that provides some PERCENTage functionality, e.g. VDEF:<name>=<ds>,PERCENT,95 for a 95th Percentile. This may yield some interesting results for you without lots of programming. I wrote some code for cacti 0.8.6-pre-h (as of December 2005) to include this, but it was not yet released and is too big to be published as a patch.
Hope this gives you some more starting points for your work
Reinhard

acceleratebiz · Post by **acceleratebiz** » Sat Jan 21, 2006 12:21 pm

Hi Reinhard, why would you be offending me? I'll try out the debug logs. Those values are from the AVERAGE cf.

I understand the basic principles of how rrdtool works. The only thing I don't understand are the raw values. What is the bandwidth value those raw values correspond to in the .rrd files? Do I multiply it by 4 for the bps? is the e+06 something other than 10^6?

Post by **gandalf** » Sun Jan 22, 2006 5:55 am

acceleratebiz wrote:...Those values are from the AVERAGE cf.

Fine. For your purpose, this is vital.

...The only thing I don't understand are the raw values. What is the bandwidth value those raw values correspond to in the .rrd files?

That depends on the exact OID you're using. I suppose, you're running standard traffic queries? Then, you may find those oids in resource/snmp_queries/interface.xml. You'll find OID=.1.3.6.1.2.1.2.2.1.10 for ifInOctets, but OID=.1.3.6.1.2.1.31.1.1.1.6 for ifHCInOctets, the 64 bit counters. The correct meaning must be derived from the MIB definition. See Management Information Base for Network Management of TCP/IP-based internets: MIB-II:

Code: Select all

ifInOctets OBJECT-TYPE
              SYNTAX  Counter
              ACCESS  read-only
              STATUS  mandatory
              DESCRIPTION
                      "The total number of octets received on the
                      interface, including framing characters."
              ::= { ifEntry 10 }

implemented as a COUNTER increasing with the start of the snmp agent.

... is the e+06 something other than 10^6?

No, nothing strange. It is an exponent notation.
Again, I'd recommend the use of VDEF as stated above. Why? It seems easier to me to let rrdtool do the maths (as long as they are correct. If not: please post an error to the rrdtool-users mailing list. Help is given very quickly)!

Reinhard

acceleratebiz · Post by **acceleratebiz** » Sun Jan 22, 2006 11:33 pm

Hi Reinhard, thanks so much for the help. I really just need something very simple. I'm very familiar with snmp and the various interface counters. All I need to know is what the <v> elements in the rrd xml dumps represent. These are .rrd files cacti generates for a switch interface for bps traffic ("SNMP - Interface Statistics" data query). I assume cacti uses the ifInOctets OID (.1.3.6.1.2.1.2.2.1.10) and ifOutOctets to store this information. But they are clearly not exactly the ifInOctet and ifOutOctet raw data because the numbers fluctuate, the raw data is always increasing.

Are the numbers bits/sec? bytes/sec? number of bytes transferred during the time interval? Not really sure and nothing obvious makes sense based on the visual graphs cacti shows.

If anybody can tell me the simple answer I'd be so happy.

I need to do the math myself (not very difficult at all) because the .rrd files are old, can't go back in time.

Post by **gandalf** » Mon Jan 23, 2006 1:14 pm

Ok, got it. The readings for ifOctectAnything are, of course, Octets <-> Bytes. The reading is a COUNTER (permanently increasing, as you stated correctly). This Byte Counter is fed to rrdtool, together with a timestamp. The way rrdtool handles COUNTERS is to store only the difference between the actual and the last reading, adjusted to (devided by) the step value. As the data is Bytes, the differences are bytes, too. But take care of time jitter. rrdtool provides a method to cope with that in a way, that it takes the difference between the actual and the last timestamp into account. That is: if data is fed not on exact 300 sec but e.g. after 305 sec, data will be adjusted by multiplying with 300/305. So, to sum up, it's a rate in bytes/sec, corrected for jitter.

Taken from http://people.ee.ethz.ch/~oetiker/webto ... rs.en.html:
DST (Data Source Type) defines the type of the DS. It can be COUNTER, DERIVE, ABSOLUTE, GAUGE. A DS declared as COUNTER will save the rate of change of the value over a step period. This assumes that the value is always increasing (the difference between the current and the previous value is greater than 0). Traffic counters on a router are an ideal candidate for using COUNTER as DST. DERIVE is the same as COUNTER, but it allows negative values as well. If you want to see the rate of change in free diskspace on your server, then you might want to use the DERIVE data type. ABSOLUTE also saves the rate of change, but it assumes that the previous value is set to 0. The difference between the current and the previous value is always equal to the current value. Thus it just stores the current value divided by the step interval (300 seconds in our example). GAUGE does not save the rate of change. It saves the actual value itself. There are no divisions or calculations. Memory consumption in a server is a typical example of gauge. The difference between the different types DSTs can be explained better with the following example:

Code: Select all

Values       = 300, 600, 900, 1200
Step         = 300 seconds
COUNTER DS   =    1,  1,   1,    1
DERIVE DS    =    1,  1,   1,    1
ABSOLUTE DS  =    1,  2,   3,    4
GAUGE DS     = 300, 600, 900, 1200

hth
Reinhard

acceleratebiz · Post by **acceleratebiz** » Mon Jan 23, 2006 1:24 pm

Thanks Reinhard, that's what I was looking for. It is a counter as the beginning of the xml dump is:
<rrd>
<ds>
<name> traffic_in </name>
<type> COUNTER </type>

same for traffic_out. Ok, so the <v> elements are simply the number of bytes that flowed through the interface during the time interval between the last measurement and that measurement.

Thanks again.

Post by **gandalf** » Mon Jan 23, 2006 2:46 pm

acceleratebiz wrote:... Ok, so the <v> elements are simply the number of bytes that flowed through the interface during the time interval between the last measurement and that measurement.

Not exactly: they are the number of bytes that flowed through the interface during the time interval between the last measurement and that measurement normalized to a second
Reinhard

acceleratebiz · Post by **acceleratebiz** » Mon Jan 23, 2006 3:42 pm

normalized to a second? so you mean if, for example, the interval is 300.2 seconds and I transferred 300.2 bytes during that time... then cacti would normalize it to 300 bytes instead of 300.2?

Doesn't seem like I really need to worry about that much. Or am I not understanding "normalized to a second?"

Post by **gandalf** » Mon Jan 23, 2006 4:09 pm

It means:
If you got 300 Bytes transferred in 300 secs, rrdtool stores 1
If you got 600 Bytes transferred in 600 secs, rrdtool stores 2 (see the example above!)
If you got 301 Bytes transferred in 301 secs, rrdtool again stores 1 as there are _exactly_ 1 Bytes/seconds transferred
Reinhard

acceleratebiz · Post by **acceleratebiz** » Mon Jan 23, 2006 4:52 pm

hmm.. so I understand it's bytes/sec. The thing is the values just don't make sense. For example, the values for traffic_in would suggest 12mbps maximum usage for all 3 days of 12/1/05 - 12/3/05. But the GRAPH shows that usage barely went below 20mbps at all during those days. The spikes don't even seem to correspond to the correct places. In fact, I can't find where this customer even used only 12mbps or less in any date around these days. Below are the values for these 3 days and I'm attaching the associated graph. I've triple checked that the xml dump is for the correct .rrd file.

 <row><v> 7.7716054292e+05 </v><v> 2.0710391694e+05 </v></row>
 <row><v> 5.8989504097e+05 </v><v> 2.7764062333e+05 </v></row>
 <row><v> 6.3496955412e+05 </v><v> 2.4216401335e+05 </v></row>
 <row><v> 8.8398377672e+05 </v><v> 2.6547934332e+05 </v></row>
 <row><v> 1.0114003340e+06 </v><v> 2.9806132369e+05 </v></row>
 <row><v> 1.4359610599e+06 </v><v> 5.0021449992e+05 </v></row>
 <row><v> 1.5936061029e+06 </v><v> 5.9943484857e+05 </v></row>
 <row><v> 1.5812199472e+06 </v><v> 5.8677147060e+05 </v></row>
 <row><v> 1.5025644074e+06 </v><v> 5.4979179153e+05 </v></row>
 <row><v> 1.2402488506e+06 </v><v> 4.2846166583e+05 </v></row>
 <row><v> 9.7692227403e+05 </v><v> 2.6980788528e+05 </v></row>
 <row><v> 8.9087687667e+05 </v><v> 2.9679333792e+05 </v></row>
 <row><v> 8.0151858389e+05 </v><v> 2.2125743681e+05 </v></row>
 <row><v> 5.6835054566e+05 </v><v> 2.0643489336e+05 </v></row>
 <row><v> 5.6335214240e+05 </v><v> 1.4679106539e+05 </v></row>
 <row><v> 6.1116198361e+05 </v><v> 1.5885624347e+05 </v></row>
 <row><v> 9.4692792381e+05 </v><v> 2.4440032289e+05 </v></row>
 <row><v> 1.4904674479e+06 </v><v> 4.8290254058e+05 </v></row>
 <row><v> 1.5536641800e+06 </v><v> 5.6036929838e+05 </v></row>
 <row><v> 1.5369923524e+06 </v><v> 5.9414011503e+05 </v></row>
 <row><v> 1.4527823902e+06 </v><v> 5.3276864958e+05 </v></row>
 <row><v> 1.2954282902e+06 </v><v> 3.7595753117e+05 </v></row>
 <row><v> 9.7878360833e+05 </v><v> 2.2205321500e+05 </v></row>
 <row><v> 1.1392233979e+06 </v><v> 1.9000440792e+05 </v></row>
 <row><v> 6.3368441917e+05 </v><v> 1.6856520028e+05 </v></row>
 <row><v> 6.7579069738e+05 </v><v> 1.2269705114e+05 </v></row>
 <row><v> 5.2705069068e+05 </v><v> 1.0521365899e+05 </v></row>
 <row><v> 5.4726489847e+05 </v><v> 8.6527462639e+04 </v></row>
 <row><v> 6.4794965917e+05 </v><v> 1.0681821583e+05 </v></row>
 <row><v> 9.1911128278e+05 </v><v> 1.6745845217e+05 </v></row>
 <row><v> 1.0321852374e+06 </v><v> 2.2377661130e+05 </v></row>
 <row><v> 1.0288642582e+06 </v><v> 2.1851523528e+05 </v></row>
 <row><v> 9.2732970167e+05 </v><v> 1.8093429681e+05 </v></row>
 <row><v> 8.7537554361e+05 </v><v> 1.6958170764e+05 </v></row>
 <row><v> 7.7587812792e+05 </v><v> 1.2967617889e+05 </v></row>
 <row><v> 7.4348643319e+05 </v><v> 1.2468322806e+05 </v></row>

Post by **gandalf** » Tue Jan 24, 2006 12:53 pm

Would you please be so kind and click on the wrench next to the graph (please _same_ settings of start/stop time as in your picture) and post the resulting rrdtool graph statement. I will go and check the maths using some examples to find the clue
Reinhard

acceleratebiz · Post by **acceleratebiz** » Tue Jan 24, 2006 4:18 pm

Reinhard, you've been extremely helpful. i think I see the problem now. I thought I was supposed to look in the rrd file ending in 169 because the url is:
http://manager.acceleratebiz.com/traffi ... 1136091600

and there's only 5 rrd files (ending in 168-172). So the xml dump from above is for the rrd file ending in 169. However, the rrdtool graph statement you had me copy and paste here indicates it really is the 172 file:

/usr/local/rrdtool-1.2.1/bin/rrdtool graph - \
--imgformat=PNG \
--start=1133413200 \
--end=1136091600 \
--title="Internap Gigabit Edge Router - Traffic - Fa0/0.2" \
--rigid \
--base=1000 \
--height=120 \
--width=500 \
--alt-autoscale-max \
--lower-limit=0 \
COMMENT:"From 2005/12/01 00\:00\:00 To 2006/01/01 00\:00\:00\c" \
COMMENT:" \n" \
--vertical-label="bits per second" \
DEF:a="/var/www/html/traffic/rra/internap_gigabit_edge_router_traffic_in_172.rrd":traffic_in:AVERAGE \
DEF:b="/var/www/html/traffic/rra/internap_gigabit_edge_router_traffic_in_172.rrd":traffic_out:AVERAGE \
CDEF:cdefa=a,8,* \
CDEF:cdefe=b,8,* \
AREA:cdefa#00CF00:"Inbound" \
GPRINT:cdefa:LAST:" Current\:%8.2lf %s" \
GPRINT:cdefa:AVERAGE:"Average\:%8.2lf %s" \
GPRINT:cdefa:MAX:"Maximum\:%8.2lf %s\n" \
LINE1:cdefe#002A97:"Outbound" \
GPRINT:cdefe:LAST:"Current\:%8.2lf %s" \
GPRINT:cdefe:AVERAGE:"Average\:%8.2lf %s" \
GPRINT:cdefe:MAX:"Maximum\:%8.2lf %s\n" \
COMMENT:"\n" \
HRULE:0#FF0000:"95th Percentile" \
COMMENT:"(0 mbit in+out)"

Cacti

values stored in RRD database

values stored in RRD database

Who is online