Report plugin ''ReportIt" - v0.7.5a (released 23/12/11)

[michaelwjoyner]

Andreas,
Thanks so much for your plugin!!! Today the boss said he wants a daily report showing max peak usage on our trunkgroups.

I read the manual and it was very well done. One thing though, 0.7.1 has the field, "Consolidation Function", and the examples in the PDF do not. It was a little confusing how the "Consolidation Functions" tie in, as it is not documented in the new version.

I am a little lost on the bottom functions, (f_max, f_dot, etc) but I will read all the post's in this thread, hoping to shed some light.

My application is simple, my data source has 3 fields, busy, idle, total. Total is always the same number, busy and idle fluctuate. I am monitoring DSO's in a trunkgroup.

I want to write a report, that shows maximum for Busy for the last 24 hours, and what % it is of the total field.

Could you please point me in the right direction, I will do the foot work, I just think I need a little more knowledge to learn what the functions are in creating the measurands.

THANKS SO MUCH FOR YOUR PLUGIN!!!

[browniebraun]

Hi Michael!

I read the manual and it was very well done. One thing though, 0.7.1 has the field, "Consolidation Function", and the examples in the PDF do not. It was a little confusing how the "Consolidation Functions" tie in, as it is not documented in the new version.

In 0.7.0 and higher versions the definition of the consolidation function has been moved from the report template to the measurands.
Now you can use all (max: MIN, MAX, AVERAGE, LAST) archives for your calculations within one report template. That makes it more flexible.
It's up to you which calculation should depend on one of those archives.

I am a little lost on the bottom functions, (f_max, f_dot, etc) but I will read all the post's in this thread, hoping to shed some light.

Oh dear! Don't do it!
Move your mouse over the functions and you will get a description at the bottom. e.g. f_max will return the highest value found in the amount of data read out from the archive you've defined.

I want to write a report, that shows maximum for Busy for the last 24 hours, and what % it is of the total field.

Maybe there is an easier way possible (I'm a little bit tired today, sorry!), but having that data source items I would create following measurands:

1.) Name = "Maxima", Abbreviation = "max", Unit = "-", CF = "MAX", Visible = false, Separate = false, Rounding = off, Calculation Formula = "f_max"

2.) Name = "max occupancy (absolute)", Abbreviation = "mocc", Unit = "-", CF = "MAX" (not important here), Visible = true, Separate = true, Rounding = false, Calculation Formula = "max:busy"
(As you can see, we use one of three interim results of the first measurand.)

3.) Name = max occupancy rate (relative)", Abbreviation = "rocc", Unit = "%", CF = "MAX" (not important here), Visible = true, Separate = true, Rounding = "Base 1000", Calculation Formula = "(max:busy/max:total)*100"


Best regards
-Andreas-

[michaelwjoyner]

Thanks so much Andreas. I bet you are tired. You have given 110% for this. I will take take the ball from here. I will also read up, all 38 pages of posts today. Your plug-in is awesome!!!!

[j-munroe]

Hi,

Has anyone else had their maxValue variable under measurands disappear? It was there before but since upgrading I've noticed it no longer is available?

Anyone else seen this...or more importantly know how to fix it?

Did you change the maximum value of your data template?The variable "maxValue" will only be available if "maximum" is neither empty nor zero.
Are there error messages listed in the Cacti log?

Best regards
-Andreas-

Sorry for the delay in getting back...didn't get the msg notifier...anyways moving on

After reading your post I did some investigation. And yes, you are correct. At some point my data template for interface traffic has changed. The orginal template had a max. value of 100000000. The template as it exists today is 0 for max. value. I've tried selecting the "Use Per-Data Source Value" for the traffic_in and traffic_out, which when I look at the actual data sources a maxValue gets added corresponding to the interface max. Should this not work to get a maxValue to be available in the measurand variables (interface traffic template)?

Maybe I should explain what I'm trying to achieve a little better. I'm interested in reporting of max. utilization of interfaces on a percentage basis. I was using the guide on sourceforge to set this out. But...couldn't continue when the maxValue was not present as a variable. I have thousands of 10M/100M/1000M/10G interfaces out there. What is the easiest way to accomplish what I'm looking for with min. effort?

Many Thanks in Advance!!!

[michaelwjoyner]

Do we want to run the cron entries as "cacti" or "root"?

[mrgenius]

Dear Friends

I am trying to use this Plugin to calculate 95th percentile and Total bandwidth.. However i am getting wrong calculation in both cases.... Can someone point out the mistake i am making here??

I am attaching the snaps of all the configuration.. Along with the graph of total bandwidth from regular graph template.. which is doing correct calculation for the same time period.

[dononeil]

Actually you are not getting the wrong calculation.....

ReportIT uses no consolidation for doing it's calculations, so peaks are maintained, true averages are calcualted, etc...

HOWEVER, by default Cacti graphs use the AVERAGE consolidation function, which means that in order to 'squish' the data to the graph, multiple points are averaged to make one data point, and then the graph and corresponding min/max/avg is based on that data.

I had exactly the same issue you did. If you change your graphs to use the MAX consolidation, both Cacti and ReportIT will then match.

In fact, I don't understand why anyone would use the AVG function anyway, it has a tendancy to kill true spikes in the data.

I am planning on documentation updates to both ReportIT and the cacti doc pages that talk about the consolidation functions of the graphs, but I just haven't had time lately.

[michaelwjoyner]

Doneil,
Thanks for mentioning about the documentation update. I have been using cacti for a little while and I still seem confused about how the cdefs work within the graphs, and ReportIt for that matter.

Your explanation helped a bit.

[dononeil]

No problem... if you search the forums for 'graph consolidation' you'll find a longer explination from me.

[michaelwjoyner]

I dying to ask.........

How do the CF's work in ReportIt? Like for example, it confuses me how it ties in. I have a report that basically shoes me daily utilization on DSO circuits. (SEE ATTACHED)

In the first measurand, I use "f_max" with the CF, "MAX".

........ Isn't this independent of the CF? Isn't this the max of all RRA's?

Second, I take the "MAX:busy" with the CF of "AVERAGE".

........ Am I taking the MAX for "busy" based on the average of busy?

I hope that makes sense. I am reading more about RRDTOOL which has been shedding some light.

Thanks!

[dononeil]

ReportIT and RRDTool use totally separate functions...

When ReportIT does all of it's calcs, it does it based on the original RAW data... which preserves peaks, valleys, etc... nothing is averaged out, or dropped in order to do the calcs. As a result, the data is very accurate. No consolidation is happening on the data, and thus is should be trusted as the 'real deal'.

RRDTool however can be deceiving. RRDTool, by default, is set to use AVERAGE as a consolidation function (not the best way to do things if you ask me). So, when it graphs, and shows it's min/max/average legend, it is based on 'massaged' data.

Here's an example how the AVERAGE consolidation works....

Lets say the graph you're plotting is only going to be 100 pixels wide. Lets say the data you're trying to graph actually is 1000 data points (10x larger than the graph pixels). RRDTool will then average the first 10 data points to make the first point on the graph, the second 10 for the second point, and so on.... and the legend that is printed below the graph with the min/max/average is based on the consolidated (averaged) 100 plot points, and not the original 1000 data points.

The MAX consolidation works similarly, but instead of averaging 10 data points to make the plot point, it uses the highest value. Since most people are only concerned with 'peak' utilizations, it's probably best to use the MAX function rather than AVERAGE. MIN is obviously the opposite of MAX.

So.... if your graphs are using the AVERAGE consolidation function, and you use ReportIT to run reports, the values/graphs will NOT match. There is ONE exemption from this statement... if your data set is small enough on the graph to have it not consolidate data because you zoomed in to the graph (ie, 100 pixels wide, <100 data points) then the data will match.

The RRDTool consolidation function of AVERAGE is good for general trending, but if you're concerned with spikes and peaks in your traffic/data then you should use the MAX consolidation.

Unfortunatly changing from AVERAGE to MAX requires that you delete/recreate all your RRD files, so it's best to set things up the way you want it from the beginning.

[michaelwjoyner]

Thats the dope I was looking for!!!!

WELL PUT, I never got any of this from the doc's.

I really do appreciate you taking your time to explain it. I finally get it.

Just left with the question, then why do we need the CF's in ReportIt?

THX AGAIN!

[dononeil]

The "Consolidation Function" in reportit is more of a Calculation Function, not an RRDTool Consolidation Function....

You need the functions in ReportIT to do calcs against peak, throughput, etc... to calculate things like % utilization, etc...

I'm working on better docs for ReportIT too... just haven't had much time to do it. Took quite a few emails and trials to figure it all out.

[browniebraun]

Just left with the question, then why do we need the CF's in ReportIt?

THX AGAIN!

The "Consolidation Function" in reportit is more of a Calculation Function, not an RRDTool Consolidation Function....

You need the functions in ReportIT to do calcs against peak, throughput, etc... to calculate things like % utilization, etc...

I'm working on better docs for ReportIT too... just haven't had much time to do it. Took quite a few emails and trials to figure it all out.

Oh dear!
I'm really sorry that I've so much to do at the moment and did not interrupt that discussion earlier. But now I've to become active before you guys are totally out of it.

As every admin should know RRDtool differentiates between "primary data points" (PDP) and "consolidated data points" (CDP).
If a mesaured value will be fetched it's a PDP, a little later those PDPs will be consolidated to CDPs in relation to the rra definition. The consolidation functions are MIN, MAX, AVERAGE AND LAST.
I hope this will also clarify that if your poller is running every 5 Minute the values contained in the 5 minute MAX archive are the same as the values stored in the 5 minute Min, Average and Last archive.

If we take a look at the 30 minute archives then we'll see that the last average value represents the average of 6 PDPs. Same applies to the MAX, MIN and LAST archive:
MAX will contain the highest value of that 6 PDPs, MIN the lowest and LAST the lastest.
I'll have to leave out all the other parameters, therefore you should read the tutorials written by Alexander Bogaerdt.

If you want to read out data from RRDs then you have to define following parameters at least beside the name of the RRD file: The consolidation function, the start and the end of the time series.
So, we have to say for example: Give us all the max values recorded during the latest 7 days.

So what does ReportIt?
ReportIt offers to fetch all data from the Max, Min, Average and Last archive for your reporting period, so that we can calculate with them.
E.g. We want to calculate with peaks, then we have to bind our measurand on consolidation function MAX!
Our second measurand should represent the average utilization which is often shown in traffic graphs. Therefore we have to use the same archive value, so we need CF "AVERAGE".

And what can I do with f_max?
The f_max function returns the highest value of an amount of data. That amount of data is defined by the consolidation function.
So if we use f_max in combination with CF MAX, then we get the highest value of the peaks. Used with Cf AVERAGE it will return the highest average value.

Best regards
-Andreas-

[browniebraun]

Sorry for the delay in getting back...didn't get the msg notifier...anyways moving on

After reading your post I did some investigation. And yes, you are correct. At some point my data template for interface traffic has changed. The orginal template had a max. value of 100000000. The template as it exists today is 0 for max. value. I've tried selecting the "Use Per-Data Source Value" for the traffic_in and traffic_out, which when I look at the actual data sources a maxValue gets added corresponding to the interface max. Should this not work to get a maxValue to be available in the measurand variables (interface traffic template)?

Maybe I should explain what I'm trying to achieve a little better. I'm interested in reporting of max. utilization of interfaces on a percentage basis. I was using the guide on sourceforge to set this out. But...couldn't continue when the maxValue was not present as a variable. I have thousands of 10M/100M/1000M/10G interfaces out there. What is the easiest way to accomplish what I'm looking for with min. effort?

Many Thanks in Advance!!!

Mmmh, I never thought about that. I'll try to allocate some time next weekend to proof which changes are necessary to support that.
I also fixed a bug some days ago which will occur with data source items selected individually. See http://forums.cacti.net/viewtopic.php?p=164770#164770. So 0.7.3 will come.

Best regards
-Andreas-