Default Datasource & RRD max values, and 64bit counters

[cgb]

Hi there,

I have observed the same symptoms as those reported in
http://forums.cacti.net/about32536.html

That is, graphs/datasources created for down interfaces default to a 10mbit maximum value in both the RRD & the datasource maximum value setting for traffic_in and traffic_out.

In a related note to my point, the default graph type choice for Interface graphs is 'In/Out bits', which uses 32bit counters.

These two behaviors result in a painful experience when you go to troubleshoot a bandwidth problem and discover your graphs have data, but it's completely wrong in certain cases.

Experimenting with Cacti, I've set 64bit counters and a maximum datasource & RRD value of 1x10^9 (1gbit) for all graphs, regardless of speed. It is an assumption that the devices support 64bit interface counters in my tests, although better handling of this could be implementing in Cacti (when adding graphs, get both to determine which counter to use...).

My tests have been in an isolated playground Cacti install and I havn't observed any negative effects so far. I was hoping other experienced Cacti users and/or developers could respond with any flaws in using 64bit counters for all graphs, and setting the maximum datasource & RRD values to be the upperlimit off all interfaces for the site.

The one drawback I could think of is if you're not using auto-scale for the graph template (default is auto-scale on), you'd expect your graphs to be out of scale, as it'd use the maximum value as an upperlimit. According to http://forums.cacti.net/about23679.html, and my own tests, if you disable auto-scale, the upperlimit will be 100 as observed by the graph debug feature and the rrdtool command line options. So I expect auto-scale will always be on, which suggests regardless of the 'maximum value', the upperlimit of a graph will be the maximum value for the dataset.

I'm not heavily familiar with rrdtool mechanics, but I suspect another drawback is if the maximum value in the RRD is inaccurately high, it will accept erroneous values from polling results. I'm not sure if that is worse than having erroneously wrapped data values that _look_ correct but are not

If the above doesn't have any negative consequences, I'd be curious to know what others think about changing the default behavior so maximum values are high, and 64bit counters would be used by default if they are available.

Thanks for any responses in advance.

Chris Bennett
cgb

[gandalf]

My tests have been in an isolated playground Cacti install and I havn't observed any negative effects so far. I was hoping other experienced Cacti users and/or developers could respond with any flaws in using 64bit counters for all graphs, and setting the maximum datasource & RRD values to be the upperlimit off all interfaces for the site.

The major flaw seems to be lack of support for 64 bit COUNTERS. In other words, not all SNMP capable devices respond to those OIDs. Still, support for SNMP V2c is not MANDATORY for SNMP devices. And if SNMP V2c is supported, support for 64 bit COUNTERS again is not MANDATORY.
That's why we can't default to that.
Arguing against using the ifSpeed as a default RRD ds max is a different point. As ifSpeed may change at any time (e.g. auto-negotiation), you may feel the need to define your own maximum or to have no maximum at all.
R.

[TheWitness]

Also, ifHighSpeed MUST be used for 64 bit counters. We should add that to the default template. Then we could be more intellegent about things at instantation. However, if things change....

Larry

[cgb]

Thanks for the quick replies - Cacti's support base is a great example to other projects.

The major flaw seems to be lack of support for 64 bit COUNTERS. In other words, not all SNMP capable devices respond to those OIDs. Still, support for SNMP V2c is not MANDATORY for SNMP devices. And if SNMP V2c is supported, support for 64 bit COUNTERS again is not MANDATORY.
That's why we can't default to that.

I was thinking Cacti could handle it gracefully by falling back to 32bit counters if 64bit counters are unavailable, and this logic would only be used when SNMP v2C is specified. This is only a problem for link speeds > 100mbit, so I would expect any device having gigabit or more interface speeds, would support 64bit counters.

Arguing against using the ifSpeed as a default RRD ds max is a different point. As ifSpeed may change at any time (e.g. auto-negotiation), you may feel the need to define your own maximum or to have no maximum at all.
R.

I am interested in considering whether having no maximum (or an artificially high maximum in the case of the RRD) has any detrimental impact on the correct operation of Cacti, and if not, it might be worth considering in future versions of Cacti so user's don't get bitten by a 1gig interface created with 10mb maximums because the interface was down. For similar reasons, interfaces that auto-negotiate to a higher rate at any time will be affected by this.

Also, ifHighSpeed MUST be used for 64 bit counters. We should add that to the default template. Then we could be more intellegent about things at instantation. However, if things change....

If the maximum value has no significant bearing on the correct operation of Cacti & RRD, then you don't need to deal with ifSpeed/ifHighSpeed at all.

Chris Bennett
cgb

[gandalf]

I was thinking Cacti could handle it gracefully by falling back to 32bit counters if 64bit counters are unavailable, and this logic would only be used when SNMP v2C is specified. This is only a problem for link speeds > 100mbit, so I would expect any device having gigabit or more interface speeds, would support 64bit counters.

As this is a corner case on 32/64 bit interface traffic alone, I hesitate to produce code for that, to be honest

I am interested in considering whether having no maximum (or an artificially high maximum in the case of the RRD) has any detrimental impact on the correct operation of Cacti, and if not, it might be worth considering in future versions of Cacti so user's don't get bitten by a 1gig interface created with 10mb maximums because the interface was down. For similar reasons, interfaces that auto-negotiate to a higher rate at any time will be affected by this.

No, it does not stop anything from working. By applying a MAX value, you will stop graphs from spiking under strange conditions. By not applying this, you may have those strange spikes (e.g. on reboot, interface reset and the like)

If the maximum value has no significant bearing on the correct operation of Cacti & RRD, then you don't need to deal with ifSpeed/ifHighSpeed at all.

Same comment. But yes, using ifHighSpeed for MAX in case of 64bit counters is again a "corner case"

These all are personal opinion, not the result of a discussion with developers
R.

[cgb]

Thanks gandalf - I might look into an out-of-band script to compare Cacti's max settings per interface to that of the actual device, periodically, to ensure appropriate counter type and maximum settings.

Chris Bennett
cgb