Virtual counter64 SNMP support

[wheelear]

Most of the computers I monitor with Cacti do not seem to support 64bit counters. And, the switches I have which do support them seem to have a bug that makes the data unreliable.
So, I decided to fix this by writing a server that would run the needed queries once every 30 seconds, keeping track of the data using a 64bit counter, so I could effectively monitor gigabit interfaces.
Then when Cacti wanted the data, present it the locally maintained 64bit value.

This has been imlemented using perl for the server, and MySQL to hold the bits to query.

Is this something anyone else would be interested in?

The way it works now, when SNMP queries for either of the 2 counter64 interface OIDs are generated, they are converted to the 32bit queries, and the results stored in the database, which the server then updates every 30 seconds.

So, now I have counter64 support for all my systems.

If there is interest, I'll make the patch/server available here.

[klavs]

I would very much like that patch/script. Have all Gigabit here interfaces here - and no graphs that works with 64-bit counters

[wheelear]

Ok, here you go.
There are 4 files in the tarball:
SnmpProxy2.pl - The Perl script which runs a query every 30 seconds
snmp_counter64.sql - MySQL Code to put a new table in the database
cacti64php.tgz - Tarball containing replacement php files
README - Installation instructions and notes

Let me know if you have any questions.
As it says in the README, this should only affect queries that are for the 64bit counters. Your current 32bit counters should remain unaffected.

You must also be using the cmd.php poller. I think I'm going to modify cactid, but for now, it requires cmd.php.

[prjctgeek]

very Creative workaround..

While looking at this idea I checked out the net-snmp site and discovered that the snmpd that comes with RH/Fedora(others?) doesn't have a flag that enables the IfXTable (64bit counters for interfaces). Reconfigure/recompile with the --enable-mfd-rewrites flag and eliminate the linux hosts from your problem list.

look at: http://www.net-snmp.org/about/ChangeLog.html.

snippet:
Ports:
Linux:
- new experimental tables/rewrites for Linux, including:
ifTable, ifXTable, inetCidrRouteTable, ipCidrRouteTable,
ipAddressTable, ipSystemStatsTable, ipNetToPhysicalTable.
- Enable these talbles by specifying --enable-mfd-rewrites to
configure.

[fletch]

I'm about to try this, but am not clear on how to "tell cacti that you want to collect 64bit network statistics for some servers" - from your README.

How do I tell cacti to collect the 64 bit counters?

thanks,
fletch.

[fletch]

answer: choose the DataSource and update bottom dropdown list to "In/Out Bits (64 Bit counters)

its working great - brilliant workaround - although I am afraid to hammer everything every 30 seconds...

thanks,
fletch

[fletch]

Ok, there is a problem with some idle servers.

the graphs for these servers using your proxy script are showing a steady level of > 100Mbit traffic when there is really next to none.

I've guessing the is due to the logic in the script assuming the initial readings wrapped - when really they are not?

Any fix for this? (reset the DB counter?)

Below is an example of the issue - one reading taken from an unmodified 32 bit cacti installation - the other from the 64-bit SnmpProxy workaround modified version. The network is not carrying 200Mbit on this server...its not computing this correctly.

thanks,
fletch.

[fletch]

The first graph is 32 bit counter (pre-patch) the second is the same web server network traffic graph post 64-bit patch - something is not right in the 64 bit version

[fletch]

Figured it out

the last32 field in the snmp_counter64 table is an int(32) which does not accept the updates from the pl script (mysql warns - data truncated) - therefore the subsequent differences between current and last grow and grow .
The fix - re-do the last32 as a varchar(20):

drop table snmp_counter64;
create table snmp_counter64(
host varchar(20) not null,
community varchar(20) not null,
oid varchar(255) not null,
version int default 1,
last32 varchar(20) not null default '0',
last64 varchar(20) not null default '0'
);

update the .pl update line to have single quotes around the last32 value:

$Query="UPDATE snmp_counter64 SET last32='$current32', last64='$last64' WHERE host='$host' AND oid='$oid'";

Cheers!
Fletch.

[kbartoletta]

Just to chime in here.... along these same lines, I am trying to graph some items on an F5 BigIP load balencer - all of the values are reported in counter64 data type (64 bit counter) - Am I hearing from your discussion above that there is no way to tell cacti that these are 64 bit counters??? Currently, my graphs are all crazy and inaccurate...

[robn]

There seems to be a bug somewhere - I ended up with 1000s of duplicate rows for one hosts - enough to ensue mysql took up 90% of the cpu time on a dual P3 1Ghz server

Anyway for now I've added a primary key to the database untill I can look through the code.

Code: Select all

CREATE TABLE `snmp_counter64` (
  `host` varchar(20) NOT NULL default '',
  `community` varchar(20) NOT NULL default '',
  `oid` varchar(255) NOT NULL default '',
  `version` int(11) default '1',
  `last32` varchar(20) NOT NULL default '0',
  `last64` varchar(20) NOT NULL default '0',
  PRIMARY KEY  (`host`,`oid`),
) ENGINE=InnoDB DEFAULT CHARSET=utf8

The only obvious difference is that its an SNMP v1 host with a FQDN

[robn]

It def doesn't like this server - everytime the poller runs I'm getting this error.

SQL Exec Failed "INSERT INTO snmp_counter64(host, community, oid, version, last32, last64) VALUES('ilab.midyorks.nhs.uk:161', 'public', '.1.3.6.1.2.1.2.2.1.16.2', 1, 108951049, 108951049)"

The graph for this host still works fine - but it would be nice to get to the bottom of this and find out why this is happening

[robn]

I've not got arround to fixing that bug yet, but I have made a few changes to the script

Basically I've changed it so it will run every ~60seconds no matter how long it takes to poll all the devices.

If it takes longer than 60seconds to poll the devices it will simply repeat as soon as its finished.

Why? Will we had an issue with it taking too long with the 30 seconds pause in it. I removed the pause and it was fine, untill this week when it started to take about 8seconds and was constantly updating the database.

These changes should mean that it will repeat striaght away when devices are slow to respond, but still delay things when all devices are responding normally.

[robn]

I've made a change to the table to reduce disk i/o by using the heap storage method. All the data is lost on mysql restart, but this has little impact on the graphs (plus solves the issue of deleted devices not getting removed from the database).

alter table cacti.snmp_counter64 engine=heap;

Code: Select all

[Faruk]

I've made a change to the table to reduce disk i/o by using the heap storage method. All the data is lost on mysql restart, but this has little impact on the graphs (plus solves the issue of deleted devices not getting removed from the database).

alter table cacti.snmp_counter64 engine=heap;

Code: Select all

Hi,

I have loaded all the scripts/files for counter 64 bit support. I want to poll some specific OID using 'SNMP-Generic OID Template'. Problem is, I can not figure out any way to tell cacti to use the data as 64 bit counter.

Need help...

Faruk