Hex to decimal conversion

[dalaro]

Hi,

I've got some devices that use the Fibre Alliance MIB. This MIB encodes 64-bit interface counters as hex strings. Cacti considers these hex-strings non-numeric and sends unknown values to the associated RRDs. There a quite a few devices that use this MIB; for example, QLogic SANBoxes and Cisco MDS devices.

People have worked around this problem by creating a separate script query for the device (e.g. this bit of advice or this cisco mds template). The query is essentially identical to normal cacti SNMP functionality, except it does hex to decimal conversion on the returned value. This is a valid approach, but it duplicates code and increases the administration overhead on those devices. It's not as tightly-integrated and easy to use, imho, as an SNMP query.

I modified Cacti to convert pure hex output values into decimal, treating them as numbers. Informational or "input" data (in the SNMP query XML sense) are not modified. Only data bound for RRDs get the conversion. I think this isreasonable because RRDs store numbers. If the user configures Cacti to send hex values into RRDs, it seems sensible to treat them as such (instead of unknown).

Now I'm using plain SNMP queries to read my hex-string counters, and things are comparatively tidier in my configuration.

I'd appreciate your thoughts on this. If it sounds like a good idea, I've got diffs.

Thanks
Dan

[dalaro]

Here are patches with the functionality I want. I'm running these on my install and polling some QLogic SANBoxes with it.

One patch is for cacti proper, the other is for spine. So far, I've only added hex-to-dec conversion under spine; haven't changed cmd.php because there hasn't been a need. If this gets integrated, though, I can supply a tested patch for cmd.php too.

Any feedback is appreciated

[BSOD2600]

Please create a feature bug, with the details/attachment of these posts. also post a link to this post in the bug. http://www.cacti.net/bugs.php

[dalaro]

Thank you. Created issue #0001271.

http://bugs.cacti.net/view.php?id=1271

[zivley]

Great to see someone has the same problem than me and have already found the workaround!
Will someone be kind and tell me how to apply those patches??
Thanks in advance
Ziv

[limuerti]

Great to see someone has the same problem than me and have already found the workaround!
Will someone be kind and tell me how to apply those patches??
Thanks in advance
Ziv

patch -p1 -N < hexdec_conv_cacti.patch

Regards

[gheppner]

This is great, a patch for cmd.php would be great as well since I'm not using spine. If I was more savvy with php I'd give it a try myself...

[gheppner]

Actually scratch that, i figured i might as well install spine.

Here is a question though. I'm polling an FC device that uses the FC alliance MIBS that you reference. The hex to decimal conversion works fine for (eg) connUnitPortStatCountTxElements, but it appears to output an octet string and not a meaningful interger i can graph.

Whats the missing step? In the absence of an external script, there must be another solution.

[dalaro]

Hi,

First, thanks Cacti team for merging my patches into the mainline. I've been updating Cacti and monitoring every switch in our SAN with these changes. I appreciate you taking the time and effort. I'm pretty late with the feedback, but hopefully better late than never.


@gheppner: I *think* you're saying you can't sample OIDs like connUnitPortStatCountTxElements without a script. If you're using up-to-date cacti with spine, this is not accurate. Maybe I misunderstood. Let me explain these patches a bit.

These patches changed spine to accept valid hexadecimal strings verbatim. So spine gets a hex string from the Fibre Alliance device, e.g. "00 00 B5 9A A9 00 AF A8". Spine decides the hex, including spaces, is valid. Spine updates the poller output SQL table with the hex string verbatim (spaces preserved).

Next, lib/poller.php comes along to suck values out of SQL and insert them into the appropriate Round Robin Archive. Before inserting values into RRDs, poller.php validates them using php's standard "is_numeric()" function. The hex string fails is_numeric() because it has spaces, and is_numeric() is sensitive to spaces. My patch adds an if-else in this vicinity; if the string is hex (ignoring spaces), then convert it to decimal via php's "hexdec()" and treat it as valid. Since php 4.10, hexdec() will use your platform's maximum signed int precision, or switch to using floats if the input is too big for an int. It's important to note that space-sensitivity is crucial here. If is_numeric() ignored spaces, or if the hex-string read by spine had no spaces, poller.php would be wrongly interpreting number-only hex values. For example, hex "01 00" (decimal 256) would be ingested into an RRA as decimal 100. Fortunately, the semantics of is_numeric() are well-specified in the PHP manual, and hex strings are normally printed with spaces by the underlying SNMP client implementation, so this doesn't happen.

In short, the hex-decimal conversion should be seamless, without any scripts, as long as the hex is valid and contains spaces.

Here's the little snmp_query XML document we use to monitor the SANBoxes. It's real simple, but it meets our needs. To set it up, I created a data query and set this as its XML file. Then I associated the stock Interface Traffic graph template using connUnitPortStatCountTxElements as traffic_out andconnUnitPortStatCountRxElements as traffic_in.

Anyway, I may have mixed this up, but I think it works. Let me know if you see a problem.

[dalaro]

If one of our SANBoxes returns a (64-bit) connUnitPortStatCountTxElements with the highest bit set, hexdec() will effectively fail by returning a rolled-over 63-bit int.

I wrote this above, but it's wrong. hexdec() returns a float in this case. I edited it out of the original.