Spine not parsing VALUE/REGEXP in resource XML

[pinnieroom]

I have the following in my resource XML

Code: Select all

<facilityrxpowerinput>
<name>Rx dBm</name>
<method>walk</method>
<source>VALUE/REGEXP:((-){0,1}([0-9]+)\.[0-9]*)</source>
<direction>output</direction>
<oid>.1.3.6.1.4.1.2865.1.4.1.1.10</oid>
</facilityrxpowerinput>

This is mainly becuase the device responds with garbage after the SNMP value.
Eg.

Code: Select all

SNMPv2-SMI::enterprises.2865.1.4.1.1.10.4194819 = STRING: "-12.457309, , , , , , "

When I poll the device with cmd.php its all good. However, I need to use spine for performance reasons.

When I use spine, the values stay in the poller_output table and don't get uploaded into the RRD files.
Plus, the values that are in the poller_output table look like the raw values...

Code: Select all

|          6247 | dBm_RX   | 2011-11-09 15:07:02 | -15.767338, , , , , ,                                              |
|          6247 | dBm_TX   | 2011-11-09 15:07:02 | 3.496529, , , , , ,                                                |
|          6265 | dBm_TX   | 2011-11-09 15:07:03 | 3.514897, , , ,                                                    |

On the next polling cycle, Cacti complains that there is data in the poller_output table (which it appears to delete before the next poll).

I've tried a few differenet regex in the resource xml, but nothing seems to work. Should spine be evaluating the regex or does the poller.php do this?

It would appear to me to be a bug in spine.

Any ideas?

[pinnieroom]

UPDATE:

It appears that spine is not capable of understanding the VALUE/REGEXP at all. It just tries to find the number buried in the result.

However, after finding the number it doesn't remove any other characters from the string. So I modified the code in poller.c like so...

Before:

Code: Select all

...
}else if ((is_numeric(snmp_oids[j].result)) || (is_multipart_output(snmp_oids[j].result))) {
	/* continue */
...

After

Code: Select all

...
}else if (is_numeric(snmp_oids[j].result)) {
	snprintf(temp_result, RESULTS_BUFFER, "%s", strip_alpha(trim(snmp_oids[j].result)));
	snprintf(snmp_oids[j].result , RESULTS_BUFFER, "%s", temp_result);
}else if (is_multipart_output(snmp_oids[j].result)) {
	/* continue */
...

There are a few places with much the same code, just slightly different variables. So I repeated this for all the occurances.

This appears to work for my situation, but it's fairly specific. It would be great if some code is added to spine to use the regular expression from the XML.

[gandalf]

I'm not sure that this is indeed a spine issue.
Today, I fixed a very similar error in lib/data_query.php. Find in in our online SVN. The fix itself is simple and was committed as r6929
R.

[NetBob]

Hello,
I've run into the same problem.
I'm running cacti with 0.8.7i wich has the corrected lib/data_query.php.
My own debugs come to the same conclusion as pinnieroom, Spine should extract the numeric value from the string.
Even though I'd rather have an xml regexp parse, in the context of is_numeric() extracting the numeric value should be enough. But I see many other cases where this hack won't work.

M

[NetBob]

To continue on my post, there is another possibility:
just before writting to the RRD in lib/rrd.php line 370, instead of doing an is_numeric() check (which retruns FALSE in php) that invalidates the string and replaces it by 'U' there could be an xml regexp enforcement, this way no changes need to be done to spine and Cacti can reuse the facilities that it already has.

[NetBob]

Progress:
actually cmd.php does not enforce VALUE/REGEXP when doing the polling, the function strip_quotes() removes all non-numeric characters from the result
hence VALUE/REGEXP is only used in the data query part to format the results in the initial walk.
to emulate this behavior in spine, is_numeric should return the same as PHP (false when non numeric characters are present instad of true if the start of the string is numeric) so as to go through

snprintf(temp_result, RESULTS_BUFFER, "%s", strip_alpha(trim(snmp_oids[j].result)));
snprintf(snmp_oids[j].result , RESULTS_BUFFER, "%s", temp_result);
in the poller.c else clause.

[bkoch]

I am having the exact same issue as pinnieroom.
I am currently running SPINE 0.8.8a
PHP Version 5.3.10-1ubuntu3.2

Running SPINE manually results in the following output:

02/25/2013 08:17:52 AM - SPINE: Poller[0] Host[1248] TH[1] DS[4114] SNMP: v2: xxx.xxx.xxx.xxx, dsname: light_level_db_in, oid: .1.3.6.1.4.1.2865.1.4.1.1.10.4194830, value: -12.518802, , , , , ,

Is there a way to poll the actual number value out from "-12.518802, , , , , ," yet?

Thanks!

[bkoch]

I was able to create a work around for pinnieroom's original problem. But this does not implement the VALUE/REGEXP function that was stated in the title.

I was able to implement the workaround by doing the following:

Download and extract the cacti-spine-0.8.8a source (http://www.cacti.net/downloads/spine/ca ... .8a.tar.gz).
tar -zxvf cacti-spine-0.8.8a.tar.gz

*Edit poller.c
--> Go to line 977
--> Add the following two lines of code as stated by NetBob:

snprintf(temp_result, RESULTS_BUFFER, "%s", strip_alpha(trim(snmp_oids[j].result)));
snprintf(snmp_oids[j].result , RESULTS_BUFFER, "%s", temp_result);

--> Save the file.
--> Recompile spine (run the following commands from within the cacti-spine-0.8.8a directory or whatever directory you extracted the cacti-spine-0.8.8a.tar.gz source code to)

./configure
make

--> Make a backup of your old spine binary on your production system: /usr/local/spine/bin/spine
Rename it to something else like spine_ORIGINAL

--> copy the new "spine" binary file from the cacti-spine-0.8.8a directory that you just ran the ./configure and make commands in to your production cacti spine/bin folder: /usr/local/spine/bin

*Now delete any graphs and their associated data sources that use the SNMP data query which is polling the funky string data, and recreate them.


After performing all these actions and waiting 2 poling periods I was able to save the values like "-12.518802" from the snmp strings of "-12.518802, , , , , ," into RRD files and graph the results.

Please let me know if this was of any help to your issues as well.

Thanks!

[bkoch]

Here is the recombpiled spine binary.

Just remove .txt from the end of the file name. The forum won't allow files with no extension to be uploaded.

[gandalf]

I will consider this for a next update
R.

[llow]

I'm just now looking at this for myself and it looks like is_numeric is improperly returning true for strings that contain non numeric data. I am looking through that function now to see why it is returning true for values that start with numbers and end with strings.

[gandalf]

I remember having read funny hints about correct use of "integer" detection with php. is_numeric has to be handled with care.

I will happily accept any bug report that allows to recreate the issue in question so I can debug and fix.
Of course, code contributions are welcomed
R.

[gandalf]

What I missed:

Value/Regexp has been defined for <direction>input</direction> only. And there it works, as I just verified again.

Nevertheless, this test on numeric output results seems to be a good move. Will do some more evaluating

R.

[sk011]

Hi all.
I have same problem
in my xml file

Code: Select all

        <ddmTemperature>
            <name>ddmTemperature</name>
            <method>walk</method>
            <oid>.1.3.6.1.4.1.5205.2.77.2.1.4.1.13</oid>
            <source>VALUE/REGEXP:\D</source>
            <direction>output</direction>
        </ddmTemperature>

device response

Code: Select all

SNMPv2-SMI::enterprises.5205.2.77.2.1.4.1.13.25 = STRING: "46.02 C"

regexp delete all non-numeric, if i change <direction> to input and try "Verbose Query"

Code: Select all

+ Found item [ddmTemperature='4602'] index: 25 [from regexp value parse]

if <direction> output in cacti.log

Code: Select all

SPINE: Poller[0] Host[38] TH[1] DS[11554] SNMP: v2: 10.1.6.18, dsname: ddmTemperature, oid: .1.3.6.1.4.1.5205.2.77.2.1.4.1.13.25, value: 46.02 C

i try recompile spine0.8.8b with edit poller.c line 977

Code: Select all

                        }else if ((is_numeric(snmp_oids[j].result)) || (is_multipart_output(snmp_oids[j].result))) {
                            /* continue */
                            snprintf(temp_result, RESULTS_BUFFER, "%s", strip_alpha(trim(snmp_oids[j].result)));
                            snprintf(snmp_oids[j].result , RESULTS_BUFFER, "%s", temp_result);
                        }else if (is_hexadecimal(snmp_oids[j].result, TRUE)) {
                            snprintf(snmp_oids[j].result, RESULTS_BUFFER, "%lld", hex2dec(snmp_oids[j].result));

PS sorry for my bad english