Data Queries.....grrrrrrrrrr

[vragosta]

I am trying to implement data queries as outlined in the walkthrough:

http://docs.cacti.net/manual:088:3a_adv ... alkthrough

The indexes for the part of the MIB I am trying to walk are embedded in the OID itself, necessitating the use of the oid_index_parse field, which I have defined as follows:

Code: Select all

<interface>
        <name>panPerfStorageTable</name>
        <description>StorageBlade Performance Statistics</description>
        <oid_index_parse>OID/REGEXP:^.{30}(.*)</oid_index_parse>
        <oid_index>.1.3.6.1.4.1.10159.1.2.2.1.1.1</oid_index>
        <fields>
                <panHwBladeHwSN>
                        <name>Index</name>
                        <method>walk</method>
                        <source>index</source>
                        <direction>input</direction>
                        <oid>.1.3.6.1.4.1.10159.1.2.2.1.1.1</oid>
                </panHwBladeHwSN>
                <panHwBladeName>
                        <name>Blade Name</name>
                        <method>walk</method>
                        <source>value</source>
                        <direction>input</direction>
                        <oid>.1.3.6.1.4.1.10159.1.2.2.1.1.26</oid>
                </panHwBladeName>
                <panHwBladeType>
                        <name>Blade Type</name>
                        <method>walk</method>
                        <source>value</source>
                        <direction>input</direction>
                        <oid>.1.3.6.1.4.1.10159.1.2.2.1.1.2</oid>
                </panHwBladeType>

This results in "verbose query" output as follows:

Code: Select all

+ Running data query [10].
+ Found type = '3' [SNMP Query].
+ Found data query XML file at '/usr/share/cacti/site/resource/snmp_queries/panPerfStorageTable.xml'
+ XML file parsed ok.
+ <oid_num_indexes> missing in XML file, 'Index Count Changed' emulated by counting oid_index entries
+ Executing SNMP walk for list of indexes @ '.1.3.6.1.4.1.10159.1.2.2.1.1.1' Index Count: 11
+ Index found at OID: '1.3.6.1.4.1.10159.1.2.2.1.1.1.17.50.53.97.51.48.52.98.50.49.48.48.48.50.56.48.48.49' value: '25a304b2100028001'
+ Index found at OID: '1.3.6.1.4.1.10159.1.2.2.1.1.1.17.50.54.52.51.48.97.97.48.49.48.48.48.50.56.48.48.49' value: '26430aa0100028001'
+ Index found at OID: '1.3.6.1.4.1.10159.1.2.2.1.1.1.17.50.54.54.51.49.101.100.53.49.48.48.48.50.56.48.48.49' value: '26631ed5100028001'
+ Index found at OID: '1.3.6.1.4.1.10159.1.2.2.1.1.1.17.50.54.57.51.48.54.48.54.51.48.48.48.50.52.48.48.49' value: '26930606300024001'
+ Index found at OID: '1.3.6.1.4.1.10159.1.2.2.1.1.1.17.50.54.98.51.50.49.52.55.49.48.48.48.50.56.48.48.49' value: '26b32147100028001'
+ Index found at OID: '1.3.6.1.4.1.10159.1.2.2.1.1.1.17.50.55.97.51.48.48.49.51.49.48.48.48.50.56.48.48.49' value: '27a30013100028001'
+ Index found at OID: '1.3.6.1.4.1.10159.1.2.2.1.1.1.17.50.56.48.52.48.48.48.49.49.48.48.48.50.56.48.48.49' value: '28040001100028001'
+ Index found at OID: '1.3.6.1.4.1.10159.1.2.2.1.1.1.17.50.56.52.52.48.48.48.99.49.48.48.48.50.56.48.48.49' value: '2844000c100028001'
+ Index found at OID: '1.3.6.1.4.1.10159.1.2.2.1.1.1.17.50.56.52.52.48.48.48.100.49.48.48.48.50.56.48.48.49' value: '2844000d100028001'
+ Index found at OID: '1.3.6.1.4.1.10159.1.2.2.1.1.1.17.50.56.52.52.48.48.48.101.49.48.48.48.50.56.48.48.49' value: '2844000e100028001'
+ Index found at OID: '1.3.6.1.4.1.10159.1.2.2.1.1.1.17.50.56.53.52.48.48.48.98.49.48.48.48.50.56.48.48.49' value: '2854000b100028001'
+ index_parse at OID: '1.3.6.1.4.1.10159.1.2.2.1.1.1.17.50.53.97.51.48.52.98.50.49.48.48.48.50.56.48.48.49' results: '17.50.53.97.51.48.52.98.50.49.48.48.48.50.56.48.48.49'
+ index_parse at OID: '1.3.6.1.4.1.10159.1.2.2.1.1.1.17.50.54.52.51.48.97.97.48.49.48.48.48.50.56.48.48.49' results: '17.50.54.52.51.48.97.97.48.49.48.48.48.50.56.48.48.49'
+ index_parse at OID: '1.3.6.1.4.1.10159.1.2.2.1.1.1.17.50.54.54.51.49.101.100.53.49.48.48.48.50.56.48.48.49' results: '17.50.54.54.51.49.101.100.53.49.48.48.48.50.56.48.48.49'
+ index_parse at OID: '1.3.6.1.4.1.10159.1.2.2.1.1.1.17.50.54.57.51.48.54.48.54.51.48.48.48.50.52.48.48.49' results: '17.50.54.57.51.48.54.48.54.51.48.48.48.50.52.48.48.49'
+ index_parse at OID: '1.3.6.1.4.1.10159.1.2.2.1.1.1.17.50.54.98.51.50.49.52.55.49.48.48.48.50.56.48.48.49' results: '17.50.54.98.51.50.49.52.55.49.48.48.48.50.56.48.48.49'
+ index_parse at OID: '1.3.6.1.4.1.10159.1.2.2.1.1.1.17.50.55.97.51.48.48.49.51.49.48.48.48.50.56.48.48.49' results: '17.50.55.97.51.48.48.49.51.49.48.48.48.50.56.48.48.49'
+ index_parse at OID: '1.3.6.1.4.1.10159.1.2.2.1.1.1.17.50.56.48.52.48.48.48.49.49.48.48.48.50.56.48.48.49' results: '17.50.56.48.52.48.48.48.49.49.48.48.48.50.56.48.48.49'
+ index_parse at OID: '1.3.6.1.4.1.10159.1.2.2.1.1.1.17.50.56.52.52.48.48.48.99.49.48.48.48.50.56.48.48.49' results: '17.50.56.52.52.48.48.48.99.49.48.48.48.50.56.48.48.49'
+ index_parse at OID: '1.3.6.1.4.1.10159.1.2.2.1.1.1.17.50.56.52.52.48.48.48.100.49.48.48.48.50.56.48.48.49' results: '17.50.56.52.52.48.48.48.100.49.48.48.48.50.56.48.48.49'
+ index_parse at OID: '1.3.6.1.4.1.10159.1.2.2.1.1.1.17.50.56.52.52.48.48.48.101.49.48.48.48.50.56.48.48.49' results: '17.50.56.52.52.48.48.48.101.49.48.48.48.50.56.48.48.49'
+ index_parse at OID: '1.3.6.1.4.1.10159.1.2.2.1.1.1.17.50.56.53.52.48.48.48.98.49.48.48.48.50.56.48.48.49' results: '17.50.56.53.52.48.48.48.98.49.48.48.48.50.56.48.48.49'
+ Located input field 'panHwBladeHwSN' [walk]
+ Executing SNMP walk for data @ '.1.3.6.1.4.1.10159.1.2.2.1.1.1'
+ Located input field 'panHwBladeName' [walk]
+ Executing SNMP walk for data @ '.1.3.6.1.4.1.10159.1.2.2.1.1.26'
+ Found item [panHwBladeName='Shelf-1,6'] index: .17.50.53.97.51.48.52.98.50.49.48.48.48.50.56.48.48.49 [from value]
+ Found item [panHwBladeName='Shelf-1,10'] index: .17.50.54.52.51.48.97.97.48.49.48.48.48.50.56.48.48.49 [from value]
+ Found item [panHwBladeName='Shelf-1,8'] index: .17.50.54.54.51.49.101.100.53.49.48.48.48.50.56.48.48.49 [from value]
+ Found item [panHwBladeName='Shelf-1,1'] index: .17.50.54.57.51.48.54.48.54.51.48.48.48.50.52.48.48.49 [from value]
+ Found item [panHwBladeName='Shelf-1,5'] index: .17.50.54.98.51.50.49.52.55.49.48.48.48.50.56.48.48.49 [from value]
+ Found item [panHwBladeName='Shelf-1,4'] index: .17.50.55.97.51.48.48.49.51.49.48.48.48.50.56.48.48.49 [from value]
+ Found item [panHwBladeName='Shelf-1,7'] index: .17.50.56.48.52.48.48.48.49.49.48.48.48.50.56.48.48.49 [from value]
+ Found item [panHwBladeName='Shelf-1,3'] index: .17.50.56.52.52.48.48.48.99.49.48.48.48.50.56.48.48.49 [from value]
+ Found item [panHwBladeName='Shelf-1,9'] index: .17.50.56.52.52.48.48.48.100.49.48.48.48.50.56.48.48.49 [from value]
+ Found item [panHwBladeName='Shelf-1,2'] index: .17.50.56.52.52.48.48.48.101.49.48.48.48.50.56.48.48.49 [from value]
+ Found item [panHwBladeName='Shelf-1,11'] index: .17.50.56.53.52.48.48.48.98.49.48.48.48.50.56.48.48.49 [from value]
+ Located input field 'panHwBladeType' [walk]
+ Executing SNMP walk for data @ '.1.3.6.1.4.1.10159.1.2.2.1.1.2'
+ Found item [panHwBladeType='Storage'] index: 17.50.53.97.51.48.52.98.50.49.48.48.48.50.56.48.48.49 [from value]
+ Found item [panHwBladeType='Storage'] index: 17.50.54.52.51.48.97.97.48.49.48.48.48.50.56.48.48.49 [from value]
+ Found item [panHwBladeType='Storage'] index: 17.50.54.54.51.49.101.100.53.49.48.48.48.50.56.48.48.49 [from value]
+ Found item [panHwBladeType='Director'] index: 17.50.54.57.51.48.54.48.54.51.48.48.48.50.52.48.48.49 [from value]
+ Found item [panHwBladeType='Storage'] index: 17.50.54.98.51.50.49.52.55.49.48.48.48.50.56.48.48.49 [from value]
+ Found item [panHwBladeType='Storage'] index: 17.50.55.97.51.48.48.49.51.49.48.48.48.50.56.48.48.49 [from value]
+ Found item [panHwBladeType='Storage'] index: 17.50.56.48.52.48.48.48.49.49.48.48.48.50.56.48.48.49 [from value]
+ Found item [panHwBladeType='Storage'] index: 17.50.56.52.52.48.48.48.99.49.48.48.48.50.56.48.48.49 [from value]
+ Found item [panHwBladeType='Storage'] index: 17.50.56.52.52.48.48.48.100.49.48.48.48.50.56.48.48.49 [from value]
+ Found item [panHwBladeType='Storage'] index: 17.50.56.52.52.48.48.48.101.49.48.48.48.50.56.48.48.49 [from value]
+ Found item [panHwBladeType='Storage'] index: 17.50.56.53.52.48.48.48.98.49.48.48.48.50.56.48.48.49 [from value]

When I click on "Create Graphs for Host", I see the following :

CreateGraphsForHost.JPG (50.92 KiB) Viewed 1447 times

So, yeah....a couple of things are amiss:

1. Why do I not see the index value?

2. Why are the "Blade Name" and "Blade Type" columns not associated with one another?


How can I fix this?

[phalek]

Because you have different indexes:

+ Found item [panHwBladeName='Shelf-1,6'] index: .17.50.53.97.51.48.52.98.50.49.48.48.48.50.56.48.48.49 [from value]

+ Found item [panHwBladeType='Storage'] index: 17.50.53.97.51.48.52.98.50.49.48.48.48.50.56.48.48.49 [from value]


Not sure, but I usually also add these 2 fields under the oid_index ones:

Code: Select all

 <index_order>panHwBladeHwSN</index_order>
    <index_order_type>alphabetic</index_order_type>

Have a look here which better fits your data query:

http://docs.cacti.net/howto:data_query_ ... _not_exist

[vragosta]

Shouldn't the OID index come from the OID specified -- <oid_index>.1.3.6.1.4.1.10159.1.2.2.1.1.1</oid_index>?

Why am I specifying oid_index and oid_index_parse fields if they are not used for every OID lookup?

+ Executing SNMP walk for list of indexes @ '.1.3.6.1.4.1.10159.1.2.2.1.1.1' Index Count: 11
+ Index found at OID: '1.3.6.1.4.1.10159.1.2.2.1.1.1.17.50.53.97.51.48.52.98.50.49.48.48.48.50.56.48.48.49' value: '25a304b2100028001'
.
.
.
+ index_parse at OID: '1.3.6.1.4.1.10159.1.2.2.1.1.1.17.50.53.97.51.48.52.98.50.49.48.48.48.50.56.48.48.49' results: '17.50.53.97.51.48.52.98.50.49.48.48.48.50.56.48.48.49'
.
.
.
+ Executing SNMP walk for data @ '.1.3.6.1.4.1.10159.1.2.2.1.1.26'
+ Found item [panHwBladeName='Shelf-1,6'] index: .17.50.53.97.51.48.52.98.50.49.48.48.48.50.56.48.48.49 [from value]
.
.
.
+ Executing SNMP walk for data @ '.1.3.6.1.4.1.10159.1.2.2.1.1.2'
+ Found item [panHwBladeType='Storage'] index: 17.50.53.97.51.48.52.98.50.49.48.48.48.50.56.48.48.49 [from value]


Why isn't index 17.50.53.97.51.48.52.98.50.49.48.48.48.50.56.48.48.49 used for each OID lookup, as this was the index located/calculated by oid_index and index_parse?

[BSOD2600]

I've actually run into this exact problem too when using oid_index_parse & OID/REGEXP (disassociated columns of info displayed). Haven't found a solution yet, but it use to work correctly several releases ago (it's been a while since I've needed to create a new Data Query).

If you look in the poller/snmp cache and pay close attention to the OIDs for this query, I bet you'll find some inconsistencies.

[phalek]

Hm

Looks like it's the regexp rule:

Code: Select all

.1.3.6.1.4.1.10159.1.2.2.1.1.1 - Index - Works
.1.3.6.1.4.1.10159.1.2.2.1.1.2 - BladeType - Works
.1.3.6.1.4.1.10159.1.2.2.1.1.26 - BladeName - Doesn't work

Try this index:

Code: Select all

<oid_index_parse>OID/REGEXP:^.{28}\.\d+\.(.*)</oid_index_parse>

[vragosta]

If you look in the poller/snmp cache and pay close attention to the OIDs for this query, I bet you'll find some inconsistencies.

Well, I'm relieved I'm not going insane, but really wish this would work as it seemingly should. Can I file a bug regarding this issue? If so, where is the appropriate place to do so?

[vragosta]

Try this index:

Code: Select all

<oid_index_parse>OID/REGEXP:^.{28}\.\d+\.(.*)</oid_index_parse>

No go...grabs the entire OID as the index.

[phalek]

What a fun ...

I usually create script server php files which doe the index parsing themselves, for some reason that always works

[vragosta]

I usually create script server php files which doe the index parsing themselves, for some reason that always works

Is there any documentation available in regards to how this is accomplished?

Thank you,

[BSOD2600]

Bug created: http://bugs.cacti.net/view.php?id=2470

For something this simple, SNMP Data Query really is the way to go... when it's working :/

http://docs.cacti.net/manual:088:3a_adv ... alkthrough is a starting point. Can also look at the existing script server scripts which come with Cacti for real examples.

[vragosta]

Bug created: http://bugs.cacti.net/view.php?id=2470

Thank you, I appreciate that.

[phalek]

Hi

Can you provide me with a snmpwalk for that oid: .1.3.6.1.4.1.10159.1.2.2.1.1.1
Please use the On swtich so it's numeric.

I want to troubleshoot the code to create a fix/patch.

Code: Select all

snmpwalk -On

[phalek]

I think I've found the bug.

In lib/data_query.php, line 302 it reads:

Code: Select all

$snmp_index = preg_replace('/' . (isset($field_array["oid_index_parse"]) ? $field_array["oid_index_parse"] : $index_parse_regexp) . '/', "\\1", $snmp_data[$i]["oid"]);

it's using the return from snmp_data which removes the first dot (.) from the oid which in return makes the REGEXP fail due to it not being able to match the string correctly.


Try this regexp to fix this:

Code: Select all

^\.{0,1}.{27,28}\.\d+\.(.*)

[BSOD2600]

phalek:
I can confirm that modified regex (well specifically: ^\.{0,1}.{25,26}\.\d+\.(.*) for my device) results in correct indexes and resulting Data Query table display.

Going to submit a patch to that bug?

[phalek]

I will once I nailed it down to the root cause or better: why that has been deliberately been removed ... since revision 2 of lib/snmp.php !