Making my own Data Query all the way down to graphs.

[askreet]

I am trying to define my own data query to create graphs for the number of connections on a Foundry ServerIron. This is my first experience trying to do all of this from the ground up.

I have defined an XML file as such:

Code: Select all

<interface>
        <name>ServerIron - Real Server Information</name>
        <description>Queries a ServerIron Load Balancer for Real Servers to be monitored</description>
        <oid_index>1.3.6.1.4.1.1991.1.1.4.24.1.1.1</oid_index>
        <oid_index_parse>OID/REGEXP:.*([0-9]*\.[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*)</oid_index_parse>
        <index_order>displayName:ipAddress:port</index_order>
        <index_order_type>numeric</index_order_type>
        <index_title_format>|chosen_order_field|</index_title_format>

        <fields>
                <ipAddress>
                        <name>IP Address</name>
                        <method>walk</method>
                        <source>value</source>
                        <direction>input</direction>
                        <oid>.1.3.6.1.4.1.1991.1.1.4.24.1.1.1</oid>
                </ipAddress>
                <port>
                        <name>Port</name>
                        <method>walk</method>
                        <source>value</source>
                        <direction>input</direction>
                        <oid>.1.3.6.1.4.1.1991.1.1.4.24.1.1.2</oid>
                </port>
               <displayName>
                        <name>Display Name</name>
                        <method>walk</method>
                        <source>value</source>
                        <direction>input</direction>
                        <oid>.1.3.6.1.4.1.1991.1.1.4.24.1.1.3</oid>
                </displayName>
                <connections>
                        <name>Connections</name>
                        <method>walk</method>
                        <source>value</source>
                        <direction>output</direction>
                        <oid>.1.3.6.1.4.1.1991.1.1.4.24.1.1.7</oid>
                </connections>
        </fields>
</interface>

The output of a walk on that OID is (Truncated, we have many real servers):

Code: Select all

SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.1.10.11.1.112.8888 = IpAddress: 10.11.1.112
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.1.10.11.1.113.8888 = IpAddress: 10.11.1.113
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.1.10.11.1.114.8888 = IpAddress: 10.11.1.114
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.1.10.11.1.115.8888 = IpAddress: 10.11.1.115
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.2.10.11.1.112.8888 = INTEGER: 8888
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.2.10.11.1.113.8888 = INTEGER: 8888
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.2.10.11.1.114.8888 = INTEGER: 8888
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.2.10.11.1.115.8888 = INTEGER: 8888
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.3.10.11.1.112.8888 = STRING: "server1"
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.3.10.11.1.113.8888 = STRING: "server2"
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.3.10.11.1.114.8888 = STRING: "server3"
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.3.10.11.1.115.8888 = STRING: "server4"
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.4.10.11.1.112.8888 = INTEGER: 0
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.4.10.11.1.113.8888 = INTEGER: 0
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.4.10.11.1.114.8888 = INTEGER: 0
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.4.10.11.1.115.8888 = INTEGER: 0
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.5.10.11.1.112.8888 = INTEGER: 6
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.5.10.11.1.113.8888 = INTEGER: 6
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.5.10.11.1.114.8888 = INTEGER: 6
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.5.10.11.1.115.8888 = INTEGER: 6
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.6.10.11.1.112.8888 = INTEGER: 0
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.6.10.11.1.113.8888 = INTEGER: 0
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.6.10.11.1.114.8888 = INTEGER: 0
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.6.10.11.1.115.8888 = INTEGER: 0
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.7.10.11.1.112.8888 = INTEGER: 0
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.7.10.11.1.113.8888 = INTEGER: 30
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.7.10.11.1.114.8888 = INTEGER: 0
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.7.10.11.1.115.8888 = INTEGER: 0
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.8.10.11.1.112.8888 = Counter32: 0
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.8.10.11.1.113.8888 = Counter32: 31461169
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.8.10.11.1.114.8888 = Counter32: 129850
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.8.10.11.1.115.8888 = Counter32: 441
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.9.10.11.1.112.8888 = Counter32: 0
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.9.10.11.1.113.8888 = Counter32: 48782
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.9.10.11.1.114.8888 = Counter32: 0
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.9.10.11.1.115.8888 = Counter32: 0
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.10.10.11.1.112.8888 = Counter32: 0
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.10.10.11.1.113.8888 = Counter32: 27911
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.10.10.11.1.114.8888 = Counter32: 0
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.10.10.11.1.115.8888 = Counter32: 0
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.11.10.11.1.112.8888 = Counter32: 0
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.11.10.11.1.113.8888 = Counter32: 62210801
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.11.10.11.1.114.8888 = Counter32: 0
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.11.10.11.1.115.8888 = Counter32: 0
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.12.10.11.1.112.8888 = Counter32: 0
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.12.10.11.1.113.8888 = Counter32: 3759499
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.12.10.11.1.114.8888 = Counter32: 0
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.12.10.11.1.115.8888 = Counter32: 0
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.13.10.11.1.112.8888 = INTEGER: 0
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.13.10.11.1.113.8888 = INTEGER: 387
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.13.10.11.1.114.8888 = INTEGER: 194
SNMPv2-SMI::enterprises.1991.1.1.4.24.1.1.13.10.11.1.115.8888 = INTEGER: 5

I define the Data Query in the straight-forward way, leaving no Associated Graph Templates at this point. I set the Data Input Method to "Get SNMP Data (Indexed)".

Next I define a Data Template using per-data source values for Name, Output Type ID, Index Value and Index Type. I define GAUGE type, and a sensible Maximum value of 10000.

Now I clone a Graph Template I like, and change all of the Template Items to use the new Data Template.

Next I go back to the Graph Template and attach the Graph Template to the Data Query. I name it 'Connections', drop-down select the Graph Template and define a suggested Name value for Data Template, as well as a suggested Title value for the graphs. Both look something like this:

|host_description| - RS: "|query_displayName|" (|query_ipAddress|:|query_port|)

Now I return to my test device (which points at a ServerIron) and add this Data Query, then select 'Create Graphs for this host'. At this point I see a large table full of graphs I can create. I select the Real Server I know will have active connections and click create. The Data Source gets created.

Here are the issues I am running into:
- The graph does not get populated. Yesterday it did, but remained empty.
- The Data Source shows:
Output Type ID: "Data query data sources must be created through New Graphs."
Index Value: [null]
Index Type: "Data query data sources must be created through New Graphs."

Shouldn't these fields be populated by the data query in question? I have templated all of my work based on the SNMP - Interface data query, but I do not get properly populated Data Sources. I assume it's because of the more complex nature of the indexing in the MIB. I just don't know enough about how Cacti deals with this to correct the issue.

Help!

[gandalf]

Please see http://docs.cacti.net/manual:087:3a_adv ... alkthrough for a complete walkthrough
R.

[askreet]

Gandalf,

Thank you very much for pointing me at this documentation, it looks very promising. I will follow it =)

Thanks again,
askreet

[askreet]

Hi Gandalf,

I followed the instructions you provided (and learned a few new things). One thing I did different was creating my own graph template rather than cloning another.

Anyway, I am still seeing the same results.


The created data sources show the above. Any ideas?

(edit)
I suspect it has something to do with my XML/Regexp. I re-created the XML while following the HOWTO, to be sure I did everything in accordance with the HOWTO. Of course, I am using an oid_index_parse, so it's a bit different than the example you provided.

Code: Select all

<interface>
        <name>ServerIron - Real Server Statistics</name>
        <description>Queries a ServerIron Load Balancer for Real Servers to be monitored</description>
        <oid_index>1.3.6.1.4.1.1991.1.1.4.24.1.1.1</oid_index>
        <oid_index_parse>OID/REGEXP:.*\.([0-9]*\.[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*)</oid_index_parse>
        <index_order>snL4RealServerPortStatisticIP:snL4RealServerPortStatisticPort:snL4RealServerPortStatisticServerName</index_order>
        <index_order_type>numeric</index_order_type>
        <index_title_format>|chosen_order_field|</index_title_format>

        <fields>
                <snL4RealServerPortStatisticIP>
                        <name>IP Address</name>
                        <method>walk</method>
                        <source>value</source>
                        <direction>input</direction>
                        <oid>.1.3.6.1.4.1.1991.1.1.4.24.1.1.1</oid>
                </snL4RealServerPortStatisticIP>
                <snL4RealServerPortStatisticPort>
                        <name>Port</name>
                        <method>walk</method>
                        <source>value</source>
                        <direction>input</direction>
                        <oid>.1.3.6.1.4.1.1991.1.1.4.24.1.1.2</oid>
                </snL4RealServerPortStatisticPort>
               <snL4RealServerPortStatisticServerName>
                        <name>Server Name</name>
                        <method>walk</method>
                        <source>value</source>
                        <direction>input</direction>
                        <oid>.1.3.6.1.4.1.1991.1.1.4.24.1.1.3</oid>
                </snL4RealServerPortStatisticServerName>
                <snL4RealServerPortStatisticCurrentConnection>
                        <name>Current Connections</name>
                        <method>walk</method>
                        <source>value</source>
                        <direction>output</direction>
                        <oid>.1.3.6.1.4.1.1991.1.1.4.24.1.1.7</oid>
                </snL4RealServerPortStatisticCurrentConnection>
        </fields>
</interface>

[askreet]

Bump, anyone got any ideas?

[askreet]

Some more information that may be helpful to anyone trying to help

The RRD files are not created in rra/.

The Poller Cache does not show any attempts to retrieve data from my test host which I've created graphs for using this data query.

[BSOD2600]

Anyway, I am still seeing the same results.


The created data sources show the above. Any ideas?

This is a sign of a Data Query, which must be created through the 'create graphs for this device' screen. Then these fields are automagically populated in the database. You cannot manually create these types of Data Sources.

[askreet]

Right, but these were made with a data query.

I actually resolved this at the end of the day yesterday. It was all an issue with my XML syntax. I am using an OID/REGEXP, which means I need a field which has an 'index' type in order to set it as the index order field.

Here is my new, working XML:

Code: Select all

<interface>
        <name>ServerIron - Real Server Statistics</name>
        <description>Queries a ServerIron Load Balancer for Real Servers to be monitored</description>
        <oid_index>1.3.6.1.4.1.1991.1.1.4.24.1.1.1</oid_index>
        <oid_index_parse>OID/REGEXP:.*\.([0-9]*\.[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*)</oid_index_parse>
        <index_order>snL4RealServerIndex</index_order>
        <index_order_type>alphabetic</index_order_type>

        <fields>
                <snL4RealServerIndex>
                        <name>Index</name>
                        <source>index</source>
                        <direction>input</direction>
                </snL4RealServerIndex>
                <snL4RealServerPortStatisticIP>
                        <name>IP Address</name>
                        <method>walk</method>
                        <source>value</source>
                        <direction>input</direction>
                        <oid>.1.3.6.1.4.1.1991.1.1.4.24.1.1.1</oid>
                </snL4RealServerPortStatisticIP>
                <snL4RealServerPortStatisticPort>
                        <name>Port</name>
                        <method>walk</method>
                        <source>value</source>
                        <direction>input</direction>
                        <oid>.1.3.6.1.4.1.1991.1.1.4.24.1.1.2</oid>
                </snL4RealServerPortStatisticPort>
               <snL4RealServerPortStatisticServerName>
                        <name>Server Name</name>
                        <method>walk</method>
                        <source>value</source>
                        <direction>input</direction>
                        <oid>.1.3.6.1.4.1.1991.1.1.4.24.1.1.3</oid>
                </snL4RealServerPortStatisticServerName>
                <snL4RealServerPortStatisticTotalConnection>
                        <name>Connections</name>
                        <method>walk</method>
                        <source>value</source>
                        <direction>output</direction>
                        <oid>.1.3.6.1.4.1.1991.1.1.4.24.1.1.8</oid>
                </snL4RealServerPortStatisticTotalConnection>
        </fields>
</interface>

[/code]

[BSOD2600]

great.

Once you get everything all working, please consider publishing to the template repository: http://docs.cacti.net/templates.guidelines

[fbrockerhoff]

Thank you for sharing - this helps me, too!

Regards,

Falk