[HOWTO] Add new MIB Variables to Cacti

[Lux]

I have seen a lot of questions on this in the last few months. So here is a (not so) quick how-to for adding new indexed MIB variables to Cacti.

Mike



HOWTO – Add a new indexed OID to Cacti

There are two possible ways to add a new indexed MIB variable to Cacti. The first, and easiest, is to modify an existing SNMP query. The second, is to create a new SNMP data query from scratch. Since modifying an existing query is simply a subset of creating a new query, only the second will be described.
This how-to will lead you through the steps of creating a completely new SNMP data query and integrating it into Cacti, using the CPU utilization of a Compaq Proliant server as an example.

Step 1: Create a new SNMP query XML file
First we need to create an XML file to tell Cacti what information to get and how to get it. The most difficult part of this step is determining the exact MIB variables to use. Proliant servers have a proprietary MIB that contains the 5 minute average CPU utilization for each processor.
MIB variables stand for one piece of information like total traffic, errors, discards, multicast packets..... However, it is often possible to have more than one source for this data (I.E. a switch can have several hundred network interfaces). To keep these interfaces separate when querying a device for a single piece of information, an index value is added to the end of the normal MIB variable. This index value is used to distinguish one interface from the others on all MIB variables associated with that interface. However, this value is not fixed. It can change any time that the device is rebooted. Thus we must tell Cacti what MIB variable contains the index, and then tell it what variables it can poll, using this index, to get some useful information. In the case of our proliant server, the index is contained at 1.3.6.1.4.1.232.11.2.3.1.1.1 The actual utilization is located at .1.3.6.1.4.1.232.11.2.3.1.1.3
The following is the XML file that needs to be created for this new query.

<interface>
<name>Get Compaq Processors</name>
<description>Queries a host for a list of monitorable processors</description>
<oid_index>.1.3.6.1.4.1.232.11.2.3.1.1.1</oid_index>

<fields>
<cpqHoCpuUtilIndex>
<name>Index</name>
<method>walk</method>
<source>value</source>
<direction>input</direction>
<oid>.1.3.6.1.4.1.232.11.2.3.1.1.1</oid>
</cpqHoCpuUtilIndex>
<cpqHoCpiUtilFiveMin>
<name>5 Minute Utilization</name>
<method>walk</method>
<source>value</source>
<direction>output</direction>
<oid>.1.3.6.1.4.1.232.11.2.3.1.1.3</oid>
</cpqHoCpiUtilFiveMin>
</fields>
</interface>

Name and description are optional fields and have no affect on the operation of Cacti. The oid_index field should contain the MIB variable that contains the index for this query.
Anything inside of the <fields> section will be queried using the index values gathered from the oid_index. There must be at least two variables in the fields section in any data query: one to use as an index and one gathering data to plot.
In the example above, there are two variables given in the fields section: cpqHoCpuUtilIndex and cpqHoCpiUtilFiveMin. The names can be anything so long as they only contain characters from a-z and A-Z. Numbers and punctuation will cause an error. T is safe to leave method set to walk for most queries. The source value allows you to do some processing on the value that the query returns. In this simple case, we will leave this set to “value”. The Direction field is very important. Input fields are fields that do not change over time. These are generally things like the index value, a description field, or a name. Input fields will be used within cacti to index the MIB mariables. Output fields are those fields which can be graphed.
Once you have created your XML file, save it in the <cacti-root>/resource/snmp_queries/ directory. We will call our file compaq_cpu.xml. Do not use any spaces or special characters in the file name as it may cause problems later.

Step 2: Create a Data Query in Cacti
Open the Data Queries menu in Cacti and click on the add button at the top right hand side of the screen. Fill in the fields as appropriate:
Name: your name for this query
Description: a description of this query… What does it do?
Xml Path: The path to your query. In this example it should read “<path_cacti>/resource/snmp_queries/compaq_cpu.xml”
Data Input Method: Select the type of query. Since this is SNMP, choose Get SNMP Data (Indexed)

The click on “Create”. This will bring up some additional fields. In the Data Input Method section, the first three options should almost always all be checked and have the following values:
Index Type: SNMP Field Name (Dropdown)
Index Value: SNMP Field Value (From User)
Output Type ID: SNMP Output Type (Dropdown)

Click on Save.

Step 3: Create a Data Template
Navigate to the data template screen and click on the “add” button at the top right hand side of the screen. In the screen that appears fill in the fields as follows:
Name (Data Templates section): the name for the data template
Name (Data Source Section): The name that cacti will automatically generate for all new data sources of this type. The author has used: |host_description| CPU |query_cpqHoCpuUtilIndex|.
Data Input Method: Get SNMP Data (Indexed)
Internal Data Source Name: Choose a name that only uses alphanumeric characters. This is for Cacti Internal use only. The author has used compaq_cpu.
Minimum Value: Since this is a % utilization, 0 is the best choice.
Maximum Value: It is impossible to have more than 100% utilization, so enter 100.

Click on Save. This will bring up yet more fields to fill in. Place a check mark next to the fields: Index Type, Index Value, and Output Type ID and click on save.

Step 4: Create a Graph Template
Navigate to the graph template screen and click on the “add” button at the top right hand side of the screen. In the screen that appears type in a name for the graph template and click on save. This will display some additional sections.
Click on the add button to the right of the Graph Template Items section. Fill in the fields as follows:
Data Source: Select the data source that you created in the previous step.
Color: Pick something that you like.
Graph Item Type: Since we are only graphing one item, select AREA.
Consolidation Function: Select Average
CDEF Function: None
Value: Leave this entry blank
Gprint Type: Normal
Text Format: This will be displayed at the bottom of the graph. Enter something logical like “CPU”
Click on save.
You can continue adding other items to display the maximum, minimum, and current CPU utilization. However, they are not necessary, and will not be covered here.
Next click on the add button to the right of the Graph Input Items. Enter a name for the data source and place a check mark next to Item #1. This will link the data source with a graph item. Click on Save.

Step 5: Create a Host Template
Open the Host Templates menu and click on the “add” button. Enter a name for the Host Template and click on save. Reopen this template. (I have not found a way to do this without having to reopen the template. If anyone has any suggestions, I would love to hear them) Select the data query that you created earlier from the drop down box under the “Associated Data Queries” section and click on add.

Step 6: Add the Graph template to the Data Query
Open up the Data Query, scroll down to the “Associated Graph Templates” section and then click on Add. Enter a name for this graph and select the graph template from the drop down window. Click on save. You should now be able to use your new MIB variables.

[oharel]

Hi Lux,

exactly what i was looking for! thanks.
i seem to have a problem though:
i followed your instructions to the letter but the rrds are not created.
i see in the data source:
Custom Data
Index Type - Data query data sources must be created through New Graphs.
Index Value 0 (i put it myself)
Output Type ID Data query data sources must be created through New Graphs.

when running verbose query on the device, i get:
Data Query Debug Information
+ Running data query [10].
+ Found type = '3' [snmp query].
+ Found data query XML file at '/var/www/htdocs/cacti-0.8.5a/resource/snmp_queries/ipricot_ebno.xml'
+ XML file parsed ok.
+ Executing SNMP walk for list of indexes @ '.1.3.6.1.4.1.6697.1.1.1.1.18.0'
+ Located input field 'ebNoOfIpricotIndex' [walk]
+ Executing SNMP walk for data @ '.1.3.6.1.4.1.6697.1.1.1.1.18.0'
+ Found item [ebNoOfIpricotIndex='120'] index: 0 [from value]

and indeed the value i am looking for is 120, as appears in ebNoOfIpricotIndex='120'

whe clicking on create graphs for this host, i get both Graph Template Name (which is fine i guess) and the data query -> index, which happens to be whatever it is at the time. i am adding a picture of this, so you know what i am talking about

then, when i click create i see:
Index Type - i put Integer
Index Value - i put 0
Output Type ID - i put ebn

what am i doing wrong here?
why no rrds are created?

thanks
--harel

[Lux]

oharel, I'm not sure that I understand where you are having difficulty. Are you trying to create your graphs from the new graphs menu or are you manually creating your data sources and graphs? Are you even getting to this point?

Mike

[oharel]

Hi Lux,

thanks for the reply, but happily enough, the problem was solved

-harel

[Guest]

I'm getting the same type of thing oharel is. I've selected multiple items in my XML, but all it shows is the index. On the edit graph page, it lists a place for me to put in custom data like OID, Output type, etc - do I need to do anything with that? Here's the XML I'm using:

<interface>
<name>Get PowerEgg Info</name>
<description>Queries a PowerEgg and returns power information</description>
<oid_index>.1.3.6.1.4.1.17373.2.3.1</oid_index>

<fields>
<ifindex>
<name>ifIndex</name>
<method>walk</method>
<source>value</source>
<direction>input</direction>
<oid>.1.3.6.1.4.1.17373.2.3</oid>
</ifindex>
<voltpk>
<name>Volts - Peak</name>
<method>walk</method>
<source>value</source>
<direction>output</direction>
<oid>.1.3.6.1.4.1.17373.2.3.1.10</oid>
</voltpk>
<voltmin>
<name>Volts - Min</name>
<method>walk</method>
<source>value</source>
<direction>output</direction>
<oid>.1.3.6.1.4.1.17373.2.3.1.9</oid>
</voltmin>
<voltmax>
<name>Volts - Max</name>
<method>walk</method>
<source>value</source>
<direction>output</direction>
<oid>.1.3.6.1.4.1.17373.2.3.1.8</oid>
</voltmax>
<volts>
<name>Volts - Current</name>
<method>walk</method>
<source>value</source>
<direction>output</direction>
<oid>.1.3.6.1.4.1.17373.2.3.1.7</oid>
</volts>
</fields>
</interface>

[JDP]

Opps, the above is me! Another followup - does this query only work for 1 OID per <interface>? Am I supposed to do something different in the XML to allow more?

It also doesn't seem to be working at all - running the graph in debug mode and I get
ERROR: opening 'E:\www/rra/test_netmon_test_xml_100.rrd': No such file or directory

[Lux]

JDP, looking at your XML file I only see one problem with your XML file: you list a different OID for the index from <oid_index> and <ifIndex> If the <ifIndex> field is the correct OID for your index, then you will have problems with the initial scanning for interfaces. If the <oid_index> is the correct version then you are going to have some really screwy results.

In threory, you should be able to track any OID that uses the same index as the one that you list in <oid_index> with the same XML file. Take a look at the interfaces.xml file and the related templates in cacti, it should help.

Mike

[Guest]

Ok, thanks Lux. I'll work on it and see if I can't figure out what's going on!

[imthemp3king]

Is there a way to have a data query index an OID and put all the output on one graph? I have been trying to get the voltage info from a Dell server in one graph using a data query, but it wants to create individual graphs for each output, even though the graph template I have created already has the what I want graphed specified.

<interface>
<name>Get Voltage Probe Readings</name>
<description>Queries a host for a list of voltage probe readings</description>
<oid_index>.1.3.6.1.4.1.674.10892.1.600.20.1.2</oid_index>

<fields>
<voltageProbeIndex>
<name>Voltage Probe Index</name>
<method>walk</method>
<source>value</source>
<direction>input</direction>
<oid>.1.3.6.1.4.1.674.10892.1.600.20.1.2</oid>
</voltageProbeIndex>
<voltageProbeLocationName>
<name>Probe Location</name>
<method>walk</method>
<source>value</source>
<direction>input</direction>
<oid>.1.3.6.1.4.1.674.10892.1.600.20.1.8</oid>
</voltageProbeLocationName>
<voltageProbeReading>
<name>Probe Reading</name>
<method>walk</method>
<source>value</source>
<direction>output</direction>
<oid>.1.3.6.1.4.1.674.10892.1.600.20.1.6</oid>
</voltageProbeReading>
</fields>
</interface>

[anon]

ERROR: opening '/opt/cacti-0.8.5a/rra/sansw8b_traffic_in_16.rrd': No such file or directory

is what i get on my graphs if i run them in debug mode... is this something stupid im doing? it would be really nice if there was somewhere i could find a document that explained the design behind the cacti system... the howto is great, but im just following it blindly without really having any clue whats going on...

thx!

[Lux]

Imthemp3king,
I'm not sure what your problem might be, but a good starting point would be to take a look at the Interface Errors/Discards graph template. That template is graphing two separate OIDs on the same graph like you would like to do. You will probably need to look at the associated data queries, but it should help..

Anon,
You may be having a problem with permissions. Do any of your other graphs work?

I have another document that roughly explans how cacti works, or rather, it explains what the different parts of the installation do (mysql, apache, rrdtool....). I'll dig that up and post it when I get the chance.

Mike

[Lux]

As promised:


MySQL: MySQL is a popular Open-Source database server. The program is used by Cacti to store the vast majority of user configurable settings. Everything from data sources, templates, graph settings, and user accounts are stored in the MySQL database. There is one major exception to this rule: the raw data used to produce the graphs. This information is stored in Round Robin Archives produced by the RRDTool.

RRDTool: RRD stands for Round Robin Database. Since Cacti was written to be a front end to the RRDtool, one of the best resources on how cacti works is actually the rrdtool homepage. RRDTool provides two services to Cacti. Firstly, it stores and manages the data collected by Cacti. Secondly, it draws the graphs that Cacti displays. The tool does not care about where the data comes from or what it represents, so long as it is in a numeric format. RRDtool stores the collected data in external files called Round Robin Archives (RRA). Each data source will have its own RRA. By default, each RRA is broken into four segments. In a default RRA, each entry in the first segment contains the data from one polling interval (5 minutes). In the second segment, each entry represents the average of 6 polling intervals, or 30 minutes. The third is an average of 4 entries in the 2nd section, or 2 hours of data collection. Finally the 4th section is an average of 12 polling intervals from the 3rd section, or 24 hours. Thus the 1st section is used to produce the daily graph, the 2nd the weekly graph, the 3rd the monthly graph, and the 4th the yearly graph. Each of these sections has a fixed number of records or data points that they can hold. When all of the records are filled, the oldest records get overwritten with new data. Because of this design, the size of the RRA file is determined at the creation of the file. It will never grow or shrink.

Apache: Apache is the most common web server in the world. It is the program that receives the user’s input and transmits the resulting output back to the user. When a user sends a request to the server, Apache sends the appropriate Cacti scripts along with any user data to the PHP interpreter. PHP processes the scripts and returns the results to Apache, which forwards these results over the network to the end user.

UCD/NET-SNMP: UCD-SNMP or NET-SNMP provides snmpwalk and snmpget along with a host of other programs that are not used by Cacti. PHP comes with its own implementation of snmpwalk and snmpget, however the PHP SNMP implementation only supports SNMP version 1. If version 2c or 3 are necessary, UCD or NET SNMP must be used. Regardless of the SNMP implementation used, the results of the snmp polls are forwarded to the RRDtool for processing. Note: NET SNMP is the successor to UCD SNMP. Cacti will happily work with either of these programs. Cactid, the new C based poller for Cacti is linked directly against net-snmp. <speculation> This should mean that a separate snmp binary is unnecessary for cacti to function. <speculation/>

PHP: PHP is a server side scripting language. In a poor analogy, PHP is the server side equivalent of Java. PHP is most commonly used to produce web pages with dynamic content such as Cacti.

Cacti: All of the above mentioned programs have a binary executable. Cacti, on the other hand, is simply a series of scripts that provide a user friendly interface to the other programs described above. It coordinates and bonds the other programs together, providing a seamless environment to the user.

Cactid: Cactid is the next generation poller for cacti. It is responsible for collecting statistics and placing them into the RRD files. Cactid is written in C and is multithreaded, meaning that the program can fork into multiple processes to complete the device polling quicker than doing everything serially. Cactid is still in the beta phase of its development, however it appears to be fairly stable for SNMP statistics gathering.

[jasper]

I've tried to follow these steps to build a new template for a Falcon monitoring device (one of those things that has about 20 or 30 inputs for building monitoring devices like temperature, humidity, etc..) but seem to be lost.

The oid's are pretty much non-standard and i had to figure out how to do a command line snmpwalk -O n to list them all. The ones i want to graph aren't indexed like interfaces, as far as i can tell, so how would the syntax go for this? Would it be easier to get the values in a script like i noticed the CPU template does?

[nrs]

When following the howto, I have a problem with step 2.2 (Create a Data Query in Cacti).

When pressing 'create' there shuld be displayed additional fields like 'index Type' / Index Value: / Output Type ID: ), but all what I get is :

Successfully located XML file and a block called "Associated Graph Templates". (no extra fields at all)

And I really wonder what's wrong. (and yes, I have selected "Data Input Method" as "get SNMP data (indexed)"

I wonder if somthing have gone wrong with the definition in "Data input methods / get SNMP data (indexed) or does it have somthing to doo with version 0.8.6 ?

[..::BFS::..]

When following the howto, I have a problem with step 2.2 (Create a Data Query in Cacti).

When pressing 'create' there shuld be displayed additional fields like 'index Type' / Index Value: / Output Type ID: ), but all what I get is :

Successfully located XML file and a block called "Associated Graph Templates". (no extra fields at all)

And I really wonder what's wrong. (and yes, I have selected "Data Input Method" as "get SNMP data (indexed)"

I wonder if somthing have gone wrong with the definition in "Data input methods / get SNMP data (indexed) or does it have somthing to doo with version 0.8.6 ?

My guess it's the different version of Cacti. I run in to the same problem...