[SOLVED] New to XML parsing, needs help howto..

[FrodeS]

Hi,

I've been using Cacti for seven years to graph traffic and other cable-modem related details at our ISP. With great success. I'm also graphic stuff like IP-address usage for documentation that instances like Ripe needs to have access to.

Now I've been challenged by a change of vendors where the previous system was easily accessible by SNMP to read out interface statistics and thew new platform only allows for SNMP to be used for alarms. All other details can be read out by using XML.

So I've gotten so far that I created a "response.xml" file and sent it to the system using curl and got a "response.xml" file back.

Now I don't know how to get the content of the response.xml file parsed and presented in several graphs. The response.xml file contains interface statistics like bandwith usage for several interfaces and I need one graph per interface.

Is there an easy way to do this or a tutorial that I've overlooked in my search in these forums?


Best regards,
FrodeS

[gandalf]

You may write an script that uses an XML parser to get raw data.
Then, the script may spit out that raw data like a standard "Data Input Method".
Unfortunately, it seems that all data is fetched in a single curl request. While this sounds optimal performance-wise, it is not easy to extend this approach when new data fields come into sight. So you may end up with a cacti data input method that spits out dozens of fields, feeding them into a HUGE rrd file with dozens of data source items.
R.

[FrodeS]

I've been using Cacti for a while, but haven't written any scripts or such so far so that might be a tricky part.

As you correctly point out we're receiving all the data for all interfaces in one single curl call resulting in getting all the data returned in one xml file. I need to have the various interface statistics shown in one graph per sub-interface due to the values often being to close to each other to be able to separate them in a graph.

Do you know of someone else that's done this successfully and that's sharing their scripts and input methods? I haven't been able to find any in my searches, but that might be due to the word xml showing up in so many posts that's it nearly impossible to find the correct post(s). Or it may be due to my lack of searching skills..

Is this something that someone at these forums would be able to help me achieve, as a consultancy based payed task if so? We're not fully dependant of such graphs in our daily operations, but it would sure be nice to have them in place.


Best regards from Norway,
Frode.

[gandalf]

Do you know of someone else that's done this successfully and that's sharing their scripts and input methods? I haven't been able to find any in my searches, but that might be due to the word xml showing up in so many posts that's it nearly impossible to find the correct post(s). Or it may be due to my lack of searching skills..

Personally, I'm not aware of any contribution of this kind during the last 4 years. And from my post count, you may deduce how much you want to rely on this statement.
In case your facing changes to the related XML files (e.g. by adding new interfaces or other data), I strictly discourage doing this in a BLOB way of solution. Even at the risk of multiple polling, you should separate fetching different parts of raw data.
You may want to create kinda "local file cache" to fetch the data only once and parse it multiple times, once for each "data group (aka: interface)".
In case you are able to provide a means to separate the data at the origin (target device), this is preferred.
For parsing the XML structure, you surely will want to use a XML library of the programming language of your liking.

For any support, I'm quite sure that remote access to a test target device is required.
R.

[xolo]

interesting for me, thanks

[FrodeS]

I've now got a perl script created that outputs as mentioned in http://www.cacti.net/downloads/docs/htm ... cacti.html.

I don't know how to move from here on how to get the output from the script graphed. I'm sure that there's a walktrough written that shows me howto but I'm not able to find it.

When I run my script with the input of <slot> and <frequency> I get the wanted data displayed on the screen like this:
frequency:4 totalRate:50.655 outputRate:39.453

So the question then is: How to get this graphed? Sorry for being a noob but I haven't worked with these details in Cacti before..


Best regards,
FrodeS

[FrodeS]

By reading and checking how other similar input methods is configured in Cacti, I got to create the other templates and data sources.

The only thing left now is to get data from the script within Cacti, all I get now when the poller runs is this line in the log file:
WARNING: Result from CMD not valid. Partial Result: U

This is the log entry when I turn on more detailed logging:
CMD: perl /var/www/cacti/scripts/slot_frequency_info.pl 15 1, output: U

It seems to me that the script doesn't get the correct output when it's run through Cacti, but works excellent when run from the shell, then the output is like this:
frequency:1 totalRate:50.649 outputRate:19.366

Best regards,
Frode.

[FrodeS]

I've found and corrected the problem. Graphing is now working as expected.

The problem was that my perl-script was trying to read from a XML-file that was in the process of being updated which made the file locked for reading.

Since the XML file being read every five minutes has got the same name every time, I added a command to copy the previous versjon of the XML file to a different filename before reading the next version of the XML file. And then I changed the perl script to read from the copied version instead of the active XML file.

All fixed - graphing fine now - problem solved.


Best regards,
Frode.