Debugging Script Data Queries

[tianye]

Hi, I hope you can help me out here. I am struggling to graph traffic going through IPSec connections on some routers. I followed your howto (thanks a lot by the way) without any obvious problems.
I created the the new data input query with corresponding xml file, the data templates and associated some graph templates and added the query and graphs to a device.

I can see that my script was called when I created the data input query twice, to get the index and to query the only input field.
But after that Cacti never attempted to start the script and to actual gather some data. On the graph side I can see some references for graphs, but without pictures.
Do you have any idea why Cacti is not gathering any data?
Any help appreciated. Dirk

[gandalf]

Due to the general aspects to be discussed here, I created a new topic.

Hi, I hope you can help me out here. I am struggling to graph traffic going through IPSec connections on some routers. I followed your howto (thanks a lot by the way) without any obvious problems.

Please refer to Howto: from snmptable to XML Graphs (Data Query walkthrough). This should be the better starting point.

I created the the new data input query with corresponding xml file, the data templates and associated some graph templates and added the query and graphs to a device.
I can see that my script was called when I created the data input query twice, to get the index and to query the only input field.

Fine. For debugging purpose, it is best to post the XML file along with the output of a run of each of the following:
- index
- query ...
- get ...
Next, goto the device that you associated this new Data Query with. Run the Verbose Query for that query and post the textual output.
Reinhard

[tianye]

Okay, here we go. The XML file is attached. Here is the script output for "index" and "query index". The parameter "get" is not supported yet. First I wanted to figure out how Cacti is working.

root@gtlnmicmw0001:/var/www/cacti/scripts/test# ./cisco-ipsec-traffic.pl vpn-router index
80146186186
62180131131
21361155200
root@gtlnmicmw0001:/var/www/cacti/scripts/test# ./cisco-ipsec-traffic.pl vpn-router query index
80146186186:80.146.186.186
62180131131:62.180.131.131
21361155200:213.61.155.200
root@gtlnmicmw0001:/var/www/cacti/scripts/test#

And the output from "verbose query"

+ Running data query [12].
+ Found type = '4 '[script query].
+ Found data query XML file at '/var/www/cacti-0.8.6h/resource/script_queries/cisco-ipsec-traffic.xml'
+ XML file parsed ok.
+ Executing script for list of indexes 'perl /var/www/cacti-0.8.6h/scripts/test/cisco-ipsec-traffic.pl vpn-router index'
+ Executing script query 'perl /var/www/cacti-0.8.6h/scripts/test/cisco-ipsec-traffic.pl vpn-router query index'
+ Found item [remIpAddress='80.146.186.186'] index: 80146186186
+ Found item [remIpAddress='62.180.131.131'] index: 62180131131
+ Found item [remIpAddress='213.61.155.200'] index: 21361155200
+ Found data query XML file at '/var/www/cacti-0.8.6h/resource/script_queries/cisco-ipsec-traffic.xml'
+ Found data query XML file at '/var/www/cacti-0.8.6h/resource/script_queries/cisco-ipsec-traffic.xml'
+ Found data query XML file at '/var/www/cacti-0.8.6h/resource/script_queries/cisco-ipsec-traffic.xml'
+ Found data query XML file at '/var/www/cacti-0.8.6h/resource/script_queries/cisco-ipsec-traffic.xml'
+ Found data query XML file at '/var/www/cacti-0.8.6h/resource/script_queries/cisco-ipsec-traffic.xml'
+ Found data query XML file at '/var/www/cacti-0.8.6h/resource/script_queries/cisco-ipsec-traffic.xml'
+ Found data query XML file at '/var/www/cacti-0.8.6h/resource/script_queries/cisco-ipsec-traffic.xml'
+ Found data query XML file at '/var/www/cacti-0.8.6h/resource/script_queries/cisco-ipsec-traffic.xml'
+ Found data query XML file at '/var/www/cacti-0.8.6h/resource/script_queries/cisco-ipsec-traffic.xml'

I can see in the script log that the script is started twice for the "verbose query", but not again. As I already mentioned

[gandalf]

First, please provide /full/path/to/perl in the XML. At least with cactid, this is recommended
Next, the <arg_*> stuff is wrong. Omit |host_description|.
Please post same output after cganging as advised
Reinhard

[tianye]

I will add the full path to perl, however I don't think that is the problem, the "verbose query" is working fine.
I need the the |host_description| stuff in the <arg *> area to pass over some information for the device to work with. This seems to be the only way. Later I found the parameter <arg_prepend> in some XML file, but I didn't find any proper documentation on the exact use. I'm just guessing that it is for the same purpose.

[tianye]

Okay, here we go with the advised changes. The XML file is attached.

The verbose query output

+ Running data query [12].
+ Found type = '4 '[script query].
+ Found data query XML file at '/var/www/cacti-0.8.6h/resource/script_queries/cisco-ipsec-traffic.xml'
+ XML file parsed ok.
+ Executing script for list of indexes '/usr/bin/perl /var/www/cacti-0.8.6h/scripts/test/cisco-ipsec-traffic.pl index'
+ Executing script query '/usr/bin/perl /var/www/cacti-0.8.6h/scripts/test/cisco-ipsec-traffic.pl query index'
+ Found item [remIpAddress='80.146.186.186'] index: 80146186186
+ Found item [remIpAddress='62.180.131.131'] index: 62180131131
+ Found item [remIpAddress='213.61.155.200'] index: 21361155200
+ Found data query XML file at '/var/www/cacti-0.8.6h/resource/script_queries/cisco-ipsec-traffic.xml'
+ Found data query XML file at '/var/www/cacti-0.8.6h/resource/script_queries/cisco-ipsec-traffic.xml'
+ Found data query XML file at '/var/www/cacti-0.8.6h/resource/script_queries/cisco-ipsec-traffic.xml'
+ Found data query XML file at '/var/www/cacti-0.8.6h/resource/script_queries/cisco-ipsec-traffic.xml'
+ Found data query XML file at '/var/www/cacti-0.8.6h/resource/script_queries/cisco-ipsec-traffic.xml'
+ Found data query XML file at '/var/www/cacti-0.8.6h/resource/script_queries/cisco-ipsec-traffic.xml'

And the script output

www-data@gtlnmicmw0001:~/cacti/scripts/test$ ./cisco-ipsec-traffic.pl index
80146186186
62180131131
21361155200
www-data@gtlnmicmw0001:~/cacti/scripts/test$ ./cisco-ipsec-traffic.pl query index
80146186186:80.146.186.186
62180131131:62.180.131.131
21361155200:213.61.155.200

[tianye]

I'm going crazy here. Seems like it is working now. But I have no clue why the changes had this effect. I have to do some further checking. Thanks a lot so far! Dirk

[gandalf]

I will add the full path to perl, however I don't think that is the problem, the "verbose query" is working fine.

Yep, I know. But you will want this to work with cactid as well, I suppose.

I need the the |host_description| stuff in the <arg *> area to pass over some information for the device to work with. This seems to be the only way. Later I found the parameter <arg_prepend> in some XML file, but I didn't find any proper documentation on the exact use. I'm just guessing that it is for the same purpose.

Yep, you're right (even when it comes to the "documentation" bit of your statement). arg_prepend should be used. Hope, documentation comes up with this
Reinhard

[gandalf]

I'm going crazy here. Seems like it is working now. But I have no clue why the changes had this effect. I have to do some further checking. Thanks a lot so far! Dirk

No need to get crazy if things got working The other way round is found more often
Ah, BTB, would you please mind posting your results (perhaps along with a bit of text and a nice screenshot) at the "Scripts and Templates" forum?
Reinhard

[tianye]

No need to get crazy if things got working The other way round is found more often

I'm beyond going crazy, close to a nervous breakdown
It was working just as long as it took to post the message...
My data gathering script was a couple of times called with the parameter "get". I would have expected to see some "query". However, suddenly it stopped working again I thought that maybe my database is messed up, so I exported all relevant templates, rebuilt the database from a backup and imported the templates again. No change. The debug information I gave before are still valid.
Is there a way to see if Cacti attempted to gather some data? I set the log level to "debug", but didn't found any relevant data.
Does anyone have a working an index script query at all? Is it possible that there is a general problem? I'm using Cacti 0.8.6h.

[gandalf]

Is there a way to see if Cacti attempted to gather some data? I set the log level to "debug", but didn't found any relevant data.
Does anyone have a working an index script query at all? Is it possible that there is a general problem? I'm using Cacti 0.8.6h.

Script Queries are seldom used, do not perform that well and (hmmm) not that good documented. AFAIK, you do already know all debugging stuff; last resort would be code debugging, sorry.
Reinhard

[tianye]

Even if it is an embarrassment I have to admit that the problem was in front of the computer and not inside. I messed up the file permissions and now everything is working. Yet one question remains: I can see that there is an entry to call the script with the parameter 'get' in the table 'poller_item' for every data source item. I guess the 'query' option would be much more efficient. Any ideas how the cacti will asked the script to do a query?
btw, thanks to lvm who already spent some time to help me with the other stuff