updated data query script, now cacti index returns 0 results

[sully6119]

I'm not sure where to begin. We made some modifications to the output of one of our in-house scripts and now when you look at the device under "Associated Data Queries" I see:

Code: Select all

Sandboxes  (Verbose Query)  Uptime Goes Backwards  	 Success [0 Items, 0 Rows] 

However, if ran from the command line:

Code: Select all

$ ./rca_query_sandboxes.py index | wc -l
48

Same number returned since before the script was updated.

Oddly, if I run:

Code: Select all

$ /u/cacti/cli$ php -q poller_reindex_hosts.php -id=4360 -d

I see:

Code: Select all

Sandboxes  (Verbose Query)  Uptime Goes Backwards  	 Success [144 Items, 48 Rows]

But if I click "Verbose Query" it goes right back to 0 Items, 0 Rows.

This is on a machine that I happen to have ssh user access to, and limited admin rights in the GUI. The actual cacti admin is at a loss here so he gave me enough privs to do my own investigation, and I've spent 2 days messing around with it. The script output isn't that much different than before. That, and it works on my development Cacti server that we use to test this stuff.

Development:

Code: Select all

+ Running data query [11].
+ Found type = '4 '[script query].
+ Found data query XML file at '/u/cacti-0.8.7e/resource/script_queries/rca_sandboxes.xml'
+ XML file parsed ok.
+ Executing script for list of indexes 'python /u/cacti-0.8.7e/scripts/rca_query_sandboxes.py index'
+ Executing script query 'python /u/cacti-0.8.7e/scripts/rca_query_sandboxes.py query node'
+ Found item [rcaNode='p9_PC_15002_c_1_1'] index: p9_PC_15002_c_1_1:<ip_address>:16000
+ Found item [rcaNode='p9_PC_15010_c_1_1'] index: p9_PC_15010_c_1_1:<ip_address>:16020
...
... more similar items...

Production:

Code: Select all

+ Running data query [37].
+ Found type = '4 '[script query].
+ Found data query XML file at '/tmp/rca_sandboxes.xml'
+ XML file parsed ok.
+ Executing script for list of indexes '/opt/python-2.4.6/bin/python /tmp/rca_query_sandboxes.py index'
+ Executing script query '/opt/python-2.4.6/bin/python /tmp/rca_query_sandboxes.py query node'
+ Executing script query '/opt/python-2.4.6/bin/python /tmp/rca_query_sandboxes.py query address'
+ Executing script query '/opt/python-2.4.6/bin/python /tmp/rca_query_sandboxes.py query port'
+ Found data query XML file at '/tmp/rca_sandboxes.xml'
+ Found data query XML file at '/tmp/rca_sandboxes.xml'
+ Found data query XML file at '/tmp/rca_sandboxes.xml'

(a hundred or so more of the previous message)

I've diff'd both the XML and script from dev to prod and there are no differences.

Does anyone have any helpful debug advice here?

[sully6119]

Resolved. Running the poller re-index as the cacti user fixed it. I was running it as me and getting the behavior described above.

I still have no idea why any of this was necessary in the first place though. If someone does, please clue me in!

[linuxpro747]

I'm having the same problem. Did you ever find a fix?

When I create a (perl or PHP) data query script, it returns the proper indexes and behaves exactly as it should according to the Script Data Query Walkthrough.

When I add the Data Query to a device in Console->Devices->"hostname"->Associated Data Queries, it will show: "Success [0 Items, 0 Rows]".

When I rebuilt the poller index for that host and clicked on "Reload Data Query" or "Verbose Query", it will show "Success [6 items, 6 rows]".

As soon as I try to add graphs to the host from this screen, however, it goes back to "Success [0 Items, 0 Rows] ".

Rebuilding the poller index for this host will start the whole cycle over.

Anyone know of a fix? If I can find one, I will add appropriate documentation to the Wiki. I have not been able to get data script queries working no matter what I've tried, even following the Script Data Query Walkthrough exactly, I still have this issue. I'm currently running Cacti version 0.8.7d.

Sorry if this question has already been asked, but I searched the forum and it doesn't let me view any results past the first page.

[sully6119]

/cacti/cli/$ php -q poller_reindex_hosts.php -id=<graph_id> -d

That is how I rebuilt the index. The trick was, I had to su to cacti user and run that re-index. If I ran it as my user I saw the same behavior you are getting. Back to 0 items.

[sully6119]

I should also add that our script was previously working without issue, but after some small modifications began acting this way.

Definitely keep permissions in mind whether it be the files themselves, or who they are executed by.

[linuxpro747]

That's the problem, though! I'm doing exactly what you just said. I rebuild the poller index for the host as the cacti user from the CLI using that exact command, and the problem just starts happening again. I can only see data once. If I do ANYTHING else, it immediately falls back to 0 results. It's driving me crazy.

[linuxpro747]

It's not a permissions issue. The script is being run as the cacti user, it is owned by the cacti user, it is in the cacti directory, etc.

Perhaps I'm not returning data properly from the script?

If I call the script like:

Code: Select all

cacti#> php -q scripts/query_linux_iostat.php $HOST index

It returns the correct disk listing, like:

Code: Select all

sda
sdb
md0
md1

When I run:

Code: Select all

cacti#> php -q scripts/query_linux_iostat.php $HOST query util

It returns:

Code: Select all

sda:0.17
sdb:1.24
md0:0.00
md1:0.00

etc...

Also, the poller reindex trick works until you run a Verbose Query in the Associated Data Query window. Then it shows "Success [0 Items, 0 Rows]" again. Rebuild the poller reindex again, it shows "Success [6 items, 6 Rows]" even though I have made no changes to any files or anything.

Also, if I try to "Add Graphs to Host" from that same page (Console->Devices->"host"), the Data Query shows nothing in the index field. The whole window is blank, even though the script returns the appropriate index from the CLI (as the cacti user).

I'm totally exhausted after a week of troubleshooting this issue.

[sully6119]

Also, the poller reindex trick works until you run a Verbose Query in the Associated Data Query window. Then it shows "Success [0 Items, 0 Rows]" again. Rebuild the poller reindex again, it shows "Success [6 items, 6 Rows]" even though I have made no changes to any files or anything.

That is the same *exact* behavior I saw. Perhaps it's a bug? Calling poller_reindex_hosts.php against that id is what fixed it for me. Did you try that yet?

The really puzzling thing is, the script was not changed between the broken stage to the working stage, so there is something screwy with the way cacti is working on it.

I'm almost scared to run Verbose Query at this point. I too spent an entire week on this. I've since created over 300 graphs now that it is working. I'm stuck in cacti hell for at least another week before I'm finished with all this...

[linuxpro747]

Yeah, I've been running the "poller_reindex_hosts.php" with the specific ID of my host. And yeah, the problem continues. Temporarily goes away like shown, but if you do a Verbose Query, it comes right back.

Also, even after it starts showing some results, it isn't showing anything for the index. My script returns the proper index, but Cacti just isn't seeing it for some reason. The associated Data Query window in "Add Graphs to Host" is totally blank, even though the script is returning the index properly (newline-terminated list of items with unique names).

I'm exhausted, stumped and brain-dead after a week of trying to fix this, thinking I was doing something wrong. After seeing the problem while following the Script Data Query Walkthrough to the letter, I'm realizing this is probably a serious bug. Has anyone else seen it? Are there are any known workarounds?

[gandalf]

There are some new patches out there. Please apply and retry. Else file a bug report as given by http://www.cacti.net/bugs.php
Reinhard

[doughairfield]

I'm also having the same problem. A reindex of that host works, and ~60 hosts up until now worked fine. Was there any resolution to this problem?