Trouble with Data Queries.

[Frances]

Hi,

I'm having a bit of a problem and I'm hoping for some help or advice. I've written a data query for Cacti and created the associated data and graph templates, but alas, it doesn't work just yet. This is what's happening:

The data query works wonderfully. The data and graph templates seem to be healthy. However, when I add my new graph to the host, RRDTool doesn't create it on it's own. (I figure this out both by the error message and by the fact that when I watch my logs with logging set to DEBUG , it reports nothing regarding this particular RRD file). Here's the odd part. If I run the poller manually as root, it creates the RRD file, queries the device, pulls valid data, and updates the RRD file. If I run the poller manually as my Cacti user, it doesn't do anything in relation to this particular host.

It might be worth noting that even when I run the poller manually as root, and repeatedly (in order to update the graph manually), it doesn't graph any valid results. This might have to do with an error in the data or graph templates. But I suspect that either way, I've got to figure out why it is that when the Cacti user runs the poller, the logs don't mention this host, while when root runs the poller, everything looks fine.

I'm sure I've missed something simple, but I can't quite figure out what it is. Any assistance would be greatly appreciated.

P.S. I've placed this in the Linux/Unix forum because I get different results running as root vs. the Cacti user. I have checked permissions and ownership thoroughly (and set them recursively in case I overlooked something).

[TheWitness]

Well, it seems as if you have 2 problems. The first is graph rendering and the second is rrd creation.

So, my question is:
1) Is the graph broken or just empty?

If it's Broken:
1) Set logging to DEBUG and view the graph. Then go back to the log and "cut and paste" the "RRDTool graph" command to a shell and redirect it's output to a file. It will either complete, or you will get a beneficial error message.
2) Post that graph command here.

If it's Empty:
1) Please post your Data Source Debug information.
2) Dump your RRD file to an XML file and make sure it has data
3) Review the Poller_Cache, look for the entry
4) Review the Debug output for errors.
5) Run cmd.php from the command line and look for obvious STDERR issues that are not captured in the log.

TheWitness

[Frances]

Hi TW, thanks for your assistance once again.

Well, it's broken (as in will not generate the RRD) until I force it's hand by manually running the appropriate rrdtool commands from commandline. The commands complete successfully, with the caveat that the graph becomes an empty graph instead of a broken graph (continually shows 'nan' for all data). I determined the proper rrdtool commands by running poller.php manually as root -- when run as root, poller.php creates the RRD. When run as cacti, poller.php updates all existing RRD's, but does not create the new one. If I take the commands poller.php runs when started as root and run those as the cacti user, no problems. Additionally, if I run it as root and generate the RRD's and then chown them so cacti can touch them, poller.php still does not update them. The poller cacche does show the objects I'm trying to poll. DEBUG output looks great -- in fact, I can verify it's getting the propler numbers -- but only when I run as root. The instance of poller running as the cacti user doesn't seem to touch the files.

[Frances]

I ran the rrdtool create command manually as the cacti user, works fine. Very strange. Here's some output in the event that it's helpful for diagnosis -

From the DS debug:

Code: Select all

/usr/bin/rrdtool create \
/home/cacti/rra/my_private_data_source_643.rrd \
--step 60  \
DS:private_data_in:GAUGE:120:-100:100 \
DS:private_data_out:GAUGE:120:-100:100 \
RRA:AVERAGE:0.5:1:500 \
RRA:AVERAGE:0.5:1:600 \
RRA:AVERAGE:0.5:6:700 \
RRA:AVERAGE:0.5:24:775 \
RRA:AVERAGE:0.5:288:797 \
RRA:MAX:0.5:1:500 \
RRA:MAX:0.5:1:600 \
RRA:MAX:0.5:6:700 \
RRA:MAX:0.5:24:775 \
RRA:MAX:0.5:288:797 \

The graph debug says simply that it can't find the file:

Code: Select all

ERROR: opening '/home/cacti/rra/my_private_data_source_643.rrd': No such file or directory

And the poller cache shows:

Code: Select all

Data Source: Host - My - Data
RRD: /home/cacti/rra/my_private_data_source_643.rrd
Action: 0, OID: .1.3.6.1.4.1.5454.1.20.3.6.1.16.513 (Host: 172.16.233.10, Community: [sanitized]

The OID is definitely legitimate.

Have I missed something simple?

Thanks again.

[rony]

Permissions on the rra files and directory?

[Frances]

The rra directory is 755, owned by cacti/cacti. The RRD's themselves are 644, also owned by cacti/cacti. To top off the permissions question (which was my first thought), if I create a new graph using an existing template -- for instance, if I use the interface.xml query and start graphing non-unicast packets on an interface where I previously wasn't, it all goes off without a hitch.

[Frances]

Could it have anything to do with the fact that I'm graphing negative values? The max value for this item is 0 (which I set to 100 since RRDTool didn't seem to like 0) and the min value is -100. I've got other custom data templates I'm using successfully, but they're not indexed, so it's a bit simpler.

[TheWitness]

You should paste that "create" logic on the command line and see what happens.

TheWitness

[Frances]

Well that's the odd thing -- it creates the graph if I do that. The graph, however, does not update. The poller cache still makes it sound like it ought to be, but no change.

[TheWitness]

definately permissions unless you can't find the entry in the poller cache.

TheWitness

[Frances]

Is there any place other than the rra directory I ought to be checking permissions for? And does that jive with the fact that new graphs, besides the onre related to this particular data query, can be created?

[Frances]

As a side note, I created the RRD as the cacti user, and it has the same permissions and ownerships as the functioning RRD's. This is bizarre.

[Frances]

Also, when I dump the RRD into XML, it turns out it has been updated numerous times ; it just shows NaN, so nothing to graph according to RRDTool.

[TheWitness]

Sent you an e-mail.

TheWitness