Troubleshooting cacti - Graphs not being generated

GoranTornqvist · Post by **GoranTornqvist** » Thu Nov 18, 2004 9:32 am

Sorry, my bad, I was also running a cmd.php process on the system, that's why I got some of errors. doh!

I still got problems though with the apache script with multi-output values, from the logfile:

11/18/2004 03:25:03 PM - CACTID: Poller[0] Host[1] SCRIPT: perl /home/admin/domains/mydomain.com/public_html/cacti-0.8.6b/scripts/apache.pl baltazar, output: reqs_sec: reqs_cur:

It's not returning any values when the poller is run through crontab (both roots and cactiuser), when I run apache.pl from the commandline it returns the values. (both as root and cactiuser)
If I run the poller from the commandline as root or cactiuser, its the same as above, no return values...hmmm.

Also applied all the patches in:
http://www.cacti.net/downloads/patches/ ... e-patched/

I also built another apache script with single return value, same error:

11/18/2004 04:40:02 PM - CACTID: Poller[0] Host[1] ERROR: Empty result [baltazar]: 'perl /home/admin/domains/mydomain.com/public_html/cacti-0.8.6b/scripts/apache-status.pl baltazar reqs_cur'

Same thing here,
apache-status.pl baltazar reqs_cur
works when I run it from the commandline.

Post by **TheWitness** » Thu Nov 18, 2004 8:49 pm

Your first post had the errors, the second post did not. You did not run in DEBUG mode as I had requested. You must run in debug mode if you want me to completely assess.

TheWitness

GoranTornqvist · Post by **GoranTornqvist** » Wed Nov 24, 2004 7:27 am

Here we go again

This is from the logfile when cactid is running in debugmode.
Look for "apache-status", its that script thats is bugging me.
Same as before, running the same script from commandline as the cactiuser is giving me a numeric value as it should.

Post by **TheWitness** » Wed Nov 24, 2004 11:09 am

What version of Cactid are you running. There was a bug in 0.8.6b that caused truncation with scripts. Upgrade to 0.8.6c and your problem shuld be solved. If you have problems with Segfaults, download the cactid.c from CVS prior to making.

TheWitness

GoranTornqvist · Post by **GoranTornqvist** » Wed Nov 24, 2004 4:17 pm

As the logfile says: "CACTID: Version 0.8.6c starting".

Is there maybe a way I can run the apache script with cactid manually...hmmm.
I just don't know where to start when it comes to debugging this error. Any ideas what I can try?

Segfaults. How do I know if cactid segfaults?
The logfile doesn't looks ok and no segfaults in the syslog.

Post by **TheWitness** » Wed Nov 24, 2004 5:02 pm

Here is the command that you will run from a shell:

"perl /home/admin/domains/mydomain.se/public_html/cacti-0.8.6b/scripts/apache-status.pl hm.mydomain.se reqs_cur"

Good Luck!!

TheWitness

GoranTornqvist · Post by **GoranTornqvist** » Tue Nov 30, 2004 9:48 am

I tried running the above command from cmdline and the result was correct.

I gave up using my own scripts and templates so I downloaded scripts that I found here in the forums.
This apache script worked great:
http://forums.cacti.net/viewtopic.php?t=4028

This mysql script didnt work though, same error message in the log as before.
http://forums.cacti.net/viewtopic.php?t ... sc&start=0

The error message is:
WARNING: Result from SCRIPT not valid. Partial Result: U...

But in the thread above there were a few of people having the same problem.

Is this a "bug"...?
Does anyone have a clue what to do?
Should I try reinstalling cacti...?

Post by **TheWitness** » Tue Nov 30, 2004 8:50 pm

Cacti 0.8.6x is more sensitive to erroneous data in script output. For example:

VariableA:ValueA VariableB:ValueB is correct, while

VariableA:ValueA mbps VariableB:ValueB is incorrect and may have worked in prior versions of Cacti.

TheWitness

GoranTornqvist · Post by **GoranTornqvist** » Thu Dec 02, 2004 6:49 am

I have the same graphs for different hosts, some work, some don't.

What I think is wrong here is:

When checking the debug output in the log I can't find a:

12/02/2004 12:05:07 PM - POLLER: Poller[0] CACTI2RRD: /usr/local/bin/rrdtool update host_blabla_x.rrd --template blabla:blabla 1101985501:1144369152:97431552

for the ones that aren't working.

I wonder if it has anything to do with that sometimes when I create a graph for a host the rrd file isn't created, so what I do is that I create the file manually with datasource debug mode on and running the:

Code: Select all

/usr/local/bin/rrdtool create \
/home/admin/domains/mydomain/public_html/cacti-0.8.6b/rra/host_mssql_trans_188.rrd \
--step 300  \
DS:mssql_trans:GAUGE:600:0:1000 \
DS:mssql_trans_sec:GAUGE:600:0:1000 \
RRA:AVERAGE:0.5:1:600 \
RRA:AVERAGE:0.5:6:700 \
RRA:AVERAGE:0.5:24:775 \
RRA:AVERAGE:0.5:288:797 \
RRA:MAX:0.5:1:600 \
RRA:MAX:0.5:6:700 \
RRA:MAX:0.5:24:775 \
RRA:MAX:0.5:288:797 \
RRA:LAST:0.5:1:600 \
RRA:LAST:0.5:6:700 \
RRA:LAST:0.5:24:775 \
RRA:LAST:0.5:288:797 \

Is there perhaps some error that is occurring here...and some mysqldata that should be inserted that isn't? What table in the db could I check?

Another thing, don't know if it has anything to do with it:
I have:
12/02/2004 12:05:08 PM - CACTID: Poller[0] DEBUG: SQLCMD: insert into poller_output (local_data_id,rrd_name,time,output) values (188,'','2004-12-02 12:05:01','active_transactions:9 transactions_per_sec:12')

But my poller_output table is always empty when checking it.

Post by **TheWitness** » Thu Dec 02, 2004 6:57 am

Check the poller cache. That's where you should look. The poller output table is cleared after every polling cycle completes.

TheWitness

royce · Post by **royce** » Thu Jan 20, 2005 2:03 pm

I have a problem that seems to be like this one. I had a tablecrash because the server ran outta memory and mysql got killed. After fixing the corrupted tables, restarting mysql and hoping everything was working cacti stopped updating graphs, and when I created new ones the RRD-files never appeared. In debug-mode, mentioned above, i could copy the debug-output (rrdtool -- create ...) and manually create the rrd's with no errors at all. Also the graph-generating debug-output generated graphs (with no data in them).

Debug says nothing is broken (no errors/warnings).. Is it possible to ask cacti recreate the rrd's? Is there a tool to verify that the corrected database is still usable?

I'm using cacti 0.8.6c on a debian box with approx 10k graphs.

I use InnoDB tables instead of MyISAM but that AFAIK sould not affect cacti since it's internal-mysql design.

The machine ran out of mem at 148queries/second, and it took an hour to repair the tables. I've also tried to empty my poller-cache.. any hints?

Christofer

Post by **TheWitness** » Thu Jan 20, 2005 9:11 pm

Run poller.php from the command line in debug mode and you will get the RRD command output. You should also check cron to make sure that you only have 1 version. It's a long shot, but might as well. The command line output should realy tell you what is going on.

TheWitness

royce · Post by **royce** » Fri Jan 21, 2005 4:46 am

It seems that I'm missing the RRDCMD -lines in debug... strange.

royce · Post by **royce** » Fri Jan 21, 2005 6:53 am

Problem solved!

Code: Select all

mysql> select count(*) from poller_output; 
+----------+ 
| count(*) | 
+----------+ 
| 9785768 | 
+----------+ 
1 row in set (0.00 sec) 

mysql> TRUNCATE TABLE poller_output;
Query OK, 0 rows affected (0.41 sec)

... did the trick

Now - I've seen this on serveral threads lately... new bug?

Christofer

Post by **rony** » Fri Jan 21, 2005 8:15 am

Not sure, did you originally install using a RPM from the cacti site, or did you install it from the distributions set?

Troubleshooting cacti - Graphs not being generated

Who is online