Troubleshooting cacti - Graphs not being generated
Moderators: Developers, Moderators
-
- Posts: 35
- Joined: Wed Nov 10, 2004 5:43 am
- Location: Stockholm
- Contact:
Troubleshooting cacti - Graphs not being generated
Hello,
I've installed cacti and cactid and its running fine with snmp queries but when I used a apache server-status script I ran into problems.
The script is returning the correct values and it was actually working in cacti once, the values came into the graphs, but then I had to remove/add the graphs because the output values were in the wrong order and after that it didn't work again.
I've tried everything I can think of, changing permissions (chmod a+w) on the rra files, etc, but nothing works. The log file isn't returning any errors.
How can I debug cacti? How can isolate the problem?
Cacti is a great product...when it's working
But it was a bit hard to learn how everything is "connected" together, with input method, templates, datasources, etc.
I've installed cacti and cactid and its running fine with snmp queries but when I used a apache server-status script I ran into problems.
The script is returning the correct values and it was actually working in cacti once, the values came into the graphs, but then I had to remove/add the graphs because the output values were in the wrong order and after that it didn't work again.
I've tried everything I can think of, changing permissions (chmod a+w) on the rra files, etc, but nothing works. The log file isn't returning any errors.
How can I debug cacti? How can isolate the problem?
Cacti is a great product...when it's working
But it was a bit hard to learn how everything is "connected" together, with input method, templates, datasources, etc.
- TheWitness
- Developer
- Posts: 17007
- Joined: Tue May 14, 2002 5:08 pm
- Location: MI, USA
- Contact:
If you are running cactid as your poller, get the latest version of the source from CVS and rebuild. The new version, complete, and to be released soon, contains some bug fixes with scripts. Otherwise, cmd.php should be fine.
TheWitness
TheWitness
True understanding begins only when we realize how little we truly understand...
Life is an adventure, let yours begin with Cacti!
Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages
For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Life is an adventure, let yours begin with Cacti!
Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages
For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
-
- Posts: 35
- Joined: Wed Nov 10, 2004 5:43 am
- Location: Stockholm
- Contact:
- TheWitness
- Developer
- Posts: 17007
- Joined: Tue May 14, 2002 5:08 pm
- Location: MI, USA
- Contact:
Please note that you must, from yesterday moving forward NOT download the CVS version as it is no longer compatible with Cacti 0.8.6x. The latest release Cactid 0.8.6c will be the last release (I hope) for Cacti 0.8.6x. Cactid 0.8.6c was release on Friday/Saturday.
TheWitness
TheWitness
True understanding begins only when we realize how little we truly understand...
Life is an adventure, let yours begin with Cacti!
Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages
For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Life is an adventure, let yours begin with Cacti!
Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages
For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
-
- Posts: 35
- Joined: Wed Nov 10, 2004 5:43 am
- Location: Stockholm
- Contact:
Sorry, was in a hurry writing previous message, had to catch the bus
I installed Cactid 0.8.6c, did not install cacti so that's still 0.8.6b
The graphs were generated for maybe 30-45 minutes, after that, nothing, see below. With some other graphs I get NaN values.
Only guess I have is that it could be a permssions problem, when I ran the poller as root it looked like it were updating for a while, like 30 minutes, and after that, nothing. But the time on the graph is correct.
The poller is normally run as another user thru crontab.
I will do some more debugging tomorrow and see what I can come up with.
I installed Cactid 0.8.6c, did not install cacti so that's still 0.8.6b
The graphs were generated for maybe 30-45 minutes, after that, nothing, see below. With some other graphs I get NaN values.
Only guess I have is that it could be a permssions problem, when I ran the poller as root it looked like it were updating for a while, like 30 minutes, and after that, nothing. But the time on the graph is correct.
The poller is normally run as another user thru crontab.
I will do some more debugging tomorrow and see what I can come up with.
- Attachments
-
- graph1.JPG (43.32 KiB) Viewed 7540 times
- TheWitness
- Developer
- Posts: 17007
- Joined: Tue May 14, 2002 5:08 pm
- Location: MI, USA
- Contact:
Run with cmd.php for a while and see if it has any difference.
True understanding begins only when we realize how little we truly understand...
Life is an adventure, let yours begin with Cacti!
Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages
For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Life is an adventure, let yours begin with Cacti!
Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages
For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
-
- Posts: 35
- Joined: Wed Nov 10, 2004 5:43 am
- Location: Stockholm
- Contact:
This is what debugmode helped me with.
From the log:
CACTID: Poller[0] Host[1] SCRIPT: perl /home/admin/domains/default/public_html/cacti-0.8.6b/scripts/apache.pl baltazar, output: reqs_sec:10.8 reqs_cur:5
CACTI2RRD: /usr/local/bin/rrdtool update /home/admin/domains/default/public_html/cacti-0.8.6b/rra/baltazar_reqs_sec_88.rrd --template reqs_sec:reqs_cur 1100610981:10.8:5
looks like the mapping between the input method and the graph is incorrect because the values aren't being inserted...or something? I cant really say
And with another script/host I get results back all and good but there isn't even a "rrdtool update" row!!! Weird...
Every once in a while when I run the poller manually from cmdline, it paintes the graph, it's not like that all the time though, sometimes it's the same, nothing in the graph.
I'm thinking about re-installing cacti on another server...because I'm running out of ideas.
Using cmd.php instead of poller.php didn't help by the way...
From the log:
CACTID: Poller[0] Host[1] SCRIPT: perl /home/admin/domains/default/public_html/cacti-0.8.6b/scripts/apache.pl baltazar, output: reqs_sec:10.8 reqs_cur:5
CACTI2RRD: /usr/local/bin/rrdtool update /home/admin/domains/default/public_html/cacti-0.8.6b/rra/baltazar_reqs_sec_88.rrd --template reqs_sec:reqs_cur 1100610981:10.8:5
looks like the mapping between the input method and the graph is incorrect because the values aren't being inserted...or something? I cant really say
And with another script/host I get results back all and good but there isn't even a "rrdtool update" row!!! Weird...
Every once in a while when I run the poller manually from cmdline, it paintes the graph, it's not like that all the time though, sometimes it's the same, nothing in the graph.
I'm thinking about re-installing cacti on another server...because I'm running out of ideas.
Using cmd.php instead of poller.php didn't help by the way...
-
- Posts: 35
- Joined: Wed Nov 10, 2004 5:43 am
- Location: Stockholm
- Contact:
This is what I get in the log, looks correct right?
It runs the command and then inserts it in poller_ouput.
11/17/2004 03:35:12 PM - CACTID: Poller[0] Host[1] SCRIPT: perl /home/admin/domains/xxx/public_html/cacti-0.8.6b/scripts/apache.pl baltazar, output: reqs_sec:18.2 reqs_cur:7
11/17/2004 03:35:12 PM - CACTID: Poller[0] DEBUG: SQLCMD: insert into poller_output (local_data_id,rrd_name,time,output) values (88,'','2004-11-17 15:35:09','reqs_sec:18.2 reqs_cur:7')
So the error must be with the generation of the graphs I guess.
I got all other scripts to work except this one. This is the only multi-output script i'm using by the way, don't know if that's got anything to do with it.
By the way, what does this mean?:
ERROR: illegal attempt to update using time 1100707800 when last update time is 1100707800 (minimum one second step)
I just thought I should pipe the output from poller.php to another "logfile" because the /var/log/cacti.log didn't show any errors, and that's what I got in the poller.php >/var/log/cacti2.log , maybe 30-40 rows of that error message.
It runs the command and then inserts it in poller_ouput.
11/17/2004 03:35:12 PM - CACTID: Poller[0] Host[1] SCRIPT: perl /home/admin/domains/xxx/public_html/cacti-0.8.6b/scripts/apache.pl baltazar, output: reqs_sec:18.2 reqs_cur:7
11/17/2004 03:35:12 PM - CACTID: Poller[0] DEBUG: SQLCMD: insert into poller_output (local_data_id,rrd_name,time,output) values (88,'','2004-11-17 15:35:09','reqs_sec:18.2 reqs_cur:7')
So the error must be with the generation of the graphs I guess.
I got all other scripts to work except this one. This is the only multi-output script i'm using by the way, don't know if that's got anything to do with it.
By the way, what does this mean?:
ERROR: illegal attempt to update using time 1100707800 when last update time is 1100707800 (minimum one second step)
I just thought I should pipe the output from poller.php to another "logfile" because the /var/log/cacti.log didn't show any errors, and that's what I got in the poller.php >/var/log/cacti2.log , maybe 30-40 rows of that error message.
It appears as though your RRD file is either corrupt or you have not completely upgraded to 0.8.6b (with patches). Prior to 0.8.6b, we had a bug in multi-format RRD files where the update times did not work correctly. This iteself would not have hosed your RRD. However, at one point in time, your clock may have been out of wack.
Before you go there though, you need to post a DEBUG run from the poller.php and let me review.
Before you go there though, you need to post a DEBUG run from the poller.php and let me review.
-
- Posts: 35
- Joined: Wed Nov 10, 2004 5:43 am
- Location: Stockholm
- Contact:
- TheWitness
- Developer
- Posts: 17007
- Joined: Tue May 14, 2002 5:08 pm
- Location: MI, USA
- Contact:
Yes.
TheWitness
TheWitness
True understanding begins only when we realize how little we truly understand...
Life is an adventure, let yours begin with Cacti!
Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages
For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Life is an adventure, let yours begin with Cacti!
Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages
For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
-
- Posts: 35
- Joined: Wed Nov 10, 2004 5:43 am
- Location: Stockholm
- Contact:
So I need to apply the patches...sorry I didn't think I had to.
how do I patch cacti? can you point me to a how-to or something?
Anyway, here's my cacti.log and poller output.
Looks like something is going wrong here, check the cacti.log file.
The apache script doesn't return anything.
But if I run the same script thru commandline it works, both as root and as the cactiuser.
how do I patch cacti? can you point me to a how-to or something?
Anyway, here's my cacti.log and poller output.
Looks like something is going wrong here, check the cacti.log file.
The apache script doesn't return anything.
But if I run the same script thru commandline it works, both as root and as the cactiuser.
- Attachments
-
- cacti.log.txt
- cacti.log from the cronjob
- (2.42 KiB) Downloaded 232 times
-
- poller.log.txt
- poller.log from the cronjob
- (30.51 KiB) Downloaded 240 times
-
- Posts: 35
- Joined: Wed Nov 10, 2004 5:43 am
- Location: Stockholm
- Contact:
And the logfiles when running poller.php from commandline.
- Attachments
-
- cacti.log.2.txt
- cacti.log from the commandline
- (74.74 KiB) Downloaded 281 times
-
- poller.log.2.txt
- poller.log from the commandline
- (34.71 KiB) Downloaded 339 times
Who is online
Users browsing this forum: No registered users and 7 guests