poller_output table problem?

Post support questions that directly relate to Linux/Unix operating systems.

Moderators: Developers, Moderators

Post Reply
User avatar
MagnaChef
Cacti User
Posts: 164
Joined: Tue Nov 25, 2003 3:58 pm
Location: Buffalo, NY

Post by MagnaChef »

For some reason the amount of records in my poller_output table seems to grow and shrink. Is this normal?
Top
User avatar
TheWitness
Developer
Posts: 17007
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:
Contact TheWitness

Post by TheWitness »

RRDtool must be crashing. Follow the procedure below:

1) Disable Cron
2) Open a Shell and Verify Count in poller_output table is something managable
3) Run poller.php from the command line.
4) Observe the output. Do you get the "Ok's"
5) While running, open another window and run top. Do you see rrdtool running?

Post your results.

TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages

For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Top
User avatar
MagnaChef
Cacti User
Posts: 164
Joined: Tue Nov 25, 2003 3:58 pm
Location: Buffalo, NY

Post by MagnaChef »

Will do. I'll run it now. Thanks!
Top
User avatar
MagnaChef
Cacti User
Posts: 164
Joined: Tue Nov 25, 2003 3:58 pm
Location: Buffalo, NY

Post by MagnaChef »

poller_output is empty.

Code: Select all

mysql> select count(*) from poller_output;
+----------+
| count(*) |
+----------+
|        0 |
+----------+
1 row in set (0.00 sec)
Thoughts?
Top
User avatar
MagnaChef
Cacti User
Posts: 164
Joined: Tue Nov 25, 2003 3:58 pm
Location: Buffalo, NY

Post by MagnaChef »

TheWitness wrote: 1) Disable Cron
Done.
TheWitness wrote: 2) Open a Shell and Verify Count in poller_output table is something managable
The value is zero (See Above Post)
TheWitness wrote: 3) Run poller.php from the command line.
Done.
TheWitness wrote: 4) Observe the output. Do you get the "Ok's"
I get them in the very beginning. I'm rerunning and piping it to a file so that I can show you the output.
TheWitness wrote: 5) While running, open another window and run top. Do you see rrdtool running?
I did not see rrdtool show up.

...I don't appear to be doing too well here :wink:

Thoughts?
Top
User avatar
TheWitness
Developer
Posts: 17007
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:
Contact TheWitness

Post by TheWitness »

Post your run results. Please follow my instructions.

TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages

For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Top
User avatar
MagnaChef
Cacti User
Posts: 164
Joined: Tue Nov 25, 2003 3:58 pm
Location: Buffalo, NY

Post by MagnaChef »

Do you want my entire debug file, or just the OK messages?

Here's a sample (kind of long...sorry)

Code: Select all

03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_cpu_282.rrd --template cpu 1142481302:1
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/localhost_proc_5271.rrd --template proc 1142481302:116
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_test_fs_reads_121.rrd --template fs_reads 1142481302:648397
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_cpu_283.rrd --template cpu 1142481302:0
OK u:0.00 s:0.02 r:1.27
OK u:0.00 s:0.02 r:1.27
OK u:0.00 s:0.02 r:1.28
OK u:0.00 s:0.02 r:1.28
OK u:0.00 s:0.02 r:1.28
OK u:0.00 s:0.02 r:1.28
OK u:0.00 s:0.02 r:1.28
OK u:0.00 s:0.02 r:1.28
OK u:0.00 s:0.02 r:1.28
OK u:0.00 s:0.02 r:1.28
OK u:0.00 s:0.02 r:1.28
OK u:0.00 s:0.02 r:1.28
OK u:0.00 s:0.02 r:1.28
OK u:0.00 s:0.02 r:1.28
OK u:0.00 s:0.02 r:1.28
OK u:0.00 s:0.02 r:1.28
OK u:0.00 s:0.02 r:1.28
OK u:0.00 s:0.02 r:1.28
OK u:0.00 s:0.02 r:1.28
OK u:0.00 s:0.02 r:1.28
OK u:0.00 s:0.02 r:1.28
OK u:0.00 s:0.02 r:1.28
OK u:0.00 s:0.02 r:1.28
OK u:0.00 s:0.02 r:1.28
OK u:0.00 s:0.02 r:1.28
OK u:0.00 s:0.02 r:1.28
OK u:0.00 s:0.02 r:1.28
OK u:0.00 s:0.02 r:1.28
OK u:0.00 s:0.02 r:1.28
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_fs_reads_275.rrd --template fs_reads 1142481302:104459934
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_fs_writes_276.rrd --template fs_writes 1142481302:40675071
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_open_files_280.rrd --template open_files 1142481302:84
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_cache_checks_330.rrd --template cache_checks 1142481302:32705733
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_cache_hits_331.rrd --template cache_hits 1142481302:31128670
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_cpu_335.rrd --template cpu 1142481302:1
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_cpu_336.rrd --template cpu 1142481302:0
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_fs_reads_328.rrd --template fs_reads 1142481302:206271335
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_fs_writes_329.rrd --template fs_writes 1142481302:65845513
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_open_files_333.rrd --template open_files 1142481302:16
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_vol_total_6829.rrd --template vol_free:vol_freeable:vol_total 1142481302:2820992:33088:4187840
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_vol_total_6830.rrd --template vol_free:vol_freeable:vol_total 1142481302:4096:0:4096
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/mortgage_test_fs_writes_122.rrd --template fs_writes 1142481302:615245
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_vol_total_6831.rrd --template vol_free:vol_freeable:vol_total 1142481302:221472:0:2063936
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_total_logins_266.rrd --template total_logins 1142481302:0
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_traffic_in_273.rrd --template traffic_in:traffic_out 1142481302:43375442:0
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_traffic_in_274.rrd --template traffic_in:traffic_out 1142481302:245970151:3075291986
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_cache_checks_486.rrd --template cache_checks 1142481302:132803449
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_vol_total_6825.rrd --template vol_free:vol_freeable:vol_total 1142481302:3040000:33792:4187840
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_cache_hits_487.rrd --template cache_hits 1142481302:127465266
OK u:0.00 s:0.02 r:1.30
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_cpu_494.rrd --template cpu 1142481302:2
OK u:0.00 s:0.02 r:1.30
OK u:0.00 s:0.02 r:1.30
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_cpu_495.rrd --template cpu 1142481302:0
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_fs_reads_484.rrd --template fs_reads 1142481302:452440872
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_fs_writes_485.rrd --template fs_writes 1142481302:329577680
OK u:0.01 s:0.02 r:1.30
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/mortgage_test_open_files_126.rrd --template open_files 1142481302:98
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_vol_total_6826.rrd --template vol_free:vol_freeable:vol_total 1142481302:4096:0:4096
OK u:0.01 s:0.02 r:1.31
OK u:0.01 s:0.03 r:1.31
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_open_files_489.rrd --template open_files 1142481302:216
OK u:0.01 s:0.03 r:1.31
OK u:0.01 s:0.03 r:1.31
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_vol_total_6827.rrd --template vol_free:vol_freeable:vol_total 1142481302:1147840:0:2063936
OK u:0.01 s:0.03 r:1.31
OK u:0.01 s:0.03 r:1.31
OK u:0.01 s:0.03 r:1.31
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/mortgage_test_total_logins_125.rrd --template total_logins 1142481302:2
OK u:0.01 s:0.03 r:1.31
OK u:0.01 s:0.03 r:1.31
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_cache_checks_501.rrd --template cache_checks 1142481302:1809051384
OK u:0.01 s:0.03 r:1.31
OK u:0.01 s:0.03 r:1.31
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_cache_hits_502.rrd --template cache_hits 1142481302:1802546201
OK u:0.01 s:0.03 r:1.31
OK u:0.01 s:0.03 r:1.31
OK u:0.01 s:0.03 r:1.31
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_cpu_510.rrd --template cpu 1142481302:1
OK u:0.01 s:0.03 r:1.31
OK u:0.01 s:0.03 r:1.31
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_cpu_511.rrd --template cpu 1142481302:0
OK u:0.01 s:0.03 r:1.31
OK u:0.01 s:0.03 r:1.31
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_fs_reads_499.rrd --template fs_reads 1142481302:3757376460
OK u:0.01 s:0.03 r:1.31
OK u:0.01 s:0.03 r:1.31
OK u:0.01 s:0.03 r:1.31
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_fs_writes_500.rrd --template fs_writes 1142481302:3315203223
OK u:0.01 s:0.03 r:1.31
OK u:0.01 s:0.03 r:1.31
OK u:0.01 s:0.03 r:1.31
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_open_files_504.rrd --template open_files 1142481302:381
OK u:0.01 s:0.03 r:1.31
OK u:0.01 s:0.03 r:1.31
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_total_logins_332.rrd --template total_logins 1142481302:0
OK u:0.01 s:0.03 r:1.31
OK u:0.01 s:0.03 r:1.31
03/15/2006 10:55:03 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti-0.8.6e/rra/servername_traffic_in_338.rrd --template traffic_in:traffic_out 1142481302:1336069411:0
OK u:0.01 s:0.03 r:1.31
OK u:0.01 s:0.03 r:1.31
OK u:0.01 s:0.03 r:1.31
OK u:0.01 s:0.03 r:1.31
OK u:0.01 s:0.03 r:1.31
OK u:0.01 s:0.03 r:1.31
OK u:0.01 s:0.03 r:1.31
OK u:0.01 s:0.03 r:1.31
OK u:0.01 s:0.03 r:1.31
OK u:0.01 s:0.03 r:1.32
Waiting on 1/1 pollers.
Top
User avatar
TheWitness
Developer
Posts: 17007
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:
Contact TheWitness

Post by TheWitness »

This looks good. How many are you missing?

Larry
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages

For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Top
User avatar
TheWitness
Developer
Posts: 17007
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:
Contact TheWitness

Post by TheWitness »

What about STDERR? Are you piping it to a file as well?

TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages

For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Top
User avatar
MagnaChef
Cacti User
Posts: 164
Joined: Tue Nov 25, 2003 3:58 pm
Location: Buffalo, NY

Post by MagnaChef »

TheWitness wrote:What about STDERR? Are you piping it to a file as well?

TheWitness
No. I will rerun it now.

Should I email it to you?
Top
User avatar
TheWitness
Developer
Posts: 17007
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:
Contact TheWitness

Post by TheWitness »

Does not mater. I am online trying to reach you to no avail.... I am switching computers now..

Larry
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages

For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Top
User avatar
MagnaChef
Cacti User
Posts: 164
Joined: Tue Nov 25, 2003 3:58 pm
Location: Buffalo, NY

Post by MagnaChef »

As you may of already seen I emailed both files over. Let me know if you need a dump of my cacti database, or anything else.
Top
User avatar
MagnaChef
Cacti User
Posts: 164
Joined: Tue Nov 25, 2003 3:58 pm
Location: Buffalo, NY

Post by MagnaChef »

UPDATE - I've created a meta-glue band-aid

I have created a temporary "let's just get our production cacti server running" band-aid script that will allow me to read the cacti.log file in (practically) real-time, and just run the RRDtool update commands (logging is set to DEBUG). I'm just running this via command line, and my graphs finally have some data!

It's a temporary hack that will at least get cacti working - even though it's not :wink: It is written in Perl and uses File::Tail (http://search.cpan.org/~mgrabnar/File-T ... .3/Tail.pm)

Code: Select all

#!/usr/bin/perl
use File::Tail;

$file=File::Tail->new("$ARGV[0]");
while (defined($line=$file->read))
{
        print "$line\n" if $line =~ /Time\:/;
        next if $line !~ /CACTI2RRD/;
        if ($line =~ /(\/usr\/bin\/rrdtool\s+update\s+.+)$/)
        {
                `$1`;
        }
}
Any thoughts on why Cacti isn't performing this step?
Top
User avatar
TheWitness
Developer
Posts: 17007
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:
Contact TheWitness

Post by TheWitness »

RRDtool is crashing likely due to a messed up update command. The way we have addressed this in the past is to force the RRDtool to not use pipes. There are less than 10 lines of code that need to be altered to accomodate this.

I was prepared to work with you on this last night. I think that if you look at the RRDexecute function, you should be able to ignore the "pipes" based updates and comment that out so that it just executes the exec of RRDtool every time until you find the problematic template/data source.

TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages

For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Top
User avatar
TheWitness
Developer
Posts: 17007
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:
Contact TheWitness

Post by TheWitness »

Magna,

If that patch I sent works for you, please post. Also, please post results of backing off of RRDtool 1.0.50. I believe that 1.0.48 was known good.

Larry
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages

For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Top
Post Reply

Return to “Help: Linux/Unix Specific”

Who is online

Users browsing this forum: No registered users and 1 guest