Gaps in graphs

Post general support questions here that do not specifically fall into the Linux or Windows categories.

Moderators: Developers, Moderators

Post Reply
evilzardoz
Cacti User
Posts: 55
Joined: Sun Dec 04, 2005 10:59 pm

Gaps in graphs

Post by evilzardoz »

All,

I've got a newly installed 0.8.6g setup on an RHEL3 box and I've got gaps in all my graphs. This includes the localhost graphs. This has been apparent from the moment it was installed.

This seems to be recurring in the log:

12/05/2005 03:44:57 PM - POLLER: Poller[0] Maximum runtime of 296 seconds exceeded. Exiting.

I am currently using cacti to poll some Cisco routers - a pair of 6500s and a 3750.

Any suggestions?
nugai
Posts: 49
Joined: Thu Nov 03, 2005 10:48 pm

Post by nugai »

I'm relatively new to Cacti myself but here are a few ideas:

1. The default polling interval is 5 minutes (300 seconds). Therefore, the hole polling sequence must be completed in less time than that.

2. If you use the SNMP Interface provided by Cacti, it appears Cacti first evaluates the device for the devices available, builds an index of ports, then compares that index to the existing one, and then pulls the data off the device. While this reads long, it probably takes even more time to execute. Try to make sure that your polling information is easy to poll (e.g. by eliminating unnecessary overhead).

3. You can improve performance by massaging the "Maximum Concurrent Poller Processes" and "Maximum Threads per Process" in the "Poller" tab of your configuration settings. Obviously, this is limited by the amount of memory and CPU power.

4. In order to shorten execution time, Cacti developers have introduced cactid (note the "d" at the end) which replaces the PHP poller with a daemon. That will be a lot quicker but I understand it's still somewhat experimental.
evilzardoz
Cacti User
Posts: 55
Joined: Sun Dec 04, 2005 10:59 pm

Post by evilzardoz »

The box is a dual Xeon 3GHz/1MB cache with 2GB RAM and a few 10KRPM SCSI disks in RAID 5. There aren't many other tasks on the box.

I've set threads to 20 to see how that goes.
evilzardoz
Cacti User
Posts: 55
Joined: Sun Dec 04, 2005 10:59 pm

Post by evilzardoz »

Tried it and I still get the same issue.

No idea what's going on here. After adding a new device I also got:

12/05/2005 04:35:01 PM - CMDPHP: Poller[0] ERROR: Either there are no items in the cache or polling is disabled
12/05/2005 04:34:57 PM - POLLER: Poller[0] Maximum runtime of 296 seconds exceeded. Exiting.

Hmm. No idea what's going on here?
User avatar
TheWitness
Developer
Posts: 17047
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

Post by TheWitness »

There may be an issue with the launching of cmd.php processes. Please set concurrent processes to 1 and perform 1 polling interval in DEBUG. Post the Cacti Log.

Second, please run "php <path_cacti>/cmd.php" from the command line and post your Cacti Log.

The "poller disabled ..." message has me concerned.

TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages


For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
evilzardoz
Cacti User
Posts: 55
Joined: Sun Dec 04, 2005 10:59 pm

Post by evilzardoz »

Due to the size of the log data I have placed a copy here:

http://yoyo.its.monash.edu.au/~zardoz/cacti_log.txt

Hopefully there's something of consequence in there.

.Running PHP 4.3.2 by the way. Are there any known issues with this release that may cause this?
evilzardoz
Cacti User
Posts: 55
Joined: Sun Dec 04, 2005 10:59 pm

Post by evilzardoz »

More info:

12/06/2005 12:35:02 PM - SYSTEM STATS: Time:2.1084 Method:cmd.php Processes:2 Threads:N/A Hosts:5 HostsPerProcess:3 DataSources:110 RRDsProcessed:78
12/06/2005 12:34:57 PM - POLLER: Poller[0] Maximum runtime of 296 seconds exceeded. Exiting.
12/06/2005 12:25:03 PM - SYSTEM STATS: Time:2.1215 Method:cmd.php Processes:2 Threads:N/A Hosts:5 HostsPerProcess:3 DataSources:110 RRDsProcessed:59
12/06/2005 12:24:57 PM - POLLER: Poller[0] Maximum runtime of 296 seconds exceeded. Exiting.

This is some log bits when I've set the limit to 2 concurrent processes. Setting it to 30 results in the "no items in cache" errors.

It seems that something is hanging. Somehow. No idea what is causing this, but:

$ ps auxwww |grep cacti
nismgr 1905 0.0 0.0 2132 888 ? S 12:39 0:00 /bin/sh -c php /var/www/cacti/poller.php > /dev/null 2>&1
nismgr 1907 0.1 0.4 16716 8744 ? S 12:39 0:00 php /var/www/cacti/poller.php

These appear to be persistent unless I take cacti out of crontab in which case they vanish after 5 minutes.

It also appears taht one run was successful (this was with one process):
12/06/2005 11:04:57 AM - POLLER: Poller[0] Maximum runtime of 296 seconds exceeded. Exiting.
12/06/2005 10:58:02 AM - SYSTEM STATS: Time:2.0363 Method:cmd.php Processes:1 Threads:N/A Hosts:5 HostsPerProcess:5 DataSources:110 RRDsProcessed:5
12/06/2005 10:58:02 AM - SYSTEM STATS: Time:181.9440 Method:cmd.php Processes:1 Threads:N/A Hosts:5 HostsPerProcess:5 DataSources:110 RRDsProcessed:115
12/06/2005 10:54:57 AM - POLLER: Poller[0] Maximum runtime of 296 seconds exceeded. Exiting.

I'm guessing this is what is suggested by "system stats".

Clues?
evilzardoz
Cacti User
Posts: 55
Joined: Sun Dec 04, 2005 10:59 pm

Post by evilzardoz »

Is there no solution?
User avatar
gandalf
Developer
Posts: 22383
Joined: Thu Dec 02, 2004 2:46 am
Location: Muenster, Germany
Contact:

Post by gandalf »

Please search all your crontabs for a duplicate cron entry for the poller. e.g. rpm install will create a /etc/cron.d/cacti crontab.
Reinhard
Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests