Poller overruns after 1.2.23 upgrade

Post general support questions here that do not specifically fall into the Linux or Windows categories.

Moderators: Developers, Moderators

KevinNoall
Cacti User
Posts: 99
Joined: Wed Feb 27, 2013 10:57 am
Location: Portland, OR
Contact:

Poller overruns after 1.2.23 upgrade

Post by KevinNoall »

I've done the 1.2.23 upgrade on a couple different systems, but on my main install my poller runtime went from ~12-20sec to 58+ and gets killed regularly.
Centos 7x, PHP 7.4.x, 10.5.15-MariaDB
I tried disabling the few plugins I had running to no avail (current thold, monitor, webseer)
I've seen some errors about mysql db going away and have seen mysql fail. Not exactly sure what would have changed to cause this trouble?
User avatar
TheWitness
Developer
Posts: 17047
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

Re: Poller overruns after 1.2.23 upgrade

Post by TheWitness »

Grab the latest spine. There were some issues with the 1.2.23 release, fixed now. Also, if upgrading from 1.1.x or 0.8.x to 1.2.23, the threading and processes moved to Data Collector > Edit.
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages


For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
KevinNoall
Cacti User
Posts: 99
Joined: Wed Feb 27, 2013 10:57 am
Location: Portland, OR
Contact:

Re: Poller overruns after 1.2.23 upgrade

Post by KevinNoall »

This seems to have resolved after grabbing the latest spine. Thanks
KevinNoall
Cacti User
Posts: 99
Joined: Wed Feb 27, 2013 10:57 am
Location: Portland, OR
Contact:

Re: Poller overruns after 1.2.23/1.2.24 upgrade

Post by KevinNoall »

After upgrading to the new cacti 1.2.24 this issue returned. I tried putting spine back to v1.2.22 but that did not resolve. all remote pollers and the main poller are all overrunning the polling time and collecting no statistics.
Nothing alarming in the logs that I can see
KevinNoall
Cacti User
Posts: 99
Joined: Wed Feb 27, 2013 10:57 am
Location: Portland, OR
Contact:

Re: Poller overruns after 1.2.23 upgrade

Post by KevinNoall »

Now I'm seeing boost errors in the log:
BOOST ERROR: Failed to retrieve any rows from archive tables
and getting emails:
WARNING: Detected Poller Boost Overrun, Possible Boost Poller Crash
User avatar
TheWitness
Developer
Posts: 17047
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

Re: Poller overruns after 1.2.23 upgrade

Post by TheWitness »

Upgrade to 1.2.24 ASAP.
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages


For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
KevinNoall
Cacti User
Posts: 99
Joined: Wed Feb 27, 2013 10:57 am
Location: Portland, OR
Contact:

Re: Poller overruns after 1.2.23 upgrade

Post by KevinNoall »

Apologies for re-using the post. this is AFTER I upgraded everything to 1.2.24
User avatar
TheWitness
Developer
Posts: 17047
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

Re: Poller overruns after 1.2.23 upgrade

Post by TheWitness »

Is it happening often? Make sure there is at least an hour between boost updates.
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages


For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
KevinNoall
Cacti User
Posts: 99
Joined: Wed Feb 27, 2013 10:57 am
Location: Portland, OR
Contact:

Re: Poller overruns after 1.2.23 upgrade

Post by KevinNoall »

I'm getting the warning emails every minute.

The setting was 30m, I've changed it to 1h.

all pollers dropped back to spine 1.2.22 with no improvement.
User avatar
TheWitness
Developer
Posts: 17047
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

Re: Poller overruns after 1.2.23 upgrade

Post by TheWitness »

How big is this install?
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages


For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
KevinNoall
Cacti User
Posts: 99
Joined: Wed Feb 27, 2013 10:57 am
Location: Portland, OR
Contact:

Re: Poller overruns after 1.2.23 upgrade

Post by KevinNoall »

15 remote pollers, but not all that many devices overall about 300
KevinNoall
Cacti User
Posts: 99
Joined: Wed Feb 27, 2013 10:57 am
Location: Portland, OR
Contact:

Re: Poller overruns after 1.2.23 upgrade

Post by KevinNoall »

Even with remote pollers all disabled, the main poller time out and it only has ~20 devices
User avatar
TheWitness
Developer
Posts: 17047
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

Re: Poller overruns after 1.2.23 upgrade

Post by TheWitness »

How about the server? CPU cores/threads? How much memory?

How many collector processes and threads in total?
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages


For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
KevinNoall
Cacti User
Posts: 99
Joined: Wed Feb 27, 2013 10:57 am
Location: Portland, OR
Contact:

Re: Poller overruns after 1.2.23 upgrade

Post by KevinNoall »

2gb RAM, 3 processors. remote pollers the same

3 processes 20 threads per process for each poller

avg polling times were in the neighborhood of ~30seconds for all the pollers prior to 1.2.24 upgrade.
User avatar
TheWitness
Developer
Posts: 17047
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

Re: Poller overruns after 1.2.23 upgrade

Post by TheWitness »

With what you're saying you really need considerably more threads on the core or the main server are they VMS if they are go to 16 cores and well more memory for sure go to like 8 gig memory at a minimum.
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages


For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest