Poller overruns after 1.2.23 upgrade
Moderators: Developers, Moderators
-
- Cacti User
- Posts: 99
- Joined: Wed Feb 27, 2013 10:57 am
- Location: Portland, OR
- Contact:
Poller overruns after 1.2.23 upgrade
I've done the 1.2.23 upgrade on a couple different systems, but on my main install my poller runtime went from ~12-20sec to 58+ and gets killed regularly.
Centos 7x, PHP 7.4.x, 10.5.15-MariaDB
I tried disabling the few plugins I had running to no avail (current thold, monitor, webseer)
I've seen some errors about mysql db going away and have seen mysql fail. Not exactly sure what would have changed to cause this trouble?
Centos 7x, PHP 7.4.x, 10.5.15-MariaDB
I tried disabling the few plugins I had running to no avail (current thold, monitor, webseer)
I've seen some errors about mysql db going away and have seen mysql fail. Not exactly sure what would have changed to cause this trouble?
- TheWitness
- Developer
- Posts: 17047
- Joined: Tue May 14, 2002 5:08 pm
- Location: MI, USA
- Contact:
Re: Poller overruns after 1.2.23 upgrade
Grab the latest spine. There were some issues with the 1.2.23 release, fixed now. Also, if upgrading from 1.1.x or 0.8.x to 1.2.23, the threading and processes moved to Data Collector > Edit.
True understanding begins only when we realize how little we truly understand...
Life is an adventure, let yours begin with Cacti!
Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages
For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Life is an adventure, let yours begin with Cacti!
Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages
For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
-
- Cacti User
- Posts: 99
- Joined: Wed Feb 27, 2013 10:57 am
- Location: Portland, OR
- Contact:
Re: Poller overruns after 1.2.23 upgrade
This seems to have resolved after grabbing the latest spine. Thanks
-
- Cacti User
- Posts: 99
- Joined: Wed Feb 27, 2013 10:57 am
- Location: Portland, OR
- Contact:
Re: Poller overruns after 1.2.23/1.2.24 upgrade
After upgrading to the new cacti 1.2.24 this issue returned. I tried putting spine back to v1.2.22 but that did not resolve. all remote pollers and the main poller are all overrunning the polling time and collecting no statistics.
Nothing alarming in the logs that I can see
Nothing alarming in the logs that I can see
-
- Cacti User
- Posts: 99
- Joined: Wed Feb 27, 2013 10:57 am
- Location: Portland, OR
- Contact:
Re: Poller overruns after 1.2.23 upgrade
Now I'm seeing boost errors in the log:
BOOST ERROR: Failed to retrieve any rows from archive tables
and getting emails:
WARNING: Detected Poller Boost Overrun, Possible Boost Poller Crash
BOOST ERROR: Failed to retrieve any rows from archive tables
and getting emails:
WARNING: Detected Poller Boost Overrun, Possible Boost Poller Crash
- TheWitness
- Developer
- Posts: 17047
- Joined: Tue May 14, 2002 5:08 pm
- Location: MI, USA
- Contact:
Re: Poller overruns after 1.2.23 upgrade
Upgrade to 1.2.24 ASAP.
True understanding begins only when we realize how little we truly understand...
Life is an adventure, let yours begin with Cacti!
Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages
For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Life is an adventure, let yours begin with Cacti!
Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages
For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
-
- Cacti User
- Posts: 99
- Joined: Wed Feb 27, 2013 10:57 am
- Location: Portland, OR
- Contact:
Re: Poller overruns after 1.2.23 upgrade
Apologies for re-using the post. this is AFTER I upgraded everything to 1.2.24
- TheWitness
- Developer
- Posts: 17047
- Joined: Tue May 14, 2002 5:08 pm
- Location: MI, USA
- Contact:
Re: Poller overruns after 1.2.23 upgrade
Is it happening often? Make sure there is at least an hour between boost updates.
True understanding begins only when we realize how little we truly understand...
Life is an adventure, let yours begin with Cacti!
Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages
For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Life is an adventure, let yours begin with Cacti!
Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages
For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
-
- Cacti User
- Posts: 99
- Joined: Wed Feb 27, 2013 10:57 am
- Location: Portland, OR
- Contact:
Re: Poller overruns after 1.2.23 upgrade
I'm getting the warning emails every minute.
The setting was 30m, I've changed it to 1h.
all pollers dropped back to spine 1.2.22 with no improvement.
The setting was 30m, I've changed it to 1h.
all pollers dropped back to spine 1.2.22 with no improvement.
- TheWitness
- Developer
- Posts: 17047
- Joined: Tue May 14, 2002 5:08 pm
- Location: MI, USA
- Contact:
Re: Poller overruns after 1.2.23 upgrade
How big is this install?
True understanding begins only when we realize how little we truly understand...
Life is an adventure, let yours begin with Cacti!
Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages
For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Life is an adventure, let yours begin with Cacti!
Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages
For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
-
- Cacti User
- Posts: 99
- Joined: Wed Feb 27, 2013 10:57 am
- Location: Portland, OR
- Contact:
Re: Poller overruns after 1.2.23 upgrade
15 remote pollers, but not all that many devices overall about 300
-
- Cacti User
- Posts: 99
- Joined: Wed Feb 27, 2013 10:57 am
- Location: Portland, OR
- Contact:
Re: Poller overruns after 1.2.23 upgrade
Even with remote pollers all disabled, the main poller time out and it only has ~20 devices
- TheWitness
- Developer
- Posts: 17047
- Joined: Tue May 14, 2002 5:08 pm
- Location: MI, USA
- Contact:
Re: Poller overruns after 1.2.23 upgrade
How about the server? CPU cores/threads? How much memory?
How many collector processes and threads in total?
How many collector processes and threads in total?
True understanding begins only when we realize how little we truly understand...
Life is an adventure, let yours begin with Cacti!
Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages
For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Life is an adventure, let yours begin with Cacti!
Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages
For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
-
- Cacti User
- Posts: 99
- Joined: Wed Feb 27, 2013 10:57 am
- Location: Portland, OR
- Contact:
Re: Poller overruns after 1.2.23 upgrade
2gb RAM, 3 processors. remote pollers the same
3 processes 20 threads per process for each poller
avg polling times were in the neighborhood of ~30seconds for all the pollers prior to 1.2.24 upgrade.
3 processes 20 threads per process for each poller
avg polling times were in the neighborhood of ~30seconds for all the pollers prior to 1.2.24 upgrade.
- TheWitness
- Developer
- Posts: 17047
- Joined: Tue May 14, 2002 5:08 pm
- Location: MI, USA
- Contact:
Re: Poller overruns after 1.2.23 upgrade
With what you're saying you really need considerably more threads on the core or the main server are they VMS if they are go to 16 cores and well more memory for sure go to like 8 gig memory at a minimum.
True understanding begins only when we realize how little we truly understand...
Life is an adventure, let yours begin with Cacti!
Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages
For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Life is an adventure, let yours begin with Cacti!
Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages
For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Who is online
Users browsing this forum: No registered users and 1 guest