Enabling Boost causes regular interval gaps in graphs

[Jeeves]

I've had issues with Boost since ye days of olde, so I had never enabled it.

After reading through some forum posts, it seemed like I was actually missing out on a useful feature, so I enabled "On-Demand RRD Updating" and forgot about it.

Since then, I've been fighting with regular gaps in my charts, for example:

eaton-boost-battery-issues.png (32.55 KiB) Viewed 1549 times

It seems like a useful feature, but obviously having regular gaps is not acceptable.

I looked for the boost log file, it didn't exist. Can anyone suggest a way to try and troubleshoot this?

[TheWitness]

Permissions?

[Jeeves]

Permissions?

Sorry, I'm not following. I'm not sure what you mean in this context?
What permissions need to be adjusted for boost?

[TheWitness]

Your web server has to be the same account as the poller and everything owned by it.

[TheWitness]

I periodically get poller failures for some devices, and to fix it, I increase the heartbeat to 1200 seconds for 300 second poller interval, and like magic the gaps disappear. You have to script update the RRDfiles though.

[Jeeves]

Your web server has to be the same account as the poller and everything owned by it.

It does. Everything is owned by apache. Which I believe is the same way cactid is setup as well.
That didn't change between enabling and disabling boost.

[TheWitness]

Code: Select all

cd /var/www/html/cacti
chown -R apache:apache *

[Jeeves]

I periodically get poller failures for some devices, and to fix it, I increase the heartbeat to 1200 seconds for 300 second poller interval, and like magic the gaps disappear. You have to script update the RRDfiles though.

This was all graphs on the system, not random graphs here and there. Every single graph was having periodic gaps, then I disabled Boost and it stopped happening.

[Jeeves]

Code: Select all

cd /var/www/html/cacti
chown -R apache:apache *

This is already how my configuration is

[TheWitness]

How often is boost running? I have 2.5M data sources, and I run boost every 2 hours.

[Jeeves]

How often is boost running? I have 2.5M data sources, and I run boost every 2 hours.

I believe this was set every 30 minutes, and I don't have remotely that many data sources.
Here's an example of what I was seeing:

sample-graphs.png (168.23 KiB) Viewed 1446 times

[TheWitness]

Is this script based or snmp based Data Source?

[Jeeves]

Is this script based or snmp based Data Source?

I had to go back and confirm. I don't believe I'm using any script queries at this point. These are all snmp based data sources.

I zoomed in a bit.
This is a smattering from different sites, different devices, different templates, different heartbeats, etc.

sample-graphs2.png (470.89 KiB) Viewed 1442 times

[Jeeves]

I started noticing there was a pattern to the issue, and started digging in. Logs indicated boost ran regularly so I turned it off and it's not happened once since.

It could be tangential to boost itself (e.g. boost caused too many database connections, boost caused too much IO, etc.) but boost caused the problem and turning it off fixed it.

Just wanted to zero in on it a bit since boost seemed useful.
If it's a problem not worth troubleshooting, that's fine too

[TheWitness]

Yea, connections are important, but it's normally spine that uses them up. The other things could be that someone is camping on the page and the page auto-refreshes just around boost time. If you have either App Armour or SELinux enabled, you may be missing an audit2allow rule to allow Apache to write to this directory too. That would do it for sure. If SELinux is enabled, put it into permissive mode, and reboot the box. Then check your log's for SELinux warnings and follow the procedure to make the RRA directory writable by Apache.