Boost plugin silent DB timeouts

[Instigater]

I have run across one nasty Boost plugin issue.

The setup is following. 2 virtual servers running on vmware.

app server:
2vCPU, 2GB RAM
db server:
2vCPU, 4GB RAM

MySQL is configured to support 2GB heap size and roughly 6 million entries. Boost is scheduled to run every 6 hours with maximum of 2 million records. Spine poller every 1 minute and direct boost table population.

Right now I have 10k data sources. Most of RRDs are quite large (16 DS).

I have also configured to run file and DB backups 3 times a day (HP Omniback).

Problem is that this backup process somehow locks up RRD file updates and MySQL table starts growing if boost file updates takes place at the same time. Automatic boost process just tries every minute to update RRDs and clear MySQL table. Very high MySQL CPU usage, high network traffic. I tried to run boost from commandline. It somehow managed to update smaller RRDs but fails with larger ones sooner or later without any notice in debug mode and drops out to bash!

Then I manualy set "boost_rrd_update_max_records_per_select" value to "1" and tried again. Boost completed without any errors. This might be an issue for other virtualized CACTI environments.

Please, make a possibility to set smaller "Maximum data sources per pass" from Boost plugin Settings page.

What is the default timeout for data fetch from MySQL?

[TheWitness]

I would say thats a problem with the backup solution. If you have support from the vendor, i would ask them how to disable locks and exclude certain tables. If they dont support those things, you bought the wrong product and could do better from the command line.

TheWitness

[TheWitness]

I can also reduce that number.

TheWitness

[Instigater]

It would be nice to make that input as standard not dropdown. Than everybody could scale as needed.

Also, problem persists even when backups are over. I also restarted app server without luck. The problem still persisted. Only setting "Maximum Data Sources Per Pass" to value "1" changed everything and boost could work out 12h backlog in just 25 minutes.

So, I guess that this is not a backup problem but a performance issue.

I can repeat it if needed.