BOOST Error - Cacti 0.8.7g (beta2)

[TheWitness]

Keep me posted. Just sitting here now waiting for your response I'm very concerned about spine b2.

TheWitness

[mcutting]

Ok. It looks as though SPINE B2 does not work too well at all. If I disable BOOST, it still only processes 205 out of a possible 2000+ RRD files. Nothing in the logs either to indicate why this is - only a mention of SPINE to being multithreaded ?

[TheWitness]

Do you have Console -> Settings -> Poller -> Spine Direct enabled? You will only see it with Boost enabled. However, spine does not know about "Boost On/Off". It only knows that variable.

So, enable boost, and uncheck it and try again. I have to fix boost to unset this variable, or spine to detect if Boost is enabled moving forward.

Still sitting here drinking Tea (uhm Coffee)

TheWitness

[TheWitness]

Sorry, the settings in the Boost sub-tab.

TheWitness

[mcutting]

Just tried this. Polling times have literally halved, which to me means it didn't complete correctly.

Another run with Boost disabled verified this - 205 RRD's updated

[mcutting]

This morning, I now see

• 06/15/2010 05:30:20 AM - SPINE: Poller[0] ERROR: SQL Failed! Error:'1114', Message:'The table 'poller_output_boost' is full', SQL Fragment:'INSERT INTO poller_output_boost (local_data_id, rrd_name, time, output) VALUES (700,'hdd_used','2010-06-15 05:30:08','7846256640'),(700,'hdd_total','2010-06-15 05:30:08','12889010176'),(701,'hdd_used','2010-06-15 05:30:08','13575172096'),(701,'hdd_total','2010-06-15 05:30:08','279346958336'),(702,'hdd_used','2010-06-15 05:30:08','560267264'),(702,'hdd_total','2010-06-15 05:30:08','6259277824'),(703,'hdd_used','2010-06-15 05:30:08','880869376'),(703,'hdd_total','2010-06-15 05:30:08','4289331200'),(704,'cpu','2010-06-15 05:30:08','0'),(705,'cpu','2010-06-15 05:30:08','0'),(706,'cpu','2010-06-15 05:30:08','0'),(707,'cpu','2010-06-15 05:30:08','0'),(708,'cpu','2010-06-15 05:30:08','0'),(709,'cpu','2010-06-15 05:30:08','0'),(710,'cpu','2010-06-15 05:30:08','0'),(711,'cpu','2010-06-15 05:30:08','1'),(712,'traffic_in','2010-06-15 05:30:08','3538679899'),(712,'traffic_out','2010-06-15 05:30:08','3701774158')

Any ideas as to what is causing this ?

Thanks

[mcutting]

Hmm - nothing happens if I try to force it from the command line either

php poller_boost.php --force

[TheWitness]

Did you do as I told you and disable direct poller population from boost?

TheWitness

[mcutting]

Yes. Also tried the php poller_boost.php --force with no luck

[TheWitness]

select * from settings where name like '%boost%';

post that

[mcutting]

select * from settings where name like '%boost%';

post that

Umm - seems to be working now ? Graphs are populating again

However, I still see

Boost Storage Statistics
Database Engine: MEMORY
Current Boost Table Size: 89 MBytes
Avg Bytes/Record: 545 Bytes
Max Record Length: 210 Bytes
Max Allowed Boost Table Size: 88 MBytes
Estimated Maximum Records: 170212 Records

Running php poller_boost.php --force doesn't seem to empty the table either.

• boost_png_cache_directory /usr/share/cacti/site/plugins/boost/cache
  boost_mysql_string_length 64000
  boost_rrd_update_string_length 2000
  boost_poller_mem_limit 1024
  boost_rrd_update_max_runtime 1200
  boost_redirect
  boost_server_enable
  boost_server_effective_user root
  boost_server_multiprocess 1
  boost_path_rrdupdate
  boost_server_hostname localhost
  boost_server_listen_port 9050
  boost_png_cache_enable on
  boost_server_clients 127.0.0.1
  boost_server_timeout 2
  boost_rrd_update_max_records_per_select 2000
  boost_rrd_update_interval 60
  boost_rrd_update_max_records 1000000
  boost_rrd_update_enable on
  boost_rrd_update_system_enable on
  boost_max_output_length 1276584366:210
  boost_last_run_time 2010-06-15 7:47:02
  boost_next_run_time 2010-06-15 8:45:56
  path_boost_log
  boost_rrd_update_lockfile /tmp/cacti-boost.lockf

[mcutting]

Now seeing

• 06/15/2010 07:50:34 AM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1114', SQL:"INSERT INTO poller_output_boost (local_data_id, rrd_name, time, output) VALUES ('1815','','2010-06-15 07:50:15','DNSQueriesPersec:362 LocalQueueLength:0 LocalRetryQueueLength:0 MessagesDeliveredPersec:148 MessagesReceivedPersec:358 MessagesSentPersec:704 Name:_Total RemoteQueueLength:0 RemoteRetryQueueLength:0'), ('1813','hdd_total','2010-06-15 07:50:15','292309966848'), ('1813','hdd_used','2010-06-15 07:50:15','27808612352'), ('1812','traffic_out','2010-06-15 07:50:15','739290272'), ('1812','traffic_in','2010-06-15 07:50:15','596868791'), ('1811','cpu','2010-06-15 07:50:15','0'), ('1810','cpu','2010-06-15 07:50:15','0'), ('1809','cpu','2010-06-15 07:50:15','0'), ('1808','cpu','2010-06-15 07:50:15','0'), ('1807','cpu','2010-06-15 07:50:15','0'), ('1806','cpu','2010-06-15 07:50:15','0'), ('1805','cpu','2010-06-15 07:50:15','0'), ('1804','cpu','2010-06-15 07:50:15','7'), ('1803','cpu','2010-06-15 07:50:15','0'), ('1801','cpu','2010-06-15 07:50:15','0'), ('1802','cpu','2010-06-15 07:50:15','0'), ('1799','cpu','2010-06-15 07:50:15','0'), ('1800','cpu','2010-06-15 07:50:15','0'), ('1798','cpu','2010-06-15 07:50:15','0'), ('1797','cpu','2010-06-15 07:50:15','0'), ('1796','cpu','2010-06-15 07:50:15','0'), ('1795','hdd_total','2010-06-15 07:50:15','4284350464'), ('1795','hdd_used','2010-06-15 07:50:15','1198522368'), ('1794','hdd_total','2010-06-15 07:50:15','6254297088'), ('1794','hdd_used','2010-06-15 07:50:15','1092747264'), ('1793','hdd_total','2010-06-15 07:50:15','72736118272'), ('1793','hdd_used','2010-06-15 07:50:15','6606717440') ON DUPLICATE KEY UPDATE output=VALUES(output)'

[mcutting]

Ok. Just truncated the table, and now it's populating again (data loss isn't a real problem).

However, BOOST does not appear to be updating the cache (it's enabled). Running poller_boost.php --force doesn't seem to do anything.

[mcutting]

*another* update (sorry about this)..

Looks like it is working, as the graphs are populating correctly.

However, looks like something *might* be busted

Current Boost Status
Boost On Demand Updating: Never Run
Total Data Sources: 1623
Total Boost Records: 7840

Despite forcing this from the command line

[mcutting]

Running php poller_boost.php --force --debug spits out

06/15/2010 09:28:09 AM - CMDPHP: Poller[0] DEBUG: Found lock, so another boost process is running

No sign of BOOST running by itself:


Current Boost Status
Boost On Demand Updating: Never Run
Total Data Sources: 1623
Total Boost Records: 28500
Boost Storage Statistics
Database Engine: MEMORY
Current Boost Table Size: 15 MBytes
Avg Bytes/Record: 545 Bytes
Max Record Length: 210 Bytes
Max Allowed Boost Table Size: 88 MBytes
Estimated Maximum Records: 170212 Records