I think I managed to fix it myself, I've just copied and pasted my blog post here:
-------8<---------------
I've been having problems with some of my Cacti graphs having intermittent large gaps in them (days)
. Looking at the Cacti logs I can see stale data errors:
Code: Select all
03/04/2012 11:36:54 AM - BOOST: Poller[0] WARNING: Stale Poller Data Found! Item Time:'1330511882', RRD Time:'1330581481' Ignoring Value!
03/04/2012 11:36:54 AM - BOOST: Poller[0] WARNING: Stale Poller Data Found! Item Time:'1330511882', RRD Time:'1330581481' Ignoring Value!
03/04/2012 11:36:54 AM - BOOST: Poller[0] WARNING: Stale Poller Data Found! Item Time:'1330512182', RRD Time:'1330581481' Ignoring Value!
03/04/2012 11:36:54 AM - BOOST: Poller[0] WARNING: Stale Poller Data Found! Item Time:'1330512182', RRD Time:'1330581481' Ignoring Value!
Checking the database I could see an excessive amount of entries in poller_output_boost:
Code: Select all
mysql> select count(*) from poller_output_boost;
+----------+
| count(*) |
+----------+
| 290491 |
+----------+
1 row in set (0.00 sec)
mysql> select * from poller_output_boost order by time desc limit 1;
+---------------+------------------+---------------------+-------------+
| local_data_id | rrd_name | time | output |
+---------------+------------------+---------------------+-------------+
| 2430 | VsConnections | 2012-03-04 12:04:03 | 0 |
+---------------+------------------+---------------------+-------------+
mysql> select * from poller_output_boost order by time asc limit 1;
+---------------+-------------+---------------------+--------+
| local_data_id | rrd_name | time | output |
+---------------+-------------+---------------------+--------+
| 2685 | traffic_in | 2012-02-29 16:13:02 | U |
+---------------+-------------+---------------------+--------+
Opening the Cacti Console -> System Utilities -> Boost Settings I found that Boost thought the
current date was 5 days in the past. Again confirming via the database:
Code: Select all
mysql> select * from settings where name ='boost_last_run_time';
+---------------------+---------------------+
| name | value |
+---------------------+---------------------+
| boost_last_run_time | 2012-02-29 16:12:04 |
+---------------------+---------------------+
1 row in set (0.00 sec)
The date and time on the server are correct, so I have no idea why this has happened (maybe someone
messed around with them without my knowledge).
I updated the column manually to be about an hour behind the current time (I have Boost set to
do a full update once an hour) and forced Boost to do a full update straight away:
Code: Select all
mysql> REPLACE INTO settings (name, value) VALUES('boost_last_run_time', '2012-03-04 11:31:24');
Query OK, 2 rows affected (0.00 sec)
php poller_boost.php --force
After that I could see that Boost had the correct time and I no longer got any stale data errors
in the logs:
Code: Select all
mysql> select * from settings where name like 'boost_%_run_time';
+---------------------+---------------------+
| name | value |
+---------------------+---------------------+
| boost_last_run_time | 2012-03-04 12:38:11 |
| boost_next_run_time | 2012-03-04 13:38:11 |
+---------------------+---------------------+
2 rows in set (0.00 sec)