Mac Track issue, MySQL failing

[MI1]

Greetings, something strange happened and now I have Cacti log full of this:

Code: Select all

12/28/2012 08:31:12 AM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'2006', SQL:"SELECT * FROM mac_track_temp_ports WHERE ip_address != '' AND (dns_hostname = '' OR dns_hostname IS NULL)"  
12/28/2012 08:31:12 AM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'2006', SQL:"SELECT last_rundate, COUNT(last_rundate) AS devices FROM mac_track_devices WHERE disabled = '' GROUP BY last_rundate ORDER BY last_rundate DESC;"  
12/28/2012 08:31:12 AM - CMDPHP: Poller[0] ERROR: SQL Cell Failed!, Error:'2006', SQL:"SELECT COUNT(*) FROM mac_track_processes WHERE device_id <> '0'"  
12/28/2012 08:31:12 AM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'2006', SQL:"SELECT * FROM mac_track_temp_ports WHERE ip_address != '' AND (dns_hostname = '' OR dns_hostname IS NULL)"  
12/28/2012 08:31:12 AM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'2006', SQL:"SELECT last_rundate, COUNT(last_rundate) AS devices FROM mac_track_devices WHERE disabled = '' GROUP BY last_rundate ORDER BY last_rundate DESC;"  
12/28/2012 08:31:12 AM - CMDPHP: Poller[0] ERROR: SQL Cell Failed!, Error:'2006', SQL:"SELECT COUNT(*) FROM mac_track_processes WHERE device_id <> '0'"  
12/28/2012 08:31:12 AM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'2006', SQL:"SELECT * FROM mac_track_temp_ports WHERE ip_address != '' AND (dns_hostname = '' OR dns_hostname IS NULL)"  
12/28/2012 08:31:12 AM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'2006', SQL:"SELECT last_rundate, COUNT(last_rundate) AS devices FROM mac_track_devices WHERE disabled = '' GROUP BY last_rundate ORDER BY last_rundate DESC;"  
12/28/2012 08:31:12 AM - CMDPHP: Poller[0] ERROR: SQL Cell Failed!, Error:'2006', SQL:"SELECT COUNT(*) FROM mac_track_processes WHERE device_id <> '0'"  
12/28/2012 08:31:09 AM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'2006', SQL:"SELECT * FROM mac_track_temp_ports WHERE ip_address != '' AND (dns_hostname = '' OR dns_hostname IS NULL)"  
12/28/2012 08:31:09 AM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'2006', SQL:"SELECT * FROM mac_track_temp_ports WHERE ip_address != '' AND (dns_hostname = '' OR dns_hostname IS NULL)"  
12/28/2012 08:31:09 AM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'2006', SQL:"SELECT last_rundate, COUNT(last_rundate) AS devices FROM mac_track_devices WHERE disabled = '' GROUP BY last_rundate ORDER BY last_rundate DESC;"  
12/28/2012 08:31:09 AM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'2006', SQL:"SELECT * FROM mac_track_temp_ports WHERE ip_address != '' AND (dns_hostname = '' OR dns_hostname IS NULL)"  
12/28/2012 08:31:09 AM - CMDPHP: Poller[0] ERROR: SQL Cell Failed!, Error:'2006', SQL:"SELECT COUNT(*) FROM mac_track_processes WHERE device_id <> '0'"  
12/28/2012 08:31:09 AM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'2006', SQL:"SELECT last_rundate, COUNT(last_rundate) AS devices FROM mac_track_devices WHERE disabled = '' GROUP BY last_rundate ORDER BY last_rundate DESC;"  
12/28/2012 08:31:09 AM - CMDPHP: Poller[0] ERROR: SQL Cell Failed!, Error:'2006', SQL:"SELECT COUNT(*) FROM mac_track_processes WHERE device_id <> '0'"  
12/28/2012 08:31:09 AM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'2006', SQL:"SELECT last_rundate, COUNT(last_rundate) AS devices FROM mac_track_devices WHERE disabled = '' GROUP BY last_rundate ORDER BY last_rundate DESC;"  
12/28/2012 08:31:09 AM - CMDPHP: Poller[0] ERROR: SQL Cell Failed!, Error:'2006', SQL:"SELECT COUNT(*) FROM mac_track_processes WHERE device_id <> '0'"  
12/28/2012 08:31:06 AM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'2006', SQL:"SELECT * FROM mac_track_temp_ports WHERE ip_address != '' AND (dns_hostname = '' OR dns_hostname IS NULL)"  
12/28/2012 08:31:06 AM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'2006', SQL:"SELECT last_rundate, COUNT(last_rundate) AS devices FROM mac_track_devices WHERE disabled = '' GROUP BY last_rundate ORDER BY last_rundate DESC;"  
12/28/2012 08:31:05 AM - CMDPHP: Poller[0] ERROR: SQL Cell Failed!, Error:'2006', SQL:"SELECT COUNT(*) FROM mac_track_processes WHERE device_id <> '0'"  
12/28/2012 08:31:06 AM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'2006', SQL:"SELECT * FROM mac_track_temp_ports WHERE ip_address != '' AND (dns_hostname = '' OR dns_hostname IS NULL)"  
12/28/2012 08:31:06 AM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'2006', SQL:"SELECT * FROM mac_track_temp_ports WHERE ip_address != '' AND (dns_hostname = '' OR dns_hostname IS NULL)"  
12/28/2012 08:31:06 AM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'2006', SQL:"SELECT last_rundate, COUNT(last_rundate) AS devices FROM mac_track_devices WHERE disabled = '' GROUP BY last_rundate ORDER BY last_rundate DESC;"  
12/28/2012 08:31:06 AM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'2006', SQL:"SELECT last_rundate, COUNT(last_rundate) AS devices FROM mac_track_devices WHERE disabled = '' GROUP BY last_rundate ORDER BY last_rundate DESC;"  
12/28/2012 08:31:05 AM - CMDPHP: Poller[0] ERROR: SQL Cell Failed!, Error:'2006', SQL:"SELECT COUNT(*) FROM mac_track_processes WHERE device_id <> '0'"  
12/28/2012 08:31:05 AM - CMDPHP: Poller[0] ERROR: SQL Cell Failed!, Error:'2006', SQL:"SELECT COUNT(*) FROM mac_track_processes WHERE device_id <> '0'"  
12/28/2012 08:31:02 AM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'2006', SQL:"SELECT * FROM mac_track_temp_ports WHERE ip_address != '' AND (dns_hostname = '' OR dns_hostname IS NULL)"  
12/28/2012 08:31:02 AM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'2006', SQL:"SELECT last_rundate, COUNT(last_rundate) AS devices FROM mac_track_devices WHERE disabled = '' GROUP BY last_rundate ORDER BY last_rundate DESC;"  
12/28/2012 08:31:02 AM - CMDPHP: Poller[0] ERROR: SQL Cell Failed!, Error:'2006', SQL:"SELECT COUNT(*) FROM mac_track_processes WHERE device_id <> '0'"  
12/28/2012 08:31:02 AM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'2006', SQL:"SELECT * FROM mac_track_temp_ports WHERE ip_address != '' AND (dns_hostname = '' OR dns_hostname IS NULL)"  
12/28/2012 08:31:02 AM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'2006', SQL:"SELECT last_rundate, COUNT(last_rundate) AS devices FROM mac_track_devices WHERE disabled = '' GROUP BY last_rundate ORDER BY last_rundate DESC;"  
12/28/2012 08:31:02 AM - CMDPHP: Poller[0] ERROR: SQL Cell Failed!, Error:'2006', SQL:"SELECT COUNT(*) FROM mac_track_processes WHERE device_id <> '0'"  
12/28/2012 08:31:02 AM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'2006', SQL:"SELECT * FROM mac_track_temp_ports WHERE ip_address != '' AND (dns_hostname = '' OR dns_hostname IS NULL)"  
12/28/2012 08:31:02 AM - CMDPHP: Poller[0] ERROR: SQL Assoc Failed!, Error:'2006', SQL:"SELECT last_rundate, COUNT(last_rundate) AS devices FROM mac_track_devices WHERE disabled = '' GROUP BY last_rundate ORDER BY last_rundate DESC;"  

and this is probably causing some kind of MySQL freeze. For whole week there are missing segments around all graphs and it looks like MySQL is under heavy load. Network usage on teMySQL graph seems to be very high [screenshot], after MySQL restart it drops and then sudenly MySQL is in some kind of sleeping state, then after few minutes / hours everything works as it shoud be. This happends last week. MySQL max connections is set to 900. Mactrack errors showed in log in the same time so I am guessing Mactrack is causing all of this somehow.

This is huge [and old] installation with around 1200 devices, 20 000 graphs and many many plugins. This issue seems to be happening just from last week. Any help is highly appreciated. Thx

Cacti 0.8.7g / no upgrade possible
Mactrack 2.9 / no update possible atm

// Edit

It looks like there is problem with 'poller_output' :

Code: Select all

cat /var/lib/mysql/hostname.err

Version: '5.1.32-1'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  (Debian)
121228 10:01:37 [ERROR] /usr/sbin/mysqld: The table 'poller_output' is full
121228 11:19:54 [ERROR] /usr/sbin/mysqld: The table 'poller_output' is full
121228 11:19:54 [ERROR] /usr/sbin/mysqld: The table 'poller_output' is full
121228 11:19:54 [ERROR] /usr/sbin/mysqld: The table 'poller_output' is full
121228 11:19:54 [ERROR] /usr/sbin/mysqld: The table 'poller_output' is full
121228 11:19:54 [ERROR] /usr/sbin/mysqld: The table 'poller_output' is full

Should increase PHP memory or recreate that table?

[MI1]

This was obviously not an MacTrack issue but simple MySQL 2006 error. Solved with by increasing MySQL communication packet value.

[jagmeetbhinder]

I have same problem and increasing max_allowed_packets size did not help. Can you please post your resolution?

Thanks