I've setup syslog 10 days ago to run on separated database on the same host with a bunch of alerts/reports/removal rules. It was running fine until Today around 3.58am when it looks like it started to fail
I think I've selected 2 partitions per day.
Logs found on /var/log/mariadb/mariadb.log
Logs from cacti.log, there is a lot of crap after syslog and weathermap got disabled and apparently MariaDB crashed or something.180626 3:58:36 [ERROR] mysqld: Incorrect information in file: './syslog/syslog.frm'
180626 3:58:36 [ERROR] mysqld: Incorrect information in file: './syslog/syslog_removed.frm'
180626 4:00:01 [ERROR] Error in accept: Too many open files
180626 4:21:54 [ERROR] Error in accept: Too many open files
180626 5:22:34 [ERROR] Error in accept: Too many open files
180626 6:55:02 [ERROR] Error in accept: Too many open files
Not very familiar with this sort of mysql issues...suggestions? ideas?26/Jun/2018 05:57:57 - CMDPHP ERRORS DETECTED - DISABLING PLUGIN 'syslog'
26/Jun/2018 05:57:57 - CMDPHP PHP ERROR Backtrace: (CactiShutdownHandler)(/lib/functions.php: 4628 cacti_debug_backtrace)
26/Jun/2018 05:57:57 - ERROR PHP ERROR in Plugin 'syslog': Call to undefined function syslog_db_fetch_row() in file: /usr/share/cacti-1.1.37/plugins/syslog/syslog_counter.php on line: 43
26/Jun/2018 05:56:59 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 05:56:59 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 05:56:00 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 05:56:00 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 05:56:00 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 05:56:00 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 05:50:01 - CMDPHP ERRORS DETECTED - DISABLING PLUGIN 'syslog'
26/Jun/2018 05:50:01 - CMDPHP PHP ERROR Backtrace: (CactiShutdownHandler)(/lib/functions.php: 4628 cacti_debug_backtrace)
26/Jun/2018 05:50:01 - ERROR PHP ERROR in Plugin 'syslog': Call to undefined function syslog_db_fetch_row() in file: /usr/share/cacti-1.1.37/plugins/syslog/syslog_counter.php on line: 43
26/Jun/2018 05:45:19 - CMDPHP ERRORS DETECTED - DISABLING PLUGIN 'syslog'
26/Jun/2018 05:45:19 - CMDPHP PHP ERROR Backtrace: (CactiShutdownHandler)(/lib/functions.php: 4628 cacti_debug_backtrace)
26/Jun/2018 05:45:19 - ERROR PHP ERROR in Plugin 'syslog': Call to undefined function syslog_db_fetch_row() in file: /usr/share/cacti-1.1.37/plugins/syslog/syslog_counter.php on line: 43
26/Jun/2018 05:40:18 - CMDPHP ERRORS DETECTED - DISABLING PLUGIN 'syslog'
26/Jun/2018 05:40:18 - CMDPHP PHP ERROR Backtrace: (CactiShutdownHandler)(/lib/functions.php: 4628 cacti_debug_backtrace)
26/Jun/2018 05:40:18 - ERROR PHP ERROR in Plugin 'syslog': Call to undefined function syslog_db_fetch_row() in file: /usr/share/cacti-1.1.37/plugins/syslog/syslog_counter.php on line: 43
26/Jun/2018 05:35:03 - CMDPHP ERRORS DETECTED - DISABLING PLUGIN 'syslog'
26/Jun/2018 05:35:03 - CMDPHP PHP ERROR Backtrace: (CactiShutdownHandler)(/lib/functions.php: 4628 cacti_debug_backtrace)
26/Jun/2018 05:35:03 - ERROR PHP ERROR in Plugin 'syslog': Call to undefined function syslog_db_fetch_row() in file: /usr/share/cacti-1.1.37/plugins/syslog/syslog_counter.php on line: 43
26/Jun/2018 05:33:00 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 05:33:00 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 05:33:00 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAI' (Errcode: 24)
26/Jun/2018 05:33:00 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 05:31:25 - CMDPHP ERRORS DETECTED - DISABLING PLUGIN 'syslog'
26/Jun/2018 05:31:25 - CMDPHP PHP ERROR Backtrace: (CactiShutdownHandler)(/lib/functions.php: 4628 cacti_debug_backtrace)
26/Jun/2018 05:31:25 - ERROR PHP ERROR in Plugin 'syslog': Call to undefined function syslog_db_fetch_row() in file: /usr/share/cacti-1.1.37/plugins/syslog/syslog_counter.php on line: 43
26/Jun/2018 05:31:00 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 05:31:00 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 05:31:00 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 05:31:00 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 05:23:29 - CMDPHP ERRORS DETECTED - DISABLING PLUGIN 'syslog'
26/Jun/2018 05:23:29 - CMDPHP PHP ERROR Backtrace: (CactiShutdownHandler)(/lib/functions.php: 4628 cacti_debug_backtrace)
26/Jun/2018 05:23:29 - ERROR PHP ERROR in Plugin 'syslog': Call to undefined function syslog_db_fetch_row() in file: /usr/share/cacti-1.1.37/plugins/syslog/syslog_counter.php on line: 43
26/Jun/2018 05:22:59 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 05:22:59 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 05:22:59 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 05:22:59 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 05:22:59 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 05:22:59 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 05:21:59 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAI' (Errcode: 24)
26/Jun/2018 05:21:59 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 05:21:09 - CMDPHP ERRORS DETECTED - DISABLING PLUGIN 'syslog'
26/Jun/2018 05:21:09 - CMDPHP PHP ERROR Backtrace: (CactiShutdownHandler)(/lib/functions.php: 4628 cacti_debug_backtrace)
26/Jun/2018 05:21:09 - ERROR PHP ERROR in Plugin 'syslog': Call to undefined function syslog_db_fetch_row() in file: /usr/share/cacti-1.1.37/plugins/syslog/syslog_counter.php on line: 43
26/Jun/2018 05:21:00 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 05:21:00 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 05:21:00 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAI' (Errcode: 24)
26/Jun/2018 05:21:00 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 05:20:00 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 05:20:00 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 05:20:00 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 05:20:00 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 05:20:00 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 05:20:00 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 05:16:00 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAI' (Errcode: 24)
26/Jun/2018 05:16:00 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 05:16:00 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAI' (Errcode: 24)
26/Jun/2018 05:16:00 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 05:15:00 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 05:15:00 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 05:12:04 - CMDPHP ERRORS DETECTED - DISABLING PLUGIN 'syslog'
26/Jun/2018 05:12:04 - CMDPHP PHP ERROR Backtrace: (CactiShutdownHandler)(/lib/functions.php: 4628 cacti_debug_backtrace)
26/Jun/2018 05:12:04 - ERROR PHP ERROR in Plugin 'syslog': Call to undefined function syslog_db_fetch_row() in file: /usr/share/cacti-1.1.37/plugins/syslog/syslog_counter.php on line: 43
26/Jun/2018 05:12:00 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 05:12:00 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 05:10:00 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 05:10:00 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 05:10:00 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 05:10:00 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 05:04:16 - CMDPHP ERRORS DETECTED - DISABLING PLUGIN 'syslog'
26/Jun/2018 05:04:16 - CMDPHP PHP ERROR Backtrace: (CactiShutdownHandler)(/lib/functions.php: 4628 cacti_debug_backtrace)
26/Jun/2018 05:04:16 - ERROR PHP ERROR in Plugin 'syslog': Call to undefined function syslog_db_fetch_row() in file: /usr/share/cacti-1.1.37/plugins/syslog/syslog_counter.php on line: 43
26/Jun/2018 04:59:20 - CMDPHP ERRORS DETECTED - DISABLING PLUGIN 'syslog'
26/Jun/2018 04:59:20 - CMDPHP PHP ERROR Backtrace: (CactiShutdownHandler)(/lib/functions.php: 4628 cacti_debug_backtrace)
26/Jun/2018 04:59:20 - ERROR PHP ERROR in Plugin 'syslog': Call to undefined function syslog_db_fetch_row() in file: /usr/share/cacti-1.1.37/plugins/syslog/syslog_counter.php on line: 43
26/Jun/2018 04:54:18 - CMDPHP ERRORS DETECTED - DISABLING PLUGIN 'syslog'
26/Jun/2018 04:54:18 - CMDPHP PHP ERROR Backtrace: (CactiShutdownHandler)(/lib/functions.php: 4628 cacti_debug_backtrace)
26/Jun/2018 04:54:18 - ERROR PHP ERROR in Plugin 'syslog': Call to undefined function syslog_db_fetch_row() in file: /usr/share/cacti-1.1.37/plugins/syslog/syslog_counter.php on line: 43
26/Jun/2018 04:49:40 - CMDPHP ERRORS DETECTED - DISABLING PLUGIN 'syslog'
26/Jun/2018 04:49:40 - CMDPHP PHP ERROR Backtrace: (CactiShutdownHandler)(/lib/functions.php: 4628 cacti_debug_backtrace)
26/Jun/2018 04:49:40 - ERROR PHP ERROR in Plugin 'syslog': Call to undefined function syslog_db_fetch_row() in file: /usr/share/cacti-1.1.37/plugins/syslog/syslog_counter.php on line: 43
26/Jun/2018 04:48:01 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 04:48:01 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 04:47:01 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 04:47:01 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 04:47:01 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 04:47:01 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 04:47:01 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 04:47:01 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 04:46:15 - CMDPHP ERRORS DETECTED - DISABLING PLUGIN 'syslog'
26/Jun/2018 04:46:15 - CMDPHP PHP ERROR Backtrace: (CactiShutdownHandler)(/lib/functions.php: 4628 cacti_debug_backtrace)
26/Jun/2018 04:46:15 - ERROR PHP ERROR in Plugin 'syslog': Call to undefined function syslog_db_fetch_row() in file: /usr/share/cacti-1.1.37/plugins/syslog/syslog_counter.php on line: 43
26/Jun/2018 04:46:01 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAI' (Errcode: 24)
26/Jun/2018 04:46:01 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 04:45:02 - DBCALL ERROR: SQL Assoc Failed!, Error: Can't read dir of './cacti1x/' (errno: 24)
26/Jun/2018 04:45:02 - DBCALL ERROR: SQL Assoc Failed!, Error:1018, SQL:'SHOW TABLE STATUS WHERE `Name` LIKE 'reportit_tmp_%' AND (UNIX_TIMESTAMP(`Update_time`) + 300) <= UNIX_TIMESTAMP()'
26/Jun/2018 04:45:02 - CMDPHP ERROR: A DB Exec Failed!, Error: Table 'cacti1x.plugin_thold_host_failed' doesn't exist
26/Jun/2018 04:45:02 - DBCALL ERROR: A DB Exec Failed!, Error:1146, SQL:'TRUNCATE plugin_thold_host_failed'
26/Jun/2018 04:37:29 - CMDPHP ERRORS DETECTED - DISABLING PLUGIN 'syslog'
26/Jun/2018 04:37:29 - CMDPHP PHP ERROR Backtrace: (CactiShutdownHandler)(/lib/functions.php: 4628 cacti_debug_backtrace)
26/Jun/2018 04:37:29 - ERROR PHP ERROR in Plugin 'syslog': Call to undefined function syslog_db_fetch_row() in file: /usr/share/cacti-1.1.37/plugins/syslog/syslog_counter.php on line: 43
26/Jun/2018 04:35:01 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_1.MAI' (Errcode: 24)
26/Jun/2018 04:35:01 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 04:35:01 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAI' (Errcode: 24)
26/Jun/2018 04:35:01 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 04:33:01 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 04:33:01 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 04:33:00 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 04:33:00 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 04:32:09 - CMDPHP ERRORS DETECTED - DISABLING PLUGIN 'syslog'
26/Jun/2018 04:32:09 - CMDPHP PHP ERROR Backtrace: (CactiShutdownHandler)(/lib/functions.php: 4628 cacti_debug_backtrace)
26/Jun/2018 04:32:09 - ERROR PHP ERROR in Plugin 'syslog': Call to undefined function syslog_db_fetch_row() in file: /usr/share/cacti-1.1.37/plugins/syslog/syslog_counter.php on line: 43
26/Jun/2018 04:32:01 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 04:32:01 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 04:30:01 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 04:30:01 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 04:23:42 - CMDPHP ERRORS DETECTED - DISABLING PLUGIN 'syslog'
26/Jun/2018 04:23:42 - CMDPHP PHP ERROR Backtrace: (CactiShutdownHandler)(/lib/functions.php: 4628 cacti_debug_backtrace)
26/Jun/2018 04:23:42 - ERROR PHP ERROR in Plugin 'syslog': Call to undefined function syslog_db_fetch_row() in file: /usr/share/cacti-1.1.37/plugins/syslog/syslog_counter.php on line: 43
2018/06/26 04:23:01 - SPINE: Poller[Main Poller] ERROR: Spine Timed Out While Waiting for Threads to End
2018/06/26 04:22:59 - SPINE: Poller[Main Poller] ERROR: Spine Timed Out While Processing Devices Internal
2018/06/26 04:22:02 - SPINE: Poller[Main Poller] ERROR: Spine Timed Out While Waiting for Threads to End
26/Jun/2018 04:22:01 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAI' (Errcode: 24)
26/Jun/2018 04:22:01 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
2018/06/26 04:22:00 - SPINE: Poller[Main Poller] ERROR: Spine Timed Out While Processing Devices Internal
26/Jun/2018 04:21:06 - CMDPHP ERRORS DETECTED - DISABLING PLUGIN 'syslog'
26/Jun/2018 04:21:06 - CMDPHP PHP ERROR Backtrace: (CactiShutdownHandler)(/lib/functions.php: 4628 cacti_debug_backtrace)
26/Jun/2018 04:21:06 - ERROR PHP ERROR in Plugin 'syslog': Call to undefined function syslog_db_fetch_row() in file: /usr/share/cacti-1.1.37/plugins/syslog/functions.php on line: 70
2018/06/26 04:21:02 - SPINE: Poller[Main Poller] ERROR: Spine Timed Out While Waiting for Threads to End
26/Jun/2018 04:21:01 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAI' (Errcode: 24)
26/Jun/2018 04:21:01 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
2018/06/26 04:21:00 - SPINE: Poller[Main Poller] ERROR: Spine Timed Out While Processing Devices Internal
26/Jun/2018 04:15:04 - CMDPHP PHP ERROR NOTICE Backtrace: (/plugins/syslog/syslog_process.php: 154 syslog_is_partitioned)(/plugins/syslog/functions.php: 71 CactiErrorHandler)(/lib/functions.php: 4590 cacti_debug_backtrace)
26/Jun/2018 04:15:04 - ERROR PHP NOTICE in Plugin 'syslog': Undefined index: Create Table in file: /usr/share/cacti-1.1.37/plugins/syslog/functions.php on line: 71
26/Jun/2018 04:15:02 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 04:15:02 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
2018/06/26 04:12:03 - SPINE: Poller[Main Poller] ERROR: Spine Timed Out While Waiting for Threads to End
2018/06/26 04:12:01 - SPINE: Poller[Main Poller] ERROR: Spine Timed Out While Processing Devices Internal
26/Jun/2018 04:10:02 - CMDPHP ERRORS DETECTED - DISABLING PLUGIN 'weathermap'
26/Jun/2018 04:10:02 - CMDPHP PHP ERROR Backtrace: (CactiShutdownHandler)(/lib/functions.php: 4628 cacti_debug_backtrace)
26/Jun/2018 04:10:02 - ERROR PHP ERROR in Plugin 'weathermap': Uncaught exception 'PDOException' with message 'SQLSTATE[HY000]: General error: 1018 Can't read dir of './cacti1x/' (errno: 24)' in /usr/share/cacti-1.1.37/plugins/weathermap/lib/datasources/WeatherMapDataSource_cactithold.php:64
26/Jun/2018 04:06:46 - CMDPHP ERROR: A DB Exec Failed!, Error: Table 'syslog.syslog_statistics' doesn't exist
26/Jun/2018 04:06:46 - DBCALL ERROR: A DB Exec Failed!, Error:1146, SQL:'TRUNCATE `syslog`.`syslog_statistics`'
26/Jun/2018 04:06:46 - CMDPHP PHP ERROR NOTICE Backtrace: (/plugins/syslog/syslog_process.php: 154 syslog_is_partitioned)(/plugins/syslog/functions.php: 71 CactiErrorHandler)(/lib/functions.php: 4590 cacti_debug_backtrace)
26/Jun/2018 04:06:46 - ERROR PHP NOTICE in Plugin 'syslog': Undefined index: Create Table in file: /usr/share/cacti-1.1.37/plugins/syslog/functions.php on line: 71
26/Jun/2018 04:06:12 - CMDPHP ERROR: A DB Exec Failed!, Error: Table 'syslog.syslog_statistics' doesn't exist
26/Jun/2018 04:06:12 - DBCALL ERROR: A DB Exec Failed!, Error:1146, SQL:'TRUNCATE `syslog`.`syslog_statistics`'
26/Jun/2018 04:06:12 - CMDPHP PHP ERROR NOTICE Backtrace: (/plugins/syslog/syslog_process.php: 154 syslog_is_partitioned)(/plugins/syslog/functions.php: 71 CactiErrorHandler)(/lib/functions.php: 4590 cacti_debug_backtrace)
26/Jun/2018 04:06:12 - ERROR PHP NOTICE in Plugin 'syslog': Undefined index: Create Table in file: /usr/share/cacti-1.1.37/plugins/syslog/functions.php on line: 71
26/Jun/2018 04:06:10 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 04:06:10 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 04:05:34 - CMDPHP ERROR: A DB Exec Failed!, Error: Table 'syslog.syslog_statistics' doesn't exist
26/Jun/2018 04:05:34 - DBCALL ERROR: A DB Exec Failed!, Error:1146, SQL:'TRUNCATE `syslog`.`syslog_statistics`'
26/Jun/2018 04:05:34 - CMDPHP PHP ERROR NOTICE Backtrace: (/plugins/syslog/syslog_process.php: 154 syslog_is_partitioned)(/plugins/syslog/functions.php: 71 CactiErrorHandler)(/lib/functions.php: 4590 cacti_debug_backtrace)
26/Jun/2018 04:05:34 - ERROR PHP NOTICE in Plugin 'syslog': Undefined index: Create Table in file: /usr/share/cacti-1.1.37/plugins/syslog/functions.php on line: 71
26/Jun/2018 04:05:33 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 04:05:33 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT sm.* FROM snmpagent_managers_notifications AS smn INNER JOIN snmpagent_managers AS sm ON sm.id = smn.manager_id WHERE smn.notification = ? AND smn.mib = ? ORDER BY sm.snmp_message_type'
26/Jun/2018 03:58:36 - CMDPHP ERROR: A DB Exec Failed!, Error: Out of resources when opening file './syslog/syslog_statistics.MYD' (Errcode: 24)
26/Jun/2018 03:58:36 - DBCALL ERROR: A DB Exec Failed!, Error:23, SQL:'TRUNCATE `syslog`.`syslog_statistics`'
26/Jun/2018 03:58:36 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 03:58:36 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT * FROM `information_schema`.`partitions` WHERE table_schema='syslog' AND table_name='syslog' ORDER BY partition_ordinal_position'
26/Jun/2018 03:58:36 - CMDPHP ERROR: A DB Exec Failed!, Error: Failed to read from the .par file
26/Jun/2018 03:58:36 - DBCALL ERROR: A DB Exec Failed!, Error:1696, SQL:'ALTER TABLE `syslog`.`syslog_removed` REORGANIZE PARTITION dMaxValue INTO ( PARTITION d20180626 VALUES LESS THAN (TO_DAYS('2018-06-27')), PARTITION dMaxValue VALUES LESS THAN MAXVALUE)'
26/Jun/2018 03:58:36 - DBCALL ERROR: SQL Assoc Failed!, Error: Out of resources when opening file '/var/tmp/#sql_2734_0.MAD' (Errcode: 24)
26/Jun/2018 03:58:36 - DBCALL ERROR: SQL Assoc Failed!, Error:23, SQL:'SELECT * FROM `information_schema`.`partitions` WHERE table_schema='syslog' AND table_name='syslog' ORDER BY partition_ordinal_position'
26/Jun/2018 03:58:36 - CMDPHP ERROR: A DB Exec Failed!, Error: Failed to read from the .par file
26/Jun/2018 03:58:36 - DBCALL ERROR: A DB Exec Failed!, Error:1696, SQL:'ALTER TABLE `syslog`.`syslog` REORGANIZE PARTITION dMaxValue INTO ( PARTITION d20180626 VALUES LESS THAN (TO_DAYS('2018-06-27')), PARTITION dMaxValue VALUES LESS THAN MAXVALUE)'
Open files limit is set to 2000 but I'm suspecting I'm having I/O issues? This is running on a VM
Code: Select all
open_files_limit = 2000
See attached. Thanks and sorry for such a looooooong post.