issues with graphs workinig

[jkay]

I am running the latest version of cacti, and since I've upgraded I've ran my linux box out of space 2 times and had to delete the log files in order for it to work. It happened again and I cleared them out but now I am not getting new graph data on any of my graphs.

Here is my error log and I'm not sure what else to try, any ideas?

2020/05/01 22:25:01 - POLLER: Poller[Main Poller] WARNING: You have 4 Devices with bad SNMP Indexes. Devices: Device[ICTDC01], Device[RDSHostA02], Device[RDSHostA03], Device[ICTBackup01] totalling 5 Data Sources. Please Either Re-Index, Delete or Disable these Data Sources.
2020/05/01 22:25:00 - SYSTEM STATS: Time:299.0395 Method:spine Processes:2 Threads:2 Hosts:122 HostsPerProcess:61 DataSources:982 RRDsProcessed:0
2020/05/01 22:25:00 - SNMPAGENT WARNING: No notification receivers configured for event: cactiNotifyPollerRuntimeExceeding (CACTI-MIB), severity: high
2020/05/01 22:25:00 - SYSTEM WARNING: Primary Admin account notifications disabled! Unable to send administrative Email.
2020/05/01 22:25:00 - POLLER: Poller[Main Poller] Maximum runtime of 298 seconds exceeded. Exiting.
2020/05/01 22:20:01 - POLLER: Poller[Main Poller] WARNING: You have 4 Devices with bad SNMP Indexes. Devices: Device[ICTDC01], Device[RDSHostA02], Device[RDSHostA03], Device[ICTBackup01] totalling 5 Data Sources. Please Either Re-Index, Delete or Disable these Data Sources.
2020/05/01 22:20:00 - SYSTEM STATS: Time:299.0252 Method:spine Processes:2 Threads:2 Hosts:122 HostsPerProcess:61 DataSources:982 RRDsProcessed:0
2020/05/01 22:20:00 - SNMPAGENT WARNING: No notification receivers configured for event: cactiNotifyPollerRuntimeExceeding (CACTI-MIB), severity: high
2020/05/01 22:20:00 - SYSTEM WARNING: Primary Admin account notifications disabled! Unable to send administrative Email.
2020/05/01 22:20:00 - POLLER: Poller[Main Poller] Maximum runtime of 298 seconds exceeded. Exiting.


Thank you!

[cigamit]

Run a MySQL Check on the DB, if you ran it out of space, than mostly have corrupted tables.

[crazyxbowler]

So I ran the below command and here is what it showed.

Any suggestions?


root@Cacti-Debian:/home/john# mysqlcheck --defaults-file=/etc/mysql/debian.cnf - -auto-repair --optimize --all-databases -p
Enter password:
cacti.aggregate_graph_templates
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.aggregate_graph_templates_graph
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.aggregate_graph_templates_item
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.aggregate_graphs
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.aggregate_graphs_graph_item
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.aggregate_graphs_items
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.automation_devices
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.automation_graph_rule_items
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.automation_graph_rules
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.automation_ips
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.automation_match_rule_items
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.automation_networks
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.automation_processes
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.automation_snmp
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.automation_snmp_items
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.automation_templates
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.automation_tree_rule_items
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.automation_tree_rules
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.cdef
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.cdef_items
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.color_template_items
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.color_templates
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.colors
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.data_debug
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.data_input
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.data_input_data
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.data_input_fields
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.data_local
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.data_source_profiles
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.data_source_profiles_cf
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.data_source_profiles_rra
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.data_source_purge_action
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.data_source_purge_temp
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.data_source_stats_daily
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.data_source_stats_hourly
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.data_source_stats_hourly_cache
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.data_source_stats_hourly_last
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.data_source_stats_monthly
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.data_source_stats_weekly
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.data_source_stats_yearly
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.data_template
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.data_template_data
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.data_template_rrd
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.external_links
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.graph_local
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.graph_template_input
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.graph_template_input_defs
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.graph_templates
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.graph_templates_gprint
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.graph_templates_graph
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.graph_templates_item
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.graph_tree
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.graph_tree_items
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.host
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.host_graph
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.host_snmp_cache
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.host_snmp_query
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.host_template
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.host_template_graph
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.host_template_snmp_query
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.plugin_config
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.plugin_db_changes
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.plugin_hooks
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.plugin_realms
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.poller
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.poller_command
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.poller_data_template_field_mappings
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.poller_item
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.poller_output
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.poller_output_boost
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.poller_output_boost_processes
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.poller_output_realtime
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.poller_reindex
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.poller_resource_cache
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.poller_time
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.processes
note : The storage engine for the table doesn't support optimize
cacti.reports
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.reports_items
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.sessions
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.settings
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.settings_tree
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.settings_user
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.settings_user_group
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.sites
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.snmp_query
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.snmp_query_graph
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.snmp_query_graph_rrd
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.snmp_query_graph_rrd_sv
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.snmp_query_graph_sv
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.snmpagent_cache
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.snmpagent_cache_notifications
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.snmpagent_cache_textual_conventions
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.snmpagent_managers
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.snmpagent_managers_notifications
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.snmpagent_mibs
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.snmpagent_notifications_log
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.user_auth
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.user_auth_cache
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.user_auth_group
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.user_auth_group_members
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.user_auth_group_perms
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.user_auth_group_realm
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.user_auth_perms
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.user_auth_realm
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.user_domains
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.user_domains_ldap
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.user_log
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.vdef
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.vdef_items
note : Table does not support optimize, doing recreate + analyze instead
status : OK
cacti.version
note : Table does not support optimize, doing recreate + analyze instead
status : OK
mysql.column_stats Table is already up to date
mysql.columns_priv Table is already up to date
mysql.db OK
mysql.event Table is already up to date
mysql.func Table is already up to date
mysql.gtid_slave_pos
note : Table does not support optimize, doing recreate + analyze instead
status : OK
mysql.help_category OK
mysql.help_keyword OK
mysql.help_relation OK
mysql.help_topic OK
mysql.host Table is already up to date
mysql.index_stats Table is already up to date
mysql.innodb_index_stats
note : Table does not support optimize, doing recreate + analyze instead
status : OK
mysql.innodb_table_stats
note : Table does not support optimize, doing recreate + analyze instead
status : OK
mysql.plugin Table is already up to date
mysql.proc OK
mysql.procs_priv Table is already up to date
mysql.proxies_priv OK
mysql.roles_mapping Table is already up to date
mysql.servers Table is already up to date
mysql.table_stats Table is already up to date
mysql.tables_priv OK
mysql.time_zone OK
mysql.time_zone_leap_second OK
mysql.time_zone_name OK
mysql.time_zone_transition OK
mysql.time_zone_transition_type OK
mysql.user OK


Thank you for your help!

[jkay]

This is what my event logs in Cacti are showing now

2020/05/04 10:25:01 - AUTOM8 [PID: 1415] Not time to Run Discovery for 'VLAN2'
2020/05/04 10:25:01 - AUTOM8 [PID: 1415] Not time to Run Discovery for 'Test Network'
2020/05/04 10:25:01 - POLLER: Poller[Main Poller] NOTE: Registering process! (maintenance, master, 1, 1417)
2020/05/04 10:25:01 - POLLER: Poller[Main Poller] NOTE: Registering process! (spikekill, master, 0, 1413)
2020/05/04 10:25:01 - POLLER: Poller[Main Poller] NOTE: Registering process! (reports, master, 0, 1411)
2020/05/04 10:25:01 - POLLER: Poller[Main Poller] NOTE: Poller Int: '300', Cron Int: '300', Time Since Last: '300', Max Runtime '298', Poller Runs: '1'
2020/05/04 10:25:00 - SYSTEM STATS: Time:299.4735 Method:spine Processes:2 Threads:2 Hosts:118 HostsPerProcess:59 DataSources:952 RRDsProcessed:0
2020/05/04 10:25:00 - SNMPAGENT WARNING: No notification receivers configured for event: cactiNotifyPollerRuntimeExceeding (CACTI-MIB), severity: high
2020/05/04 10:25:00 - SYSTEM WARNING: Primary Admin account notifications disabled! Unable to send administrative Email.
2020/05/04 10:25:00 - POLLER: Poller[Main Poller] Maximum runtime of 298 seconds exceeded. Exiting.

[jkay]

Any suggestions?

Thanks

[TheWitness]

Run spine manually from the command line to verify its working. There are lots of examples if you search google.

[jkay]

I ran the poller manually and I think it ran fine.

I also did an upgrade to cacti from 1.2.11 to 1.2.12 and still having the same issue.

This is what I am getting in my logs

2020/05/07 10:40:01 - POLLER: Poller[Main Poller] NOTE: Poller Int: '300', Cron Int: '300', Time Since Last: '300.04', Max Runtime '298', Poller Runs: '1'
2020/05/07 10:40:00 - POLLER: Poller[Main Poller] NOTE: Registering process! (reports, master, 0, 6361)
2020/05/07 10:40:00 - AUTOM8 [PID: 6365] Not time to Run Discovery for 'VLAN2'
2020/05/07 10:40:00 - AUTOM8 [PID: 6365] Not time to Run Discovery for 'Test Network'
2020/05/07 10:40:00 - POLLER: Poller[Main Poller] NOTE: Registering process! (maintenance, master, 1, 6367)
2020/05/07 10:40:00 - POLLER: Poller[Main Poller] NOTE: Registering process! (spikekill, master, 0, 6363)
2020/05/07 10:40:00 - SYSTEM STATS: Time:299.0619 Method:spine Processes:10 Threads:4 Hosts:118 HostsPerProcess:12 DataSources:952 RRDsProcessed:0
2020/05/07 10:40:00 - SNMPAGENT WARNING: No notification receivers configured for event: cactiNotifyPollerRuntimeExceeding (CACTI-MIB), severity: high
2020/05/07 10:40:00 - SYSTEM WARNING: Primary Admin account notifications disabled! Unable to send administrative Email.
2020/05/07 10:40:00 - POLLER: Poller[Main Poller] Maximum runtime of 298 seconds exceeded. Exiting.
2020/05/07 10:35:01 - POLLER: Poller[Main Poller] NOTE: Poller Int: '300', Cron Int: '300', Time Since Last: '300.16', Max Runtime '298', Poller Runs: '1'
2020/05/07 10:35:01 - AUTOM8 [PID: 4743] Not time to Run Discovery for 'VLAN2'
2020/05/07 10:35:01 - AUTOM8 [PID: 4743] Not time to Run Discovery for 'Test Network'
2020/05/07 10:35:01 - POLLER: Poller[Main Poller] NOTE: Registering process! (reports, master, 0, 4739)
2020/05/07 10:35:01 - POLLER: Poller[Main Poller] NOTE: Registering process! (maintenance, master, 1, 4745)
2020/05/07 10:35:01 - POLLER: Poller[Main Poller] NOTE: Registering process! (spikekill, master, 0, 4741)
2020/05/07 10:35:00 - SYSTEM STATS: Time:298.9172 Method:spine Processes:10 Threads:4 Hosts:118 HostsPerProcess:12 DataSources:952 RRDsProcessed:0
2020/05/07 10:35:00 - SNMPAGENT WARNING: No notification receivers configured for event: cactiNotifyPollerRuntimeExceeding (CACTI-MIB), severity: high
2020/05/07 10:35:00 - SYSTEM WARNING: Primary Admin account notifications disabled! Unable to send administrative Email.