cc , and took 1.21 seconds, Subject 'Cacti System Warning - Maximum runtime of 298 seconds exceeded for poller id 1. Exi

[eliassal]

Hi,
Cacti was working fine, All of a sudden, since yesterday I started getting emails indicating

Maximum runtime of 298 seconds exceeded for poller id 1. Exiting.

when I visit the "Logs" Tab, I see



I changed the cron job to 6 minutes as follows

*/6 * * * * apache /usr/bin/php /usr/share/cacti/poller.php > /dev/null 2>&1

I played with Poller sync interval and Cron Daemon but it keeps creating this error and nothing is updated in the graphs, here is a snapshot of my cacti config is as follows (which runs on fedora 35, cacti version is 1.2.19)

[eliassal]

I have just rebooted the fedora box still getting

POLLER: Poller[Main Poller] PID[417866] Maximum runtime of 298 seconds exceeded. Exiting.

in the logs

[camerabob]

One of your polled devices is taking too long to respond. You need to find out which one and fix it.

[eliassal]

Can you please tell me how? Is there somewhere in cacti that indcates this? Thanks

[eliassal]

I disabled 2 devices which were in recovery state since several hours, enabled the poller, same thing in addition to a very strange message indicating that admin has removed cacti.log which is not the case!!!



whereas the log file exists and cacti noted right now

[camerabob]

I see you have only 10 hosts. Disable them one at a time or maybe two, and check after a couple of polling cycles. When you have located the slow responders, adjust the polling settings on the host information page. (Console > Management > Devices) Depending on how you are trying to reach these hosts, you may have to tweak the PING settings or the number of SNMP OIDs per requests. Play around, but don't get too drastic.

[eliassal]

I disabled 2 devices which were in recovery state since several hours, enabled the poller, same thing in addition to a very strange message indicating that admin has removed cacti.log which is not the case!!!



whereas the log file exists and cacti noted right now

[eliassal]

This will make go mad. I disabled all devices and kept only 1, same result for the poller

[eliassal]

Also I changed poller interval to 5 minutes and cron/daemon to 5 as well, in the log I get

2022-02-22 15:38:04 - AUTOM8 [PID: 2389319] Not time to Run Discovery for 'mydomain.net'

in spite of the fact that network discovery is disabled

[camerabob]

Perhaps you found the offending device? Try to disable that one and enable another.

[eliassal]

Same thing Sir, I disabled and enabled another 1 which is my desktop, renables the poller, Cacti kept saying "unknown" status for the machine and in the log I got the exact same message.

2022-02-22 17:36:03 - MAILER INFO: Mail successfully sent via SMTP from 'Cacti Sierac <cactionPup29@salam.hd.free.fr>', to 'Administrator <salam@salam.hd.free.fr>', cc '', and took 1.12 seconds, Subject 'Cacti System Warning'
2022-02-22 17:36:02 - POLLER: Poller[Main Poller] PID[2759398] WARNING: Cron is out of sync with the Poller Interval! The Poller Interval is '300' seconds, with a maximum of a '300' second Cron, but 360.5 seconds have passed since the last poll!
2022-02-22 17:36:02 - POLLER: Poller[Main Poller] PID[2759398] NOTE: Poller Int: '300', Cron Int: '300', Time Since Last: '360.52', Max Runtime '298', Poller Runs: '1'
2022-02-22 17:35:03 - AUTOM8 [PID: 2759123] Not time to Run Discovery for 'salam.net'
2022-02-22 17:35:03 - POLLER: Poller[Main Poller] PID[2759119] NOTE: Registering process! (reports, master, 0, 2759119)
2022-02-22 17:35:03 - POLLER: Poller[Main Poller] PID[2759125] NOTE: Registering process! (maintenance, master, 1, 2759125)
2022-02-22 17:35:03 - POLLER: Poller[Main Poller] PID[2759121] NOTE: Registering process! (spikekill, master, 0, 2759121)
2022-02-22 17:35:02 - SYSTEM STATS: Time:298.7003 Method:cmd.php Processes:1 Threads:1 Hosts:1 HostsPerProcess:1 DataSources:186 RRDsProcessed:0
2022-02-22 17:35:02 - MAILER INFO: Mail successfully sent via SMTP from 'Cacti Sierac <cactionPup29@salam.hd.free.fr>', to 'Administrator <salam@salam.hd.free.fr>', cc '', and took 1.11 seconds, Subject 'Cacti System Warning'
2022-02-22 17:35:01 - POLLER: Poller[Main Poller] PID[2723564] Maximum runtime of 298 seconds exceeded. Exiting.

I have the impression that Cacti is tuck by itself, I will repeat my question, is there any way or tool to see what is going wrong?

[camerabob]

What are your settings for this device? SNMP and PING, SNMP or PING, SNMP only, PING only? Are you sure the device is reachable by the Cacti server?

[eliassal]

Yes, when I do
snmpwalk -v2c -c community salam7:161 1.3
or snmpwalk -v2c -c community myrouter:161 1.3

I get thousands of counters

SNMPv2-MIB::snmpOutTooBigs.0 = Counter32: 0
SNMPv2-MIB::snmpOutNoSuchNames.0 = Counter32: 0
SNMPv2-MIB::snmpOutBadValues.0 = Counter32: 0
SNMPv2-MIB::snmpOutGenErrs.0 = Counter32: 0
SNMPv2-MIB::snmpOutGetRequests.0 = Counter32: 0
SNMPv2-MIB::snmpOutGetNexts.0 = Counter32: 0
SNMPv2-MIB::snmpOutSetRequests.0 = Counter32: 0
SNMPv2-MIB::snmpOutGetResponses.0 = Counter32: 471
SNMPv2-MIB::snmpOutTraps.0 = Counter32: 7
SNMPv2-MIB::snmpEnableAuthenTraps.0 = INTEGER: enabled(1)
HOST-RESOURCES-MIB::hrProcessorFrwID.0 = Wrong Type (should be OBJECT IDENTIFIER): INTEGER: 1
HOST-RESOURCES-MIB::hrProcessorLoad.0 = INTEGER: 10
SNMPv2-SMI::enterprises.7367.1.1.0 = STRING: "No sensor"
SNMPv2-SMI::enterprises.7367.3.1.0 = STRING: "Vigor2926ac"
SNMPv2-SMI::enterprises.7367.3.2.0 = STRING: "3.9.6.1 V385_v2926_3.9.6.1_Release"
SNMPv2-SMI::enterprises.7367.3.3.0 = STRING: "Apr 29 2021 16:04:28"
SNMPv2-SMI::enterprises.7367.3.4.0 = STRING: "Resrved"
SNMPv2-SMI::enterprises.7367.3.5.0 = STRING: "France"
SNMPv2-SMI::enterprises.7367.3.6.0 = STRING: "0x30"
SNMPv2-SMI::enterprises.7367.3.7.0 = INTEGER: 54
SNMPv2-SMI::enterprises.7367.3.8.0 = STRING: "001DAA661748"

[eliassal]

I tried to refresh some collectors on the device, in the log I see in red

2022-02-23 09:07:46 - CMDPHP PHP ERROR WARNING Backtrace: (/host.php[116]:host_reload_query(), /host.php[574]:run_data_query(), /lib/data_query.php[108]:query_script_host(), /lib/data_query.php[647]:CactiErrorHandler())
2022-02-23 09:07:46 - ERROR PHP WARNING: Undefined array key "script_path" in file: /usr/share/cacti/lib/data_query.php on line: 647
2022-02-23 09:07:46 - CMDPHP PHP ERROR WARNING Backtrace: (/host.php[116]:host_reload_query(), /host.php[574]:run_data_query(), /lib/data_query.php[108]:query_script_host(), /lib/data_query.php[647]:CactiErrorHandler())
2022-02-23 09:07:46 - ERROR PHP WARNING: Undefined array key "arg_index" in file: /usr/share/cacti/lib/data_query.php on line: 647
2022-02-23 09:06:03 - SYSTEM WARNING: System Polling is Disabled! Therefore, data collection from the poller will be suspended till re-enabled.

[camerabob]

My level of troubleshooting is exhausted at this point.