Timeout issues.

[jpmallory]

I've got about 70ish devices I'm polling. Most of them work with no issue. I've got a handful that are timing out. I've run snmpget from the command line to those devices and the results come back in less than a second. I also run nagios on this box, which also polls via snmp, and there doesn't appear to be any issues there. I've been trying different timeout settings, but I haven't found a solution.

Here are some relevant lines from the log:

2021/06/25 11:16:42 - SNMPAGENT WARNING: No notification receivers configured for event: cactiNotifyDeviceFailedPoll (CACTI-MIB), severity: medium
2021/06/25 11:16:41 - SPINE: Poller[Main Poller] PID[13994] Device[server1] HT[1] DS[server1 - Traffic - 10.10.10.104 - ens160] Graphs[server1 - Traffic - 10.10.10.104 (ens160)] WARNING: SNMP timeout detected [5000 ms], ignoring host '10.10.10.104'
2021/06/25 11:16:41 - SPINE: Poller[Main Poller] PID[13994] Device[server1] HT[1] DS[server1 - Traffic - 10.10.10.104 - ens160] Graphs[server1 - Traffic - 10.10.10.104 (ens160)] WARNING: SNMP timeout detected [5000 ms], ignoring host '10.10.10.104'
2021/06/25 11:16:41 - SPINE: Poller[Main Poller] PID[13994] Device[server1] HT[1] DS[server1 - CPU - User] Graphs[server1 - CPU Utilization] WARNING: SNMP timeout detected [5000 ms], ignoring host '10.10.10.104'
2021/06/25 11:16:41 - SPINE: Poller[Main Poller] PID[13994] Device[server1] HT[1] DS[server1 - CPU - System] Graphs[server1 - CPU Utilization] WARNING: SNMP timeout detected [5000 ms], ignoring host '10.10.10.104'
2021/06/25 11:16:41 - SPINE: Poller[Main Poller] PID[13994] Device[server1] HT[1] DS[server1 - CPU - Idle] Graphs[server1 - CPU Utilization] WARNING: SNMP timeout detected [5000 ms], ignoring host '10.10.10.104'
2021/06/25 11:16:41 - SPINE: Poller[Main Poller] PID[13994] Device[server1] HT[1] DS[server1 - Processes] Graphs[server1 - Processes] WARNING: SNMP timeout detected [5000 ms], ignoring host '10.10.10.104'
2021/06/25 11:16:41 - SPINE: Poller[Main Poller] PID[13994] Device[server1] HT[1] DS[server1 - Memory - Total] Graphs[server1 - Memory Usage] WARNING: SNMP timeout detected [5000 ms], ignoring host '10.10.10.104'
2021/06/25 11:16:41 - SPINE: Poller[Main Poller] PID[13994] Device[server1] HT[1] DS[server1 - Memory - Free] Graphs[server1 - Memory Usage] WARNING: SNMP timeout detected [5000 ms], ignoring host '10.10.10.104'
2021/06/25 11:16:41 - SPINE: Poller[Main Poller] PID[13994] Device[server1] HT[1] DS[server1 - Memory - Cache] Graphs[server1 - Memory Usage] WARNING: SNMP timeout detected [5000 ms], ignoring host '10.10.10.104'
2021/06/25 11:16:41 - SPINE: Poller[Main Poller] PID[13994] Device[server1] HT[1] DS[server1 - Memory - Buffers] Graphs[server1 - Memory Usage] WARNING: SNMP timeout detected [5000 ms], ignoring host '10.10.10.104'
2021/06/25 11:16:21 - CMDPHP WARNING: SNMP Error:'snmp3_get(): No response from 10.10.10.104', Device:'10.10.10.104', OID:'.1.3.6.1.2.1.25.2.3.1.5.61'
2021/06/25 11:16:11 - CMDPHP WARNING: SNMP Error:'snmp3_get(): No response from 10.10.10.104', Device:'10.10.10.104', OID:'.1.3.6.1.2.1.25.2.3.1.6.61'
2021/06/25 11:16:01 - CMDPHP WARNING: SNMP Error:'snmp3_get(): No response from 10.10.10.104', Device:'10.10.10.104', OID:'.1.3.6.1.2.1.1.3.0'
2021/06/25 11:15:51 - CMDPHP WARNING: SNMP Error:'snmp3_get(): No response from 10.10.10.104', Device:'10.10.10.104', OID:'.1.3.6.1.2.1.1.3.0'
2021/06/25 11:15:36 - SPINE: Poller[Main Poller] PID[13994] Device[server2] HT[1] DS[server2 - Traffic - 10.100.43 - ens160] Graphs[server2 - Traffic - 10.100.43 (ens160)] WARNING: SNMP timeout detected [500 ms], ignoring host '10.100.43'
2021/06/25 11:15:36 - SPINE: Poller[Main Poller] PID[13994] Device[server2] HT[1] DS[server2 - Traffic - 10.100.43 - ens160] Graphs[server2 - Traffic - 10.100.43 (ens160)] WARNING: SNMP timeout detected [500 ms], ignoring host '10.100.43'
2021/06/25 11:15:36 - SPINE: Poller[Main Poller] PID[13994] Device[server2] HT[1] DS[server2 - CPU - User] Graphs[server2 - CPU Utilization] WARNING: SNMP timeout detected [500 ms], ignoring host '10.100.43'
2021/06/25 11:15:36 - SPINE: Poller[Main Poller] PID[13994] Device[server2] HT[1] DS[server2 - CPU - System] Graphs[server2 - CPU Utilization] WARNING: SNMP timeout detected [500 ms], ignoring host '10.100.43'
2021/06/25 11:15:36 - SPINE: Poller[Main Poller] PID[13994] Device[server2] HT[1] DS[server2 - CPU - Idle] Graphs[server2 - CPU Utilization] WARNING: SNMP timeout detected [500 ms], ignoring host '10.100.43'
2021/06/25 11:15:36 - SPINE: Poller[Main Poller] PID[13994] Device[server2] HT[1] DS[server2 - Processes] Graphs[server2 - Processes] WARNING: SNMP timeout detected [500 ms], ignoring host '10.100.43'
2021/06/25 11:15:36 - SPINE: Poller[Main Poller] PID[13994] Device[server2] HT[1] DS[server2 - Memory - Total] Graphs[server2 - Memory Usage] WARNING: SNMP timeout detected [500 ms], ignoring host '10.100.43'
2021/06/25 11:15:36 - SPINE: Poller[Main Poller] PID[13994] Device[server2] HT[1] DS[server2 - Memory - Free] Graphs[server2 - Memory Usage] WARNING: SNMP timeout detected [500 ms], ignoring host '10.100.43'
2021/06/25 11:15:36 - SPINE: Poller[Main Poller] PID[13994] Device[server2] HT[1] DS[server2 - Memory - Cache] Graphs[server2 - Memory Usage] WARNING: SNMP timeout detected [500 ms], ignoring host '10.100.43'
2021/06/25 11:15:36 - SPINE: Poller[Main Poller] PID[13994] Device[server2] HT[1] DS[server2 - Memory - Buffers] Graphs[server2 - Memory Usage] WARNING: SNMP timeout detected [500 ms], ignoring host '10.100.43'
2021/06/25 11:15:34 - CMDPHP WARNING: SNMP Error:'snmp3_get(): No response from 10.100.43', Device:'10.100.43', OID:'.1.3.6.1.2.1.25.2.3.1.5.78'
2021/06/25 11:15:33 - CMDPHP WARNING: SNMP Error:'snmp3_get(): No response from 10.100.43', Device:'10.100.43', OID:'.1.3.6.1.2.1.25.2.3.1.6.78'
2021/06/25 11:15:32 - CMDPHP WARNING: SNMP Error:'snmp3_get(): No response from 10.100.43', Device:'10.100.43', OID:'.1.3.6.1.2.1.25.2.3.1.5.77'
2021/06/25 11:15:31 - CMDPHP WARNING: SNMP Error:'snmp3_get(): No response from 10.100.43', Device:'10.100.43', OID:'.1.3.6.1.2.1.25.2.3.1.6.77'
2021/06/25 11:15:30 - CMDPHP WARNING: SNMP Error:'snmp3_get(): No response from 10.100.43', Device:'10.100.43', OID:'.1.3.6.1.2.1.25.2.3.1.5.76'
2021/06/25 11:15:29 - CMDPHP WARNING: SNMP Error:'snmp3_get(): No response from 10.100.43', Device:'10.100.43', OID:'.1.3.6.1.2.1.25.2.3.1.6.76'

Any ideas?

[netniV]

i believe that if you are using snmp v3 you have to be careful which snmp library you have enabled (phpsnmp vs netsnmp). If you search the forums, you should see some previous comments on this.

[jpmallory]

Thanks. SNMPv3 appears to be a factor here. I just changed a couple of the problem hosts to use SNMPv1, and they started graphing.

What I'm not finding is where to tell cacti/spine which library to use. I was going to remove phpsnmp, but it wants to remove the spine package as well. (This is a redhat system).

[netniV]

Yeah, you shouldn't remove it so much as disable if you want to disable it. However, I would search back through previous forum posts as there are likely a few referring to this exact issue.