Cacti & Spine 0.8.8f + SNMPv3 w/ SHA+AES128 not working

[ryanjwh]

I've searched quite a bit around the forums and the 'net but haven't found a working solution for an issue we're having.

Cacti version: 0.8.8f
Spine version: 0.8.8f

We're deploying new hosts outside our perimeter so we're mandating SNMPv3 (authPriv) for obvious security reasons. I've added SNMPv3 default settings (user, pass, hash method, key, cipher, *no* context) to Cacti based on successful tests with snmpwalk:

Code: Select all

root@<hostname>(cacti)# snmpwalk -v 3 -a SHA -A <hidden> -u <hidden> -x AES -X <hidden> -l AuthPriv -n "" <hostname> system.sysUpTime.0
DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (213210) 0:35:32.10

(We can also just not specify -n, but it's here to make it explicit.)

When adding the host as SNMPv3 in Cacti, it queries the host perfectly, allows me to add all graphs include SNMP data query-based graphs (partitions, network interfaces, etc), and all looks well. Here's the top of the host properties page output:

Code: Select all

<Pretty name> (FQDN)
*Data Source List
*Graph List
SNMP Information
System:Linux <hostname> 3.10.0-123.el7.x86_64 #1
SMP Mon Jun 30 12:09:22 UTC 2014 x86_64
Uptime: 251456 (0 days, 0 hours, 41 minutes)
Hostname: <hostname>
Location: Unknown
Contact: blah@blah.com

However, once the poller tries to run, it claims the host is down and never starts populating the graphs. It appears Spine isn't working correctly with our SNMPv3 options and I'm not sure why. I tried taking a pcap and dropping into Wireshark, but Wireshark appears to only be able to decrypt SNMPv3 packets for DES, not AES, so it just says the packet is encrypted so I can't see if spine is passing the correct parameters in the SNMP request.

Here is spine debug output:

When host is set to use ICMP availability check:

Code: Select all

root@<hostname>(cacti)# /usr/bin/spine -H 1946 -R -S -V 5
SPINE: Using spine config file [/etc/spine.conf]
WARNING: DB_PreG is no longer supported
12/08/2015 09:42:16 AM - SPINE: Poller[0] DEBUG: The path_php_server variable is /srv/http/site/cacti.<domain>/html/script_server.php
12/08/2015 09:42:16 AM - SPINE: Poller[0] DEBUG: The path_cactilog variable is /srv/logs/<domain>/cacti/cacti.log
DEBUG: The log_destination variable is 4 (STDOUT)
DEBUG: The path_php variable is /usr/bin/php
DEBUG: The availability_method variable is 2
DEBUG: The ping_recovery_count variable is 3
DEBUG: The ping_failure_count variable is 2
DEBUG: The ping_method variable is 2
DEBUG: The ping_retries variable is 1
DEBUG: The ping_timeout variable is 400
DEBUG: The snmp_retries variable is 3
DEBUG: The log_perror variable is 1
DEBUG: The log_pwarn variable is 1
DEBUG: The boost_redirect variable is 0
DEBUG: The log_pstats variable is 1
DEBUG: The threads variable is 100
DEBUG: The polling interval is 300 seconds
DEBUG: The number of concurrent processes is 1
DEBUG: The script timeout is 10
DEBUG: The number of php script servers to run is 10
DEBUG: Host List to be polled='1946', TotalPHPScripts='1
DEBUG: The PHP Script Server is Required
DEBUG: The Maximum SNMP OID Get Size is 40
Version 0.8.8f starting
DEBUG: MySQL is Thread Safe!
DEBUG: Spine is running asroot.
SPINE: Initializing Net-SNMP API
DEBUG: Issues with SNMP Header Version information, assuming old version of Net-SNMP.
SPINE: Initializing PHP Script Server(s)
DEBUG: SS[0] PHP Script Server Routine Starting
DEBUG: SS[0] PHP Script Server About to FORK Child Process
DEBUG: SS[0] PHP Script Server Child FORK Success
DEBUG: SS[0] Confirmed PHP Script Server running using readfd[7], writefd[6]
DEBUG: SS[1] PHP Script Server Routine Starting
DEBUG: SS[1] PHP Script Server About to FORK Child Process
DEBUG: SS[1] PHP Script Server Child FORK Success
DEBUG: SS[1] Confirmed PHP Script Server running using readfd[9], writefd[8]
DEBUG: SS[2] PHP Script Server Routine Starting
DEBUG: SS[2] PHP Script Server About to FORK Child Process
DEBUG: SS[2] PHP Script Server Child FORK Success
DEBUG: SS[2] Confirmed PHP Script Server running using readfd[11], writefd[10]
DEBUG: SS[3] PHP Script Server Routine Starting
DEBUG: SS[3] PHP Script Server About to FORK Child Process
DEBUG: SS[3] PHP Script Server Child FORK Success
DEBUG: SS[3] Confirmed PHP Script Server running using readfd[13], writefd[12]
DEBUG: SS[4] PHP Script Server Routine Starting
DEBUG: SS[4] PHP Script Server About to FORK Child Process
DEBUG: SS[4] PHP Script Server Child FORK Success
DEBUG: SS[4] Confirmed PHP Script Server running using readfd[15], writefd[14]
DEBUG: SS[5] PHP Script Server Routine Starting
DEBUG: SS[5] PHP Script Server About to FORK Child Process
DEBUG: SS[5] PHP Script Server Child FORK Success
DEBUG: SS[5] Confirmed PHP Script Server running using readfd[17], writefd[16]
DEBUG: SS[6] PHP Script Server Routine Starting
DEBUG: SS[6] PHP Script Server About to FORK Child Process
DEBUG: SS[6] PHP Script Server Child FORK Success
DEBUG: SS[6] Confirmed PHP Script Server running using readfd[19], writefd[18]
DEBUG: SS[7] PHP Script Server Routine Starting
DEBUG: SS[7] PHP Script Server About to FORK Child Process
DEBUG: SS[7] PHP Script Server Child FORK Success
DEBUG: SS[7] Confirmed PHP Script Server running using readfd[21], writefd[20]
DEBUG: SS[8] PHP Script Server Routine Starting
DEBUG: SS[8] PHP Script Server About to FORK Child Process
DEBUG: SS[8] PHP Script Server Child FORK Success
DEBUG: SS[8] Confirmed PHP Script Server running using readfd[23], writefd[22]
DEBUG: SS[9] PHP Script Server Routine Starting
DEBUG: SS[9] PHP Script Server About to FORK Child Process
DEBUG: SS[9] PHP Script Server Child FORK Success
DEBUG: SS[9] Confirmed PHP Script Server running using readfd[25], writefd[24]
NOTE: Spine will support multithread device polling.
DEBUG: Initial Value of Active Threads is 0
DEBUG: Valid Thread to be Created
DEBUG: In Poller, About to Start Polling of Host
DEBUG: The Value of Active Threads is 1
DEBUG: Valid Thread to be Created
DEBUG: In Poller, About to Start Polling of Host
DEBUG: The Value of Active Threads is 2
Host[0] TH[1] Total Time: 0.0012 Seconds
Host[0] TH[1] DEBUG: HOST COMPLETE: About to Exit Host Polling Thread Function
DEBUG: The Value of Active Threads is 1
Host[1946] DEBUG: Entering ICMP Ping
Host[1946] DEBUG: ICMP Host Alive, Try Count:1, Time:151.2420 ms
Host[1946] PING: Result ICMP: Host is Alive
Host[1946] TH[1] RECACHE: Processing 13 items in the auto reindex cache for '<hostname>
Host[1946] TH[1] Recache DataQuery[1] OID: .1.3.6.1.2.1.1.3.0, output: U
Host[1946] TH[1] Recache DataQuery[2] OID: .1.3.6.1.2.1.1.3.0, output: U
Host[1946] TH[1] Recache DataQuery[8] OID: .1.3.6.1.2.1.1.3.0, output: U
Host[1946] TH[1] Recache DataQuery[22] OID: .1.3.6.1.2.1.1.3.0, output: U
Host[1946] TH[1] Recache DataQuery[23] OID: .1.3.6.1.2.1.1.3.0, output: U
Host[1946] TH[1] Recache DataQuery[24] OID: .1.3.6.1.2.1.1.3.0, output: U
Host[1946] TH[1] Recache DataQuery[25] OID: .1.3.6.1.2.1.1.3.0, output: U
Host[1946] TH[1] Recache DataQuery[44] OID: .1.3.6.1.2.1.1.3.0, output: U
Host[1946] TH[1] Recache DataQuery[45] OID: .1.3.6.1.2.1.1.3.0, output: U
Host[1946] TH[1] Recache DataQuery[46] OID: .1.3.6.1.2.1.1.3.0, output: U
Host[1946] TH[1] Recache DataQuery[47] OID: .1.3.6.1.2.1.1.3.0, output: U
Host[1946] TH[1] Recache DataQuery[48] OID: .1.3.6.1.2.1.1.3.0, output: U
Host[1946] TH[1] Recache DataQuery[49] OID: .1.3.6.1.2.1.1.3.0, output: U
Host[1946] TH[1] NOTE: There are '61' Polling Items for this Host
Host[1946] TH[1] Total Time:    40 Seconds
Host[1946] TH[1] DEBUG: HOST COMPLETE: About to Exit Host Polling Thread Function
DEBUG: The Value of Active Threads is 0
DEBUG: Thread Cleanup Complete
DEBUG: SS[0] Script Server Shutdown Started
DEBUG: SS[1] Script Server Shutdown Started
DEBUG: SS[2] Script Server Shutdown Started
DEBUG: SS[3] Script Server Shutdown Started
DEBUG: SS[4] Script Server Shutdown Started
DEBUG: SS[5] Script Server Shutdown Started
DEBUG: SS[6] Script Server Shutdown Started
DEBUG: SS[7] Script Server Shutdown Started
DEBUG: SS[8] Script Server Shutdown Started
DEBUG: SS[9] Script Server Shutdown Started
DEBUG: PHP Script Server Pipes Closed
DEBUG: Allocated Variable Memory Freed
DEBUG: MYSQL Free & Close Completed
DEBUG: Net-SNMP Close Completed
Time: 41.3461 s, Threads: 100, Hosts: 2

When host is set to use UDP SNMP availability check:

Code: Select all

root@<hostname>(cacti)# /usr/bin/spine -H 1946 -R -S -V 5
SPINE: Using spine config file [/etc/spine.conf]
WARNING: DB_PreG is no longer supported
12/08/2015 11:45:32 AM - SPINE: Poller[0] DEBUG: The path_php_server variable is /srv/http/site/cacti.<domain>/html/script_server.php
12/08/2015 11:45:32 AM - SPINE: Poller[0] DEBUG: The path_cactilog variable is /srv/logs/<domain>/cacti/cacti.log
DEBUG: The log_destination variable is 4 (STDOUT)
DEBUG: The path_php variable is /usr/bin/php
DEBUG: The availability_method variable is 2
DEBUG: The ping_recovery_count variable is 3
DEBUG: The ping_failure_count variable is 2
DEBUG: The ping_method variable is 2
DEBUG: The ping_retries variable is 1
DEBUG: The ping_timeout variable is 400
DEBUG: The snmp_retries variable is 3
DEBUG: The log_perror variable is 1
DEBUG: The log_pwarn variable is 1
DEBUG: The boost_redirect variable is 0
DEBUG: The log_pstats variable is 1
DEBUG: The threads variable is 100
DEBUG: The polling interval is 300 seconds
DEBUG: The number of concurrent processes is 1
DEBUG: The script timeout is 10
DEBUG: The number of php script servers to run is 10
DEBUG: Host List to be polled='1946', TotalPHPScripts='1
DEBUG: The PHP Script Server is Required
DEBUG: The Maximum SNMP OID Get Size is 40
Version 0.8.8f starting
DEBUG: MySQL is Thread Safe!
DEBUG: Spine is running asroot.
SPINE: Initializing Net-SNMP API
DEBUG: Issues with SNMP Header Version information, assuming old version of Net-SNMP.
SPINE: Initializing PHP Script Server(s)
DEBUG: SS[0] PHP Script Server Routine Starting
DEBUG: SS[0] PHP Script Server About to FORK Child Process
DEBUG: SS[0] PHP Script Server Child FORK Success
DEBUG: SS[0] Confirmed PHP Script Server running using readfd[7], writefd[6]
DEBUG: SS[1] PHP Script Server Routine Starting
DEBUG: SS[1] PHP Script Server About to FORK Child Process
DEBUG: SS[1] PHP Script Server Child FORK Success
DEBUG: SS[1] Confirmed PHP Script Server running using readfd[9], writefd[8]
DEBUG: SS[2] PHP Script Server Routine Starting
DEBUG: SS[2] PHP Script Server About to FORK Child Process
DEBUG: SS[2] PHP Script Server Child FORK Success
DEBUG: SS[2] Confirmed PHP Script Server running using readfd[11], writefd[10]
DEBUG: SS[3] PHP Script Server Routine Starting
DEBUG: SS[3] PHP Script Server About to FORK Child Process
DEBUG: SS[3] PHP Script Server Child FORK Success
DEBUG: SS[3] Confirmed PHP Script Server running using readfd[13], writefd[12]
DEBUG: SS[4] PHP Script Server Routine Starting
DEBUG: SS[4] PHP Script Server About to FORK Child Process
DEBUG: SS[4] PHP Script Server Child FORK Success
DEBUG: SS[4] Confirmed PHP Script Server running using readfd[15], writefd[14]
DEBUG: SS[5] PHP Script Server Routine Starting
DEBUG: SS[5] PHP Script Server About to FORK Child Process
DEBUG: SS[5] PHP Script Server Child FORK Success
DEBUG: SS[5] Confirmed PHP Script Server running using readfd[17], writefd[16]
DEBUG: SS[6] PHP Script Server Routine Starting
DEBUG: SS[6] PHP Script Server About to FORK Child Process
DEBUG: SS[6] PHP Script Server Child FORK Success
DEBUG: SS[6] Confirmed PHP Script Server running using readfd[19], writefd[18]
DEBUG: SS[7] PHP Script Server Routine Starting
DEBUG: SS[7] PHP Script Server About to FORK Child Process
DEBUG: SS[7] PHP Script Server Child FORK Success
DEBUG: SS[7] Confirmed PHP Script Server running using readfd[21], writefd[20]
DEBUG: SS[8] PHP Script Server Routine Starting
DEBUG: SS[8] PHP Script Server About to FORK Child Process
DEBUG: SS[8] PHP Script Server Child FORK Success
DEBUG: SS[8] Confirmed PHP Script Server running using readfd[23], writefd[22]
DEBUG: SS[9] PHP Script Server Routine Starting
DEBUG: SS[9] PHP Script Server About to FORK Child Process
DEBUG: SS[9] PHP Script Server Child FORK Success
DEBUG: SS[9] Confirmed PHP Script Server running using readfd[25], writefd[24]
NOTE: Spine will support multithread device polling.
DEBUG: Initial Value of Active Threads is 0
DEBUG: Valid Thread to be Created
DEBUG: In Poller, About to Start Polling of Host
DEBUG: The Value of Active Threads is 1
DEBUG: Valid Thread to be Created
DEBUG: In Poller, About to Start Polling of Host
DEBUG: The Value of Active Threads is 2
Host[0] TH[1] Total Time: 0.00098 Seconds
Host[0] TH[1] DEBUG: HOST COMPLETE: About to Exit Host Polling Thread Function
DEBUG: The Value of Active Threads is 1
Host[1946] DEBUG: Entering SNMP Ping
Host[1946] SNMP Ping Error: Unknown error: 2
Host[1946] SNMP Result: Host did not respond to SNMP
Host[1946] TH[1] NOTE: There are '61' Polling Items for this Host
Host[1946] TH[1] Total Time:    40 Seconds
Host[1946] TH[1] DEBUG: HOST COMPLETE: About to Exit Host Polling Thread Function
DEBUG: The Value of Active Threads is 0
DEBUG: Thread Cleanup Complete
DEBUG: SS[0] Script Server Shutdown Started
DEBUG: SS[1] Script Server Shutdown Started
DEBUG: SS[2] Script Server Shutdown Started
DEBUG: SS[3] Script Server Shutdown Started
DEBUG: SS[4] Script Server Shutdown Started
DEBUG: SS[5] Script Server Shutdown Started
DEBUG: SS[6] Script Server Shutdown Started
DEBUG: SS[7] Script Server Shutdown Started
DEBUG: SS[8] Script Server Shutdown Started
DEBUG: SS[9] Script Server Shutdown Started
DEBUG: PHP Script Server Pipes Closed
DEBUG: Allocated Variable Memory Freed
DEBUG: MYSQL Free & Close Completed
DEBUG: Net-SNMP Close Completed
Time: 41.2753 s, Threads: 100, Hosts: 2

Has anyone else run across this? I saw that spine 1.0.0 is available for compilation on github, but in the release notes I see nothing about SNMPv3 so it doesn't seem like that would help us.

[ryanjwh]

Ping. I can't imagine there's nobody using SNMPv3 w/ SHA+AES on 0.8.8f w/ Spine? Does anyone have this working on 0.8.8f? Is this a known issue with Spine?

[ryanjwh]

Pinging this thread again. I'm happy to do any additional legwork to provide more information as needed to troubleshoot this. I've had several folks on our Ops team look at the problem and we're convinced it's an issue in Spine. Before I ask one of our internal developers to look at Spine's source code to see if they can figure it out, I'd love to know if this is a known issue, if there's a known workaround, etc.

Thanks all.

[ryanjwh]

Today, I decided to see if DES worked to confirm whether it was an AES-specific problem. I created a second new SNMPv3 user on my test host, this time with DES as the encryption method. Once again, I can use snmpwalk from bash with SHA and DES specified and get instant good response. I also updated the host in Cacti to use the new username and DES (instead of AES) encryption, and the Cacti web UI is able to query the host properly, show name, uptime, etc.

However, same as before, the poller says the host is down. When I run the poller as root from bash for just that host ID, it just says the host timed out/isn't responding.

Does SNMPv3 w/ SHA and DES *or* AES work at all in Spine?

[ryanjwh]

More data: I manually ran cmd.php from the command line and it works because the web UI then reports the host as recovering/up and I get graphs.

Code: Select all

root@cacti901(cacti)# /usr/bin/php /srv/cacti/cacti/cmd.php 1941 1941

Also of note, I ran tcpdump as follows while trying to poll with spine and don't see a single packet leave the host (where with cmd.php and/or snmpwalk I see all of the encrypted packets), so it doesn't appear Spine is even sending any packets out of the interface:

Code: Select all

root@cacti901(~)# tcpdump -vv -A -T snmp -s 0 "(dst port 162) or (src port 161) or (dst port 161) and (host <ip_address>)"

We can't really use cmd.php in production as we have tens of thousands of data sources and cmd.php doesn't nearly scale enough.

So, it appears directly related to a problem in spine, since polling with cmd.php is fine.

[ryanjwh]

After a bunch of host and datasource cleanup we've switched our poller to cmd.php (away from spine) for now. The average run time for cmd.php is 298 seconds, just under the wire to avoid graph gaps.

I'd really like to be able to go back to using spine (which averages about 200 seconds run time for the same number of datasources), but can't until I can get some assistance troubleshooting why it's broken wrt SNMPv3.

I'm happy to provide any diagnostics requested. Anyone around to assist?

[cigamit]

The average run time for cmd.php is 298 seconds, just under the wire to avoid graph gaps.

If I recall correctly, it is hardcoded to stop at 298, so you may be missing data if it is hitting 298.

[ryanjwh]

Ah, that *did* look fishy! I suspected something was cutting it off at 298 seconds; it seemed too convenient. Does it happen to log what datasources it couldn't get to if it's cut off at 298 seconds?

[ryanjwh]

Indeed, I had to switch back to Spine as I realized about 5% of my graphs were no longer updating, presumably because they were fitting into the 298 second cmd.php poller runs.

[ryanjwh]

Ah! I think I've found a lead!

I tested compiling Spine again on the same system with the same configure options to dig through the compile output, and it looks like Spine is saying it doesn't need encryption support in Net-SNMP? If that's the case I assume that means it's compiling without OpenSSL which would prevent it from being able to use SHA and AES at least, maybe DES too?

Code: Select all

root@spine701(cacti-spine-0.8.8f)# ./configure | grep crypto
checking for CRYPTO_realloc in -lcrypto... yes
checking if Net-SNMP needs crypto support... no

[ryanjwh]

I found this in the spine configure.ac:

Code: Select all

# Net-SNMP includes v3 support and insists on crypto unless compiled --without-openssl
AC_MSG_CHECKING([if Net-SNMP needs crypto support])
AC_TRY_COMPILE([#include <net-snmp-config.h>], [exit(USE_OPENSSL != 1);],
  [  AC_MSG_RESULT(yes)
     SNMP_SSL=yes
  ],
  AC_MSG_RESULT(no)
)

We're just using the stock Net-SNMP on RHEL 6.5, so this tells me that the configure for spine doesn't see that Net-SNMP was compiled with openssl required. However, Net-SNMP must have openssl support because I can use SNMPv3 just fine with snmpwalk and the cmd.php poller. Is there a command line option with ./configure that I can use to force it to include openssl?

Also, FWIW, I did run ./bootstrap and everything checks out.

[pantaley]

Hi,
We have the exact same problem. An year ago we decided to use cacti for one of our projects and during that time we concluded that using spine and snmpV3 is not working. We were hoping that it will get fixed in the next few releases. But it looks like the problem remain. Currently we are using cmp.php, but we are at 260-270 secs, so we are getting very close to the 298 secs limit. We have 507 hosts at the moment.

I will also dig in the next week, because we are intending to have at least another 500 hosts. I will update you if I have any luck.

P.S. I also cannot believe that no one is using cacti + spine + snmpV3.

[ryanjwh]

Thanks for the reply, pantaley. Definitely let me know what you find. At this point I'm pretty sure it's an issue with compiling on RHEL/CentOS 6.x (at least 6.5 in our case) where ./configure is unable to detect crypto support properly. Fixing that is beyond my configure/compile/make knowledge.

[pantaley]

Hi,
I have small progress. It must be something with the server configuration or the build process of spine as you said. On our CentOS the ./configure of spine is trowing(check config.log) a lot of errors about snmp too and it is not working.

So I tested fresh new installation on Debian and cacti works fine with spine and snmpv3.


01/07/2016 09:50:01 PM - SYSTEM STATS: Time:0.2651 Method:spine Processes:1 Threads:1 Hosts:2 HostsPerProcess:2 DataSources:8 RRDsProcessed:8



root@testVM110:/tmp/cacti-spine-0.8.8f# ./spine -H 2 -R -V 5 -S
SPINE: Using spine config file [spine.conf]
01/07/2016 09:43:53 PM - SPINE: Poller[0] DEBUG: The path_php_server variable is /var/www/cacti/script_server.php
01/07/2016 09:43:53 PM - SPINE: Poller[0] DEBUG: The path_cactilog variable is /usr/share/cacti/site/log/cacti.log
DEBUG: The log_destination variable is 4 (STDOUT)
DEBUG: The path_php variable is /usr/bin/php
DEBUG: The availability_method variable is 0
DEBUG: The ping_recovery_count variable is 0
DEBUG: The ping_failure_count variable is 0
DEBUG: The ping_method variable is 0
DEBUG: The ping_retries variable is 0
DEBUG: The ping_timeout variable is 0
DEBUG: The snmp_retries variable is 0
DEBUG: The log_perror variable is 0
DEBUG: The log_pwarn variable is 0
DEBUG: The boost_redirect variable is 0
DEBUG: The log_pstats variable is 0
DEBUG: The threads variable is 5
DEBUG: The polling interval is the system default
DEBUG: The number of concurrent processes is 1
DEBUG: The script timeout is 25
DEBUG: The number of php script servers to run is 2
DEBUG: Host List to be polled='2', TotalPHPScripts='0
DEBUG: The PHP Script Server is Not Required
DEBUG: The Maximum SNMP OID Get Size is 25
Version 0.8.8f starting
DEBUG: MySQL is Thread Safe!
DEBUG: Spine is running asroot.
SPINE: Initializing Net-SNMP API
DEBUG: Issues with SNMP Header Version information, assuming old version of Net-SNMP.
SPINE: Initializing PHP Script Server(s)
NOTE: Spine will support multithread device polling.
DEBUG: Initial Value of Active Threads is 0
DEBUG: Valid Thread to be Created
DEBUG: In Poller, About to Start Polling of Host
DEBUG: The Value of Active Threads is 1
DEBUG: Valid Thread to be Created
Host[0] TH[1] Total Time: 0.0011 Seconds
Host[0] TH[1] DEBUG: HOST COMPLETE: About to Exit Host Polling Thread Function
DEBUG: In Poller, About to Start Polling of Host
DEBUG: The Value of Active Threads is 2
DEBUG: The Value of Active Threads is 1
Host[2] DEBUG: Entering SNMP Ping
Host[2] SNMP Result: Host responded to SNMP
Host[2] TH[1] RECACHE: Processing 3 items in the auto reindex cache for '69.89.0.222
Host[2] TH[1] Recache DataQuery[10] OID: .1.3.6.1.2.1.1.3.0, output: 5853300
Host[2] TH[1] Recache DataQuery[11] OID: .1.3.6.1.2.1.1.3.0, output: 5853300
Host[2] TH[1] Recache DataQuery[12] OID: .1.3.6.1.2.1.1.3.0, output: 5853300
Host[2] TH[1] NOTE: There are '8' Polling Items for this Host
Host[2] TH[1] DS[8] SNMP: v3: 69.89.0.222, dsname: cpu_5, oid: .1.3.6.1.4.1.2021.10.1.3.1, value: 0.00
Host[2] TH[1] DS[9] SNMP: v3: 69.89.0.222, dsname: memory_buffers, oid: .1.3.6.1.4.1.2021.4.14.0, value: 131296
Host[2] TH[1] DS[10] SNMP: v3: 69.89.0.222, dsname: memory_caches, oid: .1.3.6.1.4.1.2021.4.15.0, value: 448708
Host[2] TH[1] DS[11] SNMP: v3: 69.89.0.222, dsname: memory_free, oid: .1.3.6.1.4.1.2021.4.6.0, value: 46992
Host[2] TH[1] DS[12] SNMP: v3: 69.89.0.222, dsname: memory_total, oid: .1.3.6.1.4.1.2021.4.5.0, value: 786432
Host[2] TH[1] DS[13] SNMP: v3: 69.89.0.222, dsname: swap_free, oid: .1.3.6.1.4.1.2021.4.4.0, value: 1048568
Host[2] TH[1] DS[14] SNMP: v3: 69.89.0.222, dsname: swap_total, oid: .1.3.6.1.4.1.2021.4.3.0, value: 1048568
Host[2] TH[1] DS[15] SNMP: v3: 69.89.0.222, dsname: uptime, oid: .1.3.6.1.2.1.25.1.1.0, value: 6467310
Host[2] TH[1] Total Time: 0.062 Seconds
Host[2] TH[1] DEBUG: HOST COMPLETE: About to Exit Host Polling Thread Function
DEBUG: The Value of Active Threads is 0
DEBUG: Thread Cleanup Complete
DEBUG: PHP Script Server Pipes Closed
DEBUG: Allocated Variable Memory Freed
DEBUG: MYSQL Free & Close Completed
DEBUG: Net-SNMP Close Completed
Time: 0.1474 s, Threads: 5, Hosts: 2

I will try on a new CentOS as well on the next day or two.

[ryanjwh]

Yep pantaley, that's exactly what I found and where I got blocked. I also saw the errors in config.log apparently having issues reading the snmp headers file(s).

Note that I think we should be careful to note whether the config.log is complaining about errors with snmp when it's trying to check for crypto support vs when it's doing other net-snmp related actions, because in configure.ac there are other net-snmp related actions just after checking for crypto. It's difficult to differentiate between which action is causing which log lines.