Percona Templates and Local Host Issues - RHEL 7.3 Maipo

UzziahParker · Post by **UzziahParker** » Wed May 30, 2018 8:18 am

Hello,

I am new to Cacti and have been tasked with setting it up for a company I am contracting with. Cacti is proving to be an insatiable beast in terms of getting it to work and I feel incredibly discouraged the longer I work with it.

That said, I was told by a co-worker that a separate site we run has cacti up and running. This site uses percona templates and seems to work just fine, though no one fiddles with anything because after it was setup the original developer for that project left and everyone else feels like Cacti is a massive house of cards just waiting to fall down. I have been trying to get graphs working on the local host, which is also running Nagios if that matters, as it will be the primary Cacti server. I don't believe this is an issue with the Cacti install or Cacti configs as I am receiving SNMP data for my devices and SOME GRAPHS seem to work. I will be more than eager to share whatever I can involving my configs, hosts.allow, snmpd.conf, or whatever else needs to be shared to bring my understanding to a higher level.

I have searched nearly all stack overflow tickets and I've browsed this forum itself quite heavily. I'm not sure if it's because I've been staring at the same thing for 2 or more weeks now, but I legitimately don't believe it is this difficult, which leads me to assume I have to have missed something.

I have logging set to DEBUG already
Here is some info to work with... Cacti Version: 1.1.28 which is located in /usr/share/cacti/
Percona Template Versions: "cacti_host_template_percona_gnu_linux_server_ht_0.8.6i-sver1.1.8" -- I assume version "0.8.6i" or "1.1.8"

uname -a:

Code: Select all

Linux <FQDN> 3.10.0-514.el7.x86_64 #1 SMP Wed Oct 19 11:24:13 EDT 2016 x86_64 x86_64 x86_64 GNU/Linux

cat /etc/*release

Code: Select all

NAME="Red Hat Enterprise Linux Server"
VERSION="7.3 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="7.3"
PRETTY_NAME="Red Hat Enterprise Linux"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.3:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.3
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.3"
Red Hat Enterprise Linux Server release 7.3 (Maipo)
Red Hat Enterprise Linux Server release 7.3 (Maipo)

Below is an example of a percona CPU info graph for the local host, which does not display any data, but RRD says its "OK"...
When I run these commands and pipe it to a file "foo.png" I receive a graph but no data in the graph.

Code: Select all

RRDtool Command:
/bin/rrdtool graph - \
--imgformat=PNG \
--start='1527683760' \
--end='1527685516' \
--pango-markup  \
--title='Local Host (<FQDN>) - CPU Usage' \
--vertical-label='CPU Utilization' \
--slope-mode \
--base=1000 \
--height=120 \
--width=500 \
--units-length '3' \
--tabwidth '30' \
--alt-autoscale-max \
--lower-limit='0' \
COMMENT:"From 2018/05/30 08\:36\:00 To 2018/05/30 09\:05\:16\c" \
COMMENT:"  \n" \
--border 1 --slope-mode \
--watermark 'Generated by Cacti®' \
DEF:a='/usr/share/cacti/rra/local_host_stat_cpu_nice_47.rrd':'STAT_CPU_user':AVERAGE \
DEF:b='/usr/share/cacti/rra/local_host_stat_cpu_nice_47.rrd':'STAT_CPU_nice':AVERAGE \
DEF:c='/usr/share/cacti/rra/local_host_stat_cpu_nice_47.rrd':'STAT_CPU_system':AVERAGE \
DEF:d='/usr/share/cacti/rra/local_host_stat_cpu_nice_47.rrd':'STAT_CPU_idle':AVERAGE \
DEF:e='/usr/share/cacti/rra/local_host_stat_cpu_nice_47.rrd':'STAT_CPU_iowait':AVERAGE \
DEF:f='/usr/share/cacti/rra/local_host_stat_cpu_nice_47.rrd':'STAT_CPU_irq':AVERAGE \
DEF:g='/usr/share/cacti/rra/local_host_stat_cpu_nice_47.rrd':'STAT_CPU_softirq':AVERAGE \
DEF:h='/usr/share/cacti/rra/local_host_stat_cpu_nice_47.rrd':'STAT_CPU_steal':AVERAGE \
DEF:i='/usr/share/cacti/rra/local_host_stat_cpu_nice_47.rrd':'STAT_CPU_guest':AVERAGE \
AREA:a#941342FF:'CPU User   '  \
GPRINT:a:LAST:'Cur\:%6.1lf%s'  \
GPRINT:a:AVERAGE:'Avg\:%6.1lf%s'  \
GPRINT:a:MAX:'Max\:%6.1lf%s\n'  \
AREA:b#435247FF:'CPU Nice   ':STACK \
GPRINT:b:LAST:'Cur\:%6.1lf%s'  \
GPRINT:b:AVERAGE:'Avg\:%6.1lf%s'  \
GPRINT:b:MAX:'Max\:%6.1lf%s\n'  \
AREA:c#00CBF6FF:'CPU System ':STACK \
GPRINT:c:LAST:'Cur\:%6.1lf%s'  \
GPRINT:c:AVERAGE:'Avg\:%6.1lf%s'  \
GPRINT:c:MAX:'Max\:%6.1lf%s\n'  \
AREA:d#BFBD82FF:'CPU Idle   ':STACK \
GPRINT:d:LAST:'Cur\:%6.1lf%s'  \
GPRINT:d:AVERAGE:'Avg\:%6.1lf%s'  \
GPRINT:d:MAX:'Max\:%6.1lf%s\n'  \
AREA:e#C5031AFF:'CPU Iowait ':STACK \
GPRINT:e:LAST:'Cur\:%6.1lf%s'  \
GPRINT:e:AVERAGE:'Avg\:%6.1lf%s'  \
GPRINT:e:MAX:'Max\:%6.1lf%s\n'  \
AREA:f#F39034FF:'CPU Irq    ':STACK \
GPRINT:f:LAST:'Cur\:%6.1lf%s'  \
GPRINT:f:AVERAGE:'Avg\:%6.1lf%s'  \
GPRINT:f:MAX:'Max\:%6.1lf%s\n'  \
AREA:g#3D282AFF:'CPU Softirq':STACK \
GPRINT:g:LAST:'Cur\:%6.1lf%s'  \
GPRINT:g:AVERAGE:'Avg\:%6.1lf%s'  \
GPRINT:g:MAX:'Max\:%6.1lf%s\n'  \
AREA:h#33369EFF:'CPU Steal  ':STACK \
GPRINT:h:LAST:'Cur\:%6.1lf%s'  \
GPRINT:h:AVERAGE:'Avg\:%6.1lf%s'  \
GPRINT:h:MAX:'Max\:%6.1lf%s\n'  \
AREA:i#08A000FF:'CPU Guest  ':STACK \
GPRINT:i:LAST:'Cur\:%6.1lf%s'  \
GPRINT:i:AVERAGE:'Avg\:%6.1lf%s'  \
GPRINT:i:MAX:'Max\:%6.1lf%s\n' 
RRDtool Says:
OK

Below is a percona graph for CPU info on ANOTHER SERVER that my Cacti Server can see and communicate with just fine, this is the SAME graph as above but the only difference is it is not applied to the localhost. The graph displays data, though it seems to have gaps between points when I zoom in which I am not sure if that is intended by Cacti or not.

Code: Select all

RRDtool Command:
/bin/rrdtool graph - \
--imgformat=PNG \
--start='1527683880' \
--end='1527685650' \
--pango-markup  \
--title='<FQDN-Name> - CPU Usage' \
--vertical-label='CPU Utilization' \
--slope-mode \
--base=1000 \
--height=120 \
--width=500 \
--units-length '3' \
--tabwidth '30' \
--alt-autoscale-max \
--lower-limit='0' \
COMMENT:"From 2018/05/30 08\:38\:00 To 2018/05/30 09\:07\:30\c" \
COMMENT:"  \n" \
--border 1 --slope-mode \
--watermark 'Generated by Cacti®' \
DEF:a='/usr/share/cacti/rra/<FQDN-Name>_stat_cpu_nice_46.rrd':'STAT_CPU_user':AVERAGE \
DEF:b='/usr/share/cacti/rra/<FQDN-Name>_stat_cpu_nice_46.rrd':'STAT_CPU_nice':AVERAGE \
DEF:c='/usr/share/cacti/rra/<FQDN-Name>_stat_cpu_nice_46.rrd':'STAT_CPU_system':AVERAGE \
DEF:d='/usr/share/cacti/rra/<FQDN-Name>_stat_cpu_nice_46.rrd':'STAT_CPU_idle':AVERAGE \
DEF:e='/usr/share/cacti/rra/<FQDN-Name>_stat_cpu_nice_46.rrd':'STAT_CPU_iowait':AVERAGE \
DEF:f='/usr/share/cacti/rra/<FQDN-Name>_stat_cpu_nice_46.rrd':'STAT_CPU_irq':AVERAGE \
DEF:g='/usr/share/cacti/rra/<FQDN-Name>_stat_cpu_nice_46.rrd':'STAT_CPU_softirq':AVERAGE \
DEF:h='/usr/share/cacti/rra/<FQDN-Name>_stat_cpu_nice_46.rrd':'STAT_CPU_steal':AVERAGE \
DEF:i='/usr/share/cacti/rra/<FQDN-Name>_stat_cpu_nice_46.rrd':'STAT_CPU_guest':AVERAGE \
AREA:a#941342FF:'CPU User   '  \
GPRINT:a:LAST:'Cur\:%6.1lf%s'  \
GPRINT:a:AVERAGE:'Avg\:%6.1lf%s'  \
GPRINT:a:MAX:'Max\:%6.1lf%s\n'  \
AREA:b#435247FF:'CPU Nice   ':STACK \
GPRINT:b:LAST:'Cur\:%6.1lf%s'  \
GPRINT:b:AVERAGE:'Avg\:%6.1lf%s'  \
GPRINT:b:MAX:'Max\:%6.1lf%s\n'  \
AREA:c#00CBF6FF:'CPU System ':STACK \
GPRINT:c:LAST:'Cur\:%6.1lf%s'  \
GPRINT:c:AVERAGE:'Avg\:%6.1lf%s'  \
GPRINT:c:MAX:'Max\:%6.1lf%s\n'  \
AREA:d#BFBD82FF:'CPU Idle   ':STACK \
GPRINT:d:LAST:'Cur\:%6.1lf%s'  \
GPRINT:d:AVERAGE:'Avg\:%6.1lf%s'  \
GPRINT:d:MAX:'Max\:%6.1lf%s\n'  \
AREA:e#C5031AFF:'CPU Iowait ':STACK \
GPRINT:e:LAST:'Cur\:%6.1lf%s'  \
GPRINT:e:AVERAGE:'Avg\:%6.1lf%s'  \
GPRINT:e:MAX:'Max\:%6.1lf%s\n'  \
AREA:f#F39034FF:'CPU Irq    ':STACK \
GPRINT:f:LAST:'Cur\:%6.1lf%s'  \
GPRINT:f:AVERAGE:'Avg\:%6.1lf%s'  \
GPRINT:f:MAX:'Max\:%6.1lf%s\n'  \
AREA:g#3D282AFF:'CPU Softirq':STACK \
GPRINT:g:LAST:'Cur\:%6.1lf%s'  \
GPRINT:g:AVERAGE:'Avg\:%6.1lf%s'  \
GPRINT:g:MAX:'Max\:%6.1lf%s\n'  \
AREA:h#33369EFF:'CPU Steal  ':STACK \
GPRINT:h:LAST:'Cur\:%6.1lf%s'  \
GPRINT:h:AVERAGE:'Avg\:%6.1lf%s'  \
GPRINT:h:MAX:'Max\:%6.1lf%s\n'  \
AREA:i#08A000FF:'CPU Guest  ':STACK \
GPRINT:i:LAST:'Cur\:%6.1lf%s'  \
GPRINT:i:AVERAGE:'Avg\:%6.1lf%s'  \
GPRINT:i:MAX:'Max\:%6.1lf%s\n' 
RRDtool Says:
OK

I am looking forward to being blessed by whatever knowledge can be dropped on me to relieve me of my ignorance. Please.
Do not hesitate to ask me for more info, I only shared what I thought was relevant based on what I have seen others post due to the problems they have faced.

netniV · Post by **netniV** » Thu May 31, 2018 9:50 am

Sorry to hear about your discouragement. We are always working to try and make sure people do not feel like they are experiencing the "house of cards" effect. As with most complicated systems, when things just work they are great and when they don't it can be troublesome. I mean, Windows is definitely a house of cards with the number of things that can easily stop a system from booting, and Linux has it's own problems. Both platforms left me banging my head recently over simple things but then people far more knowlegable than me have been through it before.

To answer your particular questions, have you use the Debug Graph options? There are some which say what data has been populated into the RRD files and you can use the 'rrdtool' command directly (if you know what you are doing).

Do you have the STATS lines from the poller which show how many RRD's, Devices and Graphs have been polled?

If you have set the debugging logging level, you may find that there is so much information you get lost in it. There is an option to debug a single device and another for a single page which would reduce the overall information.

Are you sure that the poller has been running ?

UzziahParker · Post by **UzziahParker** » Thu May 31, 2018 10:22 am

Thanks for responding,

1)I have used the graph debug options to get the snippets of code I uploaded that shows the RRD tool a "Ok". If there is more to this than I know, please let me know so I can post it.

2) I am not sure what you mean by STATS lines. Below is what I could find by running a simple grep command on the log file...Should there be something else I can look for?

Code: Select all

sudo grep -runH --color STATS /usr/share/cacti/log/cacti.log

/usr/share/cacti/log/cacti.log:168599:30/05/2018 20:32:04 - SYSTEM DSSTATS STATS: Type:HOURLY, Time:0.0048 
/usr/share/cacti/log/cacti.log:169020:30/05/2018 20:33:05 - SYSTEM STATS: Time:3.5721 Method:cmd.php Processes:1 Threads:N/A Hosts:2 HostsPerProcess:2 DataSources:39 RRDsProcessed:39
/usr/share/cacti/log/cacti.log:169108:30/05/2018 20:33:05 - SYSTEM DSSTATS STATS: Type:HOURLY, Time:0.0049 
/usr/share/cacti/log/cacti.log:169529:30/05/2018 20:34:04 - SYSTEM STATS: Time:3.4606 Method:cmd.php Processes:1 Threads:N/A Hosts:2 HostsPerProcess:2 DataSources:39 RRDsProcessed:39
/usr/share/cacti/log/cacti.log:169621:30/05/2018 20:34:05 - SYSTEM DSSTATS STATS: Type:HOURLY, Time:0.0055 
/usr/share/cacti/log/cacti.log:170069:30/05/2018 20:35:05 - SYSTEM STATS: Time:3.4818 Method:cmd.php Processes:1 Threads:N/A Hosts:2 HostsPerProcess:2 DataSources:39 RRDsProcessed:39
/usr/share/cacti/log/cacti.log:170157:30/05/2018 20:35:05 - SYSTEM DSSTATS STATS: Type:HOURLY, Time:0.0050 
/usr/share/cacti/log/cacti.log:170578:30/05/2018 20:36:05 - SYSTEM STATS: Time:3.5144 Method:cmd.php Processes:1 Threads:N/A Hosts:2 HostsPerProcess:2 DataSources:39 RRDsProcessed:39

3) As far as there being to much information to look at, I'm not concerned. I hope it doesn't sound cocky but I'd rather there be a ton of data in a log file to look through just encase something shows up that is valuable. I can always grep and parse my way to exactly what I need.

4) I am confident that the poller is running...

Code: Select all

sudo grep -runH --color POLLER /usr/share/cacti/log/cacti.log

/usr/share/cacti/log/cacti.log:29:30/05/2018 15:05:01 - POLLER: Poller[1] NOTE: Poller Int: '60', Cron Int: '300', Time Since Last: '300.84', Max Runtime '298', Poller Runs: '5'
/usr/share/cacti/log/cacti.log:40:30/05/2018 15:05:01 - POLLER: Poller[1] DEBUG: About to Spawn a Remote Process [CMD: /bin/php, ARGS: -q "/usr/share/cacti/cmd.php" --poller=1 --first=0 --last=2 --mibs]
/usr/share/cacti/log/cacti.log:50:30/05/2018 15:05:01 - POLLER: Poller[1] DEBUG: About to Spawn a Remote Process [CMD: /bin/php, ARGS: -q "/usr/share/cacti/cmd.php" --poller=1 --first=3 --last=3 --mibs]
/usr/share/cacti/log/cacti.log:93:30/05/2018 15:05:02 - POLLER: Poller[1] Device[0] DS[48] WARNING: Result from CMD not valid.  Partial Result: U
/usr/share/cacti/log/cacti.log:94:30/05/2018 15:05:02 - POLLER: Poller[1] Device[0] DS[48] CMD: /bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host '' --type proc_stat --items gw,gx,gy,gz,hg,hh,hi,hj,hk, output: U
/usr/share/cacti/log/cacti.log:100:30/05/2018 15:05:02 - POLLER: Poller[1] Device[2] STATUS: Device '<IP Address>' is UP.
/usr/share/cacti/log/cacti.log:113:30/05/2018 15:05:02 - POLLER: Poller[1] Device[3] STATUS: Device 'localhost' is UP.
/usr/share/cacti/log/cacti.log:126:30/05/2018 15:05:02 - POLLER: Poller[1] Device[3] DS[47] CMD: /bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host 'localhost' --type proc_stat --items gw,gx,gy,gz,hg,hh,hi,hj,hk, output: gw:-1 gx:-1 gy:-1 gz:-1 hg:-1 hh:-1 hi:-1 hj:-1 hk:-1
/usr/share/cacti/log/cacti.log:127:30/05/2018 15:05:02 - POLLER: Poller[1] Device[2] DS[46] CMD: /bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host '<IP Address>' --type proc_stat --items gw,gx,gy,gz,hg,hh,hi,hj,hk, output: gw:12077117 gx:442901 gy:5230428 gz:807172682 hg:3259504 hh:0 hi:129715 hj:0 hk:0
/usr/share/cacti/log/cacti.log:128:30/05/2018 15:05:02 - POLLER: Poller[1] Device[2] DS[49] CMD: /bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host '<IP Address>' --type proc_stat --items gw,gx,gy,gz,hg,hh,hi,hj,hk, output: gw:12077117 gx:442901 gy:5230428 gz:807172682 hg:3259504 hh:0 hi:129715 hj:0 hk:0
/usr/share/cacti/log/cacti.log:129:30/05/2018 15:05:02 - POLLER: Poller[1] Device[3] DS[64] CMD: /bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host 'localhost' --type apache --items gh, output: gh:-1
/usr/share/cacti/log/cacti.log:130:30/05/2018 15:05:02 - POLLER: Poller[1] Device[2] DS[50] CMD: /bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host '<IP Address>' --type diskstats --items jk --device '', output: jk:-1
/usr/share/cacti/log/cacti.log:131:30/05/2018 15:05:02 - POLLER: Poller[1] Device[3] DS[65] CMD: /bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host 'localhost' --type apache --items gk, output: gk:-1
/usr/share/cacti/log/cacti.log:132:30/05/2018 15:05:02 - POLLER: Poller[1] Device[2] DS[51] CMD: /bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host '<IP Address>' --type df --items nj,nk --volume '', output: nj:-1 nk:-1
/usr/share/cacti/log/cacti.log:133:30/05/2018 15:05:02 - POLLER: Poller[1] Device[3] DS[66] CMD: /bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host 'localhost' --type apache --items gg, output: gg:-1
/usr/share/cacti/log/cacti.log:134:30/05/2018 15:05:02 - POLLER: Poller[1] Device[3] DS[67] CMD: /bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host 'localhost' --type apache --items gl,gm,gn,go,gp,gq,gr,gs,gt,gu,gv, output: gl:0 gm:0 gn:0 go:0 gp:0 gq:0 gr:0 gs:0 gt:0 gu:0 gv:0
/usr/share/cacti/log/cacti.log:135:30/05/2018 15:05:02 - POLLER: Poller[1] Device[2] DS[52] CMD: /bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host '<IP Address>' --type netdev --items nl,nq --device '|host_description| - Apache CPU Load', output: nl:-1 nq:-1
/usr/share/cacti/log/cacti.log:136:30/05/2018 15:05:02 - POLLER: Poller[1] Device[3] DS[68] CMD: /bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host 'localhost' --type apache --items gi,gj, output: gi:-1 gj:-1
/usr/share/cacti/log/cacti.log:137:30/05/2018 15:05:02 - POLLER: Poller[1] Device[2] DS[53] CMD: /bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host '<IP Address>' --type apache --items gk, output: gk:-1
/usr/share/cacti/log/cacti.log:138:30/05/2018 15:05:02 - POLLER: Poller[1] Device[3] DS[69] CMD: /bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host 'localhost' --type proc_stat --items hm, output: hm:-1
/usr/share/cacti/log/cacti.log:139:30/05/2018 15:05:02 - POLLER: Poller[1] Device[2] DS[54] CMD: /bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host '<IP Address>' --type proc_stat --items hm, output: hm:4322308161
/usr/share/cacti/log/cacti.log:140:30/05/2018 15:05:02 - POLLER: Poller[1] Device[3] DS[70] CMD: /bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host 'localhost' --type diskstats --items jl,jm --device '', output: jl:-1 jm:-1
/usr/share/cacti/log/cacti.log:141:30/05/2018 15:05:02 - POLLER: Poller[1] Device[2] DS[55] CMD: /bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host '<IP Address>' --type diskstats --items jl,jm --device '', output: jl:-1 jm:-1
/usr/share/cacti/log/cacti.log:142:30/05/2018 15:05:02 - POLLER: Poller[1] Device[3] DS[71] CMD: /bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host 'localhost' --type diskstats --items jk --device '', output: jk:-1
/usr/share/cacti/log/cacti.log:143:30/05/2018 15:05:02 - POLLER: Poller[1] Device[2] DS[56] CMD: /bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host '<IP Address>' --type diskstats --items iw,ix,jg,jh --device '', output: iw:-1 ix:-1 jg:-1 jh:-1
/usr/share/cacti/log/cacti.log:144:30/05/2018 15:05:02 - POLLER: Poller[1] Device[2] DS[57] CMD: /bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host '<IP Address>' --type diskstats --items iz,jj --device '', output: iz:-1 jj:-1
/usr/share/cacti/log/cacti.log:145:30/05/2018 15:05:02 - POLLER: Poller[1] Device[3] DS[72] CMD: /bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host 'localhost' --type diskstats --items iw,ix,jg,jh --device '', output: iw:-1 ix:-1 jg:-1 jh:-1
/usr/share/cacti/log/cacti.log:146:30/05/2018 15:05:03 - POLLER: Poller[1] Device[2] DS[59] CMD: /bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host '<IP Address>' --type proc_stat --items hl, output: hl:3098915586
/usr/share/cacti/log/cacti.log:147:30/05/2018 15:05:03 - POLLER: Poller[1] Device[3] DS[73] CMD: /bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host 'localhost' --type diskstats --items iz,jj --device '', output: iz:-1 jj:-1
/usr/share/cacti/log/cacti.log:148:30/05/2018 15:05:03 - POLLER: Poller[1] Device[3] DS[74] CMD: /bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host 'localhost' --type diskstats --items iw,iz,jg,jj --device '', output: iw:-1 iz:-1 jg:-1 jj:-1
/usr/share/cacti/log/cacti.log:151:30/05/2018 15:05:03 - POLLER: Poller[1] Device[3] DS[75] CMD: /bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host 'localhost' --type diskstats --items iy,ji --device '', output: iy:-1 ji:-1
/usr/share/cacti/log/cacti.log:152:30/05/2018 15:05:03 - POLLER: Poller[1] Device[3] DS[76] CMD: /bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host 'localhost' --type df --items nj,nk --volume '', output: nj:-1 nk:-1

netniV · Post by **netniV** » Fri Jun 01, 2018 5:35 am

Cocky eh? Not a problem as long as you know what you are doing (unlike me

)

So looking over the quick logs you posted, it would seem to be using the ss_get_by_ssh script. One thing to be wary of when using that, the SSH key (if you are using one) should be under the poller user's home directory. If not, that may explain the -1 values that are being returned by every call (at least it seems that way to me).

As for graph debugging, the part I was referring is the cog icon next to a graph when you are viewing it.

UzziahParker · Post by **UzziahParker** » Fri Jun 01, 2018 6:46 am

No no I'm sorry, I didn't mean to come across as arragent or cocky. I just to my limited understanding, 100% sure the poller was in fact running. I numbered my responses to try and keep it organized to the suggestions you gave me in the order you gave me them is all.

1) Hmmm I am not positive about if the SSH Key which I believe had to get setup at some point is in the users home dir. I believe I have the poller set to be owned by root, which might be a problem? Should it be related to cacti:cacti instead or should the perms on the poller change at all?

Code: Select all

cat /etc/passwd | grep --color cacti
        cacti:x:169:169::/usr/share/cacti:/sbin/nologin

ls -la /usr/share/cacti/ | grep -unH poll
        (standard input):64:-rw-r--r--    1 root   root    29028 Nov 19  2017 poller_automation.php
        (standard input):65:-rw-r--r--    1 root   root    14739 Nov 19  2017 poller_boost.php
        (standard input):66:-rw-r--r--    1 root   root     5528 Nov 19  2017 poller_commands.php
        (standard input):67:-rw-r--r--    1 root   root     8589 Nov 19  2017 poller_dsstats.php
        (standard input):68:-rw-r--r--    1 root   root     3641 Nov 19  2017 poller_export.php
        (standard input):69:-rw-r--r--    1 root   root    15513 Nov 19  2017 poller_maintenance.php
**    (standard input):70:-rwxr-xr-x    1 root   root    24658 Nov 19  2017 poller.php
        (standard input):71:-rw-r--r--    1 root   root    10051 Nov 19  2017 poller_realtime.php
        (standard input):72:-rw-r--r--    1 root   root     9008 Nov 19  2017 poller_recovery.php
        (standard input):73:-rw-r--r--    1 root   root     5644 Nov 19  2017 poller_reports.php
        (standard input):74:-rw-r--r--    1 root   root    25439 Nov 19  2017 pollers.php
        (standard input):75:-rw-r--r--    1 root   root     7160 Nov 19  2017 poller_spikekill.php
        (standard input):76:-rwxr--r--    1 root   root      983 May 24 08:06 rebuild_poller_cache.php

When it comes to the SSH key I know that I on my cacti box I made a ".ssh" dir under "/usr/share/cacti/" and then in that dir I did "ssh-keygen blah blah" which produced a id_rsa.pub file. I took that file's contents, which is my public key for the cacti box, and put it in the "/home/cacti/.ssh/authorized_keys" file on my agent box(s).

2) Please see the attached image in response to the graph debug, which shows nothing I think.

I am thankful to be discussing this with you though.

netniV · Post by **netniV** » Fri Jun 01, 2018 10:07 am

I still believe that this is an SSH key issue. What flavour of Linux are you running? Also, is your poller user using the cacti user? If so, I would imagine the home directory for the cacti u see is actually /home/cacti and therefore the .ssh directory should be under there.

If you want this to also work live from the website using a realtime test, you would also want the same key under the home directory of the web server user (apache or www-data)

UzziahParker · Post by **UzziahParker** » Fri Jun 01, 2018 2:42 pm

I am using RHEL 7.3 Maipo.

Sorry for the late response, work got busy. I have the cacti user on my agent that is being polled. My primary server which is where cacti is installed keeps the ssh info in /usr/share/cacti/.ssh/id_rsa.

On the agent its in /home/cacti/.ssh/authorized_keys.

This code runs fine from my cacti server, it ssh's into the machine just fine and in the /var/log/audit/audit.log file it shows me connecting fine. However then it will randomly start spitting SSH USER_AUTH errors from the same IP address as my <agent FQDN>.

Code: Select all

sudo ssh -i /usr/share/cacti/.ssh/id_rsa cacti@<agent FQDN>



<below is the audit.log from the agent box. Shows me connecting fine, but then throwing more errors for some reason even though I am connected?>
type=CRYPTO_KEY_USER 'op=destroy kind=server <filler> direction=? spid=63703 suid=0  exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=CRYPTO_KEY_USER 'op=destroy kind=server <filler> direction=? spid=63703 suid=0  exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=CRYPTO_KEY_USER 'op=destroy kind=server <filler> direction=? spid=63703 suid=0  exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=CRYPTO_KEY_USER 'op=destroy kind=server <filler> direction=? spid=63703 suid=0  exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=CRYPTO_SESSION 'op=start direction=from-server cipher=aes128-ctr ksize=128 mac=umac-64-etm@openssh.com pfs=curve25519-sha256 spid=63703 suid=74 rport=26751 laddr=<agent IP> lport=22  exe="/usr/sbin/sshd" hostname=? addr=<Cacti Server> terminal=? res=success'
type=CRYPTO_SESSION 'op=start direction=from-client cipher=aes128-ctr ksize=128 mac=umac-64-etm@openssh.com pfs=curve25519-sha256 spid=63703 suid=74 rport=26751 laddr=<agent IP> lport=22  exe="/usr/sbin/sshd" hostname=? addr=<Cacti Server> terminal=? res=success'
type=USER_AUTH 'op=pubkey_auth rport=26751 acct="cacti" exe="/usr/sbin/sshd" hostname=? addr=<Cacti Server> terminal=? res=success'
type=USER_AUTH 'op=key algo=ssh-rsa size=4096 <filler> rport=26751 acct="cacti" exe="/usr/sbin/sshd" hostname=? addr=<Cacti Server> terminal=? res=success'
type=USER_ACCT 'op=PAM:accounting grantors=pam_unix,pam_localuser acct="cacti" exe="/usr/sbin/sshd" hostname=<Cacti Server> addr=<Cacti Server> terminal=ssh res=success'
type=CRYPTO_KEY_USER 'op=destroy kind=session fp=? direction=both spid=63703 suid=74 rport=26751 laddr=<agent IP> lport=22  exe="/usr/sbin/sshd" hostname=? addr=<Cacti Server> terminal=? res=success'
type=USER_AUTH 'op=success acct="cacti" exe="/usr/sbin/sshd" hostname=? addr=<Cacti Server> terminal=ssh res=success'
type=CRED_ACQ 'op=PAM:setcred grantors=pam_env,pam_unix,pam_duo acct="cacti" exe="/usr/sbin/sshd" hostname=<Cacti Server> addr=<Cacti Server> terminal=ssh res=success'
type=USER_START 'op=PAM:session_open grantors=pam_selinux,pam_loginuid,pam_selinux,pam_namespace,pam_keyinit,pam_keyinit,pam_limits,pam_systemd,pam_unix,pam_ldap,pam_lastlog acct="cacti" exe="/usr/sbin/sshd" hostname=<Cacti Server> addr=<Cacti Server> terminal=ssh res=success'
type=CRYPTO_KEY_USER 'op=destroy kind=server <filler> direction=? spid=63704 suid=0  exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=CRYPTO_KEY_USER 'op=destroy kind=server <filler> direction=? spid=63704 suid=0  exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=CRYPTO_KEY_USER 'op=destroy kind=server <filler> direction=? spid=63704 suid=0  exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=CRYPTO_KEY_USER 'op=destroy kind=server <filler> direction=? spid=63704 suid=0  exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=CRED_ACQ 'op=PAM:setcred grantors=pam_env,pam_unix,pam_duo acct="cacti" exe="/usr/sbin/sshd" hostname=<Cacti Server> addr=<Cacti Server> terminal=ssh res=success'
type=USER_LOGIN 'op=login id=42908 exe="/usr/sbin/sshd" hostname=<Cacti Server> addr=<Cacti Server> terminal=/dev/pts/1 res=success'
type=USER_START 'op=login id=42908 exe="/usr/sbin/sshd" hostname=<Cacti Server> addr=<Cacti Server> terminal=/dev/pts/1 res=success'
type=CRYPTO_KEY_USER 'op=destroy kind=server <filler> direction=? spid=63705 suid=42908  exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'


type=CRYPTO_KEY_USER 'op=destroy kind=server <filler> direction=? spid=63779 suid=0  exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=CRYPTO_KEY_USER 'op=destroy kind=server <filler> direction=? spid=63779 suid=0  exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=CRYPTO_KEY_USER 'op=destroy kind=server <filler> direction=? spid=63779 suid=0  exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=CRYPTO_KEY_USER 'op=destroy kind=server <filler> direction=? spid=63779 suid=0  exe="/usr/sbin/sshd" hostname=? addr=? terminal=? res=success'
type=CRYPTO_SESSION 'op=start direction=from-server cipher=aes128-ctr ksize=128 mac=umac-64-etm@openssh.com pfs=curve25519-sha256 spid=63779 suid=74 rport=27159 laddr=<agent IP> lport=22  exe="/usr/sbin/sshd" hostname=? addr=<Cacti Server> terminal=? res=success'
type=CRYPTO_SESSION 'op=start direction=from-client cipher=aes128-ctr ksize=128 mac=umac-64-etm@openssh.com pfs=curve25519-sha256 spid=63779 suid=74 rport=27159 laddr=<agent IP> lport=22  exe="/usr/sbin/sshd" hostname=? addr=<Cacti Server> terminal=? res=success'
type=USER_AUTH 'op=PAM:authentication grantors=? acct="cacti" exe="/usr/sbin/sshd" hostname=<Cacti Server> addr=<Cacti Server> terminal=ssh res=failed'
type=USER_AUTH 'op=challenge-response acct="cacti" exe="/usr/sbin/sshd" hostname=? addr=<Cacti Server> terminal=ssh res=failed'
type=USER_AUTH 'op=PAM:authentication grantors=? acct="cacti" exe="/usr/sbin/sshd" hostname=<Cacti Server> addr=<Cacti Server> terminal=ssh res=failed'
type=USER_AUTH 'op=challenge-response acct="cacti" exe="/usr/sbin/sshd" hostname=? addr=<Cacti Server> terminal=ssh res=failed'
type=USER_AUTH 'op=PAM:authentication grantors=? acct="cacti" exe="/usr/sbin/sshd" hostname=<Cacti Server> addr=<Cacti Server> terminal=ssh res=failed'
type=USER_AUTH 'op=challenge-response acct="cacti" exe="/usr/sbin/sshd" hostname=? addr=<Cacti Server> terminal=ssh res=failed'
type=USER_AUTH 'op=none acct="cacti" exe="/usr/sbin/sshd" hostname=? addr=<Cacti Server> terminal=ssh res=failed'
type=USER_AUTH 'op=maxtries exceeded acct="cacti" exe="/usr/sbin/sshd" hostname=? addr=<Cacti Server> terminal=ssh res=failed'
type=CRYPTO_KEY_USER 'op=destroy kind=session fp=? direction=both spid=63779 suid=74 rport=27159 laddr=<agent IP> lport=22  exe="/usr/sbin/sshd" hostname=? addr=<Cacti Server> terminal=? res=success'

netniV · Post by **netniV** » Sat Jun 02, 2018 11:58 am

OK, now that test works as root. Is your poller running as root? If not, the test in invalidated. Use the sudo's -u parameter to specify the actual user the poller runs as. On Debian, for example, this would be specified in the /etc/cron.d/cacti file that cron uses to spawn the poller. You are looking for the first word after the timing columns (just in case you weren't familiar with it).

UzziahParker · Post by **UzziahParker** » Mon Jun 04, 2018 6:17 am

I am sorry I didn't respond over the weekend. Hope you had a good time yourself though,

I believe the poller is running as the cacti user, which is what I thought was supposed to be the case?

Code: Select all

sudo ls -la /etc/cron.d/
-rw-------    1 root root   76 Mar 19 14:54 cacti


sudo vim /etc/cron.d/cacti
*/5 * * * *     cacti   /usr/bin/php /usr/share/cacti/poller.php > /dev/null 2>&1

netniV · Post by **netniV** » Mon Jun 04, 2018 6:50 am

Yeah looks right. And as you said you had the file in the .ssh folder. Have you tried doing the sudo -u ?

UzziahParker · Post by **UzziahParker** » Mon Jun 04, 2018 7:13 am

As in adding "sudo -u" before "cacti" in the cron.d/cacti file?

UzziahParker · Post by **UzziahParker** » Mon Jun 04, 2018 9:08 am

I added "sudo -u" before the user portion in the cron.d/cacti file. I have not noticed any changes in behavior.

I did a quick search on the most recent errors and warnings on the cacti log file, you can see below. Is there any other info I can get for you to help in this troubleshooting?

Code: Select all

06/04/2018 09:52:06 AM
<cacti server>$ sudo tail -n 5000 /var/log/cacti/cacti.log | grep -iunH --color error

49:04/06/2018 09:25:54 - POLLER: Poller[1] WARNING: Invalid Response(s), Errors[1] Device[2] Thread[1] DS[48]
246:04/06/2018 09:25:54 - DBCALL DEVEL: SQL Assoc: "SELECT id, description, hostname, status,  disabled, status_event_count, status_fail_date, status_rec_date,  status_last_error, min_time, max_time, cur_time, avg_time,  total_polls, failed_polls, availability, snmp_engine_id  FROM host  ORDER BY id ASC"
248:04/06/2018 09:25:54 - DBCALL DEVEL: SQL Exec: "INSERT INTO `snmpagent_cache`       (name, value, oid)       VALUES ('cactiApplDeviceStatus', '3', '.1.3.6.1.4.1.23925.1.14.1.4.2'), ('cactiApplDeviceEventCount', '0', '.1.3.6.1.4.1.23925.1.14.1.5.2'), ('cactiApplDeviceFailDate', '2018-05-23 13:10:02', '.1.3.6.1.4.1.23925.1.14.1.6.2'), ('cactiApplDeviceRecoveryDate', '2018-05-23 14:45:01', '.1.3.6.1.4.1.23925.1.14.1.7.2'), ('cactiApplDeviceLastError', 'Device did not respond to SNMP, UDP Ping Success (0.06 ms)', '.1.3.6.1.4.1.23925.1.14.1.8.2')      ON DUPLICATE KEY UPDATE value=VALUES(value)"
252:04/06/2018 09:25:54 - DBCALL DEVEL: SQL Exec: "INSERT INTO `snmpagent_cache`       (name, value, oid)       VALUES ('cactiApplDeviceStatus', '3', '.1.3.6.1.4.1.23925.1.14.1.4.3'), ('cactiApplDeviceEventCount', '0', '.1.3.6.1.4.1.23925.1.14.1.5.3'), ('cactiApplDeviceFailDate', '2018-05-29 15:04:02', '.1.3.6.1.4.1.23925.1.14.1.6.3'), ('cactiApplDeviceRecoveryDate', '2018-05-29 15:09:01', '.1.3.6.1.4.1.23925.1.14.1.7.3'), ('cactiApplDeviceLastError', 'Device did not respond to SNMP, UDP Ping Success (0.07 ms)', '.1.3.6.1.4.1.23925.1.14.1.8.3')      ON DUPLICATE KEY UPDATE value=VALUES(value)"
334:04/06/2018 09:26:01 - DBCALL DEVEL: SQL Exec: "UPDATE host SET  status = ?,  status_event_count = ?,  status_fail_date = ?,  status_rec_date = ?,  status_last_error = ?,  min_time = ?,  max_time = ?,  cur_time = ?,  avg_time = ?,  total_polls = ?,  failed_polls = ?,  availability = ?  WHERE hostname = ?"
350:04/06/2018 09:26:01 - DBCALL DEVEL: SQL Exec: "UPDATE host SET  status = ?,  status_event_count = ?,  status_fail_date = ?,  status_rec_date = ?,  status_last_error = ?,  min_time = ?,  max_time = ?,  cur_time = ?,  avg_time = ?,  total_polls = ?,  failed_polls = ?,  availability = ?  WHERE hostname = ?"
395:04/06/2018 09:26:03 - POLLER: Poller[1] WARNING: Invalid Response(s), Errors[1] Device[2] Thread[1] DS[48]
748:04/06/2018 09:26:04 - DBCALL DEVEL: SQL Assoc: "SELECT id, description, hostname, status,  disabled, status_event_count, status_fail_date, status_rec_date,  status_last_error, min_time, max_time, cur_time, avg_time,  total_polls, failed_polls, availability, snmp_engine_id  FROM host  ORDER BY id ASC"
750:04/06/2018 09:26:04 - DBCALL DEVEL: SQL Exec: "INSERT INTO `snmpagent_cache`       (name, value, oid)       VALUES ('cactiApplDeviceStatus', '3', '.1.3.6.1.4.1.23925.1.14.1.4.2'), ('cactiApplDeviceEventCount', '0', '.1.3.6.1.4.1.23925.1.14.1.5.2'), ('cactiApplDeviceFailDate', '2018-05-23 13:10:02', '.1.3.6.1.4.1.23925.1.14.1.6.2'), ('cactiApplDeviceRecoveryDate', '2018-05-23 14:45:01', '.1.3.6.1.4.1.23925.1.14.1.7.2'), ('cactiApplDeviceLastError', 'Device did not respond to SNMP, UDP Ping Success (0.06 ms)', '.1.3.6.1.4.1.23925.1.14.1.8.2')      ON DUPLICATE KEY UPDATE value=VALUES(value)"
754:04/06/2018 09:26:04 - DBCALL DEVEL: SQL Exec: "INSERT INTO `snmpagent_cache`       (name, value, oid)       VALUES ('cactiApplDeviceStatus', '3', '.1.3.6.1.4.1.23925.1.14.1.4.3'), ('cactiApplDeviceEventCount', '0', '.1.3.6.1.4.1.23925.1.14.1.5.3'), ('cactiApplDeviceFailDate', '2018-05-29 15:04:02', '.1.3.6.1.4.1.23925.1.14.1.6.3'), ('cactiApplDeviceRecoveryDate', '2018-05-29 15:09:01', '.1.3.6.1.4.1.23925.1.14.1.7.3'), ('cactiApplDeviceLastError', 'Device did not respond to SNMP, UDP Ping Success (0.07 ms)', '.1.3.6.1.4.1.23925.1.14.1.8.3')      ON DUPLICATE KEY UPDATE value=VALUES(value)"
843:04/06/2018 09:27:01 - DBCALL DEVEL: SQL Exec: "UPDATE host SET  status = ?,  status_event_count = ?,  status_fail_date = ?,  status_rec_date = ?,  status_last_error = ?,  min_time = ?,  max_time = ?,  cur_time = ?,  avg_time = ?,  total_polls = ?,  failed_polls = ?,  availability = ?  WHERE hostname = ?"
848:04/06/2018 09:27:01 - DBCALL DEVEL: SQL Exec: "UPDATE host SET  status = ?,  status_event_count = ?,  status_fail_date = ?,  status_rec_date = ?,  status_last_error = ?,  min_time = ?,  max_time = ?,  cur_time = ?,  avg_time = ?,  total_polls = ?,  failed_polls = ?,  availability = ?  WHERE hostname = ?"
894:04/06/2018 09:27:02 - POLLER: Poller[1] WARNING: Invalid Response(s), Errors[1] Device[2] Thread[1] DS[48]
1257:04/06/2018 09:27:05 - DBCALL DEVEL: SQL Assoc: "SELECT id, description, hostname, status,  disabled, status_event_count, status_fail_date, status_rec_date,  status_last_error, min_time, max_time, cur_time, avg_time,  total_polls, failed_polls, availability, snmp_engine_id  FROM host  ORDER BY id ASC"
1259:04/06/2018 09:27:05 - DBCALL DEVEL: SQL Exec: "INSERT INTO `snmpagent_cache`       (name, value, oid)       VALUES ('cactiApplDeviceStatus', '3', '.1.3.6.1.4.1.23925.1.14.1.4.2'), ('cactiApplDeviceEventCount', '0', '.1.3.6.1.4.1.23925.1.14.1.5.2'), ('cactiApplDeviceFailDate', '2018-05-23 13:10:02', '.1.3.6.1.4.1.23925.1.14.1.6.2'), ('cactiApplDeviceRecoveryDate', '2018-05-23 14:45:01', '.1.3.6.1.4.1.23925.1.14.1.7.2'), ('cactiApplDeviceLastError', 'Device did not respond to SNMP, UDP Ping Success (0.06 ms)', '.1.3.6.1.4.1.23925.1.14.1.8.2')      ON DUPLICATE KEY UPDATE value=VALUES(value)"
1263:04/06/2018 09:27:05 - DBCALL DEVEL: SQL Exec: "INSERT INTO `snmpagent_cache`       (name, value, oid)       VALUES ('cactiApplDeviceStatus', '3', '.1.3.6.1.4.1.23925.1.14.1.4.3'), ('cactiApplDeviceEventCount', '0', '.1.3.6.1.4.1.23925.1.14.1.5.3'), ('cactiApplDeviceFailDate', '2018-05-29 15:04:02', '.1.3.6.1.4.1.23925.1.14.1.6.3'), ('cactiApplDeviceRecoveryDate', '2018-05-29 15:09:01', '.1.3.6.1.4.1.23925.1.14.1.7.3'), ('cactiApplDeviceLastError', 'Device did not respond to SNMP, UDP Ping Success (0.07 ms)', '.1.3.6.1.4.1.23925.1.14.1.8.3')      ON DUPLICATE KEY UPDATE value=VALUES(value)"
1351:04/06/2018 09:28:01 - DBCALL DEVEL: SQL Exec: "UPDATE host SET  status = ?,  status_event_count = ?,  status_fail_date = ?,  status_rec_date = ?,  status_last_error = ?,  min_time = ?,  max_time = ?,  cur_time = ?,  avg_time = ?,  total_polls = ?,  failed_polls = ?,  availability = ?  WHERE hostname = ?"
1358:04/06/2018 09:28:01 - DBCALL DEVEL: SQL Exec: "UPDATE host SET  status = ?,  status_event_count = ?,  status_fail_date = ?,  status_rec_date = ?,  status_last_error = ?,  min_time = ?,  max_time = ?,  cur_time = ?,  avg_time = ?,  total_polls = ?,  failed_polls = ?,  availability = ?  WHERE hostname = ?"
1688:04/06/2018 09:29:03 - POLLER: Poller[1] WARNING: Invalid Response(s), Errors[1] Device[2] Thread[1] DS[48]
1899:04/06/2018 09:29:04 - DBCALL DEVEL: SQL Exec: "UPDATE host SET  status = ?,  status_event_count = ?,  status_fail_date = ?,  status_rec_date = ?,  status_last_error = ?,  min_time = ?,  max_time = ?,  cur_time = ?,  avg_time = ?,  total_polls = ?,  failed_polls = ?,  availability = ?  WHERE hostname = ?"
1920:04/06/2018 09:29:04 - DBCALL DEVEL: SQL Exec: "UPDATE host SET  status = ?,  status_event_count = ?,  status_fail_date = ?,  status_rec_date = ?,  status_last_error = ?,  min_time = ?,  max_time = ?,  cur_time = ?,  avg_time = ?,  total_polls = ?,  failed_polls = ?,  availability = ?  WHERE hostname = ?"
1962:04/06/2018 09:29:05 - POLLER: Poller[1] WARNING: Invalid Response(s), Errors[1] Device[2] Thread[1] DS[48]
2315:04/06/2018 09:29:06 - DBCALL DEVEL: SQL Assoc: "SELECT id, description, hostname, status,  disabled, status_event_count, status_fail_date, status_rec_date,  status_last_error, min_time, max_time, cur_time, avg_time,  total_polls, failed_polls, availability, snmp_engine_id  FROM host  ORDER BY id ASC"
2317:04/06/2018 09:29:06 - DBCALL DEVEL: SQL Exec: "INSERT INTO `snmpagent_cache`       (name, value, oid)       VALUES ('cactiApplDeviceStatus', '3', '.1.3.6.1.4.1.23925.1.14.1.4.2'), ('cactiApplDeviceEventCount', '0', '.1.3.6.1.4.1.23925.1.14.1.5.2'), ('cactiApplDeviceFailDate', '2018-05-23 13:10:02', '.1.3.6.1.4.1.23925.1.14.1.6.2'), ('cactiApplDeviceRecoveryDate', '2018-05-23 14:45:01', '.1.3.6.1.4.1.23925.1.14.1.7.2'), ('cactiApplDeviceLastError', 'Device did not respond to SNMP, UDP Ping Success (0.06 ms)', '.1.3.6.1.4.1.23925.1.14.1.8.2')      ON DUPLICATE KEY UPDATE value=VALUES(value)"
2321:04/06/2018 09:29:06 - DBCALL DEVEL: SQL Exec: "INSERT INTO `snmpagent_cache`       (name, value, oid)       VALUES ('cactiApplDeviceStatus', '3', '.1.3.6.1.4.1.23925.1.14.1.4.3'), ('cactiApplDeviceEventCount', '0', '.1.3.6.1.4.1.23925.1.14.1.5.3'), ('cactiApplDeviceFailDate', '2018-05-29 15:04:02', '.1.3.6.1.4.1.23925.1.14.1.6.3'), ('cactiApplDeviceRecoveryDate', '2018-05-29 15:09:01', '.1.3.6.1.4.1.23925.1.14.1.7.3'), ('cactiApplDeviceLastError', 'Device did not respond to SNMP, UDP Ping Success (0.07 ms)', '.1.3.6.1.4.1.23925.1.14.1.8.3')      ON DUPLICATE KEY UPDATE value=VALUES(value)"
2432:04/06/2018 09:30:02 - DBCALL DEVEL: SQL Exec: "UPDATE host SET  status = ?,  status_event_count = ?,  status_fail_date = ?,  status_rec_date = ?,  status_last_error = ?,  min_time = ?,  max_time = ?,  cur_time = ?,  avg_time = ?,  total_polls = ?,  failed_polls = ?,  availability = ?  WHERE hostname = ?"
2450:04/06/2018 09:30:02 - DBCALL DEVEL: SQL Exec: "UPDATE host SET  status = ?,  status_event_count = ?,  status_fail_date = ?,  status_rec_date = ?,  status_last_error = ?,  min_time = ?,  max_time = ?,  cur_time = ?,  avg_time = ?,  total_polls = ?,  failed_polls = ?,  availability = ?  WHERE hostname = ?"
2492:04/06/2018 09:30:03 - POLLER: Poller[1] WARNING: Invalid Response(s), Errors[1] Device[2] Thread[1] DS[48]
2855:04/06/2018 09:30:05 - DBCALL DEVEL: SQL Assoc: "SELECT id, description, hostname, status,  disabled, status_event_count, status_fail_date, status_rec_date,  status_last_error, min_time, max_time, cur_time, avg_time,  total_polls, failed_polls, availability, snmp_engine_id  FROM host  ORDER BY id ASC"
2857:04/06/2018 09:30:05 - DBCALL DEVEL: SQL Exec: "INSERT INTO `snmpagent_cache`       (name, value, oid)       VALUES ('cactiApplDeviceStatus', '3', '.1.3.6.1.4.1.23925.1.14.1.4.2'), ('cactiApplDeviceEventCount', '0', '.1.3.6.1.4.1.23925.1.14.1.5.2'), ('cactiApplDeviceFailDate', '2018-05-23 13:10:02', '.1.3.6.1.4.1.23925.1.14.1.6.2'), ('cactiApplDeviceRecoveryDate', '2018-05-23 14:45:01', '.1.3.6.1.4.1.23925.1.14.1.7.2'), ('cactiApplDeviceLastError', 'Device did not respond to SNMP, UDP Ping Success (0.06 ms)', '.1.3.6.1.4.1.23925.1.14.1.8.2')      ON DUPLICATE KEY UPDATE value=VALUES(value)"
2861:04/06/2018 09:30:05 - DBCALL DEVEL: SQL Exec: "INSERT INTO `snmpagent_cache`       (name, value, oid)       VALUES ('cactiApplDeviceStatus', '3', '.1.3.6.1.4.1.23925.1.14.1.4.3'), ('cactiApplDeviceEventCount', '0', '.1.3.6.1.4.1.23925.1.14.1.5.3'), ('cactiApplDeviceFailDate', '2018-05-29 15:04:02', '.1.3.6.1.4.1.23925.1.14.1.6.3'), ('cactiApplDeviceRecoveryDate', '2018-05-29 15:09:01', '.1.3.6.1.4.1.23925.1.14.1.7.3'), ('cactiApplDeviceLastError', 'Device did not respond to SNMP, UDP Ping Success (0.07 ms)', '.1.3.6.1.4.1.23925.1.14.1.8.3')      ON DUPLICATE KEY UPDATE value=VALUES(value)"
2931:04/06/2018 09:31:02 - DBCALL DEVEL: SQL Exec: "UPDATE host SET  status = ?,  status_event_count = ?,  status_fail_date = ?,  status_rec_date = ?,  status_last_error = ?,  min_time = ?,  max_time = ?,  cur_time = ?,  avg_time = ?,  total_polls = ?,  failed_polls = ?,  availability = ?  WHERE hostname = ?"
2962:04/06/2018 09:31:02 - DBCALL DEVEL: SQL Exec: "UPDATE host SET  status = ?,  status_event_count = ?,  status_fail_date = ?,  status_rec_date = ?,  status_last_error = ?,  min_time = ?,  max_time = ?,  cur_time = ?,  avg_time = ?,  total_polls = ?,  failed_polls = ?,  availability = ?  WHERE hostname = ?"
3261:04/06/2018 09:31:51 - POLLER: Poller[1] WARNING: Invalid Response(s), Errors[1] Device[2] Thread[1] DS[48]
3458:04/06/2018 09:31:52 - DBCALL DEVEL: SQL Assoc: "SELECT id, description, hostname, status,  disabled, status_event_count, status_fail_date, status_rec_date,  status_last_error, min_time, max_time, cur_time, avg_time,  total_polls, failed_polls, availability, snmp_engine_id  FROM host  ORDER BY id ASC"
3460:04/06/2018 09:31:52 - DBCALL DEVEL: SQL Exec: "INSERT INTO `snmpagent_cache`       (name, value, oid)       VALUES ('cactiApplDeviceStatus', '3', '.1.3.6.1.4.1.23925.1.14.1.4.2'), ('cactiApplDeviceEventCount', '0', '.1.3.6.1.4.1.23925.1.14.1.5.2'), ('cactiApplDeviceFailDate', '2018-05-23 13:10:02', '.1.3.6.1.4.1.23925.1.14.1.6.2'), ('cactiApplDeviceRecoveryDate', '2018-05-23 14:45:01', '.1.3.6.1.4.1.23925.1.14.1.7.2'), ('cactiApplDeviceLastError', 'Device did not respond to SNMP, UDP Ping Success (0.06 ms)', '.1.3.6.1.4.1.23925.1.14.1.8.2')      ON DUPLICATE KEY UPDATE value=VALUES(value)"
3464:04/06/2018 09:31:52 - DBCALL DEVEL: SQL Exec: "INSERT INTO `snmpagent_cache`       (name, value, oid)       VALUES ('cactiApplDeviceStatus', '3', '.1.3.6.1.4.1.23925.1.14.1.4.3'), ('cactiApplDeviceEventCount', '0', '.1.3.6.1.4.1.23925.1.14.1.5.3'), ('cactiApplDeviceFailDate', '2018-05-29 15:04:02', '.1.3.6.1.4.1.23925.1.14.1.6.3'), ('cactiApplDeviceRecoveryDate', '2018-05-29 15:09:01', '.1.3.6.1.4.1.23925.1.14.1.7.3'), ('cactiApplDeviceLastError', 'Device did not respond to SNMP, UDP Ping Success (0.07 ms)', '.1.3.6.1.4.1.23925.1.14.1.8.3')      ON DUPLICATE KEY UPDATE value=VALUES(value)"
3546:04/06/2018 09:32:01 - DBCALL DEVEL: SQL Exec: "UPDATE host SET  status = ?,  status_event_count = ?,  status_fail_date = ?,  status_rec_date = ?,  status_last_error = ?,  min_time = ?,  max_time = ?,  cur_time = ?,  avg_time = ?,  total_polls = ?,  failed_polls = ?,  availability = ?  WHERE hostname = ?"
3562:04/06/2018 09:32:01 - DBCALL DEVEL: SQL Exec: "UPDATE host SET  status = ?,  status_event_count = ?,  status_fail_date = ?,  status_rec_date = ?,  status_last_error = ?,  min_time = ?,  max_time = ?,  cur_time = ?,  avg_time = ?,  total_polls = ?,  failed_polls = ?,  availability = ?  WHERE hostname = ?"
3607:04/06/2018 09:32:03 - POLLER: Poller[1] WARNING: Invalid Response(s), Errors[1] Device[2] Thread[1] DS[48]
3960:04/06/2018 09:32:04 - DBCALL DEVEL: SQL Assoc: "SELECT id, description, hostname, status,  disabled, status_event_count, status_fail_date, status_rec_date,  status_last_error, min_time, max_time, cur_time, avg_time,  total_polls, failed_polls, availability, snmp_engine_id  FROM host  ORDER BY id ASC"
3962:04/06/2018 09:32:04 - DBCALL DEVEL: SQL Exec: "INSERT INTO `snmpagent_cache`       (name, value, oid)       VALUES ('cactiApplDeviceStatus', '3', '.1.3.6.1.4.1.23925.1.14.1.4.2'), ('cactiApplDeviceEventCount', '0', '.1.3.6.1.4.1.23925.1.14.1.5.2'), ('cactiApplDeviceFailDate', '2018-05-23 13:10:02', '.1.3.6.1.4.1.23925.1.14.1.6.2'), ('cactiApplDeviceRecoveryDate', '2018-05-23 14:45:01', '.1.3.6.1.4.1.23925.1.14.1.7.2'), ('cactiApplDeviceLastError', 'Device did not respond to SNMP, UDP Ping Success (0.06 ms)', '.1.3.6.1.4.1.23925.1.14.1.8.2')      ON DUPLICATE KEY UPDATE value=VALUES(value)"
3966:04/06/2018 09:32:04 - DBCALL DEVEL: SQL Exec: "INSERT INTO `snmpagent_cache`       (name, value, oid)       VALUES ('cactiApplDeviceStatus', '3', '.1.3.6.1.4.1.23925.1.14.1.4.3'), ('cactiApplDeviceEventCount', '0', '.1.3.6.1.4.1.23925.1.14.1.5.3'), ('cactiApplDeviceFailDate', '2018-05-29 15:04:02', '.1.3.6.1.4.1.23925.1.14.1.6.3'), ('cactiApplDeviceRecoveryDate', '2018-05-29 15:09:01', '.1.3.6.1.4.1.23925.1.14.1.7.3'), ('cactiApplDeviceLastError', 'Device did not respond to SNMP, UDP Ping Success (0.07 ms)', '.1.3.6.1.4.1.23925.1.14.1.8.3')      ON DUPLICATE KEY UPDATE value=VALUES(value)"
4048:04/06/2018 09:33:01 - DBCALL DEVEL: SQL Exec: "UPDATE host SET  status = ?,  status_event_count = ?,  status_fail_date = ?,  status_rec_date = ?,  status_last_error = ?,  min_time = ?,  max_time = ?,  cur_time = ?,  avg_time = ?,  total_polls = ?,  failed_polls = ?,  availability = ?  WHERE hostname = ?"
4063:04/06/2018 09:33:01 - DBCALL DEVEL: SQL Exec: "UPDATE host SET  status = ?,  status_event_count = ?,  status_fail_date = ?,  status_rec_date = ?,  status_last_error = ?,  min_time = ?,  max_time = ?,  cur_time = ?,  avg_time = ?,  total_polls = ?,  failed_polls = ?,  availability = ?  WHERE hostname = ?"
4107:04/06/2018 09:33:03 - POLLER: Poller[1] WARNING: Invalid Response(s), Errors[1] Device[2] Thread[1] DS[48]
4469:04/06/2018 09:33:05 - DBCALL DEVEL: SQL Assoc: "SELECT id, description, hostname, status,  disabled, status_event_count, status_fail_date, status_rec_date,  status_last_error, min_time, max_time, cur_time, avg_time,  total_polls, failed_polls, availability, snmp_engine_id  FROM host  ORDER BY id ASC"
4471:04/06/2018 09:33:05 - DBCALL DEVEL: SQL Exec: "INSERT INTO `snmpagent_cache`       (name, value, oid)       VALUES ('cactiApplDeviceStatus', '3', '.1.3.6.1.4.1.23925.1.14.1.4.2'), ('cactiApplDeviceEventCount', '0', '.1.3.6.1.4.1.23925.1.14.1.5.2'), ('cactiApplDeviceFailDate', '2018-05-23 13:10:02', '.1.3.6.1.4.1.23925.1.14.1.6.2'), ('cactiApplDeviceRecoveryDate', '2018-05-23 14:45:01', '.1.3.6.1.4.1.23925.1.14.1.7.2'), ('cactiApplDeviceLastError', 'Device did not respond to SNMP, UDP Ping Success (0.06 ms)', '.1.3.6.1.4.1.23925.1.14.1.8.2')      ON DUPLICATE KEY UPDATE value=VALUES(value)"
4475:04/06/2018 09:33:05 - DBCALL DEVEL: SQL Exec: "INSERT INTO `snmpagent_cache`       (name, value, oid)       VALUES ('cactiApplDeviceStatus', '3', '.1.3.6.1.4.1.23925.1.14.1.4.3'), ('cactiApplDeviceEventCount', '0', '.1.3.6.1.4.1.23925.1.14.1.5.3'), ('cactiApplDeviceFailDate', '2018-05-29 15:04:02', '.1.3.6.1.4.1.23925.1.14.1.6.3'), ('cactiApplDeviceRecoveryDate', '2018-05-29 15:09:01', '.1.3.6.1.4.1.23925.1.14.1.7.3'), ('cactiApplDeviceLastError', 'Device did not respond to SNMP, UDP Ping Success (0.07 ms)', '.1.3.6.1.4.1.23925.1.14.1.8.3')      ON DUPLICATE KEY UPDATE value=VALUES(value)"
4555:04/06/2018 09:34:01 - DBCALL DEVEL: SQL Exec: "UPDATE host SET  status = ?,  status_event_count = ?,  status_fail_date = ?,  status_rec_date = ?,  status_last_error = ?,  min_time = ?,  max_time = ?,  cur_time = ?,  avg_time = ?,  total_polls = ?,  failed_polls = ?,  availability = ?  WHERE hostname = ?"
4575:04/06/2018 09:34:01 - DBCALL DEVEL: SQL Exec: "UPDATE host SET  status = ?,  status_event_count = ?,  status_fail_date = ?,  status_rec_date = ?,  status_last_error = ?,  min_time = ?,  max_time = ?,  cur_time = ?,  avg_time = ?,  total_polls = ?,  failed_polls = ?,  availability = ?  WHERE hostname = ?"
4970:04/06/2018 09:35:00 - DBCALL DEVEL: SQL Assoc: "SELECT id, description, hostname, status,  disabled, status_event_count, status_fail_date, status_rec_date,  status_last_error, min_time, max_time, cur_time, avg_time,  total_polls, failed_polls, availability, snmp_engine_id  FROM host  ORDER BY id ASC"
4972:04/06/2018 09:35:00 - DBCALL DEVEL: SQL Exec: "INSERT INTO `snmpagent_cache`       (name, value, oid)       VALUES ('cactiApplDeviceStatus', '3', '.1.3.6.1.4.1.23925.1.14.1.4.2'), ('cactiApplDeviceEventCount', '0', '.1.3.6.1.4.1.23925.1.14.1.5.2'), ('cactiApplDeviceFailDate', '2018-05-23 13:10:02', '.1.3.6.1.4.1.23925.1.14.1.6.2'), ('cactiApplDeviceRecoveryDate', '2018-05-23 14:45:01', '.1.3.6.1.4.1.23925.1.14.1.7.2'), ('cactiApplDeviceLastError', 'Device did not respond to SNMP, UDP Ping Success (0.06 ms)', '.1.3.6.1.4.1.23925.1.14.1.8.2')      ON DUPLICATE KEY UPDATE value=VALUES(value)"
4976:04/06/2018 09:35:00 - DBCALL DEVEL: SQL Exec: "INSERT INTO `snmpagent_cache`       (name, value, oid)       VALUES ('cactiApplDeviceStatus', '3', '.1.3.6.1.4.1.23925.1.14.1.4.3'), ('cactiApplDeviceEventCount', '0', '.1.3.6.1.4.1.23925.1.14.1.5.3'), ('cactiApplDeviceFailDate', '2018-05-29 15:04:02', '.1.3.6.1.4.1.23925.1.14.1.6.3'), ('cactiApplDeviceRecoveryDate', '2018-05-29 15:09:01', '.1.3.6.1.4.1.23925.1.14.1.7.3'), ('cactiApplDeviceLastError', 'Device did not respond to SNMP, UDP Ping Success (0.07 ms)', '.1.3.6.1.4.1.23925.1.14.1.8.3')      ON DUPLICATE KEY UPDATE value=VALUES(value)"
4997:04/06/2018 09:35:03 - POLLER: Poller[1] WARNING: Invalid Response(s), Errors[1] Device[2] Thread[1] DS[48]

<cacti server>$ sudo tail -n 5000 /var/log/cacti/cacti.log | grep -iunH --color warn

49:04/06/2018 09:25:54 - POLLER: Poller[1] WARNING: Invalid Response(s), Errors[1] Device[2] Thread[1] DS[48]
328:04/06/2018 09:26:01 - POLLER: Poller[1] Device[0] DS[48] WARNING: Result from CMD not valid.  Partial Result: U
395:04/06/2018 09:26:03 - POLLER: Poller[1] WARNING: Invalid Response(s), Errors[1] Device[2] Thread[1] DS[48]
833:04/06/2018 09:27:01 - POLLER: Poller[1] Device[0] DS[48] WARNING: Result from CMD not valid.  Partial Result: U
894:04/06/2018 09:27:02 - POLLER: Poller[1] WARNING: Invalid Response(s), Errors[1] Device[2] Thread[1] DS[48]
1349:04/06/2018 09:28:01 - POLLER: Poller[1] Device[0] DS[48] WARNING: Result from CMD not valid.  Partial Result: U
1687:04/06/2018 09:29:03 - CMDPHP WARNING: cmd.php poller over ran its polling intervale and therefore ending
1688:04/06/2018 09:29:03 - POLLER: Poller[1] WARNING: Invalid Response(s), Errors[1] Device[2] Thread[1] DS[48]
1891:04/06/2018 09:29:04 - POLLER: Poller[1] Device[0] DS[48] WARNING: Result from CMD not valid.  Partial Result: U
1962:04/06/2018 09:29:05 - POLLER: Poller[1] WARNING: Invalid Response(s), Errors[1] Device[2] Thread[1] DS[48]
2428:04/06/2018 09:30:02 - POLLER: Poller[1] Device[0] DS[48] WARNING: Result from CMD not valid.  Partial Result: U
2492:04/06/2018 09:30:03 - POLLER: Poller[1] WARNING: Invalid Response(s), Errors[1] Device[2] Thread[1] DS[48]
2922:04/06/2018 09:31:02 - POLLER: Poller[1] Device[0] DS[48] WARNING: Result from CMD not valid.  Partial Result: U
3261:04/06/2018 09:31:51 - POLLER: Poller[1] WARNING: Invalid Response(s), Errors[1] Device[2] Thread[1] DS[48]
3540:04/06/2018 09:32:01 - POLLER: Poller[1] Device[0] DS[48] WARNING: Result from CMD not valid.  Partial Result: U
3607:04/06/2018 09:32:03 - POLLER: Poller[1] WARNING: Invalid Response(s), Errors[1] Device[2] Thread[1] DS[48]
4042:04/06/2018 09:33:01 - POLLER: Poller[1] Device[0] DS[48] WARNING: Result from CMD not valid.  Partial Result: U
4107:04/06/2018 09:33:03 - POLLER: Poller[1] WARNING: Invalid Response(s), Errors[1] Device[2] Thread[1] DS[48]
4549:04/06/2018 09:34:01 - POLLER: Poller[1] Device[0] DS[48] WARNING: Result from CMD not valid.  Partial Result: U
4996:04/06/2018 09:35:03 - CMDPHP WARNING: cmd.php poller over ran its polling intervale and therefore ending
4997:04/06/2018 09:35:03 - POLLER: Poller[1] WARNING: Invalid Response(s), Errors[1] Device[2] Thread[1] DS[48]

netniV · Post by **netniV** » Tue Jun 05, 2018 4:46 am

The sudo -u was for you to run the SSH script as the poller user at the command line. To test as the cacti user, run the following command (you will need to be root to not specify the password of the cacti user) replacing <hostname> with the correct name:

sudo -u cacti /bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host '<hostname>' --type proc_stat --items gw,gx,gy,gz,hg,hh,hi,hj,hk,

Your cron file should look as you had it original (as I read that you had changed that):

*/5 * * * * cacti /usr/bin/php /usr/share/cacti/poller.php > /dev/null 2>&1

Having more thoroughly reviewed your logs, I have the following questions:

I noticed that the first entry doesn't even have a hostname which is invalid so is that because you removed it?
Have you setup the localhost correctly for SNMP? When checking localhost, the script is returning data, but it's always returning -1 (notice the output line).
Have you checked the Percona github site to see if others have a similar issue?
Have you any SNMP security in place that would prevent unregistered systems querying the servers?

The good news is, you have cacti installed and the poller is running. The bad news is, it's not cacti itself but the third party script that isn't returning data. ss_get_by_ssh.php must be installed by the Percona templates (or you as part of that) since I don't have that on any of my cacti installations but then I don't have the percona ones either.

It may mean that you don't have the required SNMP or some additional modules that Percona script requires. It can also be a security issue if you have locked down the servers that are allowed to query. Percona do have a GitHub page so it may be worth a quick search on there to see if someone has had a similar issue.

netniV · Post by **netniV** » Tue Jun 05, 2018 4:51 am

Additionally, this is the only thing I've stumbled which may also have an impact:
https://www.percona.com/doc/percona-mon ... lates.html

It is possible the poller user doesn't have access to the stats file.

UzziahParker · Post by **UzziahParker** » Tue Jun 05, 2018 6:25 am

I tried to run that command but no matter how I tried to run it, there always was a password prompt, to which if I ignored and never entered anything it would eventually just spit out all the -1's.

Code: Select all

[root@<FQDN>]$ sudo -u cacti /bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host '<target FQDN>' --type proc_stat --items gw,gx,gy,gz,hg,hh,hi,hj,hk,
Password: 
gw:-1 gx:-1 gy:-1 gz:-1 hg:-1 hh:-1 hi:-1 hj:-1 hk:-1

1)I noticed that the first entry doesn't even have a hostname which is invalid so is that because you removed it?
I have removed all hostnames for ambiguity, normally replacing them with <FQDN> or something of the likes.

2)Have you setup the localhost correctly for SNMP? When checking localhost, the script is returning data, but it's always returning -1 (notice the output line).
Yes I believe I have.

Code: Select all

[cc@<cacti server FQDN>]$ sudo snmpwalk -Os -v 2c -c <community password> localhost system
sysDescr.0 = STRING: Linux <FQDN> 3.10.0-693.11.6.el7.x86_64 #1 SMP Thu Dec 28 14:23:39 EST 2017 x86_64
sysObjectID.0 = OID: netSnmpAgentOIDs.10
sysUpTimeInstance = Timeticks: (31773564) 3 days, 16:15:35.64
sysContact.0 = STRING: Admin <Admin@example> (/etc/snmp/snmpd.conf)
sysName.0 = STRING: <FQDN>
sysLocation.0 = STRING: PDC (/etc/snmp/snmpd.conf)
sysORLastChange.0 = Timeticks: (5) 0:00:00.05
sysORID.1 = OID: snmpMPDCompliance
sysORID.2 = OID: usmMIBCompliance
sysORID.3 = OID: snmpFrameworkMIBCompliance
sysORID.4 = OID: snmpMIB
sysORID.5 = OID: tcpMIB
sysORID.6 = OID: ip
sysORID.7 = OID: udpMIB
sysORID.8 = OID: vacmBasicGroup
sysORID.9 = OID: snmpNotifyFullCompliance
sysORID.10 = OID: notificationLogMIB
sysORDescr.1 = STRING: The MIB for Message Processing and Dispatching.
sysORDescr.2 = STRING: The management information definitions for the SNMP User-based Security Model.
sysORDescr.3 = STRING: The SNMP Management Architecture MIB.
sysORDescr.4 = STRING: The MIB module for SNMPv2 entities
sysORDescr.5 = STRING: The MIB module for managing TCP implementations
sysORDescr.6 = STRING: The MIB module for managing IP and ICMP implementations
sysORDescr.7 = STRING: The MIB module for managing UDP implementations
sysORDescr.8 = STRING: View-based Access Control Model for SNMP.
sysORDescr.9 = STRING: The MIB modules for managing SNMP Notification, plus filtering.
sysORDescr.10 = STRING: The MIB module for logging SNMP Notifications.
sysORUpTime.1 = Timeticks: (5) 0:00:00.05
sysORUpTime.2 = Timeticks: (5) 0:00:00.05
sysORUpTime.3 = Timeticks: (5) 0:00:00.05
sysORUpTime.4 = Timeticks: (5) 0:00:00.05
sysORUpTime.5 = Timeticks: (5) 0:00:00.05
sysORUpTime.6 = Timeticks: (5) 0:00:00.05
sysORUpTime.7 = Timeticks: (5) 0:00:00.05
sysORUpTime.8 = Timeticks: (5) 0:00:00.05
sysORUpTime.9 = Timeticks: (5) 0:00:00.05
sysORUpTime.10 = Timeticks: (5) 0:00:00.05

3) Have you checked the Percona github site to see if others have a similar issue?
From the googling I've done and asking around my company which even has a similar cacti set up for another cluster of servers using Percona, I've not been able to diagnose what seems to be causing the issues I am having. I'll be having a group meeting with the team that is using Percona and Cacti later today to get their perspectives on this issue outside of playing email-tag.

4) Have you any SNMP security in place that would prevent unregistered systems querying the servers?
The only thing that comes to mind is the community string setup using version 2 of snmp, however I have only ever used 1 community string and it accepts it without issue. Should I enter anything else, it fails out as it should (see below).

Code: Select all

[cc@<cacti server FQDN>]$ sudo snmpwalk -Os -v 2c -c aorihvnauirgheih2094tugjiwg localhost system
Timeout: No Response from localhost

Outside of anything related to the cacti install I know we have a rather strict firewall setup but my network team has clarified that there shouldn't be anything blocking this.

I'll look into the possibility that the poller user can't access the stats file. Wanted to get my response to your larger post up for you to look at, or others, sooner rather than later.

Cacti

Percona Templates and Local Host Issues - RHEL 7.3 Maipo

Percona Templates and Local Host Issues - RHEL 7.3 Maipo

Re: Percona Templates and Local Host Issues - RHEL 7.3 Maipo

Re: Percona Templates and Local Host Issues - RHEL 7.3 Maipo

Re: Percona Templates and Local Host Issues - RHEL 7.3 Maipo

Re: Percona Templates and Local Host Issues - RHEL 7.3 Maipo

Re: Percona Templates and Local Host Issues - RHEL 7.3 Maipo

Re: Percona Templates and Local Host Issues - RHEL 7.3 Maipo

Re: Percona Templates and Local Host Issues - RHEL 7.3 Maipo

Re: Percona Templates and Local Host Issues - RHEL 7.3 Maipo

Re: Percona Templates and Local Host Issues - RHEL 7.3 Maipo

Re: Percona Templates and Local Host Issues - RHEL 7.3 Maipo

Re: Percona Templates and Local Host Issues - RHEL 7.3 Maipo

Re: Percona Templates and Local Host Issues - RHEL 7.3 Maipo

Re: Percona Templates and Local Host Issues - RHEL 7.3 Maipo

Re: Percona Templates and Local Host Issues - RHEL 7.3 Maipo

Who is online