[CentOS 7][Cacti 1.1.38] Problem with device status

[pberger]

You are hiding the pieces that I'm thinking are relevant. At one time I found that if there was an apostrophe in the snmp location that the devices status would never change. Can you remove any special characters from snmp location and contact just to see if that resolves your issue?

Only thing hidden is the customer name and the location of 1 device.. . doubt you would find that relevant in any way...

Have actually upgraded from 1.1.38 to 1.2.2. No errors during upgrading, same issue though.
Locations are mostly normal. Good amount of them isnt even filled in so doubt they can affect the outcome.
Looked at them, but most of them are just normal with only alphabetical characters.

(to be fully honest.. snmp location is just a textfield it loads, no relation to device status regardless)

[pberger]

Fully removed my cacti database, remade a cacti database and imported the default empty cacti.sql. Then created a few device.. issue still is there.
Checked the php and apache logs more but could not find a clue still

[netniV]

Put the device into debug mode, see if that shows you anything more.

[pberger]

Put the device into debug mode, see if that shows you anything more.

Thanks for the recommendation. It did not tell me much but i did think of something due to that.
I thought.. what if it does not detect any changes and cant write anyhing down because it cant alter or read a certain file.
So i decided i was going through all the cacti-related files and looking at their permissions.
Well.. everything has read/write permissions to t for the cactiuser except for 1 file which had only read rights on it.
Spine.conf ...
Now.. Spine.conf only held read permissions forr the cacti user. For a second i thought.. "ow, theres only settings in there so surely he doesnt need write ones", but then i hesitated so i changed it to read/write to make sure.

5 minutes later. Devices change status correctly.

Now there are still 2 issues left:
- It only updates its status every 5 minutes instead of the 1 minute shown in the settings
- Graphs arent being filled yet

It is however a good first step

[netniV]

Can you open a bug for that on the Spine repo? It should not need READ/WRITE but only READ for the configuration file.

[pberger]

Will do.

Atm struggling getting the graphs to update their data to.. example of debug snippet. RRDtool says "OK" but seeing as the data not only says it updated more than a day ago but also simply doesnt get any data at all. I doubt that

Code: Select all

15/03/2019 13:39:30 - WEBLOG CACTI2RRD: /usr/bin/rrdtool graph - --imgformat=PNG --start='1552567140' --end='1552653499' --pango-markup --title='Bassastaal-Meraki-MX64-RM - CPU Usage' --vertical-label='percent' --slope-mode --base=1000 --height=120 --width=500 --tabwidth '30' --rigid --alt-autoscale-max --lower-limit='0' COMMENT:"From 14/03/2019 13\:39\:00 To 15/03/2019 13\:38\:19\c" COMMENT:" \n" --color BACK#F3F3F3 --color CANVAS#FDFDFD --color SHADEA#CBCBCB --color SHADEB#999999 --color FONT#000000 --color AXIS#2C4D43 --color ARROW#2C4D43 --color FRAME#2C4D43 --border 1 --font TITLE:11:'Arial' --font AXIS:8:'Arial' --font LEGEND:8:'Courier' --font UNIT:8:'Arial' --font WATERMARK:6:'Arial' --slope-mode --watermark 'Generated by Cacti®' DEF:a='/usr/share/cacti/rra/bassastaal-887_5min_cpu_4321.rrd' DS[Bassastaal-Meraki-MX64-RM - 5 Minute CPU] Graphs[Bassastaal-Meraki-MX64-RM - CPU Usage]:'5min_cpu':AVERAGE AREA:a#FF0000FF:'CPU Usage' GPRINT:a:LAST:'Current\:%8.0lf' GPRINT:a:AVERAGE:'Average\:%8.0lf' GPRINT:a:MAX:'Maximum\:%8.0lf\n' COMMENT:' \n' COMMENT:'Last Updated \: Thu 14 Mar 08\:25\:15 CET 2019' 

Code: Select all

RRDtool opdracht:

/usr/bin/rrdtool graph - \
--imgformat=PNG \
--start='1552567380' \
--end='1552653760' \
--pango-markup  \
--title='Bassastaal-Meraki-MX64-RM - CPU Usage' \
--vertical-label='percent' \
--slope-mode \
--base=1000 \
--height=120 \
--width=500 \
--tabwidth '30' \
--rigid \
--alt-autoscale-max \
--lower-limit='0' \
COMMENT:"From 14/03/2019 13\:43\:00 To 15/03/2019 13\:42\:40\c" \
COMMENT:"  \n" \
--color BACK#F3F3F3 \
--color CANVAS#FDFDFD \
--color SHADEA#CBCBCB \
--color SHADEB#999999 \
--color FONT#000000 \
--color AXIS#2C4D43 \
--color ARROW#2C4D43 \
--color FRAME#2C4D43 \
--border 1 --font TITLE:11:'Arial' \
--font AXIS:8:'Arial' \
--font LEGEND:8:'Courier' \
--font UNIT:8:'Arial' \
--font WATERMARK:6:'Arial' \
--slope-mode \
--watermark 'Generated by Cacti®' \
DEF:a='/usr/share/cacti/rra/bassastaal-887_5min_cpu_4321.rrd':'5min_cpu':AVERAGE \
AREA:a#FF0000FF:'CPU Usage'  \
GPRINT:a:LAST:'Current\:%8.0lf'  \
GPRINT:a:AVERAGE:'Average\:%8.0lf'  \
GPRINT:a:MAX:'Maximum\:%8.0lf\n'  \
COMMENT:' \n'  \
COMMENT:'Last Updated \: Thu 14 Mar 08\:25\:15 CET 2019' 

RRDtool zegt:
OK

[camerabob]

Make sure you choose the correct option. Or as opposed to AND, or SNMP only/PING only...

device.png (3.13 KiB) Viewed 801 times

[pberger]

Make sure you choose the correct option. Or as opposed to AND, or SNMP only/PING only...

device.png

Yep. Made sure it is all on OR.
Do have a fully working Cacti 0.8.8 server from which i can just check the monitor settings.


Did some more investigation. Cleared my RRA folder fully, cacti simply remade the rrd files in there so cacti does have enough permissions to do that kind of stuff.
Wondering wether the rrdtool might be the issue, but that one seems to installed correctly as far as i can see (newest one available for Centos repo .. the 1.4.8 something)

[pberger]

SNMP walk on the cacti server to one of the routers does retrieve me the data that cacti can retrieve.
So basically..
we have graphs/rrds created by cacti, meaning it can reach that.
it retrieves the info (shows it at device info within cacti)
What it doesnt do is put that data in the rrd's

example of when i do rrdtool info on one of my RRD's

Code: Select all

rrd_version = "0003"
step = 60
last_update = 1552917311
header_size = 3704
ds[5min_cpu].index = 0
ds[5min_cpu].type = "GAUGE"
ds[5min_cpu].minimal_heartbeat = 120
ds[5min_cpu].min = 0.0000000000e+00
ds[5min_cpu].max = 1.0000000000e+02
ds[5min_cpu].last_ds = "5"
ds[5min_cpu].value = NaN
ds[5min_cpu].unknown_sec = 11
rra[0].cf = "AVERAGE"
rra[0].rows = 600
rra[0].cur_row = 195
rra[0].pdp_per_row = 1
rra[0].xff = 5.0000000000e-01
rra[0].cdp_prep[0].value = NaN
rra[0].cdp_prep[0].unknown_datapoints = 0
rra[1].cf = "AVERAGE"
rra[1].rows = 700
rra[1].cur_row = 613
rra[1].pdp_per_row = 6
rra[1].xff = 5.0000000000e-01
rra[1].cdp_prep[0].value = 0.0000000000e+00
rra[1].cdp_prep[0].unknown_datapoints = 1
rra[2].cf = "AVERAGE"
rra[2].rows = 775
rra[2].cur_row = 114
rra[2].pdp_per_row = 24
rra[2].xff = 5.0000000000e-01
rra[2].cdp_prep[0].value = 0.0000000000e+00
rra[2].cdp_prep[0].unknown_datapoints = 19
rra[3].cf = "AVERAGE"
rra[3].rows = 797
rra[3].cur_row = 668
rra[3].pdp_per_row = 288
rra[3].xff = 5.0000000000e-01
rra[3].cdp_prep[0].value = NaN
rra[3].cdp_prep[0].unknown_datapoints = 259
rra[4].cf = "MIN"
rra[4].rows = 600
rra[4].cur_row = 181
rra[4].pdp_per_row = 1
rra[4].xff = 5.0000000000e-01
rra[4].cdp_prep[0].value = NaN
rra[4].cdp_prep[0].unknown_datapoints = 0
rra[5].cf = "MIN"
rra[5].rows = 700
rra[5].cur_row = 304
rra[5].pdp_per_row = 6
rra[5].xff = 5.0000000000e-01
rra[5].cdp_prep[0].value = inf
rra[5].cdp_prep[0].unknown_datapoints = 1
rra[6].cf = "MIN"
rra[6].rows = 775
rra[6].cur_row = 703
rra[6].pdp_per_row = 24
rra[6].xff = 5.0000000000e-01
rra[6].cdp_prep[0].value = inf
rra[6].cdp_prep[0].unknown_datapoints = 19
rra[7].cf = "MIN"
rra[7].rows = 797
rra[7].cur_row = 10
rra[7].pdp_per_row = 288
rra[7].xff = 5.0000000000e-01
rra[7].cdp_prep[0].value = NaN
rra[7].cdp_prep[0].unknown_datapoints = 259
rra[8].cf = "MAX"
rra[8].rows = 600
rra[8].cur_row = 293
rra[8].pdp_per_row = 1
rra[8].xff = 5.0000000000e-01
rra[8].cdp_prep[0].value = NaN
rra[8].cdp_prep[0].unknown_datapoints = 0
rra[9].cf = "MAX"
rra[9].rows = 700
rra[9].cur_row = 216
rra[9].pdp_per_row = 6
rra[9].xff = 5.0000000000e-01
rra[9].cdp_prep[0].value = -inf
rra[9].cdp_prep[0].unknown_datapoints = 1
rra[10].cf = "MAX"
rra[10].rows = 775
rra[10].cur_row = 111
rra[10].pdp_per_row = 24
rra[10].xff = 5.0000000000e-01
rra[10].cdp_prep[0].value = -inf
rra[10].cdp_prep[0].unknown_datapoints = 19
rra[11].cf = "MAX"
rra[11].rows = 797
rra[11].cur_row = 427
rra[11].pdp_per_row = 288
rra[11].xff = 5.0000000000e-01
rra[11].cdp_prep[0].value = NaN
rra[11].cdp_prep[0].unknown_datapoints = 259
rra[12].cf = "LAST"
rra[12].rows = 600
rra[12].cur_row = 137
rra[12].pdp_per_row = 1
rra[12].xff = 5.0000000000e-01
rra[12].cdp_prep[0].value = NaN
rra[12].cdp_prep[0].unknown_datapoints = 0
rra[13].cf = "LAST"
rra[13].rows = 700
rra[13].cur_row = 476
rra[13].pdp_per_row = 6
rra[13].xff = 5.0000000000e-01
rra[13].cdp_prep[0].value = NaN
rra[13].cdp_prep[0].unknown_datapoints = 1
rra[14].cf = "LAST"
rra[14].rows = 775
rra[14].cur_row = 681
rra[14].pdp_per_row = 24
rra[14].xff = 5.0000000000e-01
rra[14].cdp_prep[0].value = NaN
rra[14].cdp_prep[0].unknown_datapoints = 19
rra[15].cf = "LAST"
rra[15].rows = 797
rra[15].cur_row = 151
rra[15].pdp_per_row = 288
rra[15].xff = 5.0000000000e-01
rra[15].cdp_prep[0].value = NaN
rra[15].cdp_prep[0].unknown_datapoints = 259

[netniV]

Sounds like a permission issue. check both directory / file permissions and the SELinux status.

[pberger]

Sounds like a permission issue. check both directory / file permissions and the SELinux status.

Checked permissions, they turned out fine (to be precise.. fully open to everything). SELinux is disabled.. best to avoid issues with it like that ;D

SOmething that might be important. When setting graphs to real-time, they do give the correct data and show a graph. When i turn real-time off, it goes blank again.

[pberger]

Well.. at one hand im happy because i fixed it. At the other hand i am still confused.

Soo.. as you might notice, my graphs are now responding and graphing correctly. This is good.
Now the reason why i am confused.
Below will be the only actions i did before it started working (no other actions has been put through on the cacti server today)
- Went to the cacti file in the cron.d
- Saw this */1 * * * * apache php /usr/share/cacti/poller.php > /usr/share/cacti/poller.log 2>&1 which is correct
- Because i refuse to trust anything of working correctly, i changed apache to cactiuser (my cacti user for database acces)
- As expected, cactiuser did not work
- Changed cactiuser back to apache

... graphs started appearing...
- Restarted the server just to make sure my eyes were not deceiving me (which i might add ive done almost twice a day while testing stuff for a week now)
.. graphs kept coming

So yes, everything i wanted to get working on Cacti works now which is great and im not going to complain about it ... but it does leave me confused

[netniV]

That would almost sound like cron hadn't picked up the change for some reason or there was a misspelling of the username.