[SOLVED] Some graphs are not updating after cacti migration

[hakiki]

Hi, I'm trying to migrate an existing cacti system (Cacti 8.8.8h, Centos 6.5) to a new server (Cacti 1.2.19, RHEL 8.5). After copying rrd files and setup upgrade on the new system, I enabled cron for the poller.

Several device graphs are working fine, but some graphs are not updating. Tried debugging method from the cacti manuals but still havent got the clue for this problem.

From what I found, the rrd files are not updated and seems poller was not running properly for the problematic devices/graphs.

Here are cacti logs sample (I filtered for one specific rrd file):

Code: Select all

- WEBLOG CACTI2RRD: /usr/bin/rrdtool info /usr/share/cacti/rra/p-d1-bna_traffic_in_125065.rrd
- WEBLOG CACTI2RRD: /usr/bin/rrdtool graphv -  --imgformat=PNG  --start='1639642218'  --end='1639728618'  --pango-markup   --title='P-D1-BNA-HundredGigE0/0/0/1-AGGR_P-D1-BNA/Hu0/0/0/1_TO_P-D1-LSM/Hu0/0/0/1_No1_100G_BE1'  --vertical-label='bits per second'  --base=1000  --height=120  --width=500  --tabwidth '30'  --rigid  --alt-autoscale-max  --lower-limit='0'  COMMENT:"From 2021-12-16 15\:10\:18 To 2021-12-17 15\:10\:18\c"  COMMENT:"  \n"  --color BACK#F3F3F3  --color CANVAS#FDFDFD  --color SHADEA#CBCBCB  --color SHADEB#999999  --color FONT#000000  --color AXIS#2C4D43  --color ARROW#2C4D43  --color FRAME#2C4D43  --border 1 --font TITLE:11:'Arial'  --font AXIS:8:'Arial'  --font LEGEND:8:'Courier'  --font UNIT:8:'Arial'  --font WATERMARK:6:'Arial'  --watermark 'Generated by Cacti®'  DEF:a='/usr/share/cacti/rra/p-d1-bna_traffic_in_125065.rrd':'traffic_in':AVERAGE  DEF:b='/usr/share/cacti/rra/p-d1-bna_traffic_in_125065.rrd':'traffic_out':AVERAGE  CDEF:cdefa='a,8,*'  CDEF:cdefe='b,8,*'  AREA:cdefa#00CF00FF:'Inbound '   GPRINT:cdefa:LAST:'Current\:%8.2lf %s'   GPRINT:cdefa:AVERAGE:'Average\:%8.2lf %s'   GPRINT:cdefa:MAX:'Maximum\:%8.2lf %s\n'   LINE1:cdefe#002A97FF:'Outbound'   GPRINT:cdefe:LAST:'Current\:%8.2lf %s'   GPRINT:cdefe:AVERAGE:'Average\:%8.2lf %s'   GPRINT:cdefe:MAX:'Maximum\:%8.2lf %s\n' 

Here are my new system info:
Cacti 1.2.19
Spine 1.2.19
RHEL 8.5
MariaDB 10.3.28
PHP 7.2.24 (cli)

Anyway I'm new to the cacti and still on learning progress.. Any inputs are welcome

[hakiki]

I took sample of 1 device whose graphs are not updating (device status: UP detected by cacti). Tried to create new graph on that device, but the graph displays "Failed to open data file, poller may not have run yet".

The data source already present in the poller cache along with the others, but from cacti debug log I didn't see such "rrdtool update .." for this device. Also, the rrd file for new graph I mentioned before still not created yet in the rra directory. Seems the poller was not running for this device somehow. Tried rebuild poller cache but it didn't solve the issue.

Here are checking result I got from Troubleshooting > Data Sources:

2021-12-20_21-03_001.png (9.17 KiB) Viewed 544 times

Tried comparing this device settings with a normally graphed device, and I found that this device desn't have "Associated graph template" other than Cisco - CPU Usage (not being graphed):

2021-12-20_21-21_001.png (20.74 KiB) Viewed 544 times

While for the normally graphed device, they have another associated graph template:

2021-12-20_21-26_001.png (22.74 KiB) Viewed 544 times

Strangely, this is the exact configuration as the old server, but the graphs are working normally there. I'm not sure whether this affect the polling behaviour or not.

Also, I noticed following log everytime cron poller executed. I ran the poller.php manually from cli and got same message after 5 minutes of running.

2021-12-20_23-20_001.png (7.82 KiB) Viewed 542 times

[camerabob]

Based on your last post, I would look to improve the spine polling. Not sure how since I don't use it, but I believe increasing threads slightly is the way. This is different between 8.x and 1.0 and newer.

[hakiki]

Hi camerabob, thanks for pointing it out.. I did change the value of processes and threads for the poller [Data Collection > Data Collectors] last night, and turns out the graphs strart to updating..

I reduced the values become processes: 4; threads: 20 since my system has 4 CPU cores (before was processes: 10; threads:30), but I got some deviations of poller running time lagging up to 2 minutes. I tried to increase processes and will observe the effect. For now I will mark this as solved.