Spine crashes on CentOS Linux release 7.9.2009 (Core)

[MarkdB]

Spine constantly crashes on CentOS Linux release 7.9.2009 (Core) on Cacti Version 1.2.25. No idea why this happens let alone how to solve it. Started with upgrade to 1.2.25

Example of part of the error from cacti_stderr.log (log is full of them)

*** Error in `/bin/spine': double free or corruption (out): 0x00007f83fc007720 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x81329)[0x7f8404726329]
/usr/lib64/mysql/libmysqlclient.so.18(+0x4d567)[0x7f8405ce3567]
/usr/lib64/mysql/libmysqlclient.so.18(+0x25755)[0x7f8405cbb755]
/usr/lib64/mysql/libmysqlclient.so.18(+0x2a4d6)[0x7f8405cc04d6]
/usr/lib64/mysql/libmysqlclient.so.18(+0x27ca4)[0x7f8405cbdca4]
/usr/lib64/mysql/libmysqlclient.so.18(mysql_close+0x1c)[0x7f8405cbdcdc]
/bin/spine[0x40d8bd]
/bin/spine[0x40fd5d]
/lib64/libpthread.so.0(+0x7ea5)[0x7f8404f02ea5]
/lib64/libc.so.6(clone+0x6d)[0x7f84047a39fd]
======= Memory map: ========
00400000-0041d000 r-xp 00000000 fd:00 535248 /usr/bin/spine
0061c000-0061d000 r--p 0001c000 fd:00 535248 /usr/bin/spine
0061d000-0061e000 rw-p 0001d000 fd:00 535248 /usr/bin/spine
0061e000-00692000 rw-p 00000000 00:00 0
00abf000-00b1e000 rw-p 00000000 00:00 0 [heap]
7f83f0000000-7f83f004c000 rw-p 00000000 00:00 0
7f83f004c000-7f83f4000000 ---p 00000000 00:00 0
7f83f5abd000-7f83fc000000 r--p 00000000 fd:00 541931 /usr/lib/locale/locale-archive
7f83fc000000-7f83fc041000 rw-p 00000000 00:00 0
7f83fc041000-7f8400000000 ---p 00000000 00:00 0
7f8401b5d000-7f8401b5e000 ---p 00000000 00:00 0
7f8401b5e000-7f840235e000 rw-p 00000000 00:00 0
7f840235e000-7f8402364000 r-xp 00000000 fd:00 527666 /usr/lib64/libnss_dns-2.17.so
7f8402364000-7f8402563000 ---p 00006000 fd:00 527666 /usr/lib64/libnss_dns-2.17.so
7f8402563000-7f8402564000 r--p 00005000 fd:00 527666 /usr/lib64/libnss_dns-2.17.so
7f8402564000-7f8402565000 rw-p 00006000 fd:00 527666 /usr/lib64/libnss_dns-2.17.so
7f8402565000-7f8402571000 r-xp 00000000 fd:00 542061 /usr/lib64/libnss_files-2.17.so
7f8402571000-7f8402770000 ---p 0000c000 fd:00 542061 /usr/lib64/libnss_files-2.17.so
7f8402770000-7f8402771000 r--p 0000b000 fd:00 542061 /usr/lib64/libnss_files-2.17.so
7f8402771000-7f8402772000 rw-p 0000c000 fd:00 542061 /usr/lib64/libnss_files-2.17.so
7f8402772000-7f8402778000 rw-p 00000000 00:00 0
7f8402778000-7f8402779000 ---p 00000000 00:00 0
7f8402779000-7f8402f79000 rw-p 00000000 00:00 0
7f8402f79000-7f8402fd9000 r-xp 00000000 fd:00 528151 /usr/lib64/libpcre.so.1.2.0
7f8402fd9000-7f84031d9000 ---p 00060000 fd:00 528151 /usr/lib64/libpcre.so.1.2.0
7f84031d9000-7f84031da000 r--p 00060000 fd:00 528151 /usr/lib64/libpcre.so.1.2.0
7f84031da000-7f84031db000 rw-p 00061000 fd:00 528151 /usr/lib64/libpcre.so.1.2.0
7f84031db000-7f84031ff000 r-xp 00000000 fd:00 528168 /usr/lib64/libselinux.so.1
7f84031ff000-7f84033fe000 ---p 00024000 fd:00 528168 /usr/lib64/libselinux.so.1
7f84033fe000-7f84033ff000 r--p 00023000 fd:00 528168 /usr/lib64/libselinux.so.1
7f84033ff000-7f8403400000 rw-p 00024000 fd:00 528168 /usr/lib64/libselinux.so.1
7f8403400000-7f8403402000 rw-p 00000000 00:00 0

[TheWitness]

Grab the latest 1.2.x branch, and then try to capture a stack trace from the core file. Post the backtrace on GitHub under the spine issues list. Lastly ensure you have enough connections:

Code: Select all

SHOW GLOBAL VARIABLES LIKE 'max_connections';
SHOW GLOBAL STATUS LIKE 'max_used_connections';

[MarkdB]

Corporate kinda pushes us to use only releases through CentOs repo. Guess this is ok (is there a recommended number?)

MariaDB [cacti]> SHOW GLOBAL VARIABLES LIKE 'max_connections';
+-----------------+-------+
| Variable_name | Value |
+-----------------+-------+
| max_connections | 151 |
+-----------------+-------+
1 row in set (0.00 sec)

MariaDB [cacti]> SHOW GLOBAL STATUS LIKE 'max_used_connections';
+----------------------+-------+
| Variable_name | Value |
+----------------------+-------+
| Max_used_connections | 20 |
+----------------------+-------+
1 row in set (0.00 sec)

MariaDB [cacti]>

[TheWitness]

Best get the backtrace then.