spine 0.8.8 always reach maximum runtime

[skyjou]

hi, i upgrade my cacti server from 0.8.7i to 0.8.8.
it seems work fine, but when i use spine 0.8.8 as the poller,
i find it always reach maximum runtime:

Code: Select all

04/05/2012 09:50:00 AM - SYSTEM STATS: Time:298.8794 Method:spine Processes:10 Threads:25 Hosts:903 HostsPerProcess:91 DataSources:3907 RRDsProcessed:3301
04/05/2012 09:50:00 AM - POLLER: Poller[0] Maximum runtime of 298 seconds exceeded. Exiting.

if i use spine 0.8.7i, it works well.

do i make any mistake somewhere?

[bfek18]

hi, i upgrade my cacti server from 0.8.7.g to 0.8.8.
it seems work fine, but when i use spine 0.8.8 as the poller,
i find it always reach maximum runtime:

Code: Select all

04/05/2012 09:50:00 AM - SYSTEM STATS: Time:298.8794 Method:spine Processes:10 Threads:25 Hosts:903 HostsPerProcess:91 DataSources:3907 RRDsProcessed:3301
04/05/2012 09:50:00 AM - POLLER: Poller[0] Maximum runtime of 298 seconds exceeded. Exiting.

if i use spine 0.8.7g, it works well.

do i make a mistake somewhere?

hi,
i have the same issue.
with the spine-0.8.7i it still works.
This is probably due to this problem: http://bugs.cacti.net/view.php?id=2185
bye

[dragossto]

test spine 0.8.8

and the reason it :

spine -V -f 0 -l 185
SPINE: Using spine config file [/etc/spine.conf]
SPINE: Version 0.8.8 starting
*** glibc detected *** spine: double free or corruption (out): 0xb6000510 ***
======= Backtrace: =========
/lib/libc.so.6(+0x714ae)[0xb72f94ae]
/lib/libc.so.6(cfree+0x70)[0xb72fd050]
spine[0x804b269]
/lib/libc.so.6(__libc_start_main+0xe6)[0xb729edb6]
spine[0x804a301]
======= Memory map: ========
08048000-0805d000 r-xp 00000000 08:01 1989497 /usr/bin/spine
0805d000-0805e000 rw-p 00014000 08:01 1989497 /usr/bin/spine
0805e000-083b7000 rw-p 00000000 00:00 0 [heap]
ae400000-ae47e000 rw-p 00000000 00:00 0
ae47e000-ae500000 ---p 00000000 00:00 0
af600000-af601000 ---p 00000000 00:00 0
af601000-afe00000 rw-p 00000000 00:00 0
b0e00000-b0e01000 ---p 00000000 00:00 0
b0e01000-b1600000 rw-p 00000000 00:00 0
b1600000-b1601000 ---p 00000000 00:00 0
b1601000-b1e00000 rw-p 00000000 00:00 0
b3e00000-b3e01000 ---p 00000000 00:00 0
b3e01000-b4600000 rw-p 00000000 00:00 0
b4600000-b46f6000 rw-p 00000000 00:00 0
b46f6000-b4700000 ---p 00000000 00:00 0
b4700000-b4777000 rw-p 00000000 00:00 0
b4777000-b4800000 ---p 00000000 00:00 0
b4800000-b4801000 ---p 00000000 00:00 0
b4801000-b5000000 rw-p 00000000 00:00 0
b6000000-b60e6000 rw-p 00000000 00:00 0
b60e6000-b6100000 ---p 00000000 00:00 0
b611b000-b612f000 r-xp 00000000 08:01 370672 /lib/libresolv-2.13.so
b612f000-b6130000 r--p 00014000 08:01 370672 /lib/libresolv-2.13.so
b6130000-b6131000 rw-p 00015000 08:01 370672 /lib/libresolv-2.13.so
b6131000-b6133000 rw-p 00000000 00:00 0
b6983000-b69ba000 r--p 00000000 08:01 1916279 /usr/lib/locale/en_US/LC_CTYPE
b69ba000-b69d5000 r-xp 00000000 08:01 1916189 /usr/lib/libgcc_s.so.1
b69d5000-b69d6000 rw-p 0001a000 08:01 1916189 /usr/lib/libgcc_s.so.1
b71ee000-b71ef000 rw-p 00000000 00:00 0
b71ef000-b71f1000 r-xp 00000000 08:01 205278 /lib/libdl-2.13.so
b71f1000-b71f2000 r--p 00001000 08:01 205278 /lib/libdl-2.13.so
b71f2000-b71f3000 rw-p 00002000 08:01 205278 /lib/libdl-2.13.so
b71f3000-b71f4000 rw-p 00000000 00:00 0
b71f4000-b7239000 r-xp 00000000 08:01 1918927 /lib/libssl.so.0.9.8
b7239000-b723d000 rw-p 00044000 08:01 1918927 /lib/libssl.so.0.9.8
b723d000-b7252000 r-xp 00000000 08:01 370618 /lib/libnsl-2.13.so
b7252000-b7253000 r--p 00014000 08:01 370618 /lib/libnsl-2.13.so
b7253000-b7254000 rw-p 00015000 08:01 370618 /lib/libnsl-2.13.so
b7254000-b7256000 rw-p 00000000 00:00 0
b7256000-b725f000 r-xp 00000000 08:01 151355 /lib/libcrypt-2.13.so
b725f000-b7260000 r--p 00009000 08:01 151355 /lib/libcrypt-2.13.so
b7260000-b7261000 rw-p 0000a000 08:01 151355 /lib/libcrypt-2.13.so
b7261000-b7288000 rw-p 00000000 00:00 0
b7288000-b73e4000 r-xp 00000000 08:01 138124 /lib/libc-2.13.so
b73e4000-b73e5000 ---p 0015c000 08:01 138124 /lib/libc-2.13.so
b73e5000-b73e7000 r--p 0015c000 08:01 138124 /lib/libc-2.13.so
b73e7000-b73e8000 rw-p 0015e000 08:01 138124 /lib/libc-2.13.so
b73e8000-b73eb000 rw-p 00000000 00:00 0
b73eb000-b740f000 r-xp 00000000 08:01 356407 /lib/libm-2.13.so
b740f000-b7410000 r--p 00023000 08:01 356407 /lib/libm-2.13.so
b7410000-b7411000 rw-p 00024000 08:01 356407 /lib/libm-2.13.so
b7411000-b7426000 r-xp 00000000 08:01 370669 /lib/libpthread-2.13.so
b7426000-b7427000 r--p 00014000 08:01 370669 /lib/libpthread-2.13.so
b7427000-b7428000 rw-p 00015000 08:01 370669 /lib/libpthread-2.13.so
b7428000-b742b000 rw-p 00000000 00:00 0
b742b000-b743e000 r-xp 00000000 08:01 1919219 /usr/lib/libz.so.1.2.5
b743e000-b743f000 rw-p 00012000 08:01 1919219 /usr/lib/libz.so.1.2.5
b743f000-b756c000 r-xp 00000000 08:01 1918926 /lib/libcrypto.so.0.9.8
b756c000-b7582000 rw-p 0012d000 08:01 1918926 /lib/libcrypto.so.0.9.8
b7582000-b7585000 rw-p 00000000 00:00 0
b7585000-b76a4000 r-xp 00000000 08:01 37838 /usr/lib/mysql/libmysqlclient_r.so.16.0.0
b76a4000-b76d8000 rw-p 0011e000 08:01 37838 /usr/lib/mysql/libmysqlclient_r.so.16.0.0
b76d8000-b76d9000 rw-p 00000000 00:00 0
b76d9000-b776f000 r-xp 00000000 08:01 2014123 /usr/lib/libnetsnmp.so.25.0.2
b776f000-b7772000 rw-p 00095000 08:01 2014123 /usr/lib/libnetsnmp.so.25.0.2
b7772000-b7792000 rw-p 00000000 00:00 0
b7794000-b779a000 r-xp 00000000 08:01 370634 /lib/libnss_dns-2.13.so
b779a000-b779b000 r--p 00005000 08:01 370634 /lib/libnss_dns-2.13.so
b779b000-b779c000 rw-p 00006000 08:01 370634 /lib/libnss_dns-2.13.so
b779c000-b77a6000 r-xp 00000000 08:01 370640 /lib/libnss_files-2.13.so
b77a6000-b77a7000 r--p 00009000 08:01 370640 /lib/libnss_files-2.13.so
b77a7000-b77a8000 rw-p 0000a000 08:01 370640 /lib/libnss_files-2.13.so
b77a8000-b77ab000 rw-p 00000000 00:00 0
b77ab000-b77c8000 r-xp 00000000 08:01 371154 /lib/ld-2.13.so
b77c8000-b77c9000 r--p 0001c000 08:01 371154 /lib/ld-2.13.so
b77c9000-b77ca000 rw-p 0001d000 08:01 371154 /lib/ld-2.13.so
bfef9000-bff1a000 rw-p 00000000 00:00 0 [stack]
ffffe000-fffff000 r-xp 00000000 00:00 0 [vdso]
Aborted

back to 0.8.7i
i forgot to specify the version for gcc, it gcc version 4.5.2 (GCC)

[dbourcier]

Same issue here ...

Rollbacking to 0.8.7i is the best workaround so far until a proper patch is provided

[c3226026]

Actually I've this on debian (on fresh install) :

spine -V -f 0 -l 185
SPINE: Using spine config file [/etc/spine.conf]
SPINE: Version 0.8.8 starting
SPINE: Time: 0.0806 s, Threads: 1, Hosts: 2

Only with localhost.

But actually I've not graphs

[gandalf]

Same issue here ...

Rollbacking to 0.8.7i is the best workaround so far until a proper patch is provided

This is indeed the best solution.
The crash occurs under specific conditions during "cleanup processing"; so in most cases all data is polled. But as 088 only includes some "cleanup patches", the result is at least better when reverting to 087i.
We're working on fixes for 088
R.

[dragossto]

good to hear
looking forward for the patch

one more question
cacti 0.8.7i will work with spine 0.8.8 ?

[gandalf]

cacti 0.8.7i will work with spine 0.8.8 ?

Yes
R.

[dbourcier]

Same issue here ...

Rollbacking to 0.8.7i is the best workaround so far until a proper patch is provided

This is indeed the best solution.
The crash occurs under specific conditions during "cleanup processing"; so in most cases all data is polled. But as 088 only includes some "cleanup patches", the result is at least better when reverting to 087i.
We're working on fixes for 088
R.

I have no worries about the fix, as always, a fix will be provided ... In my case I'm running Cacti 0.8.8 with Spine 0.8.7i and it's working fine, no need to rollback the Cacti installation, rollbacking spine was enough

[mans39]

Hello,

I have the same issue on RHEL 5.8 x64 with cacti & spine 0.8.8

I have different message on glibc issue :

*** glibc detected *** /usr/bin/spine: free(): invalid pointer: 0x0000000004a839a0 ***

*** glibc detected *** /usr/bin/spine: corrupted double-linked list: 0x00000000183a6cc0 ***

And sometimes the poller works without error.

Is there a patch to this problem?

Thank's

[gandalf]

Yes, current SVN has the correct version. We expect to publish a fixed spine088a soon
R.