Spine 0.8.8 multiple number of processes and growing

[seco]

Hello.

We are transitioning our cacti setup to spine from cmd.php
Some issues still not cleared yet:
1) Spine 0.8.8 was build from sources.
It works well but number of spine processes in system are growing permanently.
They are not closed automaticaly and only kill helps.

Code: Select all

# ps aux | grep spine | grep -v grep 
www      5105   0.0  0.1 113120  11760  ??  I    12:30PM      0:00.42 /usr/local/bin/spine 0 899
www      5162   0.0  0.2 117088  14708  ??  S    12:33PM      0:00.39 /usr/local/bin/spine 0 899
www      5496   0.0  0.2 113120  15888  ??  I    12:35PM      0:00.50 /usr/local/bin/spine 0 899
www      5543   0.0  0.1 113120  10444  ??  I    12:38PM      0:00.36 /usr/local/bin/spine 0 899
www      6271   0.0  0.2 113120  14916  ??  S    12:48PM      0:00.37 /usr/local/bin/spine 0 899
...
www     14734   0.0  0.2 117216  17548  ??  S     2:40PM      0:00.41 /usr/local/bin/spine 0 899
www     14779   0.0  0.2 113120  14688  ??  S     2:43PM      0:00.35 /usr/local/bin/spine 0 899
www     94896   0.0  0.2 113120  15824  ??  I    12:10PM      0:00.42 /usr/local/bin/spine 0 899
www     96098   0.0  0.1 109024  11400  ??  I    12:21PM      0:00.36 /usr/local/bin/spine 0 899
www     96152   0.0  0.1 117216  12188  ??  S    12:23PM      0:00.34 /usr/local/bin/spine 0 899
#
# ps aux | grep spine | wc -l
      58

Also tried Setuid spine exec file.
# ls -lf | grep spine
-rwsr-sr-x  1 root  wheel   239379 Apr 13 12:34 spine

I suppose smthg with rights are in the middle.

Cacti settings:
Poller Type: Spine
Poller Interval: 1min
Cron Interval: 5min
Maximum Concurrent Poller Processes: 1
Balance Process Load: yes
Maximum Threads per Process: 6
Enable direct population of poller_output_boost table by spine: yes

And it could not be placed on production server.

In attachment could be find 1 day progress.

Dedicated box

Code: Select all

# sysctl -a | egrep -i 'hw.machine|hw.model|hw.ncpu|hw.*mem:'
hw.machine: amd64
hw.model: Intel(R) Xeon(R) CPU           L5410  @ 2.33GHz
hw.ncpu: 4
hw.physmem: 8568328192
hw.usermem: 7252586496
hw.machine_arch: amd64
hw.realmem: 9395240960

# uname -a
FreeBSD cacti 9.0-RELEASE FreeBSD 9.0-RELEASE #0: Tue Mar 27 09:14:02 EEST 2012     sc@cacti:/usr/obj/usr/src/sys/MYK amd64
# snmpwalk -V
NET-SNMP version: 5.7.1
# php -v
PHP 5.3.10 with Suhosin-Patch (cli) (built: Mar 27 2012 13:11:52)

2) we have ~600 hosts, and if some part of network (ex:30 hosts) is not available, overall performance of cacti/spine falls significantly.
(spine polling exceeds 60 seconds and gaps appears)
what settings would be recommended for medium size setups?
available settings: icmp only/snmp only/icmp or snmp ?
max snmp timeout?
how number of thread per device would help?
Are BorisL patches already integrated in current version of spine ? ( http://bugs.cacti.net/view.php?id=2070 )

ps: sometime while debugging (./spine -R -V 5 -S) spine even get segm fault.

Any help would be appreciated.

[gandalf]

Unfortunately, spine088 is broken. Please either revert to spine087i or wait for spine088a. Both will work with Cacti 088
R.

[toe_cutter]

I noticed this aswell, but only once after a reboot. Killing all the processes fixed the problem and polling went back to normal.
Also got FreeBSD9 with spine088 from svn, can't remember rev tho, ill edit tomorrow with that. Will also consider downgrading to 088i.

Edit: was running rev7115. Now downgraded to 087i r6951.

[gandalf]

In case you're run the current spine from 088 repo, you should be fine. The issue has been worked around already in SVN. But it's a minor issue; so no need for a change in case you don't get any dumps
R.