Submit Your CMD.PHP vs. SPINE Metrics Here

[hmorandell]

Hello,

I just noticed a strange (but good) behaviour:

This are my items:
Hosts 1363
Graphs 4115
Data Sources
Script/Command: 201
SNMP: 270
SNMP Query: 3817
Script - Script Server (PHP): 4
Total: 4292


I used to have this configuration:

SPINE:
Concurrent Processes 1
Max Threads 50

It took about 55 seconds for the polling

I changed it now to:
SPINE:
Concurrent Processes 1
Max Threads 35

and suddenly it only takes ~40 seconds to run the polling and the CPU load is only half now!!!!


In the description it says using a higher number for max threads the performance using spine will improve. For me exactly the different happens!!!! Maybe this info is useful to someone...

[gandalf]

In the description it says using a higher number for max threads the performance using spine will improve. For me exactly the different happens!!!! Maybe this info is useful to someone...

It does not make sense to increase threads too high. The "old" estimate was using a value between 10-15. On very huge environments (> 100,000 items), values around 50 may be sane.
R.

[MrRat]

05/19/2012 10:17:11 PM - SYSTEM STATS: Time:10.0804 Method:spine Processes:1 Threads:100 Hosts:948 HostsPerProcess:948 DataSources:35864 RRDsProcessed:0
05/19/2012 10:16:32 PM - SYSTEM HMIB STATS: time:20.0000 processes:50 hosts:324
05/19/2012 10:16:17 PM - SYSTEM SYSLOG STATS:Time:4.97 Deletes:0 Incoming:368 Removes:937 XFers:0 Alerts:21 Alarms:0 Reports:0
05/19/2012 10:16:11 PM - SYSTEM THOLD STATS: Time:0.0155 Tholds:0 TotalHosts:947 DownHosts:1 NewDownHosts:0
05/19/2012 10:16:11 PM - SYSTEM STATS: Time:10.2817 Method:spine Processes:1 Threads:100 Hosts:948 HostsPerProcess:948 DataSources:35863 RRDsProcessed:0
05/19/2012 10:15:33 PM - SYSTEM HMIB STATS: time:19.9300 processes:50 hosts:324
05/19/2012 10:15:18 PM - SYSTEM SYSLOG STATS:Time:5.35 Deletes:0 Incoming:372 Removes:1231 XFers:0 Alerts:21 Alarms:0 Reports:0
05/19/2012 10:15:13 PM - SYSTEM THOLD STATS: Time:0.0103 Tholds:0 TotalHosts:947 DownHosts:1 NewDownHosts:0
05/19/2012 10:15:13 PM - SYSTEM STATS: Time:11.5381 Method:spine Processes:1 Threads:100 Hosts:948 HostsPerProcess:948 DataSources:35862 RRDsProcessed:0
05/19/2012 10:14:32 PM - SYSTEM HMIB STATS: time:20.7100 processes:50 hosts:324
05/19/2012 10:14:18 PM - SYSTEM SYSLOG STATS:Time:7.15 Deletes:0 Incoming:390 Removes:1256 XFers:0 Alerts:21 Alarms:0 Reports:0
05/19/2012 10:14:11 PM - SYSTEM THOLD STATS: Time:0.0087 Tholds:0 TotalHosts:947 DownHosts:1 NewDownHosts:0
05/19/2012 10:14:11 PM - SYSTEM STATS: Time:10.2068 Method:spine Processes:1 Threads:100 Hosts:948 HostsPerProcess:948 DataSources:35864 RRDsProcessed:0
05/19/2012 10:13:32 PM - SYSTEM HMIB STATS: time:19.5300 processes:50 hosts:324
05/19/2012 10:13:18 PM - SYSTEM SYSLOG STATS:Time:5.75 Deletes:0 Incoming:412 Removes:1330 XFers:0 Alerts:21 Alarms:0 Reports:0
05/19/2012 10:13:12 PM - SYSTEM THOLD STATS: Time:0.0093 Tholds:0 TotalHosts:947 DownHosts:1 NewDownHosts:0

[udo0404]

Intel(R) Xeon(R) CPU E5520 @ 2.27GHz, 8 cores
Ubuntu Linux 10.04.3 / Linux 2.6.32-39-server on x86_64

06/05/2012 02:40:34 PM - SYSTEM THOLD STATS: Time:18.1463 Tholds:6074 DownHosts:0
06/05/2012 02:40:16 PM - SYSTEM STATS: Time:14.6556 Method:spine Processes:5 Threads:12 Hosts:362 HostsPerProcess:73 DataSources:40685 RRDsProcessed:17537
06/05/2012 11:53:44 AM - SYSTEM MACTRACK STATS: Time:807.6897 ConcurrentProcesses:7 Devices:314
06/05/2012 11:50:33 AM - SYSTEM THOLD STATS: Time:15.8732 Tholds:6050 DownHosts:0

Regards

Udo

[Instigater]

App Vserver 2x2.3Ghz Opteron; 2GB RAM
DB Vserver 2z2.3Ghz Opteron; 4GB RAM

+ Boost + some other plugins + custom video rendering scripts from weathermap images (4 videos of past 48hours are rendered every hour).

07/27/2012 11:04:19 AM - SYSTEM STATS: Time:17.5515 Method:spine Processes:2 Threads:64 Hosts:664 HostsPerProcess:332 DataSources:23996 RRDsProcessed:0

Had to use a lot of threads to overcome data collection blocking when some hosts respond realy slow and delays data aquisition from other hosts.

[dragossto]

Intel(R) Core(TM)2 Quad CPU Q9450 @ 2.66GHz ; 4G RAM
Slackware 13.37 x86

SYSTEM MACTRACK STATS: Time:594.6040 ConcurrentProcesses:5 Devices:194
SYSTEM STATS: Time:8.8410 Method:spine Processes:4 Threads:18 Hosts:295 HostsPerProcess:74 DataSources:5643 RRDsProcessed:2836

[ikorzha758]

Linux version 2.6.32-279.9.1.el6.x86_64 (mockbuild@c6b9.bsys.dev.centos.org) (gcc version 4.4.6 20120305 (Red Hat 4.4.6-4) (GCC) ) #1 SMP Tue Sep 25 21:43:11 UTC 2012
QTY: 2 Quad CORE Intel(R) Xeon(R) CPU X5647 @ 2.93GHz
8 GIG of RAM. 15K RPM SAS Drive RAID 0

12/07/2012 10:29:20 AM - SYSTEM SYSLOG STATS:Time:0.07 Deletes:0 Incoming:0 Removes:191 XFers:0 Alerts:0 Alarms:0 Reports:0
12/07/2012 10:29:20 AM - SYSTEM THOLD STATS: Time:0.0120 Tholds:0 TotalHosts:285 DownHosts:2 NewDownHosts:0
12/07/2012 10:29:20 AM - SYSTEM STATS: Time:18.1005 Method:spine Processes:10 Threads:12 Hosts:286 HostsPerProcess:29 DataSources:25961 RRDsProcessed:0
12/07/2012 10:28:19 AM - SYSTEM SYSLOG STATS:Time:0.08 Deletes:0 Incoming:0 Removes:147 XFers:0 Alerts:0 Alarms:0 Reports:0
12/07/2012 10:28:19 AM - SYSTEM THOLD STATS: Time:0.0084 Tholds:0 TotalHosts:285 DownHosts:2 NewDownHosts:0
12/07/2012 10:28:19 AM - SYSTEM STATS: Time:18.1703 Method:spine Processes:10 Threads:12 Hosts:286 HostsPerProcess:29 DataSources:25961 RRDsProcessed:0
12/07/2012 10:27:19 AM - SYSTEM SYSLOG STATS:Time:0.08 Deletes:0 Incoming:0 Removes:137 XFers:0 Alerts:0 Alarms:0 Reports:0
12/07/2012 10:27:19 AM - SYSTEM THOLD STATS: Time:0.0130 Tholds:0 TotalHosts:285 DownHosts:2 NewDownHosts:0
12/07/2012 10:27:19 AM - SYSTEM STATS: Time:18.2227 Method:spine Processes:10 Threads:12 Hosts:286 HostsPerProcess:29 DataSources:25961 RRDsProcessed:0

[juliop]

General Information
Date Fri, 17 May 2013 12:40:16 -0500
Cacti Version 0.8.8a
Cacti OS unix
SNMP Version NET-SNMP version: 5.4.3
RRDTool Version RRDTool 1.4.x
Hosts 106
Graphs 592
Data Sources Script/Command: 5
SNMP: 357
SNMP Query: 330
Script - Script Server (PHP): 45
Total: 737
Poller Information
Interval 60
Type SPINE 0.8.8a Copyright 2002-2012 by The Cacti Group
Items Action[0]: 1017
Action[1]: 5
Action[2]: 45
Total: 1067
Concurrent Processes 8
Max Threads 10
PHP Servers 1
Script Timeout 25
Max OID 10
Last Run Statistics Time:8.1935 Method:spine Processes:8 Threads:10 Hosts:106 HostsPerProcess:14 DataSources:1008 RRDsProcessed:667
PHP Information
PHP Version 5.4.6-1ubuntu1.2
PHP OS Linux
PHP uname Linux mgmt-ede-01 3.5.0-17-generic #28-Ubuntu SMP Tue Oct 9 19:31:23 UTC 2012 x86_64
PHP SNMP Installed
max_execution_time 30
memory_limit 128M

SERVER: HP Z10 WORKSTATION (XEON E3 1200 - 8GiB RAM DDR3)

WITH CMD.PHP: 58 sec
WITH SPINE: 8 sec

[ScorpionSting]

Replace hostnames below with X due to security of environment.

Technical Support
General Information
Date Thu, 30 May 2013 10:53:30 +1000
Cacti Version 0.8.8a
Cacti OS unix
SNMP Version NET-SNMP version: 5.7.2
RRDTool Version RRDTool 1.4.x
Hosts 16
Graphs 56
Data Sources Script/Command: 21
SNMP: 22
Script Query: 8
Script - Script Server (PHP): 5
Script Query - Script Server: 13
Total: 69
Poller Information
Interval 300
Type SPINE 0.8.8a Copyright 2002-2012 by The Cacti Group
Items Action[0]: 22
Action[1]: 37
Action[2]: 26
Total: 85
Concurrent Processes 1
Max Threads 1
PHP Servers 1
Script Timeout 25
Max OID 10
Last Run Statistics Time:7.8489 Method:spine Processes:1 Threads:1 Hosts:17 HostsPerProcess:17 DataSources:85 RRDsProcessed:62
PHP Information
PHP Version 5.4.15
PHP OS Linux
PHP uname Linux X 3.7.10-1.4-desktop #1 SMP PREEMPT Fri Apr 19 12:06:34 UTC 2013 (8ef74f8) x86_64
PHP SNMP Installed
max_execution_time 30
memory_limit 128M

Ignore the 300 secs, was creating spine conf......spine appears to be slower and more unreliable in my environment....

05/30/2013 12:20:03 AM - SYSTEM STATS: Time:1.1889 Method:cmd.php Processes:1 Threads:N/A Hosts:17 HostsPerProcess:17 DataSources:85 RRDsProcessed:69
05/30/2013 12:25:02 AM - SYSTEM STATS: Time:1.2038 Method:cmd.php Processes:1 Threads:N/A Hosts:17 HostsPerProcess:17 DataSources:85 RRDsProcessed:69
05/30/2013 12:30:03 AM - SYSTEM STATS: Time:1.2820 Method:cmd.php Processes:1 Threads:N/A Hosts:17 HostsPerProcess:17 DataSources:85 RRDsProcessed:69
05/30/2013 12:35:03 AM - SYSTEM STATS: Time:2.2669 Method:cmd.php Processes:1 Threads:N/A Hosts:17 HostsPerProcess:17 DataSources:85 RRDsProcessed:58
05/30/2013 12:45:00 AM - POLLER: Poller[0] Maximum runtime of 298 seconds exceeded. Exiting.
05/30/2013 12:45:00 AM - SYSTEM STATS: Time:298.0159 Method:spine Processes:1 Threads:1 Hosts:17 HostsPerProcess:17 DataSources:85 RRDsProcessed:0
05/30/2013 12:45:09 AM - SYSTEM STATS: Time:7.8489 Method:spine Processes:1 Threads:1 Hosts:17 HostsPerProcess:17 DataSources:85 RRDsProcessed:62
05/30/2013 10:50:02 AM - SPINE: Poller[0] Host[2] Hostname[X] ERROR: HOST EVENT: Host is DOWN Message: UDP: Ping timed out
05/30/2013 10:50:03 AM - SPINE: Poller[0] Host[3] Hostname[X] ERROR: HOST EVENT: Host is DOWN Message: UDP: Ping timed out
05/30/2013 10:50:05 AM - SPINE: Poller[0] Host[10] Hostname[X] ERROR: HOST EVENT: Host is DOWN Message: UDP: Ping timed out
05/30/2013 10:50:05 AM - SPINE: Poller[0] Host[12] Hostname[X] ERROR: HOST EVENT: Host is DOWN Message: UDP: Ping timed out
05/30/2013 10:50:06 AM - SPINE: Poller[0] Host[13] Hostname[X] ERROR: HOST EVENT: Host is DOWN Message: UDP: Ping timed out
05/30/2013 10:50:07 AM - SPINE: Poller[0] Host[14] Hostname[X] ERROR: HOST EVENT: Host is DOWN Message: UDP: Ping timed out
05/30/2013 10:50:08 AM - SPINE: Poller[0] Host[15] Hostname[X] ERROR: HOST EVENT: Host is DOWN Message: UDP: Ping timed out
05/30/2013 12:50:08 AM - SYSTEM STATS: Time:7.2506 Method:spine Processes:1 Threads:1 Hosts:17 HostsPerProcess:17 DataSources:85 RRDsProcessed:62

[reimlima]

Technical Support
General Information
Date Thu, 06 Mar 2014 14:13:23 -0300
Cacti Version 0.8.7e
Cacti OS unix
SNMP Version NET-SNMP version: 5.3.2.2
Hosts 4333
Graphs 155880
Data Sources Script - Script Server (PHP): 254535
Total: 254535
Poller Information
Interval 300
Type spine
Items Action[2]: 195995
Total: 195995
Concurrent Processes 1
Max Threads 20
PHP Servers 10
Script Timeout 15
Max OID 40
Last Run Statistics Time:229.9647 Method:spine Processes:1 Threads:20 Hosts:3280 HostsPerProcess:3280 DataSources:195995 RRDsProcessed:0
PHP Information
PHP Version 5.3.1
PHP OS Linux

=)

[apfjunior]

Intel(R) Xeon(R) Dual-Core CPU 5160 @ 3.00GHz, 20 GB RAM
Debian GNU/Linux 7.1
6 discos de 146 GB/cada (15RPM), em RAID 1+0.

Cacti Version 0.8.8b
Hosts 5234

Data Sources Script/Command: 5165
SNMP: 16296
SNMP Query: 11844
Script Query: 6
Script - Script Server (PHP): 1
Total: 33312

Intervalo 300s

Last Run Statistics Time:43.4856 Method:spine Processes:4 Threads:20

Coloquei o diretório /var/www/cacti/rra em partição dedicada, bem como para o /var/lib/mysql.

As opções de montagem adotadas foram

type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered)

[romuald]

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 1
Core(s) per socket: 1
Socket(s): 8
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 15
Stepping: 3
CPU MHz: 1995.812
BogoMIPS: 3991.62
Hypervisor vendor: KVM
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 4096K
NUMA node0 CPU(s): 0-7

RAM 16Gb

Cacti Version 0.8.8b
Cacti OS Ubuntu 14.04.1 LTS
SNMP Version NET-SNMP version: 5.7.2
RRDTool Version RRDTool 1.4.x



02/04/2015 06:02:09 PM - SYSTEM STATS: Time:7.8075 Method:spine Processes:1 Threads:25 Hosts:36 HostsPerProcess:36 DataSources:20261 RRDsProcessed:0
02/04/2015 06:01:40 PM - SYSTEM BOOST STATS: Time:31.3300 RRDUpdates:587064

Interval 60
Type SPINE 0.8.8b Copyright 2002-2013 by The Cacti Group

[jimcjulsonjr]

So this is a work in progress, but got things working pretty well here...

<VIRTUAL MACHINE INFO>
OS: Ubuntu 14.04.1 LTS
CACTI VERSION: 0.8.8b
CPU: 8 x Cores
RAM: 16GB
DISK: Running on a VNX via iSCSI attached storage

TWEAKS IN PLAY: Good ol' fashioned Cacti is all.


10/27/2015 11:52:05 AM - SYSTEM STATS: Time:123.8662 Method:spine Processes:4 Threads:4 Hosts:298 HostsPerProcess:75 DataSources:100075 RRDsProcessed:40297

[VFMTV]

Hi,
Spine is a really great tool. With this poller we can reduce our monitoring server and have better performance than other charged monitroing tools.
Thank you very much for this great tool.

this is our following server configuration:

virtual server with reserved ressources:
CPU: 1x 1,2GHz
RAM: 1x 2GB RAM
VM is able to get half host CPU ressources

needed time with cmd.php: between 130s and 240s
needed time with spine: between 15s and 25s


Following the Cacti configuration:

Technical Support
General Information
Date Thu, 29 Oct 2015 10:13:40 +0100
Cacti Version 0.8.8f
Cacti OS unix
SNMP Version NET-SNMP version: 5.5
RRDTool Version RRDTool 1.3.x
Hosts 263
Graphs 12388
Data Sources Script/Command: 21
SNMP: 535
SNMP Query: 11955
Script Query: 1
Script Query - Script Server: 4
Total: 12516
Poller Information
Interval 300
Type SPINE 0.8.8f Copyright 2002-2015 by The Cacti Group
Items Action[0]: 24006
Action[1]: 21
Action[2]: 6
Total: 24033
Concurrent Processes 5
Max Threads 4
PHP Servers 1
Script Timeout 25
Max OID 10
Last Run Statistics Time:22.7638 Method:spine Processes:5 Threads:4 Hosts:264 HostsPerProcess:53 DataSources:24033 RRDsProcessed:12318

[phalek]

Currently working on a hardware appliance for SMB or remote offices. It's polling simulated Cisco CMTS devices which have a very, very large number of interfaces, so a high datasource density.

This is from a virtual test system:
OS: Centos 6
CACTI VERSION: 0.8.8b

11/05/2015 04:03:45 AM - SYSTEM STATS: Time:42.7337 Method:spine Processes:1 Threads:8 Hosts:16 HostsPerProcess:16 DataSources:114576 RRDsProcessed:48061
11/05/2015 04:04:28 AM - SYSTEM STATS: Time:26.4977 Method:spine Processes:1 Threads:8 Hosts:16 HostsPerProcess:16 DataSources:114576 RRDsProcessed:48061
11/05/2015 04:05:28 AM - SYSTEM STATS: Time:27.4840 Method:spine Processes:1 Threads:8 Hosts:16 HostsPerProcess:16 DataSources:114576 RRDsProcessed:48061

This plain Cacti.