Script Server Data slowing Spine

[rvd]

Hi everybody,
I created a new host template for weblogic hosts. Graph and data templates for this host type are linked to a perl script - I can't write in php.
This script gives output values on 9 various data (heapused, heaptotal, threadtotal an so on) for a weblogic host, indexed on weblogic server name.
I followed then the HowTo and created xml file.
When I call the script in command line with get option, the answer is quite fast.
But when I finally added a new host of this type and created graphs for it the poller is getting MUCH slower.
The host contains 12 WL servers and the 9 values are asked, so 108 more DS.
This is the difference between before and after adding the host

05/16/2008 08:50:26 AM - SYSTEM STATS: Time:24.3523 Method:spine Processes:2 Threads:20 Hosts:85 HostsPerProcess:43 DataSources:1210 RRDsProcessed:726
05/16/2008 08:56:03 AM - SYSTEM STATS: Time:62.0576 Method:spine Processes:2 Threads:20 Hosts:86 HostsPerProcess:43 DataSources:1318 RRDsProcessed:798

My interrogation is about spine. Launching debug mode shows me a unique spine process executes the 108 calls to the script sequentially, whatever is the number of threads. It takes about 36 seconds.
Is there a way to have the 108 calls executed not sequentially ?
My other interrogation is about 'Script Data' and 'Script Server Data' in 'Data Queries'. I don't really understand the difference ?

Using spine 0.8.7 and cacti 0.8.7b

[nate450]

try re-writing the script so you only have 1 DS per host instead of 9.

I have several scripts that return a dozen or more data points to cacti for one data source, which seems to allow it to scale much higher than it otherwise would be able to. Extra work is involved in creating the graphs and stuff though.

My cacti install:

Code: Select all

05/16/2008 06:35:13 PM - SYSTEM STATS: Time:11.7292 Method:spine Processes:1 Threads:30 Hosts:305 HostsPerProcess:305 DataSources:337 RRDsProcessed:333

Doesn't look like much, but when you see how many data points are actually in those 333 data files, there are 6,773 points of data, or an average of 20 points per file. I have cacti running every minute. It's on a dual proc quad core system 16GB memory and is averaging 11% cpu usage.

[gandalf]

My interrogation is about spine. Launching debug mode shows me a unique spine process executes the 108 calls to the script sequentially, whatever is the number of threads. It takes about 36 seconds.
Is there a way to have the 108 calls executed not sequentially ?
My other interrogation is about 'Script Data' and 'Script Server Data' in 'Data Queries'. I don't really understand the difference ?

That's by design; and sometimes I hate it.
Cacti splits up all hosts between the spine process. This results in executing all Scripts/Script Server Scripts/SNMP stuff in a single spine process.

Difference between Scrip and Script Server is:
1. Script Server is available for PHP scripts only
2. Script Server starts a single PHP execution environment and then LOADs all Script Server Scripts into it (include!). The Script Server then calls the function that is provided by the Script Server Script
Reinhard

[rvd]

Cacti splits up all hosts between the spine process. This results in executing all Scripts/Script Server Scripts/SNMP stuff in a single spine process

A pity cacti doesn't dispatch job by DS rather than by host. You may actually have hosts with a few DS and other ones with a hundred ones.

Difference between Scrip and Script Server is:
1. Script Server is available for PHP scripts only
2. Script Server starts a single PHP execution environment and then LOADs all Script Server Scripts into it (include!). The Script Server then calls the function that is provided by the Script Server Script

Thx for this explanation

[gandalf]

I'm quite sure you're right.
Especially with the new cli functions, it will be a common issue, that hosts of same kind have id's in increasing sequence. This is a good chance to have multiple spine processes in a very unbalanced way.
Please create a feature request at http://bugs.cacti.net
Reinhard

[aleu]

I'm quite sure you're right.
Especially with the new cli functions, it will be a common issue, that hosts of same kind have id's in increasing sequence. This is a good chance to have multiple spine processes in a very unbalanced way.
Please create a feature request at http://bugs.cacti.net
Reinhard

rvd, have you created this feature request? It seems that I am facing exactly the same problem...

[rvd]

I'm quite sure you're right.
Especially with the new cli functions, it will be a common issue, that hosts of same kind have id's in increasing sequence. This is a good chance to have multiple spine processes in a very unbalanced way.
Please create a feature request at http://bugs.cacti.net
Reinhard

rvd, have you created this feature request? It seems that I am facing exactly the same problem...

See http://bugs.cacti.net/view.php?id=1250
Sorry for late reply...