Spine performance expectations

[chickenlicker]

Hi folks,

I've read the manual online and that Spine can process like 20'000 data sources in under 60 seconds, but I'm having trouble getting it to process even 2000 within 60 seconds.

I have 2100 routers and switches that I'd like to poll with a 1-minute interval to graph traffic statistics of just one interface per device.
The most I've ever been able to get out of cacti is 800-ish devices/1600-ish data sources before it runs over the 60 second poll interval.

Have I misinterpreted what spine can do? Should it be able to poll all 2100 devices within 60 seconds or will that require multiple remote pollers?
I've tried threads, processes and reducing timeout values but I just can't get near the 20'000 data sources that has been quoted.

At the moment I'm running 10 processes and 30 threads for 519 hosts, 1280 data sources and it's taking between 30 to 50 seconds to get through them.
I've tried both the old 0.8 version of cacti with php5.5 and the new 1.2.1 version of cacti with php7.1.3 and Spine 1.2.1 via cygwin. Edit: both windows servers.

Is anyone successfully polling tens of thousands of data sources within 60 seconds? If so -- how did you do it?

[netniV]

It all comes down to the hardware you are utilising. When you start talking about a lot of devices, you need a physical, not virtual machine. You need fast disks and copious memory. You may also need to implement Boost (that batches up RRD file updates).

[chickenlicker]

Thanks netniV, I think I'll split it up over a few remote pollers.
I've been googling and searching through the cacti documentation but can't find much info on that feature.
Are you able to tell me if devices are added to the different pollers individually? Or do you add all of your devices to the main poller and then the load is split up automatically across the available pollers?

EG: If I have 3000 hosts and 5 remote pollers. Do I have to add a portion to poller1, poller2, poller3/etc. Or does the main poller go "okay, I'll split those 3000 hosts over 5 pollers so poller1 you get <x>, poller2 you get <y>/etc"

I feel like the answer to that question is probably the former - each remote poller is responsible for whatever devices you add to that individual poller.
If that's true... are you able to tell me if each device's data ends up in the same database table? Or does each remote poller maintain its own database records?

At the moment I pull some statistics straight out of the database for use in other applications but if I'm wondering if I'll lose that ability if I move to a remote poller setup?
I'd have to query all 5 remote poller database and collate the data for use in the other application?

[netniV]

You assign devices to individual pollers yourself. We don't currently have an automatic option as the poller could sit anywhere in the network.

[chickenlicker]

Thanks netniV.
I ended up installing cacti on a few servers (not in remote poller mode) and wrote my own script to dish out all my hosts across the available servers.
Seems to be working well so far.

[netniV]

If you neaten it up, we could always have it posted here for others to use

[ok1mgj]

Hi folks.
Spine is fast enough, i`ve had troubles with slow disk performance.
SATA III speed is really slow for bigger cacti instalation.
I`m talking about 1 minute pooling and more than 18GB of rrd files.
Booster is not usable for me, because i need graphs NOW, not in five minutes, or more.
At the end I moved RRD files to RAM drive.
Now on 8 core machine performance looks like:
06/18/2019 11:44:14 AM - SYSTEM STATS: Time:12.9067 Method:spine Processes:4 Threads:32 Hosts:671 HostsPerProcess:168 DataSources:176506 RRDsProcessed:73032

edit: DO NOT FORGET TO BACKUP RRD FILES a few times a day to harddrives, and after server reboot move them back to ramdrive

[Osiris]

SSD, or even better than that PCIe SSD, or just use Boost, and you will be free and clear.