Plugin Multi-Pollerserver (Cacticluster) 0.8.8e till 0.8.8h

[der_doc]

NEWS first new version of Pluin is out!
The new version 0.8.8e till 8.8.h of the Plugin is online for cacti on unix.

Short info:
I hope the next versions of the plugin get more the faster as this version. I plane the next update steps soon as posible.
For the german users i have written a document to install step by step. I hope one user translate it to english. pleace.


Download from:
http://www.cacti-multipollerserver.de/index.php

----[ Changelog

--- 0.8.8h ---
-change: work correcktly with cacti 0.8.8h.

--- 0.8.8g ---
-change: work correcktly with cacti 0.8.8g.

--- 0.8.8f ---
-change: work correcktly with cacti 0.8.8f.

--- 0.8.8e ---
-change: work correcktly with cacti 0.8.8e.

--- 0.8.8d ---
-fix: path correction in setup.php (thx Nino Kambach).
PHP Notice: Undefined variable: poller_server_id in /usr/share/cacti/poller.php on line 533
-change: work correcktly with cacti 0.8.8d.

--- 0.8.8c ---
-new: completly overhauled the backend. Now the installation works without root privileges.
-change: work correcktly with cacti 0.8.8c.

--- 0.2.3 ---
-new: fallback all poller to poller_id from master if update cacti to 089 before the plugin is prepared (sorry) but all settings are saved
-change: better query for hostcount.
thanks to prosodie from cacti.net

--- 0.2.2 ---
-new: Now the masterserver is ready for polling.
-new: Choose the devices on an listing for the pollerserver
-new: Show number of polling hosts

-fix: Tidy sourcecode
-fix: Failuremessage in pollerlog on polling (include once)

--- 0.2.1 ---
-new: update routine inserted
-fix: Error when pollerserver ist not registerd or is disabled
match small errors

--- 0.2 ---
-bug: Fix match error

--- 0.1 ---
Initial Release
################################################################

Hi community,
the pluginis is now ready to download.


Download from:
http://www.cacti-multipollerserver.de/index.php

But attention i give you no warranty for currectly working in your prod system. Please test it on an testsystem. !!

The installmanual and the page are only in german (my speech)
http://www.cacti-multipollerserver.de/i ... oadbereich
My english ist not so good but i hope you can forgive me.

Author: André Leinhos
E-Mail: info@cacti-multipollerserver.de
Homepage: http://www.cacti-multipollerserver.de/index.php
Version: 0.8.8h
Date: 17.06.2016


The idea of this plugin is born in the middel of the year 2011. The first version are not with the features that install automatic etc.
The real reason to start are to many devices and not enough time to polling.
With the feature that you can work virtual with cacti the eventualities are nearly endless. In this year i would create an plugin that you can use the multipollerserver on your own system. The reason that the plugin is only for unix is that the filehandel over NFS in unix is very fast and uncomplicated. If in the future are give an version for windows is not sure.

Now you can create one masterserver, he handel all the installed plugins and you can make all configurations of the system.
In my installation the database is on other server and all files are on an storage and mounted with NFS on the pollerserver.
The work of the pollerserver are easy ... polle the allotted device. . The real limit of pollerserver is only the hardware. My installation is on an range of vmware servers, so i can look at the feeling of the servers an i can create an new pollerser by copy an working server and to customize it.

So now enough spoken my me. Test the plugin an tell me your impression or wishes.

Features:

• One master take the main (plugin,configuration) work.
• A couple of pollerserver share the polling process.
• The Update is update ready.
• No existing data are lost on update.
• No existing data are lost on disable the plugin.
• One ore more variant configuration of sever or infrastructure are possible.
• VMWare installations are possible.

Roadmap:

• Fix any Bugs
• include an backup poller configuration on every pollerserver.
• more better features i think that update the plugin

Future:

• Cluster of mysqlserver.
• Make the plugin ready for work on split locations.

Same Pics:


Pluginmanger


Installform 1


Installform 2


Installform 3


Usermanagement


Pollerserver 1


Pollerserver 2


Pollerserver 3


Logfile

Same installoptions:


Option 1

Option 2

Option 3 (professional installation)


More informations for the old version are on the page or in this form http://forums.cacti.net/viewtopic.php?p=212942#p212942


Cu
der_doc

[paravis]

Hello!

I have installed the multi-poller server plugin, and it seems that both of my servers are polling to some of the same hosts. How can I debug the plugin to find out which servers are polling which hosts?

Our goal is to use this plugin to take network statistics from multiple remote sites in order to test reliability and latency, etc. from multiple sources and destinations.

Thanks for any help!
~Laz

[paravis]

Actually, please disregard.

Was having a little bit of a mix-up with the backup functions.

Would you please explain the backup_mode and what each one does?

I am sure that this function is not built into your script, but like I said, my goal is to have only a few hosts checked for latency/packet loss from a few satellite networks. Do not want them to fail over and start polling other systems. Currently I have set all pollers to backup_mode 0, which seems to make it happy.

In case I do want to configure a secondary server for "failover" operations, which mode would that be?

Thanks so much for creating a wonderful plugin. This is a great feature!

[Oyah27]

Hello,

Currently i use Cacti 088a and i would like to try the plugins MultiPollerServer, but it's for 088c. So where can i find this plugins for Cacti 088a ?

thanks in advance

[phalek]

Anyone using the Multipoller plugin and wanting to separate the rrd files to different NFS/NAS/SAN storages, look here for an enhancement to the Structured RRA Path setting which uncludes the poller_id as well.

http://forums.cacti.net/viewtopic.php?f=7&t=55239

Old RRA Path structure

old_rra_directory_structure.png (8.46 KiB) Viewed 17561 times

New RRA Path structure

new_rra_directory_structure.png (13.7 KiB) Viewed 17561 times

[blackstrip]

Greate work.
I always use it, very nice to saw this update.
thx der_doc.

Gut gemacht! Gut gemacht! (from google translate)

[roxy]

I am running Version 0.8.8a and Linux platform. I am running into an issue with high polling time and periodic gaps in my graphs. I have tried adjusting the number of threads and also tried to use multiple processes instead of one. But still there is no improvement on the total polling time.

05/04/2016 03:09:26 PM - SYSTEM STATS: Time:264.7665 Method:spine Processes:24 Threads:20 Hosts:2262 HostsPerProcess:95 DataSources:254876 RRDsProcessed:92164
05/04/2016 03:05:27 PM - SYSTEM STATS: Time:324.9506 Method:spine Processes:24 Threads:20 Hosts:2262 HostsPerProcess:95 DataSources:254876 RRDsProcessed:92176
05/04/2016 02:58:41 PM - SYSTEM STATS: Time:218.8851 Method:spine Processes:24 Threads:20 Hosts:2262 HostsPerProcess:95 DataSources:254876 RRDsProcessed:92194
05/04/2016 02:53:43 PM - SYSTEM STATS: Time:221.7222 Method:spine Processes:24 Threads:20 Hosts:2262 HostsPerProcess:95 DataSources:254876 RRDsProcessed:92183
05/04/2016 02:48:38 PM - SYSTEM STATS: Time:216.6530 Method:spine Processes:24 Threads:20 Hosts:2262 HostsPerProcess:95 DataSources:254876 RRDsProcessed:92133
05/04/2016 02:43:41 PM - SYSTEM STATS: Time:219.5430 Method:spine Processes:24 Threads:20 Hosts:2262 HostsPerProcess:95 DataSources:254876 RRDsProcessed:92177
05/04/2016 02:39:56 PM - SYSTEM STATS: Time:294.3475 Method:spine Processes:24 Threads:20 Hosts:2262 HostsPerProcess:95 DataSources:254876 RRDsProcessed:92147
05/04/2016 02:33:47 PM - SYSTEM STATS: Time:225.3324 Method:spine Processes:24 Threads:20 Hosts:2262 HostsPerProcess:95 DataSources:254876 RRDsProcessed:92177

Cacti poller is scheduled every 5 mins. I am using a good configuration box here. 24 CPU, 6 Cores each, 64GB memory.

After visiting few topics on forum, it seems like Cacti multipoller plugin can help to improve performance here.
Can it be possible to configure multiple poller in a single server so that I can assign each poller to a class of network devices? This will help to complete polling time within 300 sec?

Is it possible to install Cacti multi-poller on Cacti 0.8.8a or I need to upgrade Cacti first to install this plugin?

[phalek]

The Multipoller plugin will only help if your issue is with the actual polling of devices.
If your issue comes from the rrd updates (high I/O), then you may be better of using Boost or go for high-speed SSD drives.

[roxy]

Hi Phalek,

I have already explained my issue in the below forum.

http://forums.cacti.net/viewtopic.php?f=21&t=55951

I have already tried with Boost plugin but did not find any improvement in performance. So I want to go for Multi poller implementation.

[der_doc]

Hi roxy,

I think if you have more from this cacti server the plugin can help. I have code them as the same reason. The cacti envirement was to big for one mashine.
With this plug in the border is the hardware or the developer if the updates are more less then soon as possible

Please let me know if you are happy with the plugin.

Best Regards
Der_Doc

[kemxoi]

Hi all,
I have 1 server use cacti 0.8.8h installed multi-pollerserver plugin (ci231-53). I configured 2 poller servers (ci231-53 and ci160-84)


But poller in ci160-84 server not run. I don't know reason. Has cacti and spine on poller server must use mysql of master server? How could master server call poller on other server?



Many thanks!

[der_doc]

Hi,

ok there are any thinks to noted.

First you must install cacti (for example 0.8.8.h) and the plugin (0.8.8.h) at all your servers! Or install at one and share the files via NFS.
Then you must use only one database and release them for the serve (i called him Poller2). (look at remote access mysql)
At every server you you enter the ipadress from the databaseserver to the global.php file.
Then reboot both and i hope xthe plugin run.

So Poller01 has got the mysqlserver, cacti and the plugin installed. Poller02 installed cacti and the plugin and use the database from Poller01


The documetation for the ubuntu installation with cacti:
http://www.cacti-multipollerserver.de/i ... l?Itemid=0

Cu
André

[kemxoi]

Hi,

ok there are any thinks to noted.

First you must install cacti (for example 0.8.8.h) and the plugin (0.8.8.h) at all your servers! Or install at one and share the files via NFS.
Then you must use only one database and release them for the serve (i called him Poller2). (look at remote access mysql)
At every server you you enter the ipadress from the databaseserver to the global.php file.
Then reboot both and i hope xthe plugin run.

So Poller01 has got the mysqlserver, cacti and the plugin installed. Poller02 installed cacti and the plugin and use the database from Poller01


The documetation for the ubuntu installation with cacti:
http://www.cacti-multipollerserver.de/i ... l?Itemid=0

Cu
André

Hi André
Plugin multipoller on 2 servers have just run and I have more question for you On Poller01, I want to view graph of device X which uses Poller02. Do I have copy rrd files of device X from rra directory on Poller02 to Poller01 right? Or I have use share storage for rra directory? Do you have another solution for this issue?
Thank for your help,

[der_doc]

Hi,

on "my" master I have istalled the follow:
- SQL Server
- NFS Server (share Cacti directory, log directory, and rrd directory)

On all other poller I have installes the follow:
- SQL Client
- NFS Client

So I can logon on every poller and I see on all the same.

I think there are many configuration options, every poller has only the rrd files how polls or or or.
My sample configuration is descriped at the document at my homepage. http://www.cacti-multipollerserver.de/i ... l?Itemid=0
But all poller must work on the same database or you must config a database cluster, this worked in my old company very good.

MfG
André

[kemxoi]

Hi,

on "my" master I have istalled the follow:
- SQL Server
- NFS Server (share Cacti directory, log directory, and rrd directory)

On all other poller I have installes the follow:
- SQL Client
- NFS Client

So I can logon on every poller and I see on all the same.

I think there are many configuration options, every poller has only the rrd files how polls or or or.
My sample configuration is descriped at the document at my homepage. http://www.cacti-multipollerserver.de/i ... l?Itemid=0
But all poller must work on the same database or you must config a database cluster, this worked in my old company very good.

MfG
André

Hi André!
Yeap, I have understood and will try to config NFS on Poller01 now I don't know why on other poller have use SQL client? Does mean database cluster?
Thank you so much, André !