The CACTI mirror setup guide (Apache / MySQL)

[pestilence]

This is a small guide i decided to writte after my search for mirroring information (low cost, just for reduntant reasons).
This guide requires MySQL version 4.0.x, rsync installed and finally Apache (doh...).
Start by setting up cacti on the master server (in case you haven't done so yet). After setting up cacti its time to create the slave server for data replication.
While still logged on the Master Server perform the following steps to setup MySQL replication
1) This creates the directories and sets the permissions so only the mysql daemon can read/write to the logs.

Code: Select all

$ mkdir /var/log/mysql/binary 
$ chown mysql: /var/log/mysql/binary 
$ chmod 770 /var/log/mysql/binary

2) Edit the MySQL Configuration.
This turns on the binary logging, tells it where to store the logs, and creates a server id.

Code: Select all

$ vim /etc/mysql/my.cnf

3) Add under the [mysqld] section.
It doesn't matter what the value for server-id is, as long as it is different from the slave's (which you will set later).
Also make sure that "bind ip_address" and "skip-networking" are commented out, so that the slave user can log in remotely.

Code: Select all

log-bin=/var/log/mysql/binary/mysql_binary_log 
binlog-do-db=database_name 
server-id=1

4) Once you're done, restart the mysql server for the changes to take effect, and for the logging to start.

Code: Select all

/etc/init.d/mysql restart

NOTE: If you were already using binary logging before this, you don't need to make any changes, except for adding a server-id. The slave can keep up with your current binary logs, so there's no reason to delete them or flush the logs.

5) Grant Replication of the Database to the Slave database (this is done still in on the Master server).

Code: Select all

# mysql -u db_admin -p
mysql> GRANT REPLICATION SLAVE ON database_name TO 'slave_user_name'@'hostname_or_ipaddress' IDENTIFIED BY 'slave_password';

6) Get a database dump.
Before getting the database dump you need to lock the tables to prevent any updates.

Code: Select all

mysql> USE database_name; 
mysql> FLUSH TABLES WITH READ LOCK;
mysql> SHOW MASTER STATUS;

The SHOW MASTER STATUS will return a table like the following:

Code: Select all

+----------------------+----------+--------------+------------------+ 
| File                 | Position | Binlog_do_db | Binlog_ignore_db | 
+----------------------+----------+--------------+------------------+ 
| mysql_binary_log.025 | 796947   | database_name|                  | 
+----------------------+----------+--------------+------------------+ 
1 row in set (0.00 sec)

WHILE STILL LOGGED INSIDE THE mysqlclient OPEN ANOTHER SHELL TO THE MASTER SERVER TO AQUIRE THE BACKUP

$ mysqldump --opt database_name > database_name.sql

7) Unlock the tables
From the original shell where you locked the tables:

Code: Select all

mysql> UNLOCK TABLES;

This are the steps on the Master Server required to replicate the Database Data.
Now on to the Slave server for the rest of the replication setup:

1) Import the mysqldump from the master server:

Code: Select all

mysql database_name < database_name.sql 

2) Edit the MySQL config file on the slave server.

Code: Select all

$ vim /etc/mysql/my.cnf

Add the following info so that our server knows that he is a slave server.

Add under the [mysqld] section.
It doesn't matter what the value for server-id is, as long as it is different from the master's.

Code: Select all

server-id=2 
# master server settings 
master-host=masterhost.com 
master-user=slave_user_name 
master-password=slave_password 
master-connect-retry=60 # num of seconds, default is 60 
replicate-do-db=database_name
replicate-ignore-table=cacti.poller_reindex
replicate-ignore-table=cacti.poller_output
replicate-ignore-table=cacti.poller_time

The last 3 lines (replicate-ignore-table) should be added so that the slave DB works properlly with its own poller (Master will try to send poller data to Slave and if you are running a poller on the Slave it will try to update).


3) Restart the MySQL server (This does not start the slaving yet)

Code: Select all

/etc/init.d/mysql restart

4) Tell the slave specifically where to start logging

This is where you use the data from SHOW MASTER STATUS; from the master mysql database.

Code: Select all

mysql> CHANGE MASTER TO MASTER_HOST='master_host_name', MASTER_USER='slave_user_name', MASTER_PASSWORD='slave_password', MASTER_LOG_FILE='mysql_binary_log.025', MASTER_LOG_POS=796947; 

Note that you don't put quotes around the value for MASTER_LOG_POS or mysql will give you an error.

5) Start the Slave Process

Code: Select all

mysql> START SLAVE;

Now you slaving should have been started, To check if everything is ok proceed with the following commands:

On the Master Server

Code: Select all

mysql> SHOW PROCESSLIST;

The above command should return a table with some information check for the field State
It should be reading:

Has sent all binlog to slave; waiting for binlog to be updated

This means that our Master Server has sent all the updates to the slave server and that its waiting for the data to update (inside the DB) to replicate the updates again.

On the slave server

Code: Select all

mysql> SHOW PROCESSLIST;

Again you should be looking after the State field. It should have an entry:

Waiting for master to send event

If it says Connecting then there is something wrong with the configuration you created or the username password you are using.


NOW ON WITH FILE REPLICATION.

This part is not mandatory, allthough cacti's database are in sync the poller will not have any problems fetching the polled items from the Database and then proceed with polling the hosts.
A initial replication should ok so that the files do get synced on each host.
To proceed with this replication i used the "cheap" way of rsync. I setup an rsync daemon on the Master Server and then proceeded with a rsync process from the Slave server to get the files synced.
This step as i mentioned above is only needed 1 time, since all data used by the poller is inside the MySQL server, if you are using any custom scripts just make sure the scripts are also available on the slave server and they are inside the cronjob (Unless you are using the slave sever only for backup purposes and you don't mind if data are up to date).

[rony]

NOW ON WITH FILE REPLICATION.

This part is not mandatory, allthough cacti's database are in sync the poller will not have any problems fetching the polled items from the Database and then proceed with polling the hosts.
A initial replication should ok so that the files do get synced on each host.
To proceed with this replication i used the "cheap" way of rsync. I setup an rsync daemon on the Master Server and then proceeded with a rsync process from the Slave server to get the files synced.
This step as i mentioned above is only needed 1 time, since all data used by the poller is inside the MySQL server, if you are using any custom scripts just make sure the scripts are also available on the slave server and they are inside the cronjob (Unless you are using the slave sever only for backup purposes and you don't mind if data are up to date).

Not correct. You do need to continuely rsync the rra directory between the servers, otherwise you statistical data will not be saved. This is because Cacti does not store the information that you see in the graphs in the database, but in RRD files in the rra directory.

Also, you would not want to run the poller on both machines, only the master. Running the poller on both machines would causes issues with the poller_output table and duplicate entries durning polling.

[fmangeant]

Hi pestilence

thanks for this howto, I was just working on MySQL replication

I just had to run

Code: Select all

mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.01 sec)

after granting the 'replication' right to my user.

[pestilence]

NOW ON WITH FILE REPLICATION.

This part is not mandatory, allthough cacti's database are in sync the poller will not have any problems fetching the polled items from the Database and then proceed with polling the hosts.
A initial replication should ok so that the files do get synced on each host.
To proceed with this replication i used the "cheap" way of rsync. I setup an rsync daemon on the Master Server and then proceeded with a rsync process from the Slave server to get the files synced.
This step as i mentioned above is only needed 1 time, since all data used by the poller is inside the MySQL server, if you are using any custom scripts just make sure the scripts are also available on the slave server and they are inside the cronjob (Unless you are using the slave sever only for backup purposes and you don't mind if data are up to date).

Not correct. You do need to continuely rsync the rra directory between the servers, otherwise you statistical data will not be saved. This is because Cacti does not store the information that you see in the graphs in the database, but in RRD files in the rra directory.

Also, you would not want to run the poller on both machines, only the master. Running the poller on both machines would causes issues with the poller_output table and duplicate entries durning polling.

I Added the poller ignore tables above in my working enviroment i use both pollers to avoid rsync procedures due to extreme file count
I only used initial rsync to replicate rrd files to the slave server and from there on i started running the poller on the Slave server as well.

[Pumpi]

Hi pestilence,

interesting guide to mirror two Cacti servers.

Do you know a solution to mirror the graph pages of the main Cacti server to second web server which is located in a DMZ to let customer have access from outside ?

Perhaps the Cacti developer team knows such a solution ?

[fmangeant]

Do you know a solution to mirror the graph pages of the main Cacti server to second web server which is located in a DMZ to let customer have access from outside ?

Hi Pumpi

we're doing this here, by using Apache's reverse proxy : http://httpd.apache.org/docs/mod/mod_proxy.html

[Pumpi]

Hi fmangeant,

thank's for this pointer. I hope I get this solution working. I just started my Linux and LAMPP knowledge with the genial Cacti solution.

Polling today 84 hosts with 1136 data sources but perhaps this solution will be moved to production and then I need a backup Cacti server and a reverse proxy server solution.

[bulek]

I only used initial rsync to replicate rrd files to the slave server and from there on i started running the poller on the Slave server as well.

Do I miss something here? This way you are double polling all your devices... or?

- Piotr

[pestilence]

I only used initial rsync to replicate rrd files to the slave server and from there on i started running the poller on the Slave server as well.

Do I miss something here? This way you are double polling all your devices... or?

- Piotr

I was double polling my devices from the beggining, i just needed a way so that cacti changes (such as tree, new graphs, deletion, DB modifications, new devices etc) would replicate to the second server without having to manually change it.
If you do want to replicate also the files then you can always disable the poller, replicate the whole DB and just create an rsync script to sync the files every 5 minutes...

[kbartoletta]

pestilence,

Is the solution that you mentioned above still working for you. I have been trying to setup the same solution but I am having problems getting my poller to poll the SNMP devices - when run it keeps saying the devices are down and not answering to SNMP but they are ... I can do snmp gets from the cacti server to the devices all day long...?

Did you start with two independant cacti server, both polling, admining both and then switch to replication?

[thenoob]

Thanks for your tutorial !

[gandalf]

Moving to HowTo's
R.

[great__F]

Hi together,

I have a Setup with 3 Server in Master-SLAVE-SLAVE Mode.

ervery 20 Minutes the Pollers lost the Communication to the PHP Server:

03/12/2015 04:50:07 PM - SPINE: Poller[3] ERROR: SS[0] PHP Script Server communications lost. Restarting PHP Script Server
03/12/2015 04:50:07 PM - SPINE: Poller[3] ERROR: SS[0] Script Server did not start properly return message was: 'U'

Do someone know whats the Issue of this? Is there a timeout Problem?

Maybe someone had this Issue and can give me a Hint.

Regards
Great__F