Syslog Plugin 2.1 backup(traditional) / restore(partitioned)

[chrisgsmith40]

Thanks again netniV for your help on thread viewtopic.php?f=14&t=58812 - unfortunately, need more direction, specifically ...

[*] WORKING: Syslog Plugin 2.1 backup (traditional) / restore (traditional) [my install, can see both files "syslog.MYI" and "syslog.MYD" in /var/lib/mysql/syslog GROWING mysql redirect-in]
[*] WORKING: Syslog Plugin 2.1 backup (partitioned) / restore (partitioned) [via mariabackup @ https://mariadb.com/kb/en/library/full- ... db-backup/ ]
[*] LOST: Syslog Plugin 2.1 backup (traditional) / restore (partitioned)

In that same /var/lib/mysql/syslog I can see all the "waiting" partitioned files like ... syslog_removed#P#d20180421.ibd, syslog_removed#P#d20180420.ibd, syslog_removed#P#d20180419.ibd, etc...

And can see through text searching the plugin's PHP source, file functions.php has functions for managing "in process" partitions , etc...

Am guessing / assuming a massage of something like the following link is needed / required? @ http://www.dbastuff.net/2014/09/mysql-d ... tions.html - but grateful for "checking in with you / others" before going off into the weeds if there's an easier way?

Regards,
Christopher

[netniV]

I'll be honest, I'm not sure as I'm not really up to speed on MySQL partitioning. I would suspect however, that in order to take a single partition and make it multiple there would be some work. I'm not sure you would be able to do that once data is populated as that would surely cause confusion between where the data should and is sitting. If the dump and reload works, it's probably a safer option anyway.

[chrisgsmith40]

Thanks netniV - this is telling me it hasn’t been approached/doco’d yet - I did have a “waking dream”, of how it can be done, am going to try and re-use/reference what’s already in the plugins *.php - #NoobOnAMission

[chrisgsmith40]

I think I got it - see attached file! Works for my circumstances, - USE AT YOUR OWN RISK - obviously, needs a serious code review!

Regards

syslog-migration.zip

One-way migration from 'traditional' to 'partitioned' SYSLOG

(4.03 KiB) Downloaded 74 times

[chrisgsmith40]

For those who care, small "post script"

BEFORE running my noob script, I would suggest TRUNCATE of the following tables from the database you're COPYING FROM ...

TRUNCATE TABLE syslog_facilities;
TRUNCATE TABLE syslog_host_facilities;
TRUNCATE TABLE syslog_hosts;
TRUNCATE TABLE syslog_programs;
TRUNCATE TABLE syslog_priorities;
TRUNCATE TABLE syslog_remove;
TRUNCATE TABLE syslog_removed;

After that, the assumption is that database you're COPYING TO has been set up, BUT the Cacti SYSLOG Plug-IN has not been enabled yet

Run my noob script - even though the script finishes within about 30min, my MariaDB-server was still "processing the partitions" for another 1hr - I just watched "top" until CPUs were idle.

Then, while in the Cacti SYSLOG Plugin directory ...

php ./syslog-process.php

(This is to primarily to process any large number of waiting rows in syslog_incoming)

Obviously, if you have your syslog / rsyslog daemon set to dump into syslog_incoming, you won't get this "completely" emptied

Then, go ahead and enable the Plug-In - and you're off