Syslog monitor addon beta

[harlequin]

<Modified 2005-12-03> ver 0.1.2b has been posted - please check further down in this topic

h.aloe is a modified version of sidewinder's aloe addon.
It's been completely revamped and updated to work with Cacti 8.6g

In brief, it's a Cacti addon that provides a color-coded, searchable front-end for a mysql syslog / eventlog database [The database can be populated by Kiwi's syslog daemon, syslog-ng, etc...].
It includes an option to integrate with Cacti's graph timespan, so you can correlate graphed items with syslog events, and has an option to output filtered data to a comma delimited text file.

• Installation Level: (Easy)
  Installation Time: 5 Minutes
  Files To Edit: 4

thanks to sidewinder for the original aloe: http://forums.cacti.net/viewtopic.php?t=3993

This is a beta version. Comments, criticisms, additions, etc. are welcome, but don't blame me if it breaks something (unlikely) or doesn't work (more likely)

Hope it's useful to someone. Cheers,
Harlequin

[TheWitness]

Niiiiicccceee. Can we integrate into the full product?

TheWitness

[Phobos182]

Bravo.

[harlequin]

Niiiiicccceee. Can we integrate into the full product?

TheWitness

Absolutely. You may want to check the code over - it's pobably a mite bit sloppy... Several things could be simplified/improved with better integration, but I tried to modify Cacti files as little as possible. Glad you like it
Harlequin

[rony]

Accually, I will be contacting you about writing it as a plugin for 0.9.0.

No planned intergration into 0.8.6.

That doesn't stop you from offering it as a addon for 0.8.6, just will not be intergrated into the 0.8.6 code tree..

[tommyj]

Looks amazing! I got it up&running but I don't get the nice color coding, how do I get that? I'm using syslog-ng for information.

Also, how about some tail -f function, would that be possible to implement?

[harlequin]

No planned intergration into 0.8.6.

That was my assumption. Let me know about the 0.9.0 plugin

Looks amazing! I got it up&running but I don't get the nice color coding, how do I get that?

Thanks much. Edit the ./include/haloe-config.php file and change the ["names"] in the color section to match what shows in your 'priorities' dropdown. For example, if you have a priority listed as 'emerg', then change
$haloe_colors["Emergency"] = "FF0000"; to
$haloe_colors["emerg"] = "FF0000";
Let me know if that helps.

Also, how about some tail -f function, would that be possible to implement?

Hmmm. As it reads and sorts from a database, not really, but it basically does the same thing with the meta-refresh. You could shorten the refresh time to reload the page every couple seconds - currently it pulls this from whatever you have set for your graph refresh time - I've included files with changes for a separate setting for the syslogs refresh rate - just replace the files and edit the new setting in haloe-config.php to your liking.
Harlequin

[tommyj]

Thanks much. Edit the ./include/haloe-config.php file and change the ["names"] in the color section to match what shows in your 'priorities' dropdown. For example, if you have a priority listed as 'emerg', then change
$haloe_colors["Emergency"] = "FF0000"; to
$haloe_colors["emerg"] = "FF0000";
Let me know if that helps.

Yes, that helped, an easy one. Don't know how I could miss that . Thanks a lot!

Another thing, would it be possible to show all entries above one severity level so it shows all entries except for example info or debug messages?

[harlequin]

Glad you got the colors fixed - I should probably document that a bit better.
I'll look into adding an 'and above' option to the priority select - makes sense. Cheers,
Harlequin

[Devil]

harelquin really cool add-on.

i get the following errors when i load the syslog page:

Code: Select all

Notice: Undefined index: haloe_pdt_change in /usr/share/webapps/cacti/0.8.6f-r1/htdocs/haloe.php on line 38

Notice: Undefined index: button_clear_x in /usr/share/webapps/cacti/0.8.6f-r1/htdocs/haloe.php on line 46

and it would be nice to have a documentation tha told me how to add hosts to monitor.

Regards
Devil

[harlequin]

Thanks. For a quick fix on the 'Notice: Undefined...' errors, you could try editing your php.ini file and setting:
display_errors = Off
(this should be Off in a production server anyway) - or - setting:
error_reporting = E_ALL & ~E_NOTICE
(not really recommended in a production server, but it'll do for a test environment).
I will fix this in the next release. If you can't change the php.ini file, let me know and I will send you a 'hack' fix.

it would be nice to have a documentation tha told me how to add hosts to monitor

Hosts are pulled from whatever is in the haloe/syslog database - any hosts that are sending logging to the db will be in the list (you need to be use an external application like Kiwi syslog deamon or syslog-ng to collect syslog info and populate the database). Hope that helps...
Harlequin

[cigamit]

Thanks. For a quick fix on the 'Notice: Undefined...' errors, you could try editing your php.ini file and setting:
display_errors = Off
(this should be Off in a production server anyway) - or - setting:
error_reporting = E_ALL & ~E_NOTICE
(not really recommended in a production server, but it'll do for a test environment).
I will fix this in the next release. If you can't change the php.ini file, let me know and I will send you a 'hack' fix.

it would be nice to have a documentation tha told me how to add hosts to monitor

Hosts are pulled from whatever is in the haloe/syslog database - any hosts that are sending logging to the db will be in the list (you need to be use an external application like Kiwi syslog deamon or syslog-ng to collect syslog info and populate the database). Hope that helps...
Harlequin

First off, I would like to say thanks for the great add-on. Its very similar to the Syslog plugin I have been working on (but not even close to finishing with the everything else I have to do).

I hope you don't mind, but I took the liberty of converting your add-on into the plugins format, it really only took about 15 minutes to do. I have also added the setting for custom refresh time. I went ahead and fixed several index errors (its good practice to disable E_ALL in production, but its also good practice to code with it on). I also fixed several other minor issues. It wasn't correctly outputting to file format for me (no database call), and the page selector was passing a variable that didn't exist.

This is fairly close to what I been hoping to for. I do see a few features that I would like see eventually added. Mainly I am looking at writing another script that runs every 5 minutes (right after normal pollings) which goes through and scans all "new" events and searches for specified ones to alert on (using user customized regex or just simple string comparisons). Possibly at the same time, have it go through and purge different ones from the database that we don't deem important (same regex concept), and also purge all events that are over XX days old (simple setting).

Overall, its looking really nice so far, and I hope you keep up the good work!

[Devil]

I installed cigamits modified version and now it works like a charm.

Just one little thing. could you change so that to time field says now instead of a specific time. then it works better.

[egarnel]

This is awesome.
I was wondering if I could get a little assistance with the syslog-ng setup?

Here is the syslog-ng.conf to push into the haloe db:

Code: Select all

# Log syslog-ng to mysql database
                                    ##
                                        destination d_mysql {
                                            pipe("/tmp/mysql.pipe"
                                            template("INSERT INTO logs (host, facility, priority, level, tag, date,
                                            time, program, msg,seq) VALUES ( '$HOST', '$SEQ',  '$PROGRAM', '$TIME', '$DATE', '$PRIORITY',
                                            '$FACILITY') ;\n") template-escape(yes));
                                             };
                                        log { source(net); destination(d_mysql);
                                        };

and here is the fifo to route syslog messages into syslog-ng

Code: Select all

#!/bin/bash

if [ -e /tmp/mysql.pipe ]; then
        while [ -e /tmp/mysql.pipe ]
                do
                        mysql -u haloe --password=haloepassword haloe < /tmp/mysql.pipe
        done
else
        mkfifo /tmp/mysql.pipe
fi

The logs table never gets populated for some reason....

Thanks for your help

[Devil]

you have some errors in you syslog-ng config.

Code: Select all

# Log syslog-ng to mysql database
                                    ##
                                        destination d_mysql {
                                            pipe("/tmp/mysql.pipe"
                                            template("INSERT INTO logs (host, facility, priority, level, tag, date,
                                            time, program, msg,seq) VALUES ( '$HOST', '$SEQ',  '$PROGRAM', '$TIME', '$DATE', '$PRIORITY',
                                            '$FACILITY') ;\n") template-escape(yes));
                                             };
                                        log { source(net); destination(d_mysql);
                                        };

should be changed to:

Code: Select all

destination d_mysql { 
pipe("/var/log/mysql.pipe" 
template("INSERT INTO logs (host, facility, priority, level, tag, date, time, program, msg) 
VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY', '$HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)); };

You see you have to match the variables with the columns in the table (basic sql). The data get inserted in the wrong columns with your declaration.

How have you declared the source net in sysloc-ng.conf?
have you created the fifo file?
have you restated the syslog-ng process?