Syslog monitor addon beta

[magoo95]

Hi,

In order to enhance this wonderfull plugin, is there a chance to be able to use the "threshold" plugin with this one?

I mean to be able to automatically send an email (by exemple) on particular syslog events received in the "haloe" database.
or send also email if you receive 3 times the same message during 1 minute from 1 equipement.
This could be very helpuff to control access list on network equipments or hosts.

Cacti will become, finally a kind of "Unicenter TNG" from CA .... and more ...

[cigamit]

Hi,

In order to enhance this wonderfull plugin, is there a chance to be able to use the "threshold" plugin with this one?

I mean to be able to automatically send an email (by exemple) on particular syslog events received in the "haloe" database.
or send also email if you receive 3 times the same message during 1 minute from 1 equipement.
This could be very helpuff to control access list on network equipments or hosts.

Cacti will become, finally a kind of "Unicenter TNG" from CA .... and more ...

As I mentioned in my other post, this is actually what I am working on. You won't need the Threshold plugin at all actually, since everything will be built into haloe. Once its done, I'll submit the changes.

[kingaru]

All,

Since this script dumps evrything in to one single database this database may become so huge especially if you have quire few routers on the network which may generate up to 50-100 Mb of logs daily. Is there any possibility to have to generate kind of "daily" databases" which could be picked up by script for analisys if that paricular date was selected ....?

Thanks
Igor

[mpdsville1]

Note: Some of the feature requests i've noted in this thread i'm
handling at my site with syslog-ng/mysql on the back end and
php-syslog-ng on the front. It mightn't be a bad idea to
take a look at php-syslog-ng and how it behaves, rather than
reinventing the wheel under a cacti tab. Just a thought

Triggering emails, I have syslog-ng tee off to a flat file
in temp and use logsurfer to trigger emails . I trash the
flat file nightly, with the long term logs stored in mysql

Log rolling.. syslog-ng script exists. I roll my syslog
databases monthly. and the php interface allows spanning
of databases thru a merge table for searches..

Maybe just porting syslog-ng into a cacti tab would save
alot of development cycles, or look it over for ideas..
Just a thought..

[egarnel]

this would work for me. I could then make use of the already setup php-syslog-ng server we have. For some reason, the fifo on the cacti server is not cooperating & I haven't had time to really chase it down

[torstentfk]

Hi,

we get 32k syslogs per day. It would be very nice to have a rolling in the mysql db for lets say one week so that the db does not grow to 1TB of data. We do not use php-syslog-ng to maintain mysql-syslogs. The backupfunktion of syslog-ng as found on the websites coulb be used...

Torsten

[harlequin]

Hi all,
Again, thanks for all the feedback & suggestions - please keep them coming.

Sorry for the long-ish post, but I wanted to clarify some of my thoughts h.aloe:

The concept behind h.aloe was to provide a cross-platform syslog frontend that allowed easy viewing/searching of syslog data, as well as quick comparison between syslog and graph events.

One of my main design criteria for h.aloe was that it should be able to use a separate (and quite likely pre-existing) syslog database without altering that database at all - I wanted this both for maximum compatibility with as many syslog-to-mysql schemes as possible, as well as to leave open the ability to utilize that same database with other software, both now and in the future.
I feel that the backend is, and should remain, pretty much outside the scope of h.aloe. Keeping that in mind, I do not want to marry h.aloe to syslog-ng (or any other syslog server) specifically. [Since Cacti requires mysql, I don't mind limiting h.aloe to that, but otherwise I'd like to keep it as open as possible]
While it does make development on this side a bit more difficult, I feel it will ultimately result in a more usable and robust piece of software.

Having said all that, I realize that the plugin version currently incorporates the syslog database into Cacti's - all I've done here was merge cigamit's excellent work into the updates I've been making. In the future I would like to separate the syslog db back out, with a separate config bit in Cacti's db, but for now I have held off serious development on the plugin side until the plugin architecture is part of the official Cacti release.
[Depending on when 0.9.0 is released, this may well change - I'll be talking to the developers & cigamit on this...]


As for rotation/archiving - while this is strictly speaking a backend function (and there are any number of scripts/tools out there for this), for the sake of simplicity I will post a file for this when I get a chance (unless someone else wants to deal with this part of it - hint, hint ). However, as my spare time is quite limited at the moment, I will be concentrating most of it on core h.aloe updates - if someone else wants to take this bit on, please let me know.
Requirements would be:
- no interruption of current logging
- platform-independant (should work as a cron job or win32 scheduled-task, like poller.php)
- configurable time-of-backup/prune (like rrd, this would have to match cron/s-t scheduling)
- configurable no. of backups to keep (compression [configurable] would be nice, too)
- ideally, I think this should be pretty much a separate 'drop-in' entity - modifications to Cacti & h.aloe should not be required


Other than that:
- I'm (still) working on user-restriction.
- In order to avoid redundancy, I'll probably hold off on notification for awhile
[looks like this will be incorporated into Cacti 0.9.0, and I think cigamit has already written this part anyway - again, I'll get with the developers/cigamit on this]

I'm sure I've missed a ton of things, but paid-programming has been quietly, insidiously, eating my brain in the background for awhile and I'm bloody tired - time for a stiff drink and a couple hours' sleep...

Cheers,
harlequin

[farhan]

Hi harlequin,

Be thankful for for your efforts.


Farhan

[farhan]

Hi harlequin,

Sorry for mistyping in my previous message. I appreciate your efforts.

Thanks,

Farhan

[eddievenus]

I agree, harlequin, your efforts are much appreciated. And I for one respect the time and effort put into this since I use this plug in almost as much as I use Cacti in general. In fact I have noticed that Cacti is so much better now with this plugin that I laugh when I see other admins just doing SNMP polling with no syslog or SNMP trap capacity. Suckers.

I do however have one request. I noticed this brought up before as well. That is the ability to delete. I think a little check box next to the message would be a great start. With a little box (or drop down menu) at the bottom that says delete. Just like on many of the Cacti config pages. This way all those annoying messages that are too sparse to filter out before hand can be caught and removed easily and quickly. Plus this will help me find all the message types that I do not want to have clogging up my database. I try to filter what syslog messages I should get, but that only works so well.

I know that you could take the idea of a delete button to much more inclusive depths than just on a per message basis, but for now that would be more than sufficient.

Thanks again for this fine piece of work. And also many thanks to Cigamit for the plugin work done on this.

[eddievenus]

Also 1 minor issue I found is that the message text filter does not take commas. If I put this string in

Interface Serial0, changed

it changes it to

Interface Serial0 changed

and does not find anything. I know that this is to sanitize the string that gets sent to the database, but there should be a way to put commas in. They are
just replaced with blank spaces, as are many other delimiters I tried. However commas are common enough in the text that it would be pretty common to find a need to search for them.

Also it does not remove blank spaces at the end of the search string. So
'Interface Serial0 ' does not work where 'Interface Serial0' does. The single quotes are just to show the trailing space, they are not part of the search string.

and seriously these issues are minor, very minor. I figured you would want to know, since this is still in beta.

eddievenus

[cigamit]

I agree, harlequin, your efforts are much appreciated. And I for one respect the time and effort put into this since I use this plug in almost as much as I use Cacti in general. In fact I have noticed that Cacti is so much better now with this plugin that I laugh when I see other admins just doing SNMP polling with no syslog or SNMP trap capacity. Suckers.

I do however have one request. I noticed this brought up before as well. That is the ability to delete. I think a little check box next to the message would be a great start. With a little box (or drop down menu) at the bottom that says delete. Just like on many of the Cacti config pages. This way all those annoying messages that are too sparse to filter out before hand can be caught and removed easily and quickly. Plus this will help me find all the message types that I do not want to have clogging up my database. I try to filter what syslog messages I should get, but that only works so well.

I know that you could take the idea of a delete button to much more inclusive depths than just on a per message basis, but for now that would be more than sufficient.

Thanks again for this fine piece of work. And also many thanks to Cigamit for the plugin work done on this.

I have now completed the alerting and deleting portion of the plugin, I did it on an older version, so I now have to import the last changes that were made to the syslog module. Once that is done, I will repost it here.

If your curious enough to want it before hand, I have released a Cacti CD that currently in beta testing (auto-installs CentOS with Cacti), but it happens to include the syslog plugin and msyslog all up and running. You should be able to find it on my site.

[eddievenus]

great,

I have been deleting by entering SQL commands right to MYSQL, but it is nice to have the delete option right in the gui. For the small stuff (like single records) entering commands via SQL syntax gets pretty unruley. I will be glad when I can easily delete individual records without typing it all out.

[cigamit]

great,

I have been deleting by entering SQL commands right to MYSQL, but it is nice to have the delete option right in the gui. For the small stuff (like single records) entering commands via SQL syntax gets pretty unruley. I will be glad when I can easily delete individual records without typing it all out.

Currently it deletes using wildcards so you could remove all of them in one go, if you need to specifically delete 1 record, while leaving all others that match it there, then I could see the possibility of adding that too. All it would require is a few extra lines of code.

For instance, currently you create a rule like so

Code: Select all

Delete messages that being with "crond[%]:%/poller.php"

will delete all message like this

Code: Select all

crond[4707]: (root) CMD (php /var/www/html/poller.php > /dev/null 2>&1)

which saves a lot of time.

[AlexK]

Thanks a lot for the wonderful plugin. I upgraded v1.1b to 1.2b to enable 'putput to file' but it is still not functioning. Whenever I do it, it outputs an empty file. Is 'output to file' bug still pending ?


Thanks!!!

Alex