Syslog monitor addon beta

jherrinike · Post by **jherrinike** » Sat Jul 01, 2006 6:19 pm

I just wanted to check and see if someone could point me in the right direction on how to solve a problem I am having getting this plugin to work. The plugin itself seems fine. The reading I have done in this message board makes me think I am having a problem with getting my syslog data passed through to my database.

I have installed syslog-ng and working as far as creating text log files from my network devices. I then added the following to my syslog-ng.conf file:

# pipe messages to /var/log/mysql.pipe to be processed by mysql
source src {unix-stream("/dev/log");
pipe("/proc/kmsg");
internal();};
destination d_mysql { pipe("/var/log/mysql.pipe" template("INSERT INTO logs
(host, facility, priority, level, tag, datetime, program, msg)
VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC',
'$PROGRAM', '$MSG' );\n") template-escape(yes)); };

log { source(src); destination(d_mysql); };

I created a pipe file using: mkfifo /var/log/mysql.pipe

Since I was really not certain about the correct database structure I created a database called syslog and the imported the syslog.sql file from the plugin into it. I also gave it the same user right as my cacti database.

I then used the following script file in order to start it and I changed the user info and paths to match my system.

#!/bin/bash

if [ -e /var/log/mysql.pipe ]; then
while [ -e /var/log/mysql.pipe ]
do
mysql -u cactiuser --password=cactipw syslog < /var/log/mysql.pipe
done
else
mkfifo /var/log/mysql.pipe
fi

When I execute the script either through crontab or manually it just hangs and no data gets into the database or into the pipe file from what I can see.

I am running on CentOS 4 with MYSQL 5.0.22

Anyone have any ideas on what I might be overlooking or things to check?

Thanks for the help

jarhead · Post by **jarhead** » Tue Jul 18, 2006 10:27 am

All,
I have a box freebsd 5.4 P16 that is running apache 1.3 mod ssl and mysql 4.1 with php5. I have all of cacti running fine with the exception of this syslog plugin. I have a working syslog-ng install. All my logs are going into the the correct files in /var/log according to the config. I have a destination setup like the examples for writing to a fifo file. I ran mkfifo /tmp/mysql.pipe and then restart syslog-ng. The permissions on the mysql.pipe file change when syslog-ng starts but there is no data going there.

If I do a tcpdump port 514 I can see the data coming in. But if I do a cat /tmp/mysql.pipe there is nothing. I can not see where the issue lies with the data not ending up in the pipe. there are no errors that I can see. Please someone help me get my data into the pipe.

Thanks

chercen · Post by **chercen** » Mon Jul 24, 2006 7:06 am

Hi all,

We are planning to install syslog-ng and cacti syslog plugin. However, I have not found anything about database size.

If you insert more and more logs into the database, what happens? shouldnt you delete old entries? Will the database explode?

Is there any way to set a maximum size for database, erasing old logs?

Any help would be appreciatted....

Regards

adrianmarsh · Post by **adrianmarsh** » Mon Jul 24, 2006 7:44 am

I think I remember reading somewhere in the config that it only keeps the last 30 days worth of logs.

chercen · Post by **chercen** » Mon Jul 24, 2006 9:16 am

adrianmarsh wrote:I think I remember reading somewhere in the config that it only keeps the last 30 days worth of logs.

Thanks adrianmarsh... anybody can confirm this?

Also, would it be possible to keep only a given size of database? i.e, imagine one of your devices starts sending lots of syslog, MB and MB of syslog.... database would die...

Regards,

adrianmarsh · Post by **adrianmarsh** » Mon Jul 24, 2006 1:17 pm

HHmmm...

I think i found the line I was thinking of, and its not time,:

$haloe_config["rows_per_page"] = "30";

I can't see a variable in the settings for it, but i do see some scripts/options that seem to suggest it may be in sync with Cacti defaults (eg 2 years)..

chercen · Post by **chercen** » Tue Jul 25, 2006 6:38 am

adrianmarsh wrote:HHmmm...

I think i found the line I was thinking of, and its not time,:

$haloe_config["rows_per_page"] = "30";

I can't see a variable in the settings for it, but i do see some scripts/options that seem to suggest it may be in sync with Cacti defaults (eg 2 years)..

2 years??

that is fine for cacti rrds (nice design, btw), but not for our database. Our network devices do generate lots of logs, they are space-hungry, and 2 years would make database explode...

Anybody got a workaround for this issue? How do you manage to keep database size within a reasonable value? Any script, database routine, option...? Sorry for all this, but I am a newbie for mysql issues

Thanks in advance,

Regards

cigamit · Post by **cigamit** » Tue Jul 25, 2006 6:55 pm

chercen wrote:2 years?? that is fine for cacti rrds (nice design, btw), but not for our database. Our network devices do generate lots of logs, they are space-hungry, and 2 years would make database explode...

Anybody got a workaround for this issue? How do you manage to keep database size within a reasonable value? Any script, database routine, option...? Sorry for all this, but I am a newbie for mysql issues

Thanks in advance,

Regards

My latest version has an option to specify the number of days to keep syslog messages. I generally just default it to 30 inorder to keep the database from getting huge.

adrianmarsh · Post by **adrianmarsh** » Tue Aug 08, 2006 2:14 am

Ah ha!! So it IS 30... I knew I'd read that somewhere...

So whats the point then of having the gui match Cactis time-scales in the drop down box then ? That was what lead me to thinking 2 years..

spoonman · Post by **spoonman** » Thu Sep 07, 2006 1:37 pm

cigamit....

I am using .4 of haloe and was wondering about the alerts feature? It doesnt seem to work when i set a rule ??? I saw some code back on post page 8 from a guy..haven't tried that yet...what's the status////Should it work or should i try that code?

Thanks

spoonman · Post by **spoonman** » Tue Sep 12, 2006 8:06 am

Anyone else using .4 have the alerts peice working??

Thanks

cigamit · Post by **cigamit** » Tue Sep 12, 2006 6:58 pm

Problem is more than likely your server is not sending the mail. It uses your php.ini mail settings to send out. If I ever get around to finsihing the standalone mail plugin, then you can have it use that plugins settings, but until then, your stuck with php.ini

spoonman · Post by **spoonman** » Wed Sep 13, 2006 6:51 am

What options do I have ....I dont use sendmail?? Can I use just smtp like thold has in its options?? My server is Linux based.

airwalk · Post by **airwalk** » Mon Oct 02, 2006 2:06 am

I'm getting "Server Error, unable to connect to fastcgi server". Has anyone an idea what this error can come from? Doesn't matter whether I install haloe as a plugin or not. I'm using cacti on win2k3 with IIS. no possiblity to test it on apache.

Though the manual run of haloe.php returns more or less good page:
"php c:\inetpub\wwwroot\cacti\plugins\haloe\haloe.php > test.php"

goosed · Post by **goosed** » Mon Oct 23, 2006 2:45 pm

Great plug-in and I believe this is exactly what I've been looking for in viewing syslog log files.

I've installed the plugin and database and believe that they are both working fine. I'm using FreeBSD with Rsyslogd, which stores syslog messages natively to a MySQL database I have that's on the same server.

Now I've been reading up on this plug-in but I'm still not 100% sure I know how it works. Is it going to pull data from my current syslog server and display that? Currently I have my Rsyslogd setup to log messages to MySQL, with a template setup pointing to what database/table/etc I want it stored in. Will I be using a new database to house these messages and not need my current syslog server?

Any help would be appreciated. Thanks.

Cacti

Syslog monitor addon beta

Help

How to avoid database growth?

Who is online