Syslog daemon on Windows (can logged to database)

General discussion about Plugins for Cacti

Moderators: Developers, Moderators

DrivesMeCrazy
Posts: 30
Joined: Sat Apr 28, 2007 3:57 am

Syslog daemon on Windows (can logged to database)

Post by DrivesMeCrazy »

Hi guys,

I just added the haloe-syslog plugin to cacti and found that it does not support the logging of syslog messages to database.

I am stucked now as my cacti and monitoring server is on windows platform.
Currently, I am monitoring a switch that will send syslog and snmp info to my cacti server.

I tried using Kiwi Syslogd, but then for advance features like log to database needs money.

Now, can anybody recommend me a syslog daemon on windows that is free and yet has the ability to log to database?

Been searching on the net all day but failed to find something as above. :(

Thanks in advance.
User avatar
streaker69
Cacti Pro User
Posts: 712
Joined: Mon Mar 27, 2006 10:35 am
Location: Psychic Amish Network Administrator

Re: Syslog daemon on Windows (can logged to database)

Post by streaker69 »

DrivesMeCrazy wrote:Hi guys,

I just added the haloe-syslog plugin to cacti and found that it does not support the logging of syslog messages to database.

I am stucked now as my cacti and monitoring server is on windows platform.
Currently, I am monitoring a switch that will send syslog and snmp info to my cacti server.

I tried using Kiwi Syslogd, but then for advance features like log to database needs money.

Now, can anybody recommend me a syslog daemon on windows that is free and yet has the ability to log to database?

Been searching on the net all day but failed to find something as above. :(

Thanks in advance.
Haloe does indeed log to a Database, if yours does not, it's not configured properly.

As for Winders boxes logging event log to Haloe, you can use WinlogD.
apperrault
Cacti User
Posts: 379
Joined: Fri Feb 16, 2007 11:37 am
Location: Emeryville, CA
Contact:

Post by apperrault »

Join the club Crazy!! I have been using cacti for probably about 6 months, and i have been looking for a windows based Syslog server that will write to a MySQL DB ever since.

If i find something, I will let everyone know.

app
[b]Cacti Version[/b] - 0.8.7b
[b]Plugin Architecture[/b] - 2.2 Beta
[b]Poller Type[/b] - CMD.php
[b]Server Info[/b] - Linux 2.6.9-78.0.1.ELsmp
[b]Web Server[/b] - Apache/2.0.52 (Red Hat)
[b]PHP[/b] - 4.3.9
[b]MySQL[/b] - 4.1.22
[b]RRDTool[/b] - 1.2.23
[b]SNMP[/b] - 5.1.2
[b]Plugins[/b][list]Global Plugin Settings (settings - v0.5)
SuperLinks (superlinks - v0.72)
Host Info (hostinfo - v0.2)
Report Creator (reports - v0.3)
Update Checker (update - v0.4)
Realtime for Cacti (realtime - v0.35)
Cacti Log View (clog - v1.1)
RRD File Cleaner (rrdclean - v0.36)
Network Discovery (discovery - v0.9)
Uptime (uptime - v0.4)[/list]
DrivesMeCrazy
Posts: 30
Joined: Sat Apr 28, 2007 3:57 am

Re: Syslog daemon on Windows (can logged to database)

Post by DrivesMeCrazy »

streaker69 wrote:
DrivesMeCrazy wrote:Hi guys,

I just added the haloe-syslog plugin to cacti and found that it does not support the logging of syslog messages to database.

I am stucked now as my cacti and monitoring server is on windows platform.
Currently, I am monitoring a switch that will send syslog and snmp info to my cacti server.

I tried using Kiwi Syslogd, but then for advance features like log to database needs money.

Now, can anybody recommend me a syslog daemon on windows that is free and yet has the ability to log to database?

Been searching on the net all day but failed to find something as above. :(

Thanks in advance.
Haloe does indeed log to a Database, if yours does not, it's not configured properly.

As for Winders boxes logging event log to Haloe, you can use WinlogD.
But I see Haloe documentation, it says that it will not captured syslog from remote machines or network device?

I am using Haloe version 0.4, what am I missing here?

Will appreciate if you could point me to the correct source or direction.
Thanks.
DrivesMeCrazy
Posts: 30
Joined: Sat Apr 28, 2007 3:57 am

Post by DrivesMeCrazy »

apperrault wrote:Join the club Crazy!! I have been using cacti for probably about 6 months, and i have been looking for a windows based Syslog server that will write to a MySQL DB ever since.

If i find something, I will let everyone know.

app
Okie, thanks in advance.

Anyway I just find that Haloe is a bit of redundant if it just only allow viewing of syslog message from a mysql database.

It should instead be a plugin that accept syslog messages from remote and local server, put it in database, send out alerts, etc. :-?
User avatar
streaker69
Cacti Pro User
Posts: 712
Joined: Mon Mar 27, 2006 10:35 am
Location: Psychic Amish Network Administrator

Re: Syslog daemon on Windows (can logged to database)

Post by streaker69 »

DrivesMeCrazy wrote: Haloe does indeed log to a Database, if yours does not, it's not configured properly.

As for Winders boxes logging event log to Haloe, you can use WinlogD.
But I see Haloe documentation, it says that it will not captured syslog from remote machines or network device?

I am using Haloe version 0.4, what am I missing here?

Will appreciate if you could point me to the correct source or direction.
Thanks.[/quote]

CactiEZ comes with Haloe already configured to log to a MySQL database. From there it was just a matter of configuring WinlogD on my windows machines to log to the syslog server.

See if following these instructions will help you get Haloe properly configured.

http://www.nmsworld.com/UNIX/Syslog-NG.htm
DrivesMeCrazy
Posts: 30
Joined: Sat Apr 28, 2007 3:57 am

Re: Syslog daemon on Windows (can logged to database)

Post by DrivesMeCrazy »

Correcting some quoting issues...
Last edited by DrivesMeCrazy on Thu Aug 09, 2007 12:25 pm, edited 1 time in total.
DrivesMeCrazy
Posts: 30
Joined: Sat Apr 28, 2007 3:57 am

Re: Syslog daemon on Windows (can logged to database)

Post by DrivesMeCrazy »

streaker69 wrote:
DrivesMeCrazy wrote:
streaker69 wrote: Haloe does indeed log to a Database, if yours does not, it's not configured properly.

As for Winders boxes logging event log to Haloe, you can use WinlogD.
But I see Haloe documentation, it says that it will not captured syslog from remote machines or network device?

I am using Haloe version 0.4, what am I missing here?

Will appreciate if you could point me to the correct source or direction.
Thanks.
CactiEZ comes with Haloe already configured to log to a MySQL database. From there it was just a matter of configuring WinlogD on my windows machines to log to the syslog server.

See if following these instructions will help you get Haloe properly configured.

http://www.nmsworld.com/UNIX/Syslog-NG.htm
I think I get where you are coming from.

However, I installed cacti and the plugin architecture separately and addon the haloe plugin lastly (not via CactiEZ way).
Furthermore my cacti is on a Windows machine, which suppose to receive all syslogs from *nix machines or network devices.

Syslog-NG; that CactiEZ is compatible with, need to be installed on a *nix machine.
And Winlogd transform windows event log to syslog format.

I am still trying to find a windows syslog daemon that can capture remote syslog messages and write to the database. :(
pepj
Cacti User
Posts: 324
Joined: Thu Sep 29, 2005 5:03 am
Location: switzerland

Re: Syslog daemon on Windows (can logged to database)

Post by pepj »

DrivesMeCrazy wrote:But I see Haloe documentation, it says that it will not captured syslog from remote machines or network device?
I am using Haloe version 0.4x, what am I missing here?
Two possibilities:
1 / using my plugin with for example KIWI (created for almost two years, but I thought nobody else needed this.)
To know how to install it read the readme file

2/ using cygwin and a unix syslog or Net-Snmp

Some changement (0.3) to function correctly with Haloe / email-alerts

HERE NEW link to syslogupd 0.4x because of new structure of cacti0.8.7 and above and syslog-plugin)
URL=http://forums.cacti.net/viewtopic.php?p=117245#117245
Last edited by pepj on Fri Nov 16, 2007 6:08 pm, edited 7 times in total.
Jean-Michel
cacti 0.8.7e | cmd & cactid (cactid 0.8.x) | Linux | MySQL Ver 14.7 Distrib 4.1.12, for Win32 | PHP v5.2.6 | Apache v2.x | Thold | Plugin Architecture | plugin "configuration manager" http://cactiusers.org/forums/topic257.html | plugin "IP subnet calculator IPv4 / IPV6" http://forums.cacti.net/viewtopic.php?t=15428 | plugin banner http://docs.cacti.net/userplugin:banner | Net-SNMP 5.5.2 | cygwin 1.5.18 of 02.07.2005
DrivesMeCrazy
Posts: 30
Joined: Sat Apr 28, 2007 3:57 am

Re: Syslog daemon on Windows (can logged to database)

Post by DrivesMeCrazy »

pepj wrote:
DrivesMeCrazy wrote:But I see Haloe documentation, it says that it will not captured syslog from remote machines or network device?
I am using Haloe version 0.4, what am I missing here?
Two possibilities:
1 / using my plugin with for example KIWI (created for almost two years, but I thought nobody else needed this.)
To know how to install it read the readme file

2/ using cygwin and a unix syslog or Net-Snmp
pepj,

thanks for sharing your work.

May I know how can i use your plugin?
I have went through the readme file and setup accordingly.
But it seems like nothing is going into haloe syslog database, even use the force option from syslogupd.

Below are my steps:

Code: Select all

Step 1: cacti's config.php
$plugins = array();
$plugins[0] = 'haloe';
$plugins[1] = 'syslogupd';

Code: Select all

Step 2: syslogupd's config.php
$syslogfile = "C:\Program Files\Syslogd\Logs\SyslogCatchAll.txt";
$trapfile   = "C:\Program Files\Syslogd\Logs\TEMPSyslogCatchAll.txt";

Code: Select all

Step 3: Syslogupd update haloe mysql database every 5 mins
Do I need to manually configure the 5mins poller or its using the cacti poller.php?

Code: Select all

Step 4: "Syslog force update"
I tried to use this option to update the database, but every time it gives me a blank page.
Is this the expected result?
Please advise. Thanks alot.
DrivesMeCrazy
Posts: 30
Joined: Sat Apr 28, 2007 3:57 am

Post by DrivesMeCrazy »

pepj,

some updates here.

I scheduled a 5min poller using syslogupd.php.
However, I still do not see any logs being written to the database. :(

Code: Select all

mysql> select * from syslog;
Empty set (0.00 sec)

mysql> select * from syslog_incoming;
Empty set (0.00 sec)
I have check the scheduled task for exit status code and manually run the command via dos prompt. No error detected.

Code: Select all

E:\cacti> E:\php\php.exe E:\cacti\plugins\syslogupd\syslogupd.php
And the funny thing i observe is now Kiwi Syslogd's log directory only contains TEMPSyslogCatchAll.txt.
The default SyslogCatchAll.txt is not in the path anymore - C:\Program Files\Syslogd\Logs\.

Any ideas!? :-?
pepj
Cacti User
Posts: 324
Joined: Thu Sep 29, 2005 5:03 am
Location: switzerland

Post by pepj »

DrivesMeCrazy wrote:I scheduled a 5min poller using syslogupd.php.
However, I still do not see any logs being written to the database. :(
You don't need a specific poller. Syslogupd use the cacti poller.
DrivesMeCrazy wrote: Step 4: "Syslog force update"
I tried to use this option to update the database, but every time it gives me a blank page.
Is this the expected result?
Yes (I will change it for the next release). You have to go back. The blank "page" is only because I was lazy (I thought nobody else needed the plugin) and I have not programmed the return. Normally you don't need to force (only for test ...)
DrivesMeCrazy wrote: Empty syslog/haloe ....
This plugin is very simple it read the file with the trap (created by Kiwi in your case) and it insert the rows in th sysog/haloe.

The trick is to configure (in your case Kiwi) the syslog to create a file with the same format we want.

In your case for Kiwi:
check your setup
(if you have the full version you don't need the syslogupd plugin (use the action -> mysql)
1/ if not
- in "action-->log to file" checked
- in "action"/logtofile "logfile format" choose : comma separatede YY-mm-dd
- in "formatting" check only the fileds: date, time, priority, facility, level, host, msg,
- in "modifier" chek "replace non printable...", default priority to "local0"
4/ send a test snmp from KIWI. Has Kiwi created a good file?

How to test syslogupd-plugin:
1/ try to create manually the syslogfile with some lines for example like this:
2007-08-17 12:54:12,Syslog.Warning,10.0.0.2,community=public enterprise=1.3.6.1.4.1.9.5.6 enterprise_mib_name=workgroup.6 uptime=-1738433785 agent_ip=10.0.0.2 generic_num=6 specific_num=6 version=Ver1 var01_mib_name=chassisTempAlarm.0 var05_oid=1.3.6.1.4.1.9.5.1.2.10.0 var05_value=0
2007-08-17 12:54:39,Local7.Notice,10.0.0.2,"277: 000276: Aug XX 12:54:38.300 gmt: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet1/0/15, changed state to down"
2/ check if your KIWI create the file with the good format
Jean-Michel
cacti 0.8.7e | cmd & cactid (cactid 0.8.x) | Linux | MySQL Ver 14.7 Distrib 4.1.12, for Win32 | PHP v5.2.6 | Apache v2.x | Thold | Plugin Architecture | plugin "configuration manager" http://cactiusers.org/forums/topic257.html | plugin "IP subnet calculator IPv4 / IPV6" http://forums.cacti.net/viewtopic.php?t=15428 | plugin banner http://docs.cacti.net/userplugin:banner | Net-SNMP 5.5.2 | cygwin 1.5.18 of 02.07.2005
DrivesMeCrazy
Posts: 30
Joined: Sat Apr 28, 2007 3:57 am

Post by DrivesMeCrazy »

Thanks pepj, I have followed ur instructions to create the format.
Sample of my log is as such:

Code: Select all

2007-08-24 10:20:46,User.Warning,192.168.0.1,NTP server 216.52.23.2 failed to respond
One thing I notice this.
Everytime when kiwi syslogd captured the syslog message and stored in SyslogCatchAll.txt, it gets renamed to TEMPSyslogCatchAll.txt.
SyslogCatchAll.txt will not exists anymore till the next syslog msg comes in and the last line of syslog msg is written into TEMPSyslogCatchAll.txt.

I traced thru your code and found the below coding (apparently this does not work for me and it doesn't even get to insert the record into database. Please help if you can.
I am totally lost right now. :(

Code: Select all

if (!file_exists($syslogfile)) exit;
//TEST fwrite($handletest,"(file exist ($syslogfile)\n");
        if (file_exists($trapfile)) unlink($trapfile);
        rename($syslogfile,$trapfile);
        $handle = fopen($trapfile, "r");
        if ($handle) {
          while (!feof($handle)) {
           $line=fgets($handle);
          if ($line!="") {
            // insert record
            $record="(".$haloe_config["hostField"].","
                 .$haloe_config["facilityField"] . ","
                 .$haloe_config["priorityField"] . ","
                 ."level,"
                 .$haloe_config["dateField"] . ","
                 .$haloe_config["timeField"] . ","
                 ."program,"
                 .$haloe_config["textField"].")";
            list($fulldate,$fulllevel,$host,$msg0) = preg_split('/,/', $line, 4);
pepj
Cacti User
Posts: 324
Joined: Thu Sep 29, 2005 5:03 am
Location: switzerland

Post by pepj »

DrivesMeCrazy wrote:Sample of my log is ...
It seeems good
DrivesMeCrazy wrote: Everytime when kiwi syslogd captured the syslog message and stored in SyslogCatchAll.txt, it gets renamed to TEMPSyslogCatchAll.txt.
SyslogCatchAll.txt will not exists anymore till ...
In this way the plugin will copy only one time the traps (the TEMPSyslogCatchAll.txt is a temporary backup) and also not lost a trap that would comes during the process. We can discuss later if you want to keep all the traps in a text file, but normally you don't need this if you have the data in the haloe SQL DB.

To check what happend when the data are copied to Haloe:
- remove in the function "syslogupd_poller" of "functions.php" in syslogupd the "//TEST " before the lines which have this.
- create a "SyslogCatchAll.txt" file with a test trap (or from KIWI click test trap)
- start "php syslogupd.php" manually from a command prompt

You should see the parameters from haloe passed to syslogupd, and the result.
Are the parameters OK?

PS: check also the parameters in the Haloe config.php
in my case I had changed

Code: Select all

$haloedb_hostname = "localhost";
$haloedb_username = "...";
$haloedb_password = "...";
$haloe_config["haloeTable"]      = "syslog";
Jean-Michel
cacti 0.8.7e | cmd & cactid (cactid 0.8.x) | Linux | MySQL Ver 14.7 Distrib 4.1.12, for Win32 | PHP v5.2.6 | Apache v2.x | Thold | Plugin Architecture | plugin "configuration manager" http://cactiusers.org/forums/topic257.html | plugin "IP subnet calculator IPv4 / IPV6" http://forums.cacti.net/viewtopic.php?t=15428 | plugin banner http://docs.cacti.net/userplugin:banner | Net-SNMP 5.5.2 | cygwin 1.5.18 of 02.07.2005
pepj
Cacti User
Posts: 324
Joined: Thu Sep 29, 2005 5:03 am
Location: switzerland

Post by pepj »

DrivesMeCrazy wrote:Please help if you can.
I am totally lost right now. :(
1/ first read the post above.
2/ install the new version of syslogupd 0.3
- I have corrected when you click on "Syslog force update" to manually load the new traps.
- I have added the debuging when you start

Code: Select all

your_path\plugins\syslogupd>php your_path\plugins\syslogupd\syslogupd.php -d
So you will see the debugging on the shell and cactilog.
PS: check also the parameters in the Haloe config.php
in my case I had changed

Code: Select all

$haloedb_hostname = "localhost"; 
$haloedb_username = "..."; 
$haloedb_password = "..."; 
$haloe_config["haloeTable"]      = "syslog"; 
Some changement (0.3) to function correctly with Haloe / email-alerts


HERE NEW link to syslogupd 0.4 because of new structure of cacti0.8.7 and above and syslog-plugin)
URL=http://forums.cacti.net/viewtopic.php?p=117245#117245
Last edited by pepj on Thu Nov 15, 2007 6:20 am, edited 3 times in total.
Jean-Michel
cacti 0.8.7e | cmd & cactid (cactid 0.8.x) | Linux | MySQL Ver 14.7 Distrib 4.1.12, for Win32 | PHP v5.2.6 | Apache v2.x | Thold | Plugin Architecture | plugin "configuration manager" http://cactiusers.org/forums/topic257.html | plugin "IP subnet calculator IPv4 / IPV6" http://forums.cacti.net/viewtopic.php?t=15428 | plugin banner http://docs.cacti.net/userplugin:banner | Net-SNMP 5.5.2 | cygwin 1.5.18 of 02.07.2005
Post Reply

Who is online

Users browsing this forum: No registered users and 3 guests