Syslog 1.0 Released

[philipz]

[

Post your feature request and bug on Mantis http://bugs.cacti.net

TheWitness

Done; 1 bug and 1 feature request posted.

Tx

[fata51]

Hello,

I'm trying to use Syslog (version 1.05) with a remote syslog-ng/database server.

I copied the "syslog" directory into the plugins cacti directory

First, I tried to modify the syslog/config.php before install the plugin into the plugin management console as following :

/* revert if you dont use the Cacti database */
$use_cacti_db = false;

if (!$use_cacti_db) {
$syslogdb_type = 'mysql';
$syslogdb_default = 'syslog';
$syslogdb_hostname = '192.168.57.49';
$syslogdb_username = 'syslog';
$syslogdb_password = '****';
$syslogdb_port = 3306;

I tested this configuration by connecting with the command "mysql -h 192.168.57.49 -u syslog -p syslog" and it's OK.

In the plugin management console, I choose "install" and then the following settings : MyISAM storage, Traditional Table (because of my MySQL server version) and 1 Year. After clicking on the "install" button, the page is loading for a while and finally ending with a blank page. When, I go back to the cacti console, the left menu has been modified, the plugin appeared as "Not Installed" in the plugin management and there is no syslog tab.

Secondly, I tried to install the plugin for a cacti database use and then modify the config.php. In this case, the plugin has been correctly installed and enabled. I changed the config.php as posted above but didn't manage to have anything displayed in the syslog tab. I checked the syslog_incoming table, it's correctly populated from syslog-ng daemon. I manually ran the command "php -q syslog_process.php --debug" and get the following :

cacti:/var/www/cacti/plugins/syslog# sudo -u syslog php -q syslog_process.php --debug
SYSLOG: Syslog Table is NOT Partitioned
SYSLOG: Deleted 0, Syslog Message(s) (older than 2009-11-10 days)
SYSLOG: Unique ID = 77
SYSLOG: Found 156749, New Message(s) to process
SYSLOG: Found 0, Removal Rule(s) to process
SYSLOG: Found 0, Alert Rules to process
SYSLOG: Moved , Message(s) to the 'syslog' table
SYSLOG: Deleted 156749, Already Processed Message(s) from incoming
SYSLOG: Updated , Hosts in the syslog hosts table
SYSLOG: Processing Reports...
SYSLOG: We have 0 Reports in the database
SYSLOG: Finished processing Reports...
11/10/2010 03:42:19 PM - SYSTEM SYSLOG STATS:Time:11.05 Deletes:0 Incoming:156749 Removes:0 XFers:0 Alerts:0 Alarms:0 Reports:0

No messages are moved to the syslog table but I don't understand why...

I also have some logs in the cacti.log file :

11/10/2010 02:40:18 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1146', SQL:"INSERT INTO `syslog`.`syslog_hosts` (host) (SELECT DISTINCT host FROM `syslog`.`syslog_incoming`) ON DUPLICATE KEY UPDATE host=VALUES(host)'
11/10/2010 02:40:18 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1146', SQL:"INSERT INTO `syslog`.`syslog` (logtime, priority_id, facility_id, host_id, message) SELECT TIMESTAMP(`date`, `time`), priority_id, facility_id, host_id, message FROM (SELECT date, time, priority_id, facility_id, host_id, message FROM syslog_incoming AS si INNER JOIN syslog_facilities AS sf ON sf.facility=si.facility INNER JOIN syslog_priorities AS sp ON sp.priority=si.priority INNER JOIN syslog_hosts AS sh ON sh.host=si.host WHERE status=3) AS merge'
11/10/2010 02:40:17 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1146', SQL:"INSERT INTO `syslog`.`syslog_host_facilities` (host_id, facility_id) SELECT host_id, facility_id FROM ((SELECT DISTINCT host, facility FROM `syslog`.`syslog_incoming`) AS s INNER JOIN `syslog`.`syslog_hosts` AS sh ON s.host=sh.host INNER JOIN `syslog`.`syslog_facilities` AS sf ON sf.facility=s.facility) ON DUPLICATE KEY UPDATE host_id=VALUES(host_id)'
11/10/2010 02:40:17 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1146', SQL:"INSERT INTO `syslog`.`syslog_hosts` (host) SELECT DISTINCT host FROM `syslog`.`syslog_incoming` ON DUPLICATE KEY UPDATE host=VALUES(host)'
11/10/2010 02:40:17 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1146', SQL:"INSERT INTO `syslog`.`syslog_priorities` (priority) SELECT DISTINCT priority FROM `syslog`.`syslog_incoming` ON DUPLICATE KEY UPDATE priority=VALUES(priority)'
11/10/2010 02:40:17 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1146', SQL:"INSERT INTO `syslog`.`syslog_facilities` (facility) SELECT DISTINCT facility FROM `syslog`.`syslog_incoming` ON DUPLICATE KEY UPDATE facility=VALUES(facility)'
11/10/2010 02:40:17 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1146', SQL:"DELETE FROM `syslog`.`syslog_removed` WHERE logtime < '2009-11-10''
11/10/2010 02:40:17 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1054', SQL:"DELETE FROM `syslog`.`syslog` WHERE logtime < '2009-11-10''

Any tips would be very appreciated

Thanks in advance,

Vincent

[Palermo]

Hi,
I have an question.
I`f followed the manual according:
http://docs.cacti.net/plugin:syslog.config

all the install procedure went well. But i don`t think its working. I don`t see syslog rolling in.
i think its more a rsyslog problem. be course the database is not filing.

What i did.

yum install rsyslog rsyslog-mysql

then updatetd the config file: "/etc/rsyslog.conf"

$ModLoad ommysql
$template cacti_syslog,"INSERT INTO syslog_incoming(facility, priority, date, time, host, message) values (%syslogfacility%, %syslogpriority%, '%timereported:::date-mysql%', '%timereported:::date-mysql%', '%HOSTNAME%', '%msg%')", SQL
*.* >localhost,syslog,cactiuser,password;cacti_syslog

then restarted rsyslogd.

Code: Select all

service rsyslogd restart

i have an error :

Code: Select all

rsyslogd: unrecognized service

So i used commando:

Code: Select all

service rsyslog restart
Shutting down system logger:                               [  OK  ]
Starting system logger:                                         [  OK  ]

Updated the IP tables:

Code: Select all

-A INPUT -m state --state NEW -m udp -p udp --dport 514 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 514 -j ACCEPT

restarted the iptables.

No updates in the database tables.

Anyone maybe an idea?

Thx in advanced.

[fata51]

Hello,

I'm trying to use Syslog (version 1.05) with a remote syslog-ng/database server.

I copied the "syslog" directory into the plugins cacti directory

First, I tried to modify the syslog/config.php before install the plugin into the plugin management console as following :

/* revert if you dont use the Cacti database */
$use_cacti_db = false;

if (!$use_cacti_db) {
$syslogdb_type = 'mysql';
$syslogdb_default = 'syslog';
$syslogdb_hostname = '192.168.57.49';
$syslogdb_username = 'syslog';
$syslogdb_password = '****';
$syslogdb_port = 3306;

I tested this configuration by connecting with the command "mysql -h 192.168.57.49 -u syslog -p syslog" and it's OK.

In the plugin management console, I choose "install" and then the following settings : MyISAM storage, Traditional Table (because of my MySQL server version) and 1 Year. After clicking on the "install" button, the page is loading for a while and finally ending with a blank page. When, I go back to the cacti console, the left menu has been modified, the plugin appeared as "Not Installed" in the plugin management and there is no syslog tab.

Secondly, I tried to install the plugin for a cacti database use and then modify the config.php. In this case, the plugin has been correctly installed and enabled. I changed the config.php as posted above but didn't manage to have anything displayed in the syslog tab. I checked the syslog_incoming table, it's correctly populated from syslog-ng daemon. I manually ran the command "php -q syslog_process.php --debug" and get the following :

cacti:/var/www/cacti/plugins/syslog# sudo -u syslog php -q syslog_process.php --debug
SYSLOG: Syslog Table is NOT Partitioned
SYSLOG: Deleted 0, Syslog Message(s) (older than 2009-11-10 days)
SYSLOG: Unique ID = 77
SYSLOG: Found 156749, New Message(s) to process
SYSLOG: Found 0, Removal Rule(s) to process
SYSLOG: Found 0, Alert Rules to process
SYSLOG: Moved , Message(s) to the 'syslog' table
SYSLOG: Deleted 156749, Already Processed Message(s) from incoming
SYSLOG: Updated , Hosts in the syslog hosts table
SYSLOG: Processing Reports...
SYSLOG: We have 0 Reports in the database
SYSLOG: Finished processing Reports...
11/10/2010 03:42:19 PM - SYSTEM SYSLOG STATS:Time:11.05 Deletes:0 Incoming:156749 Removes:0 XFers:0 Alerts:0 Alarms:0 Reports:0

No messages are moved to the syslog table but I don't understand why...

I also have some logs in the cacti.log file :

11/10/2010 02:40:18 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1146', SQL:"INSERT INTO `syslog`.`syslog_hosts` (host) (SELECT DISTINCT host FROM `syslog`.`syslog_incoming`) ON DUPLICATE KEY UPDATE host=VALUES(host)'
11/10/2010 02:40:18 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1146', SQL:"INSERT INTO `syslog`.`syslog` (logtime, priority_id, facility_id, host_id, message) SELECT TIMESTAMP(`date`, `time`), priority_id, facility_id, host_id, message FROM (SELECT date, time, priority_id, facility_id, host_id, message FROM syslog_incoming AS si INNER JOIN syslog_facilities AS sf ON sf.facility=si.facility INNER JOIN syslog_priorities AS sp ON sp.priority=si.priority INNER JOIN syslog_hosts AS sh ON sh.host=si.host WHERE status=3) AS merge'
11/10/2010 02:40:17 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1146', SQL:"INSERT INTO `syslog`.`syslog_host_facilities` (host_id, facility_id) SELECT host_id, facility_id FROM ((SELECT DISTINCT host, facility FROM `syslog`.`syslog_incoming`) AS s INNER JOIN `syslog`.`syslog_hosts` AS sh ON s.host=sh.host INNER JOIN `syslog`.`syslog_facilities` AS sf ON sf.facility=s.facility) ON DUPLICATE KEY UPDATE host_id=VALUES(host_id)'
11/10/2010 02:40:17 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1146', SQL:"INSERT INTO `syslog`.`syslog_hosts` (host) SELECT DISTINCT host FROM `syslog`.`syslog_incoming` ON DUPLICATE KEY UPDATE host=VALUES(host)'
11/10/2010 02:40:17 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1146', SQL:"INSERT INTO `syslog`.`syslog_priorities` (priority) SELECT DISTINCT priority FROM `syslog`.`syslog_incoming` ON DUPLICATE KEY UPDATE priority=VALUES(priority)'
11/10/2010 02:40:17 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1146', SQL:"INSERT INTO `syslog`.`syslog_facilities` (facility) SELECT DISTINCT facility FROM `syslog`.`syslog_incoming` ON DUPLICATE KEY UPDATE facility=VALUES(facility)'
11/10/2010 02:40:17 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1146', SQL:"DELETE FROM `syslog`.`syslog_removed` WHERE logtime < '2009-11-10''
11/10/2010 02:40:17 PM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1054', SQL:"DELETE FROM `syslog`.`syslog` WHERE logtime < '2009-11-10''

Any tips would be very appreciated

Thanks in advance,

Vincent

Hello,

I'm kind of stuck with this thing, any clue?

Thanks,

Vincent

[trungtano]

Hi,

I have just upgrade my Syslog to 1.04 and Setting to 0.7 running on cacti 0.8.7g but my plugin Syslog seem not running some function:

- It does not display the colour as level define, in tab Alert Log it just have colour of Notice Level other Level have the same colour



Do you have any Idea?

Thanks for help

[armstrongzhao]

Does Mysql 5.5 support syslog 1.05?

[hpsmartyz]

Hello

thanks for this new version!
I have an issue related to the warning priority level.
I have logs with such priority level, but none of them has the color associated with it. Further, when I filter the logs using either Debug, Info++, Notice++, Warning++ the warning logs do not show up in the list.
I am using syslog-ng v2.0.10-2

Thanks for your help.
If needed I can dig in the code if you tell me were to look at

[armstrongzhao]

My OS enviroment is centos 5.5 (64bit) cacti 0.8.7 mysql 5.5 syslog 1.05 rsyslog 3.22
The latest syslog version is cool with mysql 5.5.

The table partition of mysql 5.5 is much quicker than 5.1.

The amount of the syslog file is 19G. Hardware CPU model name : Intel(R) Xeon(R) CPU 5110 @ 1.60GHz, Memory 2G. The server works well.

I disabled rsyslog host name resolution, rsyslog -x. Some of hosts can display IP in the GUI, but others still display host names. How to deal with it?

[TheWitness]

Syslog 1.21 will be released approximately the same time as Cacti 0.8.7h.

[JohnYYC]

I am currently using Syslog version 1.22 on Cacti 0.8.7g. I was wondering if it was possible to disable the "Statistics" I have no need for them and it seems to fill up my database at a pretty steady rate. I looked under Settings but couldn't find anything.

If it is not currently possible could I make a feature request for the option to enable/disable Statistics?

[TheWitness]

I am currently using Syslog version 1.22 on Cacti 0.8.7g. I was wondering if it was possible to disable the "Statistics" I have no need for them and it seems to fill up my database at a pretty steady rate. I looked under Settings but couldn't find anything.

If it is not currently possible could I make a feature request for the option to enable/disable Statistics?

Yea, it's not a bad idea, the performance is horrible right now anyway.

[mastrboy]

Since I'm unable to create a account on the dokuwiki at http://docs.cacti.net/ I'll just post this stuff here:

Rsyslog tips,

Tip1, howto only log messages from a subnet/external host into the database:

Code: Select all

if $fromhost-ip startswith '192.168.0.' then :ommysql:localhost,cacti,DB_USER,DB_PASSWORD;cacti_syslog
& ~

The '& ~' discards the message after putting it into SQL, meaning your normal /var/log files won't get flooded.

Tip2, Strip timestamps from cisco syslog messages before inserting them, messages that don't match the rule will get inserted as is.

Code: Select all

# Cacti template with cisco regexp stripping
$Template cacti_syslog,"INSERT INTO syslog_incoming(facility, priority, date, time, host, message) VALUES ('%syslogfacility-text%', '%syslogpriority-text%',  '%timereported:::date-mysql%', '%timereported:::date-mysql%', '%HOSTNAME%', '%msg:R,ERE,2,FIELD:(.*[0-2][0-9]:[0-5][0-9]:[0-5][0-9]: )(.*)--end%')", SQL

Tip3, async logging for very busy systems with file backlog in case the database goes down.

Code: Select all

# Async logging
$WorkDirectory /var/log/rsyslog
$ActionQueueType LinkedList   # use asynchronous processing
$ActionQueueFileName srvrfwd  # set file name, also enables disk mode
$ActionResumeRetryCount -1    # infinite retries on insert failure
$ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down

Syslog plugin tip about colors, we've been having problems with the colors randomly appear in syslog, commenting out some lines i setup.php solves this:
Line 1336, file setup.php:

Code: Select all

// changing colours should not need a re-initialize of the session, and sometimes $_SESSION["syslog_* is set, but empty...
//      if (!isset($_SESSION["syslog_colors"])) {
                foreach($syslog_levels as $level) {
                        $syslog_colors[$level] = db_fetch_cell("SELECT hex FROM colors WHERE id=" . read_config_option("syslog_" . $level . "_bg"));
                        $syslog_text_colors[$level] = db_fetch_cell("SELECT hex FROM colors WHERE id=" . read_config_option("syslog_" . $level . "_fg"));
                }

                $_SESSION["syslog_colors"] = $syslog_colors;
                $_SESSION["syslog_text_colors"] = $syslog_text_colors;
//      }else{
//              $syslog_colors = $_SESSION["syslog_colors"];
//              $syslog_text_colors = $_SESSION["syslog_text_colors"];
//      }

[ltitusl]

Thanks for all the great info, super helpful!

[gurulee]

I am not seeing any remote hosts in my syslog, only localhost messages like the one below:

Code: Select all

(root) CMD (php /var/www/html/poller.php > /dev/null 2>&1)

Can someone help me?

[gurulee]

mastrboy, Thank you for the tips!

But I am seeing hosts in my Syslog screen and verified that data is populating in the mysql syslog tables, BUT there's no messages showing in the Cacti Syslog GUI...

Any ideas, anyone?