Sendmail & Mailscanner version 2

[warnesj]

>>> UPDATED November 7, 2007 <<<

As I promised, here is the latest vesion of my Sendmail and MailScanner scripts and templates. This is an update to my original post. Visually there are no differences to the graphs. See, here are what the graphs are looking like now,

Not too much different, just a bit cleaner with the new version of RRDTool that Cacti is using.

Now to what has changed. This version runs significantly differently. There are basically two scripts, a maillog watcher script and a read counters script. Both of these scripts run on your mail server.

NOTE: All the scripts and an empty counter file is attached at the bottom of this post.

maillog Watcher Script
First, the source code to the script. I store these scripts in /opt/watchmaillog on my mail servers, but you can put them wherever you like on your server. Just update the first couple of lines of the code to reflect where you've put the files.
/opt/watchmaillog/watchmaillog.sh

Code: Select all

#!/usr/bin/perl
#
# Daemon used to watch the maillog messages for certain messages and trigger events when
# certain messages occur
#
# By Jason Warnes
#
# Change Log
# ~~~~~~~~~~
# 2006-08-22: Initial release
# 2006-09-05: Fixed signal handling
#             Added /var/run/watchmaillog.pid file for logrotate usage
# 2006-09-07: Added proper signal handling (Thanks pvenezia!)
#             Added SpamAssassin spamd checking support for SPAM (Thanks raiten!)
# 2006-09-18: Added new item mess_waiting, which is the number of messages MailScanner
#             detects when a new batch is started.
# 2006-11-02: Changed the way SPAM messages were detected so the script works
#             on servers configured for less verbose logging. (Thanks sdetroch!)
# 2006-11-08: Added new item mess_rejected, which is the number of rejected
#             messages by Sendmail.
# 2007-02-06: Fixed <MAILLOG> close statement at end of main program. (Thanks Avenger!)
#             Fixed warning messages about uninitialized $line used in pattern
#             matching (Thanks Avenger!)
# 2007-05-04: Properly closed the maillog file on SIGHUP received.  (Thansks thomasch!)

$debug=0;       # 1=Debug messages are displayed, 0=No debug messages are displayed
$daemon=1;      # 1=Daemonize the program, 0=Run interactive
$syslog=1;      # 1=Log stuff to syslog, 0=No logging to syslog
$self="/opt/watchmaillog/watchmaillog.sh";  # Location of this script
$counterfile="/opt/watchmaillog/watchmaillog_counters";	# Location to store the counter file
$resetfile="/opt/watchmaillog/watchmaillog_reset";	# Location of the reset counter flag file
$pidfile="/var/run/watchmaillog.pid";	# Location of the running process ID file (used in logrotate)

use Sys::Syslog;
use POSIX;
use Time::HiRes qw( gettimeofday tv_interval );

 $|=1;

my $sigset = POSIX::SigSet->new();
my $hupaction = POSIX::SigAction->new('hup_signal_handler',
                                     $sigset,
                                     &POSIX::SA_NODEFER);
my $osigaction = POSIX::SigAction->new('signal_handler',
                                     $sigset,
                                     &POSIX::SA_NODEFER);
POSIX::sigaction(&POSIX::SIGHUP, $hupaction);
POSIX::sigaction(&POSIX::SIGINT, $osigaction);
POSIX::sigaction(&POSIX::SIGTERM, $osigaction);


if($daemon){
        $pid=fork;
	if($pid) {
		open(PID,">".$pidfile) or die "Cannot open PID file: $!.";
			print PID ("$pid\n");	# Write the PID out to the PID file for logrotate
		close(PID);
	}
        exit if $pid;
        die "Couldn't fork : $!" unless defined($pid);
        setsid() or die "Can't start a new session: $!";
	$time_to_die=0;
}

sub signal_handler {
        $time_to_die=1;
}

sub hup_signal_handler {
      if($debug){print "got SIGHUP\n";}
      close(MAILLOG);
      exec($self) or die "Couldn't restart: $!\n";
}

if($syslog){openlog("watchmaillog","pid","daemon");}
if($syslog){syslog("notice","Starting.");}
if($debug){print("watchmaillog is starting.\n");}

# Main part of the program
open(MAILLOG, "tail -n 0 -f /var/log/maillog|") or die "Cannot open maillog: $!.";
my $line="";
while(!$time_to_die){
	$line=<MAILLOG>;
	# Look for received messages where the sender is not from our domain(s)
               if(($line=~/from\=/) && ($line!~/\@domain1.com|\@domain2.com/)){
		$item="mess_recv";
		&readcounterfile;
		$counter{$item}++;
		if($debug){print("Found an inbound message, incrementing the message recieve counter to $counter{$item}.\n");}
		&writecounterfile;
	}
	# Look for messages sent to our domain(s), indicates an inbound message relayed to an internal server
	if(($line=~/stat\=Sent/) && ($line=~/\@domain1.com|\@domain2.com/)){
		$item="mess_relay";
		&readcounterfile;
		$counter{$item}++;
		if($debug){print("Found an clean inbound message, incrementing the clean message recieve counter to $counter{$item}.\n");}
		&writecounterfile;
	}
	# Look for sent messages to NOT our email domain(s), indicates an outbound message
	if(($line=~/stat\=Sent/) && ($line!~/\@domain1.com|\@domain2.com/)){
		$item="mess_sent";
		&readcounterfile;
		$counter{$item}++;
		if($debug){print("Found an outbound message, incrementing the message sent counter to $counter{$item}.\n");}
		&writecounterfile;
	}
	# Look for rejected messages
	if((($line=~/ruleset/) && ($line=~/reject\=/)) || ($line =~/rejecting/)){
		$item="mess_rejected";
		&readcounterfile;
		$counter{$item}++;
		if($debug){print("Found a rejected message, incrementing the message rejected counter to $counter{$item}.\n");}
		&writecounterfile;
	}
	# Look for MailScanner spam scanning batch results
	if($line=~/Spam\ Checks\:\ Found/){
		$item="spam";
		$spam_count_pos = index($line,"Spam\ Checks\:\ Found");
		$spam_count_pos2 = index($line, "\ spam\ messages");
		$spam_count = substr($line,($spam_count_pos+19),($spam_count_pos2-($spam_count_pos+19)));
		&readcounterfile;
		$counter{$item}=$counter{$item}+$spam_count;
		if($debug){print("Found $spam_count SPAM in the MailScanner batch, incrementing the spam counter to $counter{$item}.\n");}
		&writecounterfile;
	}
	# Look for MainScanner virus scanning batch results
	if($line=~/Virus\ Scanning\:\ Found/){
		$item="virus";
		$virus_count_pos = index($line,"Virus\ Scanning\:\ Found");
		$virus_count_pos2 = index($line, "\ viruses");
		$virus_count = substr($line,($virus_count_pos+22),($virus_count_pos2-($virus_count_pos+22)));
		&readcounterfile;
		$counter{$item}=$counter{$item}+$virus_count;
		if($debug){print("Found $virus_count viruses in the MailScanner batch, incrementing the virus counter to $counter{$item}.\n");}
		&writecounterfile;
	}
	# Look for MailScanner waiting messages
	if($line=~/New\ Batch\:\ Found/){
		$item="mess_waiting";
		$mess_waiting_pos = index($line,"New\ Batch\:\ Found");
		$mess_waiting_pos2 = index($line,"\ messages\ waiting");
		$mess_waiting = substr($line,($mess_waiting_pos+17),($mess_waiting_pos2-($mess_waiting_pos+17)));
		&readcounterfile;
		$counter{$item}=$mess_waiting;
		if($debug){print("Mailscanner found $mess_waiting messages waiting, setting the mess_waiting counter to $counter{$item}.\n");}
		&writecounterfile;
	}
}
close(MAILLOG);
if($debug){print("watchmaillog is ending.\n");}
if($syslog){syslog("notice","Ending.");}
unlink($pidfile);

# Subroutine to read the contents of the counter file
sub readcounterfile {
	# Read the counter values from the file
	if($debug){print("Reading contents of counter file.\n");}
	open(COUNTER,$counterfile);
	while($line=<COUNTER>){
		@line=split(/\:/,$line);
		chop($line[1]); # Drop the trailing LF off the value
		# Check for reset counter flag file
		if(-e $resetfile."_".$line[0]){
			if($debug){print("Reset counter flag file found for counter $line[0], resetting counter value to 0.\n");}
			$counter{$line[0]}=0;
			unlink($resetfile."_".$line[0]);
		} else {
			$counter{$line[0]}=$line[1];
		}
		if($debug){print("Counter $line[0] = $counter{$line[0]}.\n");}
	}
	close(COUNTER);
}

# Subrouting to write the contents of the counter file
sub writecounterfile {
	if($debug){print("Writing counter values to counter file.\n");}
	open(COUNTER,">".$counterfile);
	# Write each counter item out to the counter file
	foreach $item (sort keys(%counter)) {
		print COUNTER ($item."\:".$counter{$item}."\n");
	}
	close(COUNTER);
	chmod(0666,$counterfile);
}

NOTE:You'll need to change domain1.com and domain2.com to whatever your domain(s) are. If you only have one domain then you can take out the domain2.com search.

WHOA you're thinking, what the heck is going on here! Well let me explain. This is the biggest change since the first version and it was done purely for performance. Now instead of scraping through the last 5000 lines of the maillog file (which was a performance killer on high load servers), now this maillog watcher script runs daemonized and constantly is watching your maillog file in realtime for particular keywords. These keywords will trigger to increment counters for messages sent, messages recieved, spam blocked, viruses detected, and messages relayed (I'll explain this one in a second). As these counters are incremented they are written out to a file on your mail server's disk (to survive reboots, etc).

Now the counters are all pretty self explanitory except for "messages relayed". To understand what messages relayed is you need to understand a little bit out my environment. My Linux Sendmail & MailScanner servers are front-end servers, meaning that these server scan inbound and outbound messages and if the messages are clean then they are relayed to internal corporate email servers. Basically you can consider messages relayed like this,

messages received - spam messages = messages relayed

Simple enough? Most of us probably won't care about messages relayed (and in fact I'm not even graphing it, because it's kind of infered in the graph automatically by the bit of green you can see on the graph).

Okay now that you've created the maillog watcher script (remember to make it executable, chmod 755 is your friend), you need to make sure it runs when you reboot your server. The simplest way is to just add it to your rc.local file. Just tag /opt/watchmaillog/watchmaillog.sh (or where ever you stored it) at the end. Let's not start the watchmaillog.sh script yet. I know you want to, but hold on, we're only about one quarter of the way there.

Next we need to create an empty counters file. Below is an empty counters file. Mine is stored in /opt/watchmaillog (which is the same place that the rest of the scripts reside for simplicity sake).
/opt/watchmaillog/watchmaillog_counters

Code: Select all

mess_recv:0
mess_rejected:0
mess_relay:0
mess_sent:0
mess_waiting:0
spam:0
virus:0

Now if you change the name or location of where the counter file is make sure to update the locations in the other scripts.

Read Counters Script
So here's the source code. Again, I've chosen to store my code in /opt/watchmaillog, but you can put it anywhere.
/opt/watchmaillog/readcounters.sh

Code: Select all

#!/usr/bin/perl
#
# Script called by Net-SNMP to read the watchmaillog counters
#
# By Jason Warnes
#
# 2006-09-18: Added new item mess_waiting, which is the number of messages MailScanner
#             detects when a new batch is started.
# 2006-10-11: Fixed bug where counter is flaged to be reset but the main watchmaillog
#             daemon hasn't run to reset the counter.  Don't reset it here, just return a
#             zero value.  (Thanks rbl!)
# 2006-11-08: Added new item mess_rejected, which is the number of rejected
#             messages by Sendmail.

$counterfile="/opt/watchmaillog/watchmaillog_counters";
$resetfile="/opt/watchmaillog/watchmaillog_reset";
$oid{"spam"}=".1.3.6.1.4.100.2";
$oid{"virus"}=".1.3.6.1.4.100.4";
$oid{"mess_recv"}=".1.3.6.1.4.100.5";
$oid{"mess_sent"}=".1.3.6.1.4.100.6";
$oid{"mess_relay"}=".1.3.6.1.4.100.7";
$oid{"mess_waiting"}=".1.3.6.1.4.100.8";
$oid{"mess_rejected"}=".1.3.6.1.4.100.9";

&readcounterfile;
# If the counter is nothing set it to zero
if($counter{$ARGV[0]}==""){$counter{$ARGV[0]}=0;}
# If a counter reset file exists set the counter to zero
if(-e $resetfile."_".$ARGV[0]){$counter{$ARGV[0]}=0;}
# Return the value of the counter
print "$oid{$ARGV[0]}\ngauge\n$counter{$ARGV[0]}\n";
&createresetflagfile;

sub readcounterfile {   # Subroutine to read the contents of the counter file
        open(COUNTER,$counterfile);
        while($line=<COUNTER>){
                @line=split(/\:/,$line);
                chop($line[1]); # Drop the trailing LF off the value
                $counter{$line[0]}=$line[1];
        }
        close(COUNTER);
}

sub createresetflagfile { 	# Subroutine to create the reset counter flag file
	open(RESET,">".$resetfile."_".$ARGV[0]);
	close(RESET);
	chmod(0666,$resetfile."_".$ARGV[0]);
}

Make sure to make the script executable. I'm again using the Net-SNMP capability to run scripts when particular SNMP OID's are queried. What happens here is when a particular OID is queried this script will return the value of the counter to the Net-SNMP and then create a 0 byte file to tell the watchmaillog.sh script to reset that counter to 0. Now you're probably wondering why wouldn't we just re-write the counter file with the value of that counter set to 0. Well (and this is from experience) when you have two processes that can write to the same file at different times the file can (and most likely will) become corrupt, and values may become unreadable. So having only one process writing to the file is an absolute must to keep the file consistent.

In order to get Net-SNMP to run the script you need to modify your snmpd.conf file and add these lines,
snmpd.conf additions

Code: Select all

pass .1.3.6.1.4.100.2 /opt/watchmaillog/readcounters.sh spam
pass .1.3.6.1.4.100.4 /opt/watchmaillog/readcounters.sh virus
pass .1.3.6.1.4.100.5 /opt/watchmaillog/readcounters.sh mess_recv
pass .1.3.6.1.4.100.6 /opt/watchmaillog/readcounters.sh mess_sent
pass .1.3.6.1.4.100.7 /opt/watchmaillog/readcounters.sh mess_relay
pass .1.3.6.1.4.100.8 /opt/watchmaillog/readcounters.sh mess_waiting
pass .1.3.6.1.4.100.9 /opt/watchmaillog/readcounters.sh mess_rejected

So you can see here the SNMP OIDs that are being used. Don't worry you won't need to remember these. The Data templates have them built in.

logrotate Configuration
In order to have logrotate restart the watchmaillog.sh script when it rolls the maillog file we need to modify the logrotate configuration for syslog because this is the one that handles the /var/log/maillog rotating. On my FC5 system I had to modify /etc/logrotate.d/syslog and now mine looks like this,
/etc/logrotate.d/syslog

Code: Select all

/var/log/messages /var/log/secure /var/log/maillog /var/log/spooler /var/log/boot.log /var/log/cron {
    sharedscripts
    postrotate
        /bin/kill -HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> /dev/null || true
        /bin/kill -HUP `cat /var/run/watchmaillog.pid 2> /dev/null` 2> /dev/null || true
    endscript
}

Basically I added a similar kill that the syslog gets.

Cacti Templates
Below are the data and graph templates that I made too. Then you'll need to,

1. Import the data and host templates.
2. Add the data sources to the Cacti device for your mail server. This is done using the Data Sources menu item on the console. The new data sources are
   • watchmaillog - MailScanner SPAM
   • watchmaillog - MailScanner Viruses
   • watchmaillog - Sendmail Messages Recieved
   • watchmaillog - Sendmail Messages Rejected
   • watchmaillog - Sendmail Messages Relayed
   • watchmaillog - Sendmail Messages Sent
   • watchmaillog - MailScanner Messages Waiting
3. Then add the watchmaillog - Sendmail & MailScanner Stats and watchmaillog - MailScanner Messages Waiting graphs to your mail server device. This is done through the Graph Management menut item in the console. Select the appropriate data sources for the graphs. They should be pretty self-explanitory as to what data source to use.

Now you can start the watchmaillog.sh script that you created a long time ago. (FINALLY hey?!).

Well I hope it works for everyone. It's been tested and running on Fedora Core 5, but the script isn't doing anything too crazy and should work on any other Linux or BSD distros with few (or no) modifications. I went into quite a bit of detail here that most probably don't care, but I wanted to make sure that people understood what was going on (in case some people wanted to hack it up a bit). And I'll try and be a bit more active in the forums to help people that may be having problems with it. But I do have a "day job" so most of my replies may come at night.

I already know what I want to do next to this version. I'm going to look at converting the counters to act more like true SNMP 32-bit integer counters, so instead of relying on the reset file to be created and the maillog watcher script noticing it and reseting the counter, I'll just let the counter wrap past the 32-bit max value and then start at zero again. This is how interface octet SNMP counters work. So I'll start a new thread once that version is ready.

[goodmasp]

Thanks very much for the updated scripts!

I have been using the last version successfully with small changes to work with Exim, our MTA of choice.



I'm fairly new to Cacti and RRDTOOL so although modifying your fine script to cope with exim won't be a problem I do have a couple of questions before I move onto the new structure.

1) Is there any downside to stating with the current Cacti templates? I would like to keep my existing graphs, I notice the OIDs remain the same.

2) I don't know about you, but we run a couple of MailScanner boxes to keep up with the mail. I like having individual graphs, but to complement this I have modified the Aggregate (2 sources) graph to show total spam bocoked using the existing data sources:



Is it possible to make a aggregate graph similar to your single host one, with all data combined to give a simple overview?

Thanks for your hard work; I am already looking forward to testing your new revision and daemonising the monitoring looks an ideal way forward.

Kind Regards,

Steve

[goodmasp]

I'm glad to report the new script/daemon works fine on our Exim/Mailscanner environment and is reporting using the "old" template for the time-being.

Previously I was monitoring the seperate log file for exim, though now both Exim and Mailscanner are reporting to a central logfile to minimise changes to the script. As most exim users will know this is achieved by using the log_file_path=syslog directive.

The changes to parse Exim logfiles are simply

Code: Select all

# Look for received messages where the sender is not from our domain(s)
                if(($line=~/\<\=/) && ($line!~/\@domain.com/)){
# Look for messages sent to our domain(s), indicates an inbound message relayed to an internal server
                if(($line=~/\=\>/) && ($line=~/\@domain.com/)){
 # Look for sent messages to NOT our email domain(s), indicates an outbound message
                if(($line=~/\=\>/) && ($line!~/\@domain.com/)){

Thanks very much!

[warnesj]

goodmasp, thanks a million for the Exim mod! Nice to see my programming madness could be modified to help.

1) Is there any downside to stating with the current Cacti templates? I would like to keep my existing graphs, I notice the OIDs remain the same.

Are you talking about using the existing data sources? I don't see any major problems with using the existing data sources, because the data that is being gathered is basically the same. The OID's are still GAUGE types, so it should be okay to re-use data sources to keep your historic data.

2) I don't know about you, but we run a couple of MailScanner boxes to keep up with the mail. I like having individual graphs, but to complement this I have modified the Aggregate (2 sources) graph to show total spam bocoked using the existing data sources:



Is it possible to make a aggregate graph similar to your single host one, with all data combined to give a simple overview?

I also run two MailScanner boxes. Great minds think alike. I have created a graph that does aggregate the two sources, but mine is a bit different than yours. I don't have each host in a different colour. I just reuse the same colour. Here is what mine looks like.

I didn't know if anyone would want it though so that's why I didn't include it in the original post. Sorry. I've attached the graph template I use to the bottom of this post. I hope it's what you're looking for. It should be easy to mod to have each host use different colours (if you want), you'll just need to modify the colours in the template once it's imported.

To use the template (you probably ran into this already, but I thought I'd mention it if anyone else tries to use it) you can't attach it to a host. When you create the graph just select the "None" host and then use the data sources from the mail servers you want. If you attach it to a specific host you can only use data sources from that host.

[warnesj]

I had a wierd problem that I never ran into before until now. When the logrotate ran to roll the my log files the watchmaillog script seemed to loose it connection to /var/log/maillog. I use this method of log "snooping" on other systems and this is the first time that the Perl script had problems getting hooked back into the log file.

I just killed the watchmaillog process and restarted it and everything came back okay but I was wondering if anyone else had this problem?

[raiten]

when i import last templates (mailscanner), i get:

Graph Template
[success] watchmaillog - Sendmail & MailScanner Stats [update]
+ Unmet Dependency: (Data Template Item)
+ Found Dependency: (GPRINT Preset) Normal
+ Unmet Dependency: (Data Template Item)
+ Found Dependency: (GPRINT Preset) Exact Numbers
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Found Dependency: (CDEF) Make Stack Negative
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)


also few remarks on script:
- why .sh for perl script ? (more like no extension or .pl, even if it doesn't matter on unix)
- prefer to use -w and "use strict" in perl script for some security ...
some messages like:
Use of uninitialized value in pattern match (m//) at ./watchmaillog.pl line 67.
Use of uninitialized value in pattern match (m//) at ./watchmaillog.pl line 76.
which comes from: "$line=~/is\ spam/"
why do you escape the space ?
- when running from command-line, Ctrl-C does not work, nor SIGHUP
^CSIGINT handler "signal_hanldler" not defined.
^CSIGINT handler "signal_hanldler" not defined.
SIGTERM handler "signal_hanldler" not defined.


Killed
- chmod: 644 seems better for me
- i would prefer an option to run the script from crontab (as unprivileged user) than daemon


else, about install:
i get the snmp part ok
imported template
adding data source
but no graph appears
there is no error message in cacti.log
a direction to look for ?

else thanks a lot for the scripts

[raiten]

the problem is in "3) graph management" in cacti.
when i select data source, in Supplemental Graph Template Data. it is not saved whatever i do ...

i have not any error messages in cacti log, system one or mysql.

coudl someone precise which entries are expected in mysql and which tables ?

thanks

[warnesj]

when i import last templates (mailscanner), i get:

Graph Template
[success] watchmaillog - Sendmail & MailScanner Stats [update]
+ Unmet Dependency: (Data Template Item)
+ Found Dependency: (GPRINT Preset) Normal
+ Unmet Dependency: (Data Template Item)
+ Found Dependency: (GPRINT Preset) Exact Numbers
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Found Dependency: (CDEF) Make Stack Negative
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)

What version of Cacti are you using? Whenever I've run into template import problems it's usually because the version that the template was created in doesn't match what version it's trying to be imported it. My verison is 0.8.6h.

also few remarks on script:
- why .sh for perl script ? (more like no extension or .pl, even if it doesn't matter on unix)

I use .sh extensions on all my scripts. It can be anything as long as it executable. Since the first like has #!/usr/bin/perl the shell will use that as the command interpreter for the script. So yeah you should be able to change the extension without too many problems.

- prefer to use -w and "use strict" in perl script for some security ...

I haven't tested it as an unpriviledged user, but I'll keep that in mind when I make the new version.

some messages like:
Use of uninitialized value in pattern match (m//) at ./watchmaillog.pl line 67.
Use of uninitialized value in pattern match (m//) at ./watchmaillog.pl line 76.
which comes from: "$line=~/is\ spam/"
why do you escape the space ?

I find that somtimes Perl doesn't like the white space when I'm trying to search for stuff. So I usually put an escape character there just so Perl knows that it's part of the whole string to search for. But it can probably be removed without a problem.

- when running from command-line, Ctrl-C does not work, nor SIGHUP
^CSIGINT handler "signal_hanldler" not defined.
^CSIGINT handler "signal_hanldler" not defined.
SIGTERM handler "signal_hanldler" not defined.


Killed

Yeah that was my bad. I didn't look close enough at the code. I've included a new version of the watchmaillog.sh script in this post, and I've updated the archive in the original post. This one will properly exit when Ctrl-C is pressed and when a SIGHUP is sent. Sorry about that.

- chmod: 644 seems better for me

That would make the script not runable, so you would need to execute perl to run the script. Again, it should work, but I haven't tested it like that.

- i would prefer an option to run the script from crontab (as unprivileged user) than daemon

Sounds okay, you'll just need to figure out how to handle the crontab schedule. By default the script will run forever, and if you have crontab schedule to kick off the script over and over you'll have multiple instances running which would be bad.

else, about install:
i get the snmp part ok
imported template
adding data source
but no graph appears
there is no error message in cacti.log
a direction to look for ?

You were having problems importing the graph template so that's probably why you don't have a graph appearing. Let's try to get that figured out and then we'll see if the graph will appear.

else thanks a lot for the scripts

You're welcome.

[warnesj]

I had a wierd problem that I never ran into before until now. When the logrotate ran to roll the my log files the watchmaillog script seemed to loose it connection to /var/log/maillog.

Well my problem came back again. Once logrotate ran My graphs went all "flat". So with the script update I've started writing out a /var/run/watchmaillog.pid file that we can use in the logrotate configuration to automatically restart the watchmaillog.sh script.

logrotate Configuration
In order to have logrotate restart the watchmaillog.sh script when it rolls the maillog file we need to modify the logrotate configuration for syslog because this is the one that handles the /var/log/maillog rotating. On my FC5 system I had to modify /etc/logrotate.d/syslog and now mine looks like this,
/etc/logrotate.d/syslog

Code: Select all

/var/log/messages /var/log/secure /var/log/maillog /var/log/spooler /var/log/boot.log /var/log/cron {
    sharedscripts
    postrotate
        /bin/kill -HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> /dev/null || true
        /bin/kill -HUP `cat /var/run/watchmaillog.pid 2> /dev/null` 2> /dev/null || true
        /opt/watchmaillog/watchmaillog.sh
    endscript
}

Basically I added a similar kill that the syslog gets and then restarted the script. HOPEFULLY this fixes my flat graphs. I've also added this section to the main post.

[raiten]

thanks for the reply

a few comment before the import problem.
- chmod: 644 is not about the script, but the counter file in the script
i run the script as nobody user in daemon mode for now, and i may want to check the files as me.
- for cron, the easy way seems to check maillog & modify counters without forking and exiting after. a file lock (pid in /var/run or /tmp) or something else could be used to avoid concurrency.

about the script, else, i move
local addresses in var $local_domains
and a rule for spam log detected by spamassassin:

# Look for MailScanner spam messages
if($line=~/is spam/
|| $line=~/spamd\: identified spam/){


else about the import problem, it's only this file:
cacti_graph_template_watchmaillog_sendmail_mailscanner_stats.xml

i downloaded it again today, try to import it, fails, reimport all others and it after, and the same.
i'm not sure only this template will make graphing fails.

i use version 0.8.6h, not sure about all the additional official patches.
one important note, my cacti is in a chrooted and systraced apache (systrace limits system-calls)
but i've managed to make it work and others scripts (default, mysql, asterisk, ...) work ok

i've nothing special when importing in system message (systrace) or apache error_log, same for cacti poller output.

i believe we can't ask for debug when importing template ? or doing it from command-line ?

thanks

[pvenezia]

when i import last templates (mailscanner), i get:

Graph Template
[success] watchmaillog - Sendmail & MailScanner Stats [update]
+ Unmet Dependency: (Data Template Item)
+ Found Dependency: (GPRINT Preset) Normal
+ Unmet Dependency: (Data Template Item)
+ Found Dependency: (GPRINT Preset) Exact Numbers
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Found Dependency: (CDEF) Make Stack Negative
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)
+ Unmet Dependency: (Data Template Item)

What version of Cacti are you using? Whenever I've run into template import problems it's usually because the version that the template was created in doesn't match what version it's trying to be imported it. My verison is 0.8.6h.

FWIW, I'm using a fresh install of 0.8.6h and I have the same problem with this template. I'm looking for a fix now.

[pvenezia]

This graph template appears to work for me in 0.8.6h. I'm still waiting for more data to fill in, but it renders properly.

[pvenezia]

I spent a little time with this to add the relaying data to the graphs and I've modified my watchmaillog.sh script to properly restart the daemon following a SIGHUP, and exit cleanly following a SIGTERM or SIGINT. Thanks for the work on this, warnesj. It's quite handy.

This tgz contains the modified watchmaillog.sh script, and two graph templates -- one modified original that imports properly (at least for me) and another that includes defs for mess_relay.

[warnesj]

a few comment before the import problem.
- chmod: 644 is not about the script, but the counter file in the script
i run the script as nobody user in daemon mode for now, and i may want to check the files as me.

Ah, I understand now. That makes sense. Yeah that's a good idea.

- for cron, the easy way seems to check maillog & modify counters without forking and exiting after. a file lock (pid in /var/run or /tmp) or something else could be used to avoid concurrency.

Seems cool. If you get it working be sure to tell me. I'd be curious to hear.

about the script, else, i move
local addresses in var $local_domains

Good idea. I was thinking of doing this for the next version too. Great minds think alike.

and a rule for spam log detected by spamassassin:

# Look for MailScanner spam messages
if($line=~/is spam/
|| $line=~/spamd\: identified spam/){

Perfect. I don't run spamd so that's a great mod. I'll be sure to include it in the next version (if you don't mind). I'll put credit's in for ya for the mod.

else about the import problem, it's only this file:
cacti_graph_template_watchmaillog_sendmail_mailscanner_stats.xml

i downloaded it again today, try to import it, fails, reimport all others and it after, and the same.
i'm not sure only this template will make graphing fails.

i use version 0.8.6h, not sure about all the additional official patches.
one important note, my cacti is in a chrooted and systraced apache (systrace limits system-calls)
but i've managed to make it work and others scripts (default, mysql, asterisk, ...) work ok

i've nothing special when importing in system message (systrace) or apache error_log, same for cacti poller output.

i believe we can't ask for debug when importing template ? or doing it from command-line ?

thanks

That last XML file is the graph unfortunately. I noticed that pvenezia has posted the XML for that graph in his post. Can you try and import his and see what happens. Otherwise I'll try and export it again and export all the dependencies individually (there are just a few CDEFS that I have in there), and maybe that will work to import.

I spent a little time with this to add the relaying data to the graphs and I've modified my watchmaillog.sh script to properly restart the daemon following a SIGHUP, and exit cleanly following a SIGTERM or SIGINT. Thanks for the work on this, warnesj. It's quite handy.

SWEETNESS!! Thanks for the new code. Can I use it in the next version? I'll be sure to put credits in for ya in the comments.

[warnesj]

This graph template appears to work for me in 0.8.6h. I'm still waiting for more data to fill in, but it renders properly.

Thanks pvenezia for the fix. Did you manage to see what the unmet dependecies were?