Walkthrough on Postifx monitoring with Cacti

[otaviof]

Hi there,

i made this script work for my configuration of Postfix/ClamAv/Amavisd-New/SpamAssasin this morning (i am using a Sarge Debian fully upgraded). I had to modify both amavisd logformat and fetch_mail_statistics.pl scripts. Here are my modifications :

/etc/amavisd/amavisd.conf

Code: Select all

$log_templ = '[?%#D|Not-Delivered|Passed] [? %#V |[? %#F |[? %#X |CLEAN|BAD-HEADER]|BANNED name/type (%F)]|INFECTED (%V)], #
[?%o|(?)|<%o>] -> [<%R>|,][? %i ||, quarantine %i], Message-ID: %m, Hits: %c';

fetch_mail_statistics.pl

Code: Select all

  while( <LOGFILE> ) {
    if ( / postfix\/smtpd/ && /client=/ && ! /127.0.0.1/ ) { $num_rec++; }
    elsif ( /status=sent/ && ! /relay=(mailfilter|procmail|local|127.0.0.1)/ ) { $num_sent++; }
    elsif ( /status=deferred/ ) { $num_deferred++; }
    elsif ( /status=bounced/ ) { $num_bounced++; }
    elsif ( /: NOQUEUE: reject: / ) { $num_reject++; }
    elsif ( / Passed CLEAN/  && /Hits: ([-+]?[0-9]*\.?[0-9]+)/) {
        $hits = $1;
        $num_clean++ if ($hits < $tag_level);
        $num_pspam++ if ($hits > $tag_level);
        }

    elsif ( / Not-Delivered CLEAN/  && /Hits: ([-+]?[0-9]*\.?[0-9]+)/) {
        $hits = $1;
        $num_clean++ if ($hits < $tag_level);
        $num_spam++ if ($hits > $tag_level);
        }

    elsif ( / Passed INFECTED/ ) { $num_infected++; }
    elsif ( / Not-Delivered INFECTED/ ) { $num_infected++; }
    elsif ( / Passed BAD-HEADER/ ) { $num_header++; }
    elsif ( / Not-Delivered BAD-HEADER/ ) { $num_header++; }
    elsif ( / Blocked BANNED/ ) { $num_banned++; }
    elsif ( / Not-Delivered BANNED/ ) { $num_banned++; }

    }

To use this modification, you must add at the beginning of the script a SPAMLVL variable with your local value. You can find this value in /etc/amavisd/amavisd.conf if you look for

$tag2_level

parameter.
Here is an exemple of such a line :

Code: Select all

$SPAMLVL  = 2.0;

I hope this will help...

Mathieu

Hey Mathieu,

Very nice job. But I have an suggestion:

You can make your script read amavisd.conf $sa_tag2_level_deflt variable automatically, if you include this modification:

<code>
#
# This defines how many seconds we will cache calculated values.
#
our $sa_tag2_level_deflt;
require q{/usr/local/etc/amavisd.conf};

$MINRESET = 50;
$SPAMLVL = $sa_tag2_level_deflt;
</code>

thank you indeed.

--
Otávio Fernandes <otaviof at gmail dot com>
http://otaviof.blogspot.com

[enash]

Can someone help here, I have read all 6 pages of this thread and found a couple other people with the same problem as I do and they have yet to get an answer. I hae a random dtat graph. as attached at the bottom. Since i am not using Spam filter on the postfix server and its handled internally. But I do use my access.db file to block out all interrnational IP addresses. I am not much of an XML guy, and when I made the changes I try to re add that xml file so it would update the template, and bam got an XML parse error. I also have helo and header restrictions on. As for the helo u must use your FQDN etc.. I wanted to replace the spam field with the addition all mail my config filters. Here is my XML. What i want is to monitor how much mail is incoming from the external, not just whats coming in from my internal exchange server and external, and be able to graph all my blocks to international IP's, bad helo responses, etc....

After i loaded this XML i had a Mail Rcvd graph. I went in and pulled the OID out of the graph temp and ran a walk against that OID and i was returning the following..

[root@mysyslogsrv01 bin]# snmpwalk -v2c -c zzzzzz localhost .1.3.6.1.4.1.2021.255.0
UCD-SNMP-MIB::ucdavis.255.0 = INTEGER: 530

[root@mysyslogsrv02 bin]# snmpwalk -v2c -c zzzzzz mysyslogsrv01 .1.3.6.1.4.1.2021.255.0
UCD-SNMP-MIB::ucdavis.255.0 = INTEGER: 341

[root@mysyslogsrv02 bin]# snmpwalk -v2c -c zzzzzz mymail01 .1.3.6.1.4.1.2021.255.0
UCD-SNMP-MIB::ucdavis.255.0 = INTEGER: 556

[root@mysyslogsrv02 bin]# snmpwalk -v2c -c zzzzzz mymail02 .1.3.6.1.4.1.2021.255.0
UCD-SNMP-MIB::ucdavis.255.0 = INTEGER: 507

[enash]

Trying to bump this up for some help.. I still have these gaps in my graph.. Is this due to the poller running every 5 mins? Here is another example. Our company is a pretty big one. And we are one of the bigger online retailers. So we have a lot of mail that is flowing in and out or our shop. In my postfix config I am blocking a ton of international IP ranges. and Im using RBL lists as well. I have a ton of hits in my log file like this

postfix/smtpd[10338]: NOQUEUE: reject: RCPT from unknown[78.172.90.86]: 504 5.5.2 <78.172.90.86>: Helo command rejected: need fully-qualified hostname;

so I know i should have more stats than this in my graph. Can anyone point me in the right directions. whether I have to adjust the graph sizes, or something on the host side. etc..

thanks...

[eugenevdm]

Here are some troubleshooting tips:

1. Turning on debugging in Caciti / Console / Settings / Poller Logging Level from "LOW - Statistics and Errors" to the next level "MEDIUM..." is very useful.

2. Check under which user your SNMPD is running. On Ubuntu check /etc/default/snmpd to see which user it's running under

3. Be sure to restart snmpd after making changes

4. For all those troubleshooting just using 'snmpwalk', snmpwalk returns empty if you haven't run the script first. So while you're testing run the script first and then snmpwalk.

5. If you're running default polling at 5 minutes I agree with the one author that "$MINRESET = 298;" should be changed from the script writer's default of 50.

6. If you get "WARNING: Result from SNMP not valid. Partial Result: No Such Instance cur" it could be that the script is not running properly. The script might not run properly due to permissions. Permissions might be off due to context of SNMPD user or script / file permissions. Run the script on the console and see if on the next poll that messages disappears. If so examine permissions.

7. Check the rights on the mailstat.db file

8. Check the rights on the script

[martin_nl]

I have installed this script, but have one problem.

I get only results in my cacti log when i add this line to a cron job, and
run it once a minute.

/usr/local/bin/fetch_mail_statistics.pl /var/log/mail.log /var/tmp/mailstats.db .1.3.6.1.4.1.2021.255 -n .1.3.6.1.4.1.2021.255.1


When i don't add this line to a cron job the cacti log only shows value: U

[jangrewe]

Thanks to everybody for their work, i finally got the script to run under Debian "squeeze".
The trick i used was adding the 'snmp' user, as which snmpd runs, to the 'adm' group - this allowed the script to read the mail log.

Code: Select all

usermod -aG adm snmp

You can find out if your snmpd's user has enough permissions to read the logfile like this (change to your respective paths):

Code: Select all

sudo -u snmp /usr/bin/fetch_mail_stats.pl /var/log/mail.log /var/lib/cacti/mailstats.db .1.3.6.1.4.1.2021.255 -g .1.3.6.1.4.1.2021.255

[Creaton]

Useful stuff. I need both for the practical results. I’ve seperated certain log functions using the "localX" logging facility to better control the log data obtained on a loaded production server.

[pexotinec]

Guys any idea ?
snmpwalk -v 1 -c public localhost .1.3.6.1.4.1.2021.255
/usr/local/bin/snmpwalk: line 1: 3310246: command not found
/usr/local/bin/snmpwalk: line 2: 0: command not found
/usr/local/bin/snmpwalk: line 3: 463: command not found
/usr/local/bin/snmpwalk: line 4: 91: command not found
/usr/local/bin/snmpwalk: line 5: 437: command not found
/usr/local/bin/snmpwalk: line 6: 2667: command not found
/usr/local/bin/snmpwalk: line 7: 0: command not found
/usr/local/bin/snmpwalk: line 8: 0: command not found
/usr/local/bin/snmpwalk: line 9: 0: command not found
/usr/local/bin/snmpwalk: line 10: 0: command not found
/usr/local/bin/snmpwalk: line 11: 0: command not found
/usr/local/bin/snmpwalk: line 12: 0: command not found

[Lewis]

hello all,

installed the script, installed the template.. changed $MINRESET = 298;
checked permissions, etc..

i get these on my cacti's log file..

Code: Select all

08/02/2015 05:35:27 PM - CMDPHP: Poller[0] Host[14] DS[171] SNMP: v1: xx.xx.xx.xx, dsname: postfix_sent, oid: .1.3.6.1.4.1.2021.255.1, output: U
08/02/2015 05:35:27 PM - CMDPHP: Poller[0] WARNING: SNMP Get Timeout for Host:'xx.xx.xx.xx', and OID:'.1.3.6.1.4.1.2021.255.1'

when i run the script locally, i get:

Code: Select all

root@:/usr/local/bin# /usr/local/bin/fetch_mail_statistics.pl /var/log/mail.log /var/tmp/mailstats.db .1.3.6.1.4.1.2021.255 -n .1.3.6.1.4.1.2021.255.1
.1.3.6.1.4.1.2021.255.2
integer
49

[Lewis]

its driving me mad... i'm trying the last 8 hours to get some data

Code: Select all

root@student:~# sudo -u snmp /usr/local/bin/fetch_mail_statistics.pl /var/log/mail.log /var/log/mailstats.db .1.3.6.1.4.1.2021.255 -n .1.3.6.1.4.1.2021.255.1
.1.3.6.1.4.1.2021.255.2
integer
967
root@student:~# snmpwalk -v 1 -c public localhost .1.3.6.1.4.1.2021.255
root@student:~# 

hello all,

installed the script, installed the template.. changed $MINRESET = 298;
checked permissions, etc..

i get these on my cacti's log file..

Code: Select all

08/02/2015 05:35:27 PM - CMDPHP: Poller[0] Host[14] DS[171] SNMP: v1: xx.xx.xx.xx, dsname: postfix_sent, oid: .1.3.6.1.4.1.2021.255.1, output: U
08/02/2015 05:35:27 PM - CMDPHP: Poller[0] WARNING: SNMP Get Timeout for Host:'xx.xx.xx.xx', and OID:'.1.3.6.1.4.1.2021.255.1'

when i run the script locally, i get:

Code: Select all

root@:/usr/local/bin# /usr/local/bin/fetch_mail_statistics.pl /var/log/mail.log /var/tmp/mailstats.db .1.3.6.1.4.1.2021.255 -n .1.3.6.1.4.1.2021.255.1
.1.3.6.1.4.1.2021.255.2
integer
49