SNMPTT/SYSLOG viewer Plugin for Cacti. v 1.4.3 (2009/02/06)

[egarnel]

http://www.cisco.com/univercd/cc/td/doc ... #wp1016309

[egarnel]

Got this working on CactiEZ

extra tips:
create /var/spool/snmptt directory
enable snmpttsystem logging in snmptt.ini
copy the snmptt.conf.generic from the examples dir from the snmptt tarball to /etc/snmp
Make sure that the user specified in snmptt.ini can access cacti db
Make sure that port 162 udp is allowed in from the devices and/or networks that you want to receive traps from

One more thing:
to send traps from other linux boxes, add the following to snmpd.conf
trapsink <ip address of cacti server>

[niobe]

Thanks for great explanation lthe, conf files make a lot more sense now.

Problem fixed too, it was the authCommunity line needed "log,execute,net"

I am now seeing all expected files:

Code: Select all

[root@server snmp]# ls -l
total 1044
drwxr-xr-x 2 root netcomms   4096 Apr 24 00:01 archive
-rw-rw-rw- 1 root root      38665 Apr 24 09:49 snmptrapd.log
-rw-r--r-- 1 root root     897117 Apr 24 09:49 snmptt.debug
-rw-rw-rw- 1 root root       8566 Apr 24 09:49 snmptt.log
-rw-r--r-- 1 root root       1292 Apr 24 09:48 snmpttsystem.log
-rw-rw-rw- 1 root root      73248 Apr 24 09:49 snmpttunknown.log

(my conf file so far just has standard MIBS that is why there is a lot of unknown)

cheers!

N

[niobe]

Hi gthe,

Excellent, I have the plugin working now. A couple of ideas I want to propose..

1) I would prefer to see the Purge and Statistics as item under Cacti System Utilities rather than their own menu. I have had a go implementing this and seems to work.
-Download the attached setup.fork.php.
-Backup your existing setup.php, then copy over it with the forked one. -Also copy the attached utilities to plugins/snmptt/lib
-Then disable/enable the plugin.
What do you think? It may just be a personal preference of mine to keep the main menu uncluttered.

2) No need I think to link to Traps & Infos in the console since you have the big SNMPTT tab at the top of the screen

3) Filtering could be enhanced - I logged 48000 traps in an hour so I would like to be able to display more rows Also filter by host and host template (can the navbar headings be hyperlinked?). If you are interested to see it I am happy to help with implementing this, let me know.

4) Seems to be a bug with filtering unknown traps - regexp does not work and paging breaks - will post details later

gthe, fantastic effort for a first plugin and much needed

N

[gthe]

Hi gthe,

Excellent, I have the plugin working now. A couple of ideas I want to propose..

1) I would prefer to see the Purge and Statistics as item under Cacti System Utilities rather than their own menu. I have had a go implementing this and seems to work.

Added to next release, thanks.

2) No need I think to link to Traps & Infos in the console since you have the big SNMPTT tab at the top of the screen

Agree

3) Filtering could be enhanced - I logged 48000 traps in an hour so I would like to be able to display more rows

will be added Row count field.

3) Also filter by host and host template (can the navbar headings be hyperlinked?).

OK

4) Seems to be a bug with filtering unknown traps - regexp does not work and paging breaks - will post details later

I will check it. (I simply not have unknown traps ). Besides unknown traps has no opportunity of creation of rules.

...........
-Then disable/enable the plugin.
...............

With PIA 2.0 and 2.1 (don't check 2.2) you need uninstall/install plugin (with delete all data in tables). It is PIA bug - hooks added only when installing plugin

Many thanks for your help and advice

[chrisgapske]

Can anyone give or post a basic example of snmptrapd.conf that would be needed.

And what is a good way to see if you are able to get traps ?

I know I a newbie for this .

[egarnel]

Code: Select all

# Configuration File
# /etc/snmp/snmptrapd.conf
# last change: Mon Oct 26 19:44:36 1998
# This file is used to set the configuration for logging on
# Flash Disk, RAM Disk and external System via Traps
# Log Levels for Flash Disk and RAM Disk
authCommunity log public
traphandle default /usr/sbin/snmptthandler
LogFlash 1 1000
LogRAM 10 32768
# Trap Sink addresses
TrapSink 0 5 public 192.168.100.29
# Generic Traps
Generic 0 1 ColdStart
Generic 1 10 WarmStart
Generic 2 10 IFDown
Generic 3 10 IFUp
Generic 4 10 InvalidCommunity
Generic 5 10 EGPdown
# Enterprises Traps
Enterprise 1 4 passwordFail
Enterprise 2 4 switchWarning
Enterprise 3 4 switchLOS
Enterprise 4 4 switchErrorLock
Enterprise 5 4 switchBackupLineFail
Enterprise 6 4 switchChangedLine
Enterprise 7 4 snmpFail
Enterprise 8 4 chanFail
Enterprise 9 4 chanLasLocCurrOOR
Enterprise 10 4 chanLasRemCurrOOR
Enterprise 11 4 chanTempOOR
Enterprise 12 4 chanClockrecFail
Enterprise 13 4 chanCommunicationWar
Enterprise 14 4 chanRecremLOS
Enterprise 15 4 chanReclocLOS
Enterprise 16 4 fanFail
Enterprise 17 4 edfaFail
Enterprise 18 4 psFail
Enterprise 19 4 psOK
Enterprise 20 4 chanRecremNoLOS
Enterprise 21 4 chanReclocNoLOS
Enterprise 22 4 chanHardwareAdd
Enterprise 23 4 chanHardwareDel
Enterprise 24 4 chanClockrecNoFail

[cigamit]

I will give this one a try and see how it goes. Interesting enough, I originally wrote a snmptt viewer such as this (if I remember correctly it was called Events plugin) before I found and worked on the syslog plugin (originally called haloe). Main problem I had back then was the slowness of snmptt when translating the results. I had some pretty graphs at the time monitoring the number of incoming traps for several hours, but I could only do roughly 100 traps a second while killing the CPU on my box, while I could do ~600 syslog messages a second and still not have many issues at all.

[TheWitness]

I will explore these things in time. Right now it's not the time. Could be a index issue and could be allayed using some tricky SQL stuff.

Larry

[niobe]

Hi gthe,

My snmptt is becoming grindingly slow..

Code: Select all

$ tail -f /var/log/mysqld_slow.log
/usr/libexec/mysqld, Version: 5.0.22-log. started with:
Tcp port: 0  Unix socket: /var/lib/mysql/mysql.sock
Time                 Id Command    Argument
# Time: 080502 15:22:01
# User@Host: netman[netman] @ localhost []
# Query_time: 11  Lock_time: 0  Rows_sent: 50  Rows_examined: 1148014
use cacti;
SELECT plugin_snmptt.*, host.description, host.host_template_id, host.id as device_id, host.status, host.disabled FROM plugin_snmptt     Left join host on (plugin_snmptt.hostname=host.hostname)          ORDER BY description ASC LIMIT 0,50;

So 11 seconds to load the front page. Here is the unknown traps page:

Code: Select all

# Time: 080502 15:28:31
# User@Host: netman[netman] @ localhost []
# Query_time: 25  Lock_time: 0  Rows_sent: 50  Rows_examined: 687318
SELECT plugin_snmptt_unknown.*, host.description, host.host_template_id, host.id as device_id, host.status, host.disabled FROM plugin_snmptt_unknown     Left join host on (plugin_snmptt_unknown.hostname=host.hostname)          ORDER BY hostname ASC LIMIT 0,50;

25 seconds with less record! Here is the problem (I think):

The query selects only 50 rows but it has to read and sort ALL of them due to the ORDER by statement. Sorting is usuallly nlogn time so it climbs disproportionately with the number of traps. Even with a 2 week purge I am likely to have several million traps in the DB at any one time. Sorting is of course a desirable feature.

By the way this is running on an eight core machine with 8GB RAM and RAID5 15k SCSI drives, so not too shabby.

Just wanted to bring this to your attention, I know you will have some idea from your mactrack post http://forums.cacti.net/viewtopic.php?t=25753

cheers,

N

[niobe]

just thinking about this I suppose an index for every sortable column is the way to solve it

[chrisgapske]

OK I could use some help.


all I seem to get it this ..
CACTI02.fiber.local 2008-05-06 17:35:10 snmptt-sys[501]: Total traps received=0,Total traps translated=0,Total traps ignored=0,Total unknown traps=0 local0 warning

CACTI02.fiber.local 2008-05-06 17:30:10 snmptt-sys[501]: Total traps received=0,Total traps translated=0,Total traps ignored=0,Total unknown traps=0


Could sombody lead me in the right direction. I know the end devices are sending traps.

[niobe]

Hi Chris,

I would break it down like this..

1) Verify traps being received on server network interface using tcpdump, wireshark or similar. Look for UDP port 162 coming in.

2) Check any firewall configuration/logs on the server to ensure it's not then blocking

3) Verify there is a service listening for snmptraps (port 162) using 'netstat -an' (works on windows and unix)

4) Verify the operation of the service (snmptrapd) by checking logs, double-checking config etc.

With my configuration snmptrapd.log still stores traps regardless of whether snmptt is running. But if snmptrapd isn't working it won't be passing traps to the handler.

5) Verify operation of the trap handler snmptt by checking logs etc.

Hope that helps to start..

N

[gthe]

I upload new version in first post.

2 niobe:
Thanks for your help. Did You read my email ? In mysql data you have errors: all data repeat 2 times
Look at this.

[niobe]

Hi gthe,

I created a few different .conf files use MIB packs from different vendors. Since these are not perfect and some re-include the standard RFC MIBS no doubt there are duplicates.

Is there a way for snmptt to find and avoid these duplicates itself?

But I think even half as many traps coming in is still quite a few, so performance could still be a concern.