Dead Host Notification - Email Flooding

Support questions about the Threshold plugin

Moderators: Developers, Moderators

skirmish
Posts: 8
Joined: Wed Mar 31, 2010 10:26 pm

Dead Host Notification - Email Flooding

Post by skirmish »

At first downed host notifications were sending one notification per minute per host with re-alerting set to 0 minutes.
This has stopped but now when a host comes up i get literally dozens of mails per host simultaneously with a "returned from DOWN" message.

Cacti Version: 0.8.7g, Plugin Architecture Version: 2.8, Thold 0.4.3 on Debian Lenny
User avatar
dragossto
Cacti User
Posts: 86
Joined: Tue May 15, 2007 5:24 am
Location: Romania
Contact:

Re: Dead Host Notification - Email Flooding

Post by dragossto »

it depend how you set the
Host Up/Down Settings from the poller category
if set Failure Count to 0 it possible to happen what you say,i don't make test whit my machine

i have it set to the value 2
and have more device monitored and collect graph from them,more then 3 years

and the re alert it for minimum the poller interval
because if set to 0 will send all the re alert at once
so will have more then one email :)

Hope to help you
Cacti - 0.8.7h
Plugin Architecture - 3.0
Poller Type - SPINE 0.8.7g
Server Info - Linux 2.6.35.10 - Slackware 13.1
Web Server - Apache/2.2.17 (Unix) DAV/2 PHP 5.3.5
MySQL - 5.1.53 ;RRDTool - 1.4.5 ;SNMP - 5.6.1
Plugins
  • (tools - v0.3) (Thold - v0.4.6) (Ntop - v0.2) (Discovery - v1.2a) (Monitor - v1.2)
skirmish
Posts: 8
Joined: Wed Mar 31, 2010 10:26 pm

Re: Dead Host Notification - Email Flooding

Post by skirmish »

Thanks for the reply dragossto.

My Settings are:

Cacti Settings (Poller)
Host Up/Down Settings
Failure Count 3
Recovery Count 3

Cacti Settings (Thresholds)
Default Alerting Options
Default Trigger Count 5
Re-Alerting 0

I thought and hoped that Re-Alerting 0 would disable re-alerting. Even if not, I don't understand why I would be getting multiple alerts that that the host is up!
User avatar
dragossto
Cacti User
Posts: 86
Joined: Tue May 15, 2007 5:24 am
Location: Romania
Contact:

Re: Dead Host Notification - Email Flooding

Post by dragossto »

how many emails did you receive ?
Cacti - 0.8.7h
Plugin Architecture - 3.0
Poller Type - SPINE 0.8.7g
Server Info - Linux 2.6.35.10 - Slackware 13.1
Web Server - Apache/2.2.17 (Unix) DAV/2 PHP 5.3.5
MySQL - 5.1.53 ;RRDTool - 1.4.5 ;SNMP - 5.6.1
Plugins
  • (tools - v0.3) (Thold - v0.4.6) (Ntop - v0.2) (Discovery - v1.2a) (Monitor - v1.2)
skirmish
Posts: 8
Joined: Wed Mar 31, 2010 10:26 pm

Re: Dead Host Notification - Email Flooding

Post by skirmish »

Hi dragossto, I estimate each host generated between 5 and 25 alerts on its way up. I started deleting them as soon as they got to more than 5 alerts per host but they were being sent for at least an hour after the host was up.
User avatar
dragossto
Cacti User
Posts: 86
Joined: Tue May 15, 2007 5:24 am
Location: Romania
Contact:

Re: Dead Host Notification - Email Flooding

Post by dragossto »

in Threshold u have another Re-Alert Cycle setting
set that one to never

another idea i don't have , because i use thold only for email notification for dead host
Cacti - 0.8.7h
Plugin Architecture - 3.0
Poller Type - SPINE 0.8.7g
Server Info - Linux 2.6.35.10 - Slackware 13.1
Web Server - Apache/2.2.17 (Unix) DAV/2 PHP 5.3.5
MySQL - 5.1.53 ;RRDTool - 1.4.5 ;SNMP - 5.6.1
Plugins
  • (tools - v0.3) (Thold - v0.4.6) (Ntop - v0.2) (Discovery - v1.2a) (Monitor - v1.2)
skirmish
Posts: 8
Joined: Wed Mar 31, 2010 10:26 pm

Re: Dead Host Notification - Email Flooding

Post by skirmish »

Thanks for the reply dragossto, the only re-alerting setting i could find was the one in

Cacti Settings (Thresholds)
Default Alerting Options
Default Trigger Count 5
Re-Alerting 0

I only have downed host notifications, no additional thresholds set. Just this morning I received 14 UP messages from a single host and 12 from another spanning a period of just over 2/12 hrs for a single event.

Thold doesn't seem to have registered this event - the thold page looks like this

Server 19 1 1 Up 0 xx.xx.xx.30 47.65 68.21 95.67
Router 5 2 2 Up 0 xx.xx.xx.1 37.81 85.09 94.73

And I haven't received any downed host notifications yet.

Are there any other settings or debug I could check? I could only find one re-alerting setting in thold settings, and that is set to 0.
User avatar
dragossto
Cacti User
Posts: 86
Joined: Tue May 15, 2007 5:24 am
Location: Romania
Contact:

Re: Dead Host Notification - Email Flooding

Post by dragossto »

ok
it little mistake in code, i understand now your problem
even it down or up you receive email only whit up :)
that it fix in my modif thold from hear,
http://forums.cacti.net/viewtopic.php?f=17&t=36235

and the re-alerting setting it only for Thresholds
i don't receive extra emails and the code also in not designed to send re alerts for one host go down or up and stay there
for a long time,only receive email when cacti declare that host down or up

only setting that affect that it under polling settings
Host Up/Down Settings
Failure Count 2
Recovery Count 1
Cacti - 0.8.7h
Plugin Architecture - 3.0
Poller Type - SPINE 0.8.7g
Server Info - Linux 2.6.35.10 - Slackware 13.1
Web Server - Apache/2.2.17 (Unix) DAV/2 PHP 5.3.5
MySQL - 5.1.53 ;RRDTool - 1.4.5 ;SNMP - 5.6.1
Plugins
  • (tools - v0.3) (Thold - v0.4.6) (Ntop - v0.2) (Discovery - v1.2a) (Monitor - v1.2)
beone
Posts: 4
Joined: Thu Mar 31, 2011 12:27 am

Re: Dead Host Notification - Email Flooding

Post by beone »

dragossto wrote:ok
it little mistake in code, i understand now your problem
even it down or up you receive email only whit up :)
that it fix in my modif thold from hear,
http://forums.cacti.net/viewtopic.php?f=17&t=36235
No, it's not that. I have the same problem, but on DOWN notifications.
I receive 3 e-mails everytime a host goes DOWN, on UP it's only one e-mail.
I already use your code from svn, but the problem remains.
User avatar
dragossto
Cacti User
Posts: 86
Joined: Tue May 15, 2007 5:24 am
Location: Romania
Contact:

Re: Dead Host Notification - Email Flooding

Post by dragossto »

hy , once again

it some thing wrong in the setting , i use
this thold ( modified by me ) for over 400 devices
and i don't have this problem

one question , did u have that ip multiple time in yours devices ?
Cacti - 0.8.7h
Plugin Architecture - 3.0
Poller Type - SPINE 0.8.7g
Server Info - Linux 2.6.35.10 - Slackware 13.1
Web Server - Apache/2.2.17 (Unix) DAV/2 PHP 5.3.5
MySQL - 5.1.53 ;RRDTool - 1.4.5 ;SNMP - 5.6.1
Plugins
  • (tools - v0.3) (Thold - v0.4.6) (Ntop - v0.2) (Discovery - v1.2a) (Monitor - v1.2)
beone
Posts: 4
Joined: Thu Mar 31, 2011 12:27 am

Re: Dead Host Notification - Email Flooding

Post by beone »

dragossto wrote:hy , once again

it some thing wrong in the setting , i use
this thold ( modified by me ) for over 400 devices
and i don't have this problem

one question , did u have that ip multiple time in yours devices ?
Well, I use cacti on +500 devices too.
And each time a host goes down I get 3 e-mails confirming downtime of the same host.
Can you point me to the code where the mails are generated? I 'll take a look.
That "3" must come from somewhere....
PS 1 device = 1 unique ip, so it's not that.

Also, I got this option Default Trigger Count set to 7 and I poll every 5 minutes, so expect an e-mail after 30minutes of down host, I get it after 10 minutes :-(
There sure is somethign wrong, but what?
EDIT: Ok, about the last part, I see it uses the setting on the poller settings page, so that's ok. But still no explanation for the multiple e-mails...

kind regards
User avatar
dragossto
Cacti User
Posts: 86
Joined: Tue May 15, 2007 5:24 am
Location: Romania
Contact:

Re: Dead Host Notification - Email Flooding

Post by dragossto »

Hello , sorry for the delay
for dead host notification count the settings from cacti
under poller settings

Host Up/Down Settings
Failure Count

to me the Failure Count it 2

and the polling interval have to be same value like in your crond (schedule pollings)

the setting for alerting in the thold setting plugin it only for Thresholds
the only one ho have influent over email body and subject it Emailing Options,but they do not have influent
over the number of emails sent , it a little strange for me , and i don't understand why
you receive 3 emails for dead hosts ,

i will try to make some test whit some tresholds make for some devices to see what happen.
if i receive more emails for dead hosts , but in my opinion that should not happen ,because the part whit email for dead host and the one whit trigger for a Threshold they are different functions.

hope to figure out why happen to you.
Cacti - 0.8.7h
Plugin Architecture - 3.0
Poller Type - SPINE 0.8.7g
Server Info - Linux 2.6.35.10 - Slackware 13.1
Web Server - Apache/2.2.17 (Unix) DAV/2 PHP 5.3.5
MySQL - 5.1.53 ;RRDTool - 1.4.5 ;SNMP - 5.6.1
Plugins
  • (tools - v0.3) (Thold - v0.4.6) (Ntop - v0.2) (Discovery - v1.2a) (Monitor - v1.2)
beone
Posts: 4
Joined: Thu Mar 31, 2011 12:27 am

Re: Dead Host Notification - Email Flooding

Post by beone »

Yes, I don't understand too.
I have no other tresholds set, only host UP/DOWN notification is activated.
MrRat
Cacti User
Posts: 136
Joined: Thu Jan 07, 2010 10:33 am

Re: Dead Host Notification - Email Flooding

Post by MrRat »

Has anyone found the solution to this issue yet? I have the same problem. I get one down and multiple up on "some" hosts not all.
mudmud
Posts: 21
Joined: Sun Oct 28, 2012 2:56 pm

Re: Dead Host Notification - Email Flooding

Post by mudmud »

Hi everyone...

Has any of you been able to solve this issue? I am monitoring about 60 devices ...polling every minute. I think I encountered this problem after starting to use the PHP mail function to send the emails. Before I was using the SMTP settings on Cacti. (<<I am not 100% sure about this yet).

Please let me know what mail function that you are using and if you've already found a solution please share.

Thanks.

D.
Post Reply

Who is online

Users browsing this forum: No registered users and 3 guests