Unable to receive alerts from Thold

[TheWitness]

0.4.8-3 was released last night.

[ARC1450]

I'm not receiving updates since I upgraded THOLD. In the log it shows that it stopped check for threshold violations almost the minute I upgraded.

I grabbed the latest (0.4.8-3).

Any clues?

Just went back to (0.4.4) and email works like a charm.

[TheWitness]

none.

[ARC1450]

Okay, so I went back to 0.4.4 and now THold reports as checking all 473 tholds and it is sending email out just fine.

*sigh* I love problems like these, especially when this crap works for everyone else.

[TheWitness]

Make sure the 'Realert Cycle' is not 'None'. Realert is working for me. So, I'm not sure what to tell you without your database.

[jjarrettc]

Got it working.

The patch file contained within cacti-plugin-0.8.7g-PA-v2.9.tar.gz did not patch poller.php correctly because the ubuntu cacti release, at least in my case, had a slightly different line structure in poller.php, causing the patch to fail. I missed this originally. I went through the patch file for poller.php line by line to verify the patches for poller.php did get inserted into poller.php. I caught that a critical section did not get added (at the bottom of poller.php). So I added the missing line which was:

api_plugin_hook('poller_bottom');

and thold did a marvelous job. Awesome plugin.

Best regards,
John

[escort146]

Hi,

It is working fine, the issue was PHP version. Thanks.


Prasanna

[ARC1450]

Okey dokey. . .here's my PHP version info:

PHP 5.2.14 (cli) (built: Jul 21 2010 18:43:55)
Copyright (c) 1997-2010 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2010 Zend Technologies

I'm running .8.7g on Windows. THold works great with v4.4, but as soon as I upgrade to v4.8, no go. I did notice in the Cacti log these lines when I upgraded:

10/28/2011 08:47:45 AM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1064', SQL:"ALTER TABLE thold_template REMOVE COLUMN bl_enabled'
10/28/2011 08:47:45 AM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1064', SQL:"ALTER TABLE thold_data REMOVE COLUMN bl_enabled'
10/28/2011 08:47:45 AM - CMDPHP: Poller[0] ERROR: A DB Exec Failed!, Error:'1060', SQL:"ALTER TABLE thold_data ADD COLUMN bl_thold_valid INT UNSIGNED NOT NULL DEFAULT 0'

3 syntax errors on the version of MySQL that came with the Windows installer. . .the first two need to have "DROP COLUMN". . .the third, I'm not sure why it was there already, but I had to do a "MODIFY COLUMN" on it.

I have no errors reported by PHP or Apache. . .

I like the look of the new THold, but I can't get it working too well. Any info that could help someone else figure out what's going on?

[edit]

And this pops up in my Cacti log now:
10/28/2011 09:21:02 AM - SYSTEM THOLD STATS: Time:0.0108 Tholds:0 Hosts:0

[edit2]

Oh, and when I try to edit an existing threshold template and save it, I get an "Error: Save Failed"

[edit3]

Yet another error: SQL Save Command Failed for Table 'thold_template'. Error was 'Unknown column 'expression' in 'field list''

Just trying to show as much erring as possible to help with troubleshooting.

[TheWitness]

What MySQL version?

[ARC1450]

mysql Ver 14.14 Distrib 5.1.50, for Win64 (unknown)

I'm wondering if I should just bomb the thold tables, uninstall the plugin and start over with it. Here's the current contents of my THOLD_DATA and THOLD_TEMPLATES tables:

Code: Select all

mysql> show columns from thold_data;
+----------------------------+------------------+------+-----+------------------+-----------------------------+
| Field                      | Type             | Null | Key | Default | Extra                       |
+----------------------------+------------------+------+-----+------------------+-----------------------------+
| id                         | int(11)          | NO   | PRI | NULL | auto_increment              |
| name                       | varchar(150)     | YES  |     | NULL |                             |
| rra_id                     | int(11)          | NO   | MUL | 0 |                             |
| data_id                    | int(11)          | NO   | MUL | 0 |                             |
| graph_id                   | int(11)          | NO   | MUL | 0 |                             |
| graph_template             | int(11)          | NO   | MUL | 0 |                             |
| data_template              | int(11)          | NO   | MUL | 0 |                             |
| thold_hi                   | varchar(100)     | YES  |     | NULL |                             |
| thold_low                  | varchar(100)     | YES  |     | NULL |                             |
| thold_fail_trigger         | int(10) unsigned | YES  |     | NULL |                             |
| thold_fail_count           | int(11)          | NO   |     | 0 |                             |
| time_hi                    | varchar(100)     | YES  |     | NULL |                             |
| time_low                   | varchar(100)     | YES  |     | NULL |                             |
| time_fail_trigger          | int(12)          | NO   |     | 1 |                             |
| time_fail_length           | int(12)          | NO   |     | 1 |                             |
| thold_alert                | int(1)           | NO   |     | 0 |                             |
| thold_enabled              | enum('on','off') | NO   | MUL | on |                             |
| thold_type                 | int(3)           | NO   |     | 0 |                             |
| bl_ref_time                | int(50) unsigned | YES  |     | NULL |                             |
| bl_ref_time_range          | int(10) unsigned | YES  |     | NULL |                             |
| bl_pct_down                | varchar(100)     | YES  |     | NULL |                             |
| bl_pct_up                  | varchar(100)     | YES  |     | NULL |                             |
| bl_fail_trigger            | int(10) unsigned | YES  |     | NULL |                             |
| bl_fail_count              | int(11) unsigned | YES  |     | NULL |                             |
| bl_alert                   | int(2)           | NO   |     | 0 |                             |
| lastread                   | varchar(100)     | YES  |     | NULL |                             |
| lasttime                   | timestamp        | NO   |     | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
| oldvalue                   | varchar(100)     | YES  |     | NULL |                             |
| repeat_alert               | int(10) unsigned | YES  |     | NULL |                             |
| notify_default             | enum('on','off') | YES  |     | NULL |                             |
| notify_extra               | text             | YES  |     | NULL |                             |
| host_id                    | int(10)          | YES  | MUL | NULL |                             |
| syslog_priority            | int(2)           | NO   |     | 3 |                             |
| data_type                  | int(12)          | NO   |     | 0 |                             |
| cdef                       | int(11)          | NO   |     | 0 |                             |
| percent_ds                 | varchar(64)      | NO   |     | |                             |
| template                   | int(11)          | NO   | MUL | 0 |                             |
| template_enabled           | char(3)          | NO   | MUL | |                             |
| tcheck                     | int(1)           | NO   | MUL | 0 |                             |
| exempt                     | char(3)          | NO   |     | off |                             |
| restored_alert             | char(3)          | NO   |     | off |                             |
| exempt_state               | tinyint(1)       | YES  |     | NULL |                             |
| exempt_begin_time          | int(2)           | YES  |     | NULL |                             |
| exempt_end_time            | int(2)           | YES  |     | NULL |                             |
| bl_thold_valid             | int(10) unsigned | NO   |     | 0 |                             |
| time_warning_fail_length   | int(12)          | NO   |     | 1 |                             |
| thold_warning_hi           | varchar(100)     | YES  |     | NULL |                             |
| thold_warning_low          | varchar(100)     | YES  |     | NULL |                             |
| thold_warning_fail_trigger | int(10) unsigned | YES  |     | NULL |                             |
| thold_warning_fail_count   | int(11)          | NO   |     | 0 |                             |
| time_warning_hi            | varchar(100)     | YES  |     | NULL |                             |
| time_warning_low           | varchar(100)     | YES  |     | NULL |                             |
| time_warning_fail_trigger  | int(12)          | NO   |     | 1 |                             |
| notify_warning_extra       | text             | YES  |     | NULL |                             |
| notify_alert               | int(10) unsigned | NO   |     | 1 |                             |
| notify_warning             | int(10) unsigned | NO   |     | 1 |                             |
+----------------------------+------------------+------+-----+------------------+-----------------------------+
56 rows in set (0.00 sec)

Code: Select all

mysql> show columns from thold_template;
+----------------------------+------------------+------+-----+---------+---------------+
| Field                      | Type             | Null | Key | Default | Extra        |
+----------------------------+------------------+------+-----+---------+---------------+
| id                         | int(11)          | NO   | PRI | NULL    | auto_icrement |
| hash                       | varchar(32)      | YES  |     | NULL    |        |
| name                       | varchar(100)     | NO   |     |         |        |
| data_template_id           | int(10)          | NO   | MUL | 0       |        |
| data_template_name         | varchar(100)     | NO   |     |         |        |
| data_source_id             | int(10)          | NO   | MUL | 0       |        |
| data_source_name           | varchar(100)     | NO   |     |         |        |
| data_source_friendly       | varchar(100)     | NO   |     |         |        |
| thold_hi                   | varchar(100)     | YES  |     | NULL    |        |
| thold_low                  | varchar(100)     | YES  |     | NULL    |        |
| thold_fail_trigger         | int(10) unsigned | YES  |     | NULL    |        |
| time_hi                    | varchar(100)     | YES  |     | NULL    |        |
| time_low                   | varchar(100)     | YES  |     | NULL    |        |
| time_fail_trigger          | int(12)          | NO   |     | 1       |        |
| time_fail_length           | int(12)          | NO   |     | 1       |        |
| thold_enabled              | enum('on','off') | NO   |     | on      |        |
| thold_type                 | int(3)           | NO   |     | 0       |        |
| bl_ref_time                | int(50) unsigned | YES  |     | NULL    |        |
| bl_ref_time_range          | int(10) unsigned | YES  |     | NULL    |        |
| bl_pct_down                | varchar(100)     | YES  |     | NULL    |        |
| bl_pct_up                  | varchar(100)     | YES  |     | NULL    |        |
| bl_fail_trigger            | int(10) unsigned | YES  |     | NULL    |        |
| bl_fail_count              | int(11) unsigned | YES  |     | NULL    |        |
| bl_alert                   | int(2)           | NO   |     | 0       |        |
| repeat_alert               | int(10) unsigned | YES  |     | NULL    |        |
| notify_default             | enum('on','off') | YES  |     | NULL    |        |
| notify_extra               | text             | YES  |     | NULL    |        |
| data_type                  | int(12)          | NO   |     | 0       |        |
| cdef                       | int(11)          | NO   |     | 0       |        |
| percent_ds                 | varchar(64)      | NO   |     |         |        |
| exempt                     | char(3)          | NO   |     | off     |        |
| restored_alert             | char(3)          | NO   |     | off     |        |
| exempt_begin_time          | int(2)           | YES  |     | NULL    |        |
| exempt_end_time            | int(2)           | YES  |     | NULL    |        |
| thold_warning_hi           | varchar(100)     | YES  |     | NULL    |        |
| thold_warning_low          | varchar(100)     | YES  |     | NULL    |        |
| thold_warning_fail_trigger | int(10) unsigned | YES  |     | NULL    |        |
| thold_warning_fail_count   | int(11)          | NO   |     | 0       |        |
| time_warning_hi            | varchar(100)     | YES  |     | NULL    |        |
| time_warning_low           | varchar(100)     | YES  |     | NULL    |        |
| time_warning_fail_trigger  | int(12)          | NO   |     | 1       |        |
| time_warning_fail_length   | int(12)          | NO   |     | 1       |        |
| notify_warning_extra       | text             | YES  |     | NULL    |        |
| notify_alert               | int(10) unsigned | NO   |     | 1       |        |
| notify_warning             | int(10) unsigned | NO   |     | 1       |        |
+----------------------------+------------------+------+-----+---------+---------------+
45 rows in set (0.01 sec)

[TheWitness]

Here is what it 'should' be.

Code: Select all

mysql> show columns from thold_data;
+----------------------------+------------------+------+-----+---------------------+----------------+
| Field                      | Type             | Null | Key | Default             | Extra          |
+----------------------------+------------------+------+-----+---------------------+----------------+
| id                         | int(11)          | NO   | PRI | NULL                | auto_increment |
| name                       | varchar(100)     | YES  |     | NULL                |                |
| rra_id                     | int(11)          | NO   | MUL | 0                   |                |
| data_id                    | int(11)          | NO   | MUL | 0                   |                |
| graph_id                   | int(11)          | NO   | MUL | 0                   |                |
| graph_template             | int(11)          | NO   |     | 0                   |                |
| data_template              | int(11)          | NO   |     | 0                   |                |
| thold_hi                   | varchar(100)     | YES  |     | NULL                |                |
| thold_low                  | varchar(100)     | YES  |     | NULL                |                |
| thold_fail_trigger         | int(10) unsigned | YES  |     | NULL                |                |
| thold_fail_count           | int(11)          | NO   |     | 0                   |                |
| time_hi                    | varchar(100)     | YES  |     | NULL                |                |
| time_low                   | varchar(100)     | YES  |     | NULL                |                |
| time_fail_trigger          | int(12)          | NO   |     | 1                   |                |
| time_fail_length           | int(12)          | NO   |     | 1                   |                |
| thold_warning_hi           | varchar(100)     | YES  |     | NULL                |                |
| thold_warning_low          | varchar(100)     | YES  |     | NULL                |                |
| thold_warning_fail_trigger | int(10) unsigned | YES  |     | NULL                |                |
| thold_warning_fail_count   | int(11)          | NO   |     | 0                   |                |
| time_warning_hi            | varchar(100)     | YES  |     | NULL                |                |
| time_warning_low           | varchar(100)     | YES  |     | NULL                |                |
| time_warning_fail_trigger  | int(12)          | NO   |     | 1                   |                |
| time_warning_fail_length   | int(12)          | NO   |     | 1                   |                |
| thold_alert                | int(1)           | NO   |     | 0                   |                |
| thold_enabled              | enum('on','off') | NO   | MUL | on                  |                |
| thold_type                 | int(3)           | NO   |     | 0                   |                |
| bl_enabled                 | enum('on','off') | NO   |     | off                 |                |
| bl_ref_time_range          | int(10) unsigned | YES  |     | NULL                |                |
| bl_pct_down                | varchar(100)     | YES  |     | NULL                |                |
| bl_pct_up                  | varchar(100)     | YES  |     | NULL                |                |
| bl_fail_trigger            | int(10) unsigned | YES  |     | NULL                |                |
| bl_fail_count              | int(11) unsigned | YES  |     | NULL                |                |
| bl_alert                   | int(2)           | NO   |     | 0                   |                |
| lastread                   | varchar(100)     | YES  |     | NULL                |                |
| lasttime                   | timestamp        | NO   |     | 0000-00-00 00:00:00 |                |
| oldvalue                   | varchar(100)     | YES  |     | NULL                |                |
| repeat_alert               | int(10) unsigned | YES  |     | NULL                |                |
| notify_default             | enum('on','off') | YES  |     | NULL                |                |
| notify_extra               | text             | YES  |     | NULL                |                |
| notify_warning_extra       | varchar(512)     | YES  |     | NULL                |                |
| notify_warning             | int(10)          | NO   |     | 0                   |                |
| notify_alert               | int(10)          | NO   |     | 0                   |                |
| host_id                    | int(10)          | YES  | MUL | NULL                |                |
| syslog_priority            | int(2)           | NO   |     | 3                   |                |
| data_type                  | int(12)          | NO   |     | 0                   |                |
| cdef                       | int(11)          | NO   |     | 0                   |                |
| percent_ds                 | varchar(64)      | NO   |     |                     |                |
| expression                 | varchar(70)      | NO   |     |                     |                |
| template                   | int(11)          | NO   | MUL | 0                   |                |
| template_enabled           | char(3)          | NO   | MUL |                     |                |
| tcheck                     | int(1)           | NO   | MUL | 0                   |                |
| exempt                     | char(3)          | NO   |     | off                 |                |
| restored_alert             | char(3)          | NO   |     | off                 |                |
| bl_thold_valid             | int(10) unsigned | NO   |     | 0                   |                |
+----------------------------+------------------+------+-----+---------------------+----------------+
54 rows in set (0.00 sec)

Code: Select all

mysql> show columns from thold_template;
+----------------------------+------------------+------+-----+---------+----------------+
| Field                      | Type             | Null | Key | Default | Extra          |
+----------------------------+------------------+------+-----+---------+----------------+
| id                         | int(11)          | NO   | PRI | NULL    | auto_increment |
| hash                       | varchar(32)      | YES  |     | NULL    |                |
| name                       | varchar(100)     | NO   |     |         |                |
| data_template_id           | int(10)          | NO   | MUL | 0       |                |
| data_template_name         | varchar(100)     | NO   |     |         |                |
| data_source_id             | int(10)          | NO   | MUL | 0       |                |
| data_source_name           | varchar(100)     | NO   |     |         |                |
| data_source_friendly       | varchar(100)     | NO   |     |         |                |
| thold_hi                   | varchar(100)     | YES  |     | NULL    |                |
| thold_low                  | varchar(100)     | YES  |     | NULL    |                |
| thold_fail_trigger         | int(10) unsigned | YES  |     | NULL    |                |
| time_hi                    | varchar(100)     | YES  |     | NULL    |                |
| time_low                   | varchar(100)     | YES  |     | NULL    |                |
| time_fail_trigger          | int(12)          | NO   |     | 1       |                |
| time_fail_length           | int(12)          | NO   |     | 1       |                |
| thold_warning_hi           | varchar(100)     | YES  |     | NULL    |                |
| thold_warning_low          | varchar(100)     | YES  |     | NULL    |                |
| thold_warning_fail_trigger | int(10) unsigned | YES  |     | NULL    |                |
| thold_warning_fail_count   | int(11)          | NO   |     | 0       |                |
| time_warning_hi            | varchar(100)     | YES  |     | NULL    |                |
| time_warning_low           | varchar(100)     | YES  |     | NULL    |                |
| time_warning_fail_trigger  | int(12)          | NO   |     | 1       |                |
| time_warning_fail_length   | int(12)          | NO   |     | 1       |                |
| thold_enabled              | enum('on','off') | NO   |     | on      |                |
| thold_type                 | int(3)           | NO   |     | 0       |                |
| bl_enabled                 | enum('on','off') | NO   |     | off     |                |
| bl_ref_time_range          | int(10) unsigned | YES  |     | NULL    |                |
| bl_pct_down                | varchar(100)     | YES  |     | NULL    |                |
| bl_pct_up                  | varchar(100)     | YES  |     | NULL    |                |
| bl_fail_trigger            | int(10) unsigned | YES  |     | NULL    |                |
| bl_fail_count              | int(11) unsigned | YES  |     | NULL    |                |
| bl_alert                   | int(2)           | NO   |     | 0       |                |
| repeat_alert               | int(10) unsigned | YES  |     | NULL    |                |
| notify_default             | enum('on','off') | YES  |     | NULL    |                |
| notify_extra               | text             | YES  |     | NULL    |                |
| notify_warning_extra       | varchar(512)     | YES  |     | NULL    |                |
| notify_warning             | int(10)          | NO   |     | 0       |                |
| notify_alert               | int(10)          | NO   |     | 0       |                |
| data_type                  | int(12)          | NO   |     | 0       |                |
| cdef                       | int(11)          | NO   |     | 0       |                |
| percent_ds                 | varchar(64)      | NO   |     |         |                |
| expression                 | varchar(70)      | NO   |     |         |                |
| exempt                     | char(3)          | NO   |     | off     |                |
| restored_alert             | char(3)          | NO   |     | off     |                |
+----------------------------+------------------+------+-----+---------+----------------+
44 rows in set (0.00 sec)

TheWitness

[ARC1450]

Odd. . .I just blew out the thold tables and reactivated the plugins. . .

bl_enabled did not get created with 4.8-3. :-\

[edit]

4.8-3 has that listed as a table to remove in the "includes/database.php" file.

[edit2]

And as a doofy aside, I can't import the XML file of the thresholds that previously existed. All I can do is paste the text.

Not sure if this is something with my setup or not, but figured it was worth noting.

[TheWitness]

bl_enabled should not be created from a new install, and should be removed from all upgrades.

[ARC1450]

Fresh install of THold and I have this popping up when trying to import:

Code: Select all

10/28/2011 10:41:43 AM - CMDPHP: Poller[0] ERROR: SQL Save Command Failed for Table 'thold_template'. Error was 'Unknown column 'bl_ref_time' in 'field list''
10/28/2011 10:41:43 AM - CMDPHP: Poller[0] ERROR: SQL Save Command Failed for Table 'thold_template'. Error was 'Unknown column 'bl_ref_time' in 'field list''
10/28/2011 10:41:43 AM - CMDPHP: Poller[0] ERROR: SQL Save Command Failed for Table 'thold_template'. Error was 'Unknown column 'bl_ref_time' in 'field list''
10/28/2011 10:41:43 AM - CMDPHP: Poller[0] ERROR: SQL Save Command Failed for Table 'thold_template'. Error was 'Unknown column 'bl_ref_time' in 'field list''
10/28/2011 10:41:43 AM - CMDPHP: Poller[0] ERROR: SQL Save Command Failed for Table 'thold_template'. Error was 'Unknown column 'bl_ref_time' in 'field list''
10/28/2011 10:41:43 AM - CMDPHP: Poller[0] ERROR: SQL Save Command Failed for Table 'thold_template'. Error was 'Unknown column 'bl_ref_time' in 'field list''
10/28/2011 10:41:43 AM - CMDPHP: Poller[0] ERROR: SQL Save Command Failed for Table 'thold_template'. Error was 'Unknown column 'bl_ref_time' in 'field list''
10/28/2011 10:41:43 AM - CMDPHP: Poller[0] ERROR: SQL Save Command Failed for Table 'thold_template'. Error was 'Unknown column 'bl_ref_time' in 'field list''
10/28/2011 10:41:43 AM - CMDPHP: Poller[0] ERROR: SQL Save Command Failed for Table 'thold_template'. Error was 'Unknown column 'bl_ref_time' in 'field list''
10/28/2011 10:41:43 AM - CMDPHP: Poller[0] ERROR: SQL Save Command Failed for Table 'thold_template'. Error was 'Unknown column 'bl_ref_time' in 'field list''
10/28/2011 10:41:43 AM - CMDPHP: Poller[0] ERROR: SQL Save Command Failed for Table 'thold_template'. Error was 'Unknown column 'bl_ref_time' in 'field list''
10/28/2011 10:41:43 AM - CMDPHP: Poller[0] ERROR: SQL Save Command Failed for Table 'thold_template'. Error was 'Unknown column 'bl_ref_time' in 'field list''
10/28/2011 10:41:43 AM - CMDPHP: Poller[0] ERROR: SQL Save Command Failed for Table 'thold_template'. Error was 'Unknown column 'bl_ref_time' in 'field list''
10/28/2011 10:41:43 AM - CMDPHP: Poller[0] ERROR: SQL Save Command Failed for Table 'thold_template'. Error was 'Unknown column 'bl_ref_time' in 'field list''
10/28/2011 10:41:43 AM - CMDPHP: Poller[0] ERROR: SQL Save Command Failed for Table 'thold_template'. Error was 'Unknown column 'bl_ref_time' in 'field list''
10/28/2011 10:41:43 AM - CMDPHP: Poller[0] ERROR: SQL Save Command Failed for Table 'thold_template'. Error was 'Unknown column 'bl_ref_time' in 'field list''
10/28/2011 10:41:43 AM - CMDPHP: Poller[0] ERROR: SQL Save Command Failed for Table 'thold_template'. Error was 'Unknown column 'bl_ref_time' in 'field list'' 

After a quick check via 'SHOW COLUMNS FROM thold_template LIKE 'bl_ref_time';, that column really doesn't exist. :-\ I noticed it doesn't exist in your tables.

What did it get migrated to, or did it get removed from the templates completely?

[TheWitness]

You need to hand edit the XML and remove that field as it no longer exists. If you re-export, it will not exist either.