Cacti THOLD alerts to Slack

[jamesb]

I've written a Python script that will turn THOLD email alerts into Slack rich messages with graph.
The script leverages Sendmail aliases and the sendmail restricted shell (smrsh) to pipe the default THOLD email message in.

You will need to edit set some global variables (within the script). Please let me know if you have any questions.

Download here: https://github.com/jamesboswell/thold-slack

Features:
* Sends Cacti THOLD message to a Slack channel of your choice
* Allows hyperlink to URL of your choice (plugins/thold/thold_graph.php)
* Hyperlink to graph of threshold breached
* Optionally includes graph directly in Slack
* RED (Alert), YELLOW (Warning), BLUE (Restored) alerts

Screenshot examples:

[MissouriSpartan]

I know this is a couple years old, but your script works perfectly!!!!

However, I'm trying to figure out how to implement putting graphs directly into the THOLD messages in Slack. I'm a little confused as to what you mean by mapping a directory to Apache.

N00b at this, I guess.

Help?

[Osiris]

Thats pretty cool.

[MoonDogg]

This is great!! anyone know if you can use sendmail and SMTP notification? I am using SMTP right now, but would have to switch to sendmail to implement this.

Update: never mind I got SendMail installed and working with all my notifications with no problem. Now working on getting the graph's in the Slack notifications.

[MoonDogg]

I know this is a couple years old, but your script works perfectly!!!!

However, I'm trying to figure out how to implement putting graphs directly into the THOLD messages in Slack. I'm a little confused as to what you mean by mapping a directory to Apache.

N00b at this, I guess.

Help?

What it is saying is that :

Code: Select all

slack_image_url     =   "http://cacti/tmp/img/"

Needs to go to the same folder as :

Code: Select all

image_path           = '/var/www/html/tmp/img/'

This is my path's and I am using ubuntu server 16.04. I also have "cacti" map to the ip address of my server in DNS.

I still have not got this to work yet. I think the script is looking for an image attachment and the new system puts inline images into the e-mail. I have tried to change it to PNG and JPEG attached images and it still does not work. The images are not getting copied to that directory. I am at a loss at this point. It also does not look like the e-mails are changing image formats when I change the settings, so I am not sure whats going on.

[jamesb]

I know this is a couple years old, but your script works perfectly!!!!

However, I'm trying to figure out how to implement putting graphs directly into the THOLD messages in Slack. I'm a little confused as to what you mean by mapping a directory to Apache.

N00b at this, I guess.

Help?

You need to configure your web server to map your image directory to a reachable URL for to embed graphs in Slack attachments. Slack posts don't actually upload the PNG per THOLD alert. The Slack notification includes a URL to the PNG file, that the Slack image proxy accesses.

Here is an example from my server, where my script outputs PNG files to /data/tmp/img in line 58. You can change line 58 to any directory you like.
https://github.com/jamesboswell/thold-s ... ack.py#L58

Code: Select all

/data/tmp/img/f6bd35eda2f34bb7824c8fbc297c1abe.png

is accessible at

Code: Select all

http://example.com/img/f6bd35eda2f34bb7824c8fbc297c1abe.png

This is done in Apache with a simple Alias and Location directives

Code: Select all

Alias /img /data/tmp/img  # this maps URL to disk location
<Location /img>
    Allow from All
    Satisfy any
</Location>

You should see something like this in your logs

Code: Select all

54.152.5.26 - - [28/Apr/2017:17:21:25 -0400] "GET /ed94329640844ae5a5458ee80c542c4f.png HTTP/1.1" 200 50572 "-" "Slack-ImgProxy (+https://api.slack.com/robots)"

You can make your URL whatever you like, you just need to map the URL to the location of the PNG files on disk.

[jamesb]

This is great!! anyone know if you can use sendmail and SMTP notification? I am using SMTP right now, but would have to switch to sendmail to implement this.

Update: never mind I got SendMail installed and working with all my notifications with no problem. Now working on getting the graph's in the Slack notifications.

Glad you got it working. As you discovered it has to be Sendmail. SMTP is using PHP to directly send the email, where as Sendmail is processing the email (and in this case piping it to my Python script)

[MoonDogg]

This is great!! anyone know if you can use sendmail and SMTP notification? I am using SMTP right now, but would have to switch to sendmail to implement this.

Update: never mind I got SendMail installed and working with all my notifications with no problem. Now working on getting the graph's in the Slack notifications.

Glad you got it working. As you discovered it has to be Sendmail. SMTP is using PHP to directly send the email, where as Sendmail is processing the email (and in this case piping it to my Python script)

I am still having issues with the graph's.. as you may have noticed I submitted an issue on your github page. On my server the images are not even showing up in the directory, so I don't know what the issue is there. And from what I just read you wrote it just links to the graph.. so does that link have to be accessible from the internet or just the network you are looking at slack on? Because my cacti server is internal only.

[jamesb]

I am still having issues with the graph's.. as you may have noticed I submitted an issue on your github page. On my server the images are not even showing up in the directory, so I don't know what the issue is there. And from what I just read you wrote it just links to the graph.. so does that link have to be accessible from the internet or just the network you are looking at slack on? Because my cacti server is internal only.

Ok, firstly, if the images aren't even showing up, check file permissions. The script is run as the mail user on my server for example

Code: Select all

# ps -ef | grep thold-slack
mail     13636 13634  0  2015 ?        00:00:00 /usr/bin/python /etc/smrsh/thold-slack.py

So my image path is writeable by "mail"

Code: Select all

image_path           = '/data/tmp/img/'

I actually have it owned by root, and set to 1777 (sticky bit)

Code: Select all

stat -c "%a %n" /data/tmp/img/
1777 /data/tmp/img/

The image files are 0644 when written by smrsh

Code: Select all

 File: ‘/data/tmp/img/fe217580bd90436b9fdd633352886fdf.png’
  Size: 50355     	Blocks: 112        IO Block: 32768  regular file
Device: 13h/19d	Inode: 166115      Links: 1
Access: (0644/-rw-r--r--)  Uid: (    8/    mail)   Gid: (   12/    mail)

[MissouriSpartan]

Well, I did the following:

Set an Alias as the following:

Code: Select all

Alias /img /data/tmp/img
<Directory /img>
        Allow from all
        Satisfy any
</Directory>

Restarted Apache2 service. Then went to check on my permissions for the /data/tmp/img. This was the result of the check:

Code: Select all

drwxrwxrwx   3 root smmsp  4096 Apr 10 12:55 data

1777 /data/tmp/img/

Then checked on the thold-slack.py owner:

Code: Select all

root      5498 18452  0 09:07 pts/1    00:00:00 grep --color=auto thold-slack

Still no graphs appearing on the alerts. There are URLs showing up, which we can click on, but the /data/tmp/img directory is not being written to with graph PNG images.

[jamesb]

Still no graphs appearing on the alerts. There are URLs showing up, which we can click on, but the /data/tmp/img directory is not being written to with graph PNG images.

Can you navigate to your Settings tab, Thresholds, and scroll down and confirm Send Alerts as Text is NOT checked?

[MissouriSpartan]

Yes. This is unchecked.

[jamesb]

@MissouriSpartan, can you forward one of your THOLD alert emails to yourself as an attachment, then open the attachment in a text editor?

You should see some lines that have "image" content type. I just want to make sure it's in there somewhere, my guess is it's A) not in there or B) it is, but my script is looking in the wrong index of the get_message_payload list returned. It works for me, but perhaps something is different with your generated emails.

Code: Select all

9014_1428530199_2765990@localhost'><br>

----ALT__MAILER_9014_1428530199_2768360--
----REL__MAILER_9014_1428530199_2768360
Content-ID: <9014_1428530199_2765990@localhost>
Content-Type: image/jpg
Content-Transfer-Encoding: base64
Content-Disposition: inline; filename="12753.jpg"

[MissouriSpartan]

I may be in over my head here..............how do you forward as an attachment with Sendmail? I'm sorry, I'm a n00b with Sendmail.

[jamesb]

I may be in over my head here..............how do you forward as an attachment with Sendmail? I'm sorry, I'm a n00b with Sendmail.

Not in Sendmail, are you sending THOLD alerts to your email now? Many programs such as Microsoft Outlook allow you to forward as an attachment.

If you can save an existing THOLD email alert that has the graphs, as an EML mail file, then we can see what's going on from there.