Boost Plugin v1.6 Released
Moderators: Developers, Moderators
- TheWitness
- Developer
- Posts: 17047
- Joined: Tue May 14, 2002 5:08 pm
- Location: MI, USA
- Contact:
Boost Plugin v1.6 Released
All,
I am proud to release the Boost Plugin to the GPL community. It has been in development ever since my visit to Linuxworld earlier this year. I did not start in earnest until about 1.5 months ago. This version has been extensively tested by our development team in addition to some generous testers that I will name later.
Here are the main features:
On-Demand RRD Updating You don't have to update 40k RRD files every 5 minutes, you can wait hours before doing updates. If a user accesses a Graph before a major update, the "On-Demand" updater takes care of it.
Network Service to handle RRD Updating This service is handy when your Web Server does not have RW access to the RRA folder. This provides an added benefit of allowing for multiple Web Servers to be deployed within this framework to host Cacti.
PNG Caching Yes, that's right. This plugin will cache PNG's for as long as they are viable. Once they expire, the first person to view the graph uses the overhead of creating the next good image.
Before you get all excited, let me express a few words of caution. First, this plugin is complicated. Therefore, before you just go ramming into your production environment, set it up in test. Do some reading on "memory" tables in MySQL in order to get the best performance possible, use of them will really reduce you system load. Convert your poller_output tabke to Memory as well.
Once you have it working, then move it to your production system. I have also included a "Utilities" component for this plugin that will allow you to track Boost statistics.
As usual, I have provided little documentation. I hope you will understand. However, the README file is sufficient for now. It does explain many aspects of this plugin.
As, I am a Windows developer (well sort of hate most of Windows anymore). Therefore, all the files in the Zip package have those nasty line break characters in them. I have a committment from Gandolf, to package a *NIX version to assist you in each of your deployments.
This plugin involves a network service and open port unless your web server has RW access to your RRA folder, then you can avoid the service. If you have Windows, this is likely a started task. If someone want's to come up with documentation on how to make it a self restarting/healing Windows service, I would greatly appreciate it.
You will also need a new version of the Plugin Architecture, Cigamit has agreed to release this right away once I release this plugin. So, you will see it soon. There is also a "modified" directory in the plugin. Make sure you copy those files over the existing Cacti installation. See Cigamit's comments later on.
As far as compatibility with other Plugins. The only issue that the development team is absolutely 100% sure is an issue with Weathermap. We have contacted the developer and asked him to see what he can do. We'll see. Howie is as busy as me these days.
Also, this plugin is for 0.8.6i only. No other versions I will even think of supporting.
So, in summary, here are the highlights:
1) New Plugin Architecture (v1.1)
2) Remove those nasty breaks until Gandolf arrives with a gz file or wait for Gandolf
3) 0.8.6i Only (Please)
4) Test First, Then Deploiy
5) Don't forget the service...
5) Post your statistics in the Main Announcements Forum
Regards,
TheWitness
Happy Holidays!!! (If you like, see my Amazon Store, my Kid's would appreciate it)
Major Contributers:
N3NCY - Ernest Wilson (I broke his system a few times)
Cigamit - Jimmy Connor (Always happy to work with this guy)
Gandolf - Reinhard Schek (aka LVM, he broke it and it broke him. Great moderator)
RCaston - Who is now imortalized in the product.
Others - You know who you are. Make sure you post your results.
I am proud to release the Boost Plugin to the GPL community. It has been in development ever since my visit to Linuxworld earlier this year. I did not start in earnest until about 1.5 months ago. This version has been extensively tested by our development team in addition to some generous testers that I will name later.
Here are the main features:
On-Demand RRD Updating You don't have to update 40k RRD files every 5 minutes, you can wait hours before doing updates. If a user accesses a Graph before a major update, the "On-Demand" updater takes care of it.
Network Service to handle RRD Updating This service is handy when your Web Server does not have RW access to the RRA folder. This provides an added benefit of allowing for multiple Web Servers to be deployed within this framework to host Cacti.
PNG Caching Yes, that's right. This plugin will cache PNG's for as long as they are viable. Once they expire, the first person to view the graph uses the overhead of creating the next good image.
Before you get all excited, let me express a few words of caution. First, this plugin is complicated. Therefore, before you just go ramming into your production environment, set it up in test. Do some reading on "memory" tables in MySQL in order to get the best performance possible, use of them will really reduce you system load. Convert your poller_output tabke to Memory as well.
Once you have it working, then move it to your production system. I have also included a "Utilities" component for this plugin that will allow you to track Boost statistics.
As usual, I have provided little documentation. I hope you will understand. However, the README file is sufficient for now. It does explain many aspects of this plugin.
As, I am a Windows developer (well sort of hate most of Windows anymore). Therefore, all the files in the Zip package have those nasty line break characters in them. I have a committment from Gandolf, to package a *NIX version to assist you in each of your deployments.
This plugin involves a network service and open port unless your web server has RW access to your RRA folder, then you can avoid the service. If you have Windows, this is likely a started task. If someone want's to come up with documentation on how to make it a self restarting/healing Windows service, I would greatly appreciate it.
You will also need a new version of the Plugin Architecture, Cigamit has agreed to release this right away once I release this plugin. So, you will see it soon. There is also a "modified" directory in the plugin. Make sure you copy those files over the existing Cacti installation. See Cigamit's comments later on.
As far as compatibility with other Plugins. The only issue that the development team is absolutely 100% sure is an issue with Weathermap. We have contacted the developer and asked him to see what he can do. We'll see. Howie is as busy as me these days.
Also, this plugin is for 0.8.6i only. No other versions I will even think of supporting.
So, in summary, here are the highlights:
1) New Plugin Architecture (v1.1)
2) Remove those nasty breaks until Gandolf arrives with a gz file or wait for Gandolf
3) 0.8.6i Only (Please)
4) Test First, Then Deploiy
5) Don't forget the service...
5) Post your statistics in the Main Announcements Forum
Regards,
TheWitness
Happy Holidays!!! (If you like, see my Amazon Store, my Kid's would appreciate it)
Major Contributers:
N3NCY - Ernest Wilson (I broke his system a few times)
Cigamit - Jimmy Connor (Always happy to work with this guy)
Gandolf - Reinhard Schek (aka LVM, he broke it and it broke him. Great moderator)
RCaston - Who is now imortalized in the product.
Others - You know who you are. Make sure you post your results.
- Attachments
-
- boost-v1.6.zip
- Boost Plugin v1.6
- (34.61 KiB) Downloaded 1710 times
-
- boost-v1.5.zip
- Boost Plugin v1.5
- (33.95 KiB) Downloaded 756 times
-
- boost-v1.4.zip
- Boost Plugin v1.4
- (33.7 KiB) Downloaded 687 times
-
- boost-v1.3.zip
- Boost Plugin v1.3
- (33.62 KiB) Downloaded 628 times
-
- boost-v1.2.zip
- Boost Plugin v1.2
- (29.74 KiB) Downloaded 1000 times
-
- boost-v1.1.zip
- Boost Plugin v1.1
- (30.36 KiB) Downloaded 890 times
-
- boost-1.0.zip
- Boost Plugin v1.0
- (75.31 KiB) Downloaded 842 times
Last edited by TheWitness on Mon Apr 14, 2008 6:05 pm, edited 16 times in total.
True understanding begins only when we realize how little we truly understand...
Life is an adventure, let yours begin with Cacti!
Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages
For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Life is an adventure, let yours begin with Cacti!
Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages
For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
- gandalf
- Developer
- Posts: 22383
- Joined: Thu Dec 02, 2004 2:46 am
- Location: Muenster, Germany
- Contact:
Here's the tar.gz for *nix (hope so, at least. I tested on my RHEL 4 system only).
Please follow instructions of README carefully when installing the cacti_rrdsvc. Remember to edit cacti_rrdsvc and define the path to your boost plugin directory or it will not work but simply exit gracefully
Reinhard
Please follow instructions of README carefully when installing the cacti_rrdsvc. Remember to edit cacti_rrdsvc and define the path to your boost plugin directory or it will not work but simply exit gracefully
Reinhard
- Attachments
-
- boost-1.0.tar.gz
- (65.96 KiB) Downloaded 1109 times
Last edited by gandalf on Sat Dec 09, 2006 11:36 am, edited 1 time in total.
- TheWitness
- Developer
- Posts: 17047
- Joined: Tue May 14, 2002 5:08 pm
- Location: MI, USA
- Contact:
Interesting. I think I know why this is happening.
What I think may be happening is that due to the changes in RC4, the boost server is getting "true" error messages from RRDtool now, and therefore, it does not discard the contents of the poller_output_boost table under those circumstances, although "some" of the updates took. This will result in gaps.
Please verify the following:
1) Change your boost server to single process,
2) Force an update of the boost server using "php poller_boost.php -f -d".
3) Allow the poll cycle to run for a few ticks, then switch back to multi-process again.
Let me know how it goes. Maybe we should just assume the results are correct, log an error to the Cacti log and discard the update values. What do you think?
For all those watching this post, simply run in Single On Demand Process mode until we have this resolved.
TheWitness
What I think may be happening is that due to the changes in RC4, the boost server is getting "true" error messages from RRDtool now, and therefore, it does not discard the contents of the poller_output_boost table under those circumstances, although "some" of the updates took. This will result in gaps.
Please verify the following:
1) Change your boost server to single process,
2) Force an update of the boost server using "php poller_boost.php -f -d".
3) Allow the poll cycle to run for a few ticks, then switch back to multi-process again.
Let me know how it goes. Maybe we should just assume the results are correct, log an error to the Cacti log and discard the update values. What do you think?
For all those watching this post, simply run in Single On Demand Process mode until we have this resolved.
TheWitness
True understanding begins only when we realize how little we truly understand...
Life is an adventure, let yours begin with Cacti!
Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages
For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Life is an adventure, let yours begin with Cacti!
Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages
For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Hi Gandalf,
I do run my cacti stuff on *nix. The boost_server runs as root.
At the meantime I managed to get the graphs going.
I did disable everything on the performance booster tab.
Ran the cacti poller by hand and it did process the rrds.
Then I again enabled everything on the tab and removed the path to the rrdupdate utility.
But anyway I get this message from the poller in the cacti log file:
SYSTEM STATS: Time:9.1476 Method:cactid Processes:5 Threads:5 Hosts:5 HostsPerProcess:1 DataSources:145 RRDsProcessed:0
But the graphs are updated definitly.
And still the poller_output_boost_processes table is empty. Maybe that's why I can't see any boost statistics on the utility page?
Dagonet.
I do run my cacti stuff on *nix. The boost_server runs as root.
At the meantime I managed to get the graphs going.
I did disable everything on the performance booster tab.
Ran the cacti poller by hand and it did process the rrds.
Then I again enabled everything on the tab and removed the path to the rrdupdate utility.
But anyway I get this message from the poller in the cacti log file:
SYSTEM STATS: Time:9.1476 Method:cactid Processes:5 Threads:5 Hosts:5 HostsPerProcess:1 DataSources:145 RRDsProcessed:0
But the graphs are updated definitly.
And still the poller_output_boost_processes table is empty. Maybe that's why I can't see any boost statistics on the utility page?
Dagonet.
- gandalf
- Developer
- Posts: 22383
- Joined: Thu Dec 02, 2004 2:46 am
- Location: Muenster, Germany
- Contact:
So be carefully, when switching back. All newly created rrd files will be owned by root in your case. And when switching back after having created a file as root, it will not get updated by cactiuser any more. So, personally, I'd like to use the same user that runs the poller.dagonet wrote:Hi Gandalf,
I do run my cacti stuff on *nix. The boost_server runs as root.
That's fine; the poller does not update rrd files any more. This is done by the boost server in turn. Please search the forums for my newest poller graph templates featuring graphs for boost server statistics.At the meantime I managed to get the graphs going.
I did disable everything on the performance booster tab.
Ran the cacti poller by hand and it did process the rrds.
Then I again enabled everything on the tab and removed the path to the rrdupdate utility.
But anyway I get this message from the poller in the cacti log file:
SYSTEM STATS: Time:9.1476 Method:cactid Processes:5 Threads:5 Hosts:5 HostsPerProcess:1 DataSources:145 RRDsProcessed:0
This table should nearly always be empty. It is filled during dynamic update processes and cleared as soon as an update succeeded. So if it is not empty, this would indicate a problem (the table serializes boost update and web graph actions).But the graphs are updated definitly.
And still the poller_output_boost_processes table is empty. Maybe that's why I can't see any boost statistics on the utility page?
Dagonet.
Reinhard
- gandalf
- Developer
- Posts: 22383
- Joined: Thu Dec 02, 2004 2:46 am
- Location: Muenster, Germany
- Contact:
I will try running single process. Currently it works flawlessly. No "broken" graphs (without graph caching, currently), no missing data. But this will need further investigation, you knowTheWitness wrote:Interesting. I think I know why this is happening.
What I think may be happening is that due to the changes in RC4, the boost server is getting "true" error messages from RRDtool now, and therefore, it does not discard the contents of the poller_output_boost table under those circumstances, although "some" of the updates took. This will result in gaps.
Please verify the following:
1) Change your boost server to single process,
2) Force an update of the boost server using "php poller_boost.php -f -d".
3) Allow the poll cycle to run for a few ticks, then switch back to multi-process again.
Let me know how it goes. Maybe we should just assume the results are correct, log an error to the Cacti log and discard the update values. What do you think?
For all those watching this post, simply run in Single On Demand Process mode until we have this resolved.
TheWitness
Reinhard
I should cut and paste my posting:)
After I finished my posting I got this error message:
Error in posting
DEBUG MODE
SQL Error : 1064 You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'WHERE forum_id = 19' at line 3
UPDATE forums SET forum_posts = forum_posts + 1, forum_last_post_id = WHERE forum_id = 19
Line : 423
File : functions_post.php
Very funny...
Hi Gandalf,
a patch would be very handy:)
But anyway:
After getting things work. I got this morning the error message "ERROR: Socket Error. Boost server is down. Contact support immediately!!".
But the boost server was running. A telnet check was positive.
A reboot solved this issue.
Now I have a problem with rrds not being updated by the boost server process. It's not all rrds, only some. Switching back to the old school poller style, the rrds are updated.
Any ideas?
Dagonet.
After I finished my posting I got this error message:
Error in posting
DEBUG MODE
SQL Error : 1064 You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'WHERE forum_id = 19' at line 3
UPDATE forums SET forum_posts = forum_posts + 1, forum_last_post_id = WHERE forum_id = 19
Line : 423
File : functions_post.php
Very funny...
Hi Gandalf,
a patch would be very handy:)
But anyway:
After getting things work. I got this morning the error message "ERROR: Socket Error. Boost server is down. Contact support immediately!!".
But the boost server was running. A telnet check was positive.
A reboot solved this issue.
Now I have a problem with rrds not being updated by the boost server process. It's not all rrds, only some. Switching back to the old school poller style, the rrds are updated.
Any ideas?
Dagonet.
- TheWitness
- Developer
- Posts: 17047
- Joined: Tue May 14, 2002 5:08 pm
- Location: MI, USA
- Contact:
Please provide the DS information to this post. I will likely not have time for it soon. Once inaddition to the DS#'s, please provide a DEBUG run log file so that I can review syntax.
Thanks,
TheWitness
Thanks,
TheWitness
True understanding begins only when we realize how little we truly understand...
Life is an adventure, let yours begin with Cacti!
Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages
For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
Life is an adventure, let yours begin with Cacti!
Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages
For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
- gandalf
- Developer
- Posts: 22383
- Joined: Thu Dec 02, 2004 2:46 am
- Location: Muenster, Germany
- Contact:
In my very first post, I forgot to chmod a+x boost_server.php. That's required and corrected in the current tar.gz.dagonet wrote: After getting things work. I got this morning the error message "ERROR: Socket Error. Boost server is down. Contact support immediately!!".
As for the patch:
You'll have to change file utilities.php. Around line 128 it should read
Code: Select all
default:
include_once("./include/top_header.php");
do_hook_function("utilities_action", $_REQUEST["action"]);
utilities();
include_once("./include/bottom_footer.php");
break;
Code: Select all
<?php
do_hook("utilities_list");
html_end_box();
}
?>
Reinhard
Who is online
Users browsing this forum: No registered users and 4 guests