Boost Plugin v1.6 Released

[TheWitness]

All,

I am proud to release the Boost Plugin to the GPL community. It has been in development ever since my visit to Linuxworld earlier this year. I did not start in earnest until about 1.5 months ago. This version has been extensively tested by our development team in addition to some generous testers that I will name later.

Here are the main features:
On-Demand RRD Updating You don't have to update 40k RRD files every 5 minutes, you can wait hours before doing updates. If a user accesses a Graph before a major update, the "On-Demand" updater takes care of it.
Network Service to handle RRD Updating This service is handy when your Web Server does not have RW access to the RRA folder. This provides an added benefit of allowing for multiple Web Servers to be deployed within this framework to host Cacti.
PNG Caching Yes, that's right. This plugin will cache PNG's for as long as they are viable. Once they expire, the first person to view the graph uses the overhead of creating the next good image.

Before you get all excited, let me express a few words of caution. First, this plugin is complicated. Therefore, before you just go ramming into your production environment, set it up in test. Do some reading on "memory" tables in MySQL in order to get the best performance possible, use of them will really reduce you system load. Convert your poller_output tabke to Memory as well.

Once you have it working, then move it to your production system. I have also included a "Utilities" component for this plugin that will allow you to track Boost statistics.

As usual, I have provided little documentation. I hope you will understand. However, the README file is sufficient for now. It does explain many aspects of this plugin.

As, I am a Windows developer (well sort of hate most of Windows anymore). Therefore, all the files in the Zip package have those nasty line break characters in them. I have a committment from Gandolf, to package a *NIX version to assist you in each of your deployments.

This plugin involves a network service and open port unless your web server has RW access to your RRA folder, then you can avoid the service. If you have Windows, this is likely a started task. If someone want's to come up with documentation on how to make it a self restarting/healing Windows service, I would greatly appreciate it.

You will also need a new version of the Plugin Architecture, Cigamit has agreed to release this right away once I release this plugin. So, you will see it soon. There is also a "modified" directory in the plugin. Make sure you copy those files over the existing Cacti installation. See Cigamit's comments later on.

As far as compatibility with other Plugins. The only issue that the development team is absolutely 100% sure is an issue with Weathermap. We have contacted the developer and asked him to see what he can do. We'll see. Howie is as busy as me these days.

Also, this plugin is for 0.8.6i only. No other versions I will even think of supporting.

So, in summary, here are the highlights:
1) New Plugin Architecture (v1.1)
2) Remove those nasty breaks until Gandolf arrives with a gz file or wait for Gandolf
3) 0.8.6i Only (Please)
4) Test First, Then Deploiy
5) Don't forget the service...
5) Post your statistics in the Main Announcements Forum

Regards,

TheWitness
Happy Holidays!!! (If you like, see my Amazon Store, my Kid's would appreciate it)

Major Contributers:
N3NCY - Ernest Wilson (I broke his system a few times)
Cigamit - Jimmy Connor (Always happy to work with this guy)
Gandolf - Reinhard Schek (aka LVM, he broke it and it broke him. Great moderator)
RCaston - Who is now imortalized in the product.
Others - You know who you are. Make sure you post your results.

[gandalf]

Here's the tar.gz for *nix (hope so, at least. I tested on my RHEL 4 system only).
Please follow instructions of README carefully when installing the cacti_rrdsvc. Remember to edit cacti_rrdsvc and define the path to your boost plugin directory or it will not work but simply exit gracefully
Reinhard

[gandalf]

I encountered the following entry in table poller_output_boost_processes

ERROR: illegal attempt to update using time 1165675503 when last update time is 1165675803 (minimum one second step)

And I've found some gaps in my graphs.
Reinhard

[TheWitness]

Interesting. I think I know why this is happening.

What I think may be happening is that due to the changes in RC4, the boost server is getting "true" error messages from RRDtool now, and therefore, it does not discard the contents of the poller_output_boost table under those circumstances, although "some" of the updates took. This will result in gaps.

Please verify the following:
1) Change your boost server to single process,
2) Force an update of the boost server using "php poller_boost.php -f -d".
3) Allow the poll cycle to run for a few ticks, then switch back to multi-process again.

Let me know how it goes. Maybe we should just assume the results are correct, log an error to the Cacti log and discard the update values. What do you think?

For all those watching this post, simply run in Single On Demand Process mode until we have this resolved.

TheWitness

[dagonet]

Hi TheWitness,
I encountered an problem with this plugin. After installing the plugin the normal cacti poller does not process the rrd files anymore resulting in gaps in my graphs.

And btw. the poller_output_boost_processes table is emtpy.

Dagonet.

[gandalf]

Are you running on *nix? Are you running the boost_server? If so, which userid?
Reinhard

[dagonet]

Hi Gandalf,
I do run my cacti stuff on *nix. The boost_server runs as root.
At the meantime I managed to get the graphs going.
I did disable everything on the performance booster tab.
Ran the cacti poller by hand and it did process the rrds.
Then I again enabled everything on the tab and removed the path to the rrdupdate utility.

But anyway I get this message from the poller in the cacti log file:
SYSTEM STATS: Time:9.1476 Method:cactid Processes:5 Threads:5 Hosts:5 HostsPerProcess:1 DataSources:145 RRDsProcessed:0

But the graphs are updated definitly.

And still the poller_output_boost_processes table is empty. Maybe that's why I can't see any boost statistics on the utility page?

Dagonet.

[gandalf]

Hi Gandalf,
I do run my cacti stuff on *nix. The boost_server runs as root.

So be carefully, when switching back. All newly created rrd files will be owned by root in your case. And when switching back after having created a file as root, it will not get updated by cactiuser any more. So, personally, I'd like to use the same user that runs the poller.

At the meantime I managed to get the graphs going.
I did disable everything on the performance booster tab.
Ran the cacti poller by hand and it did process the rrds.
Then I again enabled everything on the tab and removed the path to the rrdupdate utility.

But anyway I get this message from the poller in the cacti log file:
SYSTEM STATS: Time:9.1476 Method:cactid Processes:5 Threads:5 Hosts:5 HostsPerProcess:1 DataSources:145 RRDsProcessed:0

That's fine; the poller does not update rrd files any more. This is done by the boost server in turn. Please search the forums for my newest poller graph templates featuring graphs for boost server statistics.

But the graphs are updated definitly.

And still the poller_output_boost_processes table is empty. Maybe that's why I can't see any boost statistics on the utility page?

Dagonet.

This table should nearly always be empty. It is filled during dynamic update processes and cleared as soon as an update succeeded. So if it is not empty, this would indicate a problem (the table serializes boost update and web graph actions).
Reinhard

[gandalf]

Interesting. I think I know why this is happening.

What I think may be happening is that due to the changes in RC4, the boost server is getting "true" error messages from RRDtool now, and therefore, it does not discard the contents of the poller_output_boost table under those circumstances, although "some" of the updates took. This will result in gaps.

Please verify the following:
1) Change your boost server to single process,
2) Force an update of the boost server using "php poller_boost.php -f -d".
3) Allow the poll cycle to run for a few ticks, then switch back to multi-process again.

Let me know how it goes. Maybe we should just assume the results are correct, log an error to the Cacti log and discard the update values. What do you think?

For all those watching this post, simply run in Single On Demand Process mode until we have this resolved.

TheWitness

I will try running single process. Currently it works flawlessly. No "broken" graphs (without graph caching, currently), no missing data. But this will need further investigation, you know
Reinhard

[dagonet]

Ah Gandalf,
now I got it. Thanks.
Still confusing is the not working "View Boost Status".

Dagonet.

[gandalf]

I suppose that is due to an error with the plugin arch.
Watch out at the very left bottom side of the "Utilities" page
If you want to, I can provide a simple patch
Reinhard

[dagonet]

I should cut and paste my posting:)
After I finished my posting I got this error message:

Error in posting

DEBUG MODE

SQL Error : 1064 You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'WHERE forum_id = 19' at line 3

UPDATE forums SET forum_posts = forum_posts + 1, forum_last_post_id = WHERE forum_id = 19

Line : 423
File : functions_post.php

Very funny...

Hi Gandalf,
a patch would be very handy:)

But anyway:
After getting things work. I got this morning the error message "ERROR: Socket Error. Boost server is down. Contact support immediately!!".
But the boost server was running. A telnet check was positive.

A reboot solved this issue.

Now I have a problem with rrds not being updated by the boost server process. It's not all rrds, only some. Switching back to the old school poller style, the rrds are updated.

Any ideas?

Dagonet.

[TheWitness]

Please provide the DS information to this post. I will likely not have time for it soon. Once inaddition to the DS#'s, please provide a DEBUG run log file so that I can review syntax.

Thanks,

TheWitness

[gandalf]

After getting things work. I got this morning the error message "ERROR: Socket Error. Boost server is down. Contact support immediately!!".

In my very first post, I forgot to chmod a+x boost_server.php. That's required and corrected in the current tar.gz.
As for the patch:
You'll have to change file utilities.php. Around line 128 it should read

Code: Select all

        default:

                include_once("./include/top_header.php");

                do_hook_function("utilities_action", $_REQUEST["action"]);

                utilities();

                include_once("./include/bottom_footer.php");
                break;

and around line 977 it should read

Code: Select all

        <?php

        do_hook("utilities_list");

        html_end_box();
}

?>

. So simple the do_hooks must be shifted some lines. Unfortunately, this gives me an undefined index error in httpd's error_log that I have not yet dealt with. But the utilities page looks nicer
Reinhard

[gandalf]

Due to some problems and a discussion with TheWitness, we strongly urge all *nix users to stick to single boost process configuration for now. It will take at least until January to find time for fixing this.
Reinhard