Submit your BOOST Statistics Here

Important information about Cacti developments that all users should be interested in.

Moderators: Developers, Moderators

User avatar
TheWitness
Developer
Posts: 17047
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

Submit your BOOST Statistics Here

Post by TheWitness »

All,

I have released the Boost Plugin, version 1.0. Once you have something good to say about boost, or bad for that matter, please post your before and after statistics as followups to this post.

Here are the main features:
On-Demand RRD Updating You don't have to update 40k RRD files every 5 minutes, you can wait hours before doing updates. If a user accesses a Graph before a major update, the "On-Demand" updater takes care of it.
Network Service to handle RRD Updating This service is handy when your Web Server does not have RW access to the RRA folder. This provides an added benefit of allowing for multiple Web Servers to be deployed within this framework to host Cacti.
PNG Caching Yes, that's right. This plugin will cache PNG's for as long as they are viable. Once they expire, the first person to view the graph uses the overhead of creating the next good image.

Regards,

TheWitness

http://forums.cacti.net/viewtopic.php?p=86053#86053
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages


For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
DAP
Posts: 43
Joined: Mon Jul 18, 2005 10:42 pm
Location: Melbourne, Australia

Post by DAP »

Seems to be working fine.

12/15/2006 05:00:23 PM - SYSTEM BOOST STATS: Time:4.2656 RRDUpdates:1022

Love the cache feature :-)

Cheers

PS. View Boost Status not working.
User avatar
gandalf
Developer
Posts: 22383
Joined: Thu Dec 02, 2004 2:46 am
Location: Muenster, Germany
Contact:

Post by gandalf »

DAP wrote:PS. View Boost Status not working.
This requires a patch to plugin arch. In the meantime, you should be able to find it on the utilities page on the very left bottom side
Reinhard
wriley
Posts: 16
Joined: Fri May 19, 2006 1:17 pm
Location: Lake St Louis, MO USA
Contact:

Post by wriley »

Boost is my new best friend! :) I had been struggling to tweak cacti to handle a huge amount of data sources. Thanks devs!

This system is monitoring nearly 70 Cisco 65xx routers, with a handful of other devices thrown in. This makes for a LOT of data sources.

SYSTEM: BOOST STATS: Time:985.7940 RRDUpdates:988752

SYSTEM: STATS: Time:59.0062 Method:cactid Processes:1 Threads:32 Hosts:74 HostsPerProcess:74 DataSources:75951 RRDsProcessed:0


Total Data Sources: 75951
Total Boost Records: 758095
Current Boost Table Size: 43 MBytes
Avg Bytes/Record: 45 Bytes
Max Allowed Boost Table Size: 4 GBytes
Estimated Maximum Records: 95443718 Records
Last edited by wriley on Mon Feb 12, 2007 1:38 pm, edited 1 time in total.
William Riley
http://workbench.freetcp.com/
User avatar
gandalf
Developer
Posts: 22383
Joined: Thu Dec 02, 2004 2:46 am
Location: Muenster, Germany
Contact:

Post by gandalf »

You may want to use my new Poller Stats, featuring Boost Statistics (runtime, amount of BOOST) http://forums.cacti.net/viewtopic.php?t=18057
Reinhard
wriley
Posts: 16
Joined: Fri May 19, 2006 1:17 pm
Location: Lake St Louis, MO USA
Contact:

Post by wriley »

gandalf wrote:You may want to use my new Poller Stats, featuring Boost Statistics (runtime, amount of BOOST) http://forums.cacti.net/viewtopic.php?t=18057
Reinhard
I was already using your templates for poller statistics and subsequently turned on the boost stats as well. Thanks for the templates!


These graphs show the improvement when Boost was activated (at about two-thirds point). I also added the final 20 hosts, hence the jump in data sources at the same time.
Attachments
Poller Statistics
Poller Statistics
poller_stats.png (27.45 KiB) Viewed 52951 times
Poller Runtime Statistics
Poller Runtime Statistics
poller_runtime.png (41.59 KiB) Viewed 52951 times
Boost Statistics
Boost Statistics
boost_statistics.png (20.13 KiB) Viewed 52951 times
William Riley
http://workbench.freetcp.com/
User avatar
gandalf
Developer
Posts: 22383
Joined: Thu Dec 02, 2004 2:46 am
Location: Muenster, Germany
Contact:

Post by gandalf »

Do not worry, that the green line in the last image stops. This is now reported in the boost stats
Reinhard
User avatar
TheWitness
Developer
Posts: 17047
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

Post by TheWitness »

That is one BIG system. Glad to see it come back down to earth. I am sure the web site behaves better now too. I hope you enjoy this plugin I wrote for you. Not a lot of host's, but Lot's of Data Sources...

TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages


For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
kanada
Cacti User
Posts: 137
Joined: Sun Aug 28, 2005 12:51 pm

Post by kanada »

Depends my configuration i test cactid polling vs. booster.
On both settings, timers are same, about 366 sec. (~6 min.)

Polling interval 10 min.

Below are some from my system graphs (cpu, load average).

Current Booster settings:

Current Boost Status
Boost On Demand Updating: Idle
Total Data Sources: 38379
Total Boost Records: 22966

Table Size Estimates
Current Boost Table Size: 16 MBytes
Avg Bytes/Record: 127 Bytes
Max Allowed Boost Table Size: 4 GBytes
Estimated Maximum Records: 33818640 Records

Previous Runtime
Last Start Time: 2007-03-13 8:36:07
Last Run Duration: 319.6620 Seconds
RRD's Updated: 28646

Run Time Configuration
Update Frequency: 20 Minutes
Next Start Time: 2007-03-13 8:56:07
Maximum Records: 1000000 Records
Maximum Allowed Runtime: 20 Minutes

Boost Server Details
Server Config Status: Enabled
Multiprocess Server: Single Process
Update Timeout: 2 Seconds
Server/Port: localhost@9050
Authorized Update Web Servers: 127.0.0.1
RRDtool Binary Used: /usr/bin/rrdtool

Seems, that Booster eat more system than cactid poller.
Its normal?

Best Regards,
Alex.
Attachments
Daily, before a gap works on booster.
Daily, before a gap works on booster.
cpu01.PNG (26.43 KiB) Viewed 52506 times
Weekly.
Weekly.
cpu02.PNG (41.03 KiB) Viewed 52506 times
Daily, before 12h works on booster.
Daily, before 12h works on booster.
load01.PNG (34.35 KiB) Viewed 52506 times
Monthly, from Week 10 system works on booster.
Monthly, from Week 10 system works on booster.
load03.PNG (46.5 KiB) Viewed 52506 times
User avatar
gandalf
Developer
Posts: 22383
Joined: Thu Dec 02, 2004 2:46 am
Location: Muenster, Germany
Contact:

Post by gandalf »

From my point of view, boost is not a replacement for cactid. IMHO, the sequence would be

1) Use cmd.php as a start
If your systems grows, you may hit the wall unless ...
2) Use cactid as a fast (by order of a magnitude) replacement next
If your system still grows (we are speaking of at least 20,000 to 30,000 to 40,000 and more data sources) this will not be fast enough. rrdtool update consumes most of the time. But rrdtool features sth. like "multi-row insert". Boost will use this. If you hit the wall again ...
3) use cactid in combination with boost
This is now the ultimate step.
Reinhard
User avatar
TheWitness
Developer
Posts: 17047
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

Post by TheWitness »

Concurring with Gandolf.

TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages


For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
kanada
Cacti User
Posts: 137
Joined: Sun Aug 28, 2005 12:51 pm

Post by kanada »

Yes Gandalf you have right, as you see now i have about 40k Data Sources on 10k Hosts, and this number will grows..
Up i post my stats cacti+cactid working on normal polling vs. cacti+cactid+boost..

Graphs shows that in the same poll time (normal and booster check hosts in ~6 min) system utilization (CPU, Memory, HDD) is higher when work boost. I still search a 'gold center' to minimize a polling time (to 5 min.) && system use.

I will post there my test results, if you don't have nothing against that.
Thank you for advice..

Best Regards,
Alex.
User avatar
TheWitness
Developer
Posts: 17047
Joined: Tue May 14, 2002 5:08 pm
Location: MI, USA
Contact:

Post by TheWitness »

Alex,

I would move your update frequency to more like every 1-3 hours.

TheWitness
True understanding begins only when we realize how little we truly understand...

Life is an adventure, let yours begin with Cacti!

Author of dozens of Cacti plugins and customization's. Advocate of LAMP, MariaDB, IBM Spectrum LSF and the world of batch. Creator of IBM Spectrum RTM, author of quite a bit of unpublished work and most of Cacti's bugs.
_________________
Official Cacti Documentation
GitHub Repository with Supported Plugins
Percona Device Packages (no support)
Interesting Device Packages


For those wondering, I'm still here, but lost in the shadows. Yearning for less bugs. Who want's a Cacti 1.3/2.0? Streams anyone?
dawald
Posts: 22
Joined: Wed Feb 02, 2005 5:26 pm
Location: Zurich, Switzerland

Post by dawald »

This plugin is just amazing, finally I'm able to write all my data into the rra's without getting gaps in the graphs.

cacti: 0.8.6j
cactid: 0.8.6i

BOOST Stats:
***********

Code: Select all

04/18/2007 06:49:12 PM - SYSTEM BOOST STATS: Time:380.0422 RRDUpdates:548371
04/18/2007 07:49:09 PM - SYSTEM BOOST STATS: Time:377.6698 RRDUpdates:543419
POLLER Stats:
***********

Code: Select all

04/18/2007 08:32:53 PM - SYSTEM STATS: Time:51.5142 Method:cactid Processes:1 Threads:15 Hosts:251 HostsPerProcess:251 DataSources:50366 RRDsProcessed:0
04/18/2007 08:37:54 PM - SYSTEM STATS: Time:52.6713 Method:cactid Processes:1 Threads:15 Hosts:251 HostsPerProcess:251 DataSources:50366 RRDsProcessed:0
04/18/2007 08:42:51 PM - SYSTEM STATS: Time:49.4776 Method:cactid Processes:1 Threads:15 Hosts:251 HostsPerProcess:251 DataSources:50366 RRDsProcessed:0
Thanks a lot 'TheWitness', this definitly r0cks!!

cheers
:lol:
dawald
Posts: 22
Joined: Wed Feb 02, 2005 5:26 pm
Location: Zurich, Switzerland

Post by dawald »

I would like to show you the impact of boost plugin on my disk i/o waittime. I also attachted a graph that show my loadavg. The first part of the graph shows statistics with the boost plugin. As soon as I switched back to the regular cactid poller method without the boost plugin, the red area (i/o wait) shows up...

cheers :roll:
Attachments
iowait (red)
iowait (red)
cacti_boost_plugin_iowait.png (38.61 KiB) Viewed 51711 times
another shot of the loadaverage
another shot of the loadaverage
cacti_boost_plugin_loadavg.png (58.67 KiB) Viewed 51711 times
Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest