Boost Plugin v1.6 Released

[TheWitness]

I have an error

this is when i run poller.php: 01/14/2008 03:49:21 PM - POLLER: Poller[0] ERROR: The path: is invalid. Can not continue

I'm using cacti 0.8.7a pia 1.4, any ideea ?

Hate those "informative" errors. If using Linux, do the following:

cd <path_to_cacti>
grep -r -I -e "Can not continue" *.php

Post the output. That will help.

TheWitness

[ptaylor874]

Hi

I've followed the instructions for running the boost server as a service on RHEL4. It appears to add the service ok because if I run chkconfig --list cacti_rrdsvc it returns as being on in runlevels 2 to 5. However when I try to start the service with service cacti_rrdsvc start command I get the following back

env: /etc/init.d/cacti_rrdsvc: No such file or directory.

Any ideas??

I was having this same problem today when I updated to Boost 1.7. (This answer might be elsewhere in this thread, but I couldn't find it.) This appears to be a problem with embedded characters (classic Windows/Unix issue)

Run this:

cd /etc/init.d/
mv cacti_rrdsvc cacti_rrdsvc.bak; tr -d '\r' < cacti_rrdsvc.bak > cacti_rrdsvc

Afterward, running /etc/init.d/cacti_rrdsvc start works.

[timi]

I have an error

this is when i run poller.php: 01/14/2008 03:49:21 PM - POLLER: Poller[0] ERROR: The path: is invalid. Can not continue

I'm using cacti 0.8.7a pia 1.4, any ideea ?

Hate those "informative" errors. If using Linux, do the following:

cd <path_to_cacti>
grep -r -I -e "Can not continue" *.php

Post the output. That will help.

TheWitness

figure it out, path_cactid should be path_spine ...

[ptaylor874]

Yesterday, I downloaded the trunk for Boost and installed it. It took care of the issues that saw with 1.7 constantly looping over the same data (the boost update process would be killed after running for 10 minutes, then state that it had done over 200,000 RRD updates)...

After letting it run for some time, it seemed to work very consistently with one exception. We have about 150 routers in Cacti, and after looking through all of them, it appeared that about 3 of them were missing regular updates to their RRDs. In the case of the most obvious one, it repeatedly had two points plotted, then three that were blank, then two more plotted, etc.

After seeing how consistently this failed for these few sites, I disabled boost... (I don't think our installation is currently large enough to need it, in truth) Just wanted to report this issue.

[TheWitness]

Yesterday, I downloaded the trunk for Boost and installed it. It took care of the issues that saw with 1.7 constantly looping over the same data (the boost update process would be killed after running for 10 minutes, then state that it had done over 200,000 RRD updates)...

After letting it run for some time, it seemed to work very consistently with one exception. We have about 150 routers in Cacti, and after looking through all of them, it appeared that about 3 of them were missing regular updates to their RRDs. In the case of the most obvious one, it repeatedly had two points plotted, then three that were blank, then two more plotted, etc.

After seeing how consistently this failed for these few sites, I disabled boost... (I don't think our installation is currently large enough to need it, in truth) Just wanted to report this issue.

This is quite odd. How many data sources do you have and what version of MySQL?

TheWitness

[mia1dolfan]

Yesterday, I downloaded the trunk for Boost and installed it....

Where is the trunk / SVN / developmental repository for boost?

[TheWitness]

svn://anonymous@cvs.cacti.net/var/svnroot/cacti_plugins

or

http://cvs.cacti.net/var/svnroot/cacti_plugins

Not sure which works.

TheWitness

[ptaylor874]

This is quite odd. How many data sources do you have and what version of MySQL?

TheWitness

We have increased the number of data sources probably by a factor of two since I posted this. We are currently at 1629 data sources and 164 hosts. All of these are polled in about 10 seconds..

We are using 1 spine process and about 30 threads. We are still trying to figure out the best settings for this old hardware. It's running on a dual processor 1 Ghz Xeon machine. We are using the version of MySQL that came with CactiEZ .3. The MySql Client API library version is 5.0.48.

[mia1dolfan]

Yesterday, I downloaded the trunk for Boost and installed it. It took care of the issues that saw with 1.7 constantly looping over the same data (the boost update process would be killed after running for 10 minutes, then state that it had done over 200,000 RRD updates)...

I had the same issue, I also downloaded the trunk and it resolved that issue.

Thanks for the link Larry

[mia1dolfan]

Code: Select all

01/20/2008 06:39:42 PM - SYSTEM STATS: Time:879.9037 Method:spine Processes:1 Threads:400 Hosts:4767 HostsPerProcess:4767 DataSources:39894 RRDsProcessed:8888
01/20/2008 06:39:52 PM - SYSTEM STATS: Time:587.4415 Method:spine Processes:1 Threads:400 Hosts:4767 HostsPerProcess:4767 DataSources:39894 RRDsProcessed:8986
01/20/2008 06:40:08 PM - SYSTEM STATS: Time:307.0815 Method:spine Processes:1 Threads:400 Hosts:4767 HostsPerProcess:4767 DataSources:39894 RRDsProcessed:6539
01/20/2008 06:45:30 PM - SYSTEM STATS: Time:328.1413 Method:spine Processes:1 Threads:400 Hosts:4767 HostsPerProcess:4767 DataSources:39894 RRDsProcessed:8625
01/20/2008 07:03:59 PM - SYSTEM STATS: Time:757.1612 Method:spine Processes:1 Threads:400 Hosts:4767 HostsPerProcess:4767 DataSources:39894 RRDsProcessed:25883
01/20/2008 07:22:17 PM - SYSTEM STATS: Time:383.1504 Method:spine Processes:1 Threads:400 Hosts:4767 HostsPerProcess:4767 DataSources:39894 RRDsProcessed:6559
01/20/2008 07:30:56 PM - SYSTEM STATS: Time:364.0999 Method:spine Processes:1 Threads:400 Hosts:4767 HostsPerProcess:4767 DataSources:39894 RRDsProcessed:13541
01/20/2008 07:36:38 PM - SYSTEM STATS: Time:217.2529 Method:spine Processes:1 Threads:400 Hosts:4767 HostsPerProcess:4767 DataSources:39894 RRDsProcessed:0
01/20/2008 07:44:17 PM - SYSTEM STATS: Time:250.6230 Method:spine Processes:1 Threads:400 Hosts:4767 HostsPerProcess:4767 DataSources:39894 RRDsProcessed:0
01/20/2008 07:50:31 PM - SYSTEM STATS: Time:218.4584 Method:spine Processes:1 Threads:200 Hosts:4767 HostsPerProcess:4767 DataSources:39894 RRDsProcessed:0
01/20/2008 08:06:05 PM - SYSTEM STATS: Time:216.0681 Method:spine Processes:1 Threads:200 Hosts:4767 HostsPerProcess:4767 DataSources:39894 RRDsProcessed:0
01/20/2008 08:12:42 PM - SYSTEM STATS: Time:213.7791 Method:spine Processes:1 Threads:100 Hosts:4767 HostsPerProcess:4767 DataSources:39894 RRDsProcessed:0
01/20/2008 08:18:34 PM - SYSTEM STATS: Time:212.6731 Method:spine Processes:1 Threads:100 Hosts:4767 HostsPerProcess:4767 DataSources:39894 RRDsProcessed:0
01/20/2008 08:23:44 PM - SYSTEM STATS: Time:222.3216 Method:spine Processes:1 Threads:100 Hosts:4767 HostsPerProcess:4767 DataSources:39894 RRDsProcessed:0

Some recent boost stats

Code: Select all

01/21/2008 10:04:48 PM - SYSTEM BOOST STATS: Time:226.7056 RRDUpdates:228736
01/21/2008 11:04:10 PM - SYSTEM BOOST STATS: Time:188.6107 RRDUpdates:234837
01/22/2008 12:06:01 AM - SYSTEM BOOST STATS: Time:300.5592 RRDUpdates:205450
01/22/2008 01:04:13 AM - SYSTEM BOOST STATS: Time:192.0678 RRDUpdates:234973
01/22/2008 02:06:57 AM - SYSTEM BOOST STATS: Time:356.5297 RRDUpdates:234887
01/22/2008 03:05:25 AM - SYSTEM BOOST STATS: Time:264.0324 RRDUpdates:234777
01/22/2008 04:09:24 AM - SYSTEM BOOST STATS: Time:498.8789 RRDUpdates:236797
01/22/2008 05:09:07 AM - SYSTEM BOOST STATS: Time:486.0653 RRDUpdates:192585
01/22/2008 06:07:17 AM - SYSTEM BOOST STATS: Time:375.2685 RRDUpdates:234750
01/22/2008 07:04:27 AM - SYSTEM BOOST STATS: Time:203.9991 RRDUpdates:231865
01/22/2008 08:04:20 AM - SYSTEM BOOST STATS: Time:197.0967 RRDUpdates:232986
01/22/2008 09:03:40 AM - SYSTEM BOOST STATS: Time:155.3274 RRDUpdates:232611
01/22/2008 10:04:27 AM - SYSTEM BOOST STATS: Time:201.5601 RRDUpdates:238418

[frdrk]

Hi

I tried boost for the first time yesterday, installing it in a testing environment.. worked quite well. Installed it in a bigger environment today, and I'm getting spammed with this in the log;

Code: Select all

01/30/2008 08:55:43 AM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "INSERT INTO poller_output_boost (local_data_id, rrd_name, time, output) VALUES ('22181','traffic_out','2008-01-30 08:55:41','64794230048'), ('22180','traffic_out','2008-01-30 08:55:41','16414404547'), ('22179','traffic_out','2008-01-30 08:55:41','3211724170395'), ('22202','traffic_out','2008-01-30 08:55:41','820331449'), ('22201','traffic_out','2008-01-30 08:55:41','1748033111979'), ('22200','traffic_out','2008-01-30 08:55:41','743573234844'), ('22199','traffic_out','2008-01-30 08:55:41','4220079045096'), ('22198','traffic_out','2008-01-30 08:55:41','44948181149'), ('22197','traffic_out','2008-01-30 08:55:41','471593053694'), ('22196','traffic_out','2008-01-30 08:55:41','1386060300288'), ('22178','traffic_out','2008-01-30 08:55:41','986156592653'), ('22195','traffic_out','2008-01-30 08:55:41','2629130706052'), ('22194','traffic_out','2008-01-30 08:55:41','2838874437108'), ('22193','traffic_out','2008-01-30 08:55:41','1060824279144'), ('22192','traffic_out','2008-01-30 08:55:41','5899595690'), ('22191','traffic_out','2008-01-30 08:55:41','91460319760'), ('22190','traffic_out','2008-01-30 08:55:41','12870033343'), ('22189','traffic_out','2008-01-30 08:55:41','42969395988'), ('22188','traffic_out','2008-01-30 08:55:41','61199322942'), ('22187','traffic_out','2008-01-30 08:55:41','30879994982'), ('22186','traffic_out','2008-01-30 08:55:41','1990971387683'), ('22177','traffic_out','2008-01-30 08:55:41','43518812407') ON DUPLICATE KEY UPDATE output=VALUES(output)"

I saw some posts about it earlier in this thread and it seems like my boost table size is bigger than max allowed boot table size, right?

In boost status i have this;
Current Boost Table Size: 16 MBytes
Max Allowed Boost Table Size: 16 MBytes

I changed my php.ini to memory_limit = 1024M, restarted apache.. but still got 16Mb max allowed boost table size. Am I changing the wrong value?

I've got around 750 hosts, 32K DS and 16K RRDs, any idea on approx how big my boost table/memory should be?


edit:
AH! Forgot about max_heap_table .

Current Boost Table Size: 17 MBytes
Avg Bytes/Record: 545 Bytes
Max Allowed Boost Table Size: 1004 MBytes
Estimated Maximum Records: 1931190 Records

Seems to be working better now.. I'll get back to you in a few hours

edit2:
Hm. Now I got no errors, but my graphs are not updating.
I'm using boost v1.5. Idea, anyone?


Regards
frdrk

[TheWitness]

1) Use SVN version
2) Cron the "php -q poller_grid.php -f" every hour or so.

I need to do a new release to address issues.

TheWitness

[frdrk]

1) Use SVN version

That is, the files in cacti_plugins/boost/trunk/ ?

[TheWitness]

Yes.

[frdrk]

Yes.

Heh. I tried SVN today (from 31 january though), I'm getting this error when I'm browsing graphs:


02/09/2008 05:03:53 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "CREATE TEMPORARY TABLE `rcaston_1202573033` ( `local_data_id` mediumint(8) unsigned NOT NULL default '0', `rrd_name` varchar(19) NOT NULL default '', `time` datetime NOT NULL default '0000-00-00 00:00:00', PRIMARY KEY (`local_data_id`,`rrd_name`,`time`)) ENGINE=MEMORY"
02/09/2008 05:03:53 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "CREATE TEMPORARY TABLE `rcaston_1202573033` ( `local_data_id` mediumint(8) unsigned NOT NULL default '0', `rrd_name` varchar(19) NOT NULL default '', `time` datetime NOT NULL default '0000-00-00 00:00:00', PRIMARY KEY (`local_data_id`,`rrd_name`,`time`)) ENGINE=MEMORY"
02/09/2008 05:03:53 PM - CMDPHP: Poller[0] ERROR: SQL Exec Failed "CREATE TEMPORARY TABLE `rcaston_1202573033` ( `local_data_id` mediumint(8) unsigned NOT NULL default '0', `rrd_name` varchar(19) NOT NULL default '', `time` datetime NOT NULL default '0000-00-00 00:00:00', PRIMARY KEY (`local_data_id`,`rrd_name`,`time`)) ENGINE=MEMORY"


Should I be getting that?

edit: It seems to happen only when I look at cached graphs.