Boost Plugin v1.6 Released

[Instigater]

Are we going to see plugin architecture and boost in main release v.0.9.0?

[TheWitness]

Yes.

[TheWitness]

All,

You must edit <path_cacti>/poller.php and add a line to it in PIA 1.1. That line reads as follows:

Code: Select all

db_execute("truncate table poller_output");

Place that line right after the following:

Code: Select all

db_execute("truncate table poller_time");

The reason for this is to reduce the likelyhood of corruption in your datasources due to the implementation of buggy templates, etc.

TheWitness

If you want to know that you have corruption, add the following instead:

Code: Select all

$rows = db_fetch_cell("SELECT count(*) FROM poller_output");

if ($rows > 0) {
   cacti_log("WRANING: Poller found partially populated poller_output table.", "POLLER");
   db_execute("truncate table poller_output");
}

If you see this warning often, it likely means that you have a either of the following:

1) Run cmd.php or cactid from outside the poller - Which would be ok
2) Have a damaged data source configuration - Repopulate your poller cache, else view the contents of the poller output table after your poller ends to find the problem data sources
3) Have 2 crons running.
4) Developed a buggy template and landed some crap in your poller_output table as a result.

I found the issue when I was performing a step around item 4 above.

[gthe]

Can i use boost 1.2 on cacti 0.8.6j ?

[TheWitness]

yes

[gthe]

yes

After I intalled, I have found some things:
1. Just like this:

Hi, Larry,
I re-created my plugin development environment. Now it's at level 0.8.6j with patches and plugin-arch V1.1. I installed the boost (there are only files in the main directory, now).
From "System Utilities", I tried the "View Boost Status" and nothing seems to happen. No error_log, no cacti.log entry (even with DEBUG level).
Reinhard

2. In readme I read:

Code: Select all

    7) Finally, you can poll the server using telnet.  To do so, start a telnet session on the port you choose to use (default 9050) and type the  "status" command followd by a carriage return.  You should receive a  favorable response. 

On my host:

Code: Select all

[test@system ~]# telnet 127.0.0.1 9050
Trying 127.0.0.1...
Connected to system.----(127.0.0.1).
Escape character is '^]'.
status
Server Status OKConnection closed by foreign host.
[test@system ~]#

It is normal? Status OK and close connection.

3. I don't see any boost's logs in cacti.log. To see this I need manually start boost_server.php and look in console.

4. Last, but main, problem with graph
In console next errors:

2007:05:22 11:48:58 - Host Connected '127.0.0.1'
2007:05:22 11:48:58 - Host Validated '127.0.0.1'
2007:05:22 11:48:58 - RRD Command 'update /var/www/html/cacti/rra/server_sys_kinel_ke_time_2646.rrd --template 1179771302 1179771602:109.9648:1:40:126:126:6620:U 1179771902 1179772802 1179773102:126.0681:1:40:126:126:6620:U 1179788402 1179790802:136.9701:1:40:126:126:6620:U 1179793202 1179794102:152.5565:1:40:126:126:6620:U'
ERROR: unknown DS name '1179771302'
2007:05:22 11:48:58 - Host Disconnect Request Received. '127.0.0.1'

but sometimes this graph updates correctly
or:

2007:05:22 11:59:42 - Host Connected '127.0.0.1'
2007:05:22 11:59:42 - Host Validated '127.0.0.1'
2007:05:22 11:59:42 - RRD Command 'update /var/www/html/cacti/rra/switch_des2108_uk_isputatel_13_traffic_in_184.rrd --template traffic_in:traffic_out 1179771302:U 1179772503:U 1179772802:1959182 1179773703:U 1179774304:U 1179775203:1959182 1179776403:1959182 1179777003:U 1179778204:1959182 1179778807:U 1179779106:U 1179780306:1959182 1179782703:U 1179785103:1959182 1179786003:1959182 1179787203:1959182 1179788103:1959182 1179789303:1959182 1179792302:1959182 1179792603:U 1179793203:1959182 1179793803:U 1179794103:U 1179794703:1959182 1179795903:1959182 1179797703:1959182 1179799203:U 1179802203:1959182 1179802803:1959182 1179804302:1959182 1179804603:1959182 1179805202:1959182 1179805803:1959182 1179806703:U 1179808803:1959182:4482245 1179809103:1959182:U 1179809403:1959182:U 1179809703:1959182:4482245 1179810003:1959182'
ERROR: expected 2 data source readings (got 1) from 1179771302:U
2007:05:22 11:59:42 - Host Disconnect Request Received. '127.0.0.1'

Before install boost I use this graph for 6-7 month withous this problemm.

My host info:

Cacti Version - 0.8.6j
Plugin Architecture - 1.1
Poller Type - Cactid v0.8.6i
Server Info - Linux 2.6.9-42.0.10.ELsmp
Web Server - Apache/2.0.52 (CentOS)
PHP - 4.3.9
PHP Extensions - yp, xml, wddx, tokenizer, sysvshm, sysvsem, standard, sockets, shmop, session, pspell, posix, pcre, overload, mime_magic, iconv, gmp, gettext, ftp, exif, dio, dbx, dba, curl, ctype, calendar, bz2, bcmath, zlib, openssl, apache2handler, dbg, gd, ldap, mysql, odbc, snmp
MySQL - 5.0.22-log
RRDTool - 1.2.18
SNMP - 5.1.2
Plugins

• Update Checker (update - v0.3)
  Network Tools (tools - v0.2)
  Host Info (hostinfo - v0.1)
  Network Discovery (discovery - v0.7)
  Device Tracking (mactrack - v0.0.1b)
  Dataquery (dataquery - v0.1.2a)
  PHP Network Weathermap (weathermap - v0.91)
  Thresholds (thold - v0.3.2)
  Poller Performance Report (pollperf - v0.32)
  ipsubnet - v0.4b
  impblinding - v0.0.1a
  Configuration manager (configmanager - v0.6)
  Create Aggregate Graphs (aggregate - v0.62)
  Poller Performance Report (pollperf - v0.32)
  Large Site Performane Booster for Cacti (boost - v1.2)
  Installed cacti_rrdsvc

05/22/2007 10:14:11 AM - SYSTEM MACTRACK STATS: Time:103.1877 ConcurrentProcesses:10 Devices:38
05/22/2007 10:17:15 AM - SYSTEM STATS: Time:133.5960 Method:cactid Processes:1 Threads:40 Hosts:126 HostsPerProcess:126 DataSources:6620 RRDsProcessed:0
05/22/2007 10:22:17 AM - SYSTEM STATS: Time:136.1693 Method:cactid Processes:1 Threads:40 Hosts:126 HostsPerProcess:126 DataSources:6620 RRDsProcessed:0
05/22/2007 10:23:38 AM - SYSTEM MACTRACK STATS: Time:80.5280 ConcurrentProcesses:10 Devices:38
05/22/2007 10:27:34 AM - SYSTEM STATS: Time:153.3411 Method:cactid Processes:1 Threads:40 Hosts:126 HostsPerProcess:126 DataSources:6620 RRDsProcessed:0

[TheWitness]

gthe,

You seem to have encountered a bug. The rrdupdate syntax on that graph in question is not correct. I would like to review this in detail with you. Time is scarce for me right now though.

TheWitness

[gthe]

gthe,

You seem to have encountered a bug. The rrdupdate syntax on that graph in question is not correct. I would like to review this in detail with you. Time is scarce for me right now though.

TheWitness

I look in my cacti log all day and found:
My graph

/usr/bin/rrdtool graph - \
--imgformat=PNG \
--start=-86400 \
--end=-300 \
--title="Switch DES-2108 [UK_24 Transportnaya_5] - Errors - FE_5/5" \
--rigid \
--base=1000 \
--height=140 \
--width=650 \
--alt-autoscale-max \
--lower-limit=0 \
--vertical-label="errors/sec" \
--slope-mode \
DEF:a="/var/www/html/cacti/rra/switch_des2108_uk_24_transportnaya_errors_in_3938.rrd":discards_in:AVERAGE \
DEF:b="/var/www/html/cacti/rra/switch_des2108_uk_24_transportnaya_errors_in_3938.rrd":errors_in:AVERAGE \
DEF:c="/var/www/html/cacti/rra/switch_des2108_uk_24_transportnaya_errors_in_3938.rrd":discards_out:AVERAGE \
DEF:d="/var/www/html/cacti/rra/switch_des2108_uk_24_transportnaya_errors_in_3938.rrd":errors_out:AVERAGE \
LINE1:a#FFAB00:"Discards In" \
GPRINTLAST:" Current\:%8.2lf %s" \
GPRINTAVERAGE:"Average\:%8.2lf %s" \
GPRINTMAX:"Maximum\:%8.2lf %s\n" \
LINE1:b#F51D30:"Errors In" \
GPRINTLAST:" Current\:%8.2lf %s" \
GPRINTAVERAGE:"Average\:%8.2lf %s" \
GPRINTMAX:"Maximum\:%8.2lf %s\n" \
LINE1:c#C4FD3D:"Discards Out" \
GPRINT:c:LAST:"Current\:%8.2lf %s" \
GPRINT:c:AVERAGE:"Average\:%8.2lf %s" \
GPRINT:c:MAX:"Maximum\:%8.2lf %s\n" \
LINE1:d#00694A:"Errors Out" \
GPRINT:d:LAST:" Current\:%8.2lf %s" \
GPRINT:d:AVERAGE:"Average\:%8.2lf %s" \
GPRINT:d:MAX:"Maximum\:%8.2lf %s\n"

From cacti.log:

05/22/2007 11:26:04 PM - CACTID: Poller[0] Host[147] DS[3938] SNMP: v1: 192.168.30.24, dsname: discards_in, oid: .1.3.6.1.2.1.2.2.1.13.5, value: 0
05/22/2007 11:26:07 PM - CACTID: Poller[0] Host[147] DS[3938] SNMP: v1: 192.168.30.24, dsname: errors_in, oid: .1.3.6.1.2.1.2.2.1.14.5, value: 4881
05/22/2007 11:26:16 PM - CACTID: Poller[0] Host[147] DS[3938] SNMP: v1: 192.168.30.24, dsname: discards_out, oid: .1.3.6.1.2.1.2.2.1.19.5, value: 0
05/22/2007 11:26:19 PM - CACTID: Poller[0] Host[147] DS[3938] SNMP: v1: 192.168.30.24, dsname: errors_out, oid: .1.3.6.1.2.1.2.2.1.20.5, value: 0
05/22/2007 11:31:04 PM - CACTID: Poller[0] Host[147] DS[3938] SNMP: v1: 192.168.30.24, dsname: discards_in, oid: .1.3.6.1.2.1.2.2.1.13.5, value: 0
05/22/2007 11:31:08 PM - CACTID: Poller[0] Host[147] DS[3938] SNMP: v1: 192.168.30.24, dsname: errors_in, oid: .1.3.6.1.2.1.2.2.1.14.5, value: 4893
05/22/2007 11:31:20 PM - CACTID: Poller[0] Host[147] DS[3938] SNMP: v1: 192.168.30.24, dsname: discards_out, oid: .1.3.6.1.2.1.2.2.1.19.5, value: 0
05/22/2007 11:35:57 PM - CACTID: Poller[0] Host[147] DS[3938] SNMP: v1: 192.168.30.24, dsname: discards_in, oid: .1.3.6.1.2.1.2.2.1.13.5, value: 0
05/22/2007 11:35:59 PM - CACTID: Poller[0] Host[147] DS[3938] SNMP: v1: 192.168.30.24, dsname: errors_in, oid: .1.3.6.1.2.1.2.2.1.14.5, value: 4893
05/22/2007 11:36:09 PM - CACTID: Poller[0] Host[147] DS[3938] SNMP: v1: 192.168.30.24, dsname: discards_out, oid: .1.3.6.1.2.1.2.2.1.19.5, value: 0
05/22/2007 11:36:13 PM - CACTID: Poller[0] Host[147] DS[3938] SNMP: v1: 192.168.30.24, dsname: errors_out, oid: .1.3.6.1.2.1.2.2.1.20.5, value: 0
05/22/2007 11:41:00 PM - CACTID: Poller[0] Host[147] DS[3938] SNMP: v1: 192.168.30.24, dsname: discards_in, oid: .1.3.6.1.2.1.2.2.1.13.5, value: 0
05/22/2007 11:41:03 PM - CACTID: Poller[0] Host[147] DS[3938] SNMP: v1: 192.168.30.24, dsname: errors_in, oid: .1.3.6.1.2.1.2.2.1.14.5, value: 4893
05/22/2007 11:41:14 PM - CACTID: Poller[0] Host[147] DS[3938] SNMP: v1: 192.168.30.24, dsname: discards_out, oid: .1.3.6.1.2.1.2.2.1.19.5, value: 0
05/22/2007 11:41:17 PM - CACTID: Poller[0] Host[147] DS[3938] SNMP: v1: 192.168.30.24, dsname: errors_out, oid: .1.3.6.1.2.1.2.2.1.20.5, value: 0

soo, I have only 3 (of 4) data source for 05/22/2007 11:31 PM
Next from table:

mysql> SELECT * FROM poller_output_boost where local_data_id=3938 order by time;
+---------------+--------------+---------------------+--------+
| local_data_id | rrd_name | time | output |
+---------------+--------------+---------------------+--------+
| 3938 | errors_in | 2007-05-22 23:31:00 | 4893 |
| 3938 | discards_out | 2007-05-22 23:31:00 | 0 |
| 3938 | discards_in | 2007-05-22 23:31:00 | 0 |
| 3938 | discards_out | 2007-05-22 23:40:54 | 0 |
| 3938 | errors_out | 2007-05-22 23:40:54 | 0 |
| 3938 | discards_in | 2007-05-22 23:40:54 | 0 |
| 3938 | errors_in | 2007-05-22 23:40:54 | 4893 |
+---------------+--------------+---------------------+--------+
7 rows in set (0.08 sec)

And finally when update graph we have error:

2007:05:23 00:18:07 - Host Connected '127.0.0.1'
2007:05:23 00:18:07 - Host Validated '127.0.0.1'
2007:05:23 00:18:07 - RRD Command 'update /var/www/html/cacti/rra/switch_des2108_uk_24_transportnaya_errors_in_3938.rrd --template discards_in:discards_out:errors_in:errors_out 1179858660:0:0:4893 1179859254:0:0:4893:0'
2007:05:23 00:18:07 - Host Disconnect Request Received. '127.0.0.1'
ERROR: expected 4 data source readings (got 3) from 1179858660:0:0:4893

so, all updates (2) for this graph are fail, even if second update are correct.

Sorry for my english

[Jeb]

I got an strange behavior of boost v1.2 (i'am using cacti 0.8.6i+PIA 1.1)

I set boost to update all rrds every 10 minutes but in fact this is what append:

Polling
Polling
Update
Polling
Polling
Polling
Update
Polling
Polling
Update
Polling
Polling
Polling
Update
...

So update are every 10minutes, 15m, 10m, 15m ...

I'am wondering if it's normal or if i have a problem...

[Jeb]

Hello TheWitness,

I don't know if you know that, but in boost_process_poller_output you delete data from MySQL in anycase.
So if update of RRDs fail you loose some data.

There is a if ($ok_to_delete) but there is only a $ok_to_delete = TRUE;
So...
I got problem of data loose because my webserver was not able to write to RRDs (yes, i read to fast your read me step 6)

I did not look to much at the code but it seems that patching is not simple :/
I think that the DELETE should be inside the foreach that make updates, that would be much simple to get $ok_to_delete by $item (instead of using an array for $ok_to_delete or something like that)...

anyway you make a test with $return_value but in my case $return_value is "OK" (i add some debug)

From what i see in boost_rrdtool_execute
rrdtool_execute($command, $output_to_stdout, $rrd_tool_output_options, $rrd_struc, "BOOST");
return "OK";

I think that a test on rrdtool_execute result should be add !!

[Cacti Jack]

05/25/2007 02:57:52 PM - BOOST: Poller[0] PROGERR: ERRNO:'2048' TYPE:'Runtime Notice' MESSAGE:'Implicit cloning object of class 'ADORecordSet_empty' because of 'zend.ze1_compatibility_mode'' IN FILE:'D:\Cacti\lib\database.php' LINE NO:'69'


My log is full of these errors; should I disable zend compatibility mode in my php.ini?

[knebb]

For a short summarize of boost's requirement and to make sure I understood all things right.

-distributed servers (poller and http) must have the rra shared between them (NFS/SMB)?
-the NFS bug for rrdtool is not related, because the write access is locally on poller server, which shares the rra as read-only.
-the http servers need also the boot plugin installed- there I can configure where the boost server is located, right?
-the open port is just used to force the rra updates?
-in normal cacti, the rras getting updates directly from the poller, in boost cacti the data is stored in MySQL and pushed to rra when needed?

If so, unfortunately I can't use boost for my purpose. I need a solution where I have a really remote http server and the poller server locally. Between the two there is only a small bandwith (broadband, 256kb uplink) available. And for security reason I won't use NFS access from internet.

However, if one of my above statements is wrong feal free to correct me.

BTW: Cacti rocks! THX to all developers!

[skurfuerst]

Hello everybody,

Thanks for your great plugin, it helps a lot to reduce the load tremendously.

In my installation, I had a strange issue that, at some timepoints, the graphs for different interfaces stopped working.
After more detailed investigation, I found out that somehow, not all entries from the poller_output were deleted (which should happen in lib/poller.php - process_poller_output() ). Sometimes, under weird circumstances I do not fully understand, a few records stayed in there.
After this timepoint, the boost plugin could not feed the data to RRDtool anymore for these particular graphs... Well, it tried, but there was only partial data supplied to "rrdtool update" (if it expected f.e. 6 values, it sometimes got only one), so "rrdtool update" returned an error message.

I fixed the problem for myself by inserting
db_execute('TRUNCATE TABLE poller_output');
at the end of process_poller_output - but I know it only fixes the symptoms, not the reason why it happens...

Maybe someone of you has a better idea why this happens? I'll be very curious for any feedback!

Greets,
Sebastian

[TheWitness]

Sebastian,

This can you log when this event takes place with my code in prior posts and then let me know if this is happening even when you are not adding/deleting graphs from Cacti? In other words, keep the config static for a while and see if any of these issues happen.

TheWitness

[Jeb]

I got the same problem, but i know why. The system has crashed during polling so that there was some data remaining in poller_output...