Boost Plugin v1.6 Released

[TheWitness]

I suspect you are running multi-process boost. Can you go back to single process? Also, what is your php memory set to in the php.ini file?

TheWitness

[leonardo_gyn]

TheWitness, here i am again with another strange problem .....

As suggested, i'm running boost plugin in my test environment. Everything runs fine, except for the 'erase old files' permissions problems i reported on a previous message and now, magically, one of my graphs stop working properly when 'On Demand RRD Update' is enabled.

It's a ping template who graphs latency and packet loss. It works just fine with On Demand RRD Update DISABLED, but graphs only one value with On Demand RRD ENABLED.

Check this screenshot:



Surpresily, with On Demand RRD enabled, only one value from the two-value datasource is graphed. The other is not.

Here's some DEBUG log, only the important lines:
(i have lowered 'Boost Update All RRD's to 10 minutes for getting logs faster.)

On Demand RRD Update ENABLED with interesting values highlighed with bold


first pooling
06/23/2007 03:05:28 PM - CMDPHP: Poller[0] Host[7] DS[108] CMD: /home/httpd/html/cacti2/scripts/ping_latency.pl 10.0.20.9, output: roundtrip:23 packetloss:0
06/23/2007 03:05:29 PM - POLLER: Poller[0] Parsed MULTI output field 'roundtrip:23' [map roundtrip->round_trip]
06/23/2007 03:05:29 PM - POLLER: Poller[0] Parsed MULTI output field 'packetloss:0' [map packetloss->packet_loss]
06/23/2007 03:05:29 PM - CMDPHP: Poller[0] DEBUG: SQL Exec: "INSERT INTO poller_output_boost (local_data_id, rrd_name, time, output) VALUES ('108','','2007-06-23 15:05:03','roundtrip:23 packetloss:0') ON DUPLICATE KEY UPDATE output=VALUES(output)"

second pooling
06/23/2007 03:10:27 PM - CMDPHP: Poller[0] Host[7] DS[108] CMD: /home/httpd/html/cacti2/scripts/ping_latency.pl 10.0.20.9, output: roundtrip:26 packetloss:0
06/23/2007 03:10:28 PM - POLLER: Poller[0] Parsed MULTI output field 'roundtrip:26' [map roundtrip->round_trip]
06/23/2007 03:10:28 PM - POLLER: Poller[0] Parsed MULTI output field 'packetloss:0' [map packetloss->packet_loss]
06/23/2007 03:10:28 PM - CMDPHP: Poller[0] DEBUG: SQL Exec: "INSERT INTO poller_output_boost (local_data_id, rrd_name, time, output) VALUES ('108','','2007-06-23 15:10:03','roundtrip:26 packetloss:0') ON DUPLICATE KEY UPDATE output=VALUES(output)"

boost RRD update
06/23/2007 03:10:29 PM - BOOST: Poller[0] Parsed MULTI output field 'roundtrip:23' [map roundtrip->round_trip]
06/23/2007 03:10:29 PM - BOOST: Poller[0] Parsed MULTI output field 'packetloss:0' [map packetloss->packet_loss]
06/23/2007 03:10:29 PM - BOOST: Poller[0] Parsed MULTI output field 'roundtrip:26' [map roundtrip->round_trip]
06/23/2007 03:10:29 PM - BOOST: Poller[0] Parsed MULTI output field 'packetloss:0' [map packetloss->packet_loss]

06/23/2007 03:10:29 PM - BOOST: Poller[0] CACTI2RRD: /usr/local/rrdtool-1.2.12/bin/rrdtool update /home/httpd/html/cacti2/rra/linux_andradina_and_packet_loss_108.rrd --template round_trip:packet_loss 1182621903:23:U 1182622203:26:U



On Demand RRD Update DISABLED with interesting values highlighed with bold

06/23/2007 03:25:28 PM - CMDPHP: Poller[0] Host[7] DS[108] CMD: /home/httpd/html/cacti2/scripts/ping_latency.pl 10.0.20.9, output: roundtrip:54 packetloss:0
06/23/2007 03:25:28 PM - POLLER: Poller[0] Parsed MULTI output field 'roundtrip:54' [map roundtrip->round_trip]
06/23/2007 03:25:28 PM - POLLER: Poller[0] Parsed MULTI output field 'packetloss:0' [map packetloss->packet_loss]
06/23/2007 03:25:28 PM - POLLER: Poller[0] CACTI2RRD: /usr/local/rrdtool-1.2.12/bin/rrdtool update /home/httpd/html/cacti2/rra/linux_andradina_and_packet_loss_108.rrd --template round_trip:packet_loss 1182623103:54:0

06/23/2007 03:30:28 PM - CMDPHP: Poller[0] Host[7] DS[108] CMD: /home/httpd/html/cacti2/scripts/ping_latency.pl 10.0.20.9, output: roundtrip:54 packetloss:0
06/23/2007 03:30:28 PM - POLLER: Poller[0] Parsed MULTI output field 'roundtrip:54' [map roundtrip->round_trip]
06/23/2007 03:30:28 PM - POLLER: Poller[0] Parsed MULTI output field 'packetloss:0' [map packetloss->packet_loss]
06/23/2007 03:30:28 PM - POLLER: Poller[0] CACTI2RRD: /usr/local/rrdtool-1.2.12/bin/rrdtool update /home/httpd/html/cacti2/rra/linux_andradina_and_packet_loss_108.rrd --template round_trip:packet_loss 1182623402:54:0



and then, watching the same graph from other hosts, i got some more interesting values. Seems when i have some packetloss, RRDs are correctly updated !!!



Question is: do Boost Plugin have problems with zero values ??? That would be the only reason for this behavior i detected here ..... Zero value is correctly parsed from data source, zero value is correctly inserted in pooler_output_boost table .... but it's not updated on RRDs.

[TheWitness]

It may, I will have to look into that. It all looks that way from what I am seeing. If so, I will release 1.3 pretty quick.

TheWitness

[TheWitness]

I just confirmed that you are correct. Please look for the following code in setup.php

Code: Select all

if (!is_numeric($matches[2]) || empty($matches[2])) {

Change it to the following:

Code: Select all

if (!is_numeric($matches[2]) || (empty($matches[2]) && ($matches[2] !=0)) {

Let me know how it goes.

TheWitness

[TheWitness]

Actually, looking at the "is_numeric" function, we need to do the following:

Code: Select all

if (!is_numeric($matches[2]) && $matches[2] !=0)) {

Try that instead.

TheWitness

[leonardo_gyn]

Your proposed code didnt compiled, but it seems it's because of an extra closing parenthesis. I have modified my code and i'm actually running with:

Code: Select all

if (!is_numeric($matches[2]) && $matches[2] !=0) {

will post results in some minutes

[leonardo_gyn]

Yeah !! Seems the problem with zero values is solved with this code change.

ZERO value updates:

06/23/2007 07:26:46 PM - BOOST: Poller[0] CACTI2RRD: /usr/local/rrdtool-1.2.23/bin/rrdtool update /home/httpd/html/cacti2/rra/roteador_luzinia_packet_loss_22.rrd --template round_trip:packet_loss 1182637202:30:0 1182637502:30:0

06/23/2007 07:26:46 PM - BOOST: Poller[0] CACTI2RRD: /usr/local/rrdtool-1.2.23/bin/rrdtool update /home/httpd/html/cacti2/rra/roteador_rodoeste_packet_loss_49.rrd --template round_trip:packet_loss 1182637203:124:0 1182637503:115:0

mixed non-ZERO and ZERO value update:

06/23/2007 07:26:46 PM - BOOST: Poller[0] CACTI2RRD: /usr/local/rrdtool-1.2.23/bin/rrdtool update /home/httpd/html/cacti2/rra/linux_luzinia_packet_loss_127.rrd --template round_trip:packet_loss 1182637285:30:0 1182637580:30:2


complete non-ZERO values (was working and continue to work):

06/23/2007 07:41:46 PM - BOOST: Poller[0] CACTI2RRD: /usr/local/rrdtool-1.2.23/bin/rrdtool update /home/httpd/html/cacti2/rra/servidor_logon_matriz_traffic_in_271.rrd --template traffic_in:traffic_out 1182637806:1545747776:2385206984 1182638107:1550521788:2389711604 1182638407:1554563966:2396018794

06/23/2007 07:41:46 PM - BOOST: Poller[0] CACTI2RRD: /usr/local/rrdtool-1.2.23/bin/rrdtool update /home/httpd/html/cacti2/rra/servidor_citrix_matriz_host_uptime_272.rrd --template host_uptime 1182637805:189280031 1182638105:189580234 1182638406:189880421

[leonardo_gyn]

TheWitness, i think i found another minor bug in boost plugin v1.2.

I was trying some new templates and making changes on them, but graphs werent updated, even some big changes on templates ! At first i tought i was modifying one template and graph was using another .... but no, that wasn't the problem.

After some more tests i realized that after a graph template change, cached image was still being displayed until next pooling process.

OK this is not a problem most of the time, but will certainly be a pain in the ass for people who is developing/trying/modifying templates

Is it possible to add some 'was template changed' check ?

[merlinthemagic]

Ya, its running multi process. Ill disable it, and php is running with a 512 cap.

MM

[TheWitness]

leonardo_gyn,

For the developer, it's best to either go into Graph Management which is real time or goto Graph Zoom when developing. Both are real time. I prefer Graph Management.

The template change may be a bit too much to expect being that we don't track such things...

TheWitness

[leonardo_gyn]

Yeah TheWitness .... indeed i used Graph Management for trying graph managements when i noticed the 'next pool running' thing !!

At worst case i think this situation deserves some lines on the README file

Hei .... did you see my previous message about file permissions which are working on command line but not over poller ?

[TheWitness]

Very good notes from SG. He knows Linux/Unix security.

TheWitness

[leonardo_gyn]

Despite the excellent notes from Simon, i was unable to get cleaning of stale files to work.

I tried both ways proposed by Simon. In both ways, i could successfully erase files from command line, but poller process didnt.

With second option (sticky bit) and correct permissions on boost_image directory, everything went exactly as described by Simon.

My boost_image directory permissions:

Code: Select all

drwxrws---  2 nobody cacti   12288 Jun 24 21:41 boost_images

Files created inside that directory (note that the sticky bit works!!):

Code: Select all

[root@cacti boost_images]# ls -la
total 284
drwxrws---  2 nobody cacti 12288 Jun 24 20:13 .
drwxr-xr-x 13   1000 users  4096 Jun 24 19:41 ..
-rw-r--r--  1 nobody cacti 30340 Jun 24 21:36 lgi_132_rrai_1.png
-rw-r--r--  1 nobody cacti 25221 Jun 24 21:36 lgi_132_rrai_2.png
-rw-r--r--  1 nobody cacti 20968 Jun 24 21:36 lgi_132_rrai_3.png
-rw-r--r--  1 nobody cacti 20695 Jun 24 21:36 lgi_132_rrai_4.png
-rw-r--r--  1 nobody cacti 34651 Jun 24 20:12 lgi_137_rrai_0_tsi_7.png
-rw-r--r--  1 nobody cacti 26642 Jun 24 20:12 lgi_139_rrai_0_tsi_7.png
-rw-r--r--  1 nobody cacti 23989 Jun 24 20:13 lgi_139_rrai_1.png
-rw-r--r--  1 nobody cacti 23126 Jun 24 20:13 lgi_139_rrai_2.png
-rw-r--r--  1 nobody cacti 21450 Jun 24 20:13 lgi_139_rrai_3.png
-rw-r--r--  1 nobody cacti 20759 Jun 24 20:13 lgi_139_rrai_4.png
[root@cacti boost_images]# 

Files created have exactly the permission showed by Simon in his exampes, 644.

Anyway, poller process didnt cleaned up files.

I could only get the poller process to clean things correctly after changing file permissions to 664 instead of the default 644. Then old files were cleaned.

It's easier for me just to setup a cron job for doing this chmod and things will be fine I just quit trying to get this working with purely file permissions stuff.

Anyway, thanks very much Simon and TheWitness for notes on this problem.

[rcaston]

http://forums.cacti.net/viewtopic.php?p=104566#104566

the Cisco IP SLA SAA data query/template set in the above thread will not work with boost enabled. When boost is disabled, it does work.

It could be do the zero/non-issue bug listed above, I've just modified my setup.php changing the following:

Code: Select all

(!is_numeric($matches[2]) && $matches[2] !=0) {

(updated: The code change above did not seem to fix the issue with the template/query))

[TheWitness]

Let me push v1.3 here. I have not had any negative feedback. So, I'll push it.

TheWitness