Problems with external script, cacti don't get data :(

[frivas]

Hi,
I've made a simple script to get the packet loss and latency to a host. I need to monitor at least 30 hosts, and for each one put a separate latency and packet loss graph.
I've modified fping source code to output just the average latency and the packet loss, and the output is like this:

[root@bender log]# fping-rrd -q -c 2 -e 192.168.173.2
out_packetloss:0 out_latency:2.31

The fping binary is setuid root, and it works without any problem using the cactiuser account:

[cactiuser@bender ~]$ /usr/local/sbin/fping-rrd -q -c 2 -e 192.168.173.2
out_packetloss:0 out_latency:2.24

But, when I try to graph anything using a shell script (I've found that cacti hardcoded the call to the external command for each host, but sometimes I need to change things like amount of packets send, so I've made a little shell script that calls my modified fping with the args that I want), CACTI gives me this warning on the log file:

05/12/2005 05:11:43 PM - CMDPHP: Poller[0] Host[35] PING: Host is Alive
05/12/2005 05:11:44 PM - CMDPHP: Poller[0] Host[35] WARNING: Result from CMD not valid. Partial Result:
05/12/2005 05:11:44 PM - CMDPHP: Poller[0] Host[35] CMD: /bin/sh /usr/local/sbin/fping-sh.sh 192.168.173.2, output: U

My fping-sh.sh script is this:

[root@bender log]# cat /usr/local/sbin/fping-sh.sh
#!/bin/sh
OPTIONS="-q -c 2 -e"
/usr/local/sbin/fping-rrd $OPTIONS $1


And the output is like this:

[root@bender log]# /usr/local/sbin/fping-sh.sh 192.168.173.2
out_packetloss:0 out_latency:0.94[root@bender log]#

(notice that it doesn't finish with a newline... I've seen that the other scripts on cacti does the same thing)

Why the script doesn't work?? the log doesn't give me a clue... Maybe it's related to the dual output that I'm doing?? is there something special that I should do to make this work??

Besides this, I created a data template very similar to the Unix - Multi Ping Host template, and 2 graph templates (one for the latency graphs, and other for the packet loss graphs). Is this the right way to do this??

Thanks a lot for your help!!

Regards,

Francisco Rivas C.

[BSOD2600]

1) turn on debugging in cacti and look again at the output of fping-sh.sh.
2) my guess would be the lack of a new line, that is messing up the data being returned properly for cacti.
3) Why did you create a new ping script, when there are several already out there?

[frivas]

I have the debug level on DEBUG, and this is all I can see related to the script being called by cacti:

05/12/2005 07:36:43 PM - CMDPHP: Poller[0] Host[35] PING: Host is Alive
05/12/2005 07:36:44 PM - CMDPHP: Poller[0] Host[35] WARNING: Result from CMD not valid. Partial Result:
05/12/2005 07:36:44 PM - CMDPHP: Poller[0] Host[35] CMD: /bin/sh /usr/local/sbin/fping-sh.sh 192.168.173.2, output: U

One thing that call my attention on the logs, is this:

05/12/2005 07:40:03 PM - POLLER: Poller[0] Parsed MULTI output field '1min:0.00' [map 1min->load_1min]
05/12/2005 07:40:03 PM - POLLER: Poller[0] Parsed MULTI output field '5min:0.01' [map 5min->load_5min]
05/12/2005 07:40:03 PM - POLLER: Poller[0] Parsed MULTI output field '10min:0.00' [map 10min->load_15min]
05/12/2005 07:40:03 PM - POLLER: Poller[0] Parsed MULTI output field '1min:0.00' [map 1min->load_1min]
05/12/2005 07:40:03 PM - POLLER: Poller[0] Parsed MULTI output field '5min:0.01' [map 5min->load_5min]
05/12/2005 07:40:03 PM - POLLER: Poller[0] Parsed MULTI output field '10min:0.00' [map 10min->load_15min]

That's the server load script being called. What's different from my XML?? why the poller identified that output as a MULTI, and my script isn't being identified as MULTI?? Remember, the output of my script is like this:
[root@bender log]# /bin/sh /usr/local/sbin/fping-sh.sh 192.168.173.2
out_packetloss:0 out_latency:1.10
[root@bender log]#

I've tried with a new line character on the output, but the result is the same: cacti doesn't like the output. Should my script be detected as a "MULTI output field" like the load average script? how can I do that??

The question about why I'm not using the other scripts, is because I want this to be portable to several platforms. Using fping's source code, I can control the output of the script without any problems, in every platform that I need. I don't depend on the external "ping" command, so I don't need to modify anything on the script being called. And, by the way, I like to learn how to do new things This is just the first step with cacti, because I need a lot more graphs... and I'm sure that I won't find any scripts to help me with all the stuff that I need. And, after all, I like programming

thanks for your help and time!!

[frivas]

I've noticed another log extract:

05/12/2005 02:15:04 PM - CMDPHP: Poller[0] Host[33] CMD: /usr/local/sbin/fping-sh.sh 192.168.173.2, output: U
05/12/2005 02:15:04 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti/rra/peer_out_packetloss_115.rrd --template 1115921701:U

The RRD have 2 DS: out_packetloss and out_latency. And cacti is only trying to update one of them... maybe is related to the MULTI issue??

[BSOD2600]

Do you have the corrent input field names, for your data input method?

I'm not a linux user, so I'm sort of shooting in the dark about why the shell script wouldnt be working w/cacti.

[frivas]

yes, the names are correct. This is what I have on the data input:

[BSOD2600]

You've viewed the poller cache to make sure cacti is properly passing the values (in this case the IP) to the script? I'm sort of out of ideas now....

[frivas]

yeap... cacti is passing the correct values, as you can see in the logs:

05/12/2005 02:15:04 PM - CMDPHP: Poller[0] Host[33] CMD: /usr/local/sbin/fping-sh.sh 192.168.173.2, output: U

That's why I don't know what's going on...
I'm gonna look into the code of the poller, I'm still in doubt about the "multi" thing

[TheWitness]

Please post your script.

TheWitness

[frivas]

The script is just this:

[frivas@bender sbin]$ cat fping-sh.sh
#!/bin/sh
OPTIONS="-q -c 2 -e"
/usr/local/sbin/fping-rrd $OPTIONS $1
echo
[frivas@bender sbin]$

the fping-rrd binary is a slightly modified version of fping, just for the output:

this is the real fping:
[frivas@bender sbin]$ ./fping -q -c 2 -e 192.168.173.2
192.168.173.2 : xmt/rcv/%loss = 2/2/0%, min/avg/max = 1.25/2.79/4.34

this is my modified fping:
[frivas@bender sbin]$ ./fping-rrd -q -c 2 -e 192.168.173.2
out_packetloss:0 out_latency:0.74

Both of the binaries are setuid root.
The output of my modified fping doesn't have a "\n" at the end, but I already tried with the return and it doesn't work. What can be wrong here?

Just for the record, the specs of the server are:
Dual Intel(R) Xeon(TM) CPU 2.40GHz w/HyperThreading
1GB Ram
Fedora Core 3

[root@bender ~]# rpm -qa | grep rrd
rrdtool-1.0.49-2.1.fc3.rf
[root@bender ~]#

thanks a lot for your help on this

[TheWitness]

According to your debug output, you are using the wrong fping:

05/12/2005 07:36:44 PM - CMDPHP: Poller[0] Host[35] CMD: /bin/sh /usr/local/sbin/fping-sh.sh 192.168.173.2, output: U

When you schedule the correct one, I still think you will need the "\n". This is because, I did not see any output from the incorrect one.

When you run DEBUG, you should see a "Partial Result:..." in it that will show you "part" of what you are looking for. In addition, make sure that you don't have any trailing spaces.

TheWitness

[frivas]

The fping-sh.sh is just the "capsule" to get fping-rrd with the correct options. I've found that when you define a data input method (for example, fping -q -c 2 -e <ip>), it can't be changed (for every host using it) after you add a graph for a host using that data input. For example, if I want to graph the host 192.168.173.2, and I defined the data input "fping with rrd", with the command "fping -q -c 2 -e <ip>", the command executed by the poller for the host 192.168.173.2 will be:

fping -q -c 2 -e 192.168.173.2

right?? but, what happens if later I decide that, instead of 2 packets, I want to send 20 packets?? I just need to change the command on the data input method. So, I put "fping -q -c 20 -e <ip>". This change will affect any new host that I graph with that data input.... but it will not affect the ones that were created before the change. That ones will call fping with the "-c 2" option, not with the "-c 20" option. That's the reason why I'm using the shell script, just to pass the options that I need to the binary.

I've checked the input method, and is pointing the right script. If I run it on the shell (even with the cactiuser id/gid), I get the same result:

[root@bender ~]# /bin/sh /usr/local/sbin/fping-sh.sh 192.168.173.2
out_packetloss:0 out_latency:1.07
[root@bender ~]#

I've added a "." just to look for trailing spaces:

[root@bender ~]# /bin/sh /usr/local/sbin/fping-sh.sh 192.168.173.2
out_packetloss:0 out_latency:0.82.

[root@bender ~]#



This is the script used for that:

[root@bender ~]# cat /usr/local/sbin/fping-sh.sh
#!/bin/sh
OPTIONS="-q -c 2 -e"
/usr/local/sbin/fping-rrd $OPTIONS $1
echo "."
echo
[root@bender ~]#

This is the last log I've got about this:

05/16/2005 10:11:42 PM - CMDPHP: Poller[0] Host[36] PING: Host is Alive
05/16/2005 10:11:42 PM - CMDPHP: Poller[0] Host[36] SNMP: SNMP not performed due to setting or ping result.
05/16/2005 10:11:44 PM - CMDPHP: Poller[0] Host[36] WARNING: Result from CMD not valid. Partial Result:
05/16/2005 10:11:44 PM - CMDPHP: Poller[0] Host[36] CMD: /bin/sh /usr/local/sbin/fping-sh.sh 192.168.173.2, output: U


The "partial result" is BLANK
It's clear for me that the poller is having some problem to execute the script, but the perms for the files are OK:

-rwsr-sr-x 1 root root 42351 May 12 12:42 fping-rrd
-rwxr-xr-x 1 root root 75 May 16 22:06 fping-sh.sh

I'm gonna do some more tests... if you have any other idea of what can be the problem here, I'll be glad to hear
BTW, I'll set setuid the fping-sh.sh script too, just to see what happen...

thanks a lot!!!!

[TheWitness]

what's that last echo for?

[frivas]

the last echo is for print an end of line.
I've found what's the problem: it seems that cacti doesn't like the end of line.

If I modify the script just like this:



[root@bender fping-2.4b2_to]# cat /usr/local/sbin/fping-sh.sh
#!/bin/sh
OPTIONS="-q -c 2 -e"
# commented out just for testing
#/usr/local/sbin/fping-rrd $OPTIONS $1
echo "out_packetloss:1 out_latency:250"


cacti sees the data without problems. The output generated is like this:

[root@bender fping-2.4b2_to]# sh /usr/local/sbin/fping-sh.sh
out_packetloss:1 out_latency:250
[root@bender fping-2.4b2_to]#

As you can see, the "echo" generates the end of line automatically, and cacti likes it.
In the modified fping (and in the original fping too!), the end of line is coded as "\n". This is the part of the code that I've modified to output just the data I need, formated for cacti:

Code: Select all

                else
                {
                        if( h->num_recv <= h->num_sent )
                        {
                /* Modificado por FRivas */
                /*              fprintf( stderr, " xmt/rcv/%%loss = %d/%d/%d%%",
                                        h->num_sent, h->num_recv, h->num_sent > 0 ?
                                        ( ( h->num_sent - h->num_recv ) * 100 ) / h->num_sent : 0 ); */
                                fprintf( stderr, "out_packetloss:%d ",
                                        h->num_sent > 0 ?
                                        ( ( h->num_sent - h->num_recv ) * 100 ) / h->num_sent : 0 );
                /* Modificado por FRivas */

                        }/* IF */
                        else
                        {
                                fprintf( stderr, " xmt/rcv/%%return = %d/%d/%d%%",
                                        h->num_sent, h->num_recv,
                                        ( ( h->num_recv * 100 ) / h->num_sent ) );

                        }/* ELSE */

                        if( h->num_recv )
                        {
                        /* Modificado por FRivas */
                                avg = h->total_time / h->num_recv;
                        /*
                                fprintf( stderr, ", min/avg/max = %s", sprint_tm( h->min_reply ) );
                                fprintf( stderr, "/%s", sprint_tm( avg ) );
                                fprintf( stderr, "/%s", sprint_tm( h->max_reply ) ); */

                                fprintf( stderr, "out_latency:%s", sprint_tm( avg ) );


                        }/* IF */
                        /* Agregador por FRivas - cuando packet loss eq 100, entregar N como latencia */
                        else
                        {
                                fprintf( stderr, "out_latency:U");

                        }
                        fprintf(stderr, "\n");

                }/* ELSE */

As you can see, I'm using '\n' too for the end of line, and cacti doesn't seems to like it. In the shell, the output is the same as the bash version. What could be wrong here?? what's the way cacti get the output string of the script being executed?? Or maybe cacti doesn't see the STDERR output?? I'll try modifing that output... but I'm still curious about the way cacti gets the results of the external scripts...

[frivas]

i've found the solution
I've changed all the STDERR for STDOUT on the C code of fping, and now cacti gets the data:

Code: Select all

05/17/2005 06:45:05 PM - CMDPHP: Poller[0] Host[37] CMD: /bin/sh /usr/local/sbin/fping-sh.sh 192.168.173.10, output: out_packetloss:0 out_latency:0.95
05/17/2005 06:45:05 PM - PHPSVR: Poller[0] DEBUG: PHP Script Server Shutdown request received, exiting
05/17/2005 06:45:05 PM - WEBLOG: Poller[0] CACTI2RRD: /usr/bin/rrdtool graph /var/www/html/graficos//thumb_122.png  --imgformat=PNG  --start=-86400  --end=-300  --title="Latencia-Equant - Latencia"  --base=1000  --height=100  --width=300  --alt-autoscale-max  --lower-limit=0  --no-legend  --vertical-label="milliseconds"  DEF:a="/var/www/html/cacti/rra/latenciaequant_out_packetloss_125.rrd":out_latency:AVERAGE  DEF:b="/var/www/html/cacti/rra/latenciaequant_out_packetloss_125.rrd":out_packetloss:AVERAGE  AREA:a#FF0000:""         LINE2:b#0000FF:""       
05/17/2005 06:45:05 PM - CMDPHP: Poller[0] Time: 1203.2826 s, Theads: N/A, Hosts: 2
05/17/2005 06:45:05 PM - CMDPHP: Poller[0] Time: 3.7683 s, Theads: N/A, Hosts: 2
05/17/2005 06:45:05 PM - POLLER: Poller[0] Parsed MULTI output field 'out_packetloss:0' [map out_packetloss->out_packetloss]
05/17/2005 06:45:05 PM - POLLER: Poller[0] Parsed MULTI output field 'out_latency:0.95' [map out_latency->out_latency]
05/17/2005 06:45:05 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /var/www/html/cacti/rra/latenciaequant_out_packetloss_125.rrd --template out_packetloss:out_latency 1116368702:0:0.95

thanks a lot for all your help and support!!