0.8.8b - '\' being added to poller_item table arg1 coll.

[superflyg2]

Greetings,

I just upgraded to 0.8.8b after suffering several purges of data after adjusting some custom data-templates in 0.8.8a. Now, it seems the poller _item table contains extra '\' characters compared to the data_input_data table. This is causing an empty result from my script. I noticed this when adjusting the snmp settings on a host with the pertinent graphs associated to it. When I rebuilt the poller cache the extra \ characters were inserted for all like data-sources across all devices. I had to use a backup of my db to get data back. Running the poller_reindex_hosts.php cli script does not trigger the error. When I create a new graph using these scripts and arguments (as_path information) the extra '\' characters seem to be installed into poller_item table from the get go even though data_input_data does not register the extra characters.

The scripts being run are pulling sflow traffic statistics from another database that match either as_path or location/as_path info.

Empty result log entries:

Code: Select all

09/06/2013 08:40:31 AM - SPINE: Poller[0] Host[779] ERROR: Empty result [sflow - Total Graph Host]: 'python /var/www/cacti/scripts/offpeer \'^1299_|^1299$|_1299$|_1299_\' ams'
09/06/2013 08:40:42 AM - SPINE: Poller[0] Host[784] ERROR: Empty result [sflow - onpeer graph host]: 'python /var/www/cacti/scripts/globas \'^8452$|^8452_\''
09/06/2013 08:40:43 AM - SPINE: Poller[0] Host[783] ERROR: Empty result [sflow - offpeer graph host]: 'python /var/www/cacti/scripts/globas \'_8452$|_8452_\''
09/06/2013 08:40:50 AM - SPINE: Poller[0] Host[783] ERROR: Empty result [sflow - offpeer graph host]: 'python /var/www/cacti/scripts/globas _8452$\\|_8452_'
09/06/2013 08:40:53 AM - SPINE: Poller[0] Host[784] ERROR: Empty result [sflow - onpeer graph host]: 'python /var/www/cacti/scripts/globas ^8452$\\|^8452_'
09/06/2013 08:40:57 AM - SPINE: Poller[0] Host[779] ERROR: Empty result [sflow - Total Graph Host]: 'python /var/www/cacti/scripts/globas \'^8452$|^8452_|_8452$|_8452_\''
09/06/2013 08:40:58 AM - SPINE: Poller[0] Host[779] ERROR: Empty result [sflow - Total Graph Host]: 'python /var/www/cacti/scripts/globas ^8452$\\|^8452_\\|_8452$\\|_8452_'

Here is output from the db showing a GOOD example. The value column of data_input_data and the arg1 column of poller_item are in sync.

Code: Select all

mysql> select local_data_id,poller_id,host_id,action,arg1 from poller_item where arg1 rlike '5511';
+---------------+-----------+---------+--------+--------------------------------------------------------------------------+
| local_data_id | poller_id | host_id | action | arg1                                                                     |
+---------------+-----------+---------+--------+--------------------------------------------------------------------------+
|          9658 |         0 |     141 |      1 | python /var/www/cacti/scripts/offpeer ^5511_\|^5511$ sjc                 |
|          9542 |         0 |     783 |      1 | python /var/www/cacti/scripts/offpeer _5511$\|_5511_ par                 |
|          9541 |         0 |     784 |      1 | python /var/www/cacti/scripts/offpeer ^5511_\|^5511$ par                 |
|          9505 |         0 |     784 |      1 | python /var/www/cacti/scripts/offpeer ^5511_\|^5511$ ord                 |
|          9506 |         0 |     783 |      1 | python /var/www/cacti/scripts/offpeer _5511$\|_5511_ ord                 |
|          9412 |         0 |     784 |      1 | python /var/www/cacti/scripts/offpeer ^5511_\|^5511$ mad                 |
|          9413 |         0 |     783 |      1 | python /var/www/cacti/scripts/offpeer _5511$\|_5511_ mad                 |
|          9394 |         0 |     784 |      1 | python /var/www/cacti/scripts/offpeer ^5511_\|^5511$ lon                 |
|          9395 |         0 |     783 |      1 | python /var/www/cacti/scripts/offpeer _5511$\|_5511_ lon                 |
|         13407 |         0 |     784 |      1 | python /var/www/cacti/scripts/globas ^5511$\|^5511_                      |
|          9079 |         0 |     138 |      1 | python /var/www/cacti/scripts/offpeer ^5511_\|^5511$ dal                 |
|          9080 |         0 |     138 |      1 | python /var/www/cacti/scripts/offpeer _5511$\|_5511_ dal                 |
|         13408 |         0 |     783 |      1 | python /var/www/cacti/scripts/globas _5511$\|_5511_                      |
|         13409 |         0 |     779 |      1 | python /var/www/cacti/scripts/globas ^5511$\|^5511_\|_5511$\|_5511_      |
|         11584 |         0 |     138 |      1 | python /var/www/cacti/scripts/offpeer ^5511_\|^5511$\|_5511$\|_5511_ dal |
|         11689 |         0 |     779 |      1 | python /var/www/cacti/scripts/offpeer ^5511_\|^5511$\|_5511$\|_5511_ lon |
|         11695 |         0 |     779 |      1 | python /var/www/cacti/scripts/offpeer ^5511_\|^5511$\|_5511$\|_5511_ mad |
|         11726 |         0 |     779 |      1 | python /var/www/cacti/scripts/offpeer ^5511_\|^5511$\|_5511$\|_5511_ ord |
|         11777 |         0 |     141 |      1 | python /var/www/cacti/scripts/offpeer ^5511_\|^5511$\|_5511$\|_5511_ sjc |
|         11738 |         0 |     779 |      1 | python /var/www/cacti/scripts/offpeer ^5511_\|^5511$\|_5511$\|_5511_ par |
|          9659 |         0 |     141 |      1 | python /var/www/cacti/scripts/offpeer _5511$\|_5511_ sjc                 |
|         14134 |         0 |     787 |      1 | python /var/www/cacti/scripts/globas ^5511_12956$\|^5511_12956_          |
+---------------+-----------+---------+--------+--------------------------------------------------------------------------+
22 rows in set (0.02 sec)

mysql> select * from data_input_data where value rlike '5511'; 
+---------------------+-----------------------+---------+--------------------------------+
| data_input_field_id | data_template_data_id | t_value | value                          |
+---------------------+-----------------------+---------+--------------------------------+
|                  47 |                  7951 |         | 5511                           |
|                  47 |                  8435 |         | ^5511$\|^5511_                 |
|                  47 |                  8436 |         | _5511$\|_5511_                 |
|                  47 |                  8437 |         | ^5511$\|^5511_\|_5511$\|_5511_ |
|                  49 |                  8540 |         | ^5511_\|^5511$                 |
|                  49 |                  8642 |         | ^5511_\|^5511$                 |
|                  49 |                  8744 |         | ^5511_\|^5511$                 |
|                  49 |                  9011 |         | ^5511_\|^5511$                 |
|                  49 |                  9149 |         | ^5511_\|^5511$                 |
|                  49 |                  9150 |         | _5511$\|_5511_                 |
|                  49 |                  9464 |         | ^5511_\|^5511$                 |
|                  49 |                  9465 |         | _5511$\|_5511_                 |
|                  49 |                  9482 |         | ^5511_\|^5511$                 |
|                  49 |                  9483 |         | _5511$\|_5511_                 |
|                  49 |                  9575 |         | ^5511_\|^5511$                 |
|                  49 |                  9576 |         | _5511$\|_5511_                 |
|                  49 |                  9611 |         | ^5511_\|^5511$                 |
|                  49 |                  9612 |         | _5511$\|_5511_                 |
|                  49 |                  9728 |         | ^5511_\|^5511$                 |
|                  49 |                  9729 |         | _5511$\|_5511_                 |
|                  49 |                 11661 |         | ^5511_\|^5511$\|_5511$\|_5511_ |
|                  49 |                 11766 |         | ^5511_\|^5511$\|_5511$\|_5511_ |
|                  49 |                 11772 |         | ^5511_\|^5511$\|_5511$\|_5511_ |
|                  49 |                 11803 |         | ^5511_\|^5511$\|_5511$\|_5511_ |
|                  49 |                 11815 |         | ^5511_\|^5511$\|_5511$\|_5511_ |
|                  49 |                 11854 |         | ^5511_\|^5511$\|_5511$\|_5511_ |
|                  47 |                 13487 |         | ^5511$\|^5511_                 |
|                  47 |                 13488 |         | _5511$\|_5511_                 |
|                  47 |                 13489 |         | ^5511$\|^5511_\|_5511$\|_5511_ |
|                  47 |                 14215 |         | ^5511_12956$\|^5511_12956_     |
+---------------------+-----------------------+---------+--------------------------------+
30 rows in set (0.00 sec)

Here is an example of the extra characters: BAD

Code: Select all

mysql> select local_data_id,poller_id,host_id,action,arg1 from poller_item where arg1 rlike '8452';
+---------------+-----------+---------+--------+------------------------------------------------------------------------+
| local_data_id | poller_id | host_id | action | arg1                                                                   |
+---------------+-----------+---------+--------+------------------------------------------------------------------------+
|         14148 |         0 |     783 |      1 | python /var/www/cacti/scripts/globas \'_8452$|_8452_\'                 |
|         14147 |         0 |     784 |      1 | python /var/www/cacti/scripts/globas \'^8452$|^8452_\'                 |
|         14149 |         0 |     779 |      1 | python /var/www/cacti/scripts/globas \'^8452$|^8452_|_8452$|_8452_\'   |
|         14153 |         0 |     784 |      1 | python /var/www/cacti/scripts/globas ^8452$\\|^8452_                   |
|         14154 |         0 |     783 |      1 | python /var/www/cacti/scripts/globas _8452$\\|_8452_                   |
|         14155 |         0 |     779 |      1 | python /var/www/cacti/scripts/globas ^8452$\\|^8452_\\|_8452$\\|_8452_ |
+---------------+-----------+---------+--------+------------------------------------------------------------------------+
6 rows in set (0.03 sec)

mysql> select * from data_input_data where value rlike '8452'; 
+---------------------+-----------------------+---------+--------------------------------+
| data_input_field_id | data_template_data_id | t_value | value                          |
+---------------------+-----------------------+---------+--------------------------------+
|                  47 |                 12405 |         | ^8452_\|^8452$\|_8452$\|_8452_ |
|                  47 |                 13356 |         | ^8452_\|^8452$\|_8452$\|_8452_ |
|                  47 |                 13441 |         | ^8452_\|^8452$\|_8452$\|_8452_ |
|                  47 |                 14235 |         | _8452$\|_8452_                 |
|                  47 |                 14236 |         | ^8452$\|^8452_\|_8452$\|_8452_ |
|                  47 |                 14234 |         | ^8452$\|^8452_                 |
|                  47 |                 14230 |         | '^8452$|^8452_|_8452$|_8452_'  |
|                  47 |                 14228 |         | '^8452$|^8452_'                |
|                  47 |                 14229 |         | '_8452$|_8452_'                |
+---------------------+-----------------------+---------+--------------------------------+
9 rows in set (0.00 sec)

Is there a patch for this? If not, I've got to step back to 0.8.8a

Thank you muchly for the help

-Gabe

[paulgevers]

Is there a patch for this? If not, I've got to step back to 0.8.8a

Well, actually, I think this is the result of a recent security fix. And yes, at first glance, it seems that you were using the hole that existed until 0.8.8b.

I would have to investigate where it exactly comes from, but command line arguments are escaped to prevent possible malicious code. On the other hand, maybe it is escaped one time too many. Would you mind reporting this in the bug tracker as a regression for you?

What is not completely clear to me if this bug is triggered only during the transition from 0.8.8a to 0.8.8b or if your results are really not what you want in 0.8.8b.

[superflyg2]

I would have to investigate where it exactly comes from, but command line arguments are escaped to prevent possible malicious code. On the other hand, maybe it is escaped one time too many. Would you mind reporting this in the bug tracker as a regression for you?

What is not completely clear to me if this bug is triggered only during the transition from 0.8.8a to 0.8.8b or if your results are really not what you want in 0.8.8b.

A quick rundown of what I am trying to do:

I have a database that is populated with sampled flow data (sflow) which has tables i need to search various locations and/or as_paths/as_dst/as_peer info. In order to query the db for the total traffic sent to a specific asn, which can include on-net and off-net traffic, i perform an sql query which includes an rlike '^asn_|^asn$|_asn_|_asn$'. Off-net and On-net lookups are similar. I have a scripts in the scripts/ directory of cacti which is called by a custom 'Data Input Method'. Because the scripts take stdin the regex characters need to be escaped via the '\'.

Here's an example, this is how I run the script locally. In this example I see an extra '\' installed in arg1 before each existing '\'.

Code: Select all

[gsnook@cholla.vm.phx2 ~]$ sudo python /var/www/cacti/scripts/offpeer ^7922_\|$7922$\|_7922_\|_7922$ atl
42945000000

Here's an alternate way to run it which I tried using after upgrading to 8b and seeing this problem (the difference in result is a different sample time). In this example I see an extra '\' inserted before the leading single quote.

Code: Select all

[gsnook@cholla.vm.phx2 ~]$ sudo python /var/www/cacti/scripts/offpeer '^7922_|$7922$|_7922_|_7922$' atl
42763000000

This bug was not immediately present upon the upgrade from 8a to 8b. I noticed it after changing the snmp settings on a couple hosts which had these graphs associated to it. Upon further testing I did a 'rebuild poller cache' via the web gui and noticed that all relevant items in the poller_item table had extra '\' in it. After that I restored a backup of my db and noticed that reindexing the host did NOT cause the extra '\' to be inserted into the poller table. Any new graphs I try to create have the extra '\' in them.

Can you kindly point me to the bug tracker and I'll report this.

Thanks,

Gabe

[paulgevers]

Can you kindly point me to the bug tracker and I'll report this.

Issue tracker is bugs.cacti.net.
[EDIT]: your issue is probably related to issue 2390.[/EDIT]

Both \ and ' are unsafe characters to use in a command from the point of view of possible command line injection.

Just to be sure, you are running cacti straight from this web-site, or did you install a package provided by your distribution?

I believe the offending line around 471 of lib/utility.php, which was changed in view of security issue CVE-2013-1434. Unfortunately, I am slightly unsure now if that line should have been changed at all, as the content of $host should be coming from cacti's own database and should have been sanitized before.

WARNING: this reverts the security change: If you want to get your system working again for now, you could change the following line:

Code: Select all

db_execute("replace into data_input_data (data_input_field_id,data_template_data_id,value) values (" . $template_field["id"] . "," . $data_source["id"] . "," . $cnn_id->qstr($host{$template_field["type_code"]}) . ")");

into

Code: Select all

db_execute("replace into data_input_data (data_input_field_id,data_template_data_id,value) values (" . $template_field["id"] . "," . $data_source["id"] . ",'" . $host{$template_field["type_code"]} . "')");

Please report back if the change above solves your issue.

[gandalf]

I'm indeed not sure, that this really removes the security patch.
Yes, this was part of the patch. But as the data which is escaped here stems from the database without any further user interaction, I'm quite sure that it is _safe_ to remove this. But I did not yet find time to really MAKE SURE.
R.

[nmorris]

I just upgraded to 0.8.8b and believe I am having this same issue. I updated utility.php per the instructions from paulgevers but it hasn't made a difference. Running spine from the command line I'm seeing this:

Code: Select all

...
09/19/2013 03:55:36 PM - SPINE: Poller[0] Host[52] SNMP Result: Host responded to SNMP
09/19/2013 03:55:36 PM - SPINE: Poller[0] Host[52] TH[1] RECACHE: Processing 2 items in the auto reindex cache for 'hostname'
09/19/2013 03:55:36 PM - SPINE: Poller[0] Host[52] TH[1] Recache DataQuery[1] OID: .1.3.6.1.2.1.1.3.0, output: 193235610
09/19/2013 03:55:36 PM - SPINE: Poller[0] Host[52] TH[1] Recache DataQuery[8] OID: .1.3.6.1.2.1.1.3.0, output: 193235610
09/19/2013 03:55:36 PM - SPINE: Poller[0] Host[52] TH[1] NOTE: There are '17' Polling Items for this Host
09/19/2013 03:55:36 PM - SPINE: Poller[0] Host[52] DEBUG: The NIFTY POPEN returned the following File Descriptor 17
NTSTATUS: NT code 0x8004100e - NT code 0x8004100e
PHP Notice:  Undefined offset: 1 in /var/www/cacti/public_html/scripts/wmi.php on line 120
09/19/2013 03:55:36 PM - SPINE: Poller[0] Host[52] TH[1] DS[1241] SCRIPT: /usr/bin/php -q /var/www/cacti/public_html/scripts/wmi.php -h \"hostname\" -u \"/etc/cacti/cactiwmi.pw\" -w \"Win32_PerfRawData_PerfOS_System\" -n \"\" -k \"\" -v \"\" -c \"Threads,SystemUpTime,SystemCallsPersec,ProcessorQueueLength,Processes,ContextSwitchesPersec\", output: U
...

If I manually execute the command without the backslashes it works:

Code: Select all

$ /usr/bin/php -q /var/www/cacti/public_html/scripts/wmi.php -h "hostname" -u "/etc/cacti/cactiwmi.pw" -w "Win32_PerfRawData_PerfOS_System" -n "" -k "" -v "" -c "Threads,SystemUpTime,SystemCallsPersec,ProcessorQueueLength,Processes,ContextSwitchesPersec"

ContextSwitchesPersec:1499701960 Processes:28 ProcessorQueueLength:0 SystemCallsPersec:158735323 SystemUpTime:130221653564995109 Threads:430$ $

But if the backslashes are present the command fails:

Code: Select all

$ /usr/bin/php -q /var/www/cacti/public_html/scripts/wmi.php -h \"hostname\" -u \"/etc/cacti/cactiwmi.pw\" -w \"Win32_PerfRawData_PerfOS_System\" -n \"\" -k \"\" -v \"\" -c \"Threads,SystemUpTime,SystemCallsPersec,ProcessorQueueLength,Processes,ContextSwitchesPersec\"
NTSTATUS: NT code 0x8004100e - NT code 0x8004100e


Return code non-zero, debug mode enabled!



/usr/local/bin/wmic --namespace='""' --authentication-file="/etc/cacti/cactiwmi.pw" //"hostname" "SELECT "Threads,SystemUpTime,SystemCallsPersec,ProcessorQueueLength,Processes,ContextSwitchesPersec" FROM "Win32_PerfRawData_PerfOS_System" WHERE ""='""'"
Exec Status: 1

PHP Notice:  Undefined offset: 1 in /var/www/cacti/public_html/scripts/wmi.php on line 120
$

I see what I think is the same bug reported here: http://bugs.cacti.net/view.php?id=2390

Are there any other suggestions for a workaround while we wait for a bug fix? I have never looked at Cacti's code before, but I'm willing to dig in a bit if I could get a pointer as to where to look and how to trace the issue. Thanks for your help.

[paulgevers]

@nmorris

Code: Select all

$ /usr/bin/php -q /var/www/cacti/public_html/scripts/wmi.php -h "hostname" -u "/etc/cacti/cactiwmi.pw" -w "Win32_PerfRawData_PerfOS_System" -n "" -k "" -v "" -c "Threads,SystemUpTime,SystemCallsPersec,ProcessorQueueLength,Processes,ContextSwitchesPersec"

ContextSwitchesPersec:1499701960 Processes:28 ProcessorQueueLength:0 SystemCallsPersec:158735323 SystemUpTime:130221653564995109 Threads:430$ $

Are there any other suggestions for a workaround while we wait for a bug fix? I have never looked at Cacti's code before, but I'm willing to dig in a bit if I could get a pointer as to where to look and how to trace the issue. Thanks for your help.

Could you try to run you manual run without quotes? I don't see if you really NEED them (it could be needed for -n, -k and -v options). If that works, you have your workaround: don't use quotes. This does not help for people that need backslashes, or really need quotes in the command line.

[gandalf]

After applying the patch, did you rebuild the poller cache?
R.

[nmorris]

@paulgevers
I did remove the quotes and that is working for me. The trouble is that some of the arguments in the command are optional, so when there is no parameter specified the command fails without the quotes. My workaround for now is to remove the quotes like you said and also remove the optional parameters so the command doesn't fail.

@gandalf
Yes. I did rebuild the poller cache several times, although now I'm not sure at what exact point I did it. I can test again and see if maybe I failed to rebuild it immediately following the change. I should be able to add the quotes back into one of my data input methods, rebuild the poller cache, then see a failure in the logs after the next polling cycle begins. I'll do this and report back with the results.

[nmorris]

I have confirmed that rebuilding the poller cache was not the issue. I reinserted the quote marks, rebuilt the poller cache and upon the next polling cycle the log showed that the quotes were escaped with a backslash and the command returned 'U' instead of the value it was supposed to return.

I wouldn't call this urgent since I was able to workaround it by eliminating the optional parameters and removing quote marks. Let me know if you would like me to test anything vis-a-vis this issue.

[phalek]

It seems like the addslashes function being used is unnecessary:

line 129, lib/utility.php

Code: Select all

$poller_items[] = api_poller_cache_item_add($data_source["host_id"], array(), $local_data_id, $data_input["rrd_step"], $action, $data_source_item_name, 1, addslashes($script_path) );

and line 279, lib/utility.php

Code: Select all

$poller_items[] = api_poller_cache_item_add($data_source["host_id"], $host_fields, $local_data_id, $data_input["rrd_step"], $action, get_data_source_item_n ame($output["data_template_rrd_id"]), sizeof($outputs), addslashes($script_path));

"addslashes($script_path)" should be "$script_path" only

The api_poller_cache_item_add function also quotes the string.

Removing the addslashes function from the mentioned lines and re-building the poller cache fixes the issue for me finally.

The bug has been updated accordingly ( http://bugs.cacti.net/view.php?id=2390 )

[dma_k]

It seems like the addslashes function being used is unnecessary:
line 129, lib/utility.php

Code: Select all

$poller_items[] = api_poller_cache_item_add($data_source["host_id"], array(), $local_data_id, $data_input["rrd_step"], $action, $data_source_item_name, 1, addslashes($script_path) );

and line 279, lib/utility.php

Code: Select all

$poller_items[] = api_poller_cache_item_add($data_source["host_id"], $host_fields, $local_data_id, $data_input["rrd_step"], $action, get_data_source_item_n ame($output["data_template_rrd_id"]), sizeof($outputs), addslashes($script_path));

"addslashes($script_path)" should be "$script_path" only
The api_poller_cache_item_add function also quotes the string.
Removing the addslashes function from the mentioned lines and re-building the poller cache fixes the issue for me finally.
The bug has been updated accordingly ( http://bugs.cacti.net/view.php?id=2390 )

I am facing the same problem. More exactly I need to pass the following parameter to the script:

Code: Select all

"(a|b|c)"

Without quotes or with escaped quotes the pipe is interpreted by shell. Current escaping approach is not correct in a sense that it should target escaping for shell and also escape pipe, semicolon, braces, ...:

Code: Select all

\(a\|b\|c\)

Then it is fine.

[decibel83]

Hi all,
I know this was resolved in the version 0.8.8c, but I'm facing the same problem in the version 0.8.8f, how is it possible?
I'm using the CactiWMI but I have the backslashes in the poller cache items:

Code: Select all

Script: /usr/bin/php -q /opt/cacti/scripts/wmi.php -h \'myserver\' -u \'/etc/cacti/cactiwmi.pw\' -w \'Win32_PerfRawData_PerfDisk_LogicalDisk\' -n \'\' -k \'Name\' -v \'C:\' -c \'DiskWritesPersec,DiskWriteBytesPersec,DiskReadsPersec,DiskReadBytesPersec,CurrentDiskQueueLength\' 

This is causing the WMI checks not to work, but they works if I manually execute the command without backslashes.

Please note that this is a new installation of the 0.8.8f version, not an upgrade from an old one.

Could you help me please?
Thank you very much!

[dma_k]

It seems like someone needs escaping, and someone – not. The most user-friendly solution to my mind is to have the extra checkbox "Execute with the shell", which, if checked, means that the argument should be passed to shell (e.g. /bin/bash -c $arg), and thus escaped (if applicable / necessary).