tool xml substitutes special characters in text parameter
Dear all, I noticed that with params of type "text" Galaxy seems to replace certain characters before passing them to the shell. As examples, it changes "@" to "__at__", "}" to "__cc__" and "\" to "X". Is this the standard behavior or am I doing something wrong ? And if it's standard, are there workarounds ? Best, Wolfgang
On Mon, Jul 28, 2014 at 11:03 AM, Wolfgang Maier <wolfgang.maier@biologie.uni-freiburg.de> wrote:
Dear all,
I noticed that with params of type "text" Galaxy seems to replace certain characters before passing them to the shell. As examples, it changes "@" to "__at__", "}" to "__cc__" and "\" to "X". Is this the standard behavior or am I doing something wrong ? And if it's standard, are there workarounds ?
Best, Wolfgang
This is standard behaviour to prevent special characters being used to construct malicious command lines. It can be configured within your tool definition using the <sanitizer> tag set: https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax#A.3Csanitizer.3E... Peter
On 28.07.2014 12:22, Peter Cock wrote:
This is standard behaviour to prevent special characters being used to construct malicious command lines. It can be configured within your tool definition using the <sanitizer> tag set:
https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax#A.3Csanitizer.3E...
Peter
Thanks a lot, Peter, that solved my problem ! Unfortunately, with this one fixed I now run into an additional one: There is one free-text text field defined in my tool, which should accept characters outside the standard ascii code range (i.e. > 127), in particular, German Umlaute äöüÄÖÜ. They are now no longer translated to Xs, but instead I get: an 'ascii' codec can't decode byte 0xc3 in position 22: ordinal not in range(128) error, while parsing the content of the field inside the <command> tag. Is there any way to make Galaxy work with things outside the ascii range (like by using a different codec)? Just in case, here is the complete traceback I'm seeing: Traceback (most recent call last): File "/home/galaxy/WMGalaxy/galaxy-dist/lib/galaxy/jobs/runners/__init__.py", line 152, in prepare_job job_wrapper.prepare() File "/home/galaxy/WMGalaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 679, in prepare self.command_line, self.extra_filenames = tool_evaluator.build() File "/home/galaxy/WMGalaxy/galaxy-dist/lib/galaxy/tools/evaluation.py", line 323, in build self.__build_command_line( ) File "/home/galaxy/WMGalaxy/galaxy-dist/lib/galaxy/tools/evaluation.py", line 339, in __build_command_line command_line = fill_template( command, context=param_dict ) File "/home/galaxy/WMGalaxy/galaxy-dist/lib/galaxy/util/template.py", line 9, in fill_template return str( Template( source=template_text, searchList=[context] ) ) File "/home/galaxy/WMGalaxy/galaxy-dist/eggs/Cheetah-2.2.2-py2.7-linux-x86_64-ucs4.egg/Cheetah/Template.py", line 1004, in __str__ return getattr(self, mainMethName)() File "cheetah_DynamicallyCompiledCheetahTemplate_1406545596_05_93125.py", line 163, in respond File "/home/galaxy/WMGalaxy/galaxy-dist/eggs/Cheetah-2.2.2-py2.7-linux-x86_64-ucs4.egg/Cheetah/DummyTransaction.py", line 47, in getvalue (%s)''' % (nonunicode, ex)) DummyResponseFailure: Looks like you're trying to mix encoded strings with Unicode strings (most likely utf-8 encoded ones) This can happen if you're using the `EncodeUnicode` filter, or if you're manually encoding strings as utf-8 before passing them in on the searchList (possible offenders: ['ascii', 'ascii', ' ', 'non-ASCII characters: \xc3\x84\xc3\x96\xc3\x9c', '/home/galaxy/WMGalaxy/galaxy-dist/database/files/000/dataset_197.dat']) ('ascii' codec can't decode byte 0xc3 in position 22: ordinal not in range(128))
On Mon, Jul 28, 2014 at 12:23 PM, Wolfgang Maier <wolfgang.maier@biologie.uni-freiburg.de> wrote:
On 28.07.2014 12:22, Peter Cock wrote:
This is standard behaviour to prevent special characters being used to construct malicious command lines. It can be configured within your tool definition using the <sanitizer> tag set:
https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax#A.3Csanitizer.3E...
Peter
Thanks a lot, Peter, that solved my problem !
Unfortunately, with this one fixed I now run into an additional one:
There is one free-text text field defined in my tool, which should accept characters outside the standard ascii code range (i.e. > 127), in particular, German Umlaute äöüÄÖÜ.
Hmm. This may be very complicated since a lot will depend on the local server/cluster's locale settings. Not everything will be UTF-8. Peter
On 28.07.2014 13:32, Peter Cock wrote:
On Mon, Jul 28, 2014 at 12:23 PM, Wolfgang Maier <wolfgang.maier@biologie.uni-freiburg.de> wrote:
On 28.07.2014 12:22, Peter Cock wrote:
This is standard behaviour to prevent special characters being used to construct malicious command lines. It can be configured within your tool definition using the <sanitizer> tag set:
https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax#A.3Csanitizer.3E...
Peter
Thanks a lot, Peter, that solved my problem !
Unfortunately, with this one fixed I now run into an additional one:
There is one free-text text field defined in my tool, which should accept characters outside the standard ascii code range (i.e. > 127), in particular, German Umlaute äöüÄÖÜ.
Hmm. This may be very complicated since a lot will depend on the local server/cluster's locale settings. Not everything will be UTF-8.
Peter
Hmm, I just thought that maybe Galaxy has some clever way of interpreting environmental variables. It's not terribly important to get this right. I guess I'll just add a remark to the parameter help then. Wolfgang
participants (2)
-
Peter Cock -
Wolfgang Maier