My plan is to refactor build_command_line out of lib/galaxy/jobs/runners/__init__.py, into its own module and add unit tests for this functionality. I'll issue a pull request and let people test it. I am also testing Nicola's PR, I will approve it shortly.
Nate yelled at me for committing code yesterday :) so I will let him figure out when to merge it. But I will approve the pull requests that I am willing to say are tested and that I will take responsibility for if something breaks in them.
-John
On Fri, Oct 11, 2013 at 9:13 AM, Björn Grüning bjoern.gruening@pharmazie.uni-freiburg.de wrote:
Ah now I got it ... there are two build_command_line functions. lib/galaxy/tools/__init__.py and lib/galaxy/jobs/runners/__init__.py.
And one is adding a trailing ';'
So either stripping the trailing semicolon, right after creating the command line from cheetah (patch was attached) or adding more logic to only add a ';' when no one is there.
Which one is preferred, should I create a pull request?
Thanks! Bjoern
If the tool produces a trailing semi-colon (and without Nicola's patch the bowtie 2 tool definitely could), then I bet build_command_line() is going to be what is adding the second semi-colon. It has not check that there isn't a semi-colon at the end of the command before adding one and extending the command. It might not be the only culprit, but it is likely one.
-John
On Fri, Oct 11, 2013 at 9:02 AM, Björn Grüning bjoern.gruening@pharmazie.uni-freiburg.de wrote:
Hi Nate,
Hi Björn,
Are you certain that SGE is responsible? I ran in to the same thing with bowtie2 with other runners. The double semicolon is invalid syntax in Bourne-compatible shells and so the tool ought to be fixed not to produce them (it happens under certain combinations of options). Have you noticed this happening with any other tools?
Yes, but only with tools from the toolshed and there I poked people to remove the trailing semicolon from the wrapper. From my observation the tool does not produce double semicolons. Only one at the end, but the Job scheduler adds one, resulting in a double one with invalid syntax.
I can't see where bowtie is producing a double semicolon. I really think SGE is adding one. At least I restructured the semicolons (attached) and it worked for us.
Thanks Nate, Bjoern
--nate
On Oct 11, 2013, at 8:57 AM, John Chilton wrote:
This sounds like a good way to solve the problem, I guess I would modify
On Thu, Oct 10, 2013 at 6:10 PM, Björn Grüning bjoern.gruening@pharmazie.uni-freiburg.de wrote:
Dear all,
some SGE instances will add automatically an semicolon add the end of each command, resulting in a disrupted job, because ';;' are not allowed.
The latest changes to the Bowtie2 wrapper resulting in such a case and a crash in our instance. An easy solution would be to fix the wrapper and omitting the trailing ';' but maybe its better to fix it once for all in the corresponding tools code, patched attached.
I'm not familiar with other Job schedulers so I'm seeking for comments ... it there any disadvantage of removing trailing ';' from the command line?
Ciao, Bjoern
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/