How does Galaxy determine that a job has failed? Does it simply see if the STDERR is empty? What happens if an application normally outputs to STDERR?
This is a problem for our local installation, as I have enabled it to run as the local user on the backend cluster. If a user has an error in the .bashrc file, it will automatically write to STDERR, and all jobs, no matter what, are labelled as failing.
David Hoover Helix Systems Staff
Yes, by default STDERR is used for determining error state. Older tools likely haven't been updated to use it, but there does exist functionality for doing more sophisticated error detection (exit codes, regex, etc), see the wiki here: http://wiki.g2.bx.psu.edu/Future/Job%20Failure%20When%20stderr
-Dannon
On Oct 22, 2012, at 11:23 AM, David Hoover hooverdm@helix.nih.gov wrote:
How does Galaxy determine that a job has failed? Does it simply see if the STDERR is empty? What happens if an application normally outputs to STDERR?
This is a problem for our local installation, as I have enabled it to run as the local user on the backend cluster. If a user has an error in the .bashrc file, it will automatically write to STDERR, and all jobs, no matter what, are labelled as failing.
David Hoover Helix Systems Staff ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
On Mon, Oct 22, 2012 at 4:23 PM, David Hoover hooverdm@helix.nih.gov wrote:
How does Galaxy determine that a job has failed?
It now depends on the individual tool's XML file.
Does it simply see if the STDERR is empty?
Why default, yes. The tool's XML can specify particular regexs to look for, or to decide based on the return code - but for the time being most of the tools still just look at stderr. See: http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax
What happens if an application normally outputs to STDERR?
Either use the new functionality in the XML definition, or what older Galaxy tools did was a wrapper script to hide/redirect stderr to avoid false positives.
This is a problem for our local installation, as I have enabled it to run as the local user on the backend cluster. If a user has an error in the .bashrc file, it will automatically write to STDERR, and all jobs, no matter what, are labelled as failing.
In which case the user should see those errors and be able to do something about it, right?
Peter
By default Galaxy checks stderr, if it's not empty - returns an error. So if your tool doesn't fail (returns 0) but you print something to stderr , your tool will still fail in Galaxy. There's stderr_wrapper.py workaround for that.
On the other hand, if you tool returns non zero but doesn't use stderr -- Galaxy ignores tools return value.
There are two ways around that:
1. Galaxy has <exit_code> tag to specify which exit codes to handle http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#A.3Cexit_code.3...
So in my tool.xml I have: <stdio> <exit_code range="1:255" level="fatal" description="XLICTRecon.exe Exception" /> </stdio>
2. Simple workaround in the Python wrapper, print something to stderr if the tool returns an error:
returncode = subprocess.call(cmd) if(returncode): sys.stderr.write('Error: returned ' + str(returncode))
-Alex
-----Original Message----- From: galaxy-dev-bounces@lists.bx.psu.edu [mailto:galaxy-dev-bounces@lists.bx.psu.edu] On Behalf Of Peter Cock Sent: Tuesday, 23 October 2012 2:30 AM To: David Hoover Cc: Galaxy Dev Subject: Re: [galaxy-dev] determination of errors
On Mon, Oct 22, 2012 at 4:23 PM, David Hoover hooverdm@helix.nih.gov wrote:
How does Galaxy determine that a job has failed?
It now depends on the individual tool's XML file.
Does it simply see if the STDERR is empty?
Why default, yes. The tool's XML can specify particular regexs to look for, or to decide based on the return code - but for the time being most of the tools still just look at stderr. See: http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax
What happens if an application normally outputs to STDERR?
Either use the new functionality in the XML definition, or what older Galaxy tools did was a wrapper script to hide/redirect stderr to avoid false positives.
This is a problem for our local installation, as I have enabled it to run as the local user on the backend cluster. If a user has an error in the .bashrc file, it will automatically write to STDERR, and all jobs, no matter what, are labelled as failing.
In which case the user should see those errors and be able to do something about it, right?
Peter ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
galaxy-dev@lists.galaxyproject.org
-
Alex.Khassapov＠csiro.au -
Dannon Baker -
David Hoover -
Peter Cock