Re: [galaxy-dev] code example for improved error handling

17 Aug 2012

      Please see http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax . 
There is a section for "<stdio>, <regex>, and <exit_code> tag sets". The documentation 
applies to the latest galaxy-dist, though most of what's mentioned (aside from updating 
stdout and stderr with warning messages) is supported in galaxy-central. 

(You can also use the fairly ugly URL http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#A.3Cstdio.3E.2C... ). 

-Scott 

----- Original Message -----
...
Hi
...
I'm writing a tools for an executable that writes results to stdout
and reports success message to stderr, so it really need the
improved error handling. Is there any sample code or documentation
available?
...
Thanks
...
Birgit Crain, Ph.D. | Sr. Professional Services Scientist | Complete
Genomics, Inc.
(650) 428-6023 office | (408) 605-3938 mobile
bcrain@completegenomics.com
...
From: Nicole Rockweiler < n.rockweiler@gmail.com >
Date: Friday, July 20, 2012 10:53 AM
To: Brian Haas < bhaas@broadinstitute.org >
Cc: " galaxy-dev@bx.psu.edu " < galaxy-dev@bx.psu.edu >
Subject: Re: [galaxy-dev] pipeline execution succeeds but galaxy
shows failure
...
Hi Brian,
...
A couple of days ago, smcmanus pushed the following change to the
repo:
...
...
Tools can now specify their own handling of stderr and stdout
regular
expressions as well as exit code ranges.
https://bitbucket.org/galaxy/galaxy-central/issue/325/allow-tool-authors-to-...
...
It looks like the documentation has yet to be written.
...
Hope this helps,
Nicole
...
On Fri, Jul 20, 2012 at 12:46 PM, Brad Chapman < chapmanb@50mail.com
...
wrote:
...
...
Brian;

...
...
...
I wrote a pipeline (xml attached) that, from what I can gather,
...
...
succeeds, but galaxy shows it as an error and doesn't make the
output
...
...
file accessible as a new data set.

...
...
Is it possible the software is writing to standard error? Galaxy
doesn't
...
check status codes, but rather check for stderr and assumes that
output
...
indicates a problem. You can wrap the problematic programs with a
little
...
script to eat up stderr and check that everything is okay:

...
...
http://wiki.g2.bx.psu.edu/Future/Job%20Failure%20When%20stderr

...
...
Brad

...
...
...
...
...
...
From the server log, I can see that the command line is being
...
...
constructed correctly, and it even indicates that it's captured
the
...
...
output, but in the display of the web browser, it just shows up
in
the
...
...
error state. The script being run exits (0) on success. Any
ideas?
...
...
...
...
Here's what the output section of my xml file looks like:
...
...
...
...
<outputs>
...
...
<data format="bam" name="coordSortedBam" label="${ tool.name }
...
...
on ${on_string}: coord-sorted read alignments"
...
...
from_work_dir="alignment/alignment.coordSorted.bam"/>
...
...
</outputs>
...
...
...
...
and here's what the server log states:
...
...
...
...
galaxy.jobs.handler INFO 2012-07-20 09:52:05,240 (30) Job
dispatched
...
...
galaxy.jobs.runners.local DEBUG 2012-07-20 09:52:05,453
executing:
...
...
alignReads.pl --target
...
...
/Users/bhaas/BioIfx/Galaxy/galaxy-dist/database/files/000/dataset_26.dat
...
...
-o alignment --aligner bowtie --single
...
...
/Users/bhaas/BioIfx/Galaxy/galaxy-dist/database/files/000/dataset_23.dat
...
...
--seqType fq
...
...
...
...
galaxy.jobs DEBUG 2012-07-20 09:52:16,673 finish(): Moved
...
...
/Users/bhaas/BioIfx/Galaxy/galaxy-dist/database/job_working_directory/000/30/alignment/alignment.coordSorted.bam
...
...
to
/Users/bhaas/BioIfx/Galaxy/galaxy-dist/database/files/000/dataset_50.dat
...
...
as directed by from_work_dir
...
...
...
...
Again, as far as I can tell, everything worked - but the browser
...
...
doesn't think so.
...
...
...
...
I've run the exact command above on the command-line, and it
exits(0)
...
...
indicating success.
...
...
Also, I've verified that when run through my galaxy instance, the
...
...
galaxy-relocated output file is as expected.
...
...
...
...
Many thanks for your help. I'm still getting my feet wet with
...
...
galaxy, reading through all the documentation and searching the
...
...
mailing list for additional help.
...
...
...
...
best regards,
...
...
...
...
-brian
...
...
...
...
...
...
--
...
...
--
...
...
Brian J. Haas
...
...
Manager, Genome Annotation and Analysis, Research and Development
...
...
The Broad Institute
...
...
http://broad.mit.edu/~bhaas
...
...
<tool id="alignreads" name="alignReads" version="0.0.1">
...
...
...
...
<description>alignReads: short read alignment tool
wrapper</description>
...
...
<requirements>
...
...
<requirement type="package">trinity</requirement>
...
...
</requirements>
...
...
<command>
...
...
...
...
alignReads.pl --target $target -o alignment --aligner
$aligner_selection.aligner
...
...
...
...
...
...
## Inputs.
...
...
#if str($inputs.paired_or_single) == "paired":
...
...
--left $inputs.left_input --right $inputs.right_input
...
...
#if $inputs.left_input.ext == 'fa':
...
...
--seqType fa
...
...
#else:
...
...
--seqType fq
...
...
#end if
...
...
#if str($inputs.library_type) != "None":
...
...
--SS_lib_type $inputs.library_type
...
...
#end if
...
...
--max_dist_between_pairs $inputs.max_dist_between_pairs
...
...
#else:
...
...
--single $inputs.input
...
...
#if str($inputs.input.ext) == 'fa':
...
...
--seqType fa
...
...
#else:
...
...
--seqType fq
...
...
#end if
...
...
#if str($inputs.library_type) != "None":
...
...
--SS_lib_type $inputs.library_type
...
...
#end if
...
...
#end if
...
...
...
...
## Additional parameters.
...
...
##if str($inputs.use_additional) == "yes":
...
...
## -- $inputs.additional_params
...
...
##end if
...
...
...
...
...
...
## direct to output
...
...
## > $trinity_log 2>&1
...
...
...
...
...
...
...
...
</command>
...
...
<inputs>
...
...
<param format="fasta" name="target" type="data" label="target"
help="Fasta sequences targeted for short-read alignment" />
...
...
...
...
<conditional name="inputs">
...
...
<param name="paired_or_single" type="select" label="Paired or
Single-end data?">
...
...
<option value="paired">Paired</option>
...
...
<option value="single">Single</option>
...
...
</param>
...
...
<when value="paired">
...
...
<param format="fasta,fastq" name="left_input" type="data"
label="Left/Forward strand reads" help=""/>
...
...
<param format="fasta,fastq" name="right_input" type="data"
label="Right/Reverse strand reads" help=""/>
...
...
<param name="library_type" type="select" label="Strand-specific
Library Type">
...
...
<option value="None">None</option>
...
...
<option value="FR">FR</option>
...
...
<option value="RF">RF</option>
...
...
</param>
...
...
<param name="max_dist_between_pairs" type="integer" value="2000"
min="1" label="max_dist_between_pairs" help="Maximum length
expected between fragment pairs as aligned to the target,
including introns where relevant."/>
...
...
...
...
...
...
</when>
...
...
<when value="single">
...
...
<param format="fasta,fastq" name="input" type="data"
label="Single-end reads" help=""/>
...
...
<param name="library_type" type="select" label="Strand-specific
Library Type">
...
...
<option value="None">None</option>
...
...
<option value="F">F</option>
...
...
<option value="R">R</option>
...
...
</param>
...
...
</when>
...
...
</conditional>
...
...
...
...
<conditional name="aligner_selection">
...
...
<param name="aligner" type="select" label="Select alignment tool
to
run">
...
...
<option value="bowtie">bowtie</option>
...
...
<option value="bwa">bwa</option>
...
...
<option value="blat">blat</option>
...
...
</param>
...
...
<when value="blat">
...
...
<param name="max_intron_length" type="integer" value="10000" min
=
"1" label="maximum intron length" help="" />
...
...
<param name="min_percent_identity" type="integer" value="95"
min="1" label="minimum percent identity" help="" />
...
...
</when>
...
...
<when value="bwa">
...
...
</when>
...
...
<when value="bowtie">
...
...
</when>
...
...
</conditional>
...
...
...
...
...
...
<!--
...
...
<conditional name="use_additional_params">
...
...
<param name="use_additional" type="select" label="Use Additional
Params?">
...
...
<option value="no">No</option>
...
...
<option value="yes">Yes</option>
...
...
</param>
...
...
<when value="no">
...
...
</when>
...
...
<when value="yes">
...
...
<param name="additional_params" type="text" value=""
label="Additional command-line parameters to aligner" help="" />
...
...
</when>
...
...
</conditional>
...
...
...
...
-->
...
...
...
...
</inputs>

...
...
...
<outputs>
...
...
<data format="bam" name="coordSortedBam" label="${ tool.name } on
${on_string}: coord-sorted read alignments"
from_work_dir="alignment/alignment.coordSorted.bam"/>
...
...
<!-- <data format="bam" name="nameSortedBam" label="${ tool.name
}
on ${on_string}: name-sorted read alignments"
from_work_dir="alignment/alignment.nameSorted.bam"/> -->
...
...
</outputs>
...
...
<tests>
...
...
</tests>
...
...
<help>
...
...
.. _Trinity: http://trinityrnaseq.sourceforge.net
...
...
</help>
...
...
</tool>
...
...
___________________________________________________________
...
...
Please keep all replies on the list by using "reply all"
...
...
in your mail client. To manage your subscriptions to this
...
...
and other Galaxy lists, please use the interface at:
...
...
...
...
http://lists.bx.psu.edu/
...
___________________________________________________________
...
Please keep all replies on the list by using "reply all"
...
in your mail client. To manage your subscriptions to this
...
and other Galaxy lists, please use the interface at:

...
...
http://lists.bx.psu.edu/

...
--
Nicole Rockweiler
Genome Technology Access Center
Washington University in St. Louis
Campus Box 8510
4444 Forest Park Avenue
Saint Louis, MO 63108
...
The contents of this e-mail and any attachments are confidential and
only for use by the intended recipient. Any unauthorized use,
distribution or copying of this message is strictly prohibited. If
you are not the intended recipient please inform the sender
immediately by reply e-mail and delete this message from your
system. Thank you for your co-operation.
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
...
http://lists.bx.psu.edu/

Re: [galaxy-dev] code example for improved error handling

Scott McManus