Multiple output files do not appear in history
Hey, I'm writing a XML for a tool that produces a number of output files. As the number is not known before the tool is run, I used the strategy outlined here: http://wiki.galaxyproject.org/Admin/Tools/Multiple%20Output%20Files#Number_o... First, I tried my tool definition in a galaxy instance in a virtual machine to not interrupt our production instance (we have a local instance set up). Here, everything worked fine. When trying the same code in our production galaxy instance, it didn't: Regardless of how many output files were produced, only one appears in the history. (the first one, corresponding to "output1" in the references wiki entry). When looking at the temp directory provided to the tool by galaxy ("$__new_file_dir__" in the wiki, it always points to "galaxy-dist/database/tmp/"), the additional output files have been created by my tool and are named correctly (e.g. primary_123_output2_visible_fastq, primary_123_output3_visible_fastq, …) Both instances are new, based on release_2013.06.03. Is there any setting in galaxy somewhere that could prevent this from working? Thanks for your help Chris PS: I wrote a little python script that just creates multiple output files, and the discrepancy between the two instances is reproducible. Here is the code for the python script and the corresponding XML: ####### import subprocess import argparse import os def main(): parser = argparse.ArgumentParser() parser.add_argument('-i', type=int) parser.add_argument('output1') parser.add_argument('output1_id') parser.add_argument('out_dir') args = parser.parse_args() print 'Wrapper arguments:', args with open(args.output1, 'w') as f: f.write('output1') for i in range(2, args.i+1): name = 'output%i'%i file_type = 'text' fname = '%s_%s_%s_%s_%s' % ('primary', args.output1_id, name, 'visible', file_type) with open(os.path.join(args.out_dir, fname), 'w') as f: f.write('output%i'%i) if __name__ == "__main__": main() ############################ ########################### <tool name="MultiOutTest" id="multiouttest"> <description>Multiple Outputs test</description> <command interpreter="python"> multiout.py -i $how_many $output1 $output1.id $__new_file_path__ </command> <inputs> <param name="how_many" type="integer" value="2" label="How many output files?"/> </inputs> <outputs> <data name="output1" format="txt"/> </outputs> <help> This is some help text. </help> </tool> #################################
Hi, I found my problem: in the production instances universe_wsgi.ini the value for "collect_outputs_from" was different that in my VM. It is set to: collect_outputs_from = job_working_directory Perhaps one could update the documentation that deals with multiple output files and mention that the $__new_file_path__ variable is not necessary, it is sufficient to name the output files correctly (i.e. primary_123_output2_visible_fastq,…) in the job work dir. Best, Chris universe_wsgi.ini On Jul 3, 2013, at 10:36 AM, Christoph Malisi <christoph.malisi@computomics.com> wrote:
Hey,
I'm writing a XML for a tool that produces a number of output files. As the number is not known before the tool is run, I used the strategy outlined here: http://wiki.galaxyproject.org/Admin/Tools/Multiple%20Output%20Files#Number_o...
First, I tried my tool definition in a galaxy instance in a virtual machine to not interrupt our production instance (we have a local instance set up). Here, everything worked fine. When trying the same code in our production galaxy instance, it didn't: Regardless of how many output files were produced, only one appears in the history. (the first one, corresponding to "output1" in the references wiki entry). When looking at the temp directory provided to the tool by galaxy ("$__new_file_dir__" in the wiki, it always points to "galaxy-dist/database/tmp/"), the additional output files have been created by my tool and are named correctly (e.g. primary_123_output2_visible_fastq, primary_123_output3_visible_fastq, …)
Both instances are new, based on release_2013.06.03.
Is there any setting in galaxy somewhere that could prevent this from working?
Thanks for your help Chris
PS: I wrote a little python script that just creates multiple output files, and the discrepancy between the two instances is reproducible. Here is the code for the python script and the corresponding XML:
####### import subprocess import argparse import os
def main(): parser = argparse.ArgumentParser()
parser.add_argument('-i', type=int)
parser.add_argument('output1') parser.add_argument('output1_id') parser.add_argument('out_dir') args = parser.parse_args() print 'Wrapper arguments:', args
with open(args.output1, 'w') as f: f.write('output1')
for i in range(2, args.i+1): name = 'output%i'%i file_type = 'text' fname = '%s_%s_%s_%s_%s' % ('primary', args.output1_id, name, 'visible', file_type) with open(os.path.join(args.out_dir, fname), 'w') as f: f.write('output%i'%i)
if __name__ == "__main__": main() ############################ ########################### <tool name="MultiOutTest" id="multiouttest"> <description>Multiple Outputs test</description> <command interpreter="python"> multiout.py -i $how_many $output1 $output1.id $__new_file_path__ </command> <inputs> <param name="how_many" type="integer" value="2" label="How many output files?"/> </inputs> <outputs> <data name="output1" format="txt"/> </outputs> <help> This is some help text. </help> </tool> #################################
Hello, Cristopher Thanks for noticing this. I've updated the wiki entry: http://wiki.galaxyproject.org/Admin/Tools/Multiple%20Output%20Files#Number_o... Does the line starting with 'Alternately,' match the behavior you're seeing? Thanks, Carl On Wed, Jul 3, 2013 at 8:59 AM, Christoph Malisi < christoph.malisi@computomics.com> wrote:
Hi,
I found my problem: in the production instances universe_wsgi.ini the value for "collect_outputs_from" was different that in my VM. It is set to: collect_outputs_from = job_working_directory
Perhaps one could update the documentation that deals with multiple output files and mention that the $__new_file_path__ variable is not necessary, it is sufficient to name the output files correctly (i.e. primary_123_output2_visible_fastq,…) in the job work dir.
Best, Chris
universe_wsgi.ini On Jul 3, 2013, at 10:36 AM, Christoph Malisi < christoph.malisi@computomics.com> wrote:
Hey,
I'm writing a XML for a tool that produces a number of output files. As the number is not known before the tool is run, I used the strategy outlined here:
http://wiki.galaxyproject.org/Admin/Tools/Multiple%20Output%20Files#Number_o...
First, I tried my tool definition in a galaxy instance in a virtual
machine to not interrupt our production instance (we have a local instance set up). Here, everything worked fine. When trying the same code in our production galaxy instance, it didn't: Regardless of how many output files were produced, only one appears in the history. (the first one, corresponding to "output1" in the references wiki entry). When looking at the temp directory provided to the tool by galaxy ("$__new_file_dir__" in the wiki, it always points to "galaxy-dist/database/tmp/"), the additional output files have been created by my tool and are named correctly (e.g. primary_123_output2_visible_fastq, primary_123_output3_visible_fastq, …)
Both instances are new, based on release_2013.06.03.
Is there any setting in galaxy somewhere that could prevent this from
working?
Thanks for your help Chris
PS: I wrote a little python script that just creates multiple output
files, and the discrepancy between the two instances is reproducible. Here is the code for the python script and the corresponding XML:
####### import subprocess import argparse import os
def main(): parser = argparse.ArgumentParser()
parser.add_argument('-i', type=int)
parser.add_argument('output1') parser.add_argument('output1_id') parser.add_argument('out_dir') args = parser.parse_args() print 'Wrapper arguments:', args
with open(args.output1, 'w') as f: f.write('output1')
for i in range(2, args.i+1): name = 'output%i'%i file_type = 'text' fname = '%s_%s_%s_%s_%s' % ('primary', args.output1_id, name,
'visible', file_type)
with open(os.path.join(args.out_dir, fname), 'w') as f: f.write('output%i'%i)
if __name__ == "__main__": main() ############################ ########################### <tool name="MultiOutTest" id="multiouttest"> <description>Multiple Outputs test</description> <command interpreter="python"> multiout.py -i $how_many $output1 $output1.id $__new_file_path__ </command> <inputs> <param name="how_many" type="integer" value="2" label="How many output
files?"/>
</inputs> <outputs> <data name="output1" format="txt"/> </outputs> <help> This is some help text. </help> </tool> #################################
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
participants (2)
-
Carl Eberhard -
Christoph Malisi