Tools with multiple input files, and multiple output files
Hi all, I'm looking for examples of tools which take multiple input files (one or more, determined at run time) and produce multiple output files (one for each input file). Any specific suggestions? I have a number of sequence filtering/renaming tools where this might be useful - in some cases taking multiple input files and producing a single output is fine, but in general I'd like to know how to preserve a one to one mapping from input files to output files. I realise this may overlap slightly with the work John is doing on dataset collections, but for now I'd like to target the current Galaxy feature set. In some of the simpler cases, if I have N input datasets and want N output files, I can just run the tool N times . This means more steps in the Galaxy GUI, but it isn't very complicated. However, for the current problem I need access to all the inputs at once for setting overall data derived parameters. Regards, Peter
Hey Peter, Have you seen this solution? https://wiki.galaxyproject.org/Admin/Tools/Multiple%20Output%20Files#Number_... It always seems to get mentioned when this topic is brought up. It has serious limitations in terms or workflow running, but it can probably be made to work for individual tool executions. Otherwise I would wait for future feature sets :), maybe other people have some good ideas however. -John On Wed, Feb 19, 2014 at 7:16 AM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
Hi all,
I'm looking for examples of tools which take multiple input files (one or more, determined at run time) and produce multiple output files (one for each input file). Any specific suggestions?
I have a number of sequence filtering/renaming tools where this might be useful - in some cases taking multiple input files and producing a single output is fine, but in general I'd like to know how to preserve a one to one mapping from input files to output files.
I realise this may overlap slightly with the work John is doing on dataset collections, but for now I'd like to target the current Galaxy feature set.
In some of the simpler cases, if I have N input datasets and want N output files, I can just run the tool N times . This means more steps in the Galaxy GUI, but it isn't very complicated.
However, for the current problem I need access to all the inputs at once for setting overall data derived parameters.
Regards,
Peter ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
On Wed, Feb 19, 2014 at 2:16 PM, John Chilton <jmchilton@bx.psu.edu> wrote:
Hey Peter,
Have you seen this solution?
https://wiki.galaxyproject.org/Admin/Tools/Multiple%20Output%20Files#Number_...
It always seems to get mentioned when this topic is brought up. It has serious limitations in terms or workflow running, but it can probably be made to work for individual tool executions.
Hmm. Any real life examples of this in use? I guess looking for the magic string $__new_file_path__ in the <command> tag is the only way to identify this?
Otherwise I would wait for future feature sets :), maybe other people have some good ideas however.
I'll ponder this a bit more then... thanks. Peter
On Wednesday, February 19, 2014, Peter Cock <p.j.a.cock@googlemail.com> wrote:
On Wed, Feb 19, 2014 at 2:16 PM, John Chilton <jmchilton@bx.psu.edu<javascript:;>> wrote:
Hey Peter,
Have you seen this solution?
https://wiki.galaxyproject.org/Admin/Tools/Multiple%20Output%20Files#Number_...
It always seems to get mentioned when this topic is brought up. It has serious limitations in terms or workflow running, but it can probably be made to work for individual tool executions.
Hmm. Any real life examples of this in use? I guess looking for the magic string $__new_file_path__ in the <command> tag is the only way to identify this?
Hi Peter, I used the current functionality here: http://toolshed.g2.bx.psu.edu/view/cjav/split_by_barcode The code for the underlining tool is available here: https://github.com/cjav/ngs-tools Hope it helps, Carlos
FYI: it is my understanding that such tools cannot be used in a workflow at this time. On Feb 19, 2014 10:08 AM, "Carlos Borroto" <carlos.borroto@gmail.com> wrote:
On Wednesday, February 19, 2014, Peter Cock <p.j.a.cock@googlemail.com> wrote:
On Wed, Feb 19, 2014 at 2:16 PM, John Chilton <jmchilton@bx.psu.edu> wrote:
Hey Peter,
Have you seen this solution?
https://wiki.galaxyproject.org/Admin/Tools/Multiple%20Output%20Files#Number_...
It always seems to get mentioned when this topic is brought up. It has serious limitations in terms or workflow running, but it can probably be made to work for individual tool executions.
Hmm. Any real life examples of this in use? I guess looking for the magic string $__new_file_path__ in the <command> tag is the only way to identify this?
Hi Peter,
I used the current functionality here: http://toolshed.g2.bx.psu.edu/view/cjav/split_by_barcode
The code for the underlining tool is available here: https://github.com/cjav/ngs-tools
Hope it helps, Carlos
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
On Feb 19, 2014 10:08 AM, "Carlos Borroto" <carlos.borroto@gmail.com> wrote:
Hi Peter,
I used the current functionality here: http://toolshed.g2.bx.psu.edu/view/cjav/split_by_barcode
The code for the underlining tool is available here: https://github.com/cjav/ngs-tools
Hope it helps, Carlos
Thanks - a non-trivial example to study always helps :) On Wed, Feb 19, 2014 at 3:33 PM, Michael Crusoe <michael.crusoe@gmail.com> wrote:
FYI: it is my understanding that such tools cannot be used in a workflow at this time.
That would be a problem :( Peter
On Wed, Feb 19, 2014 at 11:04 AM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
On Wed, Feb 19, 2014 at 3:33 PM, Michael Crusoe <michael.crusoe@gmail.com> wrote:
FYI: it is my understanding that such tools cannot be used in a workflow at this time.
That would be a problem :(
Well, I haven't checked but I don't see why it would be a problem if your tool is the last one in the workflow. It cannot be an intermediary step as there is no easy way to link the outputs of a this kind of tool to the next tool. --Carlos
participants (4)
-
Carlos Borroto -
John Chilton -
Michael Crusoe -
Peter Cock