Ketan,

Have you taken a look at galaxy's built-in parallelism framework?  For a great current example of a tool using this, look at Peter's NCBI BLAST+ wrappers.  https://github.com/peterjc/galaxy_blast

-Dannon

On Thu, Feb 6, 2014 at 10:32 AM, Ketan Maheshwari <ketancmaheshwari@gmail.com> wrote:

Hi John, Alex, All,

Elaborating on the motivation behind my question of running tools within tool. First, running a tool in parallel at large-scale. For example, if I need to find a pattern from 1000 files via Galaxy Select tool from Text and Filter tool-group, I am limited by providing one file at a time to the tool which will take a long time to finish. Please correct me if there is a more sophisticated way to approach this problem. Second, related concern is running a tool in parallel on one or more HPC resources. 

We want to write a generic wrapper Galaxy tool, powered by Swift parallel framework such that it can run any arbitrary Galaxy tool in parallel on HPC resources. Currently, we have developed this capability but for external executables which is not a most secure way of using Galaxy as I understand from previous conversation.

Having such a wrapper tool in a standard way is desirable so that it preserves the tool contract and binding within Galaxy environment. That is maintaining the history and metadata conventions of Galaxy.

Thanks,

Ketan

On Wed, Feb 5, 2014 at 3:53 PM, John Chilton <chilton@msi.umn.edu> wrote:

Galaxy has an API that is capable of running tools - certainly this is
one path forward on something like this. I am not sure it is the best
path forward though. Probably the best way to enhance Galaxy's
execution capabilities is to extend the Galaxy core framework itself -
this has its own downsides though.

If you can offer more details about how you would like to enhance
Galaxy - what it cannot do that you would like it to do - I or others
may be able to provide more specific ideas. Otherwise, sorry I have
not been or more help.

-John



On Tue, Feb 4, 2014 at 2:51 PM, Ketan Maheshwari <ketan@mcs.anl.gov> wrote:

> Hi,
>
> This is a question I posted to galaxy user mailing list a while back and was
> redirected to dev for possible answers:
> Is it possible in Galaxy to design a tool whose sole purpose is to run other
> tools. This is motivated by our desire to enhance execution capabilities of
> existing tools via a generic tool which acts as a wrapper.
>
> Thanks,
> Ketan
>

> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

--
Ketan


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/