Re: [galaxy-dev] routing to a cluster or not on a per-tool basis

7 May 2013

      On Tue, Apr 30, 2013 at 2:04 PM, Dannon Baker <dannon.baker@gmail.com> wrote:
...
Hey Dan,
Sure, you can configure per-tool job runners.  This wiki page
(http://wiki.galaxyproject.org/Admin/Config/Jobs) should get you started,
but let me know if you run into any trouble.
Thanks!

Following up on this.

As I mentioned before, this Galaxy instance was already configured to
use the cluster by default. I did not set up the cluster or the Galaxy
instance. This particular instance did not use the job_conf.xml file.
But I updated it and set up a job_conf.xml and I was able to set it up
so that some tools would be run on the local machine.

However, in trying to get it set up so some jobs would run on the
cluster (which is what happened by default before I came along) I ran
into problems. Previously, it looked like the cluster stuff was
configured in universe_wsgi.ini, and I guessed that these were the
relevant lines:

start_job_runners = drmaa
default_cluster_job_runner = drmaa://-t 12:00 -A noaccount/

So I tried to move that setup to job_conf.xml:

?xml version="1.0"?>
<!-- A sample job config that explicitly configures job running the
way it is configured by default (if there is no explicit config). -->
<job_conf>
    <plugins>
        <plugin id="local" type="runner"
load="galaxy.jobs.runners.local:LocalJobRunner" workers="4"/>
        <plugin id="drmaa" type="runner"
load="galaxy.jobs.runners.drmaa:DRMAAJobRunner"/>
    </plugins>
    <handlers>
        <handler id="main"/>
    </handlers>
    <destinations default="local">
        <destination id="local" runner="local"/>
        <destination id="cluster" runner="drmaa">
            <param id="nativeSpecification">-t 12:00 -A noaccount/</param>
        </destination>
    </destinations>
    <tools>
    <!-- Tools can be configured to use specific destinations or handlers,
         identified by either the "id" or "tags" attribute.  If assigned to
         a tag, a handler or destination that matches that tag will be
         chosen at random.
     -->
        <tool id="mytesttool" destination="cluster"/>
    </tools>
</job_conf>

When I run 'my test tool', however, it just hangs.

Any tips?
Thanks,
Dan
...
On Tue, Apr 30, 2013 at 4:47 PM, Dan Tenenbaum <dtenenba@fhcrc.org> wrote:
...
Hi,
I have some tools that run really quickly without using any kind of
cluster.
I would prefer not to run these tools on a cluster, as the overhead of
submitting these jobs makes them take much longer than they otherwise
would.
I have other tools that are computationally intensive and need to be
run on a cluster.
I would like to expose all these tools in the same Galaxy instance,
but have some tools run on the cluster and others not.
Is this possible?
Thanks,
Dan
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/