Dynamic job runner not being dispatched for a tool - galaxy-dev

8 Oct 2013


      Hi List,
I'm a bit baffled by a new tool I've tried to add to Galaxy. The job runner does not dispatch it correctly.
I added the tool via the toolshed, set up the executable on the cluster side, and attempted to hook it up to the dynamic job runner. The destination always ended up set as local.
Works fine if the default runner is pbs, but if it's local by default, it will only dispatch locally.
Attempting to work backwards, I copied the wrapper to a local tool directory, and pared everything down to a minimum to run. There must be something missing. My trinity_all dynamic definition works fine, Newbler doesn't. Is there anything obvious here?
~~~~~tool_conf.xml~~~~~~~~~~~~~~~~~~~~
<?xml version="1.0"?>
<toolbox>
  <section name="Assembly" id="assembly">
    <tool file="sr_assembly/runAssembly.xml" />
    <tool file="ngs_rna/trinity_all.xml" />
  </section>
<!-- I have to have two sections in order for any sections to show up at all -->
  <section name="Send Data" id="send">
    <tool file="data_destination/epigraph.xml" />
    <tool file="data_destination/epigraph_test.xml" />
    <tool file="genomespace/genomespace_exporter.xml" />
  </section>
</toolbox>
~~~~~tools/sr_assembly/runAssembly.xml~~~~~~~~~~~~~~~~~~~~
<tool id="runAssembly" name="runAssembly" version="1.0.1">
<description>De novo assembly of Roche/454 reads using Newbler</description>
<command>
runAssembly $sanger_input -o $newbler_metrics
</command>
<inputs>
    <!-- READSEQ INFILES -->
    <param name="sanger_input" type="data" format="fasta,fastqsanger" label="SE Fasta/Fastq file"/>
    <!-- for walltime control -->
    <param name="walltime" type="select" label="How long will your job need">
          <option selected="true" value="1">8 hrs</option>
          <option value="48">48 hrs</option>
    </param>
  </inputs>
  <outputs>
   <data name="newbler_metrics" format="txt" />
  </outputs>
</tool>
~~~~~job_conf.xml~~~~~~~~~~~~~~~~~~~~
<?xml version="1.0"?>
<job_conf>
  <plugins workers="4">
    <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" />
    <plugin id="pbs" type="runner" load="galaxy.jobs.runners.pbs:PBSJobRunner" />
  </plugins>
  <handlers default="handlers">
    <handler id="handler0" tags="handlers"/>
  </handlers>
  <destinations default="local">
    <destination id="local" runner="local"/>
    <destination id="pbs_normal" runner="pbs" tags="mycluster">
      <param id="Resource_List">nodes=1:ppn=4,vmem=32gb,walltime=24:00:00</param>
    </destination>
    <destination id="newbler_d" runner="dynamic">
      <param id="type">python</param>
      <param id="function">newbler</param>
    </destination>
    <destination id="trinity_d" runner="dynamic">
      <param id="type">python</param>
      <param id="function">trinity_all</param>
    </destination>
  </destinations>
  <tools>
    <tool id="runAssembly" destination="newbler_d"/>
    <tool id="trinity_all" destination="trinity_d"/>
  </tools>
  <limits>
  </limits>
</job_conf>
~~~~~lib/galaxy/jobs/rules/dynamicjobs.py~~~~~~~~~~~~~~~~~~~~
import os
from galaxy.jobs import JobDestination
def newbler(job):
    incoming = dict( [ ( p.name, p.value ) for p in job.parameters ] )
    walltime_para = incoming["walltime"]
    walltime = int(walltime_para.replace(""",""))
    params = {}
    string = "vmem=500gb,nodes=1:ppn=32,walltime=%d:00:00"%walltime
    params["Resource_List"] = string
    return JobDestination(runner="pbs", params=params)
def trinity_all(job):
    incoming = dict( [ ( p.name, p.value ) for p in job.parameters ] )
    walltime_para = incoming["walltime"]
    walltime = int(walltime_para.replace(""",""))
    params = {}
    string = "vmem=500gb,nodes=1:ppn=32,walltime=%d:00:00"%walltime
    params["Resource_List"] = string
    return JobDestination(runner="pbs", params=params)
I re-installed the Roche 454 toolsuite from the toolshed once more to see if it just needed a kick. I copied the tool id into my job_conf file to make sure I made no mistakes on the id. I tried to send it to dynamic job runner and to pbs runner. I pulled and merged this galaxy about a month ago, so it should be mostly up to date - but I've had so many issues that I may have to just re-install from scratch.
Thanks a ton for any advice,
Carrie Ganote