Re: [galaxy-user] suggestion for multithreading
Peter Cock wrote:
2011/6/2 Nate Coraor <nate(a)bx.psu.edu>:
> Hi Louise-Amélie,
>
> I haven't done anything with this code yet, but I wanted to let you know
> that we'll eventually be adding it, I'm just going to change the
> implementation slightly. I'd like to merge the functionality of the csv
> into an xml config I'm already working on (but haven't yet fully decided
> on the syntax). And it should be possible for tools to access these
> parameters in the command line template. A lot of our NGS tools have
> the number of threads to use hardcoded in the tool config, which is bad.
>
> --nate
On a related point, I've previously suggested a $variable could be defined
for use in tool XML wrappers to set the number of threads. This number
could come from a general configuration file, or perhaps via the cluster
settings - the point is the tool doesn't need to know, it just gets told how
many threads it is allowed.
Yeah, that's the idea.
job_conf.xml:
<?xml version="1.0"?>
<job_conf>
<destinations>
<destination name="default" runner="local"/>
<destination name="pbs_default" runner="pbs">
<native_param name="queue">batch</native_param>
</destination>
</destinations>
<tools>
<tool id="upload1" destination="default"/>
<tool id="bowtie_wrapper" destination="pbs_default">
<resource type="cores">8</resource>
</tool>
</tools>
</job_conf>
pbs.py then knows to translate '<resource
type="cores">8</resource>' to
'-l nodes=1:ppn=8'.
Your tool can access that value a bunch, like $__resources__.cores.
The same should be possible for other consumables.
--nate
Peter