Peter Cock wrote:
On Thu, Jun 2, 2011 at 6:23 PM, Nate Coraor nate@bx.psu.edu wrote:
Peter Cock wrote:
On a related point, I've previously suggested a $variable could be defined for use in tool XML wrappers to set the number of threads. This number could come from a general configuration file, or perhaps via the cluster settings - the point is the tool doesn't need to know, it just gets told how many threads it is allowed.
Yeah, that's the idea.
job_conf.xml:
<?xml version="1.0"?>
<job_conf> <destinations> <destination name="default" runner="local"/> <destination name="pbs_default" runner="pbs"> <native_param name="queue">batch</native_param> </destination> </destinations> <tools> <tool id="upload1" destination="default"/> <tool id="bowtie_wrapper" destination="pbs_default"> <resource type="cores">8</resource> </tool> </tools> </job_conf>
pbs.py then knows to translate '<resource type="cores">8</resource>' to '-l nodes=1:ppn=8'.
Your tool can access that value a bunch, like $__resources__.cores.
The same should be possible for other consumables.
Sounds good. Would there be a global default setting somewhere in universe.ini or elsewhere for when job_conf.xml didn't set a value?
default_cluster_job_runner will remain for backwards compatibility, but we'll ship a sample job_conf.xml that runs everything locally by default.
--nate
Peter