dear colleagues,
at the university of oslo, we develop a galaxy-based portal for natural language processing (LAP: Language Analysis Portal). jobs are submitted to a compute cluster via DRMAA and SLURM. current development is against the galaxy release of march 2015.
i am wondering about fine-grained control of job resources. our goal is that most users need not look past the ‘Use default job resource parameters’ toggle in the job configuration dialogue.
as i understand it, i think we can populate the ‘nativeSpecification’ parameter in ‘job_conf.xml’ with SLURM-specific command-line options to set defaults, for example the project, maximum run-time, number of cores, memory per core, and such. i assume these defaults will be combined with and overwritten by ‘custom’ job resource parameters, in case any are specified in the job configuration dialogue?
i tried to track the flow of information from ‘lib/galaxy/jobs/runners/drmaa.py’ via ‘scripts/drmaa_external_runner.py’ into the drmaa-python egg, but i could not easily work out where the merging of ‘nativeSpecification’ and custom resource parameters happens; presumably at about the same time as an actual job script file is created, for submission to SLURM? could someone point me in the right direction here?
—more importantly, maybe: we would like to establish per-tool resource defaults. for example, some of our tools require substantially more memory per core than others. i cannot easily find a way of associating resource default with individual tools. i looked at the tool configuration syntax, ‘job_conf.xml.sample_advanced’, and ‘job_resource_params_conf.xml.sample’, as well as at the following documentation pages:
https://wiki.galaxyproject.org/Admin/Config/Jobs https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster
i am hoping i am overlooking something :-). is there a way to define job resource defaults on a per-tool basis?
with warmest thanks in advance, oe