I am not sure I understand what you mean, but if you are setting the job to run with a ppn of greater than one but GALAXY_SLOTS is being evaluated as 1 then you have likely uncovered a bug - maybe in the runner, your job configuration, or the GALAXY_SLOTS logic for pbs/torque. I have created a Galaxy tool for debugging the problem - https://gist.github.com/jmchilton/9548516. If you could run in your environment and assign it multiple cores it should spit out some interesting information relevant to debugging the problem. It will create Galaxy datasets corresponding to the runtime-computed Galaxy slots, the contents of PBS_NODEFILE which Galaxy attempts to use to compute Galaxy slots, and the full contents of your worker node environment. This last one may contain sensitive information - but if you could post the first two or send them to me directly it would hopefully help debug the problem. -John On Wed, Mar 12, 2014 at 12:37 PM, Geert Vandeweyer <geert.vandeweyer2@uantwerpen.be> wrote:
How would those statements translate to pbs/torque (pbs_python)
I request resources using the nodes=1:ppn:4 syntax. The -pe argument is not available it seems...
Best,
Geert
On 03/12/2014 04:43 PM, Björn Grüning wrote:
Hi Geert,
you can give every tool a different amount of SLOTS via the job_conf.xml file.
https://wiki.galaxyproject.org/Admin/Config/Jobs
For example:
<tool id="toolshed.g2.bx.psu.edu/repos/iuc/gatk2/gatk2_variant_recalibrator" destination="12cores_24G" />
where 12cores_24G is defined with a parallel SGE environment.
Cheers, Bjoern
Am 12.03.2014 16:32, schrieb Geert Vandeweyer:
Hi,
Is there documentation for the proper setup and utilisation of the \${GALAXY_SLOTS:-4} style options?
I noticed that tools with this setting, including BWA and GATK run single threaded, with the following settings respectively:
BWA \${GALAXY_SLOTS:-4}" === ps output: python
/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/bwa_wrappers/b4427dbb6ced/bwa_wrappers/bwa_wrapper.py --threads=1 --fileSource=indexed --ref=/galaxy/galaxy_references/hg19/bwa-0.5.9/base/hg19.fasta --do_not_build_index --input1=/galaxy/galaxy-dist/database/files/093/dataset_93021.dat --input2=/galaxy/galaxy-dist/database/files/093/dataset_93023.dat --output=/galaxy/galaxy-dist/database/files/000/112/dataset_112301.dat --genAlignType=paired --params=pre_set --suppressHeader=false
xml config: --threads="\${GALAXY_SLOTS:-4}"
GATK Base Recalibrator \${GALAXY_SLOTS:-8} ============== python
/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/iuc/gatk2/8bcc13094767/gatk2/gatk2_wrapper.py --max_jvm_heap 6g --stdout /galaxy/galaxy-dist/database/files/000/112/dataset_112292.dat -d -I /galaxy/galaxy-dist/database/files/000/112/dataset_112289.dat bam gatk_input -d /galaxy/galaxy-dist/database/files/_metadata_files/012/metadata_12686.dat bam_index gatk_input -p java -jar "$GATK2_PATH/GenomeAnalysisTK.jar" -T "BaseRecalibrator" $GATK2_SITE_OPTIONS --num_cpu_threads_per_data_thread ${GALAXY_SLOTS:-8} --no_standard_covs -R "/galaxy/galaxy_references/hg19/srma-0.1.15/hg19.fasta" --out "/galaxy/galaxy-dist/database/files/000/112/dataset_112291.dat" -cov "ContextCovariate" -cov "CycleCovariate" -d --knownSites:dbsnp,%(file_type)s /galaxy/galaxy-dist/database/files/000/101/dataset_101086.dat vcf input_dbsnp_0 -p --pedigreeValidationType "STRICT" -d --intervals /galaxy/galaxy-dist/database/files/000/100/dataset_100129.dat bed input_intervals_0 -p --interval_set_rule "UNION" -p --downsampling_type "NONE" -p --baq "OFF" --baqGapOpenPenalty "40.0" --defaultBaseQualities "-1" --validation_strictness "STRICT" --interval_merging "ALL"
java -Xmx6g -jar /galaxy/galaxy-dist/tool-data/shared/jars/gatk2//GenomeAnalysisTK.jar -T BaseRecalibrator --num_cpu_threads_per_data_thread 1 --no_standard_covs -R /galaxy/galaxy_references/hg19/srma-0.1.15/hg19.fasta --out /galaxy/galaxy-dist/database/files/000/112/dataset_112291.dat -cov ContextCovariate -cov CycleCovariate --pedigreeValidationType STRICT --interval_set_rule UNION --downsampling_type NONE --baq OFF --baqGapOpenPenalty 40.0 --defaultBaseQualities -1 --validation_strictness STRICT --interval_merging ALL -I /tmp/tmp-gatk-fIuncH/gatk_input.bam --knownSites:dbsnp,vcf /tmp/tmp-gatk-fIuncH/input_dbsnp_0.vcf --intervals /tmp/tmp-gatk-fIuncH/input_intervals_0.bed
Best,
Geert
--
Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandeweyer@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/