Hi Hakeem, you have to _add_ a destination for each number of cores you need, and then specify this destination in the corresponding tools as I said in the previous email, i.e.:
<job_conf> <plugins> <plugin id="drmaa" type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner" workers="4"/> </plugins> <handlers default="handlers"> <!--handler id="main" /--> <handler id="handler0" tags="handlers"/> <!--handler id="handler1" tags="handlers"/--> </handlers> <destinations default="lsf"> <destination id="lsf" runner="drmaa"/> <destination id="lsf_4t" runner="drmaa"> <param id="nativeSpecification">-n 4 -R "span[hosts=1]"</param> </destination> <destination id="lsf_32t" runner="drmaa"> <param id="nativeSpecification">-n 32 -R "span[hosts=1]"</param> </destination> </destinations> <tools> <tool id="spades" destination="lsf_4t"/> <tool id="bwa" destination="lsf_32t"/> </tools> </job_conf>
Cheers, Nicola
Il 04.08.2015 14:12 Hakeem Almabrazi ha scritto:
Nicola,
I have added your suggestion and seems to work J. Is there a way to control these parameters based on a tool? For example, lets say I want to require more cores for certain tools such as bwa. How can I do that?
Here is my job_conf.xml. Please let me know if you have better suggestion to it.
<job_conf> <plugins> <plugin id="drmaa" type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner" workers="4"/> </plugins> <handlers default="handlers"> <!--handler id="main" /--> <handler id="handler0" tags="handlers"/> <!--handler id="handler1" tags="handlers"/--> </handlers> <destinations default="lsf"> <!--destination id="lsf" runner="drmaa"/--> <!-- suggested by Nicola--> <destination id="lsf" runner="drmaa"> <param id="nativeSpecification">-n 32 -R "span[hosts=1]"</param> </destination> </destinations> </job_conf>
Regards,
Hak
FROM: Nicola Soranzo [mailto:nsoranzo@tiscali.it] SENT: Tuesday, August 04, 2015 2:37 PM TO: Hakeem Almabrazi CC: galaxy-dev@lists.galaxyproject.org SUBJECT: RE: [galaxy-dev] LSF cluster wierd behaviours!
Yes, I'm using LSF with Galaxy at TGAC.
Unfortunately I don't have suggestions for your other problems.
Ciao, Nicola
Il 04.08.2015 13:27 Hakeem Almabrazi ha scritto:
Thank you Nicola,
I will play with these options and see how it works.
Have you played with LSF and Galaxy setup?
Thank you,
-----Original Message-----
From: Nicola Soranzo [mailto:nsoranzo@tiscali.it [1]]
Sent: Tuesday, August 04, 2015 2:23 PM
To: Hakeem Almabrazi
Cc: galaxy-dev@lists.galaxyproject.org [2]Subject: Re: [galaxy-dev] LSF cluster wierd behaviours!
Il 04.08.2015 07:57 Hakeem Almabrazi ha scritto:
Hello everyone, I was able to setup galaxy to work with our HPC cluster using the LSF scheduler. So far so good except with few exceptions: ... 3. Finally, how do I control the resources (i.e cores for a job ) given to a submitted job on Galaxy?
Hi Hakeem,
you need to specify new destinations in config/job_conf.xml, for example if you want to submit jobs asking for 4 cores on the same cluster node, use:-q queue_name -n 4 -R "span[hosts=1]"
Then in the section of config/job_conf.xml add
for each tool that should use 4 cores.
More info at:
https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster [3]
https://wiki.galaxyproject.org/Admin/Config/Jobs [4]
Ciao,
Nicola
Connetti gratis il mondo con la nuova indoona: hai la chat, le chiamate, le video chiamate e persino le chiamate di gruppo.
E chiami gratis anche i numeri fissi e mobili nel mondo!
Scarica subito l'app Vai su https://www.indoona.com/ [5]
Disclaimer: This email and its attachments may be confidential and are intended solely for the use of the individual to whom it is addressed. If you are not the intended recipient, any reading, printing, storage, disclosure, copying or any other action taken in respect of this e-mail is prohibited and may be unlawful. If you are not the intended recipient, please notify the sender immediately by using the reply function and then permanently delete what you have received. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Sidra Medical and Research Center.
Connetti gratis il mondo con la nuova indoona: hai la chat, le chiamate, le video chiamate e persino le chiamate di gruppo. E chiami gratis anche i numeri fissi e mobili nel mondo! Scarica subito l'app Vai su https://www.indoona.com/ [6] Disclaimer: This email and its attachments may be confidential and are intended solely for the use of the individual to whom it is addressed. If you are not the intended recipient, any reading, printing, storage, disclosure, copying or any other action taken in respect of this e-mail is prohibited and may be unlawful. If you are not the intended recipient, please notify the sender immediately by using the reply function and then permanently delete what you have received. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Sidra Medical and Research Center.
Links: ------ [1] mailto:nsoranzo@tiscali.it [2] mailto:galaxy-dev@lists.galaxyproject.org [3] https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster [4] https://wiki.galaxyproject.org/Admin/Config/Jobs [5] https://www.indoona.com/ [6] https://www.indoona.com/
Connetti gratis il mondo con la nuova indoona: hai la chat, le chiamate, le video chiamate e persino le chiamate di gruppo. E chiami gratis anche i numeri fissi e mobili nel mondo! Scarica subito l’app Vai su https://www.indoona.com/