Galaxy and Cluster options
Hi! Is there a way to specify parameters for the job submission on the cluster from the tool configuration file? Let's say I want tool A to be submitted with "-pe smp 8" but tool B submitted with "-l mem_free=500". Is it already supported? I could not find it in the wiki. ... or do I have to start digging in the code and try to implement it?! ^_^' Best regards, Tanguy
Hi Tanguy, You can set that at the very bottom of your universe_wsgi.ini file. I did it myself with Torque to set a different behavior for a couple of tools, it works fine. The related Wiki page is here: http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Cluster :) Best, L-A Le 17/02/2012 09:43, Tanguy LE CARROUR a écrit :
Hi!
Is there a way to specify parameters for the job submission on the cluster from the tool configuration file?
Let's say I want tool A to be submitted with "-pe smp 8" but tool B submitted with "-l mem_free=500".
Is it already supported? I could not find it in the wiki. ... or do I have to start digging in the code and try to implement it?! ^_^'
Best regards, Tanguy
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Hi Tanguy, L-A is correct in that you can set using the tool id different cluster options for different tools. For example I submit different tools to different queues: default_cluster_job_runner = drmaa://-q all.q -V/ upload1 = drmaa://-q short.q -V/ tophat = drmaa://-q long.q -V/ cufflinks = drmaa://-q long.q -V/ But I don't think this is what you want, as this is an one time setting that you can't change at runtime. I think there was some talk about dynamic DRMAA options base on the input size and tool, but can't tell what is the status on that feature. Hope it helps, Carlos 2012/2/17 Louise-Amélie Schmitt <louise-amelie.schmitt@embl.de>:
Hi Tanguy,
You can set that at the very bottom of your universe_wsgi.ini file. I did it myself with Torque to set a different behavior for a couple of tools, it works fine. The related Wiki page is here: http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Cluster :)
Best, L-A
Le 17/02/2012 09:43, Tanguy LE CARROUR a écrit :
Hi!
Is there a way to specify parameters for the job submission on the cluster from the tool configuration file?
Let's say I want tool A to be submitted with "-pe smp 8" but tool B submitted with "-l mem_free=500".
Is it already supported? I could not find it in the wiki. ... or do I have to start digging in the code and try to implement it?! ^_^'
Best regards, Tanguy
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
This was recently brought up in a related thread by Nate, the code is in a pull request on bitbucket. Changing the subject to see if that rings any additional bells… Here's the original post by John Chilton and link to the fork: http://galaxy-development-list-archive.2308389.n4.nabble.com/Defining-Job-Ru... https://bitbucket.org/galaxy/galaxy-central/pull-request/12/dynamic-job-runn... chris On Feb 17, 2012, at 9:28 AM, Carlos Borroto wrote:
Hi Tanguy,
L-A is correct in that you can set using the tool id different cluster options for different tools. For example I submit different tools to different queues: default_cluster_job_runner = drmaa://-q all.q -V/ upload1 = drmaa://-q short.q -V/ tophat = drmaa://-q long.q -V/ cufflinks = drmaa://-q long.q -V/
But I don't think this is what you want, as this is an one time setting that you can't change at runtime. I think there was some talk about dynamic DRMAA options base on the input size and tool, but can't tell what is the status on that feature.
Hope it helps, Carlos
2012/2/17 Louise-Amélie Schmitt <louise-amelie.schmitt@embl.de>:
Hi Tanguy,
You can set that at the very bottom of your universe_wsgi.ini file. I did it myself with Torque to set a different behavior for a couple of tools, it works fine. The related Wiki page is here: http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Cluster :)
Best, L-A
Le 17/02/2012 09:43, Tanguy LE CARROUR a écrit :
Hi!
Is there a way to specify parameters for the job submission on the cluster from the tool configuration file?
Let's say I want tool A to be submitted with "-pe smp 8" but tool B submitted with "-l mem_free=500".
Is it already supported? I could not find it in the wiki. ... or do I have to start digging in the code and try to implement it?! ^_^'
Best regards, Tanguy
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
participants (4)
-
Carlos Borroto -
Fields, Christopher J -
Louise-Amélie Schmitt -
Tanguy LE CARROUR