DRMAA options for SGE
Hi, I am trying to configure the proper memory resource requests for my Galaxy tool. This is what I have under the tool_runners section of universe_wsgi.ini [galaxy:tool_runners] ... mytoolname = drmaa://-l mem_free=1G -l mem_token=1G -l h_vmem=1G/ ... When I execute my tool on Galaxy, I get the error below in the shell that I ran "sh run.sh": galaxy.jobs.runners.drmaa ERROR 2011-07-26 09:54:01,930 Uncaught exception queueing job Traceback (most recent call last): File "/home/kmnip/galaxy/galaxy-central/lib/galaxy/jobs/runners/drmaa.py", line 112, in run_next self.queue_job( obj ) File "/home/kmnip/galaxy/galaxy-central/lib/galaxy/jobs/runners/drmaa.py", line 177, in queue_job job_id = self.ds.runJob(jt) File "/home/kmnip/galaxy/galaxy-central/eggs/drmaa-0.4b3-py2.4.egg/drmaa/__init__.py", line 331, in runJob File "/home/kmnip/galaxy/galaxy-central/eggs/drmaa-0.4b3-py2.4.egg/drmaa/helpers.py", line 213, in c File "/home/kmnip/galaxy/galaxy-central/eggs/drmaa-0.4b3-py2.4.egg/drmaa/errors.py", line 90, in error_check DeniedByDrmException: code 17: error: no suitable queues All the flags I used work with qsub commands on the SGE cluster I use. The tool runs when I comment out the line in universe_wsgi.ini. Thanks, Ka Ming
Answering myself here... I don't see the error anymore after adding "-w n": [galaxy:tool_runners] ... mytoolname = drmaa://-w n -l mem_free=1G -l mem_token=1G -l h_vmem=1G/ ... Is this what other Galaxy admins do when using drmaa native options for their SGE? Ka Ming ________________________________________ From: galaxy-dev-bounces@lists.bx.psu.edu [galaxy-dev-bounces@lists.bx.psu.edu] On Behalf Of Ka Ming Nip [kmnip@bcgsc.ca] Sent: July 26, 2011 10:39 AM To: galaxy-dev@lists.bx.psu.edu Subject: [galaxy-dev] DRMAA options for SGE Hi, I am trying to configure the proper memory resource requests for my Galaxy tool. This is what I have under the tool_runners section of universe_wsgi.ini [galaxy:tool_runners] ... mytoolname = drmaa://-l mem_free=1G -l mem_token=1G -l h_vmem=1G/ ... When I execute my tool on Galaxy, I get the error below in the shell that I ran "sh run.sh": galaxy.jobs.runners.drmaa ERROR 2011-07-26 09:54:01,930 Uncaught exception queueing job Traceback (most recent call last): File "/home/kmnip/galaxy/galaxy-central/lib/galaxy/jobs/runners/drmaa.py", line 112, in run_next self.queue_job( obj ) File "/home/kmnip/galaxy/galaxy-central/lib/galaxy/jobs/runners/drmaa.py", line 177, in queue_job job_id = self.ds.runJob(jt) File "/home/kmnip/galaxy/galaxy-central/eggs/drmaa-0.4b3-py2.4.egg/drmaa/__init__.py", line 331, in runJob File "/home/kmnip/galaxy/galaxy-central/eggs/drmaa-0.4b3-py2.4.egg/drmaa/helpers.py", line 213, in c File "/home/kmnip/galaxy/galaxy-central/eggs/drmaa-0.4b3-py2.4.egg/drmaa/errors.py", line 90, in error_check DeniedByDrmException: code 17: error: no suitable queues All the flags I used work with qsub commands on the SGE cluster I use. The tool runs when I comment out the line in universe_wsgi.ini. Thanks, Ka Ming ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Hi Ming, just an idea, "-w n" might be hiding the error reporting. Is your jobs getting submitted and executed correctly? For the queuing , you can add "galaxy" at the end, makes it drmaa://-w n -l mem_free=1G -l mem_token=1G -l h_vmem=1G/galaxy here galaxy is queue name. With Regards, ---------------- Ambarish Biswas, University of Otago Department of Biochemistry, Dunedin, New Zealand, Tel: +64(22)0855647 Fax: +64(0)3 479 7866 On Thu, Jul 28, 2011 at 6:21 AM, Ka Ming Nip <kmnip@bcgsc.ca> wrote:
Answering myself here...
I don't see the error anymore after adding "-w n":
[galaxy:tool_runners] ... mytoolname = drmaa://-w n -l mem_free=1G -l mem_token=1G -l h_vmem=1G/ ...
Is this what other Galaxy admins do when using drmaa native options for their SGE?
Ka Ming ________________________________________ From: galaxy-dev-bounces@lists.bx.psu.edu [ galaxy-dev-bounces@lists.bx.psu.edu] On Behalf Of Ka Ming Nip [ kmnip@bcgsc.ca] Sent: July 26, 2011 10:39 AM To: galaxy-dev@lists.bx.psu.edu Subject: [galaxy-dev] DRMAA options for SGE
Hi,
I am trying to configure the proper memory resource requests for my Galaxy tool. This is what I have under the tool_runners section of universe_wsgi.ini
[galaxy:tool_runners] ... mytoolname = drmaa://-l mem_free=1G -l mem_token=1G -l h_vmem=1G/ ...
When I execute my tool on Galaxy, I get the error below in the shell that I ran "sh run.sh":
galaxy.jobs.runners.drmaa ERROR 2011-07-26 09:54:01,930 Uncaught exception queueing job Traceback (most recent call last): File "/home/kmnip/galaxy/galaxy-central/lib/galaxy/jobs/runners/drmaa.py", line 112, in run_next self.queue_job( obj ) File "/home/kmnip/galaxy/galaxy-central/lib/galaxy/jobs/runners/drmaa.py", line 177, in queue_job job_id = self.ds.runJob(jt) File "/home/kmnip/galaxy/galaxy-central/eggs/drmaa-0.4b3-py2.4.egg/drmaa/__init__.py", line 331, in runJob File "/home/kmnip/galaxy/galaxy-central/eggs/drmaa-0.4b3-py2.4.egg/drmaa/helpers.py", line 213, in c File "/home/kmnip/galaxy/galaxy-central/eggs/drmaa-0.4b3-py2.4.egg/drmaa/errors.py", line 90, in error_check DeniedByDrmException: code 17: error: no suitable queues
All the flags I used work with qsub commands on the SGE cluster I use. The tool runs when I comment out the line in universe_wsgi.ini.
Thanks, Ka Ming ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Hi Ambarish, Using what I had in my previous message: mytoolname = drmaa://-w n -l mem_free=1G -l mem_token=1G -l h_vmem=1G/ my jobs do get submitted to the cluster and I see the other options from qstat of the jobs ie. hard resource_list: mem_free=1G,mem_token=1G,h_vmem=1G Ka Ming ________________________________________ From: ambarish biswas [ambarishbiswas@gmail.com] Sent: July 27, 2011 4:18 PM To: Ka Ming Nip Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] DRMAA options for SGE Hi Ming, just an idea, "-w n" might be hiding the error reporting. Is your jobs getting submitted and executed correctly? For the queuing , you can add "galaxy" at the end, makes it drmaa://-w n -l mem_free=1G -l mem_token=1G -l h_vmem=1G/galaxy here galaxy is queue name. With Regards, ---------------- Ambarish Biswas, University of Otago Department of Biochemistry, Dunedin, New Zealand, Tel: +64(22)0855647 Fax: +64(0)3 479 7866 On Thu, Jul 28, 2011 at 6:21 AM, Ka Ming Nip <kmnip@bcgsc.ca<mailto:kmnip@bcgsc.ca>> wrote: Answering myself here... I don't see the error anymore after adding "-w n": [galaxy:tool_runners] ... mytoolname = drmaa://-w n -l mem_free=1G -l mem_token=1G -l h_vmem=1G/ ... Is this what other Galaxy admins do when using drmaa native options for their SGE? Ka Ming ________________________________________ From: galaxy-dev-bounces@lists.bx.psu.edu<mailto:galaxy-dev-bounces@lists.bx.psu.edu> [galaxy-dev-bounces@lists.bx.psu.edu<mailto:galaxy-dev-bounces@lists.bx.psu.edu>] On Behalf Of Ka Ming Nip [kmnip@bcgsc.ca<mailto:kmnip@bcgsc.ca>] Sent: July 26, 2011 10:39 AM To: galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu> Subject: [galaxy-dev] DRMAA options for SGE Hi, I am trying to configure the proper memory resource requests for my Galaxy tool. This is what I have under the tool_runners section of universe_wsgi.ini [galaxy:tool_runners] ... mytoolname = drmaa://-l mem_free=1G -l mem_token=1G -l h_vmem=1G/ ... When I execute my tool on Galaxy, I get the error below in the shell that I ran "sh run.sh": galaxy.jobs.runners.drmaa ERROR 2011-07-26 09:54:01,930 Uncaught exception queueing job Traceback (most recent call last): File "/home/kmnip/galaxy/galaxy-central/lib/galaxy/jobs/runners/drmaa.py", line 112, in run_next self.queue_job( obj ) File "/home/kmnip/galaxy/galaxy-central/lib/galaxy/jobs/runners/drmaa.py", line 177, in queue_job job_id = self.ds.runJob(jt) File "/home/kmnip/galaxy/galaxy-central/eggs/drmaa-0.4b3-py2.4.egg/drmaa/__init__.py", line 331, in runJob File "/home/kmnip/galaxy/galaxy-central/eggs/drmaa-0.4b3-py2.4.egg/drmaa/helpers.py", line 213, in c File "/home/kmnip/galaxy/galaxy-central/eggs/drmaa-0.4b3-py2.4.egg/drmaa/errors.py", line 90, in error_check DeniedByDrmException: code 17: error: no suitable queues All the flags I used work with qsub commands on the SGE cluster I use. The tool runs when I comment out the line in universe_wsgi.ini. Thanks, Ka Ming ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Hi, this is what is working for me, the the "Output not returned from cluster" error is also disappeared. default_cluster_job_runner = drmaa://-q galaxy -V/ if this works, then check your other options to test. With Regards, ---------------- Ambarish Biswas, University of Otago Department of Biochemistry, Dunedin, New Zealand, Tel: +64(22)0855647 Fax: +64(0)3 479 7866 On Fri, Jul 29, 2011 at 6:32 AM, Ka Ming Nip <kmnip@bcgsc.ca> wrote:
Hi Ambarish,
Using what I had in my previous message: mytoolname = drmaa://-w n -l mem_free=1G -l mem_token=1G -l h_vmem=1G/ my jobs do get submitted to the cluster and I see the other options from qstat of the jobs ie. hard resource_list: mem_free=1G,mem_token=1G,h_vmem=1G
Ka Ming ________________________________________ From: ambarish biswas [ambarishbiswas@gmail.com] Sent: July 27, 2011 4:18 PM To: Ka Ming Nip Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] DRMAA options for SGE
Hi Ming, just an idea, "-w n" might be hiding the error reporting. Is your jobs getting submitted and executed correctly?
For the queuing , you can add "galaxy" at the end, makes it
drmaa://-w n -l mem_free=1G -l mem_token=1G -l h_vmem=1G/galaxy
here galaxy is queue name.
With Regards, ---------------- Ambarish Biswas, University of Otago Department of Biochemistry, Dunedin, New Zealand, Tel: +64(22)0855647 Fax: +64(0)3 479 7866
On Thu, Jul 28, 2011 at 6:21 AM, Ka Ming Nip <kmnip@bcgsc.ca<mailto: kmnip@bcgsc.ca>> wrote: Answering myself here...
I don't see the error anymore after adding "-w n":
[galaxy:tool_runners] ... mytoolname = drmaa://-w n -l mem_free=1G -l mem_token=1G -l h_vmem=1G/ ...
Is this what other Galaxy admins do when using drmaa native options for their SGE?
Ka Ming ________________________________________ From: galaxy-dev-bounces@lists.bx.psu.edu<mailto: galaxy-dev-bounces@lists.bx.psu.edu> [galaxy-dev-bounces@lists.bx.psu.edu <mailto:galaxy-dev-bounces@lists.bx.psu.edu>] On Behalf Of Ka Ming Nip [ kmnip@bcgsc.ca<mailto:kmnip@bcgsc.ca>] Sent: July 26, 2011 10:39 AM To: galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu> Subject: [galaxy-dev] DRMAA options for SGE
Hi,
I am trying to configure the proper memory resource requests for my Galaxy tool. This is what I have under the tool_runners section of universe_wsgi.ini
[galaxy:tool_runners] ... mytoolname = drmaa://-l mem_free=1G -l mem_token=1G -l h_vmem=1G/ ...
When I execute my tool on Galaxy, I get the error below in the shell that I ran "sh run.sh":
galaxy.jobs.runners.drmaa ERROR 2011-07-26 09:54:01,930 Uncaught exception queueing job Traceback (most recent call last): File "/home/kmnip/galaxy/galaxy-central/lib/galaxy/jobs/runners/drmaa.py", line 112, in run_next self.queue_job( obj ) File "/home/kmnip/galaxy/galaxy-central/lib/galaxy/jobs/runners/drmaa.py", line 177, in queue_job job_id = self.ds.runJob(jt) File "/home/kmnip/galaxy/galaxy-central/eggs/drmaa-0.4b3-py2.4.egg/drmaa/__init__.py", line 331, in runJob File "/home/kmnip/galaxy/galaxy-central/eggs/drmaa-0.4b3-py2.4.egg/drmaa/helpers.py", line 213, in c File "/home/kmnip/galaxy/galaxy-central/eggs/drmaa-0.4b3-py2.4.egg/drmaa/errors.py", line 90, in error_check DeniedByDrmException: code 17: error: no suitable queues
All the flags I used work with qsub commands on the SGE cluster I use. The tool runs when I comment out the line in universe_wsgi.ini.
Thanks, Ka Ming ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
participants (2)
-
ambarish biswas -
Ka Ming Nip