Galaxy submitting jobs to remote cluster
Hello: I am trying to configure Galaxy to submit jobs to a remote cluster (specifically, an XSEDE resource to which I don't have administrative access). I'm familiar with LWR and have read another response about configuring the Galaxy host as a PBS submit host. What is the best approach to accomplish what I'm after? I have experimented with the cli job runner with the SecureShell and Torque plugins. With some minimal modifications, I can run jobs via this mechanism, but file staging fails. I haven't looked into the details of the staging failure yet. Is this a viable approach? Is the cli job runner supported? (the universe_wsgi.ini file says available job runners include pbs and drmaa) I've found a similar question on the galaxy-dev mailing list, with mention of some relevant recent work: http://dev.list.galaxyproject.org/Remote-Job-Submission-from-Local-Install-t... What is the status of the work referenced in that post? Thanks, Tom Uram
On Jan 7, 2013, at 6:12 PM, Thomas Uram wrote:
Hello:
I am trying to configure Galaxy to submit jobs to a remote cluster (specifically, an XSEDE resource to which I don't have administrative access). I'm familiar with LWR and have read another response about configuring the Galaxy host as a PBS submit host. What is the best approach to accomplish what I'm after?
I have experimented with the cli job runner with the SecureShell and Torque plugins. With some minimal modifications, I can run jobs via this mechanism, but file staging fails. I haven't looked into the details of the staging failure yet. Is this a viable approach? Is the cli job runner supported? (the universe_wsgi.ini file says available job runners include pbs and drmaa)
Hi Tom, I don't have direct experience with LWR, but either it or the cli runner are probably your best bets since they should not require anything you to do things that would require admin privileges on the XSEDE resource. There are some tricky things you will have to do, like make a copy of Galaxy that appears at the same absolute path on the XSEDE resource as it does on the Galaxy server. And as you've found, there is nothing in the provided cli plugins to do staging, so you will have to add that.
I've found a similar question on the galaxy-dev mailing list, with mention of some relevant recent work:
http://dev.list.galaxyproject.org/Remote-Job-Submission-from-Local-Install-t...
What is the status of the work referenced in that post?
This work is still in progress, unfortunately it's been delayed by a number of factors. It is still high on our priority list, however. --nate
Thanks, Tom Uram
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
participants (2)
-
Nate Coraor
-
Thomas Uram