I have installed pbs_python module on our cluster to interface Galaxy with TORQUE. I am able to submit and run jobs on our cluster as the Galaxy user. I need to run Galaxy jobs as the real user. The instruction in the user guide is for DRMAA interface. How can I configure running jobs as real user for TORQUE?
thank,
Ping
Running jobs as the "real" user is not available with the PBS job runner - one has to use the DRMAA interface to submit jobs as the real user.
I have created a Trello card to add this functionality: https://trello.com/c/OddS8bMP
Would be happy to field pull requests to add this - because I doubt anyone on the core team will be able to get to this anytime soon. Sorry I don't have better news.
-John
On Sun, Jul 20, 2014 at 11:26 PM, Ping Luo luop0812@gmail.com wrote:
I have installed pbs_python module on our cluster to interface Galaxy with TORQUE. I am able to submit and run jobs on our cluster as the Galaxy user. I need to run Galaxy jobs as the real user. The instruction in the user guide is for DRMAA interface. How can I configure running jobs as real user for TORQUE?
thank,
Ping
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
On Jul 22, 2014, at 2:13 PM, John Chilton jmchilton@gmail.com wrote:
Running jobs as the "real" user is not available with the PBS job runner - one has to use the DRMAA interface to submit jobs as the real user.
That said, the DRMAA runner plugin is compatible with Torque, you just need to compile and use pbs-drmaa, rather than Torque's libdrmaa:
http://apps.man.poznan.pl/trac/pbs-drmaa
--nate
I have created a Trello card to add this functionality: https://trello.com/c/OddS8bMP
Would be happy to field pull requests to add this - because I doubt anyone on the core team will be able to get to this anytime soon. Sorry I don't have better news.
-John
On Sun, Jul 20, 2014 at 11:26 PM, Ping Luo luop0812@gmail.com wrote:
I have installed pbs_python module on our cluster to interface Galaxy with TORQUE. I am able to submit and run jobs on our cluster as the Galaxy user. I need to run Galaxy jobs as the real user. The instruction in the user guide is for DRMAA interface. How can I configure running jobs as real user for TORQUE?
thank,
Ping
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Nate and John,
I tried with pbs-drmaa and it crashed my galaxy server when the server was started and tried to load the drmaa library . However, I am able to make the drmaa library that comes with TORQUE working with galaxy if sudo is not use.. With sudo, there are still problems I am working to resolve.
Ping
On Thu, Jul 24, 2014 at 10:10 AM, Nate Coraor nate@bx.psu.edu wrote:
On Jul 22, 2014, at 2:13 PM, John Chilton jmchilton@gmail.com wrote:
Running jobs as the "real" user is not available with the PBS job runner - one has to use the DRMAA interface to submit jobs as the real user.
That said, the DRMAA runner plugin is compatible with Torque, you just need to compile and use pbs-drmaa, rather than Torque's libdrmaa:
http://apps.man.poznan.pl/trac/pbs-drmaa--nate
I have created a Trello card to add this functionality: https://trello.com/c/OddS8bMP
Would be happy to field pull requests to add this - because I doubt anyone on the core team will be able to get to this anytime soon. Sorry I don't have better news.
-John
On Sun, Jul 20, 2014 at 11:26 PM, Ping Luo luop0812@gmail.com wrote:
I have installed pbs_python module on our cluster to interface Galaxy with TORQUE. I am able to submit and run jobs on our cluster as the Galaxy user. I need to run Galaxy jobs as the real user. The instruction in the user guide is for DRMAA interface. How can I configure running jobs as real user for TORQUE?
thank,
Ping
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Nate and John,
I managed to make sudo work with galaxy and TORQUE with some fixes:
1. the configuration for external runner/killer/chown cannot be read from ether universe_wsgi.ini or job_conf.xml. I haven't looked into why but just hard coded them into the galaxy source code. 2. drmaa cannot recognize the key word 'arg' and 'email'. Again I did a dirty fix by deleting them in both drmaa.py and drmaa_external_runner.py. 3. 'sudo -E' will rename LD_LIBRARY_PATH to LIBRARY_PATH. So I have to create a wrapper for each of the three scripts (runner/killer/chown) to correct this. 4. The sudo file should now contain the three wrappers. Here I would like to point out an error in the user guide. Even for people who doesn't need the wrappers, the chown script shouldn't not be '/bin/chown' in the sudo file, but the python script /path/to/external_chown_script.py.
After these steps, I am able to run galaxy jobs as the real user. Now I need to see how to fix 1 and 2 in an elegant way.
Ping
On Thu, Jul 24, 2014 at 12:05 PM, Ping Luo luop0812@gmail.com wrote:
Nate and John,
I tried with pbs-drmaa and it crashed my galaxy server when the server was started and tried to load the drmaa library . However, I am able to make the drmaa library that comes with TORQUE working with galaxy if sudo is not use.. With sudo, there are still problems I am working to resolve.
Ping
On Thu, Jul 24, 2014 at 10:10 AM, Nate Coraor nate@bx.psu.edu wrote:
On Jul 22, 2014, at 2:13 PM, John Chilton jmchilton@gmail.com wrote:
Running jobs as the "real" user is not available with the PBS job runner - one has to use the DRMAA interface to submit jobs as the real user.
That said, the DRMAA runner plugin is compatible with Torque, you just need to compile and use pbs-drmaa, rather than Torque's libdrmaa:
http://apps.man.poznan.pl/trac/pbs-drmaa--nate
I have created a Trello card to add this functionality: https://trello.com/c/OddS8bMP
Would be happy to field pull requests to add this - because I doubt anyone on the core team will be able to get to this anytime soon. Sorry I don't have better news.
-John
On Sun, Jul 20, 2014 at 11:26 PM, Ping Luo luop0812@gmail.com wrote:
I have installed pbs_python module on our cluster to interface Galaxy with TORQUE. I am able to submit and run jobs on our cluster as the Galaxy user. I need to run Galaxy jobs as the real user. The instruction in the user guide is for DRMAA interface. How can I configure running jobs as real user for TORQUE?
thank,
Ping
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
galaxy-dev@lists.galaxyproject.org
-
John Chilton -
Nate Coraor -
Ping Luo