I've configured my local instance of galaxy to run with drmaa. Unfortunately, there is a problem that after some time, galaxy loses its ability to talk to Torque, hence Galaxy thinks long-running jobs have failed and its unable to submit new jobs.
I have no idea how this stuff works underneath so I thought I'd try setting up pbs_python instead, but I get an error when I run the command given onthe Config/Cluster Wiki
(galaxy_env)[galaxy@genomes galaxy-dist]$ LIBTORQUE=/campus/apps/amd64_linux26/packages/torque-2.4.8/lib/libtorque.so python scripts/scramble.py pbs_python fetch_one(): Using existing source, remove to download again:
/home/galaxy/galaxy-dist/scripts/scramble/archives/pbs_python-4.1.0.tar.gz unpack_source(): Removing old build directory at:
/home/galaxy/galaxy-dist/scripts/scramble/build/py2.6-linux-x86_64-ucs2/pbs_python unpack_source(): Unpacked to:
/home/galaxy/galaxy-dist/scripts/scramble/build/py2.6-linux-x86_64-ucs2/pbs_python copy_build_script(): Using build script /home/galaxy/galaxy-dist/scripts/scramble/scripts/pbs_python.py run_scramble_script(): Beginning build run_scramble_script(): Executing in /home/galaxy/galaxy-dist/scripts/scramble/build/py2.6-linux-x86_64-ucs2/pbs_python: /home/galaxy/galaxy_env/bin/python -ES scramble.py Traceback (most recent call last): File "scramble.py", line 1, in <module> import os, sys, shutil ImportError: No module named shutil Traceback (most recent call last): File "scripts/scramble.py", line 44, in <module> egg.scramble() File "/home/galaxy/galaxy-dist/lib/galaxy/eggs/scramble.py", line 58, in scramble self.run_scramble_script() File "/home/galaxy/galaxy-dist/lib/galaxy/eggs/scramble.py", line 207, in run_scramble_script raise ScrambleFailure( self, "%s(): Egg build failed for %s %s" % ( sys._getframe().f_code.co_name, self.name, self.version ) ) galaxy.eggs.scramble.ScrambleFailure: run_scramble_script(): Egg build failed for pbs_python 4.1.0 (galaxy_env)[galaxy@genomes galaxy-dist]$
It looks like it can't find shutil. I'm also running under a virtualenv as indicated by the ProductionServer wiki page, and not sure this makes a difference or not. Does anyone know how to correct this?
Ryan Golhar wrote:
I've configured my local instance of galaxy to run with drmaa. Unfortunately, there is a problem that after some time, galaxy loses its ability to talk to Torque, hence Galaxy thinks long-running jobs have failed and its unable to submit new jobs.
I have no idea how this stuff works underneath so I thought I'd try setting up pbs_python instead, but I get an error when I run the command given onthe Config/Cluster Wiki
(galaxy_env)[galaxy@genomes galaxy-dist]$ LIBTORQUE=/campus/apps/amd64_linux26/packages/torque-2.4.8/lib/libtorque.so python scripts/scramble.py pbs_python
This one should be:
$ LIBTORQUE_DIR=/campus/apps/amd64_linux26/packages/torque-2.4.8/lib python scripts/scramble.py pbs_python
fetch_one(): Using existing source, remove to download again:
/home/galaxy/galaxy-dist/scripts/scramble/archives/pbs_python-4.1.0.tar.gz unpack_source(): Removing old build directory at:
/home/galaxy/galaxy-dist/scripts/scramble/build/py2.6-linux-x86_64-ucs2/pbs_python unpack_source(): Unpacked to:
/home/galaxy/galaxy-dist/scripts/scramble/build/py2.6-linux-x86_64-ucs2/pbs_python copy_build_script(): Using build script /home/galaxy/galaxy-dist/scripts/scramble/scripts/pbs_python.py run_scramble_script(): Beginning build run_scramble_script(): Executing in /home/galaxy/galaxy-dist/scripts/scramble/build/py2.6-linux-x86_64-ucs2/pbs_python: /home/galaxy/galaxy_env/bin/python -ES scramble.py Traceback (most recent call last): File "scramble.py", line 1, in <module> import os, sys, shutil ImportError: No module named shutil Traceback (most recent call last): File "scripts/scramble.py", line 44, in <module> egg.scramble() File "/home/galaxy/galaxy-dist/lib/galaxy/eggs/scramble.py", line 58, in scramble self.run_scramble_script() File "/home/galaxy/galaxy-dist/lib/galaxy/eggs/scramble.py", line 207, in run_scramble_script raise ScrambleFailure( self, "%s(): Egg build failed for %s %s" % ( sys._getframe().f_code.co_name, self.name, self.version ) ) galaxy.eggs.scramble.ScrambleFailure: run_scramble_script(): Egg build failed for pbs_python 4.1.0 (galaxy_env)[galaxy@genomes galaxy-dist]$
It looks like it can't find shutil. I'm also running under a virtualenv as indicated by the ProductionServer wiki page, and not sure this makes a difference or not. Does anyone know how to correct this?
This is a conflict between virtualenv and the '-S' option to Python. I have just committed a fix for this, and it will be available in our stable distribution soon. In the meantime, you can work around this with:
$ cd /home/galaxy/galaxy-dist/scripts/scramble/build/py2.6-linux-x86_64-ucs2/pbs_python $ LIBTORQUE_DIR=/campus/apps/amd64_linux26/packages/torque-2.4.8/lib /home/galaxy/galaxy_env/bin/python scramble.py $ cp dist/pbs_python-4.1.0-py2.6-linux-x86_64-ucs4.egg /home/galaxy/galaxy-dist/eggs
--nate
To manage your subscriptions to this and other Galaxy lists, please use the interface at:
galaxy-dev@lists.galaxyproject.org
-
Nate Coraor -
Ryan Golhar