On Dec 29, 2011, at 4:29 PM, Ryan Golhar wrote:
Hi - I've just finished installing a ROCKS 5.4.3 cluster and am now adding Galaxy to it. We are using torque instead of SGE. I see the Wiki docs at http://wiki.g2.bx.psu.edu/Admin/Config/Performance/Cluster state I need to scramble pbs_python by doing:
LIBTORQUE_DIR=/path/to/libtorque python scripts/scramble.py -e pbs_python
I have libtorque installed in /usr/lib64:
[galaxy@bic x86_64]$ ll /usr/lib64/libtorque.* -rw-r--r-- 1 root root 536862 Dec 28 12:32 /usr/lib64/libtorque.a -rwxr-xr-x 1 root root 810 Dec 28 12:32 /usr/lib64/libtorque.la lrwxrwxrwx 1 root root 18 Dec 28 13:50 /usr/lib64/libtorque.so -> libtorque.so.2.0.0 lrwxrwxrwx 1 root root 14 Dec 28 13:50 /usr/lib64/libtorque.so.0 -> libtorque.so.2 lrwxrwxrwx 1 root root 18 Dec 28 13:50 /usr/lib64/libtorque.so.2 -> libtorque.so.2.0.0 -rwxr-xr-x 1 root root 766431 Dec 28 12:32 /usr/lib64/libtorque.so.2.0.0
So I then run (and get the error shown at the bottom): [galaxy@bic x86_64]$ cd [galaxy@bic ~]$ cd galaxy-dist/ [galaxy@bic galaxy-dist]$ LIBTORQUE_DIR=/usr/lib64/libtorque.so.2.0.0 python scripts/scramble.py -e pbs_python
Hi Ryan,
$LIBTORQUE_DIR should be the path to the directory containing libtorque, so try:
[galaxy@bic galaxy-dist]$ LIBTORQUE_DIR=/usr/lib64 python scripts/scramble.py -e pbs_python
Although it looks like the build (below) succeeded anyway since libtorque.so is in a system lib directory.
fetch_one(): Trying to fetch: http://eggs.g2.bx.psu.edu/pbs_python/pbs_python-4.1.0.tar.gz fetch_one(): Fetched to: /home/galaxy/galaxy-dist/scripts/scramble/archives/pbs_python-4.1.0.tar.gz unpack_source(): Unpacked to: /home/galaxy/galaxy-dist/scripts/scramble/build/py2.6-linux-x86_64-ucs2/pbs_python copy_build_script(): Using build script /home/galaxy/galaxy-dist/scripts/scramble/scripts/pbs_python.py run_scramble_script(): Beginning build run_scramble_script(): Executing in /home/galaxy/galaxy-dist/scripts/scramble/build/py2.6-linux-x86_64-ucs2/pbs_python: /share/apps/Python-2.6.7/bin/python scramble.py
This script requires setuptools version 0.6c11 to run (even to display help). I will attempt to download it for you (from http://pypi.python.org/packages/2.6/s/setuptools/), but you may need to enable firewall access for this script first. I will start the download in 8 seconds.
(Note: if this machine does not have network access, please obtain the file
http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c11-py2.6.egg
and place it in this directory before rerunning this script.)
Downloading http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c11-py2.6.egg checking for pbs-config... /usr/bin/pbs-config Found torque version: 2.5.9 checking for python... /share/apps/Python-2.6.7/bin/python checking for python version... 2.6 checking for python platform... linux2 checking for python script directory... ${prefix}/lib/python2.6/site-packages checking for python extension module directory... ${exec_prefix}/lib/python2.6/site-packages configure: creating ./config.status config.status: creating Makefile config.status: creating setup.py scramble(): Patching setup.py running egg_info creating src/pbs_python.egg-info writing src/pbs_python.egg-info/PKG-INFO writing top-level names to src/pbs_python.egg-info/top_level.txt writing dependency_links to src/pbs_python.egg-info/dependency_links.txt writing manifest file 'src/pbs_python.egg-info/SOURCES.txt' reading manifest file 'src/pbs_python.egg-info/SOURCES.txt' writing manifest file 'src/pbs_python.egg-info/SOURCES.txt' running bdist_egg installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py creating build creating build/lib.linux-x86_64-2.6 copying src/pbs.py -> build/lib.linux-x86_64-2.6 copying src/PBSQuery.py -> build/lib.linux-x86_64-2.6 running build_ext building '_pbs' extension creating build/temp.linux-x86_64-2.6 creating build/temp.linux-x86_64-2.6/src gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DTORQUE_2_4 -I/share/apps/Python-2.6.7/include/python2.6 -c src/pbs_wrap.c -o build/temp.linux-x86_64-2.6/src/pbs_wrap.o src/pbs_wrap.c:2622: warning: function declaration isn't a prototype src/pbs_wrap.c:2630: warning: function declaration isn't a prototype gcc -pthread -shared build/temp.linux-x86_64-2.6/src/pbs_wrap.o -L/usr/lib64 -ltorque -o build/lib.linux-x86_64-2.6/_pbs.so creating build/bdist.linux-x86_64 creating build/bdist.linux-x86_64/egg copying build/lib.linux-x86_64-2.6/pbs.py -> build/bdist.linux-x86_64/egg copying build/lib.linux-x86_64-2.6/PBSQuery.py -> build/bdist.linux-x86_64/egg copying build/lib.linux-x86_64-2.6/_pbs.so -> build/bdist.linux-x86_64/egg byte-compiling build/bdist.linux-x86_64/egg/pbs.py to pbs.pyc byte-compiling build/bdist.linux-x86_64/egg/PBSQuery.py to PBSQuery.pyc creating stub loader for _pbs.so byte-compiling build/bdist.linux-x86_64/egg/_pbs.py to _pbs.pyc creating build/bdist.linux-x86_64/egg/EGG-INFO copying src/pbs_python.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO copying src/pbs_python.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying src/pbs_python.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying src/pbs_python.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO writing build/bdist.linux-x86_64/egg/EGG-INFO/native_libs.txt zip_safe flag not set; analyzing archive contents... creating dist creating 'dist/pbs_python-4.1.0-py2.6-linux-x86_64.egg' and adding 'build/bdist.linux-x86_64/egg' to it removing 'build/bdist.linux-x86_64/egg' (and everything under it) Traceback (most recent call last): File "scripts/scramble.py", line 50, in <module> egg.scramble() File "/home/galaxy/galaxy-dist/lib/galaxy/eggs/scramble.py", line 60, in scramble raise ScrambleFailure( self, "%s(): Egg build for %s did not appear to fail, but no egg found to copy from expected path:\n %s" % ( sys._getframe().f_code.co_name, self.name, new_egg ) ) galaxy.eggs.scramble.ScrambleFailure: scramble(): Egg build for pbs_python did not appear to fail, but no egg found to copy from expected path: /home/galaxy/galaxy-dist/scripts/scramble/build/py2.6-linux-x86_64-ucs2/pbs_python/dist/pbs_python-4.1.0-py2.6-linux-x86_64-ucs2.egg [galaxy@bic galaxy-dist]$ cd scripts/scramble [galaxy@bic scramble]$ ll total 20 drwxrwxr-x 2 galaxy galaxy 4096 Dec 29 16:21 archives drwxrwxr-x 3 galaxy galaxy 4096 Dec 29 16:21 build drwxrwxr-x 2 galaxy galaxy 4096 Dec 29 16:22 lib drwxrwxr-x 7 galaxy galaxy 4096 Dec 29 13:25 patches drwxrwxr-x 2 galaxy galaxy 4096 Dec 29 13:25 scripts [galaxy@bic scramble]$ cd build/ [galaxy@bic build]$ ll total 4 drwxrwxr-x 3 galaxy galaxy 4096 Dec 29 16:21 py2.6-linux-x86_64-ucs2 [galaxy@bic build]$ cd py2.6-linux-x86_64-ucs2/ [galaxy@bic py2.6-linux-x86_64-ucs2]$ ll total 4 drwx------ 7 galaxy galaxy 4096 Dec 29 16:22 pbs_python [galaxy@bic py2.6-linux-x86_64-ucs2]$ cd pbs_python/dist/ [galaxy@bic dist]$ ll total 180 -rw-rw-r-- 1 galaxy galaxy 178869 Dec 29 16:22 pbs_python-4.1.0-py2.6-linux-x86_64.egg
I see the egg is created, but not with the name expected by galaxy. How do I fix this?
Could you make sure $PYTHONPATH is unset before starting? There are monkeypatches in lib/galaxy/__init__.py that are supposed to add the -ucsX to the platform string that are apparently not being applied here.
--nate
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: