I was successful in getting the bps_python egg to load finally. However, there is an issue that remains. When I submit a job it is sent to the local runner and not to pbs. Where should I start looking to change this work?
Thanks, Iry
On 7/10/12 12:00 PM, "galaxy-dev-request@lists.bx.psu.edu" galaxy-dev-request@lists.bx.psu.edu wrote:
Message: 3 Date: Mon, 9 Jul 2012 17:24:25 +0000 From: Iry Witham Iry.Witham@jax.org To: "Dorset, Daniel C" daniel.dorset@Vanderbilt.Edu Cc: "galaxy-dev@lists.bx.psu.edu" galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] galaxy-dev Digest, Vol 73, Issue 10 Message-ID: CC207DDE.8F49%iry.witham@jax.org Content-Type: text/plain; charset="us-ascii"
Hi Daniel,
I did use -e previously. The ?e was a typo in my post. I ran it again and it tries, but fails to generate the file(s) as shown:
:galaxy@jaxgalaxydev01:/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist> LIBTORQUE_DIR=/usr/local/lib/ python scripts/scramble.py -e pbs_python fetch_one(): Trying to fetch: http://eggs.g2.bx.psu.edu/pbs_python/pbs_python-4.1.0.tar.gz fetch_one(): Fetched to:
/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist/scripts/scramble/archives/pbs _ python-4.1.0.tar.gz unpack_source(): Unpacked to:
/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist/scripts/scramble/build/py2.6- l inux-x86_64-ucs4/pbs_python copy_build_script(): Using build script /hpcdata/galaxy-dev/galaxy-setup/galaxy-dist/scripts/scramble/scripts/pbs_ p ython.py run_scramble_script(): Beginning build run_scramble_script(): Executing in /hpcdata/galaxy-dev/galaxy-setup/galaxy-dist/scripts/scramble/build/py2.6- l inux-x86_64-ucs4/pbs_python: /usr/bin/python scramble.py
This script requires setuptools version 0.6c11 to run (even to display help). I will attempt to download it for you (from http://pypi.python.org/packages/2.6/s/setuptools/), but you may need to enable firewall access for this script first. I will start the download in 8 seconds.
(Note: if this machine does not have network access, please obtain the file
http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c11-py2.6.e g g
and place it in this directory before rerunning this script.)
Downloading http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c11-py2.6.e g g checking for pbs-config... /usr/local/lib//../bin/pbs-config Found torque version: 2.5.8 checking for python... /usr/bin/python checking for python version... 2.6 checking for python platform... linux2 checking for python script directory... ${prefix}/lib64/python2.6/site-packages checking for python extension module directory... ${exec_prefix}/lib64/python2.6/site-packages configure: creating ./config.status config.status: creating Makefile config.status: creating setup.py scramble(): Patching setup.py running egg_info creating src/pbs_python.egg-info writing src/pbs_python.egg-info/PKG-INFO writing top-level names to src/pbs_python.egg-info/top_level.txt writing dependency_links to src/pbs_python.egg-info/dependency_links.txt writing manifest file 'src/pbs_python.egg-info/SOURCES.txt' reading manifest file 'src/pbs_python.egg-info/SOURCES.txt' writing manifest file 'src/pbs_python.egg-info/SOURCES.txt' running bdist_egg installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py creating build creating build/lib.linux-x86_64-2.6 copying src/pbs.py -> build/lib.linux-x86_64-2.6 copying src/PBSQuery.py -> build/lib.linux-x86_64-2.6 running build_ext building '_pbs' extension creating build/temp.linux-x86_64-2.6 creating build/temp.linux-x86_64-2.6/src gcc -pthread -fno-strict-aliasing -DNDEBUG -fmessage-length=0 -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -g -fwrapv -fPIC -DTORQUE_2_4 -I/usr/include/python2.6 -c src/pbs_wrap.c -o build/temp.linux-x86_64-2.6/src/pbs_wrap.o gcc -pthread -shared build/temp.linux-x86_64-2.6/src/pbs_wrap.o -L/usr/local/lib -L/usr/lib64 -ltorque -lpython2.6 -o build/lib.linux-x86_64-2.6/_pbs.so creating build/bdist.linux-x86_64 creating build/bdist.linux-x86_64/egg copying build/lib.linux-x86_64-2.6/PBSQuery.py -> build/bdist.linux-x86_64/egg copying build/lib.linux-x86_64-2.6/pbs.py -> build/bdist.linux-x86_64/egg copying build/lib.linux-x86_64-2.6/_pbs.so -> build/bdist.linux-x86_64/egg byte-compiling build/bdist.linux-x86_64/egg/PBSQuery.py to PBSQuery.pyc byte-compiling build/bdist.linux-x86_64/egg/pbs.py to pbs.pyc creating stub loader for _pbs.so byte-compiling build/bdist.linux-x86_64/egg/_pbs.py to _pbs.pyc creating build/bdist.linux-x86_64/egg/EGG-INFO copying src/pbs_python.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO copying src/pbs_python.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying src/pbs_python.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying src/pbs_python.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO writing build/bdist.linux-x86_64/egg/EGG-INFO/native_libs.txt zip_safe flag not set; analyzing archive contents... creating dist creating 'dist/pbs_python-4.1.0-py2.6-linux-x86_64.egg' and adding 'build/bdist.linux-x86_64/egg' to it removing 'build/bdist.linux-x86_64/egg' (and everything under it) Traceback (most recent call last): File "scripts/scramble.py", line 50, in <module> egg.scramble() File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist/lib/galaxy/eggs/scramble.py" , line 60, in scramble raise ScrambleFailure( self, "%s(): Egg build for %s did not appear to fail, but no egg found to copy from expected path:\n %s" % ( sys._getframe().f_code.co_name, self.name, new_egg ) ) galaxy.eggs.scramble.ScrambleFailure: scramble(): Egg build for pbs_python did not appear to fail, but no egg found to copy from expected path:
/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist/scripts/scramble/build/py2.6- l inux-x86_64-ucs4/pbs_python/dist/pbs_python-4.1.0-py2.6-linux-x86_64-ucs4. e gg"
Iry
On 7/7/12 3:10 PM, "Dorset, Daniel C" daniel.dorset@Vanderbilt.Edu wrote:
From: galaxy-dev-bounces@lists.bx.psu.edu [galaxy-dev-bounces@lists.bx.psu.edu] On Behalf Of galaxy-dev-request@lists.bx.psu.edu [galaxy-dev-request@lists.bx.psu.edu] Sent: Saturday, July 07, 2012 11:00 AM To: galaxy-dev@lists.bx.psu.edu Subject: galaxy-dev Digest, Vol 73, Issue 10
Send galaxy-dev mailing list submissions to galaxy-dev@lists.bx.psu.edu
To subscribe or unsubscribe via the World Wide Web, visit http://lists.bx.psu.edu/listinfo/galaxy-dev or, via email, send a message with subject or body 'help' to galaxy-dev-request@lists.bx.psu.edu
You can reach the person managing the list at galaxy-dev-owner@lists.bx.psu.edu
When replying, please edit your Subject line so it is more specific than "Re: Contents of galaxy-dev digest..."
HEY! This is important! If you reply to a thread in a digest, please
- Change the subject of your response from "Galaxy-dev Digest Vol ..."
to the original subject for the thread. 2. Strip out everything else in the digest that is not part of the thread you are responding to.
Why?
- This will keep the subject meaningful. People will have some idea
from the subject line if they should read it or not. 2. Not doing this greatly increases the number of emails that match search queries, but that aren't actually informative.
Today's Topics:
- pbs_python issues (Iry Witham)
- Which css file does welcome.html use? (Dorset, Daniel C)
- Re: Which css file does welcome.html use? (Jeremy Goecks)
Hi Ira,
I'm in charge of a Galaxy instance that's also deployed on a PBS Cluster, and I went through this process a couple of months ago. If I remember correctly, when you run your Python script, you're supposed to run it with "-e" not "?e" Assuming that you are pointing to the correct directory, it looks like everything else about your command is correct.
Have you tried that?
Message: 1 Date: Fri, 6 Jul 2012 16:32:14 +0000 From: Iry Witham Iry.Witham@jax.org To: "galaxy-dev@lists.bx.psu.edu" galaxy-dev@lists.bx.psu.edu Subject: [galaxy-dev] pbs_python issues Message-ID: CC1C8B4D.8EAD%iry.witham@jax.org Content-Type: text/plain; charset="windows-1252"
I am having an issue with pbs_python loading, but not submitting jobs to the cluster.
The first issue is that when I ran 'LIBTORQUE_DIR=/usr/local/lib/ python scripts/scramble.py ?e pbs_python' it failed to generate the pbs_python egg in the /galaxy-dist/eggs/ directory.
Secondly I copied the pbs_python-4.1.0-py2.6-linux-x86_64-ucs4.egg from my production galaxy server and restarted my test server. It appears to start, but when I attempt to run anything it uses the local runner and fails. The runner0.log shows the pbsjobrunner is loaded is below:
galaxy.jobs.handler DEBUG 2012-07-06 10:08:45,681 Loaded job runner: galaxy.jobs.runners.pbs:PBSJobRunner galaxy.jobs.handler INFO 2012-07-06 10:08:45,682 job handler stop queue started galaxy.jobs.manager INFO 2012-07-06 10:08:45,687 job manager queue started galaxy.jobs.manager INFO 2012-07-06 10:08:45,688 job manager stop queue started galaxy.jobs.handler INFO 2012-07-06 10:08:45,692 job handler queue started
I can't seem to get past this point since I upgraded. Can someone point me in the right direction?
Thanks, Iry
The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.