July 2012 - galaxy-dev - lists.galaxyproject.org

Tool migration errors
by Anthonius deBoer 11 Jul '12

11 Jul '12

I'm getting this error when I try to run the tool migration: sh ./scripts/migrate_tools/0003_tools.sh install_dependencies No handlers could be found for logger "docutils" Repositories will be installed into configured tool_path location ../shed_tools Adding new row (or updating an existing row) for repository 'freebayes' in the tool_shed_repository table. Traceback (most recent call last): File "./scripts/migrate_tools/migrate_tools.py", line 21, in <module> app = MigrateToolsApplication( sys.argv[ 1 ] ) File "/home/tdeboer/code/galaxy-copy/lib/galaxy/tool_shed/migrate/common.py", line 150, in __init__ install_dependencies=install_dependencies ) File "/home/tdeboer/code/galaxy-copy/lib/galaxy/tool_shed/install_manager.py", line 37, in __init__ self.install_repository( repository_elem, install_dependencies ) File "/home/tdeboer/code/galaxy-copy/lib/galaxy/tool_shed/install_manager.py", line 258, in install_repository install_dependencies=install_dependencies ) File "/home/tdeboer/code/galaxy-copy/lib/galaxy/tool_shed/install_manager.py", line 174, in handle_repository_contents tool_dependencies=tool_dependencies ) File "/home/tdeboer/code/galaxy-copy/lib/galaxy/util/shed_util.py", line 1231, in handle_tool_dependencies tool_dependency = install_package( app, elem, tool_shed_repository, tool_dependencies=tool_dependencies ) File "/home/tdeboer/code/galaxy-copy/lib/galaxy/tool_shed/tool_dependencies/install_util.py", line 52, in install_package install_dir = get_tool_dependency_install_dir( app, tool_shed_repository, package_name, package_version ) File "/home/tdeboer/code/galaxy-copy/lib/galaxy/tool_shed/tool_dependencies/install_util.py", line 42, in get_tool_dependency_install_dir repository.installed_changeset_revision ) ) File "/home/tdeboer/code/galaxy_env/lib/python2.6/posixpath.py", line 67, in join elif path == '' or path.endswith('/'): AttributeError: 'NoneType' object has no attribute 'endswith' Any ideas? Thanks Thon

3 7

Where are the datatype associations stored?
by Anthonius deBoer 11 Jul '12

11 Jul '12

Hi, I had some more bad luck trying to install the GMAP stuff from the TOOLSHEd but revision 10 is broken since it does not allow you to select a section and just stores the programs in the main tree... But when I tried to remove the tool I got the following error for every run, since the sniffers where still looking for a module called GMAP... Which table should I nuke to get rid of this association? Where are the datatype associations stored? There does not seem to be an XML file for this, so I hope i can just remove some lines from the postgres database table but there are no obvious table names I can find... WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.gmap:IntervalAnnotation' to sniff_order: No module named gmap WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.gmap:SpliceSiteAnnotation' to sniff_order: No module named gmap WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.gmap:IntronAnnotation' to sniff_order: No module named gmap WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.gmap:SNPAnnotation' to sniff_order: No module named gmap WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.gmap:IntervalAnnotation' to sniff_order: No module named gmap WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.gmap:SpliceSiteAnnotation' to sniff_order: No module named gmap WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.gmap:IntronAnnotation' to sniff_order: No module named gmap WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.gmap:SNPAnnotation' to sniff_order: No module named gmap Thanks

2 2

javascript
by Cathy Riemer 11 Jul '12

11 Jul '12

Hi all, I just now tried visiting Galaxy main with my browser's javascript turned off. Formerly the left Tools panel would appear with all of the sections expanded, however now it is empty except for "Workflows". Also, some of the top menu bar items work without scripting, but others (including Help and User) do not. Ideally Galaxy would be usable without scripting, but if that is not feasible then I suggest checking whether javascript is available (as many other sites do) and displaying a prominent explanatory message if it is not. This would be much more helpful than an inexplicably near-empty tool panel and broken menus. In case it is relevant, I'm running Firefox 3.6.23 with the NoScript extension (2.4.7), on CentOS Linux with kernel 2.6.18-238.12.1.el5. Thanks, -Cathy

2 2

is there a way too automatically select references based on the genome of the dataset?
by Langhorst, Brad 11 Jul '12

11 Jul '12

I would like the reference list to be automatically selected based on the dbkey of the selected input... I think I've seen this, but I can'f seem to find an example. Anybody know? Should be some easy jquery i guess. Brad -- Brad Langhorst langhorst(a)neb.com<mailto:langhorst@neb.com> 978-380-7564

2 1

Output file permissions different pbs:/// VS local:/// job runners
by Rob Syme 11 Jul '12

11 Jul '12

Hi all. When I run the FastQC tool using the default local job runner, it produces a folder of files with permission 664: <snip> -rw-rw-r-- 1 galaxy galaxy 17K Jul 5 17:29 duplication_levels.png -rw-rw-r-- 1 galaxy galaxy 1.6K Jul 5 17:29 error.png -rw-rw-r-- 1 galaxy galaxy 7.5K Jul 5 17:29 fastqc_data.txt <snip> When I switch to using the(local) pbs queue, it produces a folder with permissions 600: <snip> -rw------- 1 galaxy galaxy 15K Jul 6 09:10 duplication_levels.png -rw------- 1 galaxy galaxy 1.6K Jul 6 09:10 error.png -rw------- 1 galaxy galaxy 8.8K Jul 6 09:10 fastqc_data.txt <snip> The final dat file (in database/files/000/dataset_xx.dat) is fine, it's just the files in the corresponding folder (database/files/000/dataset_xx_files/*) that have the incorrect permissions. These more restrictive permissions mean that the files cannot be read by apache, resulting a bunch of broken links when viewing the tool's output. Does anybody know why this might be happening or how to resolve it? Many thanks. -r

2 1

Galaxy cleaning up after it self after jobs fail
by Anthonius deBoer 11 Jul '12

11 Jul '12

I am running galaxy on a system that normally runs all jobs on the cluster, but some jobs need to be run locally... But when I restart galaxy when these jobs are running, those jobs fail (and I am fine with that)... But then when I check the logs, I see a lot of this kind of errors and the word CRITICAL scares me a little... It complains about certain directories not being empty (although they look empty to me when I look at them).... But if it needs to clean up should it not expect stuff to be in those working directories? And why is this a critical error and why don't those directories get removed anyway...It clutters my database and I fear this was the reason for weird behaviour I saw earlier, since it may re-use some of these directories after a while ? galaxy.objectstore CRITICAL 2012-07-05 16:57:24,964 /mnt/ngs/analysis/tdeboer/galaxy-data/database2/job_working_directory/003/3350 delete error [Errno 39] Directory not empty: '/mnt/ngs/analysis/tdeboer/galaxy-data/database2/job_working_directory/003/3350' Thanks Thon

2 1

Re: [galaxy-dev] Byte-range support for IGV ?
by Chebbi Mohamed Amine 11 Jul '12

11 Jul '12

HI Geert ! I did well : <Location "/"> XSendFile on XSendFilePath / </Location> and <Location "/"> # Compress all uncompressed content. SetOutputFilter DEFLATE SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary SetEnvIfNoCase Request_URI \.(?:t?gz|zip|bz2)$ no-gzip dont-vary </Location> But it dosen't fix it. I think that il is may be a problem of server byte-range accepting? Thanks Amine 2012/7/4 Geert Vandeweyer <geert.vandeweyer2(a)ua.ac.be> > Hi, > > If I remember correctly, I solved this issue by enabling the xsendfile > module in apache and setting galaxy to use the proxy for file downloading. > > best regards, > > Geert > > > On 07/04/2012 11:15 AM, Chebbi Mohamed Amine wrote: > > HELLO ! > > I just updated our local galaxy instance and just wanted to try the new > IGV display application. When I try to display a BAM file whith the local > IGV, I encounter the following message : > > "HTTP and FTP access to BAM files require byte-range support" > > Has anyone encountred the same probleme? > > Thanks in advances > > Regards > Amine > > > > > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ > > > > -- > > Geert Vandeweyer, Ph.D. > Department of Medical Genetics > University of Antwerp > Prins Boudewijnlaan 43 > 2650 Edegem > Belgium > Tel: +32 (0)3 275 97 56 > E-mail: geert.vandeweyer@ua.ac.behttp://ua.ac.be/cognitivegeneticshttp://www.linkedin.com/pub/geert-vandeweyer/26/457/726 > >

2 2

Unable to queue job, resubmitting might help
by Geert Vandeweyer 11 Jul '12

11 Jul '12

Hi, Today I ran into a cluster error on our local instance using latest galaxy-dist and torque/pbs with the python-pbs binding. Under heavy load of the galaxy process, it appears that the handler processes failed to contact the pbs-server, although the pbs_server was still up and running. after that, a lot of the following statements kept appearing in the handler.log file: galaxy.jobs.runners.pbs DEBUG 2012-07-11 17:39:06,649 (11647/12788.pbs_master_address) Skipping state check because PBS server connection failed After restarting the galaxy process (run.sh), everything worked again, with no changes to the pbs_server. Would it be possible to setup some checks for this failure? Like: - contact system admin - restart galaxy - auto retry job submission after a while as to not crash workflows. best regards, Geert Vandeweyer

1 0

Re: [galaxy-dev] pbs_python issues
by Iry Witham 11 Jul '12

11 Jul '12

I was successful in getting the bps_python egg to load finally. However, there is an issue that remains. When I submit a job it is sent to the local runner and not to pbs. Where should I start looking to change this work? Thanks, Iry On 7/10/12 12:00 PM, "galaxy-dev-request(a)lists.bx.psu.edu" <galaxy-dev-request(a)lists.bx.psu.edu> wrote: > >Message: 3 >Date: Mon, 9 Jul 2012 17:24:25 +0000 >From: Iry Witham <Iry.Witham(a)jax.org> >To: "Dorset, Daniel C" <daniel.dorset(a)Vanderbilt.Edu> >Cc: "galaxy-dev(a)lists.bx.psu.edu" <galaxy-dev(a)lists.bx.psu.edu> >Subject: Re: [galaxy-dev] galaxy-dev Digest, Vol 73, Issue 10 >Message-ID: <CC207DDE.8F49%iry.witham(a)jax.org> >Content-Type: text/plain; charset="us-ascii" > >Hi Daniel, > > I did use -e previously. The ?e was a typo in my post. I ran it again >and it tries, but fails to generate the file(s) as shown: > >:galaxy@jaxgalaxydev01:/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist> >LIBTORQUE_DIR=/usr/local/lib/ python scripts/scramble.py -e pbs_python >fetch_one(): Trying to fetch: > http://eggs.g2.bx.psu.edu/pbs_python/pbs_python-4.1.0.tar.gz >fetch_one(): Fetched to: > >/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist/scripts/scramble/archives/pbs >_ >python-4.1.0.tar.gz >unpack_source(): Unpacked to: > >/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist/scripts/scramble/build/py2.6- >l >inux-x86_64-ucs4/pbs_python >copy_build_script(): Using build script >/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist/scripts/scramble/scripts/pbs_ >p >ython.py >run_scramble_script(): Beginning build >run_scramble_script(): Executing in >/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist/scripts/scramble/build/py2.6- >l >inux-x86_64-ucs4/pbs_python: > /usr/bin/python scramble.py > >-------------------------------------------------------------------------- >- >This script requires setuptools version 0.6c11 to run (even to display >help). I will attempt to download it for you (from >http://pypi.python.org/packages/2.6/s/setuptools/) but >you may need to enable firewall access for this script first. >I will start the download in 8 seconds. > >(Note: if this machine does not have network access, please obtain the >file > > >http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c11-py2.6.e >g >g > >and place it in this directory before rerunning this script.) >-------------------------------------------------------------------------- >- >Downloading >http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c11-py2.6.e >g >g >checking for pbs-config... /usr/local/lib//../bin/pbs-config >Found torque version: 2.5.8 >checking for python... /usr/bin/python >checking for python version... 2.6 >checking for python platform... linux2 >checking for python script directory... >${prefix}/lib64/python2.6/site-packages >checking for python extension module directory... >${exec_prefix}/lib64/python2.6/site-packages >configure: creating ./config.status >config.status: creating Makefile >config.status: creating setup.py >scramble(): Patching setup.py >running egg_info >creating src/pbs_python.egg-info >writing src/pbs_python.egg-info/PKG-INFO >writing top-level names to src/pbs_python.egg-info/top_level.txt >writing dependency_links to src/pbs_python.egg-info/dependency_links.txt >writing manifest file 'src/pbs_python.egg-info/SOURCES.txt' >reading manifest file 'src/pbs_python.egg-info/SOURCES.txt' >writing manifest file 'src/pbs_python.egg-info/SOURCES.txt' >running bdist_egg >installing library code to build/bdist.linux-x86_64/egg >running install_lib >running build_py >creating build >creating build/lib.linux-x86_64-2.6 >copying src/pbs.py -> build/lib.linux-x86_64-2.6 >copying src/PBSQuery.py -> build/lib.linux-x86_64-2.6 >running build_ext >building '_pbs' extension >creating build/temp.linux-x86_64-2.6 >creating build/temp.linux-x86_64-2.6/src >gcc -pthread -fno-strict-aliasing -DNDEBUG -fmessage-length=0 -O2 -Wall >-D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables >-fasynchronous-unwind-tables -g -fwrapv -fPIC -DTORQUE_2_4 >-I/usr/include/python2.6 -c src/pbs_wrap.c -o >build/temp.linux-x86_64-2.6/src/pbs_wrap.o >gcc -pthread -shared build/temp.linux-x86_64-2.6/src/pbs_wrap.o >-L/usr/local/lib -L/usr/lib64 -ltorque -lpython2.6 -o >build/lib.linux-x86_64-2.6/_pbs.so >creating build/bdist.linux-x86_64 >creating build/bdist.linux-x86_64/egg >copying build/lib.linux-x86_64-2.6/PBSQuery.py -> >build/bdist.linux-x86_64/egg >copying build/lib.linux-x86_64-2.6/pbs.py -> build/bdist.linux-x86_64/egg >copying build/lib.linux-x86_64-2.6/_pbs.so -> build/bdist.linux-x86_64/egg >byte-compiling build/bdist.linux-x86_64/egg/PBSQuery.py to PBSQuery.pyc >byte-compiling build/bdist.linux-x86_64/egg/pbs.py to pbs.pyc >creating stub loader for _pbs.so >byte-compiling build/bdist.linux-x86_64/egg/_pbs.py to _pbs.pyc >creating build/bdist.linux-x86_64/egg/EGG-INFO >copying src/pbs_python.egg-info/PKG-INFO -> >build/bdist.linux-x86_64/egg/EGG-INFO >copying src/pbs_python.egg-info/SOURCES.txt -> >build/bdist.linux-x86_64/egg/EGG-INFO >copying src/pbs_python.egg-info/dependency_links.txt -> >build/bdist.linux-x86_64/egg/EGG-INFO >copying src/pbs_python.egg-info/top_level.txt -> >build/bdist.linux-x86_64/egg/EGG-INFO >writing build/bdist.linux-x86_64/egg/EGG-INFO/native_libs.txt >zip_safe flag not set; analyzing archive contents... >creating dist >creating 'dist/pbs_python-4.1.0-py2.6-linux-x86_64.egg' and adding >'build/bdist.linux-x86_64/egg' to it >removing 'build/bdist.linux-x86_64/egg' (and everything under it) >Traceback (most recent call last): > File "scripts/scramble.py", line 50, in <module> > egg.scramble() > File >"/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist/lib/galaxy/eggs/scramble.py" >, > line 60, in scramble > raise ScrambleFailure( self, "%s(): Egg build for %s did not appear to >fail, but no egg found to copy from expected path:\n %s" % ( >sys._getframe().f_code.co_name, self.name, new_egg ) ) >galaxy.eggs.scramble.ScrambleFailure: scramble(): Egg build for pbs_python >did not appear to fail, but no egg found to copy from expected path: > >/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist/scripts/scramble/build/py2.6- >l >inux-x86_64-ucs4/pbs_python/dist/pbs_python-4.1.0-py2.6-linux-x86_64-ucs4. >e >gg" > >Iry > >On 7/7/12 3:10 PM, "Dorset, Daniel C" <daniel.dorset(a)Vanderbilt.Edu> >wrote: > >> >>________________________________________ >>From: galaxy-dev-bounces(a)lists.bx.psu.edu >>[galaxy-dev-bounces(a)lists.bx.psu.edu] On Behalf Of >>galaxy-dev-request(a)lists.bx.psu.edu [galaxy-dev-request(a)lists.bx.psu.edu] >>Sent: Saturday, July 07, 2012 11:00 AM >>To: galaxy-dev(a)lists.bx.psu.edu >>Subject: galaxy-dev Digest, Vol 73, Issue 10 >> >>Send galaxy-dev mailing list submissions to >> galaxy-dev(a)lists.bx.psu.edu >> >>To subscribe or unsubscribe via the World Wide Web, visit >> http://lists.bx.psu.edu/listinfo/galaxy-dev >>or, via email, send a message with subject or body 'help' to >> galaxy-dev-request(a)lists.bx.psu.edu >> >>You can reach the person managing the list at >> galaxy-dev-owner(a)lists.bx.psu.edu >> >>When replying, please edit your Subject line so it is more specific >>than "Re: Contents of galaxy-dev digest..." >> >> >>HEY! This is important! If you reply to a thread in a digest, please >>1. Change the subject of your response from "Galaxy-dev Digest Vol ..." >>to the original subject for the thread. >>2. Strip out everything else in the digest that is not part of the thread >>you are responding to. >> >>Why? >>1. This will keep the subject meaningful. People will have some idea >>from the subject line if they should read it or not. >>2. Not doing this greatly increases the number of emails that match >>search queries, but that aren't actually informative. >> >>Today's Topics: >> >> 1. pbs_python issues (Iry Witham) >> 2. Which css file does welcome.html use? (Dorset, Daniel C) >> 3. Re: Which css file does welcome.html use? (Jeremy Goecks) >> >> >>Hi Ira, >> >>I'm in charge of a Galaxy instance that's also deployed on a PBS Cluster, >>and I went through this process a couple of months ago. If I remember >>correctly, when you run your Python script, you're supposed to run it >>with "-e" not "?e" Assuming that you are pointing to the correct >>directory, it looks like everything else about your command is correct. >> >>Have you tried that? >> >>---------------------------------------------------------------------- >> >>Message: 1 >>Date: Fri, 6 Jul 2012 16:32:14 +0000 >>From: Iry Witham <Iry.Witham(a)jax.org> >>To: "galaxy-dev(a)lists.bx.psu.edu" <galaxy-dev(a)lists.bx.psu.edu> >>Subject: [galaxy-dev] pbs_python issues >>Message-ID: <CC1C8B4D.8EAD%iry.witham(a)jax.org> >>Content-Type: text/plain; charset="windows-1252" >> >>I am having an issue with pbs_python loading, but not submitting jobs to >>the cluster. >> >>The first issue is that when I ran 'LIBTORQUE_DIR=/usr/local/lib/ python >>scripts/scramble.py ?e pbs_python' it failed to generate the pbs_python >>egg in the /galaxy-dist/eggs/ directory. >> >>Secondly I copied the pbs_python-4.1.0-py2.6-linux-x86_64-ucs4.egg from >>my production galaxy server and restarted my test server. It appears to >>start, but when I attempt to run anything it uses the local runner and >>fails. The runner0.log shows the pbsjobrunner is loaded is below: >> >>galaxy.jobs.handler DEBUG 2012-07-06 10:08:45,681 Loaded job runner: >>galaxy.jobs.runners.pbs:PBSJobRunner >>galaxy.jobs.handler INFO 2012-07-06 10:08:45,682 job handler stop queue >>started >>galaxy.jobs.manager INFO 2012-07-06 10:08:45,687 job manager queue >>started >>galaxy.jobs.manager INFO 2012-07-06 10:08:45,688 job manager stop queue >>started >>galaxy.jobs.handler INFO 2012-07-06 10:08:45,692 job handler queue >>started >> >>I can't seem to get past this point since I upgraded. Can someone point >>me in the right direction? >> >>Thanks, >>Iry >> >> >>The information in this email, including attachments, may be confidential >>and is intended solely for the addressee(s). If you believe you received >>this email by mistake, please notify the sender by return email as soon >>as possible. >>

1 0

DRMAA.NoActiveSessionError
by Johannes Eichner 11 Jul '12

11 Jul '12

Dear Galaxy-Dev-Team, I just restarted our local Galaxy instance at the University of Tuebingen, Germany, and obtained a DRMAA.NoActiveSessionError. You can find an excerpt from my shell below. The error can be avoided by disabling the SGE support in the universe.wsgi.ini. However, as our galaxy instance runs on a virtual machine, which requires the cluster environment for proper job processing, I would very much appreciate your help on this issue. Kind regards, Johannes Traceback (most recent call last): File "/rahome/webservices/galaxy/lib/galaxy/web/buildapp.py", line 82, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/rahome/webservices/galaxy/lib/galaxy/app.py", line 124, in __init__ self.job_manager = jobs.JobManager( self ) File "/rahome/webservices/galaxy/lib/galaxy/jobs/__init__.py", line 44, in __init__ self.dispatcher = DefaultJobDispatcher( app ) File "/rahome/webservices/galaxy/lib/galaxy/jobs/__init__.py", line 1190, in __init__ self._load_plugin( name ) File "/rahome/webservices/galaxy/lib/galaxy/jobs/__init__.py", line 1207, in _load_plugin self.job_runners[name] = runner( self.app ) File "/rahome/webservices/galaxy/lib/galaxy/jobs/runners/sge.py", line 96, in __init__ self.ds.init( self.default_cell ) File "build/bdist.linux-x86_64/egg/DRMAA.py", line 186, in init errParse(eno, estr) File "build/bdist.linux-x86_64/egg/DRMAA.py", line 167, in errParse else: raise _errindex[errno-_errMin](errstring) InternalError: cl_com_setup_commlib failed: timeout while waiting for thread start Exception DRMAA.NoActiveSessionError: NoActiveSessionError('',) in <bound method Session.__del__ of <DRMAA.Session object at 0x4ecfb50>> ignored -- Dipl.-Inf. (Bioinf.) Johannes Eichner Univ. Tuebingen, WSI-RA, Sand 1, D-72076 Tuebingen, Germany Phone: (+49/0) 7071 / 29 78970 Fax: (+49/0) 7071 / 29 5091

1 0