April 2012 - galaxy-dev - lists.galaxyproject.org

Galaxy loading never ends
by Makis Ladoukakis 04 Apr '12

04 Apr '12

Dear Galaxy users, I have installed a local Galaxy instance in my university's server. Recently I encountered a problem whenever I try to load Galaxy into my browser. When I press the link it starts loading but it never ends resulting with a timeout error. Has anyone encountered this problem before? Thank you in advance, Makis Ladoukakis

1 0

run cluster jobs as real user
by Andreas Kuntzagk 04 Apr '12

04 Apr '12

Hi, I (re-)installed galaxy on our server and managed to set it up to run jobs through our cluster (GridEngine). It's also running behind an Apache proxy now. But now I'm wondering how to setup to have the cluster jobs run as the real cluster users. I found the documentation on this topic a little thin. We use NIS for managing users and passwords across the cluster. How do I connect that to Galaxy? regards, Andreas -- Andreas Kuntzagk SystemAdministrator Berlin Institute for Medical Systems Biology at the Max-Delbrueck-Center for Molecular Medicine Robert-Roessle-Str. 10, 13125 Berlin, Germany http://www.mdc-berlin.de/en/bimsb/BIMSB_groups/Dieterich

3 6

Production environment and Apache's sendfile
by Yuri D'Elia 04 Apr '12

04 Apr '12

Hi everyone. I've set up a galaxy production environment and it seems to be working. I've identified a couple of issues though: If I enable apache_xsendfile, the little "eye" icon ("Display data in browser") on each dataset stops working. I get the following instead: The requested URL /datasets/7e93e3cf926fb2b3/display/ was not found on this server My document root is not the same as the galaxy root (as suggested). Everything else seems to work. Is this a bug? Second question: I want to rotate the galaxy log file (as specified with paster.py serve --log-file. I can't find a way to reopen the logfile in galaxy without taking down the whole service, or a way to redirect the log to a pipe together while using --daemon. I cannot let the file grow forever. Any suggestions? Right now I've setup logrotate to "copytruncate" the log (the log is opened in append anyway), but it's not a very nice solution. Does "paster" support some standard signal such as SIGUSR1 to re-open log files? Thanks.

2 2

file download returns empty file or webpage fails with ERR_CONNECTION_CLOSED
by Derrick LIN 03 Apr '12

03 Apr '12

Hi guys, I have setup a test galaxy and configured Apache as proxy. It's been running fine when I did all testing on my machine with Chrome. Recently, I let some of my colleagues to try it out, then they experience some file download issues. On one machine, both FireFox and IE8 downloaded empty files (zero byte). But on another Linux machine, FireFox downloaded a complete file successfully. On two MacBook machines, Chrome encountered webpage error (only the right hand side) as soon as download button was pressed. Saying: *This webpage is not available* * * *error 100 (net::ERR_CONNECTION_CLOSED): The server unexpectedly closed the connection.* I have seems someone posted the exact same issue before but no solution has been given. I enabled Apache xsend file: <Location "/"> XSendFile on XSendFileAllowAbove on</Location> And configured galaxy to be aware of this in universe_wsgi.ini apache_xsendfile = True. Any help will be appreciated. Thanks Derrick

2 1

incorrect tmp dir handling in picard AddOrReplaceReadGroups
by Matt Shirley 03 Apr '12

03 Apr '12

Every time I run Picard's AddOrReplaceReadGroups, my system tmp directory fills, and the tool stops. I'm left scratching my head, because it seems that the JVM sets the system /tmp directory on my local galaxy-central branch, instead of the "new_file_path" I have defined in universe_wsgi.ini. This issue seems to have been fixed in commit 07560293b167<https://bitbucket.org/galaxy/galaxy-central/changeset/07560293b167>, and I can confirm that my picard_wrapper.py includes the amended line. I am using OpenJDK: java version "1.6.0_20" OpenJDK Runtime Environment (IcedTea6 1.9.10) (rhel-1.40.1.9.10.el6_1-x86_64) OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode) Since my config is pointing to /scratch/tmp, but the JVM is using /tmp, and other tools in galaxy are respecting the config, what could be the disconnect? Thank you, Matt -- Matt Shirley Ph.D Candidate - BCMB Pevsner Lab <http://pevsnerlab.kennedykrieger.org/> Johns Hopkins Medicine

2 1

Missing tool previews in tool shed, e.g. hmmer.xml
by Peter Cock 03 Apr '12

03 Apr '12

Hello all, I was looking at Edward's updated HMMER wrapper on the toolshed, http://toolshed.g2.bx.psu.edu/ There is a preview offered for the (simple) hmmpress.xml only, giving the impression that Edward's repository isn't very useful. Why isn't anything shown for the more complex hmmer.xml? If there is a problem rendering the preview, it would still be useful to list the tool in the table with its description, version and requirements. Edward - I have a query (which might be what the Tool Shed preview is unhappy about), where is the "hmmer" file format referenced in hmmer.xml defined? In hmmpress.xml you use "hmm" (as an input) and "hmmpressed" (as an output) which are both defined in hmmer.py, so is this an accidental inconsistency? Thanks, Peter P.S. Edward, hmmpress.xml is missing hmmpress as a requirement.

2 4

Toolshed error when browsing own repo's
by Joachim Jacob 03 Apr '12

03 Apr '12

Hi all, On a local Toolshed freshly installed, I got these errors when browsing my repository. Perhaps a bug? Cheers, Joachim Server Error URL: http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o… <http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o…> Module paste.exceptions.errormiddleware:*143* in |__call__| |>> <http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o…>app_iter *=* self*.*application*(*environ*,* start_response*)*| Module paste.debug.prints:*98* in |__call__| |>> <http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o…>environ*,* self*.*app*)*| Module paste.wsgilib:*539* in |intercept_output| |>> <http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o…>app_iter *=* application*(*environ*,* replacement_start_response*)*| Module paste.recursive:*80* in |__call__| |>> <http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o…>*return* self*.*application*(*environ*,* start_response*)*| Module paste.httpexceptions:*632* in |__call__| |>> <http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o…>*return* self*.*application*(*environ*,* start_response*)*| Module galaxy.web.framework.base:*160* in |__call__| |>> <http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o…>body *=* method*(* trans*,* ****kwargs *)*| Module galaxy.webapps.community.controllers.repository:*985* in |browse_repositories| |>> <http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o…>*return* self*.*repository_list_grid*(* trans*,* ****kwd *)*| Module galaxy.web.framework.helpers.grids:*277* in |__call__| |>> <http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o…>kwargs*=*kwargs *)*| Module galaxy.web.framework:*722* in |fill_template| |>> <http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o…>*return* self*.*fill_template_mako*(* filename*,* ****kwargs *)*| Module galaxy.web.framework:*733* in |fill_template_mako| |>> <http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o…>*return* template*.*render*(* ****data *)*| Module mako.template:*296* in |render| |>> <http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o…>*return* runtime*.*_render*(*self*,* self*.*callable_*,* args*,* data*)*| Module mako.runtime:*660* in |_render| |>> <http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o…>****_kwargs_for_callable*(*callable_*,* data*)**)*| Module mako.runtime:*692* in |_render_context| |>> <http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o…>_exec_template*(*inherit*,* lclcontext*,* args*=*args*,* kwargs*=*kwargs*)*| Module mako.runtime:*718* in |_exec_template| |>> <http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o…>callable_*(*context*,* ***args*,* ****kwargs*)*| Module _base_mako:*42* in |render_body| |>> <http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o…>__M_writer*(*unicode*(*next*.*body*(**)**)**)*| Module _grid_base_mako:*119* in |render_body| |>> <http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o…>__M_writer*(*unicode*(*self*.*grid_body*(* grid *)**)**)*| Module _grid_base_mako:*790* in |render_grid_body| |>> <http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o…>__M_writer*(*unicode*(*self*.*make_grid*(* grid *)**)**)*| Module _grid_base_mako:*819* in |render_make_grid| |>> <http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o…>__M_writer*(*unicode*(*self*.*render_grid_table*(* grid*,* show_item_checkboxes *)**)**)*| Module _grid_base_mako:*768* in |render_render_grid_table| |>> <http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o…>__M_writer*(*unicode*(*render_grid_table_body_contents*(* grid*,* show_item_checkboxes *)**)**)*| Module _grid_base_mako:*664* in |render_grid_table_body_contents| |>> <http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o…>*return* render_render_grid_table_body_contents*(*context*,*grid*,*show_item_checkboxes*)*| Module _grid_base_mako:*322* in |render_render_grid_table_body_contents| |>> <http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o…>value *=* column*.*get_value*(* trans*,* grid*,* item *)*| Module galaxy.webapps.community.controllers.repository:*84* in |get_value| |>> <http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o…>select_field *=* build_changeset_revision_select_field*(* trans*,* repository *)*| Module galaxy.webapps.community.controllers.common:*992* in |build_changeset_revision_select_field| |>> <http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o…>repo *=* hg*.*repository*(* get_configured_ui*(**)**,* repository*.*repo_path *)*| Module galaxy.webapps.community.controllers.common:*757* in |get_configured_ui| |>> <http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o…>_ui*.*setconfig*(* 'ui'*,* 'quiet'*,* True *)*| Module mercurial.ui:*113* in |setconfig| |>> <http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o…>self*.*fixconfig*(**)*| Module mercurial.ui:*100* in |fixconfig| |>> <http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o…>self*.*quiet *=* *not* self*.*debugflag *and* self*.*configbool*(*'ui'*,* 'quiet'*)*| Module mercurial.ui:*134* in |configbool| |>> <http://192.168.10.26:9009/repository/browse_repositories?webapp=community&o…>*if* v*.*lower*(**)* *not* *in* _booleans*:*| *AttributeError: 'bool' object has no attribute 'lower' * -- Joachim Jacob, PhD Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib

2 10

JSONDecodeERROR thrown, job left in "running" state.
by Ann Black-Ziegelbein 03 Apr '12

03 Apr '12

Hi all - In running a samtools sort, we experienced the following exception from Galaxy: galaxy.jobs.runners.drmaa ERROR 2012-04-01 12:01:27,200 Job wrapper finish method failed Traceback (most recent call last): File "/Groups/Galaxy/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py", line 286, in finish_job drm_job_state.job_wrapper.finish( stdout, stderr ) File "/Groups/Galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 609, in finish ( not self.external_output_metadata.external_metadata_set_successfully( dataset, self.sa_session ) \ File "/Groups/Galaxy/galaxy-dist/lib/galaxy/datatypes/metadata.py", line 602, in external_metadata_set_successfully rval, rstring = simplejson.load( open( metadata_files.filename_results_code ) ) File "/Groups/Galaxy/galaxy-dist/eggs/simplejson-2.1.1-py2.7-linux-x86_64-ucs2.egg/simplejson/__init__.py", line 328, in load use_decimal=use_decimal, **kw) File "/Groups/Galaxy/galaxy-dist/eggs/simplejson-2.1.1-py2.7-linux-x86_64-ucs2.egg/simplejson/__init__.py", line 384, in loads return _default_decoder.decode(s) File "/Groups/Galaxy/galaxy-dist/eggs/simplejson-2.1.1-py2.7-linux-x86_64-ucs2.egg/simplejson/decoder.py", line 405, in decode raise JSONDecodeError("Extra data", s, end, len(s)) JSONDecodeError: Extra data: line 1 column 41 - line 1 column 44 (char 41 - 44) galaxy.jobs DEBUG 2012-04-02 10:07:16,655 dispatching job 4738 to drmaa runner galaxy.jobs INFO 2012-04-02 10:07:16,736 job 4738 dispatched This resulted in a "hung" job from galaxy's perspective. The job was left with the status "running" so the workflow could not fail. Unfortunately, the json file was not preserved - it was removed. We have not hit this problem before. minimally, if this problem arrises, it would be nice for galaxy to preserve the json file for PD and fail the job. Thoughts? Anyone else hit this? Thanks, Ann

2 1

bz2 error and An error occurred running this job: Job output not returned from cluster
by Robert Chase 03 Apr '12

03 Apr '12

Hello, I am trying to fix something that I see in my paster.log file. At the end of the file, after everything has been loaded and the server is running, I get an error message saying that bz2 cannot be imported. When I start a python interpreter session and type in "import bz2" it runs without incident. I can even get the doc info. I am also getting the error An error occurred running this job: Job output not returned from cluster when I run an "add column" to a basic data set from the text manipulation tools file. I'm wondering if they are related. We are currently trying to check our queueing system to make sure it can correctly deliver these back from the execution hosts to our filesystem. -Robert Chase Similar bugs appears here: http://gmod.827538.n3.nabble.com/Job-output-not-returned-from-cluster-td342… http://osdir.com/ml/galaxy-source-control/2009-03/msg00015.html Python 2.7.1 (r271:86832, Mar 13 2012, 17:18:00) [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import bz2 >>> print bz2.__doc__ The python bz2 module provides a comprehensive interface for the bz2 compression library. It implements a complete file interface, one shot (de)compression functions, and types for sequential (de)compression. 7.0.0.1:8080 Traceback (most recent call last): File "./scripts/paster.py", line 34, in <module> command.run() File "/share/shared/galaxy-dist/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/command.py", line 84, in run invoke(command, command_name, options, args[1:]) File "/share/shared/galaxy-dist/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/command.py", line 123, in invoke exit_code = runner.run(args) File "/share/shared/galaxy-dist/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/command.py", line 218, in run result = self.command() File "/share/shared/galaxy-dist/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/serve.py", line 276, in command relative_to=base, global_conf=vars) File "/share/shared/galaxy-dist/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/serve.py", line 313, in loadapp **kw) File "/share/shared/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/deploy/loadwsgi.py", line 204, in loadapp return loadobj(APP, uri, name=name, **kw) File "/share/shared/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/deploy/loadwsgi.py", line 224, in loadobj global_conf=global_conf) File "/share/shared/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/deploy/loadwsgi.py", line 248, in loadcontext global_conf=global_conf) File "/share/shared/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/deploy/loadwsgi.py", line 278, in _loadconfig return loader.get_context(object_type, name, global_conf) File "/share/shared/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/deploy/loadwsgi.py", line 413, in get_context section) File "/share/shared/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/deploy/loadwsgi.py", line 458, in _context_from_explicit value = import_string(found_expr) File "/share/shared/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/deploy/loadwsgi.py", line 18, in import_string return pkg_resources.EntryPoint.parse("x="+s).load(False) File "/share/shared/galaxy-dist/lib/pkg_resources.py", line 1954, in load entry = __import__(self.module_name, globals(),globals(), ['__name__']) File "/share/shared/galaxy-dist/lib/galaxy/web/buildapp.py", line 19, in <module> from galaxy import config, jobs, util, tools File "/share/shared/galaxy-dist/lib/galaxy/jobs/__init__.py", line 4, in <module> from galaxy import util, model File "/share/shared/galaxy-dist/lib/galaxy/model/__init__.py", line 13, in <module> import galaxy.datatypes.registry File "/share/shared/galaxy-dist/lib/galaxy/datatypes/registry.py", line 5, in <module> import data, tabular, interval, images, sequence, qualityscore, genetics, xml, coverage, tracks, chrominfo, binary, assembly, ngsindex, wsf File "/share/shared/galaxy-dist/lib/galaxy/datatypes/tabular.py", line 15, in <module> from sniff import * File "/share/shared/galaxy-dist/lib/galaxy/datatypes/sniff.py", line 7, in <module> from galaxy.datatypes.checkers import * File "/share/shared/galaxy-dist/lib/galaxy/datatypes/checkers.py", line 1, in <module> import os, gzip, re, gzip, zipfile, binascii, bz2, imghdr ImportError: No module named bz2

2 1

Re: [galaxy-dev] [galaxy-user] Blast2GO local instance Re: Table with gene count reads
by Peter Cock 03 Apr '12

03 Apr '12

On Fri, Mar 23, 2012 at 10:45 PM, Luciano Cosme <cosme.simple(a)gmail.com> wrote: > Howdy, > Thanks Jen, I will try it tomorrow. > > I installed Blast2Go from the Toolshed in my local instance of Galaxy and > when I try to run it I get the following error: > > Index file named 'blast2go.loc' is required by tool but not available. > > I logged as admin and the installation did not gave me any error. From > the terminal: > > galaxy.util.shed_util DEBUG 2012-03-23 17:23:37,088 Installing repository > 'blast2go' > galaxy.util.shed_util DEBUG 2012-03-23 17:23:37,088 Cloning > http://testtoolshed.g2.bx.psu.edu/repos/peterjc/blast2go > destination directory: blast2go > requesting all changes > adding changesets > adding manifests > adding file changes > added 2 changesets with 6 changes to 3 files > updating to branch default > 3 files updated, 0 files merged, 0 files removed, 0 files unresolved > galaxy.util.shed_util DEBUG 2012-03-23 17:23:42,798 Updating cloned > repository to revision "7b53cc52e7ed" > > > Anyway, I was thinking to use it because most of my differentially > expressed genes are unknown. I was thinking to use Blast2GO to get them at > least clustered in functional groups. I am not sure if that would be the > best approach to find what might be the function of these genes. I also > checked the list of public services that might have this tool, and Berkeley > BOP is listed, but it seems that they no longer have the server or it was > down when I checked (or the link is broken http://galaxy.berkeleybop.org/) > > > Thank you. > > Luciano Hi Luciano The Galaxy team normal direct questions about local Galaxy installs to the galaxy-dev mailing list (CC'd). Have you seen the README file that comes with the Blast2GO wrapper? Perhaps the 'install from toolshed' could be tweaked to make this kind of documentation more visible... In order to use the Blast2GO wrapper in Galaxy you must first install b2g4pipe (Blast2GO for pipelines) and create your own blast2go.loc based on the blast2go.loc.sample given. You will probably also want to install a local Blast2GO database. None of that can be automated at the moment. The blast2go.loc file is a list of Blast2GO profiles, which can be used to set configuration options like weights and thresholds. The primary use is to set the database, so you can have an entry for the public database in Spain, or an entry for a local database which would be MUCH faster. For example, locally we have Blast2GO in our Galaxy with three databases right now - our local one which is fast, and the recent and latest public databases in Spain (which are comparatively slow because of all the network access required). Peter

2 3