galaxy-dev December 2014

galaxy-dev@lists.galaxyproject.org

49 participants
67 discussions

Strange issue with datatypes in Galaxy version of last week
by Lukasse, Pieter 12 Dec '14

12 Dec '14

Hi, Apparently a new datatypes bug has been introduced in the Galaxy version of last week: I have a tool that generates a csv file. Tool configuration is like this: <outputs> <data name="simOut" format="msclust.csv" label="${tool.name} on ${on_string} - SIM file"/> The datatypes has this entry: <datatype extension="msclust.csv" type="galaxy.datatypes.tabular:Tabular" mimetype="text/csv" display_in_upload="true" subclass="true"/> At first Galaxy output seems fine: [cid:image001.png@01CF4847.DE4A1A70] But when I download the file, the downloaded file name will be : Galaxy40-[MsClust_on_data_1_-_SIM_file]- When I go to "edit attributes" and just click "save", then upon downloading again, the file name is correct... Galaxy40-[MsClust_on_data_1_-_SIM_file].msclust.csv . Anyone faced this issue? Thanks and regards, Pieter. Pieter Lukasse Wageningen UR, Plant Research International Departments of Bioscience and Bioinformatics Wageningen Campus, Building 107, Droevendaalsesteeg 1, 6708 PB, Wageningen, the Netherlands +31-317481122; skype: pieter.lukasse.wur http://www.pri.wur.nl<http://www.pri.wur.nl/>

3 5

Nginx + HTTPS
by Nicholas Kline 11 Dec '14

11 Dec '14

Hello. I am trying to configure our local production Galaxy server to always use HTTPS on nginx. When I visit http://localhost I see the main galaxy webpage where I can sign in. However, when I enter my administrator credentials and click Login, the page tries to redirect to HTTPS but then displays an 'unable to connect' error. What changes do I need to make to my nginx.conf file so that our galaxy server always uses HTTPS. My nginx.conf file is included below. Thanks :) ################## # nginx.conf ################## user www-data; worker_processes 4; pid /run/nginx.pid; events { worker_connections 768; # multi_accept on; } http { ## # Basic Settings ## sendfile on; tcp_nopush on; tcp_nodelay on; keepalive_timeout 65; types_hash_max_size 2048; # server_tokens off; # server_names_hash_bucket_size 64; # server_name_in_redirect off; include /etc/nginx/mime.types; default_type application/octet-stream; ## # Logging Settings ## access_log /var/log/nginx/access.log; error_log /var/log/nginx/error.log; ## # Gzip Settings ## gzip on; gzip_http_version 1.1; gzip_vary on; gzip_comp_level 4; gzip_proxied any; gzip_types text/plain text/css application/x-javascript text/xml application/xml text/javascript application/json; gzip_buffers 16 8k; gzip_disable "MSIE [1-6].(?!.*SV1)"; ## # nginx-naxsi config ## # Uncomment it if you installed nginx-naxsi ## #include /etc/nginx/naxsi_core.rules; ## # nginx-passenger config ## # Uncomment it if you installed nginx-passenger ## #passenger_root /usr; #passenger_ruby /usr/bin/ruby; ## # Virtual Host Configs ## include /etc/nginx/conf.d/*.conf; #include /etc/nginx/sites-enabled/*; ## # Galaxy ## upstream galaxy_app { # As defined in galaxy config file universe_wsgi.ini server localhost:8080; server localhost:8081; server localhost:8082; server localhost:8083; server localhost:8084; server localhost:8085; server localhost:8086; server localhost:8087; server localhost:8088; server localhost:8089; } server { # Maximum upload size that can be handled by POST requests through nginx. # Disabled because using FTP instead of web interface. #client_max_body_size 10G; location / { proxy_pass http://galaxy_app; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; # If using SSL, makes Galaxy aware of what type of URL it should generate for external sites like Biomart. proxy_set_header X-URL-SCHEME https; } # nginx is more efficient at serving static content. # expires 24h is for caching purposes. location /static { alias /home/galaxy/galaxy-dist/static; expires 24h; } location /static/style { alias /home/galaxy/galaxy-dist/static/june_2007_style/blue; expires 24h; } location /static/scripts { alias /home/galaxy/galaxy-dist/static/scripts/packed; } location /favicon.ico { alias /home/galaxy/galaxy-dist/static/favicon.ico; } location /robots.txt { alias /home/galaxy/galaxy-dist/static/robots.txt; } # Use nginx to send files (downloads) instead of Galaxy. location /_x_accel_redirect/ { internal; alias /; } } }

2 1

order of tools installed from toolshed
by Wolfgang Maier 10 Dec '14

10 Dec '14

Dear all, I'm currently testing a software package we developed for installation from the testtoolshed. Things seem to work fine mostly, but there is one thing I don't know how to solve (if it can be solved). Our package consists of 14 individual tools, which I uploaded as 14 separate repositories together with a repository suite definition that's taking care of installing all tools. Now, somewhat naively as it seems, I expected the order of repository declarations in the suite definition file to determine the order of the tools in the Galaxy Tools Bar, but, instead, they are added in seemingly random order. So I'm looking for a way to predefine the final order of the tools so that it makes sense for a typical workflow. Any advice how to address this? Thanks for your help, Wolfgang

3 6

Unable to add tools from tool shed
by Ryan G 10 Dec '14

10 Dec '14

I'm following the guide on https://wiki.galaxyproject.org/Admin/Tools/AddToolFromToolShedTutorial to add a tool from the tool shed. The tutorial uses 'bwa_base', but I don't see that. So instead, I'm using 'bwa_mem'. I choose 'Preview and Install', then 'Install to Galaxy'. I then select to have it installed in 'NGS: Mapping' tool panel, then click Install. The page then shows two tools: bwa_mem, and package_bwa_0_7_7. Both of which show 'Error' as the status. Clicking on 'bwa_mem' shows some information with "This repository is not installed correctly (see the Repository Installation Error below)...." What does this mean? And how do I fix it?

3 3

"Repository dependency definitions" broken on Test Tool Shed
by Peter Cock 10 Dec '14

10 Dec '14

Hi all, The "Repository dependency definitions" link is working on the Main Tool Shed, but fails with the following exception on the Test Tool Shed: URL: http://testtoolshed.g2.bx.psu.edu/repository/browse_repository_dependencies File '/var/opt/galaxy/toolshed_data/test_toolshed/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/var/opt/galaxy/toolshed_data/test_toolshed/eggs/Paste-1.7.5.1-py2.7.egg/paste/debug/prints.py', line 106 in __call__ environ, self.app) File '/var/opt/galaxy/toolshed_data/test_toolshed/eggs/Paste-1.7.5.1-py2.7.egg/paste/wsgilib.py', line 543 in intercept_output app_iter = application(environ, replacement_start_response) File '/var/opt/galaxy/toolshed_data/test_toolshed/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/var/opt/galaxy/toolshed_data/test_toolshed/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/var/opt/galaxy/toolshed_data/test_toolshed/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '/var/opt/galaxy/toolshed_data/test_toolshed/lib/galaxy/web/framework/base.py', line 190 in handle_request body = method( trans, **kwargs ) File '/var/opt/galaxy/toolshed_data/test_toolshed/lib/galaxy/webapps/tool_shed/controllers/repository.py', line 765 in browse_repository_dependencies return self.repository_dependencies_grid( trans, **kwd ) File '/var/opt/galaxy/toolshed_data/test_toolshed/lib/galaxy/web/framework/helpers/grids.py', line 306 in __call__ kwargs=kwargs ) File '/var/opt/galaxy/toolshed_data/test_toolshed/lib/galaxy/web/framework/webapp.py', line 772 in fill_template return self.fill_template_mako( filename, **kwargs ) File '/var/opt/galaxy/toolshed_data/test_toolshed/lib/galaxy/web/framework/webapp.py', line 787 in fill_template_mako return template.render( **data ) File '/var/opt/galaxy/toolshed_data/test_toolshed/eggs/Mako-0.4.1-py2.7.egg/mako/template.py', line 296 in render return runtime._render(self, self.callable_, args, data) File '/var/opt/galaxy/toolshed_data/test_toolshed/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 660 in _render **_kwargs_for_callable(callable_, data)) File '/var/opt/galaxy/toolshed_data/test_toolshed/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 692 in _render_context _exec_template(inherit, lclcontext, args=args, kwargs=kwargs) File '/var/opt/galaxy/toolshed_data/test_toolshed/eggs/Mako-0.4.1-py2.7.egg/mako/runtime.py', line 718 in _exec_template callable_(context, *args, **kwargs) File '/var/opt/galaxy/toolshed_data/test_toolshed/database/compiled_templates/community/base.mako.py', line 66 in render_body __M_writer(unicode(next.body())) File '/var/opt/galaxy/toolshed_data/test_toolshed/database/compiled_templates/community/grid_base.mako.py', line 91 in render_body __M_writer(unicode(self.load())) File '/var/opt/galaxy/toolshed_data/test_toolshed/database/compiled_templates/community/grid_base.mako.py', line 120 in render_load __M_writer(unicode( h.dumps( self.get_grid_config( embedded=embedded, insert=insert ) ) )) File '/var/opt/galaxy/toolshed_data/test_toolshed/database/compiled_templates/community/grid_base.mako.py', line 303 in render_get_grid_config value = column.get_value( trans, grid, item ) File '/var/opt/galaxy/toolshed_data/test_toolshed/lib/tool_shed/grids/repository_grids.py', line 1505 in get_value required_repository_metadata_id = trans.security.encode_id( required_repository_metadata.id ) AttributeError: 'NoneType' object has no attribute 'id' Regards, Peter

1 0

Manage jobs interface error
by C. Ch. 10 Dec '14

10 Dec '14

Hi all, I have had the following problem while trying to access the "manage jobs" page. File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker)File '/home/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response)File '/home/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response)File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response )File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 190 in handle_request body = method( trans, **kwargs )File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/decorators.py', line 87 in decorator return func( self, trans, *args, **kwargs )File '/home/galaxy/galaxy-dist/lib/galaxy/web/base/controllers/admin.py', line 1110 in jobs job_lock = trans.app.job_manager.job_handler.job_queue.job_lockAttributeError: 'NoopQueue' object has no attribute 'job_lock' It persists through galaxy restarts (I'm installing a production server so restarts happen several times a day). At some point I could access the interface but now I can not. I think I could access the interface when I had the [server:main] still in the galaxy.ini. I have two web servers (web0 and web1) and two handlers (handler0 and handler1) in galaxy.ini, as shown in the wiki documentation. I also have the job_conf.xml defining a local plugin with the two handlers and the local destination. How can I get the job manager interface back? Thanks!Cristian

2 4

Tool Shed Categories
by Alam, Khalid K. (MU-Student) 10 Dec '14

10 Dec '14

Hi; I’m getting ready to submit several (5) tools for processing data related to combinatorial selection technologies (aptamer and (deoxy)ribozyme selections, phage display, etc.) and would like to propose a new category for the Tool Shed entitled “Combinatorial Selections” where these repositories can be published. Khalid Kamal Alam University of Missouri Department of Biochemistry 415 Life Sciences Center 1201 Rollins St. Columbia, MO 65211 <mailto:khalidkalam@mail.missouri.edu>KhalidKAlam<mailto:KhalidKAlam@mail.missouri.edu>@mail.missouri.edu<mailto:kkafy3@mail.missouri.edu> @BiochemPhD

3 6

Failure in uploading datasets
by David Matthews 09 Dec '14

09 Dec '14

HI, We have had a copy of Galaxy working on a login node on our cluster here for a few weeks. We have tried to get it to work with the queue but could not and so reverted back to the login node for now. However, when we try to upload data (via URL or from the desktop) we now get this error: Traceback (most recent call last): File "/isys/galaxy/galaxy-dist/tools/data_source/upload.py", line 10, in <module> import galaxy.model File "/isys/galaxy/galaxy-dist/lib/galaxy/model/__init__.py", line 27, in <module> import galaxy.datatypes.registry File "/isys/galaxy/galaxy-dist/lib/galaxy/datatypes/registry.py", line 10, in <module> import data File "/isys/galaxy/galaxy-dist/lib/galaxy/datatypes/data.py", line 2, in <module> import metadata File "/isys/galaxy/galaxy-dist/lib/galaxy/datatypes/metadata.py", line 22, in <module> from galaxy.util import listify File "/isys/galaxy/galaxy-dist/lib/galaxy/util/__init__.py", line 38, in <module> eggs.require( 'docutils' ) File "/isys/galaxy/galaxy-dist/lib/galaxy/eggs/__init__.py", line 411, in require return c[req.project_name].require() File "/isys/galaxy/galaxy-dist/lib/galaxy/eggs/__init__.py", line 239, in require dists = self.resolve() File "/isys/galaxy/galaxy-dist/lib/galaxy/eggs/__init__.py", line 192, in resolve if e.args[1].key != e.args[0].key: IndexError: tuple index out of range We have not seen this before and are stuck. Any suggestions welcome! Best Wishes, David. _________________________________ Dr David A. Matthews Senior Lecturer in Virology Room E49 Department of Cellular and Molecular Medicine, School of Medical Sciences University Walk, University of Bristol Bristol. BS8 1TD U.K. Tel. +44 117 3312058 Fax. +44 117 3312091 D.A.Matthews(a)bristol.ac.uk <mailto:D.A.Matthews@bristol.ac.uk>

3 2

Custom Builds not accessible for user
by Ulf Schaefer 08 Dec '14

08 Dec '14

Dear Galaxy Dev One of our users has managed to lock himself out of his Custom Builds. An 'Internal Server Error' appears and this shows up in the web process log: --- Error - <type 'exceptions.IOError'>: [Errno 2] No such file or directory: '/phengs/galaxy/database/files/000/127/dataset_127764.dat' URL: http://158.119.147.85/user/dbkeys File '/phengs/hpc_storage/home/galaxy_hpc/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/phengs/hpc_storage/home/galaxy_hpc/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/phengs/hpc_storage/home/galaxy_hpc/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/phengs/hpc_storage/home/galaxy_hpc/galaxy-dist/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '/phengs/hpc_storage/home/galaxy_hpc/galaxy-dist/lib/galaxy/web/framework/base.py', line 190 in handle_request body = method( trans, **kwargs ) File '/phengs/hpc_storage/home/galaxy_hpc/galaxy-dist/lib/galaxy/web/framework/__init__.py', line 98 in decorator return func( self, trans, *args, **kwargs ) File '/phengs/hpc_storage/home/galaxy_hpc/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/user.py', line 1680 in dbkeys chrom_count = int( open( chrom_count_dataset.file_name ).readline() ) IOError: [Errno 2] No such file or directory: '/phengs/galaxy/database/files/000/127/dataset_127764.dat' CGI Variables ------------- CONTENT_LENGTH: '0' HTTP_ACCEPT: 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' HTTP_ACCEPT_ENCODING: 'gzip' HTTP_ACCEPT_LANGUAGE: 'en-GB,en;q=0.8,en-US;q=0.6,de;q=0.4' HTTP_CONNECTION: 'Keep-Alive' HTTP_COOKIE: 'galaxysession=c6ca0ddb55be603a1e24419432331618f5a7191d55ca4813cd28cd8c84abb88b122c85b9044b10ba' HTTP_HOST: '158.119.147.85' HTTP_REFERER: 'http://158.119.147.85/root' HTTP_USER_AGENT: 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.71 Safari/537.36' HTTP_VIA: '1.1 TMGCOL001' HTTP_X_FORWARDED_FOR: '158.119.150.18' HTTP_X_FORWARDED_HOST: '158.119.147.85' HTTP_X_FORWARDED_SERVER: 'bioinformatics-galaxy.phe.org.uk' ORGINAL_HTTP_HOST: 'localhost:8084' ORGINAL_REMOTE_ADDR: '127.0.0.1' PATH_INFO: '/user/dbkeys' REMOTE_ADDR: '158.119.150.18' REQUEST_METHOD: 'GET' SERVER_NAME: '0.0.0.0' SERVER_PORT: '8084' SERVER_PROTOCOL: 'HTTP/1.1' WSGI Variables -------------- application: <paste.recursive.RecursiveMiddleware object at 0x8fe2c90> is_api_request: False paste.cookies: (<SimpleCookie: galaxysession='c6ca0ddb55be603a1e24419432331618f5a7191d55ca4813cd28cd8c84abb88b122c85b9044b10ba'>, 'galaxysession=c6ca0ddb55be603a1e24419432331618f5a7191d55ca4813cd28cd8c84abb88b122c85b9044b10ba') paste.expected_exceptions: [<class 'paste.httpexceptions.HTTPException'>] paste.httpexceptions: <paste.httpexceptions.HTTPExceptionHandler object at 0x8fe2510> paste.httpserver.thread_pool: <paste.httpserver.ThreadPool object at 0x9332210> paste.recursive.forward: <paste.recursive.Forwarder from /> paste.recursive.include: <paste.recursive.Includer from /> paste.recursive.include_app_iter: <paste.recursive.IncluderAppIter from /> paste.recursive.script_name: '' paste.throw_errors: True request_id: '965c3b007ecc11e4b771005056983c1e' webob._parsed_query_vars: (MultiDict([]), '') wsgi process: 'Multithreaded' --- The file it is looking for (dataset_127764.dat) is indeed not there and I suspect that once it contained a genome of a custom build that this user had added. How it was removed I cannot tell. I don't need to get this particular genome back, but I need to stop Galaxy from looking for it, so that the user can get back into the interface. Does anyone know how I can wipe all Custom Builds for a given user? Thanks a lot for your help Best regards Ulf -- Ulf Schaefer, PhD Bioinformatics Scientist Bioinformatics Unit - Infectious Disease Informatics Public Health England 61 Colindale Ave London NW9 5EQ ulf.schaefer(a)phe.gov.uk Tel: 020 832 77296 http://www.gov.uk/phe Protecting and improving the nation’s health ************************************************************************** The information contained in the EMail and any attachments is confidential and intended solely and for the attention and use of the named addressee(s). It may not be disclosed to any other person without the express authority of Public Health England, or the intended recipient, or both. If you are not the intended recipient, you must not disclose, copy, distribute or retain this message or any part of it. This footnote also confirms that this EMail has been swept for computer viruses by Symantec.Cloud, but please re-sweep any attachments before opening or saving. http://www.gov.uk/PHE **************************************************************************

2 3

No handlers could be found for logger "galaxy.util.json"
by C. Ch. 07 Dec '14

07 Dec '14

Hi All, While trying to use the data_manager_fetch_genome_all_fasta to create a reference genome I get the following error: No handlers could be found for logger "galaxy.util.json" I'm on the latest version of galaxy. Any idea what is wrong? Thanks!Cristian

2 5

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

galaxy-dev December 2014