August 2014 - galaxy-dev - lists.galaxyproject.org

Adding Reference Genomes, builds.txt vs manual_builds.txt
by Lance Parsons 04 Aug '14

04 Aug '14

I noticed that the instructions at https://wiki.galaxyproject.org/Admin/DataIntegration direct admins to update the builds.txt file when adding a new reference genome to local Galaxy instances. Up to this point I had been actually modifying the manual_builds.txt file and then rebuilding the builds.txt file using the updateucsc.sh script. This allows me to continue to use the updateuscs.sh script without overwriting my changes and also generates a BUILD.len file in the chrom/ directory. It does require some scripting since one needs to add the chromosome lengths to manual_builds.txt, but this is relatively straight forward using an indexed fasta file (.fai) or similar. Perhaps a larger issue is that the manual_builds.txt file is currently under source control in mercurial (instead of being generated by a .sample version). So I have two questions: 1) What would be "best practice" for adding a new genome? Updating builds.txt or manual_builds.txt? 2) Can the manual_builds.txt file be generated from a sample and moved out of source control? (I'll add a Trello card if this seems reasonable). -- Lance Parsons - Scientific Programmer 134 Carl C. Icahn Laboratory Lewis-Sigler Institute for Integrative Genomics Princeton University

1 0

XLS TO CSV
by Mert Mehnur KIRKALI 04 Aug '14

04 Aug '14

Hello, How can i convert xls file to csv file on galaxy ? Is that possible ? Best Regards,Mert.

4 4

creating datatype repository : no module named binary
by Geert Vandeweyer 04 Aug '14

04 Aug '14

Hi all, I'm trying to create a new binary datatype in galaxy, but I'm running into troubles. The current installer is available at https://testtoolshed.g2.bx.psu.edu/view/geert-vandeweyer/cuffquant_datatype When I install the repository to a local server, I get the errors below (no module binary). Since I'm new to defining datatypes, any insights on what I'm doing wrong is welcome :-) Best, Geert tool_shed.util.repository_dependency_util DEBUG 2014-08-04 14:39:07,313 Building repository dependency relationships... 127.0.0.1 - - [04/Aug/2014:14:39:03 +0200] "POST /admin_toolshed/prepare_for_install HTTP/1.1" 200 - "http://127.0.0.1:8080/admin_toolshed/prepare_for_install?tool_shed_url=http…" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/34.0.1847.116 Chrome/34.0.1847.116 Safari/537.36" 127.0.0.1 - - [04/Aug/2014:14:39:10 +0200] "POST /admin_toolshed/repository_installation_status_updates HTTP/1.1" 200 - "http://127.0.0.1:8080/admin_toolshed/prepare_for_install" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/34.0.1847.116 Chrome/34.0.1847.116 Safari/537.36" galaxy.datatypes.registry DEBUG 2014-08-04 14:39:11,362 Loading datatypes from /home/gvandeweyer/galaxy-dist/galaxy-dist/database/tmp/tmp-toolshed-acalpdGx1tEP galaxy.datatypes.registry DEBUG 2014-08-04 14:39:11,363 Exception importing proprietary code file /home/gvandeweyer/galaxy-dist/shed_tools/testtoolshed.g2.bx.psu.edu/repos/geert-vandeweyer/cuffquant_datatype/f7801f8191e0/cuffquant_datatype/binary: No module named binary galaxy.datatypes.registry ERROR 2014-08-04 14:39:11,363 Error importing datatype module galaxy.datatypes.binary: 'module' object has no attribute 'Cuffquant' Traceback (most recent call last): File "/home/gvandeweyer/galaxy-dist/galaxy-dist/lib/galaxy/datatypes/registry.py", line 207, in load_datatypes datatype_class = getattr( module, datatype_class_name ) AttributeError: 'module' object has no attribute 'Cuffquant' galaxy.datatypes.registry ERROR 2014-08-04 14:39:11,364 Error calling method Cuffquant from class <module 'galaxy.datatypes.binary' from '/home/gvandeweyer/galaxy-dist/galaxy-dist/lib/galaxy/datatypes/binary.pyc'>: 'module' object has no attribute 'Cuffquant' Traceback (most recent call last): File "/home/gvandeweyer/galaxy-dist/galaxy-dist/lib/galaxy/datatypes/registry.py", line 349, in load_datatype_sniffers aclass = getattr( module, datatype_class_name )() AttributeError: 'module' object has no attribute 'Cuffquant' -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandeweyer(a)ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer

2 1

Patch for libxslt and libxml packages
by Jan Kanis 04 Aug '14

04 Aug '14

Hi all, I was told that updates to packages owned by devteam should just be sent here, since there is no public repo. I have patches for the libxml2 and libxslt tool_dependency.xml's, that set some more environment variables that my tool needs (specifically, the python lxml package needs some of these). Can they be applied to the official packages? Jan

1 0

August 2014 Galaxy Newsletter
by Dave Clements 01 Aug '14

01 Aug '14

Hello all, The August 2014 Galaxy Update Newsletter <https://wiki.galaxyproject.org/GalaxyUpdates/2014_08> is out. Here's a preview of what's inside: - GCC2015 will be held in Norwich, United Kingdom, 6-8 July 2015 <https://wiki.galaxyproject.org/GalaxyUpdates/2014_08#GCC2015:_Norwich.2C_Un…> - GCC2014 was a tremendous success <https://wiki.galaxyproject.org/GalaxyUpdates/2014_08#GCC2014_Report>. Watch the talks, review the slides, posters, and BoF writeups. - ISMB and BOSC 2014 slides, posters and videos <https://wiki.galaxyproject.org/GalaxyUpdates/2014_08#Galaxy_.40_ISMB_and_BO…> too. - 53 new papers <https://wiki.galaxyproject.org/GalaxyUpdates/2014_08#New_Papers> - One new public server (IM-PET) <https://wiki.galaxyproject.org/GalaxyUpdates/2014_08#New_Public_Servers> - Who's hiring? <https://wiki.galaxyproject.org/GalaxyUpdates/2014_08#Who.27s_Hiring> It turns out that lots of places are. Happy August, Dave Clements and the Galaxy Team <https://wiki.galaxyproject.org/GalaxyTeam> -- http://galaxyproject.org/ http://getgalaxy.org/ http://usegalaxy.org/ https://wiki.galaxyproject.org/

1 0

new_file_path questions
by Geert Vandeweyer 01 Aug '14

01 Aug '14

Hi, I'm running into issues with our galaxy installation regarding the new_file_path setting: 1/ clean up : ======== Can I safely delete all contents in the tmp folder using a cronjob, or manually when galaxy is (not) running. I have gathered almost 700Gb and >72K files on data there apparently. I suppose they are remnants of crashed jobs 2/ Should it be accessible by all nodes ======================= Since I have that much data there, I suppose it does. However, it would make sense to use a seperate (local /tmp/ ?) folder for toolshed actions such as installations. As an example, I tried to install the package_perl from the test toolshed. On our production server, hosting the new_file_path on a NFS share with the 700GB / 72K files, this process ran for over 24 hours, then crashed the main process with the error below. On my development box (laptop, no network drives, single galaxy process), the package installed in under five minutes. Best, Geert ========== tool_shed.galaxy_install.install_manager ERROR 2014-08-01 05:10:20,656 Error installing tool dependency perl version 5.18.1. Traceback (most recent call last): File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py", line 110, in install_and_build_package_via_fabric tool_dependency = self.install_and_build_package( app, tool_shed_repository, tool_dependency, actions_dict ) File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py", line 94, in install_and_build_package initial_download=False ) File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/recipe_manager.py", line 32, in execute_step initial_download=initial_download ) File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/step_handler.py", line 510, in execute_step return_output=False ) File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/install_environment.py", line 148, in handle_command context.flush() File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/scoping.py", line 114, in do return getattr(self.registry(), name)(*args, **kwargs) File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 1718, in flush self._flush(objects) File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 1807, in _flush transaction.rollback(_capture_exception=True) File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 386, in rollback transaction._rollback_impl() File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 414, in _rollback_impl t[1].rollback() File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 2034, in rollback self._do_rollback() File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 2071, in _do_rollback self.connection._rollback_impl() File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1265, in _rollback_impl self._handle_dbapi_exception(e, None, None, None, None) File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1262, in _rollback_impl self.engine.dialect.do_rollback(self.connection) File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/dialects/mysql/base.py", line 1910, in do_rollback connection.rollback() OperationalError: (OperationalError) (2006, 'MySQL server has gone away') None None tool_shed.util.common_install_util ERROR 2014-08-01 05:10:20,743 Error installing tool dependency perl version 5.18.1: Can't reconnect until invalid transaction is rolled back (original cause: InvalidRequestError: Can't reconnect until invalid transaction is rolled back) 'UPDATE tool_dependency SET update_time=%s, status=%s, error_message=%s WHERE tool_dependency.id = %s' [{'status': 'Error', 'error_message': ' File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py", line 110, in install_and_build_package_via_fabric\n tool_dependency = self.install_and_build_package( app, tool_shed_repository, tool_dependency, actions_dict )\n File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py", line 94, in install_and_build_package\n initial_download=False )\n File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/recipe_manager.py", line 32, in execute_step\n initial_download=initial_download )\n File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/step_handler.py", line 510, in execute_step\n return_output=False )\n File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/install_environment.py", line 148, in handle_command\n context.flush()\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/scoping.py", line 114, in do\n return getattr(self.registry(), name)(*args, **kwargs)\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 1718, in flush\n self._flush(objects)\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 1807, in _flush\n transaction.rollback(_capture_exception=True)\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 386, in rollback\n transaction._rollback_impl()\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 414, in _rollback_impl\n t[1].rollback()\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 2034, in rollback\n self._do_rollback()\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 2071, in _do_rollback\n self.connection._rollback_impl()\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1265, in _rollback_impl\n self._handle_dbapi_exception(e, None, None, None, None)\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1262, in _rollback_impl\n self.engine.dialect.do_rollback(self.connection)\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/dialects/mysql/base.py", line 1910, in do_rollback\n connection.rollback()\n\n(OperationalError) (2006, \'MySQL server has gone away\') None None', u'tool_dependency_id': 326L}] Traceback (most recent call last): File "/galaxy/galaxy-dist/lib/tool_shed/util/common_install_util.py", line 554, in install_specified_tool_dependencies from_tool_migration_manager=from_tool_migration_manager ) File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py", line 313, in install_package action_elem=None ) File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py", line 176, in install_via_fabric tool_dependency = self.install_and_build_package_via_fabric( app, tool_shed_repository, tool_dependency, actions_dict ) File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py", line 119, in install_and_build_package_via_fabric remove_installation_path=False ) File "/galaxy/galaxy-dist/lib/tool_shed/util/tool_dependency_util.py", line 423, in handle_tool_dependency_installation_error sa_session.flush() File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/scoping.py", line 114, in do return getattr(self.registry(), name)(*args, **kwargs) File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 1718, in flush self._flush(objects) File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 1789, in _flush flush_context.execute() File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/unitofwork.py", line 331, in execute rec.execute(self) File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/unitofwork.py", line 475, in execute uow File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/persistence.py", line 59, in save_obj mapper, table, update) File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/persistence.py", line 485, in _emit_update_statements execute(statement, params) File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1449, in execute params) File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1584, in _execute_clauseelement compiled_sql, distilled_params File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1651, in _execute_context None, None) File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1645, in _execute_context conn = self._revalidate_connection() File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1044, in _revalidate_connection "Can't reconnect until invalid " StatementError: Can't reconnect until invalid transaction is rolled back (original cause: InvalidRequestError: Can't reconnect until invalid transaction is rolled back) 'UPDATE tool_dependency SET update_time=%s, status=%s, error_message=%s WHERE tool_dependency.id = %s' [{'status': 'Error', 'error_message': ' File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py", line 110, in install_and_build_package_via_fabric\n tool_dependency = self.install_and_build_package( app, tool_shed_repository, tool_dependency, actions_dict )\n File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py", line 94, in install_and_build_package\n initial_download=False )\n File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/recipe_manager.py", line 32, in execute_step\n initial_download=initial_download )\n File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/step_handler.py", line 510, in execute_step\n return_output=False )\n File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/install_environment.py", line 148, in handle_command\n context.flush()\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/scoping.py", line 114, in do\n return getattr(self.registry(), name)(*args, **kwargs)\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 1718, in flush\n self._flush(objects)\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 1807, in _flush\n transaction.rollback(_capture_exception=True)\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 386, in rollback\n transaction._rollback_impl()\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 414, in _rollback_impl\n t[1].rollback()\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 2034, in rollback\n self._do_rollback()\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 2071, in _do_rollback\n self.connection._rollback_impl()\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1265, in _rollback_impl\n self._handle_dbapi_exception(e, None, None, None, None)\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1262, in _rollback_impl\n self.engine.dialect.do_rollback(self.connection)\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/dialects/mysql/base.py", line 1910, in do_rollback\n connection.rollback()\n\n(OperationalError) (2006, \'MySQL server has gone away\') None None', u'tool_dependency_id': 326L}] 143.169.236.220 - - [31/Jul/2014:14:54:50 +0200] "POST /galaxy/admin_toolshed/manage_repositories HTTP/1.1" 500 - "http://biominavm-galaxy.biomina.be/galaxy/admin_toolshed/prepare_for_install" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/34.0.1847.116 Chrome/34.0.1847.116 Safari/537.36" Error - <class 'sqlalchemy.exc.InvalidRequestError'>: This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: Can't reconnect until invalid transaction is rolled back (original cause: InvalidRequestError: Can't reconnect until invalid transaction is rolled back) 'UPDATE tool_dependency SET update_time=%s, status=%s, error_message=%s WHERE tool_dependency.id = %s' [{'status': 'Error', 'error_message': ' File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py", line 110, in install_and_build_package_via_fabric\n tool_dependency = self.install_and_build_package( app, tool_shed_repository, tool_dependency, actions_dict )\n File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py", line 94, in install_and_build_package\n initial_download=False )\n File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/recipe_manager.py", line 32, in execute_step\n initial_download=initial_download )\n File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/step_handler.py", line 510, in execute_step\n return_output=False )\n File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/install_environment.py", line 148, in handle_command\n context.flush()\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/scoping.py", line 114, in do\n return getattr(self.registry(), name)(*args, **kwargs)\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 1718, in flush\n self._flush(objects)\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 1807, in _flush\n transaction.rollback(_capture_exception=True)\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 386, in rollback\n transaction._rollback_impl()\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 414, in _rollback_impl\n t[1].rollback()\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 2034, in rollback\n self._do_rollback()\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 2071, in _do_rollback\n self.connection._rollback_impl()\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1265, in _rollback_impl\n self._handle_dbapi_exception(e, None, None, None, None)\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1262, in _rollback_impl\n self.engine.dialect.do_rollback(self.connection)\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/dialects/mysql/base.py", line 1910, in do_rollback\n connection.rollback()\n\n(OperationalError) (2006, \'MySQL server has gone away\') None None', u'tool_dependency_id': 326L}] URL: http://biominavm-galaxy.biomina.be/galaxy/admin_toolshed/manage_repositories File '/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 190 in handle_request body = method( trans, **kwargs ) File '/galaxy/galaxy-dist/lib/galaxy/web/framework/__init__.py', line 377 in decorator return func( self, trans, *args, **kwargs ) File '/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py', line 724 in manage_repositories **decoded_kwd ) File '/galaxy/galaxy-dist/lib/galaxy/web/framework/__init__.py', line 377 in decorator return func( self, trans, *args, **kwargs ) File '/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py', line 645 in install_tool_shed_repositories reinstalling=reinstalling ) File '/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/repository_util.py', line 629 in install_tool_shed_repository from_tool_migration_manager=False ) File '/galaxy/galaxy-dist/lib/tool_shed/util/common_install_util.py', line 566 in install_specified_tool_dependencies remove_installation_path=False ) File '/galaxy/galaxy-dist/lib/tool_shed/util/tool_dependency_util.py', line 400 in handle_tool_dependency_installation_error tool_shed_repository = tool_dependency.tool_shed_repository File '/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/attributes.py', line 168 in __get__ File '/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/attributes.py', line 453 in get File '/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/strategies.py', line 490 in _load_for_state File '/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/strategies.py', line 531 in _get_ident_for_use_get File '/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/mapper.py', line 1646 in _get_state_attr_by_column File '/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/attributes.py', line 451 in get File '/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/state.py', line 285 in __call__ File '/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/mapper.py', line 1714 in _load_scalar_attributes File '/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/query.py', line 2512 in _load_on_ident File '/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/query.py', line 2184 in one File '/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/query.py', line 2227 in __iter__ File '/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/query.py', line 2240 in _execute_and_instances File '/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/query.py', line 2231 in _connection_from_session File '/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py', line 777 in connection close_with_result=close_with_result) File '/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py', line 781 in _connection_for_bind return self.transaction._connection_for_bind(engine) File '/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py', line 289 in _connection_for_bind self._assert_is_active() File '/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py', line 213 in _assert_is_active % self._rollback_exception InvalidRequestError: This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: Can't reconnect until invalid transaction is rolled back (original cause: InvalidRequestError: Can't reconnect until invalid transaction is rolled back) 'UPDATE tool_dependency SET update_time=%s, status=%s, error_message=%s WHERE tool_dependency.id = %s' [{'status': 'Error', 'error_message': ' File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py", line 110, in install_and_build_package_via_fabric\n tool_dependency = self.install_and_build_package( app, tool_shed_repository, tool_dependency, actions_dict )\n File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/install_manager.py", line 94, in install_and_build_package\n initial_download=False )\n File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/recipe_manager.py", line 32, in execute_step\n initial_download=initial_download )\n File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/step_handler.py", line 510, in execute_step\n return_output=False )\n File "/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/tool_dependencies/recipe/install_environment.py", line 148, in handle_command\n context.flush()\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/scoping.py", line 114, in do\n return getattr(self.registry(), name)(*args, **kwargs)\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 1718, in flush\n self._flush(objects)\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 1807, in _flush\n transaction.rollback(_capture_exception=True)\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 386, in rollback\n transaction._rollback_impl()\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 414, in _rollback_impl\n t[1].rollback()\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 2034, in rollback\n self._do_rollback()\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 2071, in _do_rollback\n self.connection._rollback_impl()\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1265, in _rollback_impl\n self._handle_dbapi_exception(e, None, None, None, None)\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1262, in _rollback_impl\n self.engine.dialect.do_rollback(self.connection)\n File "/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/dialects/mysql/base.py", line 1910, in do_rollback\n connection.rollback()\n\n(OperationalError) (2006, \'MySQL server has gone away\') None None', u'tool_dependency_id': 326L}] CGI Variables ------------- CONTENT_LENGTH: '-1' CONTENT_TYPE: 'application/x-www-form-urlencoded; charset=UTF-8' HTTP_ACCEPT: 'text/html, */*; q=0.01' HTTP_ACCEPT_ENCODING: 'gzip,deflate,sdch' HTTP_ACCEPT_LANGUAGE: 'nl-NL,nl;q=0.8,en-US;q=0.6,en;q=0.4' HTTP_CACHE_CONTROL: 'max-age=0' HTTP_CONNECTION: 'Keep-Alive' HTTP_COOKIE: '__utma=94419599.1127508766.1403299856.1403299856.1405949987.2; __utmz=94419599.1403299856.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); galaxysession=6730c1c7f605dc205ea1a59142b8e9026339899a15206ea6665be6a34bb4fa5a635956369e96da7d' HTTP_HOST: 'biominavm-galaxy.biomina.be' HTTP_ORIGIN: 'http://biominavm-galaxy.biomina.be' HTTP_REFERER: 'http://biominavm-galaxy.biomina.be/galaxy/admin_toolshed/prepare_for_install' HTTP_USER_AGENT: 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/34.0.1847.116 Chrome/34.0.1847.116 Safari/537.36' HTTP_VIA: '1.1 biominavm-galaxy.biomina.be' HTTP_X_REQUESTED_WITH: 'XMLHttpRequest' PATH_INFO: '/admin_toolshed/manage_repositories' REMOTE_ADDR: '143.169.236.220' REQUEST_METHOD: 'POST' SCRIPT_NAME: '/galaxy' SERVER_NAME: 'biominavm-galaxy.biomina.be' SERVER_PORT: '8080' SERVER_PROTOCOL: 'HTTP/1.1' WSGI Variables -------------- application: <paste.recursive.RecursiveMiddleware object at 0xad8c1d0> is_api_request: False paste.cookies: (<SimpleCookie: __utma='94419599.1127508766.1403299856.1403299856.1405949987.2' __utmz='94419599.1403299856.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)' galaxysession='6730c1c7f605dc205ea1a59142b8e9026339899a15206ea6665be6a34bb4fa5a635956369e96da7d'>, '__utma=94419599.1127508766.1403299856.1403299856.1405949987.2; __utmz=94419599.1403299856.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); galaxysession=6730c1c7f605dc205ea1a59142b8e9026339899a15206ea6665be6a34bb4fa5a635956369e96da7d') paste.expected_exceptions: [<class 'paste.httpexceptions.HTTPException'>] paste.httpexceptions: <paste.httpexceptions.HTTPExceptionHandler object at 0xa8df9d0> paste.httpserver.thread_pool: <paste.httpserver.ThreadPool object at 0x2ca6450> paste.recursive.forward: <paste.recursive.Forwarder from /galaxy> paste.recursive.include: <paste.recursive.Includer from /galaxy> paste.recursive.include_app_iter: <paste.recursive.IncluderAppIter from /galaxy> paste.recursive.script_name: '/galaxy' paste.throw_errors: True request_id: 'dc296d0a18b111e484a100163602f2ed' webob._parsed_post_vars: (MultiDict([('operation', 'install'), ('tool_shed_repository_ids', "['dc4272dc4bc65f47']"), ('encoded_kwd', '8466491acfb960dfb39d0377c52cdd7698cac2ef:7b22737461747573223a2022646f6e65222c20226861735f7265706f7369746f72795f646570656e64656e63696573223a2066616c73652c2022696e636c756465735f746f6f6c735f666f725f646973706c61795f696e5f746f6f6c5f70616e656c223a2066616c73652c2022746f6f6c5f736865645f7265706f7369746f72795f696473223a205b2264633432373264633462633635663437225d2c2022736865645f746f6f6c5f636f6e66223a20222e2f736865645f746f6f6c5f636f6e662e786d6c222c2022696e7374616c6c5f7265706f7369746f72795f646570656e64656e63696573223a2066616c73652c2022746f6f6c5f70617468223a20222e2e2f736865645f746f6f6c73222c20227265706f5f696e666f5f6469637473223a205b7b227061636b6167655f7065726c5f355f3138223a205b22436f6e7461696e73206120746f6f6c20646570656e64656e637920646566696e6974696f6e207468617420646f776e6c6f61647320616e6420636f6d70696c65732076657273696f6e20352e31382e78206f662074686520746865207065726c206c61...t([('op...e')])>) webob._parsed_query_vars: (MultiDict([]), '') wsgi process: 'Multithreaded' ------------------------------------------------------------ -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail: geert.vandeweyer(a)ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer

2 1

TestToolShed failure, Exception: History in error state.
by Peter Cock 01 Aug '14

01 Aug '14

Hi all, I'm not sure when this started (having hardly looked at my Tool Shed test results since GCC2014), but I think this is a fairly recent problem with my BLAST RBH tests failing (which has held me back from posting this to the main Tool Shed). This could be some silly mistake in my tar-ball, but usually missing test files and the like get an explicit error. The tests are passing on my GitHub/TravisCI setup (using Twill and the API backend): e.g. https://travis-ci.org/peterjc/galaxy_blast/builds/30592097 Here is the current error (the same for the last few Test Tool Shed runs), https://testtoolshed.g2.bx.psu.edu/view/peterjc/blast_rbh Traceback (most recent call last): File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 108, in test_tool self.do_it( td ) File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 35, in do_it self._verify_outputs( testdef, test_history, shed_tool_id, data_list, galaxy_interactor ) File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 69, in _verify_outputs galaxy_interactor.verify_output( history, output_data, output_testdef=output_testdef, shed_tool_id=shed_tool_id, maxseconds=maxseconds ) File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py", line 53, in verify_output self.wait_for_history( history_id, maxseconds ) File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py", line 107, in wait_for_history self.twill_test_case.wait_for( lambda: not self.__history_ready( history_id ), maxseconds=maxseconds) File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/base/twilltestcase.py", line 2453, in wait_for result = func() File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py", line 107, in <lambda> self.twill_test_case.wait_for( lambda: not self.__history_ready( history_id ), maxseconds=maxseconds) File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py", line 257, in __history_ready return self._state_ready( state, error_msg="History in error state." ) File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py", line 316, in _state_ready raise Exception( error_msg ) Exception: History in error state. Is a more detailed log available which might help debug this? Thanks, Peter As an aside, this looks like the Test Tool Shed is still using the Twill backend for the functional tests?

4 5

bx version conflict causes failure for some tools (e.g. subtract interval tool)
by Langhorst, Brad 01 Aug '14

01 Aug '14

see: https://biostar.usegalaxy.org/p/8095/ I can work around the problem by adding a stdio block to these tools, how should the underlying problem be solved? Brad -- Bradley W. Langhorst, Ph.D. Applications and Product Development Scientist

1 0

Error while installing tools from main toolshed: "HTTPerror : HTTP error 307 : temporary redirect"
by Perley, Danielle 01 Aug '14

01 Aug '14

Hi, I'm trying to install tools from the main toolshed, but I keep getting the same error regardless of the tool that I'm installing HTTPError: HTTP Error 307: Temporary Redirect URL: http://localhost:8080/admin_toolshed/prepare_for_install?tool_shed_url=http… Module weberror.evalexception.middleware:364 in respond view >> app_iter = self.application(environ, detect_start_response) Module paste.recursive:84 in __call__ view >> return self.application(environ, start_response) Module paste.httpexceptions:633 in __call__ view >> return self.application(environ, start_response) Module galaxy.web.framework.base:132 in __call__ view >> return self.handle_request( environ, start_response ) Module galaxy.web.framework.base:190 in handle_request view >> body = method( trans, **kwargs ) Module galaxy.web.framework:377 in decorator view >> return func( self, trans, *args, **kwargs ) Module galaxy.webapps.galaxy.controllers.admin_toolshed:1116 in prepare_for_install view >> updating=updating ) Module tool_shed.util.common_install_util:110 in get_dependencies_for_repository view >> all_repo_info_dict = get_required_repo_info_dicts( trans, tool_shed_url, util.listify( repo_info_dict ) ) Module tool_shed.util.common_install_util:451 in get_required_repo_info_dicts view >> response = urllib2.urlopen( request ).read() Module urllib2:127 in urlopen view >> return _opener.open(url, data, timeout) Module urllib2:410 in open view >> response = meth(req, response) Module urllib2:523 in http_response view >> 'http', request, response, code, msg, hdrs) Module urllib2:442 in error view >> result = self._call_chain(*args) Module urllib2:382 in _call_chain view >> result = func(*args) Module urllib2:608 in http_error_302 view >> new = self.redirect_request(req, fp, code, msg, headers, newurl) Module urllib2:569 in redirect_request view >> raise HTTPError(req.get_full_url(), code, msg, headers, fp) HTTPError: HTTP Error 307: Temporary Redirect I've just installed Galaxy locally for the first time on a Mac OS 10.9.3. The only changes I've made to the universe_wsg.ini file are: setting host=0.0.0.0 (The error also occurred with host=127.0.0.1?) specifying a full path for Tool_depenendency_dir giving myself administrator privileges with admin_users=myemail any ideas? Danielle

1 0

Toolshed tool_dependency.xml git archive configure issue
by Dooley, Damion 01 Aug '14

01 Aug '14

Hi folks, Having a difficult time with coding the "configure" step in this new tool_dependency.xml file, namely that an error "/bin/sh: configure: command not found" results when I try to install my beta development tool from a local test toolshed (which is downloading and installing other tools just fine). <actions> <action type="shell_command">git clone https://github.com/myuser/mygitstuff.git .</action> <action type="shell_command">git reset --hard 1819482db93bf0025c2588b5264cc0ec8c61029c</action> <action type="shell_command">configure --disable-gui --prefix=$INSTALL_DIR</action> <action type="make_install"></action> <action type="move_directory_files"> <source_directory>bin</source_directory> <destination_directory>$INSTALL_DIR/bin</destination_directory> </action> <action type="set_environment"> <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable> </action> </actions> Any hints about why this isn't being found or what the /bin/sh needs to get it going? When I do a git reset --hard, the "configure" file is definitely there, with executable flag set. Any other permission oddities? I've also tried <action type="autoconf"></action> , which yeilds same error. (Skipping that line and doing <action type="make_install"></action> at least finds and runs "make", though it errors out for lack of input data). Regards, Damion Hsiao lab, BC Public Health Microbiology & Reference Laboratory, BC Centre for Disease Control 655 West 12th Avenue, Vancouver, British Columbia, V5Z 4R4 Canada

2 5