- galaxy-dev - lists.galaxyproject.org

clustalomega from toolshed installation issue
by Isabelle Phan 07 Oct '14

07 Oct '14

Hello, I installed clustalomega from the galaxy main toolshed using the admin interface of our local galaxy install. When I run the tool, I get this message: Dataset generation errors Dataset 55: co_alignment.fasta Tool execution generated the following error message: Error invoking command: clustalo --force --threads=1 --maxnumseq=300000 --maxseqlen=15000 -o /opt/galaxy-dist/database/files/000/dataset_158.dat -l /opt/galaxy-dist/database/files/000/dataset_159.dat -v -i /opt/galaxy-dist/database/files/000/dataset_153.dat [Errno 2] No such file or directory I've tried reloading my input file, reloading the tool's configuration, resetting the metadata on the clustalomega repository, deleting the tool completely and re-installing. No luck. If I try to rerun the job again, I get non-sensical errors that seem to originate from other tools, as if the database was completely garbled. Each of those tools work fine. Only clustalomega behaves oddly. I have no access to our galaxy server, so any solution would have to be implemented from the admin interface. thanks for any hints, Isabelle

2 2

Apache proxy, SSL, and UCSC get data
by Lance Parsons 07 Oct '14

07 Oct '14

I'm having trouble after setting up SSL on our Apache proxy for Galaxy. When I try to import data from UCSC, Galaxy tries to go to the http (non-ssl) url for UCSC and thus generates a security error in the browser and the request is blocked. I have added RequestHeader set X-URL-SCHEME https to the httpd.conf and that appears to be working. Is there something else I need to do? Should I patch the data_source tools to use https in the xml tool definitions? Thanks for any advice. -- Lance Parsons - Scientific Programmer 134 Carl C. Icahn Laboratory Lewis-Sigler Institute for Integrative Genomics Princeton University

1 0

Central tool shed installations fail due to redirect
by Kandalaft, Iyad 07 Oct '14

07 Oct '14

Hi Everyone I was recently trying to update tools that I installed from the central tool shed. However, it seems that a temporary redirect on the central tool shed server is causing my galaxy installation to throw a stack trace. Anyone else encountered this? Thanks in advance. URL: http://galaxy-devtest.biodiversity.agr.gc.ca/admin_toolshed/update_to_chang… Module galaxy.web.framework.middleware.error:149 in __call__ >> <http://galaxy-devtest.biodiversity.agr.gc.ca/admin_toolshed/update_to_chang…> app_iter = self.application(environ, sr_checker) Module paste.lint:170 in lint_app >> <http://galaxy-devtest.biodiversity.agr.gc.ca/admin_toolshed/update_to_chang…> iterator = application(environ, start_response_wrapper) Module paste.recursive:84 in __call__ >> <http://galaxy-devtest.biodiversity.agr.gc.ca/admin_toolshed/update_to_chang…> return self.application(environ, start_response) Module galaxy.web.framework.middleware.remoteuser:92 in __call__ >> <http://galaxy-devtest.biodiversity.agr.gc.ca/admin_toolshed/update_to_chang…> return self.app( environ, start_response ) Module paste.httpexceptions:633 in __call__ >> <http://galaxy-devtest.biodiversity.agr.gc.ca/admin_toolshed/update_to_chang…> return self.application(environ, start_response) Module galaxy.web.framework.base:132 in __call__ >> <http://galaxy-devtest.biodiversity.agr.gc.ca/admin_toolshed/update_to_chang…> return self.handle_request( environ, start_response ) Module galaxy.web.framework.base:190 in handle_request >> <http://galaxy-devtest.biodiversity.agr.gc.ca/admin_toolshed/update_to_chang…> body = method( trans, **kwargs ) Module galaxy.web.framework:377 in decorator >> <http://galaxy-devtest.biodiversity.agr.gc.ca/admin_toolshed/update_to_chang…> return func( self, trans, *args, **kwargs ) Module galaxy.webapps.galaxy.controllers.admin_toolshed:1926 in update_to_changeset_revision >> <http://galaxy-devtest.biodiversity.agr.gc.ca/admin_toolshed/update_to_chang…> return self.prepare_for_install( trans, **new_kwd ) Module galaxy.web.framework:377 in decorator >> <http://galaxy-devtest.biodiversity.agr.gc.ca/admin_toolshed/update_to_chang…> return func( self, trans, *args, **kwargs ) Module galaxy.webapps.galaxy.controllers.admin_toolshed:1122 in prepare_for_install >> <http://galaxy-devtest.biodiversity.agr.gc.ca/admin_toolshed/update_to_chang…> updating=updating ) Module tool_shed.util.common_install_util:110 in get_dependencies_for_repository >> <http://galaxy-devtest.biodiversity.agr.gc.ca/admin_toolshed/update_to_chang…> all_repo_info_dict = get_required_repo_info_dicts( trans, tool_shed_url, util.listify( repo_info_dict ) ) Module tool_shed.util.common_install_util:451 in get_required_repo_info_dicts >> <http://galaxy-devtest.biodiversity.agr.gc.ca/admin_toolshed/update_to_chang…> response = urllib2.urlopen( request ).read() Module urllib2:126 in urlopen >> <http://galaxy-devtest.biodiversity.agr.gc.ca/admin_toolshed/update_to_chang…> return _opener.open(url, data, timeout) Module urllib2:397 in open >> <http://galaxy-devtest.biodiversity.agr.gc.ca/admin_toolshed/update_to_chang…> response = meth(req, response) Module urllib2:510 in http_response >> <http://galaxy-devtest.biodiversity.agr.gc.ca/admin_toolshed/update_to_chang…> 'http', request, response, code, msg, hdrs) Module urllib2:429 in error >> <http://galaxy-devtest.biodiversity.agr.gc.ca/admin_toolshed/update_to_chang…> result = self._call_chain(*args) Module urllib2:369 in _call_chain >> <http://galaxy-devtest.biodiversity.agr.gc.ca/admin_toolshed/update_to_chang…> result = func(*args) Module urllib2:595 in http_error_302 >> <http://galaxy-devtest.biodiversity.agr.gc.ca/admin_toolshed/update_to_chang…> new = self.redirect_request(req, fp, code, msg, headers, newurl) Module urllib2:556 in redirect_request >> <http://galaxy-devtest.biodiversity.agr.gc.ca/admin_toolshed/update_to_chang…> raise HTTPError(req.get_full_url(), code, msg, headers, fp) HTTPError: HTTP Error 307: Temporary Redirect Iyad Kandalaft Bioinformatics Programmer Microbial Biodiversity Bioinformatics Science & Technology Branch Agriculture & Agri-Food Canada Iyad.Kandalaft(a)agr.gc.ca | (613) 759-1228

1 1

installing R bioconductor package DNAcopy gfortran setup_r_environment
by Stef van Lieshout 07 Oct '14

07 Oct '14

Hi all, I'm trying to install R/bioconductor packages in galaxy and have trouble with some of them. I made a dependency package ("package_qdnaseq_1_0_5" in the testtoolshed) that should install several R (bioconductor) packages using "setup_r_environment" and for each package the <package>http://link/to/package.tar.gz</package> structure. It works for many but fails for the package "DNAcopy" with the error below complaining about gfortran, but that seems to be installed on my systems (tested two of them, ubuntu 12.04). When I install this with "install.packages" in my default R installation it installs without problems.. Anyway know how to deal with or debug this? Thanks, Stef ----- install_environment.STDOUT DEBUG 2014-10-07 14:04:13,176 Error in dyn.load(file, DLLpath = DLLpath, ...) : unable to load shared object '/home/stef/BASE/tools/galaxy/galaxy-dist/tool-dependency/qdnaseq/1.0.5/stef/package_qdnaseq_1_0_5/42096296e06f/DNAcopy/libs/DNAcopy.so': /home/stef/BASE/tools/galaxy/galaxy-dist/tool-dependency/qdnaseq/1.0.5/stef/package_qdnaseq_1_0_5/42096296e06f/DNAcopy/libs/DNAcopy.so: undefined symbol: _gfortran_os_error Error: loading failed Execution halted Error in dyn.load(file, DLLpath = DLLpath, ...) : ERROR: loading failed * removing ‘/home/stef/BASE/tools/galaxy/galaxy-dist/tool-dependency/qdnaseq/1.0.5/stef/package_qdnaseq_1_0_5/42096296e06f/DNAcopy’

1 0

CfP: Reproducible Science Hackathon - Curriculum & Workflow Development
by Tracy Teal 06 Oct '14

06 Oct '14

It would be great to have someone from the Galaxy community involved in this reproducible science hack-a-thon, as Galaxy is such a positive contribution to workflows in reproducible science. ================================== Reproducible Science Hackathon: Curriculum & Workflow Development ================================== Goal: Develop a short course curriculum for tools, resources, and practices for reproducible science. Location: NESCent, Durham, NC Dates and times: December 8 - 11, 2014; 9 am - 5pm each day To apply: http://goo.gl/GzS118 ; deadline October 10, 2014. Successful applicants will be notified by October 17th. Synopsis Making science more reproducible has enormous potential to accelerate scientific advance, including for practicing individuals. Despite this, the tools and approaches that are already available are rarely taught. To address this, we are organizing a 4-day workshop aimed at developing, and later teaching, a short course curriculum for tools, resources, and practices for reproducible science. A part of the workshop will also be devoted to addressing gaps that hinder the broad adoption of such resources. The event will be held December 8-11,2014, in Durham, NC, at the National Evolutionary Synthesis Center (NESCent). We aim to assemble a diverse and interdisciplinary group of participants, and invite those interested to apply by Oct 10 at http://goo.gl/GzS118. The full Call for Participation can be found here: http://goo.gl/IzACr3

1 0

Get Data - url upload (connection timed out)
by Elizabeth Chia 03 Oct '14

03 Oct '14

Dear * I have a local galaxy instance installed on a cluster. I've been trying to upload data by URL but kept getting this error: *"unable to fetch <url of data> [Errno socket error] [Errno 110] connection timed out". *I've tried with and without file compression format and still gives me that error. Am I suppose to modify/edit any particular config file? Can't seem to find a similar thread or answer to this but perhaps I may have missed. Apologies if I have missed. Would appreciate if anyone can please advise. Thanks. Best regards liz

2 2

Simple standard for API use of a global user/key that all loaded tools can draw upon?
by Dooley, Damion 02 Oct '14

02 Oct '14

Do any of you see the need to establish a way for tools to use a global (current site) API url/key combination? From a security perspective I'm assuming this is basically ok since tools run under the galaxy user anyways (and can directly read and write files to their heart's content). I understand that a particular user's API key could help limit what they have access to. My tool however wants to do some prepatory form display and post-job work on behalf of the Galaxy system in addition to working with user-centric data; I presume others have similar use-case needs. I think it would amount to having a standard GALAXY_TOOL_API_URL and GALAXY_TOOL_API_KEY managed in universe_wsgi.ini (left blank if not desired by sys admin). The key would be generated by a galaxy admin using a real admin user id. These would be available as python or shell environment variables when it came time to execute a tool's <command interpreter="python"> etc. work. The wee other detail is that I'm using <code file="versioned_data_form.py" /> in my tool xml because there is API information I have to retrieve to set up the form with. So the url and key need to be accessible in that execution environment too?!! Am I missing the boat; alternately has this already been talked about?! Thanks for advice ... Damion Hsiao lab, BC Public Health Microbiology & Reference Laboratory, BC Centre for Disease Control 655 West 12th Avenue, Vancouver, British Columbia, V5Z 4R4 Canada

2 3

Re: [galaxy-dev] Internal Server error when attemptng to use Admin Tab
by John Chilton 02 Oct '14

02 Oct '14

Thanks for the additional details - I am responding on list for people who may encounter similar problems in the future. The job running code will provide both core tools and tool shed tools with the same PYTHONPATH - but some tools might setup a Python virtualenv for instance which would replace the PYTHONPATH completely I believe. Since gsummary is not in the tool shed - it will not be able to find any of the tool shed dependencies. These rpy issues are tough ones. It should be possible to hack something together if you change the requirements in gsummary to be package requirements for R and Rpy and then find working env.sh files in tool_dependency_dir and copy them to tool_dependency_dir/R/2.11/env.sh and tool_dependency_dir/Rpy/1.0.3/env.sh respectively. The tool shed is suppose to solve all of these pesky dependency problems - but not all the tools have been migrated and fixed yet. Thanks for your patience and for using Galaxy! -John On Wed, Oct 1, 2014 at 1:36 PM, David Bernick <dbernick(a)soe.ucsc.edu> wrote: > Hi John - > > Yes, we are using MySQL. This event happened as a result of attempting to add/delete tools from our instance (Rpy_1_0_3 and R_2_11). > We ended up reloading the database from our nightly backup, and that got us going again. > > The issue I was trying to solve when I "broke" the database was Summary_statistics not being able to find its Rpy module. I now have R and Rpy installed, such that tool-shed tools are happy with it, but gsummary.py is still unable to find Rpy. We are attempting to solve this, but are as yet without a fix. In the meantime, I have installed datamash, and that is working, It seems that tool-shed tools have a different PYTHONPATH than galaxy-dist tools? > > As it happens, we also had a few cases with multiple instances of tool-shed tools being installed. Since this server is brand new, it was fine to delete them. I suppose that maybe we are now fitting in MySQL BLOBs now?? > > thanks again - > David > > > > On Oct 1, 2014, at 9:44 AM, John Chilton <jmchilton(a)gmail.com> wrote: > >> Hello David, >> >> We don't have a workaround yet - but we (Dan did most of the >> investigating) understand the problem I think. You are using a MySQL >> database I assume? (Ignore the rest of this e-mail if not). I think >> MySQL has lower limits on the size of BLOBs that can be stored in the >> database than postgres - and we think the tool shed is tried to >> storing a really large blob of metadata in the database and it got >> truncated and now Galaxy is failing because it cannot load that entry >> as JSON. >> >> The work around would be something like trying to find the truncated >> column in the tool_repositories table - deleting it and references - >> changing that column to a LONGBLOB and reinstalling that tool. >> >> I have created a Trello card for this bug report. >> >> https://trello.com/c/RbW6pOd2 >> >> It is said a lot but we strongly encourage not using MySQL. >> >> It might be worth trying this patch >> (https://bitbucket.org/galaxy/galaxy-central/pull-request/511/allow-model-ob…) >> from Dan as well. I am not sure what the tool shed would do if it >> cannot find the metadata though. >> >> -John >> >> On Mon, Sep 29, 2014 at 11:41 AM, David Bernick <dbernick(a)soe.ucsc.edu> wrote: >>> Good morning - >>> >>> I have acquired an Internal server error on our local galaxy install. It >>> seems to be an unterminated string. How do I resolve this? >>> thanks in advance - >>> David Bernick >>> UCSC >>> >>> Here is the relevant portion of the log (I hope). >>> >>> Error - <type 'exceptions.ValueError'>: Unterminated string starting at: >>> line 1 column 65528 (char 65528) >>> URL: http://campusrocks.cse.ucsc.edu:8080/admin/index >>> File >>> '/projects/galaxy/bin/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', >>> line 149 in __call__ >>> app_iter = self.application(environ, sr_checker) >>> File >>> '/projects/galaxy/bin/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py', >>> line 84 in __call__ >>> return self.application(environ, start_response) >>> File >>> '/projects/galaxy/bin/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py', >>> line 633 in __call__ >>> return self.application(environ, start_response) >>> File '/projects/galaxy/bin/galaxy-dist/lib/galaxy/web/framework/base.py', >>> line 132 in __call__ >>> return self.handle_request( environ, start_response ) >>> File '/projects/galaxy/bin/galaxy-dist/lib/galaxy/web/framework/base.py', >>> line 190 in handle_request >>> body = method( trans, **kwargs ) >>> File >>> '/projects/galaxy/bin/galaxy-dist/lib/galaxy/web/framework/__init__.py', >>> line 383 in decorator >>> return func( self, trans, *args, **kwargs ) >>> File >>> '/projects/galaxy/bin/galaxy-dist/lib/galaxy/web/base/controllers/admin.py', >>> line 34 in index >>> installing_repository_ids = >>> suc.get_ids_of_tool_shed_repositories_being_installed( trans.app, >>> as_string=True ) >>> File >>> '/projects/galaxy/bin/galaxy-dist/lib/tool_shed/util/shed_util_common.py', >>> line 277 in get_ids_of_tool_shed_repositories_being_installed >>> app.install_model.ToolShedRepository.status == loading_datatypes_status ) >>> ): >>> File >>> '/projects/galaxy/bin/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/query.py', >>> line 2348 in instances >>> File >>> '/projects/galaxy/bin/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/mapper.py', >>> line 2120 in _instance >>> File >>> '/projects/galaxy/bin/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/mapper.py', >>> line 1974 in populate_state >>> File >>> '/projects/galaxy/bin/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/strategies.py', >>> line 150 in fetch_col >>> File >>> '/projects/galaxy/bin/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/types.py', >>> line 658 in process >>> File '/projects/galaxy/bin/galaxy-dist/lib/galaxy/model/custom_types.py', >>> line 52 in process_result_value >>> return json_decoder.decode( str( _sniffnfix_pg9_hex(value) ) ) >>> File '/usr/lib64/python2.6/json/decoder.py', line 319 in decode >>> obj, end = self.raw_decode(s, idx=_w(s, 0).end()) >>> File '/usr/lib64/python2.6/json/decoder.py', line 336 in raw_decode >>> obj, end = self._scanner.iterscan(s, **kw).next() >>> File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan >>> rval, next_pos = action(m, context) >>> File '/usr/lib64/python2.6/json/decoder.py', line 183 in JSONObject >>> value, end = iterscan(s, idx=end, context=context).next() >>> File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan >>> rval, next_pos = action(m, context) >>> File '/usr/lib64/python2.6/json/decoder.py', line 217 in JSONArray >>> value, end = iterscan(s, idx=end, context=context).next() >>> File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan >>> rval, next_pos = action(m, context) >>> File '/usr/lib64/python2.6/json/decoder.py', line 183 in JSONObject >>> value, end = iterscan(s, idx=end, context=context).next() >>> File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan >>> rval, next_pos = action(m, context) >>> File '/usr/lib64/python2.6/json/decoder.py', line 217 in JSONArray >>> value, end = iterscan(s, idx=end, context=context).next() >>> File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan >>> rval, next_pos = action(m, context) >>> File '/usr/lib64/python2.6/json/decoder.py', line 183 in JSONObject >>> value, end = iterscan(s, idx=end, context=context).next() >>> File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan >>> rval, next_pos = action(m, context) >>> File '/usr/lib64/python2.6/json/decoder.py', line 217 in JSONArray >>> value, end = iterscan(s, idx=end, context=context).next() >>> File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan >>> rval, next_pos = action(m, context) >>> File '/usr/lib64/python2.6/json/decoder.py', line 217 in JSONArray >>> value, end = iterscan(s, idx=end, context=context).next() >>> File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan >>> rval, next_pos = action(m, context) >>> File '/usr/lib64/python2.6/json/decoder.py', line 155 in JSONString >>> return scanstring(match.string, match.end(), encoding, strict) >>> ValueError: Unterminated string starting at: line 1 column 65528 (char >>> 65528) >>> >>> >>> ___________________________________________________________ >>> Please keep all replies on the list by using "reply all" >>> in your mail client. To manage your subscriptions to this >>> and other Galaxy lists, please use the interface at: >>> http://lists.bx.psu.edu/ >>> >>> To search Galaxy mailing lists use the unified search at: >>> http://galaxyproject.org/search/mailinglists/ >

2 1

'KeyError' in workflow extraction when using newer tool version
by Peter Briggs 02 Oct '14

02 Oct '14

Dear all One of my users has encountered a problem with extracting a workflow from a history in our local Galaxy installation (tag latest_2014.08.11, changeset 14342:db520d154716). The history was created with a version of a local tool which has since been updated, and the extraction page warns that the newer version of the tool will be used for those datasets i.e.: Dataset was created with tool version "0.0.4", but workflow extraction will use version "0.0.7". However when the extraction is run it fails immediately: the user sees the "Internal Server Error" page, and in the log I see: ===== Error - <type 'exceptions.KeyError'>: 'platform' URL: https://galaxy.ls.manchester.ac.uk/workflow/build_from_current_history File '/home/galaxy/production/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/home/galaxy/production/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/home/galaxy/production/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/home/galaxy/production/galaxy-dist/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '/home/galaxy/production/galaxy-dist/lib/galaxy/web/framework/base.py', line 190 in handle_request body = method( trans, **kwargs ) File '/home/galaxy/production/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/workflow.py', line 1220 in build_from_current_history workflow_name=workflow_name File '/home/galaxy/production/galaxy-dist/lib/galaxy/workflow/extract.py', line 30 in extract_workflow steps = extract_steps( trans, history=history, job_ids=job_ids, dataset_ids=dataset_ids, dataset_collection_ids=dataset_collection_ids ) File '/home/galaxy/production/galaxy-dist/lib/galaxy/workflow/extract.py', line 104 in extract_steps tool_inputs, associations = step_inputs( trans, job ) File '/home/galaxy/production/galaxy-dist/lib/galaxy/workflow/extract.py', line 271 in step_inputs associations = __cleanup_param_values( tool.inputs, param_values ) File '/home/galaxy/production/galaxy-dist/lib/galaxy/workflow/extract.py', line 330 in __cleanup_param_values cleanup( "", inputs, values ) File '/home/galaxy/production/galaxy-dist/lib/galaxy/workflow/extract.py', line 327 in cleanup group_values = values[input.name] KeyError: 'platform' ===== 'platform' is a conditional parameter that was introduced into the tool between the two versions. It looks like when tying to use the newer verison of the new tool, that the new parameter is not getting picked up. Is this expected behaviour? (Btw I've seen a similar issue i.e. the "KeyError: 'platform' exception when I have the two versions of this tool installed from a local toolshed, and try switching between to the newer one from the older one.) Any help or suggestions are greatly appreciated. Many thanks Peter -- Peter Briggs peter.briggs(a)manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482

2 2

please subscribe
by Ioana Cristina Chintauan-Marquier 02 Oct '14

02 Oct '14

thank you! -- Dr. Ioana C. Chintauan-Marquier Muséum National d'Histoire Naturelle Institut de SYstématique Evolution Biodiversité (ISYEB) UMR MNHN/CNRS/UPMC/EPHE 7205 57 Rue Cuvier, CP50 (Entomologie, équipe Phylevo) F-75231 PARIS cedex 05 France Tel: +33 (0)1 40 79 34 15/+33 (0)6 02 27 32 41 Email: imarquier(a)mnhn.fr, icchintauan.marquier(a)gmail.com <icchintauan.marquier(a)gmail.com>

1 0