Branch: refs/heads/dev Home: https://github.com/galaxyproject/galaxy Commit: cd2cf494fe1ede4ae5b43417f0da21b2ca6473a0 https://github.com/galaxyproject/galaxy/commit/cd2cf494fe1ede4ae5b43417f0da2... Author: John Chilton <jmchilton@gmail.com> Date: 2017-01-23 (Mon, 23 Jan 2017) Changed paths: M lib/galaxy/tools/deps/__init__.py M lib/galaxy/tools/deps/requirements.py M lib/galaxy/tools/deps/resolvers/__init__.py M lib/galaxy/tools/deps/resolvers/conda.py M lib/galaxy/tools/xsd/galaxy.xsd A test/functional/tools/requirement_specification_1.xml A test/functional/tools/requirement_specification_2.xml Log Message: ----------- Allow finer grain specification of requirements. This adapts CWL-style ``SoftwareRequirement`` ``specs`` to solve #1927. Here I'm trying to implement the CWL specification in a way that helps enable the feasibility of Conda packaging in Galaxy. It is a delicate balancing act aimed to upset as many interested parties as I can. To understand the problem - consider the ``blast+`` requirement found in the Galaxy wrappers. It looks something like this: ``` <requirement type="package" version="2.2.31" name="blast+"> ``` Great, that works for Galaxy and Tool Shed packages. It doesn't work for bioconda at all. I think this problem is fairly uncommon - most packages have simple names shared across Debian, Brew, Conda, etc... - but it does happen in some cases that there are inconsistencies. Some people have taken to duplicating the requirement - this is bad and should not be done since they are mutually exclusive and Galaxy will attempt to resolve both. This introduces the following syntax for tools with profile >= 16.10: ``` <requirement type="package" version="2.2.31" name="blast+"> <specification uri="https://anaconda.org/bioconda/blast" /> <specification uri="https://packages.debian.org/sid/ncbi-blast+" version="2.2.31-3" /> </requirement> ``` This allows finer grain resolution of the requirement without sacrificing the abstract name at the top. It allows the name and the version to be adapted by resolvers as needed (hopefully rarely so). This syntax is the future facing one, but obviously this tool would not work on older Galaxy versions. To remedy this - an alternative syntax can be used for tools targetting Galaxy verions pre-16.10: ``` <requirement type="package" version="2.2" specification_uris="https://anaconda.org/bioconda/blast@2.2.31,https://packages.debian.org/jessie/ncbi-blast+@2.2.29-3">blast+</requirement> ``` This syntax sucks - but it does give newer Galaxies the ability to resolve the specifications without breaking the more simple functionality for older Galaxies. For more information on the CWL side of this - checkout the discussion on https://github.com/common-workflow-language/cwltool/pull/214. The CWL specification information is defined at http://www.commonwl.org/v1.0/CommandLineTool.html#SoftwarePackage. Conflicts: lib/galaxy/tools/deps/__init__.py test/functional/tools/samples_tool_conf.xml Commit: 330d71d37ce7eaa60b5c8c20ae83aeb36fec4a6c https://github.com/galaxyproject/galaxy/commit/330d71d37ce7eaa60b5c8c20ae83a... Author: John Chilton <jmchilton@gmail.com> Date: 2017-01-23 (Mon, 23 Jan 2017) Changed paths: M lib/galaxy/tools/deps/requirements.py M lib/galaxy/tools/xsd/galaxy.xsd R test/functional/tools/requirement_specification_1.xml R test/functional/tools/requirement_specification_2.xml Log Message: ----------- Remove tool XML modifications for requirement specifications. Per discussion on #3117 - some people were in favor of the idea but some were opposed. I think it is still important to implement this for CWL tools and useful from a library perspective. This just removes changes to Galaxy tool XML. Commit: baf9b82847fc526af56ed8efa7aba4d31d62f400 https://github.com/galaxyproject/galaxy/commit/baf9b82847fc526af56ed8efa7aba... Author: John Chilton <jmchilton@gmail.com> Date: 2017-01-23 (Mon, 23 Jan 2017) Changed paths: M lib/galaxy/tools/deps/__init__.py M lib/galaxy/tools/deps/resolvers/__init__.py M lib/galaxy/tools/deps/resolvers/conda.py Log Message: ----------- More modular config option handling for dependency resolvers. The handling of global configuration options for Conda specific options was hacky and a poor design. This is much more modular and allows the elegant cascading option sources for all resolvers. Commit: 49a11073a49242fbd86e3ec05fb5029194ec23fd https://github.com/galaxyproject/galaxy/commit/49a11073a49242fbd86e3ec05fb50... Author: John Chilton <jmchilton@gmail.com> Date: 2017-01-23 (Mon, 23 Jan 2017) Changed paths: M lib/galaxy/tools/deps/resolvers/__init__.py M lib/galaxy/tools/deps/resolvers/conda.py Log Message: ----------- Give resolver's with a resolve_all method an interface. This follows the existing pattern within the module. Commit: 8e20e71556cfc90da548243d390deb52d4e9110d https://github.com/galaxyproject/galaxy/commit/8e20e71556cfc90da548243d390de... Author: John Chilton <jmchilton@gmail.com> Date: 2017-01-23 (Mon, 23 Jan 2017) Changed paths: M lib/galaxy/tools/deps/__init__.py M lib/galaxy/tools/deps/requirements.py M lib/galaxy/tools/deps/resolvers/__init__.py M lib/galaxy/tools/deps/resolvers/brewed_tool_shed_packages.py M lib/galaxy/tools/deps/resolvers/conda.py M lib/galaxy/tools/deps/resolvers/galaxy_packages.py M lib/galaxy/tools/deps/resolvers/homebrew.py M lib/galaxy/tools/deps/resolvers/modules.py M test/unit/tools/test_tool_deps.py Log Message: ----------- Change interface of depependency resolver's resolve for consistency. If resolve_all takes in requirements, then so should resolve. Commit: 35b2b735ff7495576de88b7469c7118f65cb2838 https://github.com/galaxyproject/galaxy/commit/35b2b735ff7495576de88b7469c71... Author: John Chilton <jmchilton@gmail.com> Date: 2017-01-23 (Mon, 23 Jan 2017) Changed paths: M lib/galaxy/tools/deps/resolvers/__init__.py M lib/galaxy/tools/deps/resolvers/conda.py Log Message: ----------- Modify specification interface with better name. Commit: 2fad74ddcf63c03fc2d7fd77fa3838de637a1bf4 https://github.com/galaxyproject/galaxy/commit/2fad74ddcf63c03fc2d7fd77fa383... Author: John Chilton <jmchilton@gmail.com> Date: 2017-01-23 (Mon, 23 Jan 2017) Changed paths: M lib/galaxy/tools/deps/resolvers/__init__.py M lib/galaxy/tools/deps/resolvers/conda.py M lib/galaxy/tools/deps/resolvers/galaxy_packages.py M lib/galaxy/tools/deps/resolvers/modules.py M test/unit/tools/test_tool_deps.py Log Message: ----------- Implement dependency mapping for environment modules, galaxy packages, and Conda. Commit: 3d9cdeb70c0170b7795c35e458c878ebe19caf1a https://github.com/galaxyproject/galaxy/commit/3d9cdeb70c0170b7795c35e458c87... Author: John Chilton <jmchilton@gmail.com> Date: 2017-01-23 (Mon, 23 Jan 2017) Changed paths: M lib/galaxy/config.py M lib/galaxy/model/orm/scripts.py M lib/galaxy/util/properties.py M run.sh A scripts/manage_tool_dependencies.py Log Message: ----------- Outline infrastructure for managing Conda installations outside of Galaxy. - Construct a DependencyManager according to Galaxy's configuration before Galaxy starts to allow auto installation of Conda. - Add script for managing tool dependencies outside of a Galaxy environment. - Refactor to allow this to be done without duplicating logic related to defaults, configuration handling, etc.... Implements #3426. Commit: 312f941e46fdbbe6198b6b0e6a3f00867946a6bd https://github.com/galaxyproject/galaxy/commit/312f941e46fdbbe6198b6b0e6a3f0... Author: John Chilton <jmchilton@gmail.com> Date: 2017-01-23 (Mon, 23 Jan 2017) Changed paths: M config/galaxy.ini.sample M lib/galaxy/tools/deps/resolvers/conda.py Log Message: ----------- Swap default for Conda "auto_init" to True. Everyone gets a Conda when Galaxy starts up. Commit: 0b7c6761c8dbb2baf7391dc4a08c4dbed2ccc7f2 https://github.com/galaxyproject/galaxy/commit/0b7c6761c8dbb2baf7391dc4a08c4... Author: John Chilton <jmchilton@gmail.com> Date: 2017-01-23 (Mon, 23 Jan 2017) Changed paths: M scripts/manage_tool_dependencies.py Log Message: ----------- Rework #3433 for changes in #3444. Commit: 9baf6d6d8d8d089a8d2c2687ef498a396006b920 https://github.com/galaxyproject/galaxy/commit/9baf6d6d8d8d089a8d2c2687ef498... Author: John Chilton <jmchilton@gmail.com> Date: 2017-01-23 (Mon, 23 Jan 2017) Changed paths: A config/local_conda_mapping.yml.sample M lib/galaxy/config.py A lib/galaxy/tools/deps/resolvers/default_conda_mapping.yml A test/functional/tools/legacy_r.xml M test/functional/tools/samples_tool_conf.xml M test/integration/test_resolvers.py Log Message: ----------- Samples and defaults for Conda mapping. - Map R to r-base and blast+ to blast for conda dependencies. - Add sample environment modules file showing off the new mapping feature. - Add integration test case to verify the default conda mapping (R -> r-base in particular). Commit: 3e3d41fb94c00a9c6b068626b6f541e714b2500e https://github.com/galaxyproject/galaxy/commit/3e3d41fb94c00a9c6b068626b6f54... Author: John Chilton <jmchilton@gmail.com> Date: 2017-01-23 (Mon, 23 Jan 2017) Changed paths: M test/unit/tools/test_tool_deps.py Log Message: ----------- Fix unit test test_tool_deps to not auto-install Conda. This became the default behavior with conda_auto_init defaulting to True. Commit: 98ee91f467039f27f4e10d730f78057d1421e833 https://github.com/galaxyproject/galaxy/commit/98ee91f467039f27f4e10d730f780... Author: Marius van den Beek <m.vandenbeek@gmail.com> Date: 2017-01-23 (Mon, 23 Jan 2017) Changed paths: M config/galaxy.ini.sample A config/local_conda_mapping.yml.sample M lib/galaxy/config.py M lib/galaxy/model/orm/scripts.py M lib/galaxy/tools/deps/__init__.py M lib/galaxy/tools/deps/requirements.py M lib/galaxy/tools/deps/resolvers/__init__.py M lib/galaxy/tools/deps/resolvers/brewed_tool_shed_packages.py M lib/galaxy/tools/deps/resolvers/conda.py A lib/galaxy/tools/deps/resolvers/default_conda_mapping.yml M lib/galaxy/tools/deps/resolvers/galaxy_packages.py M lib/galaxy/tools/deps/resolvers/homebrew.py M lib/galaxy/tools/deps/resolvers/modules.py M lib/galaxy/util/properties.py M run.sh A scripts/manage_tool_dependencies.py A test/functional/tools/legacy_r.xml M test/functional/tools/samples_tool_conf.xml M test/integration/test_resolvers.py M test/unit/tools/test_tool_deps.py Log Message: ----------- Merge pull request #3444 from jmchilton/deps_framework_overhaul Implement Dependency Resolver Mapping Compare: https://github.com/galaxyproject/galaxy/compare/5bd8a0fa0232...98ee91f46703