Branch: refs/heads/release_16.10 Home: https://github.com/galaxyproject/galaxy Commit: 94edc94b73e047612ae51ca2e075f17ac8e238bb https://github.com/galaxyproject/galaxy/commit/94edc94b73e047612ae51ca2e075f... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2016-11-29 (Tue, 29 Nov 2016) Changed paths: M config/galaxy.ini.sample M lib/galaxy/config.py M lib/galaxy/tools/__init__.py M lib/galaxy/tools/deps/__init__.py M lib/galaxy/tools/deps/resolvers/conda.py Log Message: ----------- Add cached dependency manager Similar to https://github.com/galaxyproject/galaxy/pull/2986, implement a mechanism that allows tool dependencies to be cached. If the `use_cached_dependency_manager` option is set to True in galaxy.ini, we build a hash of the combination of a tools' requirements, and store the resulting environment in a directory specified by the `tool_dependency_cache_dir` option in galaxy.ini. Commit: 0a9b6d8755dee530232ab893a3bad199407aa0bc https://github.com/galaxyproject/galaxy/commit/0a9b6d8755dee530232ab893a3bad... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2016-11-29 (Tue, 29 Nov 2016) Changed paths: M config/galaxy.ini.sample Log Message: ----------- Better phrasing of cache options in galaxy.ini.sample (Thanks @nsoranzo) Commit: 2b1765751b29008aecbd782f28269ac4e63db239 https://github.com/galaxyproject/galaxy/commit/2b1765751b29008aecbd782f28269... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2016-11-29 (Tue, 29 Nov 2016) Changed paths: M config/galaxy.ini.sample M lib/galaxy/tools/deps/__init__.py M lib/galaxy/tools/deps/conda_util.py M lib/galaxy/tools/deps/resolvers/__init__.py M lib/galaxy/tools/deps/resolvers/conda.py Log Message: ----------- Rework cached dependencies to be more robust to updates or changes in depedencies, folder structure and resolver configuration. Instead of hashing name, type and version of a dependency, hash the json representation of the dependencies returned by the dependency resolver, which include the path to the environment and the depedency type. This is only applied to resolvers whose cacheable attribute is set to True (conda-only, currently). Commit: f919a98601252ab8f87bffff6e449105e770aadf https://github.com/galaxyproject/galaxy/commit/f919a98601252ab8f87bffff6e449... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2016-11-29 (Tue, 29 Nov 2016) Changed paths: M lib/galaxy/tools/__init__.py M lib/galaxy/tools/deps/__init__.py M lib/galaxy/tools/deps/resolvers/__init__.py M lib/galaxy/tools/deps/resolvers/conda.py M lib/tool_shed/galaxy_install/install_manager.py Log Message: ----------- Build dependency cache at install time and only activate cached environments if they exist. Commit: 82f30e7645156b5f35c2b4cd77314e6275663979 https://github.com/galaxyproject/galaxy/commit/82f30e7645156b5f35c2b4cd77314... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2016-11-29 (Tue, 29 Nov 2016) Changed paths: M lib/galaxy/tools/deps/requirements.py M lib/tool_shed/galaxy_install/install_manager.py Log Message: ----------- Only attempt to build cache once per install and override __eq__ for ToolRequirement, to simplify checking if ToolRequirements are already installed/cached. Commit: 64bc9241ed785d4d552c5fac21cd412558c54567 https://github.com/galaxyproject/galaxy/commit/64bc9241ed785d4d552c5fac21cd4... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2016-11-29 (Tue, 29 Nov 2016) Changed paths: M lib/galaxy/tools/deps/resolvers/conda.py Log Message: ----------- Fix NameError Commit: cf769dd09deb73624b22628208fa103cc4f27150 https://github.com/galaxyproject/galaxy/commit/cf769dd09deb73624b22628208fa1... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2016-11-29 (Tue, 29 Nov 2016) Changed paths: M lib/galaxy/datatypes/converters/fasta_to_2bit.xml M lib/galaxy/webapps/galaxy/api/tools.py M lib/galaxy/webapps/galaxy/buildapp.py Log Message: ----------- Add API to install resolver tool dependencies and to build a tool dependency cache (if activated in galaxy.ini). An example to install dependencies for the twobit converter: ``` import bioblend.galaxy url = 'http://localhost:8080/' api_key = 'admin_api_key' tool_id = 'CONVERTER_fasta_to_2bit' endpoint = "api/tools/%s/install_dependencies" % tool_id gi = bioblend.galaxy.GalaxyInstance(url, api_key) gi.make_post_request("/".join((url, endpoint)), payload={'id': tool_id}) ``` If `use_cached_dependency_manager` is activated in the galaxy.ini, a cached environment can be built like this: ``` endpoint = "api/tools/%s/build_dependency_cache" % tool_id gi.make_post_request("/".join((url, endpoint)), payload={'id': tool_id}) ``` Commit: cc7cd47ef54a09efbf8272d1e024f6f2ac92594b https://github.com/galaxyproject/galaxy/commit/cc7cd47ef54a09efbf8272d1e024f... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2016-11-29 (Tue, 29 Nov 2016) Changed paths: M test/integration/test_resolvers.py Log Message: ----------- Add integration test-case for install tool dependency API endpoint Commit: f10fd793a68ed9413bec708daa48cef65769d09f https://github.com/galaxyproject/galaxy/commit/f10fd793a68ed9413bec708daa48c... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2016-11-29 (Tue, 29 Nov 2016) Changed paths: M lib/galaxy/datatypes/converters/fasta_to_2bit.xml Log Message: ----------- Install the correct converter dependency Commit: 045a2f0d471f7b4186aff625f3cea9abfa549fa2 https://github.com/galaxyproject/galaxy/commit/045a2f0d471f7b4186aff625f3cea... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2016-11-29 (Tue, 29 Nov 2016) Changed paths: M lib/galaxy/tools/__init__.py M lib/galaxy/tools/deps/__init__.py M lib/galaxy/webapps/galaxy/api/tools.py Log Message: ----------- Add `force_rebuild` param to build_dependency_cache This will remove a pre-existing cached dependency directory. Commit: 16b49d9140edbf116f83b51b957cd61c4f2d91df https://github.com/galaxyproject/galaxy/commit/16b49d9140edbf116f83b51b957cd... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2016-11-29 (Tue, 29 Nov 2016) Changed paths: M lib/galaxy/tools/deps/resolvers/conda.py M lib/galaxy/tools/deps/views.py M lib/galaxy/webapps/galaxy/api/tool_dependencies.py M lib/galaxy/webapps/galaxy/buildapp.py M test/integration/test_resolvers.py Log Message: ----------- Add dependency_resolvers/clean endpoint Commit: 1efd4d4b3b99ec40071aa4d6fb7cf4030c51e2e0 https://github.com/galaxyproject/galaxy/commit/1efd4d4b3b99ec40071aa4d6fb7cf... Author: Bjoern Gruening <bjoern.gruening@gmail.com> Date: 2016-11-29 (Tue, 29 Nov 2016) Changed paths: M lib/galaxy/tools/deps/conda_util.py Log Message: ----------- add a conda clean function to remove downloaded tarballs after installation Commit: 2a4ccac4039342c98d71bc72177aa951db6c4ba1 https://github.com/galaxyproject/galaxy/commit/2a4ccac4039342c98d71bc72177aa... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2016-11-29 (Tue, 29 Nov 2016) Changed paths: M lib/galaxy/tools/deps/conda_util.py M lib/galaxy/tools/deps/resolvers/conda.py M lib/galaxy/tools/deps/views.py M lib/galaxy/webapps/galaxy/api/tool_dependencies.py M lib/galaxy/webapps/galaxy/buildapp.py M test/integration/test_resolvers.py Log Message: ----------- Connect `clean` endpoint properly and add `-y` to exec_clean Commit: d6f0e6b37c318a0202e0e266828e6325100cab19 https://github.com/galaxyproject/galaxy/commit/d6f0e6b37c318a0202e0e266828e6... Author: Martin Cech <cech.marten@gmail.com> Date: 2016-11-29 (Tue, 29 Nov 2016) Changed paths: M config/galaxy.ini.sample M lib/galaxy/config.py M lib/galaxy/datatypes/converters/fasta_to_2bit.xml M lib/galaxy/tools/__init__.py M lib/galaxy/tools/deps/__init__.py M lib/galaxy/tools/deps/conda_util.py M lib/galaxy/tools/deps/requirements.py M lib/galaxy/tools/deps/resolvers/__init__.py M lib/galaxy/tools/deps/resolvers/conda.py M lib/galaxy/tools/deps/views.py M lib/galaxy/webapps/galaxy/api/tool_dependencies.py M lib/galaxy/webapps/galaxy/api/tools.py M lib/galaxy/webapps/galaxy/buildapp.py M lib/tool_shed/galaxy_install/install_manager.py M test/integration/test_resolvers.py Log Message: ----------- Merge pull request #3227 from mvdbeek/cached_deps_16.10 [16.10] Backport #3106 and #3222: Cached conda environments and API to manage them Compare: https://github.com/galaxyproject/galaxy/compare/e3f8b724648c...d6f0e6b37c31