Branch: refs/heads/dev Home: https://github.com/galaxyproject/galaxy Commit: f7820e82380f7ea69ba56037a3f463352ef36807 https://github.com/galaxyproject/galaxy/commit/f7820e82380f7ea69ba56037a3f46... Author: mvdbeek m.vandenbeek@gmail.com Date: 2017-12-01 (Fri, 01 Dec 2017)

Changed paths: M lib/galaxy/tools/data/__init__.py A lib/galaxy/util/renamed_temporary_file.py

Log Message: ----------- Do not write shed_tool_data_table_config in place

This should prevent errors such as: ``` galaxy.util ERROR 2017-11-17 09:20:12,842 Error parsing file /data/users/mvandenb/gx/config/shed_tool_data_table_conf.xml Traceback (most recent call last): File "lib/galaxy/util/__init__.py", line 217, in parse_xml root = tree.parse(fname, parser=ElementTree.XMLParser(target=DoctypeSafeCallbackTarget())) File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 657, in parse self._root = parser.close() File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1654, in close self._raiseerror(v) File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1506, in _raiseerror raise err ParseError: no element found: line 1, column 0 ``` reported in https://github.com/galaxyproject/galaxy/issues/5031.

Commit: a2b8e445d85b50265da52c06f017c9c78be953b6 https://github.com/galaxyproject/galaxy/commit/a2b8e445d85b50265da52c06f017c... Author: mvdbeek m.vandenbeek@gmail.com Date: 2017-12-01 (Fri, 01 Dec 2017)

Changed paths: M lib/galaxy/model/tool_shed_install/__init__.py

Log Message: ----------- Fix installation of data manager revision updates

Without this updates of data managers fails with ``` AttributeError: 'NoneType' object has no attribute 'get' File "galaxy/web/framework/middleware/sentry.py", line 40, in __call__ iterable = self.application(environ, start_response) File "/bioinfo/guests/mvandenb/galaxy/.venv/local/lib/python2.7/site-packages/paste/recursive.py", line 85, in __call__ return self.application(environ, start_response) File "/bioinfo/guests/mvandenb/galaxy/.venv/local/lib/python2.7/site-packages/paste/httpexceptions.py", line 640, in __call__ return self.application(environ, start_response) File "galaxy/web/framework/base.py", line 136, in __call__ return self.handle_request(environ, start_response) File "galaxy/web/framework/base.py", line 215, in handle_request body = method(trans, **kwargs) File "galaxy/web/framework/decorators.py", line 98, in decorator return func(self, trans, *args, **kwargs) File "galaxy/webapps/galaxy/controllers/admin_toolshed.py", line 1849, in update_to_changeset_revision persist=True) File "tool_shed/galaxy_install/metadata/installed_repository_metadata_manager.py", line 30, in __init__ metadata_dict=metadata_dict, user=None) File "tool_shed/metadata/metadata_generator.py", line 63, in __init__ self.metadata_dict = {'shed_config_filename': self.shed_config_dict.get('config_filename', None)} ```

Commit: d44a20cef098abf905f8525c42eb7f37d39f4ad9 https://github.com/galaxyproject/galaxy/commit/d44a20cef098abf905f8525c42eb7... Author: mvdbeek m.vandenbeek@gmail.com Date: 2017-12-01 (Fri, 01 Dec 2017)

Changed paths: M lib/tool_shed/galaxy_install/install_manager.py M lib/tool_shed/metadata/metadata_generator.py

Log Message: ----------- Use a temporary ToolDataTableManager when generating metadata

Otherwise loaded data tables bleed into the app-wide data tables. This can be seen with lots of data tables pointing to (non-existing, at that point) temporary files after a metadata reset in the galaxy admin panel, but it can also cause failure to install tools with entirely new data tables.

Commit: 561f71ec50ee6708913d57955456faa1932c4cac https://github.com/galaxyproject/galaxy/commit/561f71ec50ee6708913d57955456f... Author: mvdbeek m.vandenbeek@gmail.com Date: 2017-12-01 (Fri, 01 Dec 2017)

Changed paths: M lib/tool_shed/tools/data_table_manager.py

Log Message: ----------- Avoid manipulating global toll data tables

Commit: c02d9363c45b7272f8b507f6c46aecf867faee5d https://github.com/galaxyproject/galaxy/commit/c02d9363c45b7272f8b507f6c46ae... Author: mvdbeek m.vandenbeek@gmail.com Date: 2017-12-01 (Fri, 01 Dec 2017)

Changed paths: M lib/tool_shed/util/tool_util.py

Log Message: ----------- Fix copying of loc.sample files

`filename` should be joined with the tool_path, as evidenced by other uses of copy_sample_file.

Commit: fe6033fb22a23d4004a70de101e3f9597d0c6912 https://github.com/galaxyproject/galaxy/commit/fe6033fb22a23d4004a70de101e3f... Author: mvdbeek m.vandenbeek@gmail.com Date: 2017-12-01 (Fri, 01 Dec 2017)

Changed paths: M lib/galaxy/webapps/tool_shed/controllers/repository.py M lib/tool_shed/tools/tool_validator.py

Log Message: ----------- Avoid global changes to app.config.tool_data_path

Commit: 7df9dce63d635d66a0a9ce5045135ab49b7ef506 https://github.com/galaxyproject/galaxy/commit/7df9dce63d635d66a0a9ce5045135... Author: mvdbeek m.vandenbeek@gmail.com Date: 2017-12-01 (Fri, 01 Dec 2017)

Changed paths: M lib/galaxy/webapps/tool_shed/controllers/upload.py M lib/tool_shed/galaxy_install/install_manager.py M lib/tool_shed/tools/data_table_manager.py M lib/tool_shed/tools/tool_validator.py M lib/tool_shed/util/commit_util.py

Log Message: ----------- Rename the ToolDataTableManager that deals with repository data to ShedToolDataTableManager to avoid confusion

Commit: b03f16ed49074ef98ba7ecd7ed9f70123e196ace https://github.com/galaxyproject/galaxy/commit/b03f16ed49074ef98ba7ecd7ed9f7... Author: mvdbeek m.vandenbeek@gmail.com Date: 2017-12-01 (Fri, 01 Dec 2017)

Changed paths: M lib/tool_shed/metadata/metadata_generator.py M lib/tool_shed/tools/tool_validator.py

Log Message: ----------- Load tools directly from source and avoid loading via toolbox

Commit: c0887d72f8d37ee7c608fa0e41d2455956e6791a https://github.com/galaxyproject/galaxy/commit/c0887d72f8d37ee7c608fa0e41d24... Author: mvdbeek m.vandenbeek@gmail.com Date: 2017-12-01 (Fri, 01 Dec 2017)

Changed paths: M lib/tool_shed/galaxy_install/install_manager.py M lib/tool_shed/tools/data_table_manager.py M lib/tool_shed/tools/tool_validator.py

Log Message: ----------- Fix missing index file check

Commit: 55e458262b4184557842312c77f70dc1f5f3903c https://github.com/galaxyproject/galaxy/commit/55e458262b4184557842312c77f70... Author: mvdbeek m.vandenbeek@gmail.com Date: 2017-12-02 (Sat, 02 Dec 2017)

Changed paths: A lib/galaxy/util/miniapp.py M lib/tool_shed/metadata/metadata_generator.py M lib/tool_shed/metadata/repository_metadata_manager.py

Log Message: ----------- Add miniapp to avoid manipulating global settings

Commit: ab21f303af8d53fe543974ed48e7d0614db2de22 https://github.com/galaxyproject/galaxy/commit/ab21f303af8d53fe543974ed48e7d... Author: mvdbeek m.vandenbeek@gmail.com Date: 2017-12-02 (Sat, 02 Dec 2017)

Changed paths: M lib/galaxy/util/miniapp.py M lib/galaxy/webapps/tool_shed/api/tools.py M lib/galaxy/webapps/tool_shed/controllers/repository.py M lib/galaxy/webapps/tool_shed/controllers/upload.py M lib/tool_shed/metadata/metadata_generator.py M lib/tool_shed/tools/tool_validator.py M lib/tool_shed/util/workflow_util.py

Log Message: ----------- Always use MiniApp when using ToolValidator

Commit: f08f4733050922c4177d84ec6f2550dd53cc8fcb https://github.com/galaxyproject/galaxy/commit/f08f4733050922c4177d84ec6f255... Author: mvdbeek m.vandenbeek@gmail.com Date: 2017-12-02 (Sat, 02 Dec 2017)

Changed paths: M lib/galaxy/tools/__init__.py

Log Message: ----------- Only parse citations if app has citations_manager

Commit: d3261096c831a8539aeeae5ccf4e6734593f91a1 https://github.com/galaxyproject/galaxy/commit/d3261096c831a8539aeeae5ccf4e6... Author: mvdbeek m.vandenbeek@gmail.com Date: 2017-12-02 (Sat, 02 Dec 2017)

Changed paths: M lib/galaxy/util/miniapp.py M lib/tool_shed/tools/tool_validator.py

Log Message: ----------- Fix linting issues and fix data manager installation

Commit: c1630c244fc4700998e0d23130f7120344b303b2 https://github.com/galaxyproject/galaxy/commit/c1630c244fc4700998e0d23130f71... Author: mvdbeek m.vandenbeek@gmail.com Date: 2017-12-02 (Sat, 02 Dec 2017)

Changed paths: A lib/galaxy/tools/repositories/validationcontext.py R lib/galaxy/util/miniapp.py M lib/galaxy/webapps/tool_shed/api/tools.py M lib/galaxy/webapps/tool_shed/controllers/repository.py M lib/tool_shed/metadata/metadata_generator.py M lib/tool_shed/util/workflow_util.py

Log Message: ----------- Move and rename MiniApp to ValidationContext

This better describes what it used for and since it is tool specific, fits better here that in utils. Thanks @jmchilton for the suggestion.

Commit: 93442f621bea51bd242f2b67e7d201bc409f9fe9 https://github.com/galaxyproject/galaxy/commit/93442f621bea51bd242f2b67e7d20... Author: mvdbeek m.vandenbeek@gmail.com Date: 2017-12-02 (Sat, 02 Dec 2017)

Changed paths: A lib/galaxy/tools/repositories.py R lib/galaxy/tools/repositories/validationcontext.py M lib/tool_shed/metadata/metadata_generator.py M lib/tool_shed/util/workflow_util.py

Log Message: ----------- Move ValidationContext to right path

Commit: 482c9cce76c56bee840a87f25b0a17b8937444f5 https://github.com/galaxyproject/galaxy/commit/482c9cce76c56bee840a87f25b0a1... Author: mvdbeek m.vandenbeek@gmail.com Date: 2017-12-02 (Sat, 02 Dec 2017)

Changed paths: M lib/tool_shed/tools/tool_validator.py

Log Message: ----------- Remove more unused variables

Commit: db88175b9fffe76cfd9013a6e6d6cfb9c573c354 https://github.com/galaxyproject/galaxy/commit/db88175b9fffe76cfd9013a6e6d6c... Author: mvdbeek m.vandenbeek@gmail.com Date: 2017-12-03 (Sun, 03 Dec 2017)

Changed paths: M lib/tool_shed/tools/data_table_manager.py M lib/tool_shed/tools/tool_validator.py

Log Message: ----------- Use ValidationContext data tables instead of separately passing in relevant data tables

Commit: fd4737d0ecade2dd3f13d51756f90899920ef2a4 https://github.com/galaxyproject/galaxy/commit/fd4737d0ecade2dd3f13d51756f90... Author: mvdbeek m.vandenbeek@gmail.com Date: 2017-12-03 (Sun, 03 Dec 2017)

Changed paths: M lib/tool_shed/tools/data_table_manager.py M lib/tool_shed/util/tool_util.py

Log Message: ----------- Don't reload tools, new indexes and data tables are recognized without a reload

Commit: f97774be82108d94fecc29dcf98f1fe833a4ece0 https://github.com/galaxyproject/galaxy/commit/f97774be82108d94fecc29dcf98f1... Author: mvdbeek m.vandenbeek@gmail.com Date: 2017-12-03 (Sun, 03 Dec 2017)

Changed paths: M lib/galaxy/tools/repositories.py

Log Message: ----------- Change ValidationContext.build_file_path to be file

Also simplify cleanup code.

Commit: 1b48ff3dd15ebde13382afdb41cfb200a3c883cc https://github.com/galaxyproject/galaxy/commit/1b48ff3dd15ebde13382afdb41cfb... Author: mvdbeek m.vandenbeek@gmail.com Date: 2017-12-03 (Sun, 03 Dec 2017)

Changed paths: M lib/galaxy/tools/parameters/basic.py

Log Message: ----------- Skip format parsing if datatypes registry contains no datatypes

Commit: 78e1478c7f9c6f3c232c6c91263c7bb28dec5c88 https://github.com/galaxyproject/galaxy/commit/78e1478c7f9c6f3c232c6c91263c7... Author: mvdbeek m.vandenbeek@gmail.com Date: 2017-12-03 (Sun, 03 Dec 2017)

Changed paths: M lib/tool_shed/metadata/repository_metadata_manager.py

Log Message: ----------- Remove unnecessary tool data table resets

Commit: e65a2490c108929043a685eeca6bb89b6ade0b9c https://github.com/galaxyproject/galaxy/commit/e65a2490c108929043a685eeca6bb... Author: mvdbeek m.vandenbeek@gmail.com Date: 2017-12-03 (Sun, 03 Dec 2017)

Changed paths: M lib/galaxy/tools/repositories.py M lib/tool_shed/tools/data_table_manager.py

Log Message: ----------- Linting fixes

Commit: 4f162f28041d6c0ee3b551d589339d4048b5b7e1 https://github.com/galaxyproject/galaxy/commit/4f162f28041d6c0ee3b551d589339... Author: mvdbeek m.vandenbeek@gmail.com Date: 2017-12-03 (Sun, 03 Dec 2017)

Changed paths: M lib/galaxy/tools/__init__.py A test/shed_functional/test_data/bowtie2_loc_sample/bowtie2_indices.loc.sample A test/unit/tools/test_tool_validation.py

Log Message: ----------- Add unit test for ValidationContext and ToolValidator

Commit: 30e2736cc4fc82cdf32437c60934a0e8e6bac453 https://github.com/galaxyproject/galaxy/commit/30e2736cc4fc82cdf32437c60934a... Author: John Chilton jmchilton@gmail.com Date: 2017-12-07 (Thu, 07 Dec 2017)

Changed paths: M lib/galaxy/model/tool_shed_install/__init__.py M lib/galaxy/tools/__init__.py M lib/galaxy/tools/data/__init__.py M lib/galaxy/tools/parameters/basic.py A lib/galaxy/tools/repositories.py A lib/galaxy/util/renamed_temporary_file.py M lib/galaxy/webapps/tool_shed/api/tools.py M lib/galaxy/webapps/tool_shed/controllers/repository.py M lib/galaxy/webapps/tool_shed/controllers/upload.py M lib/tool_shed/galaxy_install/install_manager.py M lib/tool_shed/metadata/metadata_generator.py M lib/tool_shed/metadata/repository_metadata_manager.py M lib/tool_shed/tools/data_table_manager.py M lib/tool_shed/tools/tool_validator.py M lib/tool_shed/util/commit_util.py M lib/tool_shed/util/tool_util.py M lib/tool_shed/util/workflow_util.py A test/shed_functional/test_data/bowtie2_loc_sample/bowtie2_indices.loc.sample A test/unit/tools/test_tool_validation.py

Log Message: ----------- Merge pull request #5119 from mvdbeek/dev_toolshed_install_improvements_2

Improve tool validation and metadata generation

Compare: https://github.com/galaxyproject/galaxy/compare/a901264ca745...30e2736cc4fc