Branch: refs/heads/release_18.05 Home: https://github.com/galaxyproject/galaxy Commit: 0ca1401bd8387717edb31a69e11d3adc09258757 https://github.com/galaxyproject/galaxy/commit/0ca1401bd8387717edb31a69e11d3... Author: John Chilton <jmchilton@gmail.com> Date: 2018-04-27 (Fri, 27 Apr 2018) Changed paths: M lib/galaxy/tools/parameters/basic.py M lib/galaxy/tools/parameters/dataset_matcher.py Log Message: ----------- ToolBuildOptimize - Eliminate check_security on dataset matcher. It was only used for collections and we need to drop it as prohibitively expensive to calculate. No need to filter collections ahead of time that way anyhow - it is the tool action's job to block the execution of datasets without permission so hopefully we aren't deriving any security value from this filter. Commit: c3303939c05ae8af9a2b11a12d5f433ee0d8072e https://github.com/galaxyproject/galaxy/commit/c3303939c05ae8af9a2b11a12d5f4... Author: John Chilton <jmchilton@gmail.com> Date: 2018-04-27 (Fri, 27 Apr 2018) Changed paths: M lib/galaxy/tools/parameters/dataset_matcher.py Log Message: ----------- ToolBuildOptimize - Optimize state checking in dataset matcher. Pre-calculate valid states, skip now unneeded helper method. Commit: 7ca2ef587cd26682e3b4b28c7cdb8907cfeae862 https://github.com/galaxyproject/galaxy/commit/7ca2ef587cd26682e3b4b28c7cdb8... Author: John Chilton <jmchilton@gmail.com> Date: 2018-04-27 (Fri, 27 Apr 2018) Changed paths: M lib/galaxy/tools/parameters/dataset_matcher.py Log Message: ----------- DatasetMatcherClean - Skip second, unneeded call to filter(). xref 8f813712f50ca21183d2efe59ee0e2a665520f95 to some degree. Commit: 412f03d8446cb362bbfe27d5832bcf37aef2d4d6 https://github.com/galaxyproject/galaxy/commit/412f03d8446cb362bbfe27d5832bc... Author: John Chilton <jmchilton@gmail.com> Date: 2018-04-27 (Fri, 27 Apr 2018) Changed paths: M lib/galaxy/tools/parameters/basic.py M lib/galaxy/tools/parameters/dataset_matcher.py Log Message: ----------- DatasetMatcherClean - Eliminate DatasetMatcher.value - it is never set. Also eliminate any logic related to it having a value. This had a purpose originally, but is no longer set. Cleaning this up makes subsequent commits a bit cleaner also. Commit: ca80232db3a93a02dcd42f2698b54e7a8d2aeef1 https://github.com/galaxyproject/galaxy/commit/ca80232db3a93a02dcd42f2698b54... Author: John Chilton <jmchilton@gmail.com> Date: 2018-04-27 (Fri, 27 Apr 2018) Changed paths: M lib/galaxy/tools/parameters/basic.py Log Message: ----------- DatasetMatcherClean - Remove unused method. It calls things that I'm changing in subsequent commits so I thought I'd just axe it now to clear things up. Commit: efe5d8b973ffdd22963ec75db7820ff1731db916 https://github.com/galaxyproject/galaxy/commit/efe5d8b973ffdd22963ec75db7820... Author: John Chilton <jmchilton@gmail.com> Date: 2018-04-27 (Fri, 27 Apr 2018) Changed paths: M lib/galaxy/tools/__init__.py M lib/galaxy/tools/parameters/basic.py M lib/galaxy/tools/parameters/dataset_matcher.py Log Message: ----------- DatasetMatcherClean - Implement DatasetMatcherFactory to reason for whole tool. In subsequent commit I'll use this central store of all the inputs for a tool to determine if summary data about collections can be used instead of processing individual datasets one at a time. Even this commit though uses the abstraction to optimize datatype checking and cache commons checks when possible - should lead to a lot fewer objects being created when processing a large history. Commit: 2eea566c10eaf2b7dad21cdff82e0a14a1893fde https://github.com/galaxyproject/galaxy/commit/2eea566c10eaf2b7dad21cdff82e0... Author: John Chilton <jmchilton@gmail.com> Date: 2018-04-27 (Fri, 27 Apr 2018) Changed paths: M lib/galaxy/model/__init__.py M lib/galaxy/tools/parameters/dataset_matcher.py M test/unit/tools/test_data_parameters.py M test/unit/tools/test_dataset_matcher.py Log Message: ----------- ToolBuildOptimize - Add SummaryDatasetCollectionMatcher. Use summary data pulled from the database for collections when possible instead of loading potentially hundreds of thousands of individual datasets. Commit: 740c4c93445d5833038e74ccb39413752eeeeb8c https://github.com/galaxyproject/galaxy/commit/740c4c93445d5833038e74ccb3941... Author: John Chilton <jmchilton@gmail.com> Date: 2018-04-27 (Fri, 27 Apr 2018) Changed paths: M lib/galaxy/model/__init__.py M lib/galaxy/tools/parameters/basic.py M test/unit/tools/test_data_parameters.py Log Message: ----------- ToolBuildOptimize - do not fetch hidden datasets for inclusion. Only fetch visible datasets into big, cached list of history datasets under consideration. Hidden datasets don't seem to be used by the fetcher or initial value stuff so it seems fine to exclude them. The advantage should be clear for histories with a large number of datasets hidden below a signficantly smaller number collections. Commit: afe938fe04f1b70d77a357afd03df8252a375770 https://github.com/galaxyproject/galaxy/commit/afe938fe04f1b70d77a357afd03df... Author: John Chilton <jmchilton@gmail.com> Date: 2018-04-27 (Fri, 27 Apr 2018) Changed paths: M lib/galaxy/model/__init__.py M lib/galaxy/tools/parameters/basic.py Log Message: ----------- ToolBuildOptimize - fetch fewer collections, prefetch more of HDCA. Low hanging fruit is to exclude hidden collections, that probably won't buy much for typical uses. This eliminates one extra tag query per dataset collection that appears in the rendered result, this probably buys us a bit more than the hidden collection thing but is still probably a good choice (unless there collections with a large number of tags :(...). This also eliminates the extra fetch of the first, outer-est collection associated the history dataset collection - not its elements just the collection. This saves a number of queries roughly equal to the number of HDCAs in the history and unlike the tag thing there is no downside really here - there will always be one collection. Before and after profiling of a tool form build: https://gist.github.com/jmchilton/d68565662f7f4b7ee2640f09fbb92962 Commit: 717e1104c1e32f79b84de6ffbc481b105087f17f https://github.com/galaxyproject/galaxy/commit/717e1104c1e32f79b84de6ffbc481... Author: Bjoern Gruening <bjoern.gruening@gmail.com> Date: 2018-04-28 (Sat, 28 Apr 2018) Changed paths: M lib/galaxy/tools/verify/__init__.py Log Message: ----------- enable testing for more BAM types Commit: abf23cdc0bb22542577e5ddefac38242c9ddf8ee https://github.com/galaxyproject/galaxy/commit/abf23cdc0bb22542577e5ddefac38... Author: Marius van den Beek <m.vandenbeek@gmail.com> Date: 2018-04-30 (Mon, 30 Apr 2018) Changed paths: M lib/galaxy/tools/verify/__init__.py Log Message: ----------- Merge pull request #6010 from bgruening/test_bam_subtypes [18.01] enable testing for more BAM types Commit: fbc7d49de1b71e7a1a0e544a9c18c5651dad9dd8 https://github.com/galaxyproject/galaxy/commit/fbc7d49de1b71e7a1a0e544a9c18c... Author: John Chilton <jmchilton@gmail.com> Date: 2018-04-30 (Mon, 30 Apr 2018) Changed paths: M lib/galaxy/tools/parameters/dataset_matcher.py Log Message: ----------- Fix for data collection parameters in tool state optimization branch. Commit: fdfe79f67edd3af8967f916badffcd13aadb78d6 https://github.com/galaxyproject/galaxy/commit/fdfe79f67edd3af8967f916badffc... Author: John Chilton <jmchilton@gmail.com> Date: 2018-04-30 (Mon, 30 Apr 2018) Changed paths: M client/galaxy/scripts/mvc/collection/list-of-pairs-collection-creator.js M lib/galaxy/webapps/galaxy/api/history_contents.py Log Message: ----------- Fix library to list of pairs collection builder. Commit: f50fe5d455594996ba4eb900e0adbf1face67de3 https://github.com/galaxyproject/galaxy/commit/f50fe5d455594996ba4eb900e0adb... Author: John Chilton <jmchilton@gmail.com> Date: 2018-04-30 (Mon, 30 Apr 2018) Changed paths: M static/scripts/bundled/analysis.bundled.js M static/scripts/bundled/analysis.bundled.js.map M static/scripts/bundled/extended.bundled.js M static/scripts/bundled/extended.bundled.js.map M static/scripts/bundled/libs.bundled.js.map M static/scripts/mvc/collection/list-of-pairs-collection-creator.js Log Message: ----------- Rebuild client. Commit: b477329842e7209b2fa2b9389e48b0606b7d8824 https://github.com/galaxyproject/galaxy/commit/b477329842e7209b2fa2b9389e48b... Author: Martin Cech <marten@bx.psu.edu> Date: 2018-04-30 (Mon, 30 Apr 2018) Changed paths: M client/galaxy/scripts/mvc/collection/list-of-pairs-collection-creator.js M lib/galaxy/webapps/galaxy/api/history_contents.py M static/scripts/bundled/analysis.bundled.js M static/scripts/bundled/analysis.bundled.js.map M static/scripts/bundled/extended.bundled.js M static/scripts/bundled/extended.bundled.js.map M static/scripts/bundled/libs.bundled.js.map M static/scripts/mvc/collection/list-of-pairs-collection-creator.js Log Message: ----------- Merge pull request #6015 from jmchilton/fix_library_paired [18.01] Fix library to list of pairs collection builder. Commit: 790ec387d7e3bebec66dd19aaa38e16e2d2266c4 https://github.com/galaxyproject/galaxy/commit/790ec387d7e3bebec66dd19aaa38e... Author: Nate Coraor <nate@bx.psu.edu> Date: 2018-04-30 (Mon, 30 Apr 2018) Changed paths: M lib/galaxy/tools/__init__.py Log Message: ----------- lastz_wrapper_2 no longer requires Galaxy's python env as of version 1.3 Commit: aebc0da8914f5a2dc3fdc7c68805b174d058f56e https://github.com/galaxyproject/galaxy/commit/aebc0da8914f5a2dc3fdc7c68805b... Author: Dannon <dannon.baker@gmail.com> Date: 2018-04-30 (Mon, 30 Apr 2018) Changed paths: M lib/galaxy/tools/__init__.py Log Message: ----------- Merge pull request #6020 from natefoo/lastz-1.3 [18.01] lastz_wrapper_2 no longer requires Galaxy's python env as of version 1.3 Commit: b35340c77f262309d225cf2a9e526ac7e67194cc https://github.com/galaxyproject/galaxy/commit/b35340c77f262309d225cf2a9e526... Author: Dannon <dannon.baker@gmail.com> Date: 2018-05-02 (Wed, 02 May 2018) Changed paths: M lib/galaxy/model/__init__.py M lib/galaxy/tools/__init__.py M lib/galaxy/tools/parameters/basic.py M lib/galaxy/tools/parameters/dataset_matcher.py M test/unit/tools/test_data_parameters.py M test/unit/tools/test_dataset_matcher.py Log Message: ----------- Merge pull request #5997 from jmchilton/1801_tool_state_opt [18.01] Fix tool state performance for large collections. Commit: 808686cd6050fbc93b5b41185dd713993d8651d0 https://github.com/galaxyproject/galaxy/commit/808686cd6050fbc93b5b41185dd71... Author: Dannon Baker <dannon.baker@gmail.com> Date: 2018-05-02 (Wed, 02 May 2018) Changed paths: M client/galaxy/scripts/mvc/collection/list-of-pairs-collection-creator.js M lib/galaxy/model/__init__.py M lib/galaxy/tools/__init__.py M lib/galaxy/tools/parameters/basic.py M lib/galaxy/tools/parameters/dataset_matcher.py M lib/galaxy/tools/verify/__init__.py M lib/galaxy/webapps/galaxy/api/history_contents.py M test/unit/tools/test_data_parameters.py M test/unit/tools/test_dataset_matcher.py Log Message: ----------- Merge branch 'release_18.01' into release_18.05 Compare: https://github.com/galaxyproject/galaxy/compare/92a4a138621e...808686cd6050 **NOTE:** This service been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/ Functionality will be removed from GitHub.com on January 31st, 2019.