February 2018 - galaxy-commits - lists.galaxyproject.org

[galaxyproject/galaxy] 49a001: Conda fixes for 18.01 - stdout redirection.
by GitHub 28 Feb '18

28 Feb '18

Branch: refs/heads/dev Home: https://github.com/galaxyproject/galaxy Commit: 49a001cf7c4bbca1dd866a0a97082d343b3ea349 https://github.com/galaxyproject/galaxy/commit/49a001cf7c4bbca1dd866a0a9708… Author: John Chilton <jmchilton(a)gmail.com> Date: 2018-02-28 (Wed, 28 Feb 2018) Changed paths: M lib/galaxy/tools/deps/conda_util.py Log Message: ----------- Conda fixes for 18.01 - stdout redirection. Broken in #5397 I think. Commit: 79a5acaaec6d4b6e4ee36998977d85a8dc769612 https://github.com/galaxyproject/galaxy/commit/79a5acaaec6d4b6e4ee36998977d… Author: John Chilton <jmchilton(a)gmail.com> Date: 2018-02-28 (Wed, 28 Feb 2018) Changed paths: M lib/galaxy/tools/cwl/util.py M lib/galaxy/tools/deps/conda_util.py M lib/galaxy/tools/deps/containers.py M lib/galaxy/tools/verify/test_data.py Log Message: ----------- Conda fixes for dev - mostly Python 3. Commit: 0dbbfc7afcbde64a068c0bb321184b219b8c51af https://github.com/galaxyproject/galaxy/commit/0dbbfc7afcbde64a068c0bb32118… Author: Nicola Soranzo <nicola.soranzo(a)earlham.ac.uk> Date: 2018-02-28 (Wed, 28 Feb 2018) Changed paths: M lib/galaxy/tools/cwl/util.py M lib/galaxy/tools/verify/test_data.py Log Message: ----------- Prettier imports Commit: 77537ad55398b004cf0c4de3e7aaa11ce15a7a60 https://github.com/galaxyproject/galaxy/commit/77537ad55398b004cf0c4de3e7aa… Author: John Chilton <jmchilton(a)gmail.com> Date: 2018-02-28 (Wed, 28 Feb 2018) Changed paths: M lib/galaxy/tools/cwl/util.py Log Message: ----------- Tweak to Python 3 fix as suggested by @nsoranzo. Commit: cbade98a233db9f7a4c8ae28ccff2bc1258d5802 https://github.com/galaxyproject/galaxy/commit/cbade98a233db9f7a4c8ae28ccff… Author: Nicola Soranzo <nsoranzo(a)tiscali.it> Date: 2018-02-28 (Wed, 28 Feb 2018) Changed paths: M lib/galaxy/tools/cwl/util.py M lib/galaxy/tools/deps/conda_util.py M lib/galaxy/tools/deps/containers.py M lib/galaxy/tools/verify/test_data.py Log Message: ----------- Merge pull request #5621 from jmchilton/galaxy_lib_update Fixes from galaxy-lib Compare: https://github.com/galaxyproject/galaxy/compare/4caaa70006ce...cbade98a233d

1 0

[galaxyproject/usegalaxy-playbook] 038df8: Updated Diffbind
by GitHub 27 Feb '18

27 Feb '18

Branch: refs/heads/master Home: https://github.com/galaxyproject/usegalaxy-playbook Commit: 038df8395fb5e039500b016d60460cb2187b67f7 https://github.com/galaxyproject/usegalaxy-playbook/commit/038df8395fb5e039… Author: Nate Coraor <nate(a)bx.psu.edu> Date: 2018-02-27 (Tue, 27 Feb 2018) Changed paths: M env/main/files/galaxy/var/integrated_tool_panel.xml M env/main/files/galaxy/var/shed_tool_conf.xml Log Message: ----------- Updated Diffbind

1 0

[galaxyproject/usegalaxy-playbook] 829ec6: Test mutable configs for installation of Diffbind ...
by GitHub 27 Feb '18

27 Feb '18

Branch: refs/heads/master Home: https://github.com/galaxyproject/usegalaxy-playbook Commit: 829ec671430825b19f440d056bfaf7d78694d885 https://github.com/galaxyproject/usegalaxy-playbook/commit/829ec671430825b1… Author: Nate Coraor <nate(a)bx.psu.edu> Date: 2018-02-27 (Tue, 27 Feb 2018) Changed paths: M env/test/files/galaxy/var/integrated_tool_panel.xml M env/test/files/galaxy/var/shed_tool_conf.xml Log Message: ----------- Test mutable configs for installation of Diffbind and format-BIOM tools

1 0

[galaxyproject/usegalaxy-playbook] afab2f: Add support for reverting Galaxy and revert.
by GitHub 27 Feb '18

27 Feb '18

Branch: refs/heads/master Home: https://github.com/galaxyproject/usegalaxy-playbook Commit: afab2f57674523e6ae8be9ae5347f3ebd57807e7 https://github.com/galaxyproject/usegalaxy-playbook/commit/afab2f57674523e6… Author: Nate Coraor <nate(a)bx.psu.edu> Date: 2018-02-27 (Tue, 27 Feb 2018) Changed paths: M env/main/files/galaxy/galaxy_version.yml M env/main/group_vars/all.yml M roles/usegalaxy_cvmfs/tasks/update.yml Log Message: ----------- Add support for reverting Galaxy and revert.

1 0

[galaxyproject/usegalaxy-playbook] 789919: Update Main
by GitHub 27 Feb '18

27 Feb '18

Branch: refs/heads/master Home: https://github.com/galaxyproject/usegalaxy-playbook Commit: 78991922b95abbf073699f86078f7fba53d70053 https://github.com/galaxyproject/usegalaxy-playbook/commit/78991922b95abbf0… Author: Martin Cech <marten(a)bx.psu.edu> Date: 2018-02-27 (Tue, 27 Feb 2018) Changed paths: M env/main/files/galaxy/galaxy_version.yml Log Message: ----------- Update Main

1 0

[galaxyproject/galaxy] f6ac13: Don't fail rest of multi-action queue when attempt...
by GitHub 27 Feb '18

27 Feb '18

Branch: refs/heads/usegalaxy Home: https://github.com/galaxyproject/galaxy Commit: f6ac13568f62731fb785e3aafafbf3aacf21a7a2 https://github.com/galaxyproject/galaxy/commit/f6ac13568f62731fb785e3aafafb… Author: John Chilton <jmchilton(a)gmail.com> Date: 2018-02-18 (Sun, 18 Feb 2018) Changed paths: M client/galaxy/scripts/mvc/history/history-view-edit.js Log Message: ----------- Don't fail rest of multi-action queue when attempting to purge a collection. Fixes https://github.com/galaxyproject/galaxy/issues/5117. Commit: 31e31e692311ba2e9f11d3eabe5253628f8e39c9 https://github.com/galaxyproject/galaxy/commit/31e31e692311ba2e9f11d3eabe52… Author: nekrut <anton(a)nekrut.org> Date: 2018-02-19 (Mon, 19 Feb 2018) Changed paths: M config/datatypes_conf.xml.sample A lib/galaxy/datatypes/converters/fastagz_to_fasta.xml M lib/galaxy/datatypes/sequence.py M lib/galaxy/datatypes/sniff.py Log Message: ----------- added fasta.gz Commit: d79717b86e0b6ecf5e540eb939888066b88081de https://github.com/galaxyproject/galaxy/commit/d79717b86e0b6ecf5e540eb93988… Author: nekrut <anton(a)nekrut.org> Date: 2018-02-19 (Mon, 19 Feb 2018) Changed paths: M lib/galaxy/datatypes/sequence.py Log Message: ----------- fixed flake8 errors Commit: 8cec1fd592a354be9b055fbb8b7f3ce72df39990 https://github.com/galaxyproject/galaxy/commit/8cec1fd592a354be9b055fbb8b7f… Author: nekrut <anton(a)nekrut.org> Date: 2018-02-19 (Mon, 19 Feb 2018) Changed paths: M lib/galaxy/datatypes/sequence.py Log Message: ----------- removing stray whitespace Commit: d34524ff8adc694e41f0e582a57f705f2ed0dd7d https://github.com/galaxyproject/galaxy/commit/d34524ff8adc694e41f0e582a57f… Author: nekrut <anton(a)nekrut.org> Date: 2018-02-19 (Mon, 19 Feb 2018) Changed paths: M lib/galaxy/datatypes/converters/fastagz_to_fasta.xml Log Message: ----------- fixed name of fastagz->fasta converter Commit: c4ce9695bbb577a0e9514c5cb71069156d33a943 https://github.com/galaxyproject/galaxy/commit/c4ce9695bbb577a0e9514c5cb710… Author: Martin Cech <marten(a)bx.psu.edu> Date: 2018-02-19 (Mon, 19 Feb 2018) Changed paths: M lib/galaxy/datatypes/converters/fastagz_to_fasta.xml Log Message: ----------- fix copypasta Commit: 9feaf85585a30820dd71762431e1ed7d5b5c3b5e https://github.com/galaxyproject/galaxy/commit/9feaf85585a30820dd71762431e1… Author: Martin Cech <marten(a)bx.psu.edu> Date: 2018-02-19 (Mon, 19 Feb 2018) Changed paths: M lib/galaxy/datatypes/sequence.py Log Message: ----------- fix more fasta copypasta Commit: 17a5d20278f7306e69e77054f92d976020e048ee https://github.com/galaxyproject/galaxy/commit/17a5d20278f7306e69e77054f92d… Author: John Chilton <jmchilton(a)gmail.com> Date: 2018-02-19 (Mon, 19 Feb 2018) Changed paths: M run_tests.sh Log Message: ----------- In test framework only build client if running Selenium tests. These shouldn't be required for any other test types I believe and it slows things down. Commit: 6a3ce34c09bd516a18bd7d8f04851e1155d6d174 https://github.com/galaxyproject/galaxy/commit/6a3ce34c09bd516a18bd7d8f0485… Author: Martin Cech <marten(a)bx.psu.edu> Date: 2018-02-19 (Mon, 19 Feb 2018) Changed paths: M client/galaxy/scripts/mvc/history/history-view-edit.js Log Message: ----------- Merge pull request #5443 from jmchilton/fix5117 [18.01] Don't silently fail rest of multi-action queue when attempting to purge a collection. Commit: 2ca339d2b7cfdde817097837033e317a19b4a801 https://github.com/galaxyproject/galaxy/commit/2ca339d2b7cfdde817097837033e… Author: Jelle Scholtalbers <j.scholtalbers(a)gmail.com> Date: 2018-02-20 (Tue, 20 Feb 2018) Changed paths: M lib/galaxy/auth/__init__.py Log Message: ----------- fix broken admin 'create new user' when registration =='challenge' Commit: 01776c617b691c467e423c7a5f60c17be9b71503 https://github.com/galaxyproject/galaxy/commit/01776c617b691c467e423c7a5f60… Author: Dannon Baker <dannon.baker(a)gmail.com> Date: 2018-02-20 (Tue, 20 Feb 2018) Changed paths: M lib/galaxy/auth/__init__.py Log Message: ----------- Merge pull request #5569 from scholtalbers/fix/broken_create_new_user_challenge [18.01] fix broken admin 'create new user' when registration =='challenge' Commit: 99f7de456ce953f36e6122270209083fe443654b https://github.com/galaxyproject/galaxy/commit/99f7de456ce953f36e6122270209… Author: Martin Cech <marten(a)bx.psu.edu> Date: 2018-02-20 (Tue, 20 Feb 2018) Changed paths: M run_tests.sh Log Message: ----------- Merge pull request #5566 from jmchilton/optimize_tests_0 [18.01] In test framework only build client if running Selenium tests. Commit: 7715838dbbbcad980a47d1a96619b6e91fa7a623 https://github.com/galaxyproject/galaxy/commit/7715838dbbbcad980a47d1a96619… Author: Dannon Baker <dannon.baker(a)gmail.com> Date: 2018-02-20 (Tue, 20 Feb 2018) Changed paths: M config/datatypes_conf.xml.sample A lib/galaxy/datatypes/converters/fastagz_to_fasta.xml M lib/galaxy/datatypes/sequence.py M lib/galaxy/datatypes/sniff.py Log Message: ----------- Merge pull request #5561 from martenson/backport-fastgz [18.01] add fasta.gz support Commit: 93d5c54946e992112841df3461512bbeb6f5c7e9 https://github.com/galaxyproject/galaxy/commit/93d5c54946e992112841df346151… Author: John Chilton <jmchilton(a)gmail.com> Date: 2018-02-21 (Wed, 21 Feb 2018) Changed paths: M lib/galaxy/web/framework/helpers/grids.py Log Message: ----------- Optimize count query in grids (part 1). Many grid operations execute a complex query twice - once to produce the total number of rows and once to iterate through the selected parts of the data. This can be more optimal in many ways - this starts us down the path to addressing that by doing two things. The first thing it does is eliminates the query all together if paging is enabled but the user has opted to see "all" the rows. There was never a reason to execute the query in that case I don't think. The second thing it does is move the calculation of the count query until after the data has been generated - this isn't more optimal by itself but it allows us to potentially calculate the count as part of the main query. I've implemented this in a broken way @ https://github.com/galaxyproject/galaxy/commit/ea404721585b5f78da1b91e57e31… but I realized it was too broad an approach and doesn't work if certain kinds of joins are used in the initial query - but a more targetted approach on a per-grid basis could work I think and moving this code after allow us to potentially do that more easily in subsequent commits. Additionally I added a note - that I think it would be best to just return None as the number of pages and issue a second query to the same endpoint with the same arguments but with count_only=true added as a query request parameter so we can render the main grid faster and then render the paging elements after the second query finishes. This would feel more responsive I think. Commit: 1a782eb4eb29158a203df9e02b0195ecfe496e37 https://github.com/galaxyproject/galaxy/commit/1a782eb4eb29158a203df9e02b01… Author: John Chilton <jmchilton(a)gmail.com> Date: 2018-02-21 (Wed, 21 Feb 2018) Changed paths: M lib/galaxy/webapps/galaxy/controllers/workflow.py Log Message: ----------- Paginate published workflow grid. @martenson mentioned he prefers the look and feel of the published history grid so this replicates that and very likely will result in quicker rendering on usegalaxy.org - though we should verify this after it is deployed since the paginated grid do result in two big join queries instead of one big query - but I suspect rendering details on fewer objects (which includes fetching tag information from the db) will saved more than enough to make up for that. Commit: af0b40ea95c5244c58fbaa545b81ef4c4eff7cf0 https://github.com/galaxyproject/galaxy/commit/af0b40ea95c5244c58fbaa545b81… Author: Dannon Baker <dannon.baker(a)gmail.com> Date: 2018-02-21 (Wed, 21 Feb 2018) Changed paths: M lib/galaxy/web/proxy/js/package.json Log Message: ----------- Bump sqlite3 dependency for web proxy Commit: 5d422898582446575dfe30d13805c9e13a3f8104 https://github.com/galaxyproject/galaxy/commit/5d422898582446575dfe30d13805… Author: Dannon Baker <dannon.baker(a)gmail.com> Date: 2018-02-21 (Wed, 21 Feb 2018) Changed paths: M lib/galaxy/web/proxy/js/package.json Log Message: ----------- Minor updates to commander, http-proxy. eventemitter3 is several major versions behind, needs testing before upgrade. Commit: 1656c7b429854323efda82accc4c190a965be8a3 https://github.com/galaxyproject/galaxy/commit/1656c7b429854323efda82accc4c… Author: John Chilton <jmchilton(a)gmail.com> Date: 2018-02-21 (Wed, 21 Feb 2018) Changed paths: M lib/galaxy/tools/parameters/output_collect.py Log Message: ----------- Fix repeated fetching dataset permissions in dynamic output collection. Commit: b5b41d6172bd7e2894efcc3b423b11a37fb92823 https://github.com/galaxyproject/galaxy/commit/b5b41d6172bd7e2894efcc3b423b… Author: Björn Grüning <bjoern(a)gruenings.eu> Date: 2018-02-21 (Wed, 21 Feb 2018) Changed paths: M lib/galaxy/web/proxy/js/package.json Log Message: ----------- Merge pull request #5575 from dannon/web_proxy_fixes [18.01] Bump sqlite3 dependency for web proxy Commit: ff73cf2e6f8e1143e82681a9cfa97d769610f2c2 https://github.com/galaxyproject/galaxy/commit/ff73cf2e6f8e1143e82681a9cfa9… Author: Dannon Baker <dannon.baker(a)gmail.com> Date: 2018-02-21 (Wed, 21 Feb 2018) Changed paths: M lib/galaxy/datatypes/binary.py Log Message: ----------- Handle header-only bam files correctly indicating there is no more data to fetch when chunking Commit: 34c5a1a1f39e4805b4b6e46d86e9db721c98199a https://github.com/galaxyproject/galaxy/commit/34c5a1a1f39e4805b4b6e46d86e9… Author: Dannon Baker <dannon.baker(a)gmail.com> Date: 2018-02-21 (Wed, 21 Feb 2018) Changed paths: M lib/galaxy/datatypes/binary.py Log Message: ----------- Two minor linting issues (deprecated e.message, unused var) Commit: 75b06187053de95f07fdbbb6caeaf3e52265238e https://github.com/galaxyproject/galaxy/commit/75b06187053de95f07fdbbb6caea… Author: Nicola Soranzo <nicola.soranzo(a)earlham.ac.uk> Date: 2018-02-21 (Wed, 21 Feb 2018) Changed paths: M tools/data_source/upload.py Log Message: ----------- Do not remove external path files during library uploads xref. https://github.com/galaxyproject/galaxy/pull/5264#issuecomment-367172820 Commit: f7f9c120eda7f9d735257eecaeb759ad50473f57 https://github.com/galaxyproject/galaxy/commit/f7f9c120eda7f9d735257eecaeb7… Author: Nicola Soranzo <nicola.soranzo(a)earlham.ac.uk> Date: 2018-02-21 (Wed, 21 Feb 2018) Changed paths: M test/integration/test_upload_configuration_options.py Log Message: ----------- Refactor BaseFtpUploadConfigurationTestCase._write_ftp_file() method Commit: f41230034689448abc6059d8e2771001961bafb9 https://github.com/galaxyproject/galaxy/commit/f41230034689448abc6059d8e277… Author: Dannon Baker <dannon.baker(a)gmail.com> Date: 2018-02-21 (Wed, 21 Feb 2018) Changed paths: M lib/galaxy/datatypes/binary.py Log Message: ----------- No need to str() it; %s does this -- thanks @nsoranzo Commit: 117b4dcd6c503c51fb376c98ae8ba38d462d0301 https://github.com/galaxyproject/galaxy/commit/117b4dcd6c503c51fb376c98ae8b… Author: Nicola Soranzo <nicola.soranzo(a)earlham.ac.uk> Date: 2018-02-21 (Wed, 21 Feb 2018) Changed paths: M test/base/populators.py M test/integration/test_upload_configuration_options.py Log Message: ----------- Update integration test to check external path files are not deleted after library upload This test would have failed before commit 75b06187053de95f07fdbbb6caeaf3e52265238e. Commit: c00e772159f6a121bc4f7759fa57f48e79c48f32 https://github.com/galaxyproject/galaxy/commit/c00e772159f6a121bc4f7759fa57… Author: John Chilton <jmchilton(a)gmail.com> Date: 2018-02-21 (Wed, 21 Feb 2018) Changed paths: M client/galaxy/scripts/mvc/grid/grid-shared.js M client/galaxy/scripts/mvc/grid/grid-view.js M client/galaxy/scripts/mvc/history/history-list.js M client/galaxy/scripts/mvc/list/list-view.js M client/galaxy/scripts/mvc/workflow/workflow.js M client/galaxy/scripts/ui/loading-indicator.js Log Message: ----------- Fix lack of feedback when slow grids are loading data from the server. Fixes #5473 (after numerous other backend optimizations also). Commit: 8afeffe95cc63832a6208e119cbfea4671a49a47 https://github.com/galaxyproject/galaxy/commit/8afeffe95cc63832a6208e119cbf… Author: John Chilton <jmchilton(a)gmail.com> Date: 2018-02-21 (Wed, 21 Feb 2018) Changed paths: M lib/galaxy/datatypes/binary.py Log Message: ----------- Merge pull request #5579 from dannon/fix_bam_nodata [18.01] Fix small bam (header-only) infinite fetching Commit: b861b9ad4d693000ba8cf492ce6b1ff057db3d34 https://github.com/galaxyproject/galaxy/commit/b861b9ad4d693000ba8cf492ce6b… Author: Martin Cech <marten(a)bx.psu.edu> Date: 2018-02-21 (Wed, 21 Feb 2018) Changed paths: M lib/galaxy/tools/toolbox/base.py Log Message: ----------- implement basic cache on toolbox to_dict objects base the cache on absence of filters allow cache for users that don't opt in for filters Commit: 8cc675eedd8bb895cf26cfc1fa5cbcc97e75f638 https://github.com/galaxyproject/galaxy/commit/8cc675eedd8bb895cf26cfc1fa5c… Author: Martin Cech <marten(a)bx.psu.edu> Date: 2018-02-21 (Wed, 21 Feb 2018) Changed paths: M lib/galaxy/webapps/galaxy/controllers/workflow.py Log Message: ----------- Merge pull request #5576 from jmchilton/workflow_paginate [18.01] Paginate published workflows grid. Commit: d9372739376f5d289740faae40e8790391d8d843 https://github.com/galaxyproject/galaxy/commit/d9372739376f5d289740faae40e8… Author: John Chilton <jmchilton(a)gmail.com> Date: 2018-02-22 (Thu, 22 Feb 2018) Changed paths: M test/base/populators.py M test/integration/test_upload_configuration_options.py M tools/data_source/upload.py Log Message: ----------- Merge pull request #5573 from nsoranzo/release_18.01_fix_filesystem_library_uploads [18.01] Do not remove external path files during library uploads Commit: 8806030f53c2b9d0002505bf5193be88d7cff8f9 https://github.com/galaxyproject/galaxy/commit/8806030f53c2b9d0002505bf5193… Author: Nicola Soranzo <nicola.soranzo(a)earlham.ac.uk> Date: 2018-02-22 (Thu, 22 Feb 2018) Changed paths: M lib/galaxy/datatypes/converters/sam_to_bam.py M lib/galaxy/dependencies/pinned-requirements.txt M lib/galaxy/dependencies/requirements.txt M lib/galaxy/jobs/runners/pulsar.py M lib/galaxy/tools/__init__.py M lib/galaxy/tools/deps/conda_util.py M lib/galaxy/tools/deps/mulled/util.py M lib/galaxy/tools/toolbox/lineages/interface.py M lib/galaxy/visualization/data_providers/genome.py Log Message: ----------- Replace distutils.version with packaging.version distutils under Python3 has this bug: https://bugs.python.org/issue14894 which causes this traceback: ``` Traceback (most recent call last): File "lib/galaxy/webapps/galaxy/buildapp.py", line 49, in app_factory app = galaxy.app.UniverseApplication(global_conf=global_conf, **kwargs) File "lib/galaxy/app.py", line 125, in __init__ self._configure_toolbox() File "lib/galaxy/config.py", line 930, in _configure_toolbox self.toolbox = tools.ToolBox(tool_configs, self.config.tool_path, self) File "lib/galaxy/tools/__init__.py", line 232, in __init__ app=app, File "lib/galaxy/tools/toolbox/base.py", line 1056, in __init__ super(BaseGalaxyToolBox, self).__init__(config_filenames, tool_root_dir, app) File "lib/galaxy/tools/toolbox/base.py", line 84, in __init__ self._load_tool_panel() File "lib/galaxy/tools/toolbox/base.py", line 358, in _load_tool_panel self.__add_tool_to_tool_panel(section_val, section, section=True) File "lib/galaxy/tools/toolbox/base.py", line 278, in __add_tool_to_tool_panel related_tool = self._lineage_in_panel(panel_dict, tool=tool) File "lib/galaxy/tools/toolbox/base.py", line 961, in _lineage_in_panel lineage_tool_versions = reversed(tool_lineage.get_versions()) File "lib/galaxy/tools/toolbox/lineages/interface.py", line 77, in get_versions return [ToolLineageVersion(tool_id, tool_version) for tool_id, tool_version in zip(self.tool_ids, self.tool_versions)] File "lib/galaxy/tools/toolbox/lineages/interface.py", line 55, in tool_ids return ["%s/%s" % (tool_id, version) for version in self.tool_versions] File "lib/galaxy/tools/toolbox/lineages/interface.py", line 49, in tool_versions return sorted(self._tool_versions, key=LooseVersion) File "/usr/lib/python3.4/distutils/version.py", line 58, in __lt__ c = self._cmp(other) File "/usr/lib/python3.4/distutils/version.py", line 343, in _cmp if self.version < other.version: TypeError: unorderable types: str() < int() ``` when a tool version contains a letter in the "wrong" place (e.g. in the first position). Fix https://github.com/galaxyproject/galaxy/issues/5408 . Commit: ce3b40cfc10e134d5acc477757404eba40ea6553 https://github.com/galaxyproject/galaxy/commit/ce3b40cfc10e134d5acc47775740… Author: Marius van den Beek <m.vandenbeek(a)gmail.com> Date: 2018-02-22 (Thu, 22 Feb 2018) Changed paths: M lib/galaxy/tools/parameters/output_collect.py Log Message: ----------- Merge pull request #5578 from jmchilton/opt_dyn_collect [18.01] Fix repeated fetching dataset permissions in dynamic output collection. Commit: fc52fe5c62700b4f8527e4edead0fba4e7cfe33f https://github.com/galaxyproject/galaxy/commit/fc52fe5c62700b4f8527e4edead0… Author: Martin Cech <marten(a)bx.psu.edu> Date: 2018-02-22 (Thu, 22 Feb 2018) Changed paths: M lib/galaxy/datatypes/converters/sam_to_bam.py M lib/galaxy/dependencies/pinned-requirements.txt M lib/galaxy/dependencies/requirements.txt M lib/galaxy/jobs/runners/pulsar.py M lib/galaxy/tools/__init__.py M lib/galaxy/tools/deps/conda_util.py M lib/galaxy/tools/deps/mulled/util.py M lib/galaxy/tools/toolbox/lineages/interface.py M lib/galaxy/visualization/data_providers/genome.py Log Message: ----------- Merge pull request #5588 from jmchilton/backport_packaging [18.01] Replace distutils.version with packaging.version Commit: 8a88cb9827b68e6795cf00496008e1aa189185ad https://github.com/galaxyproject/galaxy/commit/8a88cb9827b68e6795cf00496008… Author: Martin Cech <marten(a)bx.psu.edu> Date: 2018-02-26 (Mon, 26 Feb 2018) Changed paths: M lib/galaxy/tools/toolbox/base.py Log Message: ----------- move the cache from toolbox level to tool level the cache remains toolbox-specific so it is stored within the toolbox object includes refactor thanks to eagle eyes Commit: e3ced2c669485f76eae7fcfb3b32aa048730964d https://github.com/galaxyproject/galaxy/commit/e3ced2c669485f76eae7fcfb3b32… Author: Dannon Baker <dannon.baker(a)gmail.com> Date: 2018-02-26 (Mon, 26 Feb 2018) Changed paths: M lib/galaxy/model/migrate/versions/0137_add_copied_from_job_id_column.py Log Message: ----------- Fix migration 0137's downgrade -- previously failed to select the correct column to drop Commit: c8407786f25c98c87f441f15222f72eeea4edc72 https://github.com/galaxyproject/galaxy/commit/c8407786f25c98c87f441f15222f… Author: Martin Cech <marten(a)bx.psu.edu> Date: 2018-02-26 (Mon, 26 Feb 2018) Changed paths: M lib/galaxy/model/migrate/versions/0137_add_copied_from_job_id_column.py Log Message: ----------- Merge pull request #5605 from dannon/migrate_fix [18.01] Fix migration 0137's downgrade Commit: d8585f6a6f87735d072635495bd49047da746624 https://github.com/galaxyproject/galaxy/commit/d8585f6a6f87735d072635495bd4… Author: Martin Cech <marten(a)bx.psu.edu> Date: 2018-02-27 (Tue, 27 Feb 2018) Changed paths: M client/galaxy/scripts/mvc/grid/grid-shared.js M client/galaxy/scripts/mvc/grid/grid-view.js M client/galaxy/scripts/mvc/history/history-list.js M client/galaxy/scripts/mvc/list/list-view.js M client/galaxy/scripts/mvc/workflow/workflow.js M client/galaxy/scripts/ui/loading-indicator.js Log Message: ----------- Merge pull request #5582 from jmchilton/indicators [18.01] Fix lack of feedback when slow grids are loading data from the server. Commit: 50cd159d4c212a36b8d3861295125ef055a296d9 https://github.com/galaxyproject/galaxy/commit/50cd159d4c212a36b8d386129512… Author: Martin Cech <marten(a)bx.psu.edu> Date: 2018-02-27 (Tue, 27 Feb 2018) Changed paths: M lib/galaxy/web/framework/helpers/grids.py Log Message: ----------- Merge pull request #5574 from jmchilton/optimize_count_query_1 [18.01] Optimize count query in grids (Part 1). Commit: a90c3a1d3f2c2a12f285f3c21e91ca364f81853e https://github.com/galaxyproject/galaxy/commit/a90c3a1d3f2c2a12f285f3c21e91… Author: John Chilton <jmchilton(a)gmail.com> Date: 2018-02-27 (Tue, 27 Feb 2018) Changed paths: M lib/galaxy/tools/toolbox/base.py Log Message: ----------- Merge pull request #5583 from martenson/backport-toolbox-cache [18.01] Backport toolbox cache Commit: ab928fe55ed1f871dbe6221e95af199579f312e1 https://github.com/galaxyproject/galaxy/commit/ab928fe55ed1f871dbe6221e95af… Author: Martin Cech <marten(a)bx.psu.edu> Date: 2018-02-27 (Tue, 27 Feb 2018) Changed paths: M client/galaxy/scripts/mvc/grid/grid-shared.js M client/galaxy/scripts/mvc/grid/grid-view.js M client/galaxy/scripts/mvc/history/history-list.js M client/galaxy/scripts/mvc/history/history-view-edit.js M client/galaxy/scripts/mvc/list/list-view.js M client/galaxy/scripts/mvc/workflow/workflow.js M client/galaxy/scripts/ui/loading-indicator.js M lib/galaxy/auth/__init__.py M lib/galaxy/datatypes/binary.py M lib/galaxy/datatypes/converters/sam_to_bam.py M lib/galaxy/dependencies/pinned-requirements.txt M lib/galaxy/dependencies/requirements.txt M lib/galaxy/jobs/runners/pulsar.py M lib/galaxy/model/migrate/versions/0137_add_copied_from_job_id_column.py M lib/galaxy/tools/__init__.py M lib/galaxy/tools/deps/conda_util.py M lib/galaxy/tools/deps/mulled/util.py M lib/galaxy/tools/parameters/output_collect.py M lib/galaxy/tools/toolbox/base.py M lib/galaxy/tools/toolbox/lineages/interface.py M lib/galaxy/visualization/data_providers/genome.py M lib/galaxy/web/framework/helpers/grids.py M lib/galaxy/web/proxy/js/package.json M lib/galaxy/webapps/galaxy/controllers/workflow.py M run_tests.sh M test/base/populators.py M test/integration/test_upload_configuration_options.py M tools/data_source/upload.py Log Message: ----------- Merge branch 'release_18.01' into usegalaxy Compare: https://github.com/galaxyproject/galaxy/compare/fa283b525f44...ab928fe55ed1

1 0

[galaxyproject/usegalaxy-playbook] a68841: Update Test
by GitHub 27 Feb '18

27 Feb '18

Branch: refs/heads/master Home: https://github.com/galaxyproject/usegalaxy-playbook Commit: a68841098821f3a5a2fde4afca042a724556e78f https://github.com/galaxyproject/usegalaxy-playbook/commit/a68841098821f3a5… Author: Martin Cech <marten(a)bx.psu.edu> Date: 2018-02-27 (Tue, 27 Feb 2018) Changed paths: M env/test/files/galaxy/galaxy_version.yml Log Message: ----------- Update Test

1 0

[galaxyproject/galaxy] 93d5c5: Optimize count query in grids (part 1).
by GitHub 27 Feb '18

27 Feb '18

Branch: refs/heads/dev Home: https://github.com/galaxyproject/galaxy Commit: 93d5c54946e992112841df3461512bbeb6f5c7e9 https://github.com/galaxyproject/galaxy/commit/93d5c54946e992112841df346151… Author: John Chilton <jmchilton(a)gmail.com> Date: 2018-02-21 (Wed, 21 Feb 2018) Changed paths: M lib/galaxy/web/framework/helpers/grids.py Log Message: ----------- Optimize count query in grids (part 1). Many grid operations execute a complex query twice - once to produce the total number of rows and once to iterate through the selected parts of the data. This can be more optimal in many ways - this starts us down the path to addressing that by doing two things. The first thing it does is eliminates the query all together if paging is enabled but the user has opted to see "all" the rows. There was never a reason to execute the query in that case I don't think. The second thing it does is move the calculation of the count query until after the data has been generated - this isn't more optimal by itself but it allows us to potentially calculate the count as part of the main query. I've implemented this in a broken way @ https://github.com/galaxyproject/galaxy/commit/ea404721585b5f78da1b91e57e31… but I realized it was too broad an approach and doesn't work if certain kinds of joins are used in the initial query - but a more targetted approach on a per-grid basis could work I think and moving this code after allow us to potentially do that more easily in subsequent commits. Additionally I added a note - that I think it would be best to just return None as the number of pages and issue a second query to the same endpoint with the same arguments but with count_only=true added as a query request parameter so we can render the main grid faster and then render the paging elements after the second query finishes. This would feel more responsive I think. Commit: 1656c7b429854323efda82accc4c190a965be8a3 https://github.com/galaxyproject/galaxy/commit/1656c7b429854323efda82accc4c… Author: John Chilton <jmchilton(a)gmail.com> Date: 2018-02-21 (Wed, 21 Feb 2018) Changed paths: M lib/galaxy/tools/parameters/output_collect.py Log Message: ----------- Fix repeated fetching dataset permissions in dynamic output collection. Commit: c00e772159f6a121bc4f7759fa57f48e79c48f32 https://github.com/galaxyproject/galaxy/commit/c00e772159f6a121bc4f7759fa57… Author: John Chilton <jmchilton(a)gmail.com> Date: 2018-02-21 (Wed, 21 Feb 2018) Changed paths: M client/galaxy/scripts/mvc/grid/grid-shared.js M client/galaxy/scripts/mvc/grid/grid-view.js M client/galaxy/scripts/mvc/history/history-list.js M client/galaxy/scripts/mvc/list/list-view.js M client/galaxy/scripts/mvc/workflow/workflow.js M client/galaxy/scripts/ui/loading-indicator.js Log Message: ----------- Fix lack of feedback when slow grids are loading data from the server. Fixes #5473 (after numerous other backend optimizations also). Commit: b861b9ad4d693000ba8cf492ce6b1ff057db3d34 https://github.com/galaxyproject/galaxy/commit/b861b9ad4d693000ba8cf492ce6b… Author: Martin Cech <marten(a)bx.psu.edu> Date: 2018-02-21 (Wed, 21 Feb 2018) Changed paths: M lib/galaxy/tools/toolbox/base.py Log Message: ----------- implement basic cache on toolbox to_dict objects base the cache on absence of filters allow cache for users that don't opt in for filters Commit: 8806030f53c2b9d0002505bf5193be88d7cff8f9 https://github.com/galaxyproject/galaxy/commit/8806030f53c2b9d0002505bf5193… Author: Nicola Soranzo <nicola.soranzo(a)earlham.ac.uk> Date: 2018-02-22 (Thu, 22 Feb 2018) Changed paths: M lib/galaxy/datatypes/converters/sam_to_bam.py M lib/galaxy/dependencies/pinned-requirements.txt M lib/galaxy/dependencies/requirements.txt M lib/galaxy/jobs/runners/pulsar.py M lib/galaxy/tools/__init__.py M lib/galaxy/tools/deps/conda_util.py M lib/galaxy/tools/deps/mulled/util.py M lib/galaxy/tools/toolbox/lineages/interface.py M lib/galaxy/visualization/data_providers/genome.py Log Message: ----------- Replace distutils.version with packaging.version distutils under Python3 has this bug: https://bugs.python.org/issue14894 which causes this traceback: ``` Traceback (most recent call last): File "lib/galaxy/webapps/galaxy/buildapp.py", line 49, in app_factory app = galaxy.app.UniverseApplication(global_conf=global_conf, **kwargs) File "lib/galaxy/app.py", line 125, in __init__ self._configure_toolbox() File "lib/galaxy/config.py", line 930, in _configure_toolbox self.toolbox = tools.ToolBox(tool_configs, self.config.tool_path, self) File "lib/galaxy/tools/__init__.py", line 232, in __init__ app=app, File "lib/galaxy/tools/toolbox/base.py", line 1056, in __init__ super(BaseGalaxyToolBox, self).__init__(config_filenames, tool_root_dir, app) File "lib/galaxy/tools/toolbox/base.py", line 84, in __init__ self._load_tool_panel() File "lib/galaxy/tools/toolbox/base.py", line 358, in _load_tool_panel self.__add_tool_to_tool_panel(section_val, section, section=True) File "lib/galaxy/tools/toolbox/base.py", line 278, in __add_tool_to_tool_panel related_tool = self._lineage_in_panel(panel_dict, tool=tool) File "lib/galaxy/tools/toolbox/base.py", line 961, in _lineage_in_panel lineage_tool_versions = reversed(tool_lineage.get_versions()) File "lib/galaxy/tools/toolbox/lineages/interface.py", line 77, in get_versions return [ToolLineageVersion(tool_id, tool_version) for tool_id, tool_version in zip(self.tool_ids, self.tool_versions)] File "lib/galaxy/tools/toolbox/lineages/interface.py", line 55, in tool_ids return ["%s/%s" % (tool_id, version) for version in self.tool_versions] File "lib/galaxy/tools/toolbox/lineages/interface.py", line 49, in tool_versions return sorted(self._tool_versions, key=LooseVersion) File "/usr/lib/python3.4/distutils/version.py", line 58, in __lt__ c = self._cmp(other) File "/usr/lib/python3.4/distutils/version.py", line 343, in _cmp if self.version < other.version: TypeError: unorderable types: str() < int() ``` when a tool version contains a letter in the "wrong" place (e.g. in the first position). Fix https://github.com/galaxyproject/galaxy/issues/5408 . Commit: ce3b40cfc10e134d5acc477757404eba40ea6553 https://github.com/galaxyproject/galaxy/commit/ce3b40cfc10e134d5acc47775740… Author: Marius van den Beek <m.vandenbeek(a)gmail.com> Date: 2018-02-22 (Thu, 22 Feb 2018) Changed paths: M lib/galaxy/tools/parameters/output_collect.py Log Message: ----------- Merge pull request #5578 from jmchilton/opt_dyn_collect [18.01] Fix repeated fetching dataset permissions in dynamic output collection. Commit: fc52fe5c62700b4f8527e4edead0fba4e7cfe33f https://github.com/galaxyproject/galaxy/commit/fc52fe5c62700b4f8527e4edead0… Author: Martin Cech <marten(a)bx.psu.edu> Date: 2018-02-22 (Thu, 22 Feb 2018) Changed paths: M lib/galaxy/datatypes/converters/sam_to_bam.py M lib/galaxy/dependencies/pinned-requirements.txt M lib/galaxy/dependencies/requirements.txt M lib/galaxy/jobs/runners/pulsar.py M lib/galaxy/tools/__init__.py M lib/galaxy/tools/deps/conda_util.py M lib/galaxy/tools/deps/mulled/util.py M lib/galaxy/tools/toolbox/lineages/interface.py M lib/galaxy/visualization/data_providers/genome.py Log Message: ----------- Merge pull request #5588 from jmchilton/backport_packaging [18.01] Replace distutils.version with packaging.version Commit: 8a88cb9827b68e6795cf00496008e1aa189185ad https://github.com/galaxyproject/galaxy/commit/8a88cb9827b68e6795cf00496008… Author: Martin Cech <marten(a)bx.psu.edu> Date: 2018-02-26 (Mon, 26 Feb 2018) Changed paths: M lib/galaxy/tools/toolbox/base.py Log Message: ----------- move the cache from toolbox level to tool level the cache remains toolbox-specific so it is stored within the toolbox object includes refactor thanks to eagle eyes Commit: e3ced2c669485f76eae7fcfb3b32aa048730964d https://github.com/galaxyproject/galaxy/commit/e3ced2c669485f76eae7fcfb3b32… Author: Dannon Baker <dannon.baker(a)gmail.com> Date: 2018-02-26 (Mon, 26 Feb 2018) Changed paths: M lib/galaxy/model/migrate/versions/0137_add_copied_from_job_id_column.py Log Message: ----------- Fix migration 0137's downgrade -- previously failed to select the correct column to drop Commit: c8407786f25c98c87f441f15222f72eeea4edc72 https://github.com/galaxyproject/galaxy/commit/c8407786f25c98c87f441f15222f… Author: Martin Cech <marten(a)bx.psu.edu> Date: 2018-02-26 (Mon, 26 Feb 2018) Changed paths: M lib/galaxy/model/migrate/versions/0137_add_copied_from_job_id_column.py Log Message: ----------- Merge pull request #5605 from dannon/migrate_fix [18.01] Fix migration 0137's downgrade Commit: d8585f6a6f87735d072635495bd49047da746624 https://github.com/galaxyproject/galaxy/commit/d8585f6a6f87735d072635495bd4… Author: Martin Cech <marten(a)bx.psu.edu> Date: 2018-02-27 (Tue, 27 Feb 2018) Changed paths: M client/galaxy/scripts/mvc/grid/grid-shared.js M client/galaxy/scripts/mvc/grid/grid-view.js M client/galaxy/scripts/mvc/history/history-list.js M client/galaxy/scripts/mvc/list/list-view.js M client/galaxy/scripts/mvc/workflow/workflow.js M client/galaxy/scripts/ui/loading-indicator.js Log Message: ----------- Merge pull request #5582 from jmchilton/indicators [18.01] Fix lack of feedback when slow grids are loading data from the server. Commit: 50cd159d4c212a36b8d3861295125ef055a296d9 https://github.com/galaxyproject/galaxy/commit/50cd159d4c212a36b8d386129512… Author: Martin Cech <marten(a)bx.psu.edu> Date: 2018-02-27 (Tue, 27 Feb 2018) Changed paths: M lib/galaxy/web/framework/helpers/grids.py Log Message: ----------- Merge pull request #5574 from jmchilton/optimize_count_query_1 [18.01] Optimize count query in grids (Part 1). Commit: a90c3a1d3f2c2a12f285f3c21e91ca364f81853e https://github.com/galaxyproject/galaxy/commit/a90c3a1d3f2c2a12f285f3c21e91… Author: John Chilton <jmchilton(a)gmail.com> Date: 2018-02-27 (Tue, 27 Feb 2018) Changed paths: M lib/galaxy/tools/toolbox/base.py Log Message: ----------- Merge pull request #5583 from martenson/backport-toolbox-cache [18.01] Backport toolbox cache Commit: 4caaa70006cead1c286c1bbf5f036860a3fd8a85 https://github.com/galaxyproject/galaxy/commit/4caaa70006cead1c286c1bbf5f03… Author: Martin Cech <marten(a)bx.psu.edu> Date: 2018-02-27 (Tue, 27 Feb 2018) Changed paths: M client/galaxy/scripts/mvc/grid/grid-shared.js M client/galaxy/scripts/mvc/grid/grid-view.js M client/galaxy/scripts/mvc/history/history-list.js M client/galaxy/scripts/mvc/list/list-view.js M client/galaxy/scripts/mvc/workflow/workflow.js M client/galaxy/scripts/ui/loading-indicator.js M lib/galaxy/model/migrate/versions/0137_add_copied_from_job_id_column.py M lib/galaxy/tools/parameters/output_collect.py M lib/galaxy/web/framework/helpers/grids.py Log Message: ----------- Merge branch 'release_18.01' into dev Compare: https://github.com/galaxyproject/galaxy/compare/73a0d72e0c98...4caaa70006ce

1 0

[galaxyproject/galaxy] b861b9: implement basic cache on toolbox to_dict objects
by GitHub 27 Feb '18

27 Feb '18

Branch: refs/heads/release_18.01 Home: https://github.com/galaxyproject/galaxy Commit: b861b9ad4d693000ba8cf492ce6b1ff057db3d34 https://github.com/galaxyproject/galaxy/commit/b861b9ad4d693000ba8cf492ce6b… Author: Martin Cech <marten(a)bx.psu.edu> Date: 2018-02-21 (Wed, 21 Feb 2018) Changed paths: M lib/galaxy/tools/toolbox/base.py Log Message: ----------- implement basic cache on toolbox to_dict objects base the cache on absence of filters allow cache for users that don't opt in for filters Commit: 8a88cb9827b68e6795cf00496008e1aa189185ad https://github.com/galaxyproject/galaxy/commit/8a88cb9827b68e6795cf00496008… Author: Martin Cech <marten(a)bx.psu.edu> Date: 2018-02-26 (Mon, 26 Feb 2018) Changed paths: M lib/galaxy/tools/toolbox/base.py Log Message: ----------- move the cache from toolbox level to tool level the cache remains toolbox-specific so it is stored within the toolbox object includes refactor thanks to eagle eyes Commit: a90c3a1d3f2c2a12f285f3c21e91ca364f81853e https://github.com/galaxyproject/galaxy/commit/a90c3a1d3f2c2a12f285f3c21e91… Author: John Chilton <jmchilton(a)gmail.com> Date: 2018-02-27 (Tue, 27 Feb 2018) Changed paths: M lib/galaxy/tools/toolbox/base.py Log Message: ----------- Merge pull request #5583 from martenson/backport-toolbox-cache [18.01] Backport toolbox cache Compare: https://github.com/galaxyproject/galaxy/compare/50cd159d4c21...a90c3a1d3f2c

1 0

[galaxyproject/galaxy] 93d5c5: Optimize count query in grids (part 1).
by GitHub 27 Feb '18

27 Feb '18

Branch: refs/heads/release_18.01 Home: https://github.com/galaxyproject/galaxy Commit: 93d5c54946e992112841df3461512bbeb6f5c7e9 https://github.com/galaxyproject/galaxy/commit/93d5c54946e992112841df346151… Author: John Chilton <jmchilton(a)gmail.com> Date: 2018-02-21 (Wed, 21 Feb 2018) Changed paths: M lib/galaxy/web/framework/helpers/grids.py Log Message: ----------- Optimize count query in grids (part 1). Many grid operations execute a complex query twice - once to produce the total number of rows and once to iterate through the selected parts of the data. This can be more optimal in many ways - this starts us down the path to addressing that by doing two things. The first thing it does is eliminates the query all together if paging is enabled but the user has opted to see "all" the rows. There was never a reason to execute the query in that case I don't think. The second thing it does is move the calculation of the count query until after the data has been generated - this isn't more optimal by itself but it allows us to potentially calculate the count as part of the main query. I've implemented this in a broken way @ https://github.com/galaxyproject/galaxy/commit/ea404721585b5f78da1b91e57e31… but I realized it was too broad an approach and doesn't work if certain kinds of joins are used in the initial query - but a more targetted approach on a per-grid basis could work I think and moving this code after allow us to potentially do that more easily in subsequent commits. Additionally I added a note - that I think it would be best to just return None as the number of pages and issue a second query to the same endpoint with the same arguments but with count_only=true added as a query request parameter so we can render the main grid faster and then render the paging elements after the second query finishes. This would feel more responsive I think. Commit: 50cd159d4c212a36b8d3861295125ef055a296d9 https://github.com/galaxyproject/galaxy/commit/50cd159d4c212a36b8d386129512… Author: Martin Cech <marten(a)bx.psu.edu> Date: 2018-02-27 (Tue, 27 Feb 2018) Changed paths: M lib/galaxy/web/framework/helpers/grids.py Log Message: ----------- Merge pull request #5574 from jmchilton/optimize_count_query_1 [18.01] Optimize count query in grids (Part 1). Compare: https://github.com/galaxyproject/galaxy/compare/d8585f6a6f87...50cd159d4c21

1 0