Branch: refs/heads/dev Home: https://github.com/galaxyproject/galaxy Commit: bd921368413002f6b57e0c6ee229b2ff06e7482d https://github.com/galaxyproject/galaxy/commit/bd921368413002f6b57e0c6ee229b... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-10 (Fri, 10 Aug 2018) Changed paths: M lib/galaxy/dependencies/pipfiles/develop/Pipfile M lib/galaxy/dependencies/pipfiles/develop/pinned-requirements.txt A pytest.ini M run_tests.sh M scripts/plugin_staging.py M test/docker/base/Dockerfile M test/docker/base/run_test_wrapper.sh Log Message: ----------- Setup pytest python3 API testing Commit: 1b3f5550548741370e70ff2b25bc404e84d2594e https://github.com/galaxyproject/galaxy/commit/1b3f5550548741370e70ff2b25bc4... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-10 (Fri, 10 Aug 2018) Changed paths: M run_tests.sh Log Message: ----------- Bump docker image version Commit: 7342a7ebbc404c275fb0635c1628eecb92b56ff5 https://github.com/galaxyproject/galaxy/commit/7342a7ebbc404c275fb0635c1628e... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-10 (Fri, 10 Aug 2018) Changed paths: M test/docker/base/Dockerfile Log Message: ----------- Avoid prepopulating database twice Commit: 4e3a981003f9bd0c99a604327e17f37e16f07f33 https://github.com/galaxyproject/galaxy/commit/4e3a981003f9bd0c99a604327e17f... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-10 (Fri, 10 Aug 2018) Changed paths: M run_tests.sh Log Message: ----------- Temporarily run tests with image published in my channel Commit: 661b3101c635fc376d9c16878fd8b23122ab4950 https://github.com/galaxyproject/galaxy/commit/661b3101c635fc376d9c16878fd8b... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-10 (Fri, 10 Aug 2018) Changed paths: A .ci/jenkins/api-py3/run_tests.sh Log Message: ----------- Add py3 build script for jenkins Commit: 39e1877ccedd36242bc93696d897f62d5c8f0126 https://github.com/galaxyproject/galaxy/commit/39e1877ccedd36242bc93696d897f... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-10 (Fri, 10 Aug 2018) Changed paths: M test/docker/base/Dockerfile Log Message: ----------- Explicitly create a python 3 virtualenv Commit: 5b02f5ccedd87a9d529d19a49950df57cc29a8cf https://github.com/galaxyproject/galaxy/commit/5b02f5ccedd87a9d529d19a49950d... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-10 (Fri, 10 Aug 2018) Changed paths: M run_tests.sh Log Message: ----------- Bump docker image version Commit: eb1cb57f424d7ab286658699d2313621ed9e1d4e https://github.com/galaxyproject/galaxy/commit/eb1cb57f424d7ab286658699d2313... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-10 (Fri, 10 Aug 2018) Changed paths: M test/docker/base/Dockerfile Log Message: ----------- Fix up Dockerfile for bdbag installation Commit: 091ba0f32c75b6046938c95b0521b3271d9c3a72 https://github.com/galaxyproject/galaxy/commit/091ba0f32c75b6046938c95b0521b... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-10 (Fri, 10 Aug 2018) Changed paths: M run_tests.sh Log Message: ----------- Use 18.09.0 tag Commit: 327ec3d4e9e9d0ab15ae07c2cd0596d6a8abbe90 https://github.com/galaxyproject/galaxy/commit/327ec3d4e9e9d0ab15ae07c2cd059... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-10 (Fri, 10 Aug 2018) Changed paths: M .ci/jenkins/api-py3/run_tests.sh Log Message: ----------- Do use python 3 Commit: a6b78e7a614aaf3e8254fdea32723f6f06678b5e https://github.com/galaxyproject/galaxy/commit/a6b78e7a614aaf3e8254fdea32723... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-10 (Fri, 10 Aug 2018) Changed paths: M test/base/populators.py Log Message: ----------- Decode bytestring returned by source_string Commit: d659d91d331eb434221940a1cddbd5458d75d8b3 https://github.com/galaxyproject/galaxy/commit/d659d91d331eb434221940a1cddbd... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-10 (Fri, 10 Aug 2018) Changed paths: M lib/galaxy/web/framework/middleware/batch.py Log Message: ----------- Decode body when using json.loads in batch middleware Commit: 61fa9711dc406dc15b434630aff3c3497f18fd52 https://github.com/galaxyproject/galaxy/commit/61fa9711dc406dc15b434630aff3c... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-10 (Fri, 10 Aug 2018) Changed paths: M test/base/populators.py M test/base/rules_test_data.py Log Message: ----------- More resource_string fixes Commit: 03f80f8aef781908f4290f613de9ee78f58b632c https://github.com/galaxyproject/galaxy/commit/03f80f8aef781908f4290f613de9e... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-10 (Fri, 10 Aug 2018) Changed paths: M lib/galaxy/model/custom_types.py Log Message: ----------- Patch LargeBinary to cast strings to bytestring I don't fully understand the issue, and we might be accumulating technical debt here. What I'm seeing without this commit is this when I run for instance the VisualizationsApiTestCase.test_create API test. ``` Traceback (most recent call last): File "/galaxy/lib/galaxy/web/framework/decorators.py", line 283, in decorator rval = func(self, trans, *args, **kwargs) File "/galaxy/lib/galaxy/webapps/galaxy/api/visualizations.py", line 55, in index visualizations = self.get_visualizations_by_user(trans, user) File "/galaxy/lib/galaxy/web/base/controller.py", line 664, in get_visualizations_by_user return query.all() File "/galaxy_venv3/lib/python3.5/site-packages/sqlalchemy/orm/query.py", line 2737, in all return list(self) File "/galaxy_venv3/lib/python3.5/site-packages/sqlalchemy/orm/loading.py", line 98, in instances util.raise_from_cause(err) File "/galaxy_venv3/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/galaxy_venv3/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 187, in reraise raise value File "/galaxy_venv3/lib/python3.5/site-packages/sqlalchemy/orm/loading.py", line 79, in instances rows = [proc(row) for row in fetch] File "/galaxy_venv3/lib/python3.5/site-packages/sqlalchemy/orm/loading.py", line 79, in <listcomp> rows = [proc(row) for row in fetch] File "/galaxy_venv3/lib/python3.5/site-packages/sqlalchemy/orm/loading.py", line 493, in _instance loaded_instance, populate_existing, populators) File "/galaxy_venv3/lib/python3.5/site-packages/sqlalchemy/orm/loading.py", line 604, in _populate_full populator(state, dict_, row) File "/galaxy_venv3/lib/python3.5/site-packages/sqlalchemy/orm/strategies.py", line 1782, in load_scalar_from_joined_new_row dict_[key] = _instance(row) File "/galaxy_venv3/lib/python3.5/site-packages/sqlalchemy/orm/loading.py", line 493, in _instance loaded_instance, populate_existing, populators) File "/galaxy_venv3/lib/python3.5/site-packages/sqlalchemy/orm/loading.py", line 593, in _populate_full dict_[key] = getter(row) File "/galaxy_venv3/lib/python3.5/site-packages/sqlalchemy/sql/type_api.py", line 1204, in process return process_value(impl_processor(value), dialect) File "/galaxy_venv3/lib/python3.5/site-packages/sqlalchemy/sql/sqltypes.py", line 902, in process value = bytes(value) TypeError: string argument without an encoding ``` I think LargeBinary isn't meant to store strings the way we do this (we dump JSON as bytestring), and somehow on python 3 we get unicode strings back from the database, which blows up here. I suspect this might be configured on the connection level (see http://docs.sqlalchemy.org/en/latest/dialects/postgresql.html#unicode), but not blowing up here seems like a good default. Commit: 2527038ffce1f7f1b987f9318671bc4fec567ec0 https://github.com/galaxyproject/galaxy/commit/2527038ffce1f7f1b987f9318671b... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-10 (Fri, 10 Aug 2018) Changed paths: M run_tests.sh M test/docker/base/run_test_wrapper.sh Log Message: ----------- Mount in run_tests script at container runtime This allows changing variables in the script without rebuilding the container Commit: 3d9c0b154ab744345808d36c8af38972524ab2c3 https://github.com/galaxyproject/galaxy/commit/3d9c0b154ab744345808d36c8af38... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-10 (Fri, 10 Aug 2018) Changed paths: M test/api/test_workflows.py M test/base/data/test_workflow_with_runtime_input.ga M tools/filters/random_lines_two_pass.py M tools/filters/randomlines.xml Log Message: ----------- Make random_lines1 consistent on py2 and py3 (again) Turns out integers are used as is in `random.seed()`, but strings are hashed (in the version 1 protocol), and the hashing isn't stable. So instead we hack around this by using `ord()` on each character and concatenating the digits. Yucky, but probably better than using numpy and putting this tool on the list of tools that inherit galaxy's environment (or adding a requirement tag and resolving with conda ...). Commit: b2fed75059767aece3c0a75c630c9f30e31f0078 https://github.com/galaxyproject/galaxy/commit/b2fed75059767aece3c0a75c630c9... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-11 (Sat, 11 Aug 2018) Changed paths: M run_tests.sh Log Message: ----------- Flip testing image back to galaxy namespace Commit: ae3d41924ac844d22c454fbd01d49a9ac3ee3b74 https://github.com/galaxyproject/galaxy/commit/ae3d41924ac844d22c454fbd01d49... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-14 (Tue, 14 Aug 2018) Changed paths: M lib/galaxy/webapps/galaxy/api/tool_shed_repositories.py M lib/tool_shed/galaxy_install/repository_dependencies/repository_dependency_manager.py M lib/tool_shed/util/encoding_util.py Log Message: ----------- py3: Fix repository installation I'm very surprised this actually worked. I suppose we called my system's `hg` ?! Commit: 4d5ea3d724cf1a7fac1c6a2f9eeac133c47c2a06 https://github.com/galaxyproject/galaxy/commit/4d5ea3d724cf1a7fac1c6a2f9eeac... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-14 (Tue, 14 Aug 2018) Changed paths: A .ci/jenkins/integration-py3/run_tests.sh M run_tests.sh Log Message: ----------- Start testing integration tests on python 3 Commit: b47c246e0c688be2f9474c36e1d12b786deb2ac0 https://github.com/galaxyproject/galaxy/commit/b47c246e0c688be2f9474c36e1d12... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-14 (Tue, 14 Aug 2018) Changed paths: M lib/galaxy/datatypes/data.py Log Message: ----------- Fix showing, creating and downloading HTML files Commit: 37b5120197d8a8bf39b52585408998252e601d82 https://github.com/galaxyproject/galaxy/commit/37b5120197d8a8bf39b5258540899... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-14 (Tue, 14 Aug 2018) Changed paths: M lib/galaxy/tools/data/__init__.py M lib/galaxy/util/renamed_temporary_file.py M lib/tool_shed/galaxy_install/install_manager.py M lib/tool_shed/galaxy_install/tools/data_manager.py M test/integration/test_data_manager_table_reload.py M test/integration/tool_sheds_conf.xml M test/integration/uses_shed.py Log Message: ----------- py3: fix installing and running data managers Commit: 167e0e912907821943bd93eab2c0986a3d31136c https://github.com/galaxyproject/galaxy/commit/167e0e912907821943bd93eab2c09... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-14 (Tue, 14 Aug 2018) Changed paths: M lib/galaxy/tools/verify/interactor.py Log Message: ----------- Fix dictify_outputs The hack of setting `.__len` doesn't work on python 3. Subclassing OrderedDict and providing list-like access using integers should have the same effect. Commit: 14b1e1acd4eedce4a86b96094a496cbb76dd4239 https://github.com/galaxyproject/galaxy/commit/14b1e1acd4eedce4a86b96094a496... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-14 (Tue, 14 Aug 2018) Changed paths: M lib/galaxy/util/none_like.py Log Message: ----------- py3: Fix optional dataset bool value This fixes cheetah like `#if $input.dataset: ... #end if`. Which is a little weird, but fastqc uses this pattern. Commit: 344f68ea6d9c48a2a6f9dd56dd434633167d8f8e https://github.com/galaxyproject/galaxy/commit/344f68ea6d9c48a2a6f9dd56dd434... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-14 (Tue, 14 Aug 2018) Changed paths: M lib/galaxy/web/stack/__init__.py M lib/galaxy/web/stack/transport.py M scripts/galaxy-main Log Message: ----------- py3: Fix uwsgi mule startup Commit: 0b30a94bbe34fb40eaf6fc97c816fcbe0586d078 https://github.com/galaxyproject/galaxy/commit/0b30a94bbe34fb40eaf6fc97c816f... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-14 (Tue, 14 Aug 2018) Changed paths: M lib/galaxy/webapps/galaxy/api/uploads.py Log Message: ----------- py3: Fix new direct API-backed uploads Commit: dc4a4b0eb5c9a07d9048e893b9fa48b43eb58c09 https://github.com/galaxyproject/galaxy/commit/dc4a4b0eb5c9a07d9048e893b9fa4... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-14 (Tue, 14 Aug 2018) Changed paths: M scripts/secret_decoder_ring.py Log Message: ----------- py3: fix secret decoder script Commit: 293ce5a4720c7fc2ec26d44985750188e3762e6b https://github.com/galaxyproject/galaxy/commit/293ce5a4720c7fc2ec26d44985750... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-14 (Tue, 14 Aug 2018) Changed paths: M test/base/driver_util.py M test/integration/test_mules_as_job_handlers.py Log Message: ----------- py3: Fix mules test cases We're now just copying the current python path. Also the handler names have actually changed, I think this hasn't been updated after a late change (it is now as it should be, based on the documentation). Commit: cfce2468451e52c72bf7f0544c1bc7e209cc9224 https://github.com/galaxyproject/galaxy/commit/cfce2468451e52c72bf7f0544c1bc... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-14 (Tue, 14 Aug 2018) Changed paths: M test/integration/test_scripts.py Log Message: ----------- Fix GRT export test Commit: 620375c9d045e0085e2b260a97b3fdd5912fb45a https://github.com/galaxyproject/galaxy/commit/620375c9d045e0085e2b260a97b3f... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-14 (Tue, 14 Aug 2018) Changed paths: M test/integration/test_mules_as_job_handlers.py Log Message: ----------- Do test mules on Jenkins, I don't see why this shouldn't work Commit: 6e7a66cdc0b9f467341af77d4d2fd5eadc9f409d https://github.com/galaxyproject/galaxy/commit/6e7a66cdc0b9f467341af77d4d2fd... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-14 (Tue, 14 Aug 2018) Changed paths: M lib/tool_shed/galaxy_install/repository_dependencies/repository_dependency_manager.py Log Message: ----------- Fix repository dependency manager Commit: dd7e4c601a3f31ce9c4fa06afc8234a39f0a5804 https://github.com/galaxyproject/galaxy/commit/dd7e4c601a3f31ce9c4fa06afc823... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-14 (Tue, 14 Aug 2018) Changed paths: M test/base/driver_util.py Log Message: ----------- Deal with docker localhost quirks Commit: fc06e78743086d3078d18e3928512cf62915bfb9 https://github.com/galaxyproject/galaxy/commit/fc06e78743086d3078d18e3928512... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-14 (Tue, 14 Aug 2018) Changed paths: M lib/galaxy/datatypes/sequence.py Log Message: ----------- py3: Fix Maf datatype Commit: dfba132d11b0f6f6d15cb9fc8c79e0645640489e https://github.com/galaxyproject/galaxy/commit/dfba132d11b0f6f6d15cb9fc8c79e... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2018-08-14 (Tue, 14 Aug 2018) Changed paths: M lib/galaxy/jobs/output_checker.py M lib/galaxy/jobs/runners/__init__.py M lib/galaxy/jobs/runners/local.py M lib/galaxy/util/__init__.py Log Message: ----------- Fix stdout/stderr exception handling Commit: 853b2d03abcfe563b12c817db84d871fbc291725 https://github.com/galaxyproject/galaxy/commit/853b2d03abcfe563b12c817db84d8... Author: John Chilton <jmchilton@gmail.com> Date: 2018-08-14 (Tue, 14 Aug 2018) Changed paths: A .ci/jenkins/api-py3/run_tests.sh A .ci/jenkins/integration-py3/run_tests.sh M lib/galaxy/datatypes/data.py M lib/galaxy/datatypes/sequence.py M lib/galaxy/dependencies/pipfiles/develop/Pipfile M lib/galaxy/dependencies/pipfiles/develop/pinned-requirements.txt M lib/galaxy/jobs/output_checker.py M lib/galaxy/jobs/runners/__init__.py M lib/galaxy/jobs/runners/local.py M lib/galaxy/model/custom_types.py M lib/galaxy/tools/data/__init__.py M lib/galaxy/tools/verify/interactor.py M lib/galaxy/util/__init__.py M lib/galaxy/util/none_like.py M lib/galaxy/util/renamed_temporary_file.py M lib/galaxy/web/framework/middleware/batch.py M lib/galaxy/web/stack/__init__.py M lib/galaxy/web/stack/transport.py M lib/galaxy/webapps/galaxy/api/tool_shed_repositories.py M lib/galaxy/webapps/galaxy/api/uploads.py M lib/tool_shed/galaxy_install/install_manager.py M lib/tool_shed/galaxy_install/repository_dependencies/repository_dependency_manager.py M lib/tool_shed/galaxy_install/tools/data_manager.py M lib/tool_shed/util/encoding_util.py A pytest.ini M run_tests.sh M scripts/galaxy-main M scripts/plugin_staging.py M scripts/secret_decoder_ring.py M test/api/test_workflows.py M test/base/data/test_workflow_with_runtime_input.ga M test/base/driver_util.py M test/base/populators.py M test/base/rules_test_data.py M test/docker/base/Dockerfile M test/docker/base/run_test_wrapper.sh M test/integration/test_data_manager_table_reload.py M test/integration/test_mules_as_job_handlers.py M test/integration/test_scripts.py M test/integration/tool_sheds_conf.xml M test/integration/uses_shed.py M tools/filters/random_lines_two_pass.py M tools/filters/randomlines.xml Log Message: ----------- Merge pull request #6565 from mvdbeek/setup_py3_test_job Setup py3 test job Compare: https://github.com/galaxyproject/galaxy/compare/164a7d7f04be...853b2d03abcf **NOTE:** This service has been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/ Functionality will be removed from GitHub.com on January 31st, 2019.