August 2018 - galaxy-commits - lists.galaxyproject.org

[galaxyproject/galaxy] c852de: Fix cron/build_chrom_db.py
by GitHub 22 Aug '18

22 Aug '18

Branch: refs/heads/dev Home: https://github.com/galaxyproject/galaxy Commit: c852de734dbc8d95ac925c103f4cad563d3f9e53 https://github.com/galaxyproject/galaxy/commit/c852de734dbc8d95ac925c103f4c… Author: Nicola Soranzo <nicola.soranzo(a)earlham.ac.uk> Date: 2018-08-22 (Wed, 22 Aug 2018) Changed paths: M cron/build_chrom_db.py Log Message: ----------- Fix cron/build_chrom_db.py Fix the following errors: ``` $ python cron/build_chrom_db.py tool-data/shared/ucsc/chrom/ tool-data/shared/ucsc/builds.txt Retrieving hg38 Failed to retrieve hg38: Problem parsing line '' Retrieving hg19 Failed to retrieve hg19: Problem parsing line '' ... ``` Broken in commit 9b8e76f9d9df0ae0ba6238d44a8f07b97fe34d6f . Commit: 926f0abde2c5ae6caf0727d890c2f90a34587cec https://github.com/galaxyproject/galaxy/commit/926f0abde2c5ae6caf0727d890c2… Author: Nicola Soranzo <nicola.soranzo(a)earlham.ac.uk> Date: 2018-08-22 (Wed, 22 Aug 2018) Changed paths: M tool-data/shared/ucsc/builds.txt.sample M tool-data/shared/ucsc/ucsc_build_sites.txt.sample Log Message: ----------- Update UCSC builds Commit: 89f237216d7d5c2030c2d620af0f95c6318ac14c https://github.com/galaxyproject/galaxy/commit/89f237216d7d5c2030c2d620af0f… Author: John Chilton <jmchilton(a)gmail.com> Date: 2018-08-22 (Wed, 22 Aug 2018) Changed paths: M cron/build_chrom_db.py M tool-data/shared/ucsc/builds.txt.sample M tool-data/shared/ucsc/ucsc_build_sites.txt.sample Log Message: ----------- Merge pull request #6623 from nsoranzo/ucsc_builds Fix cron/build_chrom_db.py and update UCSC builds Compare: https://github.com/galaxyproject/galaxy/compare/498e7d576bb4...89f237216d7d **NOTE:** This service has been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/ Functionality will be removed from GitHub.com on January 31st, 2019.

1 0

[galaxyproject/galaxy] 7882a0: Allow configuring object stores on a per-job-desti...
by GitHub 22 Aug '18

22 Aug '18

Branch: refs/heads/dev Home: https://github.com/galaxyproject/galaxy Commit: 7882a067c8cbcf564aff605b91beea8d7849fb00 https://github.com/galaxyproject/galaxy/commit/7882a067c8cbcf564aff605b91be… Author: John Chilton <jmchilton(a)gmail.com> Date: 2018-08-06 (Mon, 06 Aug 2018) Changed paths: M lib/galaxy/jobs/__init__.py M lib/galaxy/objectstore/__init__.py A test/integration/objectstore_selection_job_conf.xml A test/integration/objectstore_selection_job_resource_parameters_conf.xml A test/integration/objectstore_selection_rules/__init__.py A test/integration/objectstore_selection_rules/rules.py A test/integration/test_objectstore_selection.py Log Message: ----------- Allow configuring object stores on a per-job-destination basis. Setup a hierarchical objectstore with ids - these ids can be zero weight for objectstores that should only be used on a per-job-destination basis. Job destinations can then set object_store_id as a param to force a particular object store from the hierarchical configuration to be used. This can be set in job_conf.xml directly for static mappings or combined with dynamic job destinations for dynamic mapping to object stores. Dynamic job destinations can dispatch on cluster conditions, user, user preferences, etc.. Additionally, job resource parameters can be used with dynamic job destinations to allow user input on the object store to use. An integration test case is included that tests and demonstrates all these different pieces. It sets up a Galaxy with a job and object store configuration that routes one tool to a specific disk store (called "static"), and routes another tool dynamically using job resource parameters that allow the user to pick between "slow, cheap" and "fast, expensive" storage options (for the test it is all just disk stores used - but you can imagine choosing between EBS and S3 or something this way). I'd be happy to see something more formal than job resource parameters or user preferences developed to help users route job outputs to particular object stores - this is just a demonstration of what can be done today after these changes using extension points already being used by admins and users. Commit: 498e7d576bb40e7f2eea47b1d787638a2c507965 https://github.com/galaxyproject/galaxy/commit/498e7d576bb40e7f2eea47b1d787… Author: Dannon <dannon.baker(a)gmail.com> Date: 2018-08-22 (Wed, 22 Aug 2018) Changed paths: M lib/galaxy/jobs/__init__.py M lib/galaxy/objectstore/__init__.py A test/integration/objectstore_selection_job_conf.xml A test/integration/objectstore_selection_job_resource_parameters_conf.xml A test/integration/objectstore_selection_rules/__init__.py A test/integration/objectstore_selection_rules/rules.py A test/integration/test_objectstore_selection.py Log Message: ----------- Merge pull request #6552 from jmchilton/per_destination_object_store Allow configuring object stores on a per-job-destination basis. Compare: https://github.com/galaxyproject/galaxy/compare/53b8f08752ed...498e7d576bb4 **NOTE:** This service has been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/ Functionality will be removed from GitHub.com on January 31st, 2019.

1 0

[galaxyproject/galaxy] 4602d4: Implement dataset permissions api
by GitHub 22 Aug '18

22 Aug '18

Branch: refs/heads/dev Home: https://github.com/galaxyproject/galaxy Commit: 4602d413cc80e7478eaa8de164aa8fde014de500 https://github.com/galaxyproject/galaxy/commit/4602d413cc80e7478eaa8de164aa… Author: Helena Rasche <hxr(a)hx42.org> Date: 2018-08-10 (Fri, 10 Aug 2018) Changed paths: M lib/galaxy/webapps/galaxy/api/datasets.py M lib/galaxy/webapps/galaxy/buildapp.py Log Message: ----------- Implement dataset permissions api Commit: 409d5fcc8057309df71e2baa1b9bd37b16a5883a https://github.com/galaxyproject/galaxy/commit/409d5fcc8057309df71e2baa1b9b… Author: Helena Rasche <hxr(a)hx42.org> Date: 2018-08-10 (Fri, 10 Aug 2018) Changed paths: M lib/galaxy/webapps/galaxy/api/datasets.py Log Message: ----------- remove unnecessary doc Commit: fe5794916496364dbabfd97f7ad3fba6d91f7336 https://github.com/galaxyproject/galaxy/commit/fe5794916496364dbabfd97f7ad3… Author: Helena Rasche <hxr(a)hx42.org> Date: 2018-08-10 (Fri, 10 Aug 2018) Changed paths: M lib/galaxy/webapps/galaxy/api/datasets.py Log Message: ----------- fix linting Commit: 19523352ced554868fa6979dfe37161be7c864dd https://github.com/galaxyproject/galaxy/commit/19523352ced554868fa6979dfe37… Author: Helena Rasche <hxr(a)hx42.org> Date: 2018-08-10 (Fri, 10 Aug 2018) Changed paths: M lib/galaxy/webapps/galaxy/api/datasets.py Log Message: ----------- Correct documentation URL Commit: f7176dcfc9c63c7230c3ddc127e4e7718be49b9e https://github.com/galaxyproject/galaxy/commit/f7176dcfc9c63c7230c3ddc127e4… Author: Helena Rasche <hxr(a)hx42.org> Date: 2018-08-10 (Fri, 10 Aug 2018) Changed paths: M lib/galaxy/webapps/galaxy/api/datasets.py Log Message: ----------- Allow exception to bubble up naturally Commit: 6067fee3d44e6106217012ed9d7f029691e5d99f https://github.com/galaxyproject/galaxy/commit/6067fee3d44e6106217012ed9d7f… Author: John Chilton <jmchilton(a)gmail.com> Date: 2018-08-10 (Fri, 10 Aug 2018) Changed paths: M lib/galaxy/managers/datasets.py M test/unit/managers/test_DatasetManager.py Log Message: ----------- Remove unused dataset deserializer stuff. An unfinished thread left in the code base. It was a fine idea but I don't think anyone is left to pick it up and run with it. Commit: 18148a9656242369220d2ba1f0f10d89f2d44f2e https://github.com/galaxyproject/galaxy/commit/18148a9656242369220d2ba1f0f1… Author: John Chilton <jmchilton(a)gmail.com> Date: 2018-08-10 (Fri, 10 Aug 2018) Changed paths: M lib/galaxy/security/__init__.py M lib/galaxy/webapps/galaxy/api/folder_contents.py M lib/galaxy/webapps/galaxy/api/library_datasets.py Log Message: ----------- Refactor dataset_is_private_to_user to consume dataset instead of LibraryDataset. More consistent with other methods in that file and allows for use without knowledge of how to go from a LD -> Dataset. Commit: 49693814a9702a855d7df166d27c7a6f4f1faba1 https://github.com/galaxyproject/galaxy/commit/49693814a9702a855d7df166d27c… Author: John Chilton <jmchilton(a)gmail.com> Date: 2018-08-10 (Fri, 10 Aug 2018) Changed paths: M lib/galaxy/managers/datasets.py M lib/galaxy/managers/hdas.py M lib/galaxy/managers/lddas.py M lib/galaxy/managers/rbac_secured.py M lib/galaxy/managers/sharable.py M lib/galaxy/managers/users.py M lib/galaxy/web/base/controller.py M lib/galaxy/webapps/galaxy/api/datasets.py M lib/galaxy/webapps/galaxy/api/history_contents.py M lib/galaxy/webapps/galaxy/api/library_datasets.py M lib/galaxy/webapps/galaxy/buildapp.py M test/api/test_dataset_collections.py M test/api/test_history_contents.py A test/api/test_roles.py M test/api/test_tools.py M test/base/api.py M test/base/populators.py Log Message: ----------- Unified dataset permission API. There was an existing library dataset permission API and one proposed for HDAs in #6461. This tries to reconcile the two approaches and move toward managers. - Proper treatment of "master_api_key" throughout the manager layers related to this. - Use PUT instead of POST, since that is more appropriate for updates I believe. - Use a similar URL pattern for library datasets, history contents, and dataset APIs. - Generalize the dataset API to allow HDA or LDDA inputs (the proposal for the new API in #6461 added an HDA-only option to the dataset API - this isn't how that API layer is meant to be used I believe). - Add an HDA only endpoint in the history contents API for this functionality to mirror the library dataset API. - Use consistent and backward compatible payload parsing for these endpoints. - Use consistent serialization for the result of these changes. - Generalize the HDA variant of this to allow different actions - such as make public and make private that were available in the library dataset variant. - Test cases for the new API endpoints, the roles API, tests for permissions during collection creation and running tools with both dataset and collection inputs. Commit: 53b8f08752edc35839f1ed9a8bc6355a528c99b4 https://github.com/galaxyproject/galaxy/commit/53b8f08752edc35839f1ed9a8bc6… Author: Helena Rasche <erasche(a)users.noreply.github.com> Date: 2018-08-22 (Wed, 22 Aug 2018) Changed paths: M lib/galaxy/managers/datasets.py M lib/galaxy/managers/hdas.py M lib/galaxy/managers/lddas.py M lib/galaxy/managers/rbac_secured.py M lib/galaxy/managers/sharable.py M lib/galaxy/managers/users.py M lib/galaxy/security/__init__.py M lib/galaxy/web/base/controller.py M lib/galaxy/webapps/galaxy/api/datasets.py M lib/galaxy/webapps/galaxy/api/folder_contents.py M lib/galaxy/webapps/galaxy/api/history_contents.py M lib/galaxy/webapps/galaxy/api/library_datasets.py M lib/galaxy/webapps/galaxy/buildapp.py M test/api/test_dataset_collections.py M test/api/test_history_contents.py A test/api/test_roles.py M test/api/test_tools.py M test/base/api.py M test/base/populators.py M test/unit/managers/test_DatasetManager.py Log Message: ----------- Merge pull request #6568 from jmchilton/dataset-permissions-api-2 Unified dataset permission API. Compare: https://github.com/galaxyproject/galaxy/compare/fc167c068c9d...53b8f08752ed **NOTE:** This service has been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/ Functionality will be removed from GitHub.com on January 31st, 2019.

1 0

[galaxyproject/galaxy] 6cce98: [minor] Fix incorrect import in display_common.mak...
by GitHub 22 Aug '18

22 Aug '18

Branch: refs/heads/dev Home: https://github.com/galaxyproject/galaxy Commit: 6cce9830a7c77b22d8baaa889b34b53faf933b67 https://github.com/galaxyproject/galaxy/commit/6cce9830a7c77b22d8baaa889b34… Author: John Chilton <jmchilton(a)gmail.com> Date: 2018-08-22 (Wed, 22 Aug 2018) Changed paths: M templates/display_common.mako Log Message: ----------- [minor] Fix incorrect import in display_common.mako. Broke a few Selenium tests, not sure which commit broke this but it was fairly recent because these tests were passing two weeks ago. Commit: fc167c068c9d3b5fcdcce4514801ea2626c7d6d3 https://github.com/galaxyproject/galaxy/commit/fc167c068c9d3b5fcdcce4514801… Author: Dannon <dannon.baker(a)gmail.com> Date: 2018-08-22 (Wed, 22 Aug 2018) Changed paths: M templates/display_common.mako Log Message: ----------- Merge pull request #6622 from jmchilton/selenium_fixes_46 [minor] Fix incorrect import in display_common.mako. Compare: https://github.com/galaxyproject/galaxy/compare/46dc4db25a5c...fc167c068c9d **NOTE:** This service has been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/ Functionality will be removed from GitHub.com on January 31st, 2019.

1 0

[galaxyproject/galaxy] 43f539: Cloud: raise an exception if a history with decode...
by GitHub 21 Aug '18

21 Aug '18

Branch: refs/heads/dev Home: https://github.com/galaxyproject/galaxy Commit: 43f539ce729f8c0be06ff0d5d8cf6f26db52a148 https://github.com/galaxyproject/galaxy/commit/43f539ce729f8c0be06ff0d5d8cf… Author: vjalili <jalili.vahid(a)gmail.com> Date: 2018-08-16 (Thu, 16 Aug 2018) Changed paths: M lib/galaxy/managers/cloud.py Log Message: ----------- Cloud: raise an exception if a history with decoded id does not exist. Commit: aa0df987bbe4790eb830b86b3dc2f2db2b8edd85 https://github.com/galaxyproject/galaxy/commit/aa0df987bbe4790eb830b86b3dc2… Author: Vahid <VJalili(a)users.noreply.github.com> Date: 2018-08-21 (Tue, 21 Aug 2018) Changed paths: M lib/galaxy/managers/cloud.py Log Message: ----------- Update doc string. Commit: 46dc4db25a5c44788ff454650e91b383fb506837 https://github.com/galaxyproject/galaxy/commit/46dc4db25a5c44788ff454650e91… Author: Dannon <dannon.baker(a)gmail.com> Date: 2018-08-21 (Tue, 21 Aug 2018) Changed paths: M lib/galaxy/managers/cloud.py Log Message: ----------- Merge pull request #6611 from VJalili/patch_cloud_upload Raise an exception if a history with decoded id does not exist. Compare: https://github.com/galaxyproject/galaxy/compare/99517c6fdf3d...46dc4db25a5c **NOTE:** This service has been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/ Functionality will be removed from GitHub.com on January 31st, 2019.

1 0

[galaxyproject/galaxy] 78a8c8: Revert unnecessary changes
by GitHub 21 Aug '18

21 Aug '18

Branch: refs/heads/dev Home: https://github.com/galaxyproject/galaxy Commit: 78a8c865a13b98bd3dc93a2b0dade86d2d6e43bf https://github.com/galaxyproject/galaxy/commit/78a8c865a13b98bd3dc93a2b0dad… Author: Nicola Soranzo <nicola.soranzo(a)earlham.ac.uk> Date: 2018-08-20 (Mon, 20 Aug 2018) Changed paths: M lib/galaxy/datatypes/sniff.py M lib/galaxy/datatypes/text.py M lib/galaxy/tools/evaluation.py M lib/galaxy/web/framework/webapp.py Log Message: ----------- Revert unnecessary changes Commit: 5bddd24d8b8994f9f59d0f7ba541321c8146a166 https://github.com/galaxyproject/galaxy/commit/5bddd24d8b8994f9f59d0f7ba541… Author: Nicola Soranzo <nicola.soranzo(a)earlham.ac.uk> Date: 2018-08-20 (Mon, 20 Aug 2018) Changed paths: M test/api/test_jobs.py Log Message: ----------- Remove unreachable return statement Commit: 4a3ed12f0af3f9b26dc9cb5b66eabb26f9dc542e https://github.com/galaxyproject/galaxy/commit/4a3ed12f0af3f9b26dc9cb5b66ea… Author: Nicola Soranzo <nicola.soranzo(a)earlham.ac.uk> Date: 2018-08-21 (Tue, 21 Aug 2018) Changed paths: M .gitignore Log Message: ----------- Ignore .pytest_cache/ , assets/ and run_integration_tests.html Commit: df8abbe3ef13c0b288becba07af4a05b8f6dc668 https://github.com/galaxyproject/galaxy/commit/df8abbe3ef13c0b288becba07af4… Author: Nicola Soranzo <nicola.soranzo(a)earlham.ac.uk> Date: 2018-08-21 (Tue, 21 Aug 2018) Changed paths: M lib/galaxy/exceptions/error_codes.py M lib/galaxy/jobs/runners/util/job_script/__init__.py M lib/galaxy/tools/deps/container_resolvers/mulled.py M lib/galaxy/web/framework/middleware/batch.py M lib/tool_shed/galaxy_install/repository_dependencies/repository_dependency_manager.py M lib/tool_shed/util/encoding_util.py M scripts/secret_decoder_ring.py M test/base/populators.py M test/base/rules_test_data.py M test/integration/test_scripts.py Log Message: ----------- Prefer `unicodify()` for binary-to-text conversion which supports non-utf-8 encodings. Commit: 5d1ced2678737e1ceebb727c3b81d4546ee6588b https://github.com/galaxyproject/galaxy/commit/5d1ced2678737e1ceebb727c3b81… Author: Nicola Soranzo <nicola.soranzo(a)earlham.ac.uk> Date: 2018-08-21 (Tue, 21 Aug 2018) Changed paths: M lib/galaxy/tools/imp_exp/export_history.py M lib/galaxy/util/checkers.py Log Message: ----------- Do not open text file in binary mode Commit: f54246e1a1d157ba43783dc2a86f40aeb5c7b7f2 https://github.com/galaxyproject/galaxy/commit/f54246e1a1d157ba43783dc2a86f… Author: Nicola Soranzo <nicola.soranzo(a)earlham.ac.uk> Date: 2018-08-21 (Tue, 21 Aug 2018) Changed paths: M lib/galaxy/tools/imp_exp/imp_history_from_archive.xml M lib/galaxy/web/framework/middleware/batch.py M test/api/test_authenticate.py Log Message: ----------- Use `encode()` method instead of `bytearray()` to encode a Unicode string to a binary one. Commit: 72f020726593bcd6026cba5a127ff615df49968f https://github.com/galaxyproject/galaxy/commit/72f020726593bcd6026cba5a127f… Author: Nicola Soranzo <nicola.soranzo(a)earlham.ac.uk> Date: 2018-08-21 (Tue, 21 Aug 2018) Changed paths: M lib/galaxy/util/__init__.py Log Message: ----------- Remove unused `binary_chars` parameter of `smart_str()` Commit: d265b8a88ada5d2e57efbc03740c697ab5c7464f https://github.com/galaxyproject/galaxy/commit/d265b8a88ada5d2e57efbc03740c… Author: Nicola Soranzo <nicola.soranzo(a)earlham.ac.uk> Date: 2018-08-21 (Tue, 21 Aug 2018) Changed paths: M lib/galaxy/util/__init__.py Log Message: ----------- Raise an exception if `unicodify()` fails the conversion Commit: 61041a7c6b09664b2fa9d8d479de4251647cec87 https://github.com/galaxyproject/galaxy/commit/61041a7c6b09664b2fa9d8d479de… Author: Nicola Soranzo <nicola.soranzo(a)earlham.ac.uk> Date: 2018-08-21 (Tue, 21 Aug 2018) Changed paths: M lib/galaxy/tools/wrappers.py Log Message: ----------- Raise `AttributeError` exception in `DatasetCollectionWrapper.__getattr__()` Commit: cd11a8c19ab672d8326cb474d7833cf532ae82cc https://github.com/galaxyproject/galaxy/commit/cd11a8c19ab672d8326cb474d783… Author: Nicola Soranzo <nicola.soranzo(a)earlham.ac.uk> Date: 2018-08-21 (Tue, 21 Aug 2018) Changed paths: M lib/galaxy/tools/wrappers.py Log Message: ----------- Use `NamedTemporaryFile` opened in text mode Commit: 670d8b1623b1ba16f2b23cc661e581600556ae61 https://github.com/galaxyproject/galaxy/commit/670d8b1623b1ba16f2b23cc661e5… Author: Nicola Soranzo <nicola.soranzo(a)earlham.ac.uk> Date: 2018-08-21 (Tue, 21 Aug 2018) Changed paths: R test/functional/twilltestcase.py M test/shed_functional/base/twilltestcase.py Log Message: ----------- Merge TwillTestCase in ShedTwillTestCase Commit: ff15f3679818dc8f0ea4610ed28cc5957ac68901 https://github.com/galaxyproject/galaxy/commit/ff15f3679818dc8f0ea4610ed28c… Author: Nicola Soranzo <nicola.soranzo(a)earlham.ac.uk> Date: 2018-08-21 (Tue, 21 Aug 2018) Changed paths: M .ci/jenkins/api-py3/run_tests.sh M .ci/jenkins/framework-py3/run_tests.sh M .ci/jenkins/integration-py3/run_tests.sh M run_tests.sh Log Message: ----------- Run Py2 API and integration tests with pytest Remove `-api_py3`, `--framework_py3` and `--integration_py3` options of `run_tests.sh`. To run the tests under Python3 it is enough to prepend `GALAXY_VIRTUAL_ENV=.venv3` (or add `--python3` when using `--dockerize`). Commit: c501fb1647b21720dfbaeedb062084813095284e https://github.com/galaxyproject/galaxy/commit/c501fb1647b21720dfbaeedb0620… Author: Nicola Soranzo <nicola.soranzo(a)earlham.ac.uk> Date: 2018-08-21 (Tue, 21 Aug 2018) Changed paths: M lib/galaxy/util/bunch.py M lib/galaxy/util/expressions.py M lib/galaxy/util/none_like.py Log Message: ----------- Alias `__nonzero__` to `__bool__` Commit: 05999f9767b9ad906a719f2e103c96e64e1b41a7 https://github.com/galaxyproject/galaxy/commit/05999f9767b9ad906a719f2e103c… Author: Nicola Soranzo <nicola.soranzo(a)earlham.ac.uk> Date: 2018-08-21 (Tue, 21 Aug 2018) Changed paths: M lib/galaxy/web/stack/__init__.py Log Message: ----------- Unicodify all farm elements Commit: ff0d28c35c2c7c5371f2101e72c00259ebd4950b https://github.com/galaxyproject/galaxy/commit/ff0d28c35c2c7c5371f2101e72c0… Author: Nicola Soranzo <nicola.soranzo(a)earlham.ac.uk> Date: 2018-08-21 (Tue, 21 Aug 2018) Changed paths: M lib/galaxy/tools/evaluation.py M lib/galaxy/util/template.py M test/functional/tools/unicode_stream.xml Log Message: ----------- Fix unicode_stream framework tests on Docker This fixes the following traceback: ``` ( unicode_stream ) > Test-1 ... galaxy.tools DEBUG 2018-08-21 09:40:59,052 Validated and populated state for tool request (0.098 ms) galaxy.tools.actions INFO 2018-08-21 09:40:59,076 Handled output named out_file1 for tool unicode_stream (1.337 ms) galaxy.tools.actions INFO 2018-08-21 09:40:59,080 Added output datasets to history (4.399 ms) galaxy.tools.actions INFO 2018-08-21 09:40:59,084 Verified access to datasets for Job[unflushed,tool_id=unicode_stream] (0.058 ms) galaxy.tools.actions INFO 2018-08-21 09:40:59,086 Setup for job Job[unflushed,tool_id=unicode_stream] complete, ready to flush (5.220 ms) galaxy.tools.actions INFO 2018-08-21 09:40:59,127 Flushed transaction for job Job[id=354,tool_id=unicode_stream] (40.627 ms) galaxy.tools.execute DEBUG 2018-08-21 09:40:59,127 Tool [unicode_stream] created job [354] (56.506 ms) galaxy.tools.execute DEBUG 2018-08-21 09:40:59,132 Executed 1 job(s) for tool unicode_stream request: (79.434 ms) galaxy.jobs.mapper DEBUG 2018-08-21 09:40:59,556 (354) Mapped job to destination id: local galaxy.jobs.handler DEBUG 2018-08-21 09:40:59,561 (354) Dispatching to local runner galaxy.jobs DEBUG 2018-08-21 09:40:59,571 (354) Working directory for job is: /tmp/tmp3drfmr6x/tmp_ywd33ar/tmpbzo4a1d9/database/job_working_directory_azry9a1y/000/354 galaxy.jobs DEBUG 2018-08-21 09:40:59,574 (354) Persisting job destination (destination id: local) galaxy.jobs.runners DEBUG 2018-08-21 09:40:59,589 Job [354] queued (27.361 ms) galaxy.jobs.handler INFO 2018-08-21 09:40:59,604 (354) Job dispatched galaxy.jobs.runners ERROR 2018-08-21 09:40:59,715 (354) Failure preparing job Traceback (most recent call last): File "/galaxy/lib/galaxy/jobs/runners/__init__.py", line 187, in prepare_job job_wrapper.prepare() File "/galaxy/lib/galaxy/jobs/__init__.py", line 874, in prepare self.command_line, self.extra_filenames, self.environment_variables = tool_evaluator.build() File "/galaxy/lib/galaxy/tools/evaluation.py", line 435, in build raise e File "/galaxy/lib/galaxy/tools/evaluation.py", line 431, in build self.__build_config_files() File "/galaxy/lib/galaxy/tools/evaluation.py", line 504, in __build_config_files self.__write_workdir_file(config_filename, config_text, param_dict, is_template=is_template) File "/galaxy/lib/galaxy/tools/evaluation.py", line 578, in __write_workdir_file f.write(value) UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-5: ordinal not in range(128) galaxy.tools.error_reports DEBUG 2018-08-21 09:40:59,869 Bug report plugin <galaxy.tools.error_reports.plugins.sentry.SentryPlugin object at 0x7f92efb81668> generated response None FAIL ``` which is due to the fact that the default encoding in the Docker image is `ANSI_X3.4-1968`. Also update `unicode_stream` test tool to IUC best practices. Commit: 99517c6fdf3d07bf98dad6f88946bca4fb82fea1 https://github.com/galaxyproject/galaxy/commit/99517c6fdf3d07bf98dad6f88946… Author: Marius van den Beek <m.vandenbeek(a)gmail.com> Date: 2018-08-21 (Tue, 21 Aug 2018) Changed paths: M .ci/jenkins/api-py3/run_tests.sh M .ci/jenkins/framework-py3/run_tests.sh M .ci/jenkins/integration-py3/run_tests.sh M .gitignore M lib/galaxy/datatypes/sniff.py M lib/galaxy/datatypes/text.py M lib/galaxy/exceptions/error_codes.py M lib/galaxy/jobs/runners/util/job_script/__init__.py M lib/galaxy/tools/deps/container_resolvers/mulled.py M lib/galaxy/tools/evaluation.py M lib/galaxy/tools/imp_exp/export_history.py M lib/galaxy/tools/imp_exp/imp_history_from_archive.xml M lib/galaxy/tools/wrappers.py M lib/galaxy/util/__init__.py M lib/galaxy/util/bunch.py M lib/galaxy/util/checkers.py M lib/galaxy/util/expressions.py M lib/galaxy/util/none_like.py M lib/galaxy/util/template.py M lib/galaxy/web/framework/middleware/batch.py M lib/galaxy/web/framework/webapp.py M lib/galaxy/web/stack/__init__.py M lib/tool_shed/galaxy_install/repository_dependencies/repository_dependency_manager.py M lib/tool_shed/util/encoding_util.py M run_tests.sh M scripts/secret_decoder_ring.py M test/api/test_authenticate.py M test/api/test_jobs.py M test/base/populators.py M test/base/rules_test_data.py M test/functional/tools/unicode_stream.xml R test/functional/twilltestcase.py M test/integration/test_scripts.py M test/shed_functional/base/twilltestcase.py Log Message: ----------- Merge pull request #6616 from nsoranzo/followup_on_6561 Py3 testing cleanups Compare: https://github.com/galaxyproject/galaxy/compare/e07cc9a96009...99517c6fdf3d **NOTE:** This service has been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/ Functionality will be removed from GitHub.com on January 31st, 2019.

1 0

[galaxyproject/usegalaxy-playbook] 90b047: Keep one plant tribes DM.
by GitHub 20 Aug '18

20 Aug '18

Branch: refs/heads/master Home: https://github.com/galaxyproject/usegalaxy-playbook Commit: 90b047b9ce7fef208e29e2d659f1124aac441660 https://github.com/galaxyproject/usegalaxy-playbook/commit/90b047b9ce7fef20… Author: Dave B <dave(a)bx.psu.edu> Date: 2018-08-20 (Mon, 20 Aug 2018) Changed paths: M env/test/files/galaxy/var/shed_data_manager_conf.xml M env/test/files/galaxy/var/shed_tool_data_table_conf.xml Log Message: ----------- Keep one plant tribes DM. **NOTE:** This service has been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/ Functionality will be removed from GitHub.com on January 31st, 2019.

1 0

[galaxyproject/usegalaxy-playbook] 395521: Update id_secret
by GitHub 20 Aug '18

20 Aug '18

Branch: refs/heads/master Home: https://github.com/galaxyproject/usegalaxy-playbook Commit: 395521c421db4aa46d6aa61b81cf034c9f55c030 https://github.com/galaxyproject/usegalaxy-playbook/commit/395521c421db4aa4… Author: Nate Coraor <nate(a)bx.psu.edu> Date: 2018-08-20 (Mon, 20 Aug 2018) Changed paths: M env/main/group_vars/galaxyservers/vault.yml Log Message: ----------- Update id_secret Commit: 000bebd7fa4afffaabab1e41507cfb8fa198dc36 https://github.com/galaxyproject/usegalaxy-playbook/commit/000bebd7fa4afffa… Author: Nate Coraor <nate(a)bx.psu.edu> Date: 2018-08-20 (Mon, 20 Aug 2018) Changed paths: M env/main/group_vars/galaxyservers/vars.yml Log Message: ----------- Remove downtime banner Compare: https://github.com/galaxyproject/usegalaxy-playbook/compare/241e79e5f057...… **NOTE:** This service has been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/ Functionality will be removed from GitHub.com on January 31st, 2019.

1 0

[galaxyproject/galaxy] 9e07d9: Handle driver shutdown, show all test in pytest
by GitHub 20 Aug '18

20 Aug '18

Branch: refs/heads/dev Home: https://github.com/galaxyproject/galaxy Commit: 9e07d9cc062bf8a21bfab72c2505191e1d3c86e4 https://github.com/galaxyproject/galaxy/commit/9e07d9cc062bf8a21bfab72c2505… Author: mvdbeek <m.vandenbeek(a)gmail.com> Date: 2018-08-16 (Thu, 16 Aug 2018) Changed paths: A test/functional/conftest.py M test/functional/test_data_managers.py M test/functional/test_toolbox.py M test/functional/test_toolbox_pytest.py Log Message: ----------- Handle driver shutdown, show all test in pytest This is pretty hackish as fixtures can't be used during test discovery. To properly shut down the driver I've added a conftest.py file that will be called when quitting pytest. We might be able to handle this a little better if we wrote a full plugin, but for now this works. Also now reports all test indexes, so you can select e.g test 2 of a tool. Commit: 2b275b7df033c323cec93c34a8089a4820fce768 https://github.com/galaxyproject/galaxy/commit/2b275b7df033c323cec93c34a808… Author: mvdbeek <m.vandenbeek(a)gmail.com> Date: 2018-08-16 (Thu, 16 Aug 2018) Changed paths: M test/functional/test_toolbox_pytest.py Log Message: ----------- Start counting test numbers at 1 and use _test_n That allows for nicer select syntax like ```` pytest test/functional/test_toolbox_pytest.py -k '(random_lines1 or Cut1) and _test_1' ``` Commit: e07cc9a96009bafb47c27199cc231213ce2c3514 https://github.com/galaxyproject/galaxy/commit/e07cc9a96009bafb47c27199cc23… Author: John Chilton <jmchilton(a)gmail.com> Date: 2018-08-20 (Mon, 20 Aug 2018) Changed paths: A test/functional/conftest.py M test/functional/test_data_managers.py M test/functional/test_toolbox.py M test/functional/test_toolbox_pytest.py Log Message: ----------- Merge pull request #6608 from mvdbeek/more_pytest_improvements Handle driver shutdown, show all tests in pytest Compare: https://github.com/galaxyproject/galaxy/compare/3c7eeecee52d...e07cc9a96009 **NOTE:** This service has been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/ Functionality will be removed from GitHub.com on January 31st, 2019.

1 0

[galaxyproject/galaxy] 899353: add dcd datatype
by GitHub 20 Aug '18

20 Aug '18

Branch: refs/heads/dev Home: https://github.com/galaxyproject/galaxy Commit: 899353d631b1a3fa0b38730009f9dfc6e97b761f https://github.com/galaxyproject/galaxy/commit/899353d631b1a3fa0b38730009f9… Author: Chris Barnett <chris.barnett(a)uct.ac.za> Date: 2018-08-16 (Thu, 16 Aug 2018) Changed paths: M lib/galaxy/datatypes/binary.py Log Message: ----------- add dcd datatype Commit: c63e4fde41c4c72a45a8d2999833804481c9c9a2 https://github.com/galaxyproject/galaxy/commit/c63e4fde41c4c72a45a8d2999833… Author: Chris Barnett <chris.barnett(a)uct.ac.za> Date: 2018-08-17 (Fri, 17 Aug 2018) Changed paths: M config/datatypes_conf.xml.sample Log Message: ----------- register dcd datatype Commit: 98e4dab4ef48617bd8e03590d19d009c730284ab https://github.com/galaxyproject/galaxy/commit/98e4dab4ef48617bd8e03590d19d… Author: Chris Barnett <chris.barnett(a)uct.ac.za> Date: 2018-08-17 (Fri, 17 Aug 2018) Changed paths: A lib/galaxy/datatypes/test/test_glucose_vacuum.dcd Log Message: ----------- add dcd example file Commit: f8833cc4b34784a033e8d0910ae09eeddab5105c https://github.com/galaxyproject/galaxy/commit/f8833cc4b34784a033e8d0910ae0… Author: Chris Barnett <chris.barnett(a)uct.ac.za> Date: 2018-08-17 (Fri, 17 Aug 2018) Changed paths: M lib/galaxy/datatypes/binary.py Log Message: ----------- PEP corrections Commit: d05e8e0cb10cd75310c59a903eed8f4e0cab7fa2 https://github.com/galaxyproject/galaxy/commit/d05e8e0cb10cd75310c59a903eed… Author: Chris Barnett <chris.barnett(a)uct.ac.za> Date: 2018-08-17 (Fri, 17 Aug 2018) Changed paths: M lib/galaxy/datatypes/binary.py Log Message: ----------- file open fix Commit: 3fc40c16a3fc9cda2284cf9e5cca91373797a52c https://github.com/galaxyproject/galaxy/commit/3fc40c16a3fc9cda2284cf9e5cca… Author: Chris Barnett <chris.barnett(a)uct.ac.za> Date: 2018-08-20 (Mon, 20 Aug 2018) Changed paths: M lib/galaxy/datatypes/binary.py Log Message: ----------- use seek. logic improved. Commit: 33794fa318e49b8eeef2e65ce59a8a5bf361db3a https://github.com/galaxyproject/galaxy/commit/33794fa318e49b8eeef2e65ce59a… Author: Chris Barnett <chris.barnett(a)uct.ac.za> Date: 2018-08-20 (Mon, 20 Aug 2018) Changed paths: M config/datatypes_conf.xml.sample M lib/galaxy/datatypes/binary.py A lib/galaxy/datatypes/test/test_charmm.vel Log Message: ----------- add vel datatype,register and example Commit: f8b797c06092695c5e5431be49d806b94e900780 https://github.com/galaxyproject/galaxy/commit/f8b797c06092695c5e5431be49d8… Author: Chris Barnett <chris.barnett(a)uct.ac.za> Date: 2018-08-20 (Mon, 20 Aug 2018) Changed paths: M lib/galaxy/datatypes/binary.py Log Message: ----------- no edam format yet. soon. Commit: e9846ae9766e29bbbb8b16ff8e5b6b654333e168 https://github.com/galaxyproject/galaxy/commit/e9846ae9766e29bbbb8b16ff8e5b… Author: Chris Barnett <chris.barnett(a)uct.ac.za> Date: 2018-08-20 (Mon, 20 Aug 2018) Changed paths: M lib/galaxy/datatypes/binary.py Log Message: ----------- edam data Commit: 3c7eeecee52d5babf8039fd48ab161555086f84f https://github.com/galaxyproject/galaxy/commit/3c7eeecee52d5babf8039fd48ab1… Author: Björn Grüning <bjoern(a)gruenings.eu> Date: 2018-08-20 (Mon, 20 Aug 2018) Changed paths: M config/datatypes_conf.xml.sample M lib/galaxy/datatypes/binary.py A lib/galaxy/datatypes/test/test_charmm.vel A lib/galaxy/datatypes/test/test_glucose_vacuum.dcd Log Message: ----------- Merge pull request #6613 from chrisbarnettster/cbb_md_dcd add dcd datatype for molecular dynamics Compare: https://github.com/galaxyproject/galaxy/compare/85f347e1c659...3c7eeecee52d **NOTE:** This service has been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/ Functionality will be removed from GitHub.com on January 31st, 2019.

1 0