Branch: refs/heads/dev Home: https://github.com/galaxyproject/galaxy Commit: 7882a067c8cbcf564aff605b91beea8d7849fb00 https://github.com/galaxyproject/galaxy/commit/7882a067c8cbcf564aff605b91bee... Author: John Chilton jmchilton@gmail.com Date: 2018-08-06 (Mon, 06 Aug 2018)

Changed paths: M lib/galaxy/jobs/__init__.py M lib/galaxy/objectstore/__init__.py A test/integration/objectstore_selection_job_conf.xml A test/integration/objectstore_selection_job_resource_parameters_conf.xml A test/integration/objectstore_selection_rules/__init__.py A test/integration/objectstore_selection_rules/rules.py A test/integration/test_objectstore_selection.py

Log Message: ----------- Allow configuring object stores on a per-job-destination basis.

Setup a hierarchical objectstore with ids - these ids can be zero weight for objectstores that should only be used on a per-job-destination basis. Job destinations can then set object_store_id as a param to force a particular object store from the hierarchical configuration to be used. This can be set in job_conf.xml directly for static mappings or combined with dynamic job destinations for dynamic mapping to object stores. Dynamic job destinations can dispatch on cluster conditions, user, user preferences, etc.. Additionally, job resource parameters can be used with dynamic job destinations to allow user input on the object store to use.

An integration test case is included that tests and demonstrates all these different pieces. It sets up a Galaxy with a job and object store configuration that routes one tool to a specific disk store (called "static"), and routes another tool dynamically using job resource parameters that allow the user to pick between "slow, cheap" and "fast, expensive" storage options (for the test it is all just disk stores used - but you can imagine choosing between EBS and S3 or something this way).

I'd be happy to see something more formal than job resource parameters or user preferences developed to help users route job outputs to particular object stores - this is just a demonstration of what can be done today after these changes using extension points already being used by admins and users.

Commit: 498e7d576bb40e7f2eea47b1d787638a2c507965 https://github.com/galaxyproject/galaxy/commit/498e7d576bb40e7f2eea47b1d7876... Author: Dannon dannon.baker@gmail.com Date: 2018-08-22 (Wed, 22 Aug 2018)

Changed paths: M lib/galaxy/jobs/__init__.py M lib/galaxy/objectstore/__init__.py A test/integration/objectstore_selection_job_conf.xml A test/integration/objectstore_selection_job_resource_parameters_conf.xml A test/integration/objectstore_selection_rules/__init__.py A test/integration/objectstore_selection_rules/rules.py A test/integration/test_objectstore_selection.py

Log Message: ----------- Merge pull request #6552 from jmchilton/per_destination_object_store

Allow configuring object stores on a per-job-destination basis.

Compare: https://github.com/galaxyproject/galaxy/compare/53b8f08752ed...498e7d576bb4 **NOTE:** This service has been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/

Functionality will be removed from GitHub.com on January 31st, 2019.