Branch: refs/heads/dev Home: https://github.com/galaxyproject/galaxy Commit: 67efd072ae98d6a34cbbc7f2270743524126bf8a https://github.com/galaxyproject/galaxy/commit/67efd072ae98d6a34cbbc7f227074... Author: John Chilton jmchilton@gmail.com Date: 2018-11-15 (Thu, 15 Nov 2018)

Changed paths: M lib/galaxy/dependencies/pipfiles/default/pinned-requirements.txt M lib/galaxy/managers/workflows.py M lib/galaxy/model/__init__.py M lib/galaxy/model/mapping.py M lib/galaxy/model/migrate/versions/0136_collection_and_workflow_state.py A lib/galaxy/model/migrate/versions/0145_add_workflow_step_input.py M lib/galaxy/webapps/tool_shed/model/__init__.py M lib/galaxy/workflow/extract.py M lib/galaxy/workflow/modules.py M lib/tool_shed/util/workflow_util.py M test/api/test_workflows.py M test/unit/test_galaxy_mapping.py M test/unit/workflows/test_modules.py M test/unit/workflows/test_render.py M test/unit/workflows/test_workflow_progress.py M test/unit/workflows/workflow_support.py

Log Message: ----------- Track workflow step input definitions in our model.

We don't track workflow step inputs in any formal way in our model currently. This has resulted in some current hacks and prevents future enhancements. This commit splits WorkflowStepConnection into two models WorkflowStepInput and WorkflowStepConnection - normalizing the previous table workflow_step_connection on input step and input name.

In terms of current hacks forced on it by restricting all of tool state to be confined to a big JSON blob in the database - we have problems distinguishing keys and values when walking tool state. As we store more and more JSON blobs inside of the giant tool state blob - the worse this problem gets. Take for instance checking for runtime parameters or the rules parameter values - these both use JSON blobs that aren't simple values, so it is hard to tell looking at the tool state blob in the database or the workflow export to tell what is a key or what is a value. Tracking state as normalized inputs with default values and explicit attributes runtime values should allow much more percise state definition and construction.

This variant of the models would also potentially allow defining runtime values with non-tool default values (so default values defined for the workflow but still explicitly settable at runtime). The combinations of overriding defaults and defining runtime values were not representable before.

In terms of future enhancements, there is a lot we cannot track with the current models - such as map/reduce options for collection operations (https://github.com/galaxyproject/galaxy/issues/4623#issuecomment-389544980). This should enable a lot of that. Obviously there are a lot of attributes defined here that are not yet utilized, but I'm using most (all?) of them downstream in the CWL branch. I'd rather populate this table fully realized and fill in the implementation around it as work continues to stream in from the CWL branch - to keep things simple and avoid extra database migrations. But I understand if this feels like speculative complexity we want to avoid despite the implementation being readily available for inspection downstream.

Commit: 15c43e777b7c66614fd9d157ab2fcabb54d7f759 https://github.com/galaxyproject/galaxy/commit/15c43e777b7c66614fd9d157ab2fc... Author: Nicola Soranzo nicola.soranzo@gmail.com Date: 2018-11-16 (Fri, 16 Nov 2018)

Changed paths: M lib/galaxy/model/mapping.py

Log Message: ----------- Update lib/galaxy/model/mapping.py

Co-Authored-By: jmchilton jmchilton@gmail.com

Commit: 0a18ed62977b8a81932841361f7fa126f8dca2dc https://github.com/galaxyproject/galaxy/commit/0a18ed62977b8a81932841361f7fa... Author: Nicola Soranzo nicola.soranzo@gmail.com Date: 2018-11-16 (Fri, 16 Nov 2018)

Changed paths: M lib/galaxy/model/migrate/versions/0145_add_workflow_step_input.py

Log Message: ----------- Update lib/galaxy/model/migrate/versions/0145_add_workflow_step_input.py

Co-Authored-By: jmchilton jmchilton@gmail.com

Commit: 00af4e21dee168fd6c54ac925414bcb95b8e7930 https://github.com/galaxyproject/galaxy/commit/00af4e21dee168fd6c54ac925414b... Author: Nicola Soranzo nicola.soranzo@gmail.com Date: 2018-11-16 (Fri, 16 Nov 2018)

Changed paths: M lib/galaxy/model/migrate/versions/0145_add_workflow_step_input.py

Log Message: ----------- Update lib/galaxy/model/migrate/versions/0145_add_workflow_step_input.py

Co-Authored-By: jmchilton jmchilton@gmail.com

Commit: f2c17e589b734afedcb807d038d0f1b8caf27f52 https://github.com/galaxyproject/galaxy/commit/f2c17e589b734afedcb807d038d0f... Author: Nicola Soranzo nicola.soranzo@earlham.ac.uk Date: 2018-11-16 (Fri, 16 Nov 2018)

Changed paths: M lib/galaxy/model/mapping.py M lib/galaxy/model/migrate/versions/0145_add_workflow_step_input.py

Log Message: ----------- Preserve updated content in migration 145 downgrade

Also add UNIQUE constraint to `("workflow_step_id", "name")` to the new `workflow_step_input` table.

Commit: f9685459a45f2e2b3395077cbfa11e9a45245021 https://github.com/galaxyproject/galaxy/commit/f9685459a45f2e2b3395077cbfa11... Author: Nicola Soranzo nicola.soranzo@gmail.com Date: 2018-11-16 (Fri, 16 Nov 2018)

Changed paths: M lib/galaxy/dependencies/pipfiles/default/pinned-requirements.txt M lib/galaxy/managers/workflows.py M lib/galaxy/model/__init__.py M lib/galaxy/model/mapping.py M lib/galaxy/model/migrate/versions/0136_collection_and_workflow_state.py A lib/galaxy/model/migrate/versions/0145_add_workflow_step_input.py M lib/galaxy/webapps/tool_shed/model/__init__.py M lib/galaxy/workflow/extract.py M lib/galaxy/workflow/modules.py M lib/tool_shed/util/workflow_util.py M test/api/test_workflows.py M test/unit/test_galaxy_mapping.py M test/unit/workflows/test_modules.py M test/unit/workflows/test_render.py M test/unit/workflows/test_workflow_progress.py M test/unit/workflows/workflow_support.py

Log Message: ----------- Merge pull request #6850 from jmchilton/workflows_track_step_inputs

Track workflow step input definitions in our model.

Compare: https://github.com/galaxyproject/galaxy/compare/7695569b50b7...f9685459a45f **NOTE:** This service has been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/

Functionality will be removed from GitHub.com on January 31st, 2019.