Branch: refs/heads/dev Home: https://github.com/galaxyproject/galaxy Commit: 185aa3968bd38cfe021f7bcec9bae02c12ae4c0b https://github.com/galaxyproject/galaxy/commit/185aa3968bd38cfe021f7bcec9bae... Author: John Chilton jmchilton@gmail.com Date: 2017-03-30 (Thu, 30 Mar 2017)
Changed paths: M lib/galaxy/workflow/modules.py M lib/galaxy/workflow/run.py M test/api/test_workflows.py
Log Message: ----------- Log when workflow scheduling is delayed about why.
Currently the logs contain fairly uninformative step delayed messages that don't allow admins to understand why parts of the workflow are being delayed during scheduling.
Here are some examples of before and after.
When running the test:
``` ./run_tests.sh -api test/api/test_workflows.py:WorkflowsApiTestCase.test_workflow_pause ```
Before these lines would show up:
``` galaxy.workflow.run DEBUG 2017-03-30 09:23:47,327 Workflow step 2 of invocation 1 invoked (174.149 ms) galaxy.workflow.run DEBUG 2017-03-30 09:23:47,347 Workflow step 3 of invocation 1 invoked (19.355 ms) galaxy.workflow.run DEBUG 2017-03-30 09:23:47,364 Workflow step 4 of invocation 1 delayed (16.262 ms) ```
Now these same lines are as follows:
``` galaxy.workflow.run DEBUG 2017-03-30 09:19:28,601 Workflow step 2 of invocation 1 invoked (172.849 ms) galaxy.workflow.run DEBUG 2017-03-30 09:19:28,619 Marking step 3 outputs delayed (executing pause step) galaxy.workflow.run DEBUG 2017-03-30 09:19:28,620 Workflow step 3 of invocation 1 invoked (17.999 ms) galaxy.workflow.run DEBUG 2017-03-30 09:19:28,633 Workflow step 4 of invocation 1 delayed (dependent step [3] delayed, so this step must be delayed) (13.398 ms) ```
Also, when running the test:
``` ./run_tests.sh -api test/api/test_workflows.py:WorkflowsApiTestCase.test_workflow_run_dynamic_output_collections_3 ```
Before these lines would be printed:
``` galaxy.workflow.run DEBUG 2017-03-30 09:25:35,910 Workflow step 4 of invocation 1 invoked (281.479 ms) galaxy.workflow.run DEBUG 2017-03-30 09:25:35,937 Workflow step 5 of invocation 1 delayed (25.904 ms) galaxy.workflow.run DEBUG 2017-03-30 09:25:35,970 Workflow step 6 of invocation 1 delayed (32.597 ms) ```
Now these same lines are as follows:
``` galaxy.workflow.run DEBUG 2017-03-30 09:27:54,270 Workflow step 4 of invocation 1 invoked (295.826 ms) galaxy.workflow.run DEBUG 2017-03-30 09:27:54,315 Workflow step 5 of invocation 1 delayed (dependent collection [1] not yet populated with datasets) (44.581 ms) galaxy.workflow.run DEBUG 2017-03-30 09:27:54,326 Workflow step 6 of invocation 1 delayed (dependent step [5] delayed, so this step must be delayed) (10.804 ms) ```
When running workflows with steps that depend explicitly on other steps (instead of implicitly between dataset connections), lines such as:
``` galaxy.workflow.run DEBUG 2017-03-30 09:33:52,966 Marking step 3 outputs delayed (workflow paused at this step waiting for review) galaxy.workflow.run DEBUG 2017-03-30 09:33:53,001 Workflow step 4 of invocation 1 delayed (dependent step [3] delayed, so this step must be delayed) (0.157 ms) galaxy.workflow.run DEBUG 2017-03-30 09:33:53,001 Workflow step 5 of invocation 1 delayed (depends on step [4] but that step has not been invoked yet) (0.084 ms) ```
and
``` galaxy.workflow.run DEBUG 2017-03-30 09:33:57,090 Workflow step 5 of invocation 1 delayed (depends on step [4] but one or more jobs created from that step have not finished yet) (0.144 ms) ```
now may appear.
Commit: b8692309484c3a671145684277201ed7aaf444dd https://github.com/galaxyproject/galaxy/commit/b8692309484c3a671145684277201... Author: John Chilton jmchilton@gmail.com Date: 2017-03-30 (Thu, 30 Mar 2017)
Changed paths: M lib/galaxy/workflow/run.py
Log Message: ----------- Re-arrange workflow scheduling logic according to @nsoranzo's suggestions.
Commit: 200e475bc7cd340401f46028fc234a1e7c2ad87c https://github.com/galaxyproject/galaxy/commit/200e475bc7cd340401f46028fc234... Author: Dannon Baker dannon.baker@gmail.com Date: 2017-03-30 (Thu, 30 Mar 2017)
Changed paths: M lib/galaxy/workflow/modules.py M lib/galaxy/workflow/run.py M test/api/test_workflows.py
Log Message: ----------- Merge pull request #3849 from jmchilton/improved_scheduling_delay_logging
Log reason when workflow scheduling of a step is delayed.
Compare: https://github.com/galaxyproject/galaxy/compare/90ac2f435662...200e475bc7cd