Branch: refs/heads/dev Home: https://github.com/galaxyproject/galaxy Commit: f3898c2136f494a52dc1fec01af379a735ba1842 https://github.com/galaxyproject/galaxy/commit/f3898c2136f494a52dc1fec01af37... Author: M Bernt m.bernt@ufz.de Date: 2019-03-04 (Mon, 04 Mar 2019)

Changed paths: M lib/galaxy/jobs/output_checker.py

Log Message: ----------- - check regexp also if FATAL was detected by exit_code - introduced functions for checking regex against stream - check regex line by line instead of on the complete stream (the original behavior is coded in check_output_regex which could be removed if linewise is OK)

Commit: b458ff54e3585914d3525c4b1f4bcf5ff0dc9f7f https://github.com/galaxyproject/galaxy/commit/b458ff54e3585914d3525c4b1f4bc... Author: M Bernt m.bernt@ufz.de Date: 2019-03-04 (Mon, 04 Mar 2019)

Changed paths: M lib/galaxy/jobs/output_checker.py

Log Message: ----------- also check all regexes

Commit: 39b8f7872076508adf35a0b348e4c31a002a9b81 https://github.com/galaxyproject/galaxy/commit/39b8f7872076508adf35a0b348e4c... Author: M Bernt m.bernt@ufz.de Date: 2019-03-04 (Mon, 04 Mar 2019)

Changed paths: M lib/galaxy/jobs/output_checker.py A test/functional/tools/detect_errors.xml

Log Message: ----------- added test and changed modification of stdout/err

before stdout and stderr were modified while testing for exit codes and regexes. this had the potential problem that later tests also checked this output which might lead to confusions.

furthermore a marker line has been introduces which differentiates Galaxy and original tool output

Commit: 959af578d07d1eb1372cd4b22055d57b0912a8ca https://github.com/galaxyproject/galaxy/commit/959af578d07d1eb1372cd4b22055d... Author: M Bernt m.bernt@ufz.de Date: 2019-03-04 (Mon, 04 Mar 2019)

Changed paths: M lib/galaxy/jobs/output_checker.py M test/functional/tools/detect_errors.xml

Log Message: ----------- fixed linting and copy paste errors

Commit: b47a8631bbc4fcc9ebbd8b8a62032246effd6f5c https://github.com/galaxyproject/galaxy/commit/b47a8631bbc4fcc9ebbd8b8a62032... Author: John Chilton jmchilton@gmail.com Date: 2019-03-04 (Mon, 04 Mar 2019)

Changed paths: M lib/galaxy/jobs/output_checker.py M lib/galaxy/model/__init__.py M lib/galaxy/model/mapping.py A lib/galaxy/model/migrate/versions/0147_job_messages.py M lib/galaxy/tools/linters/tests.py M lib/galaxy/tools/verify/interactor.py M lib/galaxy/webapps/galaxy/api/jobs.py M templates/show_params.mako M test/api/test_jobs.py M test/functional/tools/samples_tool_conf.xml M test/unit/tools/test_parsing.py

Log Message: ----------- Track job failure reasons in a more structured way.

Commit: d6ccc595847a940fe1149c52f27d2786f825dea3 https://github.com/galaxyproject/galaxy/commit/d6ccc595847a940fe1149c52f27d2... Author: John Chilton jmchilton@gmail.com Date: 2019-03-04 (Mon, 04 Mar 2019)

Changed paths: M lib/galaxy/jobs/__init__.py M lib/galaxy/jobs/output_checker.py M test/unit/jobs/test_job_output_checker.py

Log Message: ----------- Move model/Tool interactions out of galaxy.jobs.output_checker.

This means output_checker can be placed into galaxy-lib without dependency problems, and we can evaluate if a tool passed without access to a job object or a Tool object as long as we have a description of the job outputs and stdio objects (already in galaxy-lib).

This is a prelude to being able to do job output checking in Pulsar without Galaxy internals.

Commit: a910ce08ca1dda188526baae9e8c4a0f631b7f1f https://github.com/galaxyproject/galaxy/commit/a910ce08ca1dda188526baae9e8c4... Author: John Chilton jmchilton@gmail.com Date: 2019-03-04 (Mon, 04 Mar 2019)

Changed paths: M lib/galaxy/jobs/output_checker.py

Log Message: ----------- Simplify, clear up output_checker code a bit.

Based on suggestions from mvdbeek.

Commit: 04f2e87b38bb9a8eec077c11747bde7446d001f6 https://github.com/galaxyproject/galaxy/commit/04f2e87b38bb9a8eec077c11747bd... Author: John Chilton jmchilton@gmail.com Date: 2019-03-04 (Mon, 04 Mar 2019)

Changed paths: M lib/galaxy/jobs/output_checker.py

Log Message: ----------- Remove unused code spotted by mvdbeek.

Commit: 8f3bb497e7d4453104546743eccb18c3b24c1a53 https://github.com/galaxyproject/galaxy/commit/8f3bb497e7d4453104546743eccb1... Author: John Chilton jmchilton@gmail.com Date: 2019-03-04 (Mon, 04 Mar 2019)

Changed paths: M lib/galaxy/jobs/runners/__init__.py

Log Message: ----------- Don't determine job failure from stderr, that makes no sense anymore.

Commit: 06f66524563f0dab134aaef950f9be16cc156394 https://github.com/galaxyproject/galaxy/commit/06f66524563f0dab134aaef950f9b... Author: John Chilton jmchilton@gmail.com Date: 2019-03-04 (Mon, 04 Mar 2019)

Changed paths: M lib/galaxy/jobs/runners/__init__.py M lib/galaxy/jobs/runners/local.py

Log Message: ----------- More consistent exit_code handling across runners.

Commit: cc7cde8ce09113690e05587e3bcd72e172956729 https://github.com/galaxyproject/galaxy/commit/cc7cde8ce09113690e05587e3bcd7... Author: John Chilton jmchilton@gmail.com Date: 2019-03-04 (Mon, 04 Mar 2019)

Changed paths: M lib/galaxy/jobs/runners/__init__.py M lib/galaxy/jobs/runners/local.py

Log Message: ----------- More consistent job finish/resubmit handling across local and async runners.

Commit: 7e5f3d83e8cfd80ecb9a301a5f72050af15bc319 https://github.com/galaxyproject/galaxy/commit/7e5f3d83e8cfd80ecb9a301a5f720... Author: John Chilton jmchilton@gmail.com Date: 2019-03-04 (Mon, 04 Mar 2019)

Changed paths: M lib/galaxy/jobs/runners/__init__.py M lib/galaxy/jobs/runners/local.py

Log Message: ----------- De-duplicate constants and parameters for shrink_stream in jobs code.

Commit: 0fe13b7e5af60b1bf7417a1a5ff981dd21cb026f https://github.com/galaxyproject/galaxy/commit/0fe13b7e5af60b1bf7417a1a5ff98... Author: John Chilton jmchilton@gmail.com Date: 2019-03-04 (Mon, 04 Mar 2019)

Changed paths: M lib/galaxy/jobs/__init__.py M lib/galaxy/jobs/command_factory.py M lib/galaxy/jobs/handler.py M lib/galaxy/jobs/runners/__init__.py M lib/galaxy/jobs/runners/pulsar.py M lib/galaxy/model/__init__.py M lib/galaxy/model/mapping.py M lib/galaxy/model/migrate/versions/0147_job_messages.py M lib/galaxy/tools/imp_exp/__init__.py M lib/galaxy/tools/verify/interactor.py M lib/galaxy/webapps/galaxy/api/jobs.py M lib/galaxy/webapps/galaxy/controllers/admin.py M test/functional/tools/job_environment_default.xml A test/integration/io_injection_job_conf.xml M test/integration/test_job_environments.py M test/unit/jobs/test_command_factory.py M test/unit/tools/test_history_imp_exp.py

Log Message: ----------- Seperate tool and job script standard I/O.

This should allow Galaxy to reason about what the tool reports vs. what dependency resolution and metadata reports, allow us to enable logging during metadata collection, and allow us to reason about the tool's I/O from inside the job script during cleanup/metadata collection at the end.

Commit: 381406e9ec13c386fe6041585ebfab7664529917 https://github.com/galaxyproject/galaxy/commit/381406e9ec13c386fe6041585ebfa... Author: Marius van den Beek m.vandenbeek@gmail.com Date: 2019-03-05 (Tue, 05 Mar 2019)

Changed paths: M lib/galaxy/jobs/__init__.py M lib/galaxy/jobs/command_factory.py M lib/galaxy/jobs/handler.py M lib/galaxy/jobs/output_checker.py M lib/galaxy/jobs/runners/__init__.py M lib/galaxy/jobs/runners/local.py M lib/galaxy/jobs/runners/pulsar.py M lib/galaxy/model/__init__.py M lib/galaxy/model/mapping.py A lib/galaxy/model/migrate/versions/0147_job_messages.py M lib/galaxy/tools/imp_exp/__init__.py M lib/galaxy/tools/linters/tests.py M lib/galaxy/tools/verify/interactor.py M lib/galaxy/webapps/galaxy/api/jobs.py M lib/galaxy/webapps/galaxy/controllers/admin.py M templates/show_params.mako M test/api/test_jobs.py A test/functional/tools/detect_errors.xml M test/functional/tools/job_environment_default.xml M test/functional/tools/samples_tool_conf.xml A test/integration/io_injection_job_conf.xml M test/integration/test_job_environments.py M test/unit/jobs/test_command_factory.py M test/unit/jobs/test_job_output_checker.py M test/unit/tools/test_history_imp_exp.py M test/unit/tools/test_parsing.py

Log Message: ----------- Merge pull request #7095 from jmchilton/structured_job_outputs

More Structured Job I/O Tracking

Compare: https://github.com/galaxyproject/galaxy/compare/503e9fd58469...381406e9ec13