Branch: refs/heads/dev Home: https://github.com/galaxyproject/galaxy Commit: c5a08186b7ff8090b5ec7191ed149c3e13c9aa90 https://github.com/galaxyproject/galaxy/commit/c5a08186b7ff8090b5ec7191ed149... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2017-09-12 (Tue, 12 Sep 2017) Changed paths: M lib/galaxy/jobs/runners/cli.py Log Message: ----------- Retry job submissions in ShellJobRunner If I submit a large amount of jobs in a very short time I sometimes see ``` galaxy.jobs.runners.cli ERROR 2017-09-11 05:03:02,404 (17633) submission failed (stderr): qsub: submit error (Invalid credential) ``` in my logs for a minor subset of jobs (2 out of 120 in the last instance that this happened). In this case waiting a little while and trying again solves the problem. By default we will try submitting the job 3 times, and sleep 10 seconds before trying again. Commit: ecd8b90e44decbef60db4359bf4837ef5b12ed1f https://github.com/galaxyproject/galaxy/commit/ecd8b90e44decbef60db4359bf483... Author: mvdbeek <m.vandenbeek@gmail.com> Date: 2017-09-13 (Wed, 13 Sep 2017) Changed paths: M lib/galaxy/jobs/runners/cli.py Log Message: ----------- Decrease log-level for resubmission msg and specifiy log level directly in if/else clause. Thx @erasche. Commit: 11d4380d2b40b4e18806f6b2d1e07f4a6eec1f8b https://github.com/galaxyproject/galaxy/commit/11d4380d2b40b4e18806f6b2d1e07... Author: Eric Rasche <hxr@hx42.org> Date: 2017-09-13 (Wed, 13 Sep 2017) Changed paths: M lib/galaxy/jobs/runners/cli.py Log Message: ----------- Merge pull request #4599 from mvdbeek/make_qsub_submissions_more_robust Retry job submissions in ShellJobRunner Compare: https://github.com/galaxyproject/galaxy/compare/e33b35def345...11d4380d2b40