Branch: refs/heads/dev Home: https://github.com/galaxyproject/galaxy Commit: 55f5235ab8b444a2ef20b03b9a03183cb5266ebb https://github.com/galaxyproject/galaxy/commit/55f5235ab8b444a2ef20b03b9a031... Author: M Bernt m.bernt@ufz.de Date: 2018-11-14 (Wed, 14 Nov 2018)
Changed paths: M lib/galaxy/dependencies/__init__.py M lib/galaxy/jobs/runners/drmaa.py A lib/galaxy/jobs/runners/univa.py M lib/galaxy/jobs/runners/util/job_script/MEMORY_STATEMENT.sh
Log Message: ----------- A new runner for DRMAA (currently UNIVA)
Reimplementation of the DRMAA runner inspired by the SLURM runner. Currently tested only for the UNIVA grid engine (but I'm optimistic that it should work as well for other drmaa systems).
This solves the problem that the current DRMAAJobRunner does not work when jobs are submitted as real user (because jobs that are started in a different drmaa session can not be accessed from the session that is open in galaxy): - this is done by resorting to command line tools qstat and qacct if the drmaa library can not be used to check the job status and to get run time information. - this has the additional advantage that if the drmaa library functions are not working (DRMAAJobRunner had implemented a repeated checking to handle this problem) the runner can still use the command line tools.
Furthermore (in contrast to the original drmaa runner) the new one tests for run time and memory violations: - memory violations are determined by comparing the used and the requested memory - run time violations are determined by checking the signal that killed the job and by comparing the used and the requested run time Where the used memory and time are determined with drmaa.wait() or qacct
Open (or better perspective): - adaptions to other grid engines. the current implementation (the command line calls and result parsing) might be specific for the Univa grid engine. to include other GEs one could determine the GE (+ version) and make the calls and result parsing depending on this.
Implementation note:
The changes in drmaa.py do not change the functionality at all, but only reorganize the code. In particular part of the function `check_watched_items` was put into a new function `check_watched_item` in order to make subclassing more convenient.
Replaces #6931 (which replaced #4275), since I did mess up with git again (there were some duplicated commits).
Commit: 72b2781d4b22cf1c6a312912ffc5333fe5db10b7 https://github.com/galaxyproject/galaxy/commit/72b2781d4b22cf1c6a312912ffc53... Author: John Chilton jmchilton@gmail.com Date: 2018-11-19 (Mon, 19 Nov 2018)
Changed paths: M lib/galaxy/dependencies/__init__.py M lib/galaxy/jobs/runners/drmaa.py A lib/galaxy/jobs/runners/univa.py M lib/galaxy/jobs/runners/util/job_script/MEMORY_STATEMENT.sh
Log Message: ----------- Merge pull request #7004 from bernt-matthias/topic/univa3
A new runner for DRMAA (currently UNIVA)
Compare: https://github.com/galaxyproject/galaxy/compare/f9685459a45f...72b2781d4b22 **NOTE:** This service has been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/
Functionality will be removed from GitHub.com on January 31st, 2019.