Branch: refs/heads/dev
Home: https://github.com/galaxyproject/galaxy
Commit: eca238d7837928c7ee962f10435fd667d61ccc4c
https://github.com/galaxyproject/galaxy/commit/eca238d7837928c7ee962f10435f…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M config/job_conf.xml.sample_advanced
M lib/galaxy/dependencies/__init__.py
M lib/galaxy/dependencies/conditional-requirements.txt
Log Message:
-----------
Make pykube a conditional requirement
Commit: c82cd04ca820d65c9279872c029d712358e4c65f
https://github.com/galaxyproject/galaxy/commit/c82cd04ca820d65c9279872c029d…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M lib/galaxy/jobs/runners/kubernetes.py
Log Message:
-----------
Use standard os.environ
Commit: 879d7c22b63733dfd5f9e1c6cbe368c97e06495d
https://github.com/galaxyproject/galaxy/commit/879d7c22b63733dfd5f9e1c6cbe3…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M test/base/integration_util.py
A test/integration/test_kubernetes_runner.py
Log Message:
-----------
Add basic kubernetes integration test
Currently times out becasue jobs aren't monitored succesfully.
Commit: 4eed32dc4c86da17a525419ba985d4d06d7a768b
https://github.com/galaxyproject/galaxy/commit/4eed32dc4c86da17a525419ba985…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M lib/galaxy/jobs/runners/cli.py
M lib/galaxy/jobs/runners/condor.py
M lib/galaxy/jobs/runners/drmaa.py
M lib/galaxy/jobs/runners/local.py
M lib/galaxy/jobs/runners/pbs.py
Log Message:
-----------
Propagate selected shell to job_script
Commit: 912fc61592a92fbed329501b38b35a591e556b2d
https://github.com/galaxyproject/galaxy/commit/912fc61592a92fbed329501b38b3…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M lib/galaxy/jobs/runners/kubernetes.py
Log Message:
-----------
Allow skipping pod_retrial
Commit: e7b89122489083f80b0c183ff0a958a58d823118
https://github.com/galaxyproject/galaxy/commit/e7b89122489083f80b0c183ff0a9…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M lib/galaxy/jobs/runners/kubernetes.py
Log Message:
-----------
Use galaxy's exisiting job_file abstraction
This allows setting environment variables, job instrumentation,
exit code checking and so on.
Commit: 3498f87d8734ce5bc3341e546842dabe29b93d40
https://github.com/galaxyproject/galaxy/commit/3498f87d8734ce5bc3341e546842…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M test/integration/test_kubernetes_runner.py
Log Message:
-----------
Use sh as job shell for kubernetes test, bash is not available in default container
Commit: 5dd10c4f86b07d9483b76c59511148bdce8949d3
https://github.com/galaxyproject/galaxy/commit/5dd10c4f86b07d9483b76c595111…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M lib/galaxy/tools/deps/dependencies.py
Log Message:
-----------
Drop mutable defaults
Commit: 1a007682737af1b466a6a8545922e17c7f42fb19
https://github.com/galaxyproject/galaxy/commit/1a007682737af1b466a6a8545922…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M lib/galaxy/jobs/runners/kubernetes.py
Log Message:
-----------
Skip broken string encoding logic
Commit: d2ed543ecd39fee1876fcbad063c1aa4c4c3ce33
https://github.com/galaxyproject/galaxy/commit/d2ed543ecd39fee1876fcbad063c…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M lib/galaxy/jobs/runners/chronos.py
M lib/galaxy/jobs/runners/godocker.py
M lib/galaxy/jobs/runners/kubernetes.py
Log Message:
-----------
Remove return statements from queue_job, this is never consumed anywhere
Commit: 1103ce1dccd0ac3dda0dca715da4cb0feafde4ee
https://github.com/galaxyproject/galaxy/commit/1103ce1dccd0ac3dda0dca715da4…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M lib/galaxy/jobs/runners/kubernetes.py
Log Message:
-----------
Remove commented out non-functional code
in kubernetes runner.
Commit: a23063b2dc3fdf82c645383fd36c3b6439b68f52
https://github.com/galaxyproject/galaxy/commit/a23063b2dc3fdf82c645383fd36c…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M lib/galaxy/jobs/runners/kubernetes.py
Log Message:
-----------
Cleanup of kubernetes exception logging
Commit: 4e39c0fc8488a4794870752f8895f61dc8d704cf
https://github.com/galaxyproject/galaxy/commit/4e39c0fc8488a4794870752f8895…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M lib/galaxy/jobs/runners/kubernetes.py
M test/integration/test_containerized_jobs.py
M test/integration/test_job_recovery.py
M test/integration/test_kubernetes_runner.py
Log Message:
-----------
Test cancelling jobs, running mulled containers on kubernetes
Commit: 9019916ae2180afe6f35b6ead45f1ae294c2bfe9
https://github.com/galaxyproject/galaxy/commit/9019916ae2180afe6f35b6ead45f…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M test/integration/test_kubernetes_runner.py
Log Message:
-----------
Fix test skipping
Commit: 34c0a2940b69c59b20bbdd8df0944332cb59e64c
https://github.com/galaxyproject/galaxy/commit/34c0a2940b69c59b20bbdd8df094…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M lib/galaxy/jobs/runners/kubernetes.py
Log Message:
-----------
Don't write fake stderr file
Commit: 5c5f199f812ff68e7d061c047956aeaa3627632f
https://github.com/galaxyproject/galaxy/commit/5c5f199f812ff68e7d061c047956…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M test/integration/test_kubernetes_runner.py
Log Message:
-----------
Add job properties test
Commit: b43f1985d71fadf3f907feeab2b529408dc05588
https://github.com/galaxyproject/galaxy/commit/b43f1985d71fadf3f907feeab2b5…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M lib/galaxy/jobs/command_factory.py
M lib/galaxy/jobs/runners/__init__.py
M lib/galaxy/jobs/runners/kubernetes.py
Log Message:
-----------
Capture stdout and stderr for kubernetes runner
There is currently no way to discriminate stdout and stderr
from kubernetes logs (https://github.com/kubernetes/kubernetes/issues/28167),
but Galaxy really needs to know about the stderr.
To work around this we capture the stdout and stderr into files using
the approach outined in https://stackoverflow.com/a/692407.
Commit: d0e749375a2e703f9f59e8794825867760ee7b44
https://github.com/galaxyproject/galaxy/commit/d0e749375a2e703f9f59e8794825…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M lib/galaxy/jobs/runners/kubernetes.py
Log Message:
-----------
Drop custom fail_job and stderr logging methods
We now log stdout and stderr to files, so this isn't needed.
Commit: 98b571e19aec1b182dbb636d23ab8a3ff290f30d
https://github.com/galaxyproject/galaxy/commit/98b571e19aec1b182dbb636d23ab…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M lib/galaxy/jobs/runners/kubernetes.py
Log Message:
-----------
Remove user-cancelled jobs from monitor queue
Commit: dc184567376ec25eba3d317c9a70d2a27d4aefed
https://github.com/galaxyproject/galaxy/commit/dc184567376ec25eba3d317c9a70…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
A lib/galaxy/util/bytesize.py
Log Message:
-----------
Add utility class for manipulating byte sizes
Commit: c61c790fccc84f06991d48dcbea70ef8181fe452
https://github.com/galaxyproject/galaxy/commit/c61c790fccc84f06991d48dcbea7…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M lib/galaxy/jobs/runners/kubernetes.py
M lib/galaxy/jobs/runners/util/job_script/CLUSTER_SLOTS_STATEMENT.sh
M test/functional/tools/samples_tool_conf.xml
M test/integration/test_kubernetes_runner.py
Log Message:
-----------
Make kubernetes pass GALAXY_SLOTS and GALAXY_MEMORY_MB
Also drops __transform_cpu_value, incoming values are always
strings, so this wasn't functional in any case.
Commit: fe02dbbf8107b8ded12cb9c76fd1686515a47dd4
https://github.com/galaxyproject/galaxy/commit/fe02dbbf8107b8ded12cb9c76fd1…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M config/job_conf.xml.sample_advanced
Log Message:
-----------
Fix typo in config/job_conf.xml.sample_advanced
Commit: 54c964011a29eca1c72360f17f5114022ad626bf
https://github.com/galaxyproject/galaxy/commit/54c964011a29eca1c72360f17f51…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M lib/galaxy/jobs/runners/kubernetes.py
M test/integration/test_kubernetes_runner.py
Log Message:
-----------
Take ceiling of kubernetes cpu value
Commit: 936679ea817151438ff76d2e14ac2557c4caa3a0
https://github.com/galaxyproject/galaxy/commit/936679ea817151438ff76d2e14ac…
Author: Pablo Moreno <pablo.a.moreno(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M lib/galaxy/jobs/runners/kubernetes.py
Log Message:
-----------
Inject env vars for k8s job to avoid changing tool environment inside container.
Commit: e52bb5daa78eb040304919299dc624491573c53f
https://github.com/galaxyproject/galaxy/commit/e52bb5daa78eb040304919299dc6…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M test/integration/test_kubernetes_runner.py
Log Message:
-----------
Import from correct module
Commit: 626657e3c2b8dc3325a4c2dee35777ac45ac8f0c
https://github.com/galaxyproject/galaxy/commit/626657e3c2b8dc3325a4c2dee357…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M lib/galaxy/jobs/command_factory.py
M lib/galaxy/jobs/runners/util/job_script/DEFAULT_JOB_FILE_TEMPLATE.sh
M lib/galaxy/jobs/runners/util/job_script/__init__.py
M test/unit/jobs/test_command_factory.py
Log Message:
-----------
Revert "job script returns tool exit code in any case"
This breaks exit code reporting and leads to pod re-scheduling.
This reverts commit 6beeec836f6c6ab479700e6a1e1accbe5069dacf.
Commit: 7cf161ba0155abf02582cbbf4636d1ca8681efbe
https://github.com/galaxyproject/galaxy/commit/7cf161ba0155abf02582cbbf4636…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M config/job_conf.xml.sample_advanced
M lib/galaxy/jobs/runners/kubernetes.py
M test/integration/test_kubernetes_runner.py
Log Message:
-----------
Allow adding multiple volume claims
Commit: e293a74e66002a28adc1045fda0e3415c6f2e08b
https://github.com/galaxyproject/galaxy/commit/e293a74e66002a28adc1045fda0e…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M test/integration/test_kubernetes_runner.py
Log Message:
-----------
Add test for tool using python dependencies
Commit: 32cad5977b102c33088111abe80887ab1a567a19
https://github.com/galaxyproject/galaxy/commit/32cad5977b102c33088111abe808…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M lib/galaxy/jobs/runners/kubernetes.py
Log Message:
-----------
Set galaxy_virtual_env to None for kubernetes runner
This prevents attempts of activating Galaxy's virtualenv
inside the container.
Commit: b931576ed1ebcdfc0f9e0d5cbab74d199e8bbb2a
https://github.com/galaxyproject/galaxy/commit/b931576ed1ebcdfc0f9e0d5cbab7…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M lib/galaxy/jobs/runners/kubernetes.py
Log Message:
-----------
Drop GALAXY_VIRTUAL_ENV env hack
GALAXY_VIRTUAL_ENV is injected through job_wrapper.galaxy_virtual_env,
which does `os.environ.get(VIRTUAL_ENV, None)`, so this didn't work.
The previous commit addresses that.
Commit: db387ab898cf47076279ac9d97407ba0fdc45e49
https://github.com/galaxyproject/galaxy/commit/db387ab898cf47076279ac9d9740…
Author: mvdbeek <m.vandenbeek(a)gmail.com>
Date: 2019-02-19 (Tue, 19 Feb 2019)
Changed paths:
M test/unit/jobs/test_command_factory.py
Log Message:
-----------
Fix typo fixed by reverted commit 6beeec8
Commit: c53fb623d8cf3af99732a3fe9d1f513f9b538c2b
https://github.com/galaxyproject/galaxy/commit/c53fb623d8cf3af99732a3fe9d1f…
Author: John Chilton <jmchilton(a)gmail.com>
Date: 2019-02-22 (Fri, 22 Feb 2019)
Changed paths:
M config/job_conf.xml.sample_advanced
M lib/galaxy/dependencies/__init__.py
M lib/galaxy/dependencies/conditional-requirements.txt
M lib/galaxy/jobs/command_factory.py
M lib/galaxy/jobs/runners/__init__.py
M lib/galaxy/jobs/runners/chronos.py
M lib/galaxy/jobs/runners/cli.py
M lib/galaxy/jobs/runners/condor.py
M lib/galaxy/jobs/runners/drmaa.py
M lib/galaxy/jobs/runners/godocker.py
M lib/galaxy/jobs/runners/kubernetes.py
M lib/galaxy/jobs/runners/local.py
M lib/galaxy/jobs/runners/pbs.py
M lib/galaxy/jobs/runners/util/job_script/CLUSTER_SLOTS_STATEMENT.sh
M lib/galaxy/jobs/runners/util/job_script/DEFAULT_JOB_FILE_TEMPLATE.sh
M lib/galaxy/jobs/runners/util/job_script/__init__.py
M lib/galaxy/tools/deps/dependencies.py
A lib/galaxy/util/bytesize.py
M test/base/integration_util.py
M test/functional/tools/samples_tool_conf.xml
M test/integration/test_containerized_jobs.py
M test/integration/test_job_recovery.py
A test/integration/test_kubernetes_runner.py
M test/unit/jobs/test_command_factory.py
Log Message:
-----------
Merge pull request #6958 from mvdbeek/kubernetes_integration_test
Kubernetes job runner integration test and enhancements
Compare: https://github.com/galaxyproject/galaxy/compare/65f0c2ac8da8...c53fb623d8cf