[galaxyproject/galaxy] 1d0e4b: New, portable metadata generation.
Branch: refs/heads/dev Home: https://github.com/galaxyproject/galaxy Commit: 1d0e4bcb2cd4eb47ef0a5c1d6027f25547292b06 https://github.com/galaxyproject/galaxy/commit/1d0e4bcb2cd4eb47ef0a5c1d6027f... Author: John Chilton <jmchilton@gmail.com> Date: 2019-03-08 (Fri, 08 Mar 2019) Changed paths: M lib/galaxy/config.py M lib/galaxy/jobs/__init__.py M lib/galaxy/metadata/__init__.py M lib/galaxy/tools/__init__.py M lib/galaxy_ext/metadata/set_metadata.py M test/unit/tools/test_metadata.py M test/unit/unittest_utils/galaxy_mock.py Log Message: ----------- New, portable metadata generation. Variant of external metadata generation that doesn't use command-line arguments or absolute paths and simply takes operates on the current, job working directory and knows what to do. - Uses relative paths allowing the directory to be staged elsewhere via Pulsar, Condor, or Docker for instance. - Avoids DB objects for hard-coded paths making everything more robust. - Avoids huge command-lines which for huge collection reduction jobs might hit system limits for command-line length. - Uses JSON dicts to communicate parameters - meaning we can avoid awkward argument parsing as this evolves (see old max_metadata_value_size handling in set_metadata.py for instance). - Uses output names instead of database IDs to key files - so file naming is consistent and human readable. Commit: 17cb35bc25b2b7a9338d23d65d3cbaa8870c93bb https://github.com/galaxyproject/galaxy/commit/17cb35bc25b2b7a9338d23d65d3cb... Author: John Chilton <jmchilton@gmail.com> Date: 2019-03-08 (Fri, 08 Mar 2019) Changed paths: M lib/galaxy/tools/provided_metadata.py A lib/galaxy_ext/metadata/_provided_metadata.py M lib/galaxy_ext/metadata/set_metadata.py Log Message: ----------- Use ToolProvidedMetadata abstraction in set_metadata. There is are somethings that happen in set_metadata.py that assume the old style galaxy.json. This sets the stage for proper external metadata collection of newer style galaxy.json files. Commit: 395380fc66c76dbbc49ef8f99792c17c23e5e3ba https://github.com/galaxyproject/galaxy/commit/395380fc66c76dbbc49ef8f99792c... Author: John Chilton <jmchilton@gmail.com> Date: 2019-03-08 (Fri, 08 Mar 2019) Changed paths: M lib/galaxy/tools/provided_metadata.py Log Message: ----------- Fix: Handle galaxy.json entries without dataset_id in galaxy.json. Commit: e7676868162497df4cdf35f86bc4afa31c2f8267 https://github.com/galaxyproject/galaxy/commit/e7676868162497df4cdf35f86bc4a... Author: John Chilton <jmchilton@gmail.com> Date: 2019-03-08 (Fri, 08 Mar 2019) Changed paths: M lib/galaxy/metadata/__init__.py Log Message: ----------- Better I/O in metadata package. Commit: f0912eb4040185df5a11cff1c7e1865a6b18f33f https://github.com/galaxyproject/galaxy/commit/f0912eb4040185df5a11cff1c7e18... Author: John Chilton <jmchilton@gmail.com> Date: 2019-03-10 (Sun, 10 Mar 2019) Changed paths: M lib/galaxy/datatypes/set_metadata_tool.xml M lib/galaxy/metadata/__init__.py M lib/galaxy/tools/actions/metadata.py M lib/galaxy_ext/metadata/set_metadata.py Log Message: ----------- Fix set_metadata executing from wrong directory when using tool. Commit: 1ca256aaba84fb101f144c67447363435ec44622 https://github.com/galaxyproject/galaxy/commit/1ca256aaba84fb101f144c6744736... Author: John Chilton <jmchilton@gmail.com> Date: 2019-03-11 (Mon, 11 Mar 2019) Changed paths: M lib/galaxy/config.py M lib/galaxy/datatypes/set_metadata_tool.xml M lib/galaxy/jobs/__init__.py M lib/galaxy/metadata/__init__.py M lib/galaxy/tools/__init__.py M lib/galaxy/tools/actions/metadata.py M lib/galaxy/tools/provided_metadata.py A lib/galaxy_ext/metadata/_provided_metadata.py M lib/galaxy_ext/metadata/set_metadata.py M test/unit/tools/test_metadata.py M test/unit/unittest_utils/galaxy_mock.py Log Message: ----------- Merge pull request #7470 from jmchilton/portable_metadata Implement portable metadata generation. Compare: https://github.com/galaxyproject/galaxy/compare/417292b5013b...1ca256aaba84
participants (1)
-
John Chilton