[galaxyproject/galaxy] 32b85e: Only permit yaml.safe_loading of data

25 Sep 2017

      Branch: refs/heads/dev
  Home:   https://github.com/galaxyproject/galaxy
  Commit: 32b85ecb591d01acbbbb9ed1bedd1a48804a0837
      https://github.com/galaxyproject/galaxy/commit/32b85ecb591d01acbbbb9ed1bedd1...
  Author: E Rasche <hxr@hx42.org>
  Date:   2017-09-25 (Mon, 25 Sep 2017)

  Changed paths:
    M lib/galaxy/containers/__init__.py
    M lib/galaxy/datatypes/registry.py
    M lib/galaxy/jobs/runners/pulsar.py
    M lib/galaxy/tools/deps/conda_compat.py
    M lib/galaxy/tools/deps/resolvers/__init__.py
    M lib/galaxy/tools/locations/dockstore.py
    M lib/galaxy/tools/parser/factory.py
    M lib/galaxy/tools/toolbox/parser.py
    M lib/galaxy/tours/__init__.py
    M lib/galaxy/util/plugin_config.py
    M lib/galaxy/util/properties.py
    M lib/galaxy/visualization/plugins/interactive_environments.py
    M lib/galaxy/webapps/config_manage.py
    M lib/galaxy/webapps/galaxy/api/users.py
    M lib/galaxy/webapps/galaxy/controllers/visualization.py
    M lib/galaxy/webhooks/__init__.py
    M scripts/grt/export.py
    M scripts/grt/upload.py
    M test/api/test_workflows.py
    M test/api/test_workflows_from_yaml.py
    M test/base/workflows_format_2/converter.py
    M test/base/workflows_format_2/main.py
    M test/galaxy_selenium/data.py
    M test/galaxy_selenium/navigates_galaxy.py
    M test/unit/workflows/workflow_support.py

  Log Message:
  -----------
  Only permit yaml.safe_loading of data

Event trusted data, belt + suspenders method.

  Commit: df41d4f4087f9b133750a2c350ea207753f30f54
      https://github.com/galaxyproject/galaxy/commit/df41d4f4087f9b133750a2c350ea2...
  Author: E Rasche <hxr@hx42.org>
  Date:   2017-09-25 (Mon, 25 Sep 2017)

  Changed paths:
    M lib/galaxy/datatypes/converters/interval_to_coverage.py
    M lib/galaxy/datatypes/converters/lped_to_pbed_converter.py
    M lib/galaxy/datatypes/converters/pbed_ldreduced_converter.py
    M lib/galaxy/datatypes/converters/pbed_to_lped_converter.py
    M lib/galaxy/datatypes/text.py
    M lib/galaxy/jobs/runners/util/job_script/__init__.py
    M lib/galaxy/jobs/runners/util/kill.py
    M lib/galaxy/jobs/transfer_manager.py

  Log Message:
  -----------
  Remove some shell=True and fix commands

  Commit: cd6a1afaa9980d958ad324fa987a07e7608bf841
      https://github.com/galaxyproject/galaxy/commit/cd6a1afaa9980d958ad324fa987a0...
  Author: E Rasche <hxr@hx42.org>
  Date:   2017-09-25 (Mon, 25 Sep 2017)

  Changed paths:
    M lib/galaxy/datatypes/sequence.py
    M lib/galaxy/datatypes/tabular.py
    M lib/galaxy/jobs/runners/pulsar.py

  Log Message:
  -----------
  Replace some os.system calls with subprocess

  Commit: 73337725bdd654904563dc54b04083a23e4a252d
      https://github.com/galaxyproject/galaxy/commit/73337725bdd654904563dc54b0408...
  Author: E Rasche <hxr@hx42.org>
  Date:   2017-09-25 (Mon, 25 Sep 2017)

  Changed paths:
    M lib/galaxy/objectstore/s3.py

  Log Message:
  -----------
  Two more commands to list

  Commit: c85db3fd68590d3d2f61b0b3f88026773d9563a0
      https://github.com/galaxyproject/galaxy/commit/c85db3fd68590d3d2f61b0b3f8802...
  Author: E Rasche <hxr@hx42.org>
  Date:   2017-09-25 (Mon, 25 Sep 2017)

  Changed paths:
    M lib/galaxy/datatypes/binary.py
    M lib/galaxy/tools/deps/mulled/mulled_build.py
    M lib/galaxy/tools/deps/mulled/mulled_build_channel.py

  Log Message:
  -----------
  Some more command refactoring

  Commit: 670897ec5ca2284c7fad453c3d595defe3cca280
      https://github.com/galaxyproject/galaxy/commit/670897ec5ca2284c7fad453c3d595...
  Author: E Rasche <hxr@hx42.org>
  Date:   2017-09-25 (Mon, 25 Sep 2017)

  Changed paths:
    M lib/galaxy/webapps/reports/controllers/system.py

  Log Message:
  -----------
  One more os command

  Commit: 9b8e76f9d9df0ae0ba6238d44a8f07b97fe34d6f
      https://github.com/galaxyproject/galaxy/commit/9b8e76f9d9df0ae0ba6238d44a8f0...
  Author: E Rasche <hxr@hx42.org>
  Date:   2017-09-25 (Mon, 25 Sep 2017)

  Changed paths:
    M cron/build_chrom_db.py
    M cron/parse_builds.py
    M cron/parse_builds_3_sites.py
    M lib/galaxy/external_services/actions.py
    M lib/galaxy/jobs/deferred/pacific_biosciences_smrt_portal.py
    M lib/galaxy/managers/citations.py
    M lib/galaxy/tools/data/__init__.py
    M lib/galaxy/tools/imp_exp/unpack_tar_gz_archive.py
    M lib/galaxy/web/proxy/__init__.py
    M lib/galaxy/webapps/config_manage.py
    M lib/galaxy/webapps/galaxy/controllers/async.py
    M lib/galaxy/webapps/galaxy/controllers/library_common.py
    M lib/galaxy/webapps/galaxy/controllers/root.py
    M lib/galaxy/webapps/galaxy/controllers/workflow.py
    M lib/galaxy/webapps/tool_shed/controllers/upload.py
    M lib/tool_shed/capsule/capsule_manager.py
    M scripts/data_libraries/build_lucene_index.py
    M scripts/edam_mapping.py
    M scripts/microbes/harvest_bacteria.py
    M scripts/tool_shed/api/export.py
    M scripts/transfer.py

  Log Message:
  -----------
  Convert urllib to requests

Which has built-in protection against urls like "file:///tmp/a"

  Commit: db0400298651f6d4c9d909406c3fe632d057bbf7
      https://github.com/galaxyproject/galaxy/commit/db0400298651f6d4c9d909406c3fe...
  Author: E Rasche <hxr@hx42.org>
  Date:   2017-09-25 (Mon, 25 Sep 2017)

  Changed paths:
    M cron/parse_builds.py
    M lib/galaxy/jobs/deferred/pacific_biosciences_smrt_portal.py

  Log Message:
  -----------
  remove unused imports

  Commit: a68d6be3a1c6e6fd85c5ae94232467f193156d53
      https://github.com/galaxyproject/galaxy/commit/a68d6be3a1c6e6fd85c5ae9423246...
  Author: E Rasche <hxr@hx42.org>
  Date:   2017-09-25 (Mon, 25 Sep 2017)

  Changed paths:
    M test/functional/webhooks/phdcomics/helper/__init__.py

  Log Message:
  -----------
  reorder imports

  Commit: 93ffc4055b145811b86191dae8b865ea45b55fcb
      https://github.com/galaxyproject/galaxy/commit/93ffc4055b145811b86191dae8b86...
  Author: E Rasche <hxr@hx42.org>
  Date:   2017-09-25 (Mon, 25 Sep 2017)

  Changed paths:
    M lib/galaxy/tools/deps/container_resolvers/mulled.py

  Log Message:
  -----------
  replace check_output with python processing and shell=False

  Commit: d79a2459a95dfda7856b9898d40d062f00456006
      https://github.com/galaxyproject/galaxy/commit/d79a2459a95dfda7856b9898d40d0...
  Author: E Rasche <hxr@hx42.org>
  Date:   2017-09-25 (Mon, 25 Sep 2017)

  Changed paths:
    M lib/galaxy/tools/parser/factory.py
    M lib/galaxy/webapps/config_manage.py

  Log Message:
  -----------
  These we will revert to their original yaml.load

It would be nice to fix them eventually, but we'll settle for an
incremental improvement.

  Commit: b3261309843f8cfec1a5a589b4628e44e6bf76df
      https://github.com/galaxyproject/galaxy/commit/b3261309843f8cfec1a5a589b4628...
  Author: E Rasche <hxr@hx42.org>
  Date:   2017-09-25 (Mon, 25 Sep 2017)

  Changed paths:
    M cron/parse_builds.py
    M cron/parse_builds_3_sites.py
    M lib/galaxy/jobs/deferred/pacific_biosciences_smrt_portal.py
    M lib/galaxy/tools/data/__init__.py
    M lib/galaxy/tools/imp_exp/unpack_tar_gz_archive.py
    M lib/tool_shed/capsule/capsule_manager.py

  Log Message:
  -----------
  import ordering

  Commit: 8f4491bfffce5d13e2199df5d60e06e3672a08c1
      https://github.com/galaxyproject/galaxy/commit/8f4491bfffce5d13e2199df5d60e0...
  Author: E Rasche <hxr@hx42.org>
  Date:   2017-09-25 (Mon, 25 Sep 2017)

  Changed paths:
    M lib/galaxy/datatypes/binary.py

  Log Message:
  -----------
  Consume stderr

  Commit: 0f7dd52988269dc4567bdbbd71be0fd273741efd
      https://github.com/galaxyproject/galaxy/commit/0f7dd52988269dc4567bdbbd71be0...
  Author: E Rasche <hxr@hx42.org>
  Date:   2017-09-25 (Mon, 25 Sep 2017)

  Changed paths:
    M lib/galaxy/datatypes/converters/sam_to_bam.py

  Log Message:
  -----------
  Copy changes over

  Commit: 75df58ae86ff1796f0bb0ea975fc1d0d518c155a
      https://github.com/galaxyproject/galaxy/commit/75df58ae86ff1796f0bb0ea975fc1...
  Author: E Rasche <hxr@hx42.org>
  Date:   2017-09-25 (Mon, 25 Sep 2017)

  Changed paths:
    M cron/parse_builds_3_sites.py

  Log Message:
  -----------
  fix

  Commit: fd92ac3eedd145635a74530ba62bb7fea7c540f4
      https://github.com/galaxyproject/galaxy/commit/fd92ac3eedd145635a74530ba62bb...
  Author: E Rasche <hxr@hx42.org>
  Date:   2017-09-25 (Mon, 25 Sep 2017)

  Changed paths:
    M lib/galaxy/datatypes/binary.py

  Log Message:
  -----------
  revert this change

  Commit: 27beb7030d99a511a428ad59211a3137a2761559
      https://github.com/galaxyproject/galaxy/commit/27beb7030d99a511a428ad59211a3...
  Author: E Rasche <hxr@hx42.org>
  Date:   2017-09-25 (Mon, 25 Sep 2017)

  Changed paths:
    M lib/galaxy/datatypes/binary.py
    M lib/galaxy/datatypes/converters/sam_to_bam.py

  Log Message:
  -----------
  revert/sync

  Commit: 1a57c06d6a81dc6f92eb4843c89a7333f874bc32
      https://github.com/galaxyproject/galaxy/commit/1a57c06d6a81dc6f92eb4843c89a7...
  Author: E Rasche <hxr@hx42.org>
  Date:   2017-09-25 (Mon, 25 Sep 2017)

  Changed paths:
    M lib/galaxy/datatypes/tabular.py
    M lib/galaxy/datatypes/text.py
    M lib/galaxy/external_services/actions.py
    M lib/galaxy/jobs/deferred/pacific_biosciences_smrt_portal.py
    M lib/galaxy/jobs/transfer_manager.py
    M lib/galaxy/objectstore/s3.py
    M lib/galaxy/tools/data/__init__.py
    M lib/galaxy/web/proxy/__init__.py
    M lib/galaxy/webapps/config_manage.py
    M lib/galaxy/webapps/galaxy/controllers/async.py
    M lib/galaxy/webapps/galaxy/controllers/library_common.py
    M lib/galaxy/webapps/galaxy/controllers/root.py
    M lib/galaxy/webapps/tool_shed/controllers/upload.py
    M lib/tool_shed/capsule/capsule_manager.py

  Log Message:
  -----------
  Fix import ordering

Thanks @nsoranzo, I appreciate all the comments here.

  Commit: 290971ca7a28eb579dc2d40c9653b9e160253634
      https://github.com/galaxyproject/galaxy/commit/290971ca7a28eb579dc2d40c9653b...
  Author: E Rasche <hxr@hx42.org>
  Date:   2017-09-25 (Mon, 25 Sep 2017)

  Changed paths:
    M cron/build_chrom_db.py
    M cron/parse_builds.py
    M lib/galaxy/datatypes/text.py
    M lib/galaxy/jobs/transfer_manager.py
    M scripts/data_libraries/build_lucene_index.py
    M scripts/edam_mapping.py
    M scripts/microbes/harvest_bacteria.py
    M scripts/tool_shed/api/export.py

  Log Message:
  -----------
  more review comments

  Commit: 15c73233c52b29ed94066c06ebb1e10c935ae28d
      https://github.com/galaxyproject/galaxy/commit/15c73233c52b29ed94066c06ebb1e...
  Author: E Rasche <hxr@hx42.org>
  Date:   2017-09-25 (Mon, 25 Sep 2017)

  Changed paths:
    M scripts/edam_mapping.py

  Log Message:
  -----------
  fix lint

  Commit: 19d03edc7ff8384a5360d71365076a2594fb0b8b
      https://github.com/galaxyproject/galaxy/commit/19d03edc7ff8384a5360d71365076...
  Author: E Rasche <hxr@hx42.org>
  Date:   2017-09-25 (Mon, 25 Sep 2017)

  Changed paths:
    M lib/galaxy/tools/deps/mulled/mulled_build_channel.py

  Log Message:
  -----------
  refactor missed call

  Commit: 626a3dc2995f0d797232aa666f1bdf3e6a5d22dd
      https://github.com/galaxyproject/galaxy/commit/626a3dc2995f0d797232aa666f1bd...
  Author: E Rasche <hxr@hx42.org>
  Date:   2017-09-25 (Mon, 25 Sep 2017)

  Changed paths:
    M lib/galaxy/tools/deps/mulled/mulled_build_channel.py

  Log Message:
  -----------
  fix ordering

  Commit: 41574d022249bae4df4b89bfb50c3a599ee5585e
      https://github.com/galaxyproject/galaxy/commit/41574d022249bae4df4b89bfb50c3...
  Author: E Rasche <hxr@hx42.org>
  Date:   2017-09-25 (Mon, 25 Sep 2017)

  Changed paths:
    M lib/galaxy/jobs/runners/util/job_script/__init__.py

  Log Message:
  -----------
  swap for subprocess.call per @nsoranzo suggestion

  Commit: 0c6dfd7a41a8df8c000b133d8833eb9963ae183e
      https://github.com/galaxyproject/galaxy/commit/0c6dfd7a41a8df8c000b133d8833e...
  Author: John Chilton <jmchilton@gmail.com>
  Date:   2017-09-25 (Mon, 25 Sep 2017)

  Changed paths:
    M cron/build_chrom_db.py
    M cron/parse_builds.py
    M cron/parse_builds_3_sites.py
    M lib/galaxy/containers/__init__.py
    M lib/galaxy/datatypes/binary.py
    M lib/galaxy/datatypes/converters/interval_to_coverage.py
    M lib/galaxy/datatypes/converters/lped_to_pbed_converter.py
    M lib/galaxy/datatypes/converters/pbed_ldreduced_converter.py
    M lib/galaxy/datatypes/converters/pbed_to_lped_converter.py
    M lib/galaxy/datatypes/converters/sam_to_bam.py
    M lib/galaxy/datatypes/registry.py
    M lib/galaxy/datatypes/sequence.py
    M lib/galaxy/datatypes/tabular.py
    M lib/galaxy/datatypes/text.py
    M lib/galaxy/external_services/actions.py
    M lib/galaxy/jobs/deferred/pacific_biosciences_smrt_portal.py
    M lib/galaxy/jobs/runners/pulsar.py
    M lib/galaxy/jobs/runners/util/job_script/__init__.py
    M lib/galaxy/jobs/runners/util/kill.py
    M lib/galaxy/jobs/transfer_manager.py
    M lib/galaxy/managers/citations.py
    M lib/galaxy/objectstore/s3.py
    M lib/galaxy/tools/data/__init__.py
    M lib/galaxy/tools/deps/conda_compat.py
    M lib/galaxy/tools/deps/container_resolvers/mulled.py
    M lib/galaxy/tools/deps/mulled/mulled_build.py
    M lib/galaxy/tools/deps/mulled/mulled_build_channel.py
    M lib/galaxy/tools/deps/resolvers/__init__.py
    M lib/galaxy/tools/imp_exp/unpack_tar_gz_archive.py
    M lib/galaxy/tools/locations/dockstore.py
    M lib/galaxy/tools/toolbox/parser.py
    M lib/galaxy/tours/__init__.py
    M lib/galaxy/util/plugin_config.py
    M lib/galaxy/util/properties.py
    M lib/galaxy/visualization/plugins/interactive_environments.py
    M lib/galaxy/web/proxy/__init__.py
    M lib/galaxy/webapps/config_manage.py
    M lib/galaxy/webapps/galaxy/api/users.py
    M lib/galaxy/webapps/galaxy/controllers/async.py
    M lib/galaxy/webapps/galaxy/controllers/library_common.py
    M lib/galaxy/webapps/galaxy/controllers/root.py
    M lib/galaxy/webapps/galaxy/controllers/visualization.py
    M lib/galaxy/webapps/galaxy/controllers/workflow.py
    M lib/galaxy/webapps/reports/controllers/system.py
    M lib/galaxy/webapps/tool_shed/controllers/upload.py
    M lib/galaxy/webhooks/__init__.py
    M lib/tool_shed/capsule/capsule_manager.py
    M scripts/data_libraries/build_lucene_index.py
    M scripts/edam_mapping.py
    M scripts/grt/export.py
    M scripts/grt/upload.py
    M scripts/microbes/harvest_bacteria.py
    M scripts/tool_shed/api/export.py
    M scripts/transfer.py
    M test/api/test_workflows.py
    M test/api/test_workflows_from_yaml.py
    M test/base/workflows_format_2/converter.py
    M test/base/workflows_format_2/main.py
    M test/functional/webhooks/phdcomics/helper/__init__.py
    M test/galaxy_selenium/data.py
    M test/galaxy_selenium/navigates_galaxy.py
    M test/unit/workflows/workflow_support.py

  Log Message:
  -----------
  Merge pull request #4604 from erasche/hardening

Various bits of hardening

Compare: https://github.com/galaxyproject/galaxy/compare/9f2582e9fbed...0c6dfd7a41a8

GitHub

tags

participants (1)