[galaxy-commits] [galaxyproject/galaxy] d5ba73: Fix hang when installing data managers

GitHub noreply at github.com
Thu Sep 8 13:52:44 EDT 2016


  Branch: refs/heads/dev
  Home:   https://github.com/galaxyproject/galaxy
  Commit: d5ba73b0c84f6bfa297d4b75296e5614d25a07cf
      https://github.com/galaxyproject/galaxy/commit/d5ba73b0c84f6bfa297d4b75296e5614d25a07cf
  Author: Marius van den Beek <m.vandenbeek at gmail.com>
  Date:   2016-09-08 (Thu, 08 Sep 2016)

  Changed paths:
    M lib/tool_shed/galaxy_install/tools/tool_panel_manager.py

  Log Message:
  -----------
  Fix hang when installing data managers


  Commit: 82c1da4709a5ae5b52ee60b2b197f91d64c5d5d3
      https://github.com/galaxyproject/galaxy/commit/82c1da4709a5ae5b52ee60b2b197f91d64c5d5d3
  Author: Marius van den Beek <m.vandenbeek at gmail.com>
  Date:   2016-09-08 (Thu, 08 Sep 2016)

  Changed paths:
    M lib/galaxy/queue_worker.py

  Log Message:
  -----------
  Add data managers to new toolbox

before replacing the old toolbox, to avoid 'Tool not found or not
accessible.' while installing new tools


  Commit: a8f794290dc1e529dd9564a95a55f4b808b2e6d9
      https://github.com/galaxyproject/galaxy/commit/a8f794290dc1e529dd9564a95a55f4b808b2e6d9
  Author: Marius van den Beek <m.vandenbeek at gmail.com>
  Date:   2016-09-08 (Thu, 08 Sep 2016)

  Changed paths:
    M lib/galaxy/tools/toolbox/base.py

  Log Message:
  -----------
  Handle xml ParseError

Occasionally the following Traceback can be seen in the log files:
```
Traceback (most recent call last):
  File "lib/galaxy/tools/toolbox/base.py", line 112, in
_init_tools_from_configs
    self._init_tools_from_config( config_filename )
  File "lib/galaxy/tools/toolbox/base.py", line 136, in
_init_tools_from_config
    tool_conf_source = get_toolbox_parser(config_filename)
  File "lib/galaxy/tools/toolbox/parser.py", line 152, in
get_toolbox_parser
    return XmlToolConfSource(config_filename)
  File "lib/galaxy/tools/toolbox/parser.py", line 40, in __init__
    tree = parse_xml(config_filename)
  File "lib/galaxy/util/__init__.py", line 210, in parse_xml
    root = tree.parse( fname, parser=ElementTree.XMLParser(
target=DoctypeSafeCallbackTarget() ) )
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 657, in parse
    self._root = parser.close()
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1654, in
close
    self._raiseerror(v)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1506, in
_raiseerror
    raise err
ParseError: unclosed token: line 2636, column 0
```
We just try parsing again, which seems to solve the issue.
This may be a problem with concurrently reading/writing to
the file.


  Commit: 89bafff06170c02e5089800a5062e3de0421ad23
      https://github.com/galaxyproject/galaxy/commit/89bafff06170c02e5089800a5062e3de0421ad23
  Author: Marius van den Beek <m.vandenbeek at gmail.com>
  Date:   2016-09-08 (Thu, 08 Sep 2016)

  Changed paths:
    M lib/tool_shed/galaxy_install/tools/data_manager.py

  Log Message:
  -----------
  Whitespace fix


  Commit: c3886b373e8954c1f7bb193cf28357aebd8cd459
      https://github.com/galaxyproject/galaxy/commit/c3886b373e8954c1f7bb193cf28357aebd8cd459
  Author: Marius van den Beek <m.vandenbeek at gmail.com>
  Date:   2016-09-08 (Thu, 08 Sep 2016)

  Changed paths:
    M lib/galaxy/config.py
    M lib/galaxy/queue_worker.py

  Log Message:
  -----------
  Specifically index new toolbox


  Commit: 6fd4c5a5be75a39e941ab4b4b4df95c9456ca1e9
      https://github.com/galaxyproject/galaxy/commit/6fd4c5a5be75a39e941ab4b4b4df95c9456ca1e9
  Author: Marius van den Beek <m.vandenbeek at gmail.com>
  Date:   2016-09-08 (Thu, 08 Sep 2016)

  Changed paths:
    M lib/galaxy/queue_worker.py

  Log Message:
  -----------
  Keep track of toolbox reload time


  Commit: 230e7df4ae293e46e4f15bf2cf811f788a9f811d
      https://github.com/galaxyproject/galaxy/commit/230e7df4ae293e46e4f15bf2cf811f788a9f811d
  Author: Marius van den Beek <m.vandenbeek at gmail.com>
  Date:   2016-09-08 (Thu, 08 Sep 2016)

  Changed paths:
    M lib/galaxy/tools/toolbox/base.py
    M lib/galaxy/tools/toolbox/cache.py

  Log Message:
  -----------
  Use cache to get the repository containing a tool

when reloading toolbox. This should prevent lots of (slow)
database requests when reloading the toolbox.


  Commit: c5406eb192a0e2355602e5e5d7eb1e218a632d07
      https://github.com/galaxyproject/galaxy/commit/c5406eb192a0e2355602e5e5d7eb1e218a632d07
  Author: Marius van den Beek <m.vandenbeek at gmail.com>
  Date:   2016-09-08 (Thu, 08 Sep 2016)

  Changed paths:
    M lib/galaxy/tools/toolbox/base.py

  Log Message:
  -----------
  Improve logic handling of loading tools


  Commit: ab9be8c311a7943d778842522a706451e9996943
      https://github.com/galaxyproject/galaxy/commit/ab9be8c311a7943d778842522a706451e9996943
  Author: Marius van den Beek <m.vandenbeek at gmail.com>
  Date:   2016-09-08 (Thu, 08 Sep 2016)

  Changed paths:
    M lib/galaxy/tools/toolbox/base.py

  Log Message:
  -----------
  Fetch cached tools repository


  Commit: 894b77ba23fa66a8852b554aacf18cc123f5d7cd
      https://github.com/galaxyproject/galaxy/commit/894b77ba23fa66a8852b554aacf18cc123f5d7cd
  Author: Marius van den Beek <m.vandenbeek at gmail.com>
  Date:   2016-09-08 (Thu, 08 Sep 2016)

  Changed paths:
    M lib/galaxy/tools/toolbox/lineages/factory.py

  Log Message:
  -----------
  Simplify lineage registration


  Commit: acc5b4ceb6525a9cf53bf39c3d6c00abf9069990
      https://github.com/galaxyproject/galaxy/commit/acc5b4ceb6525a9cf53bf39c3d6c00abf9069990
  Author: Marius van den Beek <m.vandenbeek at gmail.com>
  Date:   2016-09-08 (Thu, 08 Sep 2016)

  Changed paths:
    M lib/galaxy/tools/toolbox/base.py
    M lib/galaxy/tools/toolbox/lineages/__init__.py
    M lib/galaxy/tools/toolbox/lineages/factory.py
    M lib/galaxy/tools/toolbox/lineages/tool_shed.py

  Log Message:
  -----------
  Load all versions in a single request


  Commit: 971227c59dd1d32b537c04c59a16e7572aa7acbd
      https://github.com/galaxyproject/galaxy/commit/971227c59dd1d32b537c04c59a16e7572aa7acbd
  Author: Marius van den Beek <m.vandenbeek at gmail.com>
  Date:   2016-09-08 (Thu, 08 Sep 2016)

  Changed paths:
    M lib/galaxy/config.py
    M lib/galaxy/model/tool_shed_install/__init__.py
    M lib/galaxy/queue_worker.py
    M lib/galaxy/tools/toolbox/base.py
    M lib/galaxy/tools/toolbox/lineages/__init__.py
    M lib/galaxy/tools/toolbox/lineages/tool_shed.py
    M lib/galaxy/tools/toolbox/panel.py
    A lib/galaxy/util/profiling.py

  Log Message:
  -----------
  Use tool version cache to boost tool panel loading


  Commit: 833af5e8c2121947e16bdfb99b2794b7a130d045
      https://github.com/galaxyproject/galaxy/commit/833af5e8c2121947e16bdfb99b2794b7a130d045
  Author: Marius van den Beek <m.vandenbeek at gmail.com>
  Date:   2016-09-08 (Thu, 08 Sep 2016)

  Changed paths:
    M lib/galaxy/model/tool_shed_install/__init__.py
    M lib/galaxy/tools/toolbox/base.py
    M lib/galaxy/tools/toolbox/lineages/__init__.py
    M lib/galaxy/tools/toolbox/lineages/factory.py
    M lib/galaxy/tools/toolbox/lineages/tool_shed.py
    M lib/galaxy/util/profiling.py

  Log Message:
  -----------
  Restore peacemeal tool registration


  Commit: 8bed867fcfcf249838b6b24a1b5985af401b9b5a
      https://github.com/galaxyproject/galaxy/commit/8bed867fcfcf249838b6b24a1b5985af401b9b5a
  Author: Marius van den Beek <m.vandenbeek at gmail.com>
  Date:   2016-09-08 (Thu, 08 Sep 2016)

  Changed paths:
    M test/unit/shed_unit/test_tool_panel_manager.py
    M test/unit/tools/test_toolbox.py
    M test/unit/tools_support.py

  Log Message:
  -----------
  Reload tool_version_cache after new installs

in the unit-testing code.


  Commit: 894ac295207430d5ec1b02a0be7ee76ce4dc2cc0
      https://github.com/galaxyproject/galaxy/commit/894ac295207430d5ec1b02a0be7ee76ce4dc2cc0
  Author: Marius van den Beek <m.vandenbeek at gmail.com>
  Date:   2016-09-08 (Thu, 08 Sep 2016)

  Changed paths:
    M lib/tool_shed/tools/tool_version_manager.py

  Log Message:
  -----------
  Reload tool_version_cache when adding new tool

version relationships to the database.


  Commit: 09173cc6070bc914759f15854367d62ca6323c79
      https://github.com/galaxyproject/galaxy/commit/09173cc6070bc914759f15854367d62ca6323c79
  Author: Marius van den Beek <m.vandenbeek at gmail.com>
  Date:   2016-09-08 (Thu, 08 Sep 2016)

  Changed paths:
    M lib/galaxy/tools/toolbox/base.py
    M lib/galaxy/tools/toolbox/lineages/__init__.py
    M lib/galaxy/tools/toolbox/lineages/factory.py
    M lib/galaxy/tools/toolbox/panel.py
    M lib/galaxy/util/profiling.py
    M lib/tool_shed/tools/tool_version_manager.py

  Log Message:
  -----------
  Rebase and linting fixes


  Commit: c08db31ab22aefa23a0e6472c4f115cff537668f
      https://github.com/galaxyproject/galaxy/commit/c08db31ab22aefa23a0e6472c4f115cff537668f
  Author: Marius van den Beek <m.vandenbeek at gmail.com>
  Date:   2016-09-08 (Thu, 08 Sep 2016)

  Changed paths:
    M lib/galaxy/tools/toolbox/lineages/tool_shed.py
    R lib/galaxy/util/profiling.py

  Log Message:
  -----------
  Document ToolVersionCache class


  Commit: 92d028cbae55d120326cfac61f2fd4fb5d425b35
      https://github.com/galaxyproject/galaxy/commit/92d028cbae55d120326cfac61f2fd4fb5d425b35
  Author: Marius van den Beek <m.vandenbeek at gmail.com>
  Date:   2016-09-08 (Thu, 08 Sep 2016)

  Changed paths:
    M test/unit/tools_support.py

  Log Message:
  -----------
  Remove unnecessary self.install_model from unittest app


  Commit: 5c45da362ad27c788fe5f7e62fd5c779b7fa24bf
      https://github.com/galaxyproject/galaxy/commit/5c45da362ad27c788fe5f7e62fd5c779b7fa24bf
  Author: Martin Cech <cech.marten at gmail.com>
  Date:   2016-09-08 (Thu, 08 Sep 2016)

  Changed paths:
    M lib/galaxy/config.py
    M lib/galaxy/model/tool_shed_install/__init__.py
    M lib/galaxy/queue_worker.py
    M lib/galaxy/tools/toolbox/base.py
    M lib/galaxy/tools/toolbox/cache.py
    M lib/galaxy/tools/toolbox/lineages/__init__.py
    M lib/galaxy/tools/toolbox/lineages/factory.py
    M lib/galaxy/tools/toolbox/lineages/tool_shed.py
    M lib/tool_shed/galaxy_install/tools/data_manager.py
    M lib/tool_shed/galaxy_install/tools/tool_panel_manager.py
    M lib/tool_shed/tools/tool_version_manager.py
    M test/unit/shed_unit/test_tool_panel_manager.py
    M test/unit/tools/test_toolbox.py
    M test/unit/tools_support.py

  Log Message:
  -----------
  Merge pull request #2901 from mvdbeek/fix_data_manager_hangs

All-around improvements to toolbox loading/reloading


Compare: https://github.com/galaxyproject/galaxy/compare/0c3b5272eb53...5c45da362ad2


More information about the galaxy-commits mailing list