Hi Peter, if you open the page for data_manager_shapeit_reference in the test ToolShed you will see that revision 5 and 6 are not installable, they both have "*This revision can be installed:* False" . If you instead select revision 3 it is installable.

Cheers, Nicola

On 17/09/18 11:51, Peter van Heusden wrote:

I'm following up on my previous email. I somehow managed to get `planemo shed_update` to work but now I seem to have an inconsistent view of the Test toolshed. This is how it looks from the Admin panel of my Galaxy server:

https://files.gitter.im/galaxyproject/Lobby/k2qH/image.png Notice that it shows 3 installable revisions for the data manager, last updated 13th of September. And this is how it looks from the Test toolshed website:

https://files.gitter.im/galaxyproject/Lobby/ymY9/image.png

Notice that it has 6 installabale revisions for the data manager. Is my Galaxy server perhaps caching something?

Peter

On Sat, 15 Sep 2018 at 09:57 Peter van Heusden <pvh@sanbi.ac.za mailto:pvh@sanbi.ac.za> wrote:

Hi there

I'm working on a data manager to keep track of the reference data
collection used by the SHAPEIT2 tool. I had to add a column to the
data manager but when I try and do a `planemo shed_update` I get
the error:

Unexpected HTTP status code: 500: {"err_msg": "Error attempting to
parse file tool_data_table_conf.xml.sample: Merging tabular data
tables with non matching columns is not allowed:
shapeit_ref:{'path': 2, 'reference_prefix': 3, 'name': 1, 'value':
0, 'map_prefix': 4} != shapeit_ref:{'name': 1, 'value': 0,
'sample_prefix': 5, 'path': 2, 'reference_prefix': 3,
'map_prefix': 4}"}

This is for data_manager_shapeit_reference in testtoolshed, which
is on github at

https://github.com/pvanheus/data_manager_shapeit_reference

As far as I can tell the columns do accord. Is there any way to
see which files are being merged to help try and debug this error?

Thanks!
Peter

---

Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/

A bit more tracing leads me to suspect that what is happening is that a merge is being called here:

https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/tools/data/__ini...

from:

https://github.com/galaxyproject/galaxy/blob/dev/lib/tool_shed/tools/data_ta...

i.e. the toolshed maintains an in-memory copy of tool-data tables and if the list of columns in an incoming tool-data table is not the same as the in-memory version the error I saw happens.

Which leaves me with the question: how is one meant to update the column list of a tool data table?

Peter

On Mon, 17 Sep 2018 at 13:23 Peter van Heusden pvh@sanbi.ac.za wrote:

Thanks - which is probably because of the previous failures with `shed_update` - which relate back to the topic of my original email. I suspect that failure from `planemo shed_update` has led to this "not installable" state. And I don't know how to solve the problem from `shed_update` which *looks* like it is coming from a clash between previous and current definitions of the tool-data table.

Peter

On Mon, 17 Sep 2018 at 12:58 Nicola Soranzo nicola.soranzo@gmail.com wrote:

...

Hi Peter, if you open the page for data_manager_shapeit_reference in the test ToolShed you will see that revision 5 and 6 are not installable, they both have "*This revision can be installed:* False" . If you instead select revision 3 it is installable.

Cheers, Nicola

On 17/09/18 11:51, Peter van Heusden wrote:

I'm following up on my previous email. I somehow managed to get `planemo shed_update` to work but now I seem to have an inconsistent view of the Test toolshed. This is how it looks from the Admin panel of my Galaxy server:

[image: https://files.gitter.im/galaxyproject/Lobby/k2qH/image.png] Notice that it shows 3 installable revisions for the data manager, last updated 13th of September. And this is how it looks from the Test toolshed website:

[image: https://files.gitter.im/galaxyproject/Lobby/ymY9/image.png]

Notice that it has 6 installabale revisions for the data manager. Is my Galaxy server perhaps caching something?

Peter

On Sat, 15 Sep 2018 at 09:57 Peter van Heusden pvh@sanbi.ac.za wrote:

...

Hi there

I'm working on a data manager to keep track of the reference data collection used by the SHAPEIT2 tool. I had to add a column to the data manager but when I try and do a `planemo shed_update` I get the error:

Unexpected HTTP status code: 500: {"err_msg": "Error attempting to parse file tool_data_table_conf.xml.sample: Merging tabular data tables with non matching columns is not allowed: shapeit_ref:{'path': 2, 'reference_prefix': 3, 'name': 1, 'value': 0, 'map_prefix': 4} != shapeit_ref:{'name': 1, 'value': 0, 'sample_prefix': 5, 'path': 2, 'reference_prefix': 3, 'map_prefix': 4}"}

This is for data_manager_shapeit_reference in testtoolshed, which is on github at

https://github.com/pvanheus/data_manager_shapeit_reference

As far as I can tell the columns do accord. Is there any way to see which files are being merged to help try and debug this error?

Thanks! Peter

---

Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/

Yes that was my conclusion, thanks Peter.

As a result I switched to a new table named (shapeit2_ref) and a new tool name (data_manager_shapeit2_reference) and from planemo side upload seemed to succeed but the tool is in an error state. I resorted to manually editing the tool-data table.

Data managers are a bit of an ugly duckling part of Galaxy - not receiving as much love as other kinds of tools.

Peter

On Tue, 18 Sep 2018 at 14:22 Peter Cock p.j.a.cock@googlemail.com wrote:

Hi Peter vH,

I wonder if adding columns is not supported - because some tools would define the table without the extra column? You might have to define a new data table instead (with largely the same content). But to be clear, I am not an expert in this area of Galaxy.

However, it does seem the IUC could lay out some *.loc file best practises for tool authors. There is currently a TODO entry here:

https://galaxy-iuc-standards.readthedocs.io/en/latest/best_practices/tool_xm...

Peter C.

On Mon, Sep 17, 2018 at 10:56 PM Peter van Heusden pvh@sanbi.ac.za wrote:

...

A bit more tracing leads me to suspect that what is happening is that a merge is being called here:

https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/tools/data/__ini...

from:

https://github.com/galaxyproject/galaxy/blob/dev/lib/tool_shed/tools/data_ta...

i.e. the toolshed maintains an in-memory copy of tool-data tables and if the list of columns in an incoming tool-data table is not the same as the in-memory version the error I saw happens.

Which leaves me with the question: how is one meant to update the column list of a tool data table?

Peter