Suggestion for improved feedback on failing tests
Hi All, My guess is that one of the most common ways in which tools will fail tests on the build-bot is when a dependency fails to install properly. I think this is what is happening to my tool xtandem http://testtoolshed.g2.bx.psu.edu/view/iracooke/xtandem One potential improvement that I think could make it easier to debug this situation would be to show test status for repositories even when they contain no tools (the test would simply attempt an install and show the installation log under the test details). This would be particularly useful for repositories that exist purely to install a dependency eg http://testtoolshed.g2.bx.psu.edu/view/iracooke/galaxy_protk Naturally a tool with proper functional tests would fail if its dependency installations fail .. but it would be a huge help in narrowing down the issue to be able to see where those failures occurred. Another point is that this is pretty much the only way of getting feedback on why a tool might fail installation on the build-bot without having a perfect clone of the build-bot locally. Cheers Ira
On Tue, May 14, 2013 at 3:32 AM, Ira Cooke <iracooke@gmail.com> wrote:
Hi All,
My guess is that one of the most common ways in which tools will fail tests on the build-bot is when a dependency fails to install properly. I think this is what is happening to my tool xtandem
You may be suffering from the same "missing test results" problem as me, see the long thread "Missing test results on (Test) Tool Shed": http://dev.list.galaxyproject.org/Missing-test-results-on-Test-Tool-Shed-td4... Peter
Hi Peter, Yes that's most likely part of it ... thanks for the link to that thread. I think I am suffering from the same issue. ... but I'm also assuming when my test results come back I'll still need to figure out where my repository dependencies failed. Last test result I saw it looked like a failure compiling ruby ... but it's hard to know what's missing to fix it. Cheers Ira On 14/05/2013, at 6:54 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
On Tue, May 14, 2013 at 3:32 AM, Ira Cooke <iracooke@gmail.com> wrote:
Hi All,
My guess is that one of the most common ways in which tools will fail tests on the build-bot is when a dependency fails to install properly. I think this is what is happening to my tool xtandem
You may be suffering from the same "missing test results" problem as me, see the long thread "Missing test results on (Test) Tool Shed": http://dev.list.galaxyproject.org/Missing-test-results-on-Test-Tool-Shed-td4...
Peter
On Tue, May 14, 2013 at 10:39 AM, Ira Cooke <iracooke@gmail.com> wrote:
Hi Peter,
Yes that's most likely part of it ... thanks for the link to that thread. I think I am suffering from the same issue.
... but I'm also assuming when my test results come back I'll still need to figure out where my repository dependencies failed. Last test result I saw it looked like a failure compiling ruby ... but it's hard to know what's missing to fix it.
Cheers Ira
Me too - I've been struggling with tests failing due to partial installs (where only the beginning of the tool_dependencies.xml is processed but no error from the installation process is show on the Tool Shed): http://dev.list.galaxyproject.org/Handling-of-tool-dependencies-xml-errors-i... Peter
We're fairly close to having a new container in the "Tool test results" container. The new container will be something like "Installation errors" and for each tool it will list the tool dependencies that have installation errors. Any installation errors will result in the tool not being tested. This new feature should hopefully be available today. Greg Von Kuster On May 13, 2013, at 10:32 PM, Ira Cooke wrote:
Hi All,
My guess is that one of the most common ways in which tools will fail tests on the build-bot is when a dependency fails to install properly. I think this is what is happening to my tool xtandem
http://testtoolshed.g2.bx.psu.edu/view/iracooke/xtandem
One potential improvement that I think could make it easier to debug this situation would be to show test status for repositories even when they contain no tools (the test would simply attempt an install and show the installation log under the test details). This would be particularly useful for repositories that exist purely to install a dependency eg
http://testtoolshed.g2.bx.psu.edu/view/iracooke/galaxy_protk
Naturally a tool with proper functional tests would fail if its dependency installations fail .. but it would be a huge help in narrowing down the issue to be able to see where those failures occurred. Another point is that this is pretty much the only way of getting feedback on why a tool might fail installation on the build-bot without having a perfect clone of the build-bot locally.
Cheers Ira
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
participants (3)
-
Greg Von Kuster -
Ira Cooke -
Peter Cock