Peter, That is the intended behavior, but I've added a trello card (https://trello.com/c/O9YmzUT4) for revisiting that decision at some point. We are definitely willing to be flexible about the testing conditions, but the primary goal of the automated testing framework was to verify functional correctness of an entire repository, to simplify the approval process for the category of "tools contained within the repository". --Dave B. On 4/25/13 04:55:50.000, Peter Cock wrote:
On Wed, Apr 24, 2013 at 10:12 AM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
..., I've still finding some other cases where despite having tests defined, nothing shows up on the Tool Shed test results page.
Here's an example where I see that two tools (promoter2 and wolf_psort) have no tests, but the results of all the other tools in this suite are not mentioned (some expected to fail/be skipped due to missing dependencies):
http://toolshed.g2.bx.psu.edu/view/peterjc/tmhmm_and_signalp Revision 13:dc958c2a963a
Equivalently:
http://testtoolshed.g2.bx.psu.edu/view/peterjc/tmhmm_and_signalp Revision: 14:0d8b1d20ce9c
--------------------------------------------------------------------------------
Here's another example where based on what happens locally, I am expecting a failure:
http://testtoolshed.g2.bx.psu.edu/view/peterjc/seq_rename Revision: 1:6ce8634e2035
This is failing for me locally, but again I suspect this is due to an issue in the framework - much like the blastxml_to_top_descr test is failing due to the datatypes not being updated, here there seems to be something similar going wrong regarding the second uploaded input file (which is used for to provide the potential column parameters). See this thread:
http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-November/003867.html http://lists.bx.psu.edu/pipermail/galaxy-dev/2013-April/014261.html
Regards,
Peter
Hi Dave,
Yet another case,
http://toolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus Revision: 8:1f546099212f
and,
http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus Revision: 5:d42346e675c4
These list five or six invalid tools which do not define any unit tests, yet there is no mention of the tools present which do have tests (and if they passed or not).
Even if ncbi_blastdbcmd_wrapper doesn't (yet) have a test, I still want to know if the ncbi_blastp_wrapper tests pass ;)
Are this further symptoms of whatever was solved here? https://trello.com/card/toolshed-automated-functional-test-bug/506338ce32ae4...
Or does would a new Trello issue be best?
Thanks,
Peter