On Tue, Apr 16, 2013 at 9:42 PM, Greg Von Kuster <greg@bx.psu.edu> wrote:
Hello Peter,

On Apr 15, 2013, at 11:04 AM, Peter Cock wrote:

The "View tool functional test results" option clearly lists failed tests,
but it is not obvious if there were any successful tests. Could that
be indicated somehow (e.g. five tests passed for tool xxx).


If I understand your request, I believe we already provide it.  Functional test results are displayed for tests that passed or failed.  For example, see the functional test results for revision 3:aa76c8dd97e6 of jjohnson's pileup_to_vcf repository, which displays the following.  Is this what you are asking for?

Thanks!


Tests that failed
Tool id: pileup_to_vcf version: 2.0
Test id test_tool_000001 (functional.test_toolbox.TestForTool_toolshed.g2.bx.psu.edu/repos/jjohnson/pileup_to_vcf/pileup_to_vcf/2.0)
Status Test failed
Stderr
Traceback Traceback (most recent call last):
  File "/var/opt/buildbot/slaves/buildbot-install-test-1/buildbot-install-test-main-tool-shed-py27/build/test/functional/test_toolbox.py", line 171, in test_tool
    self.do_it( td, shed_tool_id=shed_tool_id )
  File "/var/opt/buildbot/slaves/buildbot-install-test-1/buildbot-install-test-main-tool-shed-py27/build/test/functional/test_toolbox.py", line 102, in do_it
    self.verify_dataset_correctness( outfile, hid=elem_hid, maxseconds=testdef.maxseconds, attributes=attributes, shed_tool_id=shed_tool_id )
  File "/var/opt/buildbo<t/slaves/buildbot-install-test-1/buildbot-install-test-main-tool-shed-py27/build/test/base/twilltestcase.py", line 798, in verify_dataset_correctness
    raise AssertionError( errmsg )
AssertionError: History item 2 different than expected
No text matching expression 'chr1\t158571283\t.\tA\tC\t.\tPASS\t.*' was found in output file.
Tests that passed successfully
Tool id: pileup_to_vcf version: 2.0
Test id test_tool_000000 (functional.test_toolbox.TestForTool_toolshed.g2.bx.psu.edu/repos/jjohnson/pileup_to_vcf/pileup_to_vcf/2.0)
Status Test passed



That does look good - maybe I've been unlucky in the examples I've looked
at due to missing dependencies and/or true results being hidden by this issue:

https://trello.com/card/toolshed-automated-functional-test-bug/506338ce32ae458f6d15e4b3/793

Thanks,

Peter