Hi Peter,


On Nov 28, 2013, at 12:30 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:

On Thu, Nov 28, 2013 at 5:16 PM, Greg Von Kuster <greg@bx.psu.edu> wrote:
Hello Peter,

On Nov 28, 2013, at 4:56 AM, Peter Cock <p.j.a.cock@googlemail.com> wrote:

Hi Greg,

Things look much better today on the Test Tool Shed, with
most of my tools under "Latest revision: all tool tests pass"
again - or as expected a few under "Latest revision: missing
tool tests" due to not yet having full test coverage.

I've only spotted one glitch so far, missing test results for:
http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus/b2795652d2b4

The above repository was not displaying the missing test components due to a problem in the previous test runs which I fixed yesterday.  The problem resulted in test result data that could not be displayed.  


Perhaps a fall back error message would be useful here (e.g. "Test failed but unable to display data"), which would be better than the "missing" test results?

 
When I introduced the new feature for stroing and displaying multiple test runs, I decided to not eliminate the bad data, but just to let enough test runs proceed that the bad data would eventually be eliminated.  I forced new test runs, and now things should be displaying as expected.  Here is the display for the above repository for the latest run this morning:



Thank you - that alerted me to a missing set of out output files for a recently added test, hopefully addressed in this revision:
http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus/1b8a92016045

But its not all good news - another test is now listed as failing, but with missing test results:

The bad data shoud now be eliminated, and there are 3 test runs displaying for this one.  Do these look crrect to you?  The oldest test run has a data of 'unknown' because I had not implemented the date deature at that point.

Automated tool test results

Test runs
2013-11-28 14:50:44
Automated test environment
Time tested: 2013-11-28 14:50:44
System: Linux 3.8.0-30-generic
Architecture: x86_64
Python version: 2.7.4
Galaxy revision: 11549:5aede225ebef
Galaxy database version: 117
Tool shed revision: 11549:5aede225ebef
Tool shed database version: 21
Tool shed mercurial version: 2.2.3
Tests that passed successfully
Tool id: get_orfs_or_cdss
Tool version: get_orfs_or_cdss
Test: test_tool_000000 (functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/peterjc/get_orfs_or_cdss/get_orfs_or_cdss/0.0.5)
Tool id: get_orfs_or_cdss
Tool version: get_orfs_or_cdss
Test: test_tool_000002 (functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/peterjc/get_orfs_or_cdss/get_orfs_or_cdss/0.0.5)
Tool id: get_orfs_or_cdss
Tool version: get_orfs_or_cdss
Test: test_tool_000003 (functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/peterjc/get_orfs_or_cdss/get_orfs_or_cdss/0.0.5)
Tests that failed
Tool id: get_orfs_or_cdss
Tool version: get_orfs_or_cdss
Test: test_tool_000001 (functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/peterjc/get_orfs_or_cdss/get_orfs_or_cdss/0.0.5)
Stderr: 
Traceback: 
Traceback (most recent call last):
  File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 101, in test_tool
    self.do_it( td )
  File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 31, in do_it
    self.__verify_outputs( testdef, test_history, shed_tool_id, data_list, galaxy_interactor )
  File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 64, in __verify_outputs
    galaxy_interactor.verify_output( history, output_data, outfile, attributes=attributes, shed_tool_id=shed_tool_id, maxseconds=maxseconds )
  File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py", line 303, in verify_output
    self.twill_test_case.verify_dataset_correctness( outfile, hid=hid, attributes=attributes, shed_tool_id=shed_tool_id, maxseconds=maxseconds )
  File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/base/twilltestcase.py", line 814, in verify_dataset_correctness
    self.wait( maxseconds=maxseconds )  # wait for job to finish
  File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/base/twilltestcase.py", line 1425, in wait
    return self.wait_for(lambda: self.get_running_datasets(), **kwds)
  File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/base/twilltestcase.py", line 1440, in wait_for
    assert slept < maxseconds
AssertionError: 
2013-11-28 09:24:20
Automated test environment
Time tested: 2013-11-28 09:24:20
System: Linux 3.8.0-30-generic
Architecture: x86_64
Python version: 2.7.4
Galaxy revision: 11548:7756b29bea29
Galaxy database version: 117
Tool shed revision: 11544:8b613225f1de
Tool shed database version: 21
Tool shed mercurial version: 2.2.3
Tests that passed successfully
Tool id: get_orfs_or_cdss
Tool version: get_orfs_or_cdss
Test: test_tool_000000 (functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/peterjc/get_orfs_or_cdss/get_orfs_or_cdss/0.0.5)
Tool id: get_orfs_or_cdss
Tool version: get_orfs_or_cdss
Test: test_tool_000001 (functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/peterjc/get_orfs_or_cdss/get_orfs_or_cdss/0.0.5)
Tool id: get_orfs_or_cdss
Tool version: get_orfs_or_cdss
Test: test_tool_000002 (functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/peterjc/get_orfs_or_cdss/get_orfs_or_cdss/0.0.5)
Tool id: get_orfs_or_cdss
Tool version: get_orfs_or_cdss
Test: test_tool_000003 (functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/peterjc/get_orfs_or_cdss/get_orfs_or_cdss/0.0.5)
unknown_2
Automated test environment
Time tested: unknown_2
System: Linux 3.8.0-30-generic
Architecture: x86_64
Python version: 2.7.4
Galaxy revision: 11526:c0384bad246d
Galaxy database version: 117
Tool shed revision:
Tool shed database version:
Tool shed mercurial version:
Tests that passed successfully
Tool id: get_orfs_or_cdss
Tool version: get_orfs_or_cdss
Test: test_tool_000000 (functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/peterjc/get_orfs_or_cdss/get_orfs_or_cdss/0.0.5)
Tool id: get_orfs_or_cdss
Tool version: get_orfs_or_cdss
Test: test_tool_000001 (functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/peterjc/get_orfs_or_cdss/get_orfs_or_cdss/0.0.5)
Tool id: get_orfs_or_cdss
Tool version: get_orfs_or_cdss
Test: test_tool_000002 (functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/peterjc/get_orfs_or_cdss/get_orfs_or_cdss/0.0.5)
Tests that failed
Tool id: get_orfs_or_cdss
Tool version: get_orfs_or_cdss
Test: test_tool_000003 (functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/peterjc/get_orfs_or_cdss/get_orfs_or_cdss/0.0.5)
Stderr: 
Traceback: 
Traceback (most recent call last):
  File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 101, in test_tool
    self.do_it( td )
  File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 31, in do_it
    self.__verify_outputs( testdef, test_history, shed_tool_id, data_list, galaxy_interactor )
  File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/functional/test_toolbox.py", line 64, in __verify_outputs
    galaxy_interactor.verify_output( history, output_data, outfile, attributes=attributes, shed_tool_id=shed_tool_id, maxseconds=maxseconds )
  File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/base/interactor.py", line 303, in verify_output
    self.twill_test_case.verify_dataset_correctness( outfile, hid=hid, attributes=attributes, shed_tool_id=shed_tool_id, maxseconds=maxseconds )
  File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/base/twilltestcase.py", line 814, in verify_dataset_correctness
    self.wait( maxseconds=maxseconds )  # wait for job to finish
  File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/base/twilltestcase.py", line 1425, in wait
    return self.wait_for(lambda: self.get_running_datasets(), **kwds)
  File "/var/opt/buildslaves/buildslave-ec2-2/buildbot-install-test-test-tool-shed-py27/build/test/base/twilltestcase.py", line 1440, in wait_for
    assert slept < maxseconds
AssertionError



Thanks,

Peter

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/