Hello Peter C. Thanks for your comments and advice Ironically after I sent that email the tests for the next tool I looked at in the toolshed had been run in the past couple of days. I'm interested because I've got a couple of tools that seem to work okay when I run the tests locally, but are reported as failing on the toolshed, and I wanted to see if I'd addressed the issue for one of them by updating the tool dependencies. (As an aside I'd also assumed that passing tests is one of the conditions for the tool to be marked as "verified", but maybe that's not the case?) However as you say, more testing locally seems to be a good way to go - thanks for your suggestions. I looked at planemo briefly a while ago and it looked good. The other issue I've had with testing is actually testing tool installation (i.e. tool_dependencies.xml) - I recall that planemo didn't deal with that side of things, so I had to set up the environment for the tests manually. Thanks again for your advice and the links, I will investigate trying to set up a local solution (including some CI testing!). Best wishes Peter On 17/03/15 13:40, Peter Cock wrote:
On Tue, Mar 17, 2015 at 1:20 PM, Peter Briggs <peter.briggs@manchester.ac.uk> wrote:
Hello
Apologies as this seems to be quite a frequent question (and I may have missed something) however: are the tool tests being executed on the main or test toolsheds at the moment?
The most recent tests results that I can find for any of the tools I've uploaded is the 29th January. I've also looked at a couple of tools from other users and the situation appears to be the same (at least in those cases).
Thanks for your help!
Best wishes
Peter
Hi Peter B.,
You're probably seeing some of my emails on this topic - there used to be a nightly test run, then dropped to every second night, but that Galaxy team have been having trouble with the system.
On a positive note, at least some tools are getting tested. This is one I updated earlier this month on 2015-03-06, but didn't include all the named test files - so the test failed:
https://toolshed.g2.bx.psu.edu/view/peterjc/sample_seqs Automated test environment Time tested: 2015-03-16 15:15:33 ... One or more test files are missing for tool sample_seqs: MID4_GLZRM4E04_rnd30.sff
I updated this on the test tool shed on 2015-03-16 but it has yet to be tested there:
https://testtoolshed.g2.bx.psu.edu/view/peterjc/sample_seqs
For now I would urge you to include as much local testing as you can, and explore setting up another testing service like TravisCI or Jenkins for your tool development. e.g.
http://blastedbio.blogspot.co.uk/2013/09/using-travis-ci-for-testing-galaxy-...
Note that John's plameno tool looks set to become extremely useful for both running the tests, and other sanity checking with its "lint" command (and I plan to use this for my TravisCI test setup one day):
https://github.com/galaxyproject/planemo
Peter C.
-- Peter Briggs peter.briggs@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482