Hi,
I have investigated the testing possibilities within a Galaxy local
instance and within a toolshed. I met some difficulties and I have 4
questions listed below.
1- Tests on tools installed in a local Galaxy instance (with
run_tests.sh) : It seems that it is only possible to test the tools
that have been manually installed (run_test.sh -id ...), not the
tools that have been installed through a toolshed. Is that true ?
When I try to launch the tests of a toolshed installed tool, I have
the following error message :
#########
Failure: ValueError (No such test TestForTool_sartools_edger) ...
ERROR
======================================================================
ERROR: Failure: ValueError (No such test
TestForTool_sartools_edger)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/projet/htdocs/galaxy-dev/galaxy-dist/eggs/nose-0.11.1-py2.7.egg/nose/failure.py",
line 39, in runTest
raise self.exc_class(self.exc_val)
ValueError: No such test TestForTool_sartools_edger
----------------------------------------------------------------------
Ran 1 test in 0.002s
FAILED (errors=1)
#########
2- Tests on tools installed in a local Galaxy instance (with
run_tests.sh) : Is it possible to test tools which have repository
or tool dependencies ? I have a tool (manually installed in
galaxy-dist/tools/) with a dependence to a datatype installed from a
toolshed. The datatype (no_unzip.zip) is defined in the
<param> input tag (in <test>) with attribute "ftype".
When running run_tests.sh, it seems that the dataype is not known :
#########
======================================================================
ERROR: test_tool_000001
(functional.test_toolbox.TestForTool_sartools_edger)
SARTools edgeR ( sartools_edger ) > Test-2
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/w/galaxy/galaxy4loraine/galaxy-dist/test/functional/test_toolbox.py",
line 268, in test_tool
self.do_it( td )
File
"/w/galaxy/galaxy4loraine/galaxy-dist/test/functional/test_toolbox.py",
line 37, in do_it
stage_data_in_history( galaxy_interactor, testdef.test_data(),
test_history, shed_tool_id )
File
"/w/galaxy/galaxy4loraine/galaxy-dist/test/base/interactor.py", line
36, in stage_data_in_history
upload_waits.append( galaxy_interactor.stage_data_async(
test_data, history, shed_tool_id ) )
File
"/w/galaxy/galaxy4loraine/galaxy-dist/test/base/interactor.py", line
182, in stage_data_async
raise Exception(submit_response)
Exception: {u'message': {u'type': u'error', u'data': {u'file_type':
u"An invalid option was selected for file_type, u'no_unzip.zip',
please verify.", u'files_metadata': [u"An invalid option was
selected for file_type, u'no_unzip.zip', please verify."]}}}
#########
3- Toolshed Install and Test framework (
install_and_test_tool_shed_repositories.sh)
: For my tool, the test fails if I don't explicitely defined the
dbkey attribute to "unknown" (dbkey="?") in the <param> input
tag (in <test>). The default dbkey value seems to be "hg17"
(whereas hg17 is not defined on the server running my toolshed). So
I have the following error message if I don't add dbkey="?" in
<test> :
#########
======================================================================
ERROR: test_tool_000001
(functional.test_toolbox.TestForTool_galaxy4loraine:9009/repos/lgueguen/sartools_1_0_2/sartools_edger/20150209)
SARTools edgeR (
galaxy4loraine:9009/repos/lgueguen/sartools_1_0_2/sartools_edger/20150209
) > Test-2
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/w/galaxy/galaxy4loraine/galaxy-dist/test/functional/test_toolbox.py",
line 268, in test_tool
self.do_it( td )
File
"/w/galaxy/galaxy4loraine/galaxy-dist/test/functional/test_toolbox.py",
line 37, in do_it
stage_data_in_history( galaxy_interactor, testdef.test_data(),
test_history, shed_tool_id )
File
"/w/galaxy/galaxy4loraine/galaxy-dist/test/base/interactor.py", line
36, in stage_data_in_history
upload_waits.append( galaxy_interactor.stage_data_async(
test_data, history, shed_tool_id ) )
File
"/w/galaxy/galaxy4loraine/galaxy-dist/test/base/interactor.py", line
182, in stage_data_async
raise Exception(submit_response)
Exception: {u'message': {u'type': u'error', u'data': {u'dbkey': u"An
invalid option was selected for dbkey, u'hg17', please verify."}}}
#########
4- Is it possible to launch the tests against an external server ?
That would be very useful to be able run the tests on a production
Galaxy instance in order to test the tools in real condition, within
the whole environment.
Thanks in advance for your answers.
Loraine
--
Loraine Guéguen
Plateforme ABiMS (Analyses and Bioinformatics for Marine Sciences)
Service Informatique et Bioinformatique
Station Biologique de Roscoff
FR 2424 CNRS UPMC
Place Georges Teissier
CS 90074
29688 Roscoff cedex
tel: ++33 (0)2 98 29 56 46 (interne : 415)
http://abims.sb-roscoff.fr/
http://www.sb-roscoff.fr/