2:06 p.m.
Hi all,
I'm working on a FASTA filter script, as per this email:
http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-November/003819.html
Current code here:
http://bitbucket.org/peterjc/galaxy-central/src/c3ac6d7a02f7/
I've added a basic test, but it won't run (see output below). I think
the interesting bit of the error output is this exception: cannot find
value/label "blastp_four_human_vs_rhodopsin.tabular" in list control,
coming from function set_form_control_value in the twill.utils module.
This is something to do with the columns parameter, a multiselect
option referencing the columns in this tabular file. Removing the
validator makes no difference. However, if I change the columns
parameter to a plain text parameter, the test passes:
$ hg diff
diff -r c3ac6d7a02f7 tools/fasta_tools/fasta_filter_by_id.xml
--- a/tools/fasta_tools/fasta_filter_by_id.xml Tue Nov 23 11:37:35 2010 +0000
+++ b/tools/fasta_tools/fasta_filter_by_id.xml Tue Nov 23 14:03:31 2010 +0000
@@ -4,9 +4,7 @@
<inputs>
<param name="input_fasta" type="data" format="fasta"
label="FASTA
file to filter on the identifiers"/>
<param name="input_tabular" type="data"
format="tabular"
label="Tabular file containing FASTA identifiers"/>
- <param name="columns" type="data_column"
data_ref="input_tabular"
multiple="True" numerical="False" label="Column(s) containing
FASTA
identifiers" help="Multi-select list - hold the appropriate key while
clicking to select multiple columns">
- <validator type="no_options" message="Pick at least one
column"/>
- </param>
+ <param name="columns" type="text" label="Column(s)
containing FASTA
identifiers" />
</inputs>
<outputs>
<data name="output_pos" format="fasta" label="With matched
ID" />
I've tried searching the provided wrappers for similar examples (column
multi-select from a tabular file). I found filters/uniq.xml (works but uses
a bed file rather than a simple tabular file) and stats/cor.xml (needs R
and rpy which I don't have installed yet) which have tests, and finally
plotting/bar_chart.xml which has no unit tests.
Would someone familiar with the internals of the Galaxy tests and
how they set tool parameters be able to try reproducing this for me?
The branch has all the unit test files required, and there are no new
dependencies needed.
Thank you,
Peter
--
Here is the output (on Linux - other tests tried pass):
$ ./run_functional_tests.sh -id fasta_filter_by_id
...
nose.plugins.manager DEBUG 2010-11-23 13:59:32,958
DefaultPluginManager load plugin sqlalchemy =
sqlalchemy.test.noseplugin:NoseSQLAlchemy
nose.plugins.manager DEBUG 2010-11-23 13:59:32,961
DefaultPluginManager load plugin nosetestdiff =
nosetestdiff.plugin:NoseTestDiff
nose.plugins.manager DEBUG 2010-11-23 13:59:32,962
DefaultPluginManager load plugin nosehtml = nosehtml.plugin:NoseHTML
Filter sequences by ID ( fasta_filter_by_id ) > Test-1 ...
galaxy.tools.actions.upload_common INFO 2010-11-23 13:59:34,905 tool
upload1 created job id 1
galaxy.jobs DEBUG 2010-11-23 13:59:42,709 dispatching job 1 to local runner
galaxy.jobs INFO 2010-11-23 13:59:42,899 job 1 dispatched
galaxy.jobs.runners.local DEBUG 2010-11-23 13:59:43,213 executing:
python /home/pjcock/repositories/galaxy-central/tools/data_source/upload.py
/home/pjcock/repositories/galaxy-central
/home/pjcock/repositories/galaxy-central/datatypes_conf.xml
/tmp/tmpOVlriT
1:/home/pjcock/repositories/galaxy-central/database/job_working_directory/1/dataset_1_files:/tmp/tmpu4uu_o/database/files/000/dataset_1.dat
galaxy.jobs.runners.local DEBUG 2010-11-23 13:59:44,033 execution
finished: python
/home/pjcock/repositories/galaxy-central/tools/data_source/upload.py
/home/pjcock/repositories/galaxy-central
/home/pjcock/repositories/galaxy-central/datatypes_conf.xml
/tmp/tmpOVlriT
1:/home/pjcock/repositories/galaxy-central/database/job_working_directory/1/dataset_1_files:/tmp/tmpu4uu_o/database/files/000/dataset_1.dat
galaxy.jobs DEBUG 2010-11-23 13:59:44,414 job 1 ended
galaxy.tools.actions.upload_common INFO 2010-11-23 13:59:49,927 tool
upload1 created job id 2
galaxy.jobs DEBUG 2010-11-23 13:59:50,067 dispatching job 2 to local runner
galaxy.jobs INFO 2010-11-23 13:59:50,702 job 2 dispatched
galaxy.jobs.runners.local DEBUG 2010-11-23 13:59:52,059 executing:
python /home/pjcock/repositories/galaxy-central/tools/data_source/upload.py
/home/pjcock/repositories/galaxy-central
/home/pjcock/repositories/galaxy-central/datatypes_conf.xml
/tmp/tmp6Z76yl
2:/home/pjcock/repositories/galaxy-central/database/job_working_directory/2/dataset_2_files:/tmp/tmpu4uu_o/database/files/000/dataset_2.dat
galaxy.jobs.runners.local DEBUG 2010-11-23 13:59:52,874 execution
finished: python
/home/pjcock/repositories/galaxy-central/tools/data_source/upload.py
/home/pjcock/repositories/galaxy-central
/home/pjcock/repositories/galaxy-central/datatypes_conf.xml
/tmp/tmp6Z76yl
2:/home/pjcock/repositories/galaxy-central/database/job_working_directory/2/dataset_2_files:/tmp/tmpu4uu_o/database/files/000/dataset_2.dat
galaxy.jobs DEBUG 2010-11-23 13:59:53,420 job 2 ended
base.twilltestcase DEBUG 2010-11-23 13:59:53,747 In submit_form,
continuing, but caught exception: cannot find value/label
"blastp_four_human_vs_rhodopsin.tabular" in list control
base.twilltestcase DEBUG 2010-11-23 13:59:54,488 ## files diff on
/home/pjcock/repositories/galaxy-central/test-data/four_human_proteins_filter_a.fasta
and /tmp/tmpijWxYGfour_human_proteins_filter_a.fasta lines_diff=0,
found diff = 61
FAIL
======================================================================
FAIL: Filter sequences by ID ( fasta_filter_by_id ) > Test-1
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/home/pjcock/repositories/galaxy-central/test/functional/test_toolbox.py",
line 160, in test_tool
self.do_it( td )
File
"/home/pjcock/repositories/galaxy-central/test/functional/test_toolbox.py",
line 101, in do_it
self.verify_dataset_correctness( outfile, hid=elem_hid,
maxseconds=testdef.maxseconds, attributes=attributes )
File "/home/pjcock/repositories/galaxy-central/test/base/twilltestcase.py",
line 656, in verify_dataset_correctness
raise AssertionError( errmsg )
AssertionError: History item 1 different than expected, difference (using diff):
--- local_file
+++ history_data
@@ -1,2 +1,61 @@
+>sp|Q9BS26|ERP44_HUMAN Endoplasmic reticulum resident protein 44
OS=Homo sapiens GN=ERP44 PE=1 SV=1
+MHPAVFLSLPDLRCSLLLLVTWVFTPVTTEITSLDTENIDEILNNADVALVNFYADWCRF
+SQMLHPIFEEASDVIKEEFPNENQVVFARVDCDQHSDIAQRYRISKYPTLKLFRNGMMMK
+REYRGQRSVKALADYIRQQKSDPIQEIRDLAEITTLDRSKRNIIGYFEQKDSDNYRVFER
+VANILHDDCAFLSAFGDVSKPERYSGDNIIYKPPGHSAPDMVYLGAMTNFDVTYNWIQDK
+CVPLVREITFENGEELTEEGLPFLILFHMKEDTESLEIFQNEVARQLISEKGTINFLHAD
+CDKFRHPLLHIQKTPADCPVIAIDSFRHMYVFGDFKDVLIPGKLKQFVFDLHSGKLHREF
+HHGPDPTDTAPGEQAQDVASSPPESSFQKLAPSEYRYTLLRDRDEL
+>sp|Q9NSY1|BMP2K_HUMAN BMP-2-inducible protein kinase OS=Homo sapiens
GN=BMP2K PE=1 SV=2
+MKKFSRMPKSEGGSGGGAAGGGAGGAGAGAGCGSGGSSVGVRVFAVGRHQVTLEESLAEG
+GFSTVFLVRTHGGIRCALKRMYVNNMPDLNVCKREITIMKELSGHKNIVGYLDCAVNSIS
+DNVWEVLILMEYCRAGQVVNQMNKKLQTGFTEPEVLQIFCDTCEAVARLHQCKTPIIHRD
+LKVENILLNDGGNYVLCDFGSATNKFLNPQKDGVNVVEEEIKKYTTLSYRAPEMINLYGG
+KPITTKADIWALGCLLYKLCFFTLPFGESQVAICDGNFTIPDNSRYSRNIHCLIRFMLEP
+DPEHRPDIFQVSYFAFKFAKKDCPVSNINNSSIPSALPEPMTASEAAARKSQIKARITDT
+IGPTETSIAPRQRPKANSATTATPSVLTIQSSATPVKVLAPGEFGNHRPKGALRPGNGPE
+ILLGQGPPQQPPQQHRVLQQLQQGDWRLQQLHLQHRHPHQQQQQQQQQQQQQQQQQQQQQ
+QQQQQQHHHHHHHHLLQDAYMQQYQHATQQQQMLQQQFLMHSVYQPQPSASQYPTMMPQY
+QQAFFQQQMLAQHQPSQQQASPEYLTSPQEFSPALVSYTSSLPAQVGTIMDSSYSANRSV
+ADKEAIANFTNQKNISNPPDMSGWNPFGEDNFSKLTEEELLDREFDLLRSNRLEERASSD
+KNVDSLSAPHNHPPEDPFGSVPFISHSGSPEKKAEHSSINQENGTANPIKNGKTSPASKD
+QRTGKKTSVQGQVQKGNDESESDFESDPPSPKSSEEEEQDDEEVLQGEQGDFNDDDTEPE
+NLGHRPLLMDSEDEEEEEKHSSDSDYEQAKAKYSDMSSVYRDRSGSGPTQDLNTILLTSA
+QLSSDVAVETPKQEFDVFGAVPFFAVRAQQPQQEKNEKNLPQHRFPAAGLEQEEFDVFTK
+APFSKKVNVQECHAVGPEAHTIPGYPKSVDVFGSTPFQPFLTSTSKSESNEDLFGLVPFD
+EITGSQQQKVKQRSLQKLSSRQRRTKQDMSKSNGKRHHGTPTSTKKTLKPTYRTPERARR
+HKKVGRRDSQSSNEFLTISDSKENISVALTDGKDRGNVLQPEESLLDPFGAKPFHSPDLS
+WHPPHQGLSDIRADHNTVLPGRPRQNSLHGSFHSADVLKMDDFGAVPFTELVVQSITPHQ
+SQQSQPVELDPFGAAPFPSKQ
+>sp|P06213|INSR_HUMAN Insulin receptor OS=Homo sapiens GN=INSR PE=1 SV=4
+MATGGRRGAAAAPLLVAVAALLLGAAGHLYPGEVCPGMDIRNNLTRLHELENCSVIEGHL
+QILLMFKTRPEDFRDLSFPKLIMITDYLLLFRVYGLESLKDLFPNLTVIRGSRLFFNYAL
+VIFEMVHLKELGLYNLMNITRGSVRIEKNNELCYLATIDWSRILDSVEDNYIVLNKDDNE
+ECGDICPGTAKGKTNCPATVINGQFVERCWTHSHCQKVCPTICKSHGCTAEGLCCHSECL
+GNCSQPDDPTKCVACRNFYLDGRCVETCPPPYYHFQDWRCVNFSFCQDLHHKCKNSRRQG
+CHQYVIHNNKCIPECPSGYTMNSSNLLCTPCLGPCPKVCHLLEGEKTIDSVTSAQELRGC
+TVINGSLIINIRGGNNLAAELEANLGLIEEISGYLKIRRSYALVSLSFFRKLRLIRGETL
-------------------- >> begin captured stdout << ---------------------
Uploaded file: four_human_proteins.fasta , ftype: fasta , extra:
{'ftype': 'fasta', 'value': 'four_human_proteins.fasta',
'children':
[]}
Uploaded file: blastp_four_human_vs_rhodopsin.tabular , ftype:
tabular , extra: {'ftype': 'tabular', 'value':
'blastp_four_human_vs_rhodopsin.tabular', 'children': []}
form 'tool_form' contains the following controls ( note the values )
control 0: <HiddenControl(tool_id=fasta_filter_by_id) (readonly)>
control 1:
<HiddenControl(tool_state=800255c1626362313139396662383639346664316133316565303361646336323163663764636566333538623a376232323566356637303631363736353566356632323361323033303263323032323639366537303735373435663636363137333734363132323361323032323331323232633230323236333666366337353664366537333232336132303232366537353663366332323263323032323639366537303735373435663734363136323735366336313732323233613230323233323232376471002e)
(readonly)>
control 2: <SelectControl(input_fasta=[*1])>
control 3: <SelectControl(input_tabular=[*2])>
control 4: <SelectControl(columns=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12])>
control 5: <SubmitControl(runtool_btn=Execute) (readonly)>
page_inputs (0) {'input_fasta': ['four_human_proteins.fasta'],
'columns': ['1'], 'input_tabular':
['blastp_four_human_vs_rhodopsin.tabular']}
--------------------- >> end captured stdout << ----------------------
-------------------- >> begin captured logging << --------------------
galaxy.tools.actions.upload_common: INFO: tool upload1 created job id 1
galaxy.jobs: DEBUG: dispatching job 1 to local runner
galaxy.jobs: INFO: job 1 dispatched
galaxy.jobs.runners.local: DEBUG: executing: python
/home/pjcock/repositories/galaxy-central/tools/data_source/upload.py
/home/pjcock/repositories/galaxy-central
/home/pjcock/repositories/galaxy-central/datatypes_conf.xml
/tmp/tmpOVlriT
1:/home/pjcock/repositories/galaxy-central/database/job_working_directory/1/dataset_1_files:/tmp/tmpu4uu_o/database/files/000/dataset_1.dat
galaxy.jobs.runners.local: DEBUG: execution finished: python
/home/pjcock/repositories/galaxy-central/tools/data_source/upload.py
/home/pjcock/repositories/galaxy-central
/home/pjcock/repositories/galaxy-central/datatypes_conf.xml
/tmp/tmpOVlriT
1:/home/pjcock/repositories/galaxy-central/database/job_working_directory/1/dataset_1_files:/tmp/tmpu4uu_o/database/files/000/dataset_1.dat
galaxy.jobs: DEBUG: job 1 ended
galaxy.tools.actions.upload_common: INFO: tool upload1 created job id 2
galaxy.jobs: DEBUG: dispatching job 2 to local runner
galaxy.jobs: INFO: job 2 dispatched
galaxy.jobs.runners.local: DEBUG: executing: python
/home/pjcock/repositories/galaxy-central/tools/data_source/upload.py
/home/pjcock/repositories/galaxy-central
/home/pjcock/repositories/galaxy-central/datatypes_conf.xml
/tmp/tmp6Z76yl
2:/home/pjcock/repositories/galaxy-central/database/job_working_directory/2/dataset_2_files:/tmp/tmpu4uu_o/database/files/000/dataset_2.dat
galaxy.jobs.runners.local: DEBUG: execution finished: python
/home/pjcock/repositories/galaxy-central/tools/data_source/upload.py
/home/pjcock/repositories/galaxy-central
/home/pjcock/repositories/galaxy-central/datatypes_conf.xml
/tmp/tmp6Z76yl
2:/home/pjcock/repositories/galaxy-central/database/job_working_directory/2/dataset_2_files:/tmp/tmpu4uu_o/database/files/000/dataset_2.dat
galaxy.jobs: DEBUG: job 2 ended
base.twilltestcase: DEBUG: In submit_form, continuing, but caught
exception: cannot find value/label
"blastp_four_human_vs_rhodopsin.tabular" in list control
base.twilltestcase: DEBUG: ## files diff on
/home/pjcock/repositories/galaxy-central/test-data/four_human_proteins_filter_a.fasta
and /tmp/tmpijWxYGfour_human_proteins_filter_a.fasta lines_diff=0,
found diff = 61
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 1 test in 21.504s
FAILED (failures=1)
functional_tests.py INFO 2010-11-23 13:59:54,494 Shutting down
functional_tests.py INFO 2010-11-23 13:59:54,494 Shutting down
embedded web server
functional_tests.py INFO 2010-11-23 13:59:54,497 Embedded web server stopped
functional_tests.py INFO 2010-11-23 13:59:54,497 Shutting down app
galaxy.jobs INFO 2010-11-23 13:59:54,497 sending stop signal to worker thread
galaxy.jobs INFO 2010-11-23 13:59:54,498 job queue stopped
galaxy.jobs.runners.local INFO 2010-11-23 13:59:54,498 sending stop
signal to worker threads
galaxy.jobs.runners.local INFO 2010-11-23 13:59:54,498 local job runner stopped
galaxy.jobs INFO 2010-11-23 13:59:54,499 sending stop signal to worker thread
galaxy.jobs INFO 2010-11-23 13:59:54,499 job stopper stopped
functional_tests.py INFO 2010-11-23 13:59:54,499 Embedded Universe
application stopped
functional_tests.py INFO 2010-11-23 13:59:54,500 Cleaning up temporary
files in /tmp/tmpu4uu_o
galaxy.jobs INFO 2010-11-23 13:59:55,410 sending stop signal to worker thread
galaxy.jobs INFO 2010-11-23 13:59:55,411 job queue stopped
galaxy.jobs.runners.local INFO 2010-11-23 13:59:55,411 sending stop
signal to worker threads
galaxy.jobs.runners.local INFO 2010-11-23 13:59:55,411 local job runner stopped
galaxy.jobs INFO 2010-11-23 13:59:55,411 sending stop signal to worker thread
galaxy.jobs INFO 2010-11-23 13:59:55,412 job stopper stopped
'run_functional_tests.sh help' for help
Note: galaxy test framework uses tool_conf.xml.sample, not tool_conf.xml