Puzzling test failure with data_column parameter
Hi all, I'm working on a FASTA filter script, as per this email: http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-November/003819.html Current code here: http://bitbucket.org/peterjc/galaxy-central/src/c3ac6d7a02f7/ I've added a basic test, but it won't run (see output below). I think the interesting bit of the error output is this exception: cannot find value/label "blastp_four_human_vs_rhodopsin.tabular" in list control, coming from function set_form_control_value in the twill.utils module. This is something to do with the columns parameter, a multiselect option referencing the columns in this tabular file. Removing the validator makes no difference. However, if I change the columns parameter to a plain text parameter, the test passes: $ hg diff diff -r c3ac6d7a02f7 tools/fasta_tools/fasta_filter_by_id.xml --- a/tools/fasta_tools/fasta_filter_by_id.xml Tue Nov 23 11:37:35 2010 +0000 +++ b/tools/fasta_tools/fasta_filter_by_id.xml Tue Nov 23 14:03:31 2010 +0000 @@ -4,9 +4,7 @@ <inputs> <param name="input_fasta" type="data" format="fasta" label="FASTA file to filter on the identifiers"/> <param name="input_tabular" type="data" format="tabular" label="Tabular file containing FASTA identifiers"/> - <param name="columns" type="data_column" data_ref="input_tabular" multiple="True" numerical="False" label="Column(s) containing FASTA identifiers" help="Multi-select list - hold the appropriate key while clicking to select multiple columns"> - <validator type="no_options" message="Pick at least one column"/> - </param> + <param name="columns" type="text" label="Column(s) containing FASTA identifiers" /> </inputs> <outputs> <data name="output_pos" format="fasta" label="With matched ID" /> I've tried searching the provided wrappers for similar examples (column multi-select from a tabular file). I found filters/uniq.xml (works but uses a bed file rather than a simple tabular file) and stats/cor.xml (needs R and rpy which I don't have installed yet) which have tests, and finally plotting/bar_chart.xml which has no unit tests. Would someone familiar with the internals of the Galaxy tests and how they set tool parameters be able to try reproducing this for me? The branch has all the unit test files required, and there are no new dependencies needed. Thank you, Peter -- Here is the output (on Linux - other tests tried pass): $ ./run_functional_tests.sh -id fasta_filter_by_id ... nose.plugins.manager DEBUG 2010-11-23 13:59:32,958 DefaultPluginManager load plugin sqlalchemy = sqlalchemy.test.noseplugin:NoseSQLAlchemy nose.plugins.manager DEBUG 2010-11-23 13:59:32,961 DefaultPluginManager load plugin nosetestdiff = nosetestdiff.plugin:NoseTestDiff nose.plugins.manager DEBUG 2010-11-23 13:59:32,962 DefaultPluginManager load plugin nosehtml = nosehtml.plugin:NoseHTML Filter sequences by ID ( fasta_filter_by_id ) > Test-1 ... galaxy.tools.actions.upload_common INFO 2010-11-23 13:59:34,905 tool upload1 created job id 1 galaxy.jobs DEBUG 2010-11-23 13:59:42,709 dispatching job 1 to local runner galaxy.jobs INFO 2010-11-23 13:59:42,899 job 1 dispatched galaxy.jobs.runners.local DEBUG 2010-11-23 13:59:43,213 executing: python /home/pjcock/repositories/galaxy-central/tools/data_source/upload.py /home/pjcock/repositories/galaxy-central /home/pjcock/repositories/galaxy-central/datatypes_conf.xml /tmp/tmpOVlriT 1:/home/pjcock/repositories/galaxy-central/database/job_working_directory/1/dataset_1_files:/tmp/tmpu4uu_o/database/files/000/dataset_1.dat galaxy.jobs.runners.local DEBUG 2010-11-23 13:59:44,033 execution finished: python /home/pjcock/repositories/galaxy-central/tools/data_source/upload.py /home/pjcock/repositories/galaxy-central /home/pjcock/repositories/galaxy-central/datatypes_conf.xml /tmp/tmpOVlriT 1:/home/pjcock/repositories/galaxy-central/database/job_working_directory/1/dataset_1_files:/tmp/tmpu4uu_o/database/files/000/dataset_1.dat galaxy.jobs DEBUG 2010-11-23 13:59:44,414 job 1 ended galaxy.tools.actions.upload_common INFO 2010-11-23 13:59:49,927 tool upload1 created job id 2 galaxy.jobs DEBUG 2010-11-23 13:59:50,067 dispatching job 2 to local runner galaxy.jobs INFO 2010-11-23 13:59:50,702 job 2 dispatched galaxy.jobs.runners.local DEBUG 2010-11-23 13:59:52,059 executing: python /home/pjcock/repositories/galaxy-central/tools/data_source/upload.py /home/pjcock/repositories/galaxy-central /home/pjcock/repositories/galaxy-central/datatypes_conf.xml /tmp/tmp6Z76yl 2:/home/pjcock/repositories/galaxy-central/database/job_working_directory/2/dataset_2_files:/tmp/tmpu4uu_o/database/files/000/dataset_2.dat galaxy.jobs.runners.local DEBUG 2010-11-23 13:59:52,874 execution finished: python /home/pjcock/repositories/galaxy-central/tools/data_source/upload.py /home/pjcock/repositories/galaxy-central /home/pjcock/repositories/galaxy-central/datatypes_conf.xml /tmp/tmp6Z76yl 2:/home/pjcock/repositories/galaxy-central/database/job_working_directory/2/dataset_2_files:/tmp/tmpu4uu_o/database/files/000/dataset_2.dat galaxy.jobs DEBUG 2010-11-23 13:59:53,420 job 2 ended base.twilltestcase DEBUG 2010-11-23 13:59:53,747 In submit_form, continuing, but caught exception: cannot find value/label "blastp_four_human_vs_rhodopsin.tabular" in list control base.twilltestcase DEBUG 2010-11-23 13:59:54,488 ## files diff on /home/pjcock/repositories/galaxy-central/test-data/four_human_proteins_filter_a.fasta and /tmp/tmpijWxYGfour_human_proteins_filter_a.fasta lines_diff=0, found diff = 61 FAIL ====================================================================== FAIL: Filter sequences by ID ( fasta_filter_by_id ) > Test-1 ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pjcock/repositories/galaxy-central/test/functional/test_toolbox.py", line 160, in test_tool self.do_it( td ) File "/home/pjcock/repositories/galaxy-central/test/functional/test_toolbox.py", line 101, in do_it self.verify_dataset_correctness( outfile, hid=elem_hid, maxseconds=testdef.maxseconds, attributes=attributes ) File "/home/pjcock/repositories/galaxy-central/test/base/twilltestcase.py", line 656, in verify_dataset_correctness raise AssertionError( errmsg ) AssertionError: History item 1 different than expected, difference (using diff): --- local_file +++ history_data @@ -1,2 +1,61 @@ +>sp|Q9BS26|ERP44_HUMAN Endoplasmic reticulum resident protein 44 OS=Homo sapiens GN=ERP44 PE=1 SV=1 +MHPAVFLSLPDLRCSLLLLVTWVFTPVTTEITSLDTENIDEILNNADVALVNFYADWCRF +SQMLHPIFEEASDVIKEEFPNENQVVFARVDCDQHSDIAQRYRISKYPTLKLFRNGMMMK +REYRGQRSVKALADYIRQQKSDPIQEIRDLAEITTLDRSKRNIIGYFEQKDSDNYRVFER +VANILHDDCAFLSAFGDVSKPERYSGDNIIYKPPGHSAPDMVYLGAMTNFDVTYNWIQDK +CVPLVREITFENGEELTEEGLPFLILFHMKEDTESLEIFQNEVARQLISEKGTINFLHAD +CDKFRHPLLHIQKTPADCPVIAIDSFRHMYVFGDFKDVLIPGKLKQFVFDLHSGKLHREF +HHGPDPTDTAPGEQAQDVASSPPESSFQKLAPSEYRYTLLRDRDEL +>sp|Q9NSY1|BMP2K_HUMAN BMP-2-inducible protein kinase OS=Homo sapiens GN=BMP2K PE=1 SV=2 +MKKFSRMPKSEGGSGGGAAGGGAGGAGAGAGCGSGGSSVGVRVFAVGRHQVTLEESLAEG +GFSTVFLVRTHGGIRCALKRMYVNNMPDLNVCKREITIMKELSGHKNIVGYLDCAVNSIS +DNVWEVLILMEYCRAGQVVNQMNKKLQTGFTEPEVLQIFCDTCEAVARLHQCKTPIIHRD +LKVENILLNDGGNYVLCDFGSATNKFLNPQKDGVNVVEEEIKKYTTLSYRAPEMINLYGG +KPITTKADIWALGCLLYKLCFFTLPFGESQVAICDGNFTIPDNSRYSRNIHCLIRFMLEP +DPEHRPDIFQVSYFAFKFAKKDCPVSNINNSSIPSALPEPMTASEAAARKSQIKARITDT +IGPTETSIAPRQRPKANSATTATPSVLTIQSSATPVKVLAPGEFGNHRPKGALRPGNGPE +ILLGQGPPQQPPQQHRVLQQLQQGDWRLQQLHLQHRHPHQQQQQQQQQQQQQQQQQQQQQ +QQQQQQHHHHHHHHLLQDAYMQQYQHATQQQQMLQQQFLMHSVYQPQPSASQYPTMMPQY +QQAFFQQQMLAQHQPSQQQASPEYLTSPQEFSPALVSYTSSLPAQVGTIMDSSYSANRSV +ADKEAIANFTNQKNISNPPDMSGWNPFGEDNFSKLTEEELLDREFDLLRSNRLEERASSD +KNVDSLSAPHNHPPEDPFGSVPFISHSGSPEKKAEHSSINQENGTANPIKNGKTSPASKD +QRTGKKTSVQGQVQKGNDESESDFESDPPSPKSSEEEEQDDEEVLQGEQGDFNDDDTEPE +NLGHRPLLMDSEDEEEEEKHSSDSDYEQAKAKYSDMSSVYRDRSGSGPTQDLNTILLTSA +QLSSDVAVETPKQEFDVFGAVPFFAVRAQQPQQEKNEKNLPQHRFPAAGLEQEEFDVFTK +APFSKKVNVQECHAVGPEAHTIPGYPKSVDVFGSTPFQPFLTSTSKSESNEDLFGLVPFD +EITGSQQQKVKQRSLQKLSSRQRRTKQDMSKSNGKRHHGTPTSTKKTLKPTYRTPERARR +HKKVGRRDSQSSNEFLTISDSKENISVALTDGKDRGNVLQPEESLLDPFGAKPFHSPDLS +WHPPHQGLSDIRADHNTVLPGRPRQNSLHGSFHSADVLKMDDFGAVPFTELVVQSITPHQ +SQQSQPVELDPFGAAPFPSKQ +>sp|P06213|INSR_HUMAN Insulin receptor OS=Homo sapiens GN=INSR PE=1 SV=4 +MATGGRRGAAAAPLLVAVAALLLGAAGHLYPGEVCPGMDIRNNLTRLHELENCSVIEGHL +QILLMFKTRPEDFRDLSFPKLIMITDYLLLFRVYGLESLKDLFPNLTVIRGSRLFFNYAL +VIFEMVHLKELGLYNLMNITRGSVRIEKNNELCYLATIDWSRILDSVEDNYIVLNKDDNE +ECGDICPGTAKGKTNCPATVINGQFVERCWTHSHCQKVCPTICKSHGCTAEGLCCHSECL +GNCSQPDDPTKCVACRNFYLDGRCVETCPPPYYHFQDWRCVNFSFCQDLHHKCKNSRRQG +CHQYVIHNNKCIPECPSGYTMNSSNLLCTPCLGPCPKVCHLLEGEKTIDSVTSAQELRGC +TVINGSLIINIRGGNNLAAELEANLGLIEEISGYLKIRRSYALVSLSFFRKLRLIRGETL -------------------- >> begin captured stdout << --------------------- Uploaded file: four_human_proteins.fasta , ftype: fasta , extra: {'ftype': 'fasta', 'value': 'four_human_proteins.fasta', 'children': []} Uploaded file: blastp_four_human_vs_rhodopsin.tabular , ftype: tabular , extra: {'ftype': 'tabular', 'value': 'blastp_four_human_vs_rhodopsin.tabular', 'children': []} form 'tool_form' contains the following controls ( note the values ) control 0: <HiddenControl(tool_id=fasta_filter_by_id) (readonly)> control 1: <HiddenControl(tool_state=800255c1626362313139396662383639346664316133316565303361646336323163663764636566333538623a376232323566356637303631363736353566356632323361323033303263323032323639366537303735373435663636363137333734363132323361323032323331323232633230323236333666366337353664366537333232336132303232366537353663366332323263323032323639366537303735373435663734363136323735366336313732323233613230323233323232376471002e) (readonly)> control 2: <SelectControl(input_fasta=[*1])> control 3: <SelectControl(input_tabular=[*2])> control 4: <SelectControl(columns=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12])> control 5: <SubmitControl(runtool_btn=Execute) (readonly)> page_inputs (0) {'input_fasta': ['four_human_proteins.fasta'], 'columns': ['1'], 'input_tabular': ['blastp_four_human_vs_rhodopsin.tabular']} --------------------- >> end captured stdout << ---------------------- -------------------- >> begin captured logging << -------------------- galaxy.tools.actions.upload_common: INFO: tool upload1 created job id 1 galaxy.jobs: DEBUG: dispatching job 1 to local runner galaxy.jobs: INFO: job 1 dispatched galaxy.jobs.runners.local: DEBUG: executing: python /home/pjcock/repositories/galaxy-central/tools/data_source/upload.py /home/pjcock/repositories/galaxy-central /home/pjcock/repositories/galaxy-central/datatypes_conf.xml /tmp/tmpOVlriT 1:/home/pjcock/repositories/galaxy-central/database/job_working_directory/1/dataset_1_files:/tmp/tmpu4uu_o/database/files/000/dataset_1.dat galaxy.jobs.runners.local: DEBUG: execution finished: python /home/pjcock/repositories/galaxy-central/tools/data_source/upload.py /home/pjcock/repositories/galaxy-central /home/pjcock/repositories/galaxy-central/datatypes_conf.xml /tmp/tmpOVlriT 1:/home/pjcock/repositories/galaxy-central/database/job_working_directory/1/dataset_1_files:/tmp/tmpu4uu_o/database/files/000/dataset_1.dat galaxy.jobs: DEBUG: job 1 ended galaxy.tools.actions.upload_common: INFO: tool upload1 created job id 2 galaxy.jobs: DEBUG: dispatching job 2 to local runner galaxy.jobs: INFO: job 2 dispatched galaxy.jobs.runners.local: DEBUG: executing: python /home/pjcock/repositories/galaxy-central/tools/data_source/upload.py /home/pjcock/repositories/galaxy-central /home/pjcock/repositories/galaxy-central/datatypes_conf.xml /tmp/tmp6Z76yl 2:/home/pjcock/repositories/galaxy-central/database/job_working_directory/2/dataset_2_files:/tmp/tmpu4uu_o/database/files/000/dataset_2.dat galaxy.jobs.runners.local: DEBUG: execution finished: python /home/pjcock/repositories/galaxy-central/tools/data_source/upload.py /home/pjcock/repositories/galaxy-central /home/pjcock/repositories/galaxy-central/datatypes_conf.xml /tmp/tmp6Z76yl 2:/home/pjcock/repositories/galaxy-central/database/job_working_directory/2/dataset_2_files:/tmp/tmpu4uu_o/database/files/000/dataset_2.dat galaxy.jobs: DEBUG: job 2 ended base.twilltestcase: DEBUG: In submit_form, continuing, but caught exception: cannot find value/label "blastp_four_human_vs_rhodopsin.tabular" in list control base.twilltestcase: DEBUG: ## files diff on /home/pjcock/repositories/galaxy-central/test-data/four_human_proteins_filter_a.fasta and /tmp/tmpijWxYGfour_human_proteins_filter_a.fasta lines_diff=0, found diff = 61 --------------------- >> end captured logging << --------------------- ---------------------------------------------------------------------- Ran 1 test in 21.504s FAILED (failures=1) functional_tests.py INFO 2010-11-23 13:59:54,494 Shutting down functional_tests.py INFO 2010-11-23 13:59:54,494 Shutting down embedded web server functional_tests.py INFO 2010-11-23 13:59:54,497 Embedded web server stopped functional_tests.py INFO 2010-11-23 13:59:54,497 Shutting down app galaxy.jobs INFO 2010-11-23 13:59:54,497 sending stop signal to worker thread galaxy.jobs INFO 2010-11-23 13:59:54,498 job queue stopped galaxy.jobs.runners.local INFO 2010-11-23 13:59:54,498 sending stop signal to worker threads galaxy.jobs.runners.local INFO 2010-11-23 13:59:54,498 local job runner stopped galaxy.jobs INFO 2010-11-23 13:59:54,499 sending stop signal to worker thread galaxy.jobs INFO 2010-11-23 13:59:54,499 job stopper stopped functional_tests.py INFO 2010-11-23 13:59:54,499 Embedded Universe application stopped functional_tests.py INFO 2010-11-23 13:59:54,500 Cleaning up temporary files in /tmp/tmpu4uu_o galaxy.jobs INFO 2010-11-23 13:59:55,410 sending stop signal to worker thread galaxy.jobs INFO 2010-11-23 13:59:55,411 job queue stopped galaxy.jobs.runners.local INFO 2010-11-23 13:59:55,411 sending stop signal to worker threads galaxy.jobs.runners.local INFO 2010-11-23 13:59:55,411 local job runner stopped galaxy.jobs INFO 2010-11-23 13:59:55,411 sending stop signal to worker thread galaxy.jobs INFO 2010-11-23 13:59:55,412 job stopper stopped 'run_functional_tests.sh help' for help Note: galaxy test framework uses tool_conf.xml.sample, not tool_conf.xml
On Tue, Nov 23, 2010 at 2:06 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
Hi all,
I'm working on a FASTA filter script, as per this email: http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-November/003819.html
Current code here: http://bitbucket.org/peterjc/galaxy-central/src/c3ac6d7a02f7/
I've added a basic test, but it won't run (see output below). I think the interesting bit of the error output is this exception: cannot find value/label "blastp_four_human_vs_rhodopsin.tabular" in list control, coming from function set_form_control_value in the twill.utils module.
This is something to do with the columns parameter, a multiselect option referencing the columns in this tabular file. Removing the validator makes no difference. However, if I change the columns parameter to a plain text parameter, the test passes:
$ hg diff diff -r c3ac6d7a02f7 tools/fasta_tools/fasta_filter_by_id.xml --- a/tools/fasta_tools/fasta_filter_by_id.xml Tue Nov 23 11:37:35 2010 +0000 +++ b/tools/fasta_tools/fasta_filter_by_id.xml Tue Nov 23 14:03:31 2010 +0000 @@ -4,9 +4,7 @@ <inputs> <param name="input_fasta" type="data" format="fasta" label="FASTA file to filter on the identifiers"/> <param name="input_tabular" type="data" format="tabular" label="Tabular file containing FASTA identifiers"/> - <param name="columns" type="data_column" data_ref="input_tabular" multiple="True" numerical="False" label="Column(s) containing FASTA identifiers" help="Multi-select list - hold the appropriate key while clicking to select multiple columns"> - <validator type="no_options" message="Pick at least one column"/> - </param> + <param name="columns" type="text" label="Column(s) containing FASTA identifiers" /> </inputs> <outputs> <data name="output_pos" format="fasta" label="With matched ID" />
I've tried searching the provided wrappers for similar examples (column multi-select from a tabular file). I found filters/uniq.xml (works but uses a bed file rather than a simple tabular file) and stats/cor.xml (needs R and rpy which I don't have installed yet) which have tests, and finally plotting/bar_chart.xml which has no unit tests.
Would someone familiar with the internals of the Galaxy tests and how they set tool parameters be able to try reproducing this for me? The branch has all the unit test files required, and there are no new dependencies needed.
Thank you,
Peter
--
Here is the output (on Linux - other tests tried pass):
$ ./run_functional_tests.sh -id fasta_filter_by_id ... base.twilltestcase DEBUG 2010-11-23 13:59:53,747 In submit_form, continuing, but caught exception: cannot find value/label "blastp_four_human_vs_rhodopsin.tabular" in list control ...
Hello again, See http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-November/003867.html for the original email and full log. The most interesting part of the logging was the caught exception (and the tool being tested was never run): cannot find value/label "blastp_four_human_vs_rhodopsin.tabular" in list control I don't recall if/how this was resolved, I've since deprecated this tool itself - replacing it with a more general version: http://toolshed.g2.bx.psu.edu/view/peterjc/seq_filter_by_id However, I've just hit what seems to be a very similar issue while trying to add what should be a simple unit test to my sequence renaming script, seq_rename.py / seq_rename.py which is in the main Tool Shed here: http://toolshed.g2.bx.psu.edu/view/peterjc/seq_rename Here's the current latest code (not yet pushed to the public Tool Shed), which is now on the test Tool Shed and on my bitbucket branch: http://testtoolshed.g2.bx.psu.edu/view/peterjc/seq_rename https://bitbucket.org/peterjc/galaxy-central/src/7750adf18b1fab953d7cccd0216... Here's the test itself, <tests> <test> <param name="input_file" value="four_human_proteins.fasta" ftype="fasta" /> <param name="input_tabular" value="four_human_proteins.rename.tabular" ftype="tabular" /> <param name="old_column" value="1" /> <param name="new_column" value="2" /> <output name="output_file" file="four_human_proteins.rename.fasta" ftype="fasta" /> </test> </tests> Now I am quite puzzled by what happens running the test, the full output is at the end of this email. Strangely the tool itself is never actually run, and instead Galaxy compares one of the input files to the expected output file and they don't match. As in my email from 2010, this is most interesting line in the output, just after the two input files are 'uploaded' to the current history: base.twilltestcase DEBUG 2013-04-22 18:20:18,061 In submit_form, continuing, but caught exception: cannot find value/label "four_human_proteins.rename.tabular" in list control It would appear that the tabular file has not been added to the potential valid datasets available to the $input_tabular parameter? This test should have been attempted on the Test Tool Shed last night, but there is nothing to show for it - something that has happened in other corner cases - I'll flag this on the other thread. Likely this is due to the exception within the test framework itself. Regards, Peter -- $ ./run_functional_tests.sh -id seq_rename ... functional_tests.py INFO 2013-04-22 18:20:02,192 Functional tests will be run against localhost:9176 nose.plugins.manager DEBUG 2013-04-22 18:20:02,212 DefaultPluginManager load plugin sqlalchemy = sqlalchemy.test.noseplugin:NoseSQLAlchemy nose.plugins.manager DEBUG 2013-04-22 18:20:02,222 DefaultPluginManager load plugin nosetestdiff = nosetestdiff.plugin:NoseTestDiff nose.plugins.manager DEBUG 2013-04-22 18:20:02,224 DefaultPluginManager load plugin nosehtml = nosehtml.plugin:NoseHTML Rename sequences ( seq_rename ) > Test-1 ... galaxy.web.framework DEBUG 2013-04-22 18:20:03,447 This request returned None from get_history(): http://localhost:9176/ galaxy.web.framework DEBUG 2013-04-22 18:20:03,447 This request returned None from get_history(): http://localhost:9176/ galaxy.web.framework DEBUG 2013-04-22 18:20:03,493 This request returned None from get_history(): http://localhost:9176/ galaxy.web.framework DEBUG 2013-04-22 18:20:03,494 This request returned None from get_history(): http://localhost:9176/ galaxy.web.framework DEBUG 2013-04-22 18:20:03,659 This request returned None from get_history(): http://localhost:9176/user/logout galaxy.web.framework DEBUG 2013-04-22 18:20:03,659 This request returned None from get_history(): http://localhost:9176/user/logout galaxy.web.framework DEBUG 2013-04-22 18:20:03,700 This request returned None from get_history(): http://localhost:9176/ galaxy.web.framework DEBUG 2013-04-22 18:20:03,700 This request returned None from get_history(): http://localhost:9176/ galaxy.tools.actions.upload_common INFO 2013-04-22 18:20:06,350 tool upload1 created job id 1 galaxy.jobs DEBUG 2013-04-22 18:20:06,649 (1) Working directory for job is: /mnt/galaxy/galaxy-central/database/job_working_directory/000/1 galaxy.jobs.handler DEBUG 2013-04-22 18:20:06,713 (1) Dispatching to local runner galaxy.jobs DEBUG 2013-04-22 18:20:07,088 (1) Persisting job destination (destination id: local:///) galaxy.jobs.handler INFO 2013-04-22 18:20:07,195 (1) Job dispatched galaxy.jobs.runners.local DEBUG 2013-04-22 18:20:07,825 (1) executing: python /mnt/galaxy/galaxy-central/tools/data_source/upload.py /mnt/galaxy/galaxy-central /tmp/tmpYyosAY/database/tmp/tmpIPAfMX /tmp/tmpYyosAY/database/tmp/tmpd7kJZZ 1:/mnt/galaxy/galaxy-central/database/job_working_directory/000/1/dataset_1_files:/tmp/tmpYyosAY/database/files/000/dataset_1.dat galaxy.jobs DEBUG 2013-04-22 18:20:07,895 (1) Persisting job destination (destination id: local:///) galaxy.jobs.runners.local DEBUG 2013-04-22 18:20:09,254 execution finished: python /mnt/galaxy/galaxy-central/tools/data_source/upload.py /mnt/galaxy/galaxy-central /tmp/tmpYyosAY/database/tmp/tmpIPAfMX /tmp/tmpYyosAY/database/tmp/tmpd7kJZZ 1:/mnt/galaxy/galaxy-central/database/job_working_directory/000/1/dataset_1_files:/tmp/tmpYyosAY/database/files/000/dataset_1.dat galaxy.jobs DEBUG 2013-04-22 18:20:09,327 Tool did not define exit code or stdio handling; checking stderr for success galaxy.jobs DEBUG 2013-04-22 18:20:09,826 job 1 ended galaxy.tools.actions.upload_common INFO 2013-04-22 18:20:11,435 tool upload1 created job id 2 galaxy.jobs DEBUG 2013-04-22 18:20:11,774 (2) Working directory for job is: /mnt/galaxy/galaxy-central/database/job_working_directory/000/2 galaxy.jobs.handler DEBUG 2013-04-22 18:20:11,839 (2) Dispatching to local runner galaxy.jobs DEBUG 2013-04-22 18:20:12,221 (2) Persisting job destination (destination id: local:///) galaxy.jobs.handler INFO 2013-04-22 18:20:12,328 (2) Job dispatched galaxy.jobs.runners.local DEBUG 2013-04-22 18:20:12,893 (2) executing: python /mnt/galaxy/galaxy-central/tools/data_source/upload.py /mnt/galaxy/galaxy-central /tmp/tmpYyosAY/database/tmp/tmpIPAfMX /tmp/tmpYyosAY/database/tmp/tmpeA1apX 2:/mnt/galaxy/galaxy-central/database/job_working_directory/000/2/dataset_2_files:/tmp/tmpYyosAY/database/files/000/dataset_2.dat galaxy.jobs DEBUG 2013-04-22 18:20:12,968 (2) Persisting job destination (destination id: local:///) galaxy.jobs.runners.local DEBUG 2013-04-22 18:20:14,497 execution finished: python /mnt/galaxy/galaxy-central/tools/data_source/upload.py /mnt/galaxy/galaxy-central /tmp/tmpYyosAY/database/tmp/tmpIPAfMX /tmp/tmpYyosAY/database/tmp/tmpeA1apX 2:/mnt/galaxy/galaxy-central/database/job_working_directory/000/2/dataset_2_files:/tmp/tmpYyosAY/database/files/000/dataset_2.dat galaxy.jobs DEBUG 2013-04-22 18:20:14,600 Tool did not define exit code or stdio handling; checking stderr for success galaxy.jobs DEBUG 2013-04-22 18:20:17,018 job 2 ended base.twilltestcase DEBUG 2013-04-22 18:20:18,061 In submit_form, continuing, but caught exception: cannot find value/label "four_human_proteins.rename.tabular" in list control base.twilltestcase INFO 2013-04-22 18:20:18,690 ## files diff on /mnt/galaxy/galaxy-central/test-data/four_human_proteins.rename.fasta and /tmp/tmpYyosAY/database/tmp/tmpAVYLNlfour_human_proteins.rename.fasta lines_diff=0, found diff = 13 ---------------------- >> begin tool stdout << ----------------------- ----------------------- >> end tool stdout << ------------------------ ---------------------- >> begin tool stderr << ----------------------- ----------------------- >> end tool stderr << ------------------------ FAIL ====================================================================== FAIL: Rename sequences ( seq_rename ) > Test-1 ---------------------------------------------------------------------- Traceback (most recent call last): File "/mnt/galaxy/galaxy-central/test/functional/test_toolbox.py", line 171, in test_tool self.do_it( td, shed_tool_id=shed_tool_id ) File "/mnt/galaxy/galaxy-central/test/functional/test_toolbox.py", line 102, in do_it self.verify_dataset_correctness( outfile, hid=elem_hid, maxseconds=testdef.maxseconds, attributes=attributes, shed_tool_id=shed_tool_id ) File "/mnt/galaxy/galaxy-central/test/base/twilltestcase.py", line 849, in verify_dataset_correctness raise AssertionError( errmsg ) AssertionError: History item 2 different than expected, difference (using diff): ( /mnt/galaxy/galaxy-central/test-data/four_human_proteins.rename.fasta v. /tmp/tmpYyosAY/database/tmp/tmpAVYLNlfour_human_proteins.rename.fasta ) --- local_file +++ history_data @@ -1,8 +1,5 @@ ->Q9BS26 Endoplasmic reticulum resident protein 44 OS=Homo sapiens GN=ERP44 PE=1 SV=1 -MHPAVFLSLPDLRCSLLLLVTWVFTPVTTEITSLDTENIDEILNNADVALVNFYADWCRFSQMLHPIFEEASDVIKEEFPNENQVVFARVDCDQHSDIAQRYRISKYPTLKLFRNGMMMKREYRGQRSVKALADYIRQQKSDPIQEIRDLAEITTLDRSKRNIIGYFEQKDSDNYRVFERVANILHDDCAFLSAFGDVSKPERYSGDNIIYKPPGHSAPDMVYLGAMTNFDVTYNWIQDKCVPLVREITFENGEELTEEGLPFLILFHMKEDTESLEIFQNEVARQLISEKGTINFLHADCDKFRHPLLHIQKTPADCPVIAIDSFRHMYVFGDFKDVLIPGKLKQFVFDLHSGKLHREFHHGPDPTDTAPGEQAQDVASSPPESSFQKLAPSEYRYTLLRDRDEL ->Q9NSY1 BMP-2-inducible protein kinase OS=Homo sapiens GN=BMP2K PE=1 SV=2 -MKKFSRMPKSEGGSGGGAAGGGAGGAGAGAGCGSGGSSVGVRVFAVGRHQVTLEESLAEGGFSTVFLVRTHGGIRCALKRMYVNNMPDLNVCKREITIMKELSGHKNIVGYLDCAVNSISDNVWEVLILMEYCRAGQVVNQMNKKLQTGFTEPEVLQIFCDTCEAVARLHQCKTPIIHRDLKVENILLNDGGNYVLCDFGSATNKFLNPQKDGVNVVEEEIKKYTTLSYRAPEMINLYGGKPITTKADIWALGCLLYKLCFFTLPFGESQVAICDGNFTIPDNSRYSRNIHCLIRFMLEPDPEHRPDIFQVSYFAFKFAKKDCPVSNINNSSIPSALPEPMTASEAAARKSQIKARITDTIGPTETSIAPRQRPKANSATTATPSVLTIQSSATPVKVLAPGEFGNHRPKGALRPGNGPEILLGQGPPQQPPQQHRVLQQLQQGDWRLQQLHLQHRHPHQQQQQQQQQQQQQQQQQQQQQQQQQQQHHHHHHHHLLQDAYMQQYQHATQQQQMLQQQFLMHSVYQPQPSASQYPTMMPQYQQAFFQQQMLAQHQPSQQQASPEYLTSPQEFSPALVSYTSSLPAQVGTIMDSSYSANRSVADKEAIANFTNQKNISNPPDMSGWNPFGEDNFSKLTEEELLDREFDLLRSNRLEERASSDKNVDSLSAPHNHPPEDPFGSVPFISHSGSPEKKAEHSSINQENGTANPIKNGKTSPASKDQRTGKKTSVQGQVQKGNDESESDFESDPPSPKSSEEEEQDDEEVLQGEQGDFNDDDTEPENLGHRPLLMDSEDEEEEEKHSSDSDYEQAKAKYSDMSSVYRDRSGSGPTQDLNTILLTSAQLSSDVAVETPKQEFDVFGAVPFFAVRAQQPQQEKNEKNLPQHRFPAAGLEQEEFDVFTKAPFSKKVNVQECHAVGPEAHTIPGYPKSVDVFGSTPFQPFLTSTSKSESNEDLFGLVPFDEITGSQQQKVKQRSLQKLSSRQRRTKQDMSKSNGKRHHGTPTSTKKTLKPTYRTPERARRHKKVGRRDSQSSNEFLTISDSKENISVALTDGKDRGNVLQPEESLLDPFGAKPFHSPDLSWHPPHQGLSDIRADHNTVLPGRPRQNSLHGSFHSADVLKMDDFGAVPFTELVVQSITPHQSQQSQPVELDPFGAAPFPSKQ ->P06213 Insulin receptor OS=Homo sapiens GN=INSR PE=1 SV=4 -MATGGRRGAAAAPLLVAVAALLLGAAGHLYPGEVCPGMDIRNNLTRLHELENCSVIEGHLQILLMFKTRPEDFRDLSFPKLIMITDYLLLFRVYGLESLKDLFPNLTVIRGSRLFFNYALVIFEMVHLKELGLYNLMNITRGSVRIEKNNELCYLATIDWSRILDSVEDNYIVLNKDDNEECGDICPGTAKGKTNCPATVINGQFVERCWTHSHCQKVCPTICKSHGCTAEGLCCHSECLGNCSQPDDPTKCVACRNFYLDGRCVETCPPPYYHFQDWRCVNFSFCQDLHHKCKNSRRQGCHQYVIHNNKCIPECPSGYTMNSSNLLCTPCLGPCPKVCHLLEGEKTIDSVTSAQELRGCTVINGSLIINIRGGNNLAAELEANLGLIEEISGYLKIRRSYALVSLSFFRKLRLIRGETLEIGNYSFYALDNQNLRQLWDWSKHNLTITQGKLFFHYNPKLCLSEIHKMEEVSGTKGRQERNDIALKTNGDQASCENELLKFSYIRTSFDKILLRWEPYWPPDFRDLLGFMLFYKEAPYQNVTEFDGQDACGSNSWTVVDIDPPLRSNDPKSQNHPGWLMRGLKPWTQYAIFVKTLVTFSDERRTYGAKSDIIYVQTDATNPSVPLDPISVSNSSSQIILKWKPPSDPNGNITHYLVFWERQAEDSELFELDYCLKGLKLPSRTWSPPFESEDSQKHNQSEYEDSAGECCSCPKTDSQILKELEESSFRKTFEDYLHNVVFVPRKTSSGTGAEDPRPSRKRRSLGDVGNVTVAVPTVAAFPNTSSTSVPTSPEEHRPFEKVVNKESLVISGLRHFTGYRIELQACNQDTPEERCSVAAYVSARTMPEAKADDIVGPVTHEIFENNVVHLMWQEPKEPNGLIVLYEVSYRRYGDEELHLCVSRKHFALERGCRLRGLSPGNYSVRIRATSLAGNGSWTEPTYFYVTDYLDVPSNIAKIIIGPLIFVFLFSVVIGSIYLFLRKRQPDGPLGPLYASSNPEYLSASDVFPCSVYVPDEWEVSREKITLLRELGQGSFGMVYEGNARDIIKGEAETRVAVKTVNESASLRERIEFLNEASVMKGFTCHHVVRLLGVVSKGQPTLVVMELMAHGDLKSYLRSLRPEAENNPGRPPPTLQEMIQMAAEIADGMAYLNAKKFVHRDLAARNCMVAHDFTVKIGDFGMTRDIYETDYYRKGGKGLLPVRWMAPESLKDGVFTTSSDMWSFGVVLWEITSLAEQPYQGLSNEQVLKFVMDGGYLDQPDNCPERVTDLMRMCWQFNPKMRPTFLEIVNLLKDDLHPSFPEVSFFHSEENKAPESEELEMEFEDMENVPLDRSSHCQREEAGGRDGGSSLGFKRSYEEHIPYTHMNGGKKNGRILTLPRSNPS ->P08100 Rhodopsin OS=Homo sapiens GN=RHO PE=1 SV=1 -MNGTEGPNFYVPFSNATGVVRSPFEYPQYYLAEPWQFSMLAAYMFLLIVLGFPINFLTLYVTVQHKKLRTPLNYILLNLAVADLFMVLGGFTSTLYTSLHGYFVFGPTGCNLEGFFATLGGEIALWSLVVLAIERYVVVCKPMSNFRFGENHAIMGVAFTWVMALACAAPPLAGWSRYIPEGLQCSCGIDYYTLKPEVNNESFVIYMFVVHFTIPMIIIFFCYGQLVFTVKEAAAQQQESATTQKAEKEVTRMVIIMVIAFLICWVPYASVAFYIFTHQGSNFGPIFMTIPAFFAKSAAIYNPVIYIMMNKQFRNCMLTTICCGKNPLGDDEASATVSKTETSQVAPA +#FASTA ID +sp|Q9BS26|ERP44_HUMAN Q9BS26 +sp|Q9NSY1|BMP2K_HUMAN Q9NSY1 +sp|P06213|INSR_HUMAN P06213 +sp|P08100|OPSD_HUMAN P08100 -------------------- >> begin captured stdout << --------------------- Uploaded file: four_human_proteins.fasta , ftype: fasta , extra: {'ftype': 'fasta', 'value': 'four_human_proteins.fasta', 'children': []} Uploaded file: four_human_proteins.rename.tabular , ftype: tabular , extra: {'ftype': 'tabular', 'value': 'four_human_proteins.rename.tabular', 'children': []} form 'tool_form' contains the following controls ( note the values ) control 0: <HiddenControl(refresh=refresh) (readonly)> control 1: <HiddenControl(tool_id=seq_rename) (readonly)> control 2: <HiddenControl(tool_state=800255f5643331613637623836313633636133663739623531386230366564393538633934343264643230303a37623232356635663730363136373635356635663232336132303330326332303232366536353737356636333666366337353664366532323361323032323563323233313563323232323263323032323639366537303735373435663636363936633635323233613230323233313232326332303232366636633634356636333666366337353664366532323361323032323563323233313563323232323263323032323639366537303735373435663734363136323735366336313732323233613230323233323232376471002e) (readonly)> control 3: <SelectControl(input_file=[*1])> control 4: <SelectControl(input_tabular=[1, *2])> control 5: <SelectControl(old_column=[*1, 2])> control 6: <SelectControl(new_column=[*1, 2])> control 7: <SubmitControl(runtool_btn=Execute) (readonly)> page_inputs (0) {'new_column': ['2'], 'input_tabular': ['four_human_proteins.rename.tabular'], 'old_column': ['1'], 'input_file': ['four_human_proteins.fasta']} --------------------- >> end captured stdout << ---------------------- -------------------- >> begin captured logging << -------------------- galaxy.web.framework: DEBUG: This request returned None from get_history(): http://localhost:9176/ galaxy.web.framework: DEBUG: This request returned None from get_history(): http://localhost:9176/ galaxy.web.framework: DEBUG: This request returned None from get_history(): http://localhost:9176/ galaxy.web.framework: DEBUG: This request returned None from get_history(): http://localhost:9176/ galaxy.web.framework: DEBUG: This request returned None from get_history(): http://localhost:9176/user/logout galaxy.web.framework: DEBUG: This request returned None from get_history(): http://localhost:9176/user/logout galaxy.web.framework: DEBUG: This request returned None from get_history(): http://localhost:9176/ galaxy.web.framework: DEBUG: This request returned None from get_history(): http://localhost:9176/ galaxy.tools.actions.upload_common: INFO: tool upload1 created job id 1 galaxy.jobs: DEBUG: (1) Working directory for job is: /mnt/galaxy/galaxy-central/database/job_working_directory/000/1 galaxy.jobs.handler: DEBUG: (1) Dispatching to local runner galaxy.jobs: DEBUG: (1) Persisting job destination (destination id: local:///) galaxy.jobs.handler: INFO: (1) Job dispatched galaxy.jobs.runners.local: DEBUG: (1) executing: python /mnt/galaxy/galaxy-central/tools/data_source/upload.py /mnt/galaxy/galaxy-central /tmp/tmpYyosAY/database/tmp/tmpIPAfMX /tmp/tmpYyosAY/database/tmp/tmpd7kJZZ 1:/mnt/galaxy/galaxy-central/database/job_working_directory/000/1/dataset_1_files:/tmp/tmpYyosAY/database/files/000/dataset_1.dat galaxy.jobs: DEBUG: (1) Persisting job destination (destination id: local:///) galaxy.jobs.runners.local: DEBUG: execution finished: python /mnt/galaxy/galaxy-central/tools/data_source/upload.py /mnt/galaxy/galaxy-central /tmp/tmpYyosAY/database/tmp/tmpIPAfMX /tmp/tmpYyosAY/database/tmp/tmpd7kJZZ 1:/mnt/galaxy/galaxy-central/database/job_working_directory/000/1/dataset_1_files:/tmp/tmpYyosAY/database/files/000/dataset_1.dat galaxy.jobs: DEBUG: Tool did not define exit code or stdio handling; checking stderr for success galaxy.jobs: DEBUG: job 1 ended galaxy.tools.actions.upload_common: INFO: tool upload1 created job id 2 galaxy.jobs: DEBUG: (2) Working directory for job is: /mnt/galaxy/galaxy-central/database/job_working_directory/000/2 galaxy.jobs.handler: DEBUG: (2) Dispatching to local runner galaxy.jobs: DEBUG: (2) Persisting job destination (destination id: local:///) galaxy.jobs.handler: INFO: (2) Job dispatched galaxy.jobs.runners.local: DEBUG: (2) executing: python /mnt/galaxy/galaxy-central/tools/data_source/upload.py /mnt/galaxy/galaxy-central /tmp/tmpYyosAY/database/tmp/tmpIPAfMX /tmp/tmpYyosAY/database/tmp/tmpeA1apX 2:/mnt/galaxy/galaxy-central/database/job_working_directory/000/2/dataset_2_files:/tmp/tmpYyosAY/database/files/000/dataset_2.dat galaxy.jobs: DEBUG: (2) Persisting job destination (destination id: local:///) galaxy.jobs.runners.local: DEBUG: execution finished: python /mnt/galaxy/galaxy-central/tools/data_source/upload.py /mnt/galaxy/galaxy-central /tmp/tmpYyosAY/database/tmp/tmpIPAfMX /tmp/tmpYyosAY/database/tmp/tmpeA1apX 2:/mnt/galaxy/galaxy-central/database/job_working_directory/000/2/dataset_2_files:/tmp/tmpYyosAY/database/files/000/dataset_2.dat galaxy.jobs: DEBUG: Tool did not define exit code or stdio handling; checking stderr for success galaxy.jobs: DEBUG: job 2 ended base.twilltestcase: DEBUG: In submit_form, continuing, but caught exception: cannot find value/label "four_human_proteins.rename.tabular" in list control base.twilltestcase: INFO: ## files diff on /mnt/galaxy/galaxy-central/test-data/four_human_proteins.rename.fasta and /tmp/tmpYyosAY/database/tmp/tmpAVYLNlfour_human_proteins.rename.fasta lines_diff=0, found diff = 13 --------------------- >> end captured logging << --------------------- ---------------------------------------------------------------------- Ran 1 test in 15.564s FAILED (failures=1)
Peter, Thank you for reporting this issue, I've committed a fix in 9508:4126ec15fd61. --Dave B. On 4/24/13 05:10:55.000, Peter Cock wrote:
On Tue, Nov 23, 2010 at 2:06 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
Hi all,
I'm working on a FASTA filter script, as per this email: http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-November/003819.html
Current code here: http://bitbucket.org/peterjc/galaxy-central/src/c3ac6d7a02f7/
I've added a basic test, but it won't run (see output below). I think the interesting bit of the error output is this exception: cannot find value/label "blastp_four_human_vs_rhodopsin.tabular" in list control, coming from function set_form_control_value in the twill.utils module.
This is something to do with the columns parameter, a multiselect option referencing the columns in this tabular file. Removing the validator makes no difference. However, if I change the columns parameter to a plain text parameter, the test passes:
$ hg diff diff -r c3ac6d7a02f7 tools/fasta_tools/fasta_filter_by_id.xml --- a/tools/fasta_tools/fasta_filter_by_id.xml Tue Nov 23 11:37:35 2010 +0000 +++ b/tools/fasta_tools/fasta_filter_by_id.xml Tue Nov 23 14:03:31 2010 +0000 @@ -4,9 +4,7 @@ <inputs> <param name="input_fasta" type="data" format="fasta" label="FASTA file to filter on the identifiers"/> <param name="input_tabular" type="data" format="tabular" label="Tabular file containing FASTA identifiers"/> - <param name="columns" type="data_column" data_ref="input_tabular" multiple="True" numerical="False" label="Column(s) containing FASTA identifiers" help="Multi-select list - hold the appropriate key while clicking to select multiple columns"> - <validator type="no_options" message="Pick at least one column"/> - </param> + <param name="columns" type="text" label="Column(s) containing FASTA identifiers" /> </inputs> <outputs> <data name="output_pos" format="fasta" label="With matched ID" />
I've tried searching the provided wrappers for similar examples (column multi-select from a tabular file). I found filters/uniq.xml (works but uses a bed file rather than a simple tabular file) and stats/cor.xml (needs R and rpy which I don't have installed yet) which have tests, and finally plotting/bar_chart.xml which has no unit tests.
Would someone familiar with the internals of the Galaxy tests and how they set tool parameters be able to try reproducing this for me? The branch has all the unit test files required, and there are no new dependencies needed.
Thank you,
Peter
--
Here is the output (on Linux - other tests tried pass):
$ ./run_functional_tests.sh -id fasta_filter_by_id ... base.twilltestcase DEBUG 2010-11-23 13:59:53,747 In submit_form, continuing, but caught exception: cannot find value/label "blastp_four_human_vs_rhodopsin.tabular" in list control ...
Hello again,
See http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-November/003867.html for the original email and full log. The most interesting part of the logging was the caught exception (and the tool being tested was never run):
cannot find value/label "blastp_four_human_vs_rhodopsin.tabular" in list control
I don't recall if/how this was resolved, I've since deprecated this tool itself - replacing it with a more general version: http://toolshed.g2.bx.psu.edu/view/peterjc/seq_filter_by_id
However, I've just hit what seems to be a very similar issue while trying to add what should be a simple unit test to my sequence renaming script, seq_rename.py / seq_rename.py which is in the main Tool Shed here: http://toolshed.g2.bx.psu.edu/view/peterjc/seq_rename
Here's the current latest code (not yet pushed to the public Tool Shed), which is now on the test Tool Shed and on my bitbucket branch: http://testtoolshed.g2.bx.psu.edu/view/peterjc/seq_rename https://bitbucket.org/peterjc/galaxy-central/src/7750adf18b1fab953d7cccd0216...
Here's the test itself,
<tests> <test> <param name="input_file" value="four_human_proteins.fasta" ftype="fasta" /> <param name="input_tabular" value="four_human_proteins.rename.tabular" ftype="tabular" /> <param name="old_column" value="1" /> <param name="new_column" value="2" /> <output name="output_file" file="four_human_proteins.rename.fasta" ftype="fasta" /> </test> </tests>
Now I am quite puzzled by what happens running the test, the full output is at the end of this email. Strangely the tool itself is never actually run, and instead Galaxy compares one of the input files to the expected output file and they don't match.
As in my email from 2010, this is most interesting line in the output, just after the two input files are 'uploaded' to the current history:
base.twilltestcase DEBUG 2013-04-22 18:20:18,061 In submit_form, continuing, but caught exception: cannot find value/label "four_human_proteins.rename.tabular" in list control
It would appear that the tabular file has not been added to the potential valid datasets available to the $input_tabular parameter?
This test should have been attempted on the Test Tool Shed last night, but there is nothing to show for it - something that has happened in other corner cases - I'll flag this on the other thread. Likely this is due to the exception within the test framework itself.
Regards,
Peter
participants (2)
-
Dave Bouvier -
Peter Cock