Optional multiple file selectors,
Hi John, As you noticed (and posted on Twitter), I've just tweaked the NCBI makeblastdb wrapper on GitHub to use a multiple file picker instead of a repeat: https://github.com/peterjc/galaxy_blast/commit/858bf541a828644ea6461b20b687c... That seems to be working fine in human testing (although not on the functional side). I've also just tried this for the masking files - but ran into a problem. Here the masks are optional, and I want the default to be no mask: https://github.com/peterjc/galaxy_blast/commit/e1bcb24ac24613395a0fe327c54cd... This works fine from the user interface perspective: <param name="input_file" type="data" multiple="true" optional="false" format="fasta" label="Input FASTA files(s)" help="One or more FASTA files" /> This requires at least one file, although a confusing red error message is given if none is selected: "History does not include a dataset of the required format / build" <param name="mask_data_file" type="data" multiple="true" optional="true" value="" format="maskinfo-asn1,maskinfo-asn1-binary" label="Optional ASN.1 file(s) containing masking data" help="As produced by NCBI masking applications (e.g. dustmasker, segmasker, windowmasker)" /> This does not require a file, and defaults to selecting a new dummy entry "Selection is Optional". This is presumably intended as a visual cue - but it means you can do weird things like picking a read file and the dummy entry, or no entries. Neither allows me to proceed, instead on pressing execute I get a red error on a new page: Error executing tool: 'NoneType' object has no attribute 'file_name' galaxy.tools ERROR 2014-02-25 14:02:47,844 Exception caught while attempting tool execution: Traceback (most recent call last): File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/__init__.py", line 1916, in __handle_tool_execute _, out_data = self.execute( trans, incoming=params, history=history, rerun_remap_job_id=rerun_remap_job_id ) File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/__init__.py", line 2424, in execute return self.tool_action.execute( self, trans, incoming=incoming, set_output_hid=set_output_hid, history=history, **kwargs ) File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/actions/__init__.py", line 287, in execute data.name = self.get_output_name( output, data, tool, on_text, trans, incoming, history, wrapped_params.params, job_params ) File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/parameters/wrapped.py", line 26, in params self.wrap_values( self.tool.inputs, params, skip_missing_values=not self.tool.check_values ) File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/parameters/wrapped.py", line 51, in wrap_values name=input.name ) File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/wrappers.py", line 215, in __init__ list.__init__( self, map( to_wrapper, datasets ) ) File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/wrappers.py", line 209, in to_wrapper real_path = dataset.file_name AttributeError: 'NoneType' object has no attribute 'file_name' Any thoughts? Is zero files a little tested corner of the multiple file support? If I have a suitable masking file in my history, and I select it, the tool runs fine. Thanks, Peter
Hey Peter, Answers inline. On Tue, Feb 25, 2014 at 8:08 AM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
Hi John,
As you noticed (and posted on Twitter), I've just tweaked the NCBI makeblastdb wrapper on GitHub to use a multiple file picker instead of a repeat:
https://github.com/peterjc/galaxy_blast/commit/858bf541a828644ea6461b20b687c...
That seems to be working fine in human testing (although not on the functional side).
Functional test support for these was added with the latest release but only for API driven tests. Can you try adding interactor="api" to your test cases? https://bitbucket.org/galaxy/galaxy-central/commits/211f30207d48101a1250983d...
I've also just tried this for the masking files - but ran into a problem. Here the masks are optional, and I want the default to be no mask:
https://github.com/peterjc/galaxy_blast/commit/e1bcb24ac24613395a0fe327c54cd...
This works fine from the user interface perspective:
<param name="input_file" type="data" multiple="true" optional="false" format="fasta" label="Input FASTA files(s)" help="One or more FASTA files" />
This requires at least one file, although a confusing red error message is given if none is selected:
"History does not include a dataset of the required format / build"
<param name="mask_data_file" type="data" multiple="true" optional="true" value="" format="maskinfo-asn1,maskinfo-asn1-binary" label="Optional ASN.1 file(s) containing masking data" help="As produced by NCBI masking applications (e.g. dustmasker, segmasker, windowmasker)" />
This does not require a file, and defaults to selecting a new dummy entry "Selection is Optional". This is presumably intended as a visual cue - but it means you can do weird things like picking a read file and the dummy entry, or no entries. Neither allows me to proceed, instead on pressing execute I get a red error on a new page:
Error executing tool: 'NoneType' object has no attribute 'file_name'
galaxy.tools ERROR 2014-02-25 14:02:47,844 Exception caught while attempting tool execution: Traceback (most recent call last): File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/__init__.py", line 1916, in __handle_tool_execute _, out_data = self.execute( trans, incoming=params, history=history, rerun_remap_job_id=rerun_remap_job_id ) File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/__init__.py", line 2424, in execute return self.tool_action.execute( self, trans, incoming=incoming, set_output_hid=set_output_hid, history=history, **kwargs ) File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/actions/__init__.py", line 287, in execute data.name = self.get_output_name( output, data, tool, on_text, trans, incoming, history, wrapped_params.params, job_params ) File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/parameters/wrapped.py", line 26, in params self.wrap_values( self.tool.inputs, params, skip_missing_values=not self.tool.check_values ) File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/parameters/wrapped.py", line 51, in wrap_values name=input.name ) File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/wrappers.py", line 215, in __init__ list.__init__( self, map( to_wrapper, datasets ) ) File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/wrappers.py", line 209, in to_wrapper real_path = dataset.file_name AttributeError: 'NoneType' object has no attribute 'file_name'
Any thoughts? Is zero files a little tested corner of the multiple file support?
Yes it is a corner case I personally had not tested. Are you are running something later than the latest dist but not the latest central? It seems like yes. Because I believe Dan has fixed the bug that produces the particular stack trace you are seeing last week - he has probably fixed the other problems you mentioned also? Can you upgrade and let me know, if not let me know and I can take a look a closer look.
If I have a suitable masking file in my history, and I select it, the tool runs fine.
Thanks,
Peter
On Tue, Feb 25, 2014 at 2:29 PM, John Chilton <jmchilton@gmail.com> wrote:
Hey Peter,
Answers inline.
On Tue, Feb 25, 2014 at 8:08 AM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
Hi John,
As you noticed (and posted on Twitter), I've just tweaked the NCBI makeblastdb wrapper on GitHub to use a multiple file picker instead of a repeat:
https://github.com/peterjc/galaxy_blast/commit/858bf541a828644ea6461b20b687c...
That seems to be working fine in human testing (although not on the functional side).
Functional test support for these was added with the latest release but only for API driven tests. Can you try adding interactor="api" to your test cases?
https://bitbucket.org/galaxy/galaxy-central/commits/211f30207d48101a1250983d...
Ah - examples without adding the api flag, e.g. https://travis-ci.org/peterjc/galaxy_blast/builds/19517715 and https://travis-ci.org/peterjc/galaxy_blast/builds/19570673 Problem where attempts to compare text log file (primary file in the blast database composite datatype) with a fasta file (see below). Suspecting this was due to a change in Galaxy, not the wrapper, I reran a previously successful TravisCI build from 5 days ago: https://travis-ci.org/peterjc/galaxy_blast/builds/19248312 This worked - so change to the makeblastdb wrapper changes seem to be the immediate trigger. Peter Snippet from failing test, AssertionError: History item 1 different than expected, difference (using diff): ( /home/travis/build/peterjc/galaxy_blast/galaxy-central-master/test-data/four_human_proteins_taxid.fasta.log v. /tmp/tmpNDhPnX/tmpj7U79F/new_files_path_LGtc1w/tmpdBuU0jfour_human_proteins_taxid.fasta.log ) --- local_file +++ history_data @@ -1,10 +1,61 @@ - - -Building a new DB, current time: 02/10/2014 18:40:09 -New DB name: four_human_proteins_taxid.fasta -New DB title: Just 4 human proteins -Sequence type: Protein -Keep Linkouts: T -Keep MBits: T -Maximum file size: 1000000000B -Adding sequences from FASTA; added 4 sequences in 0.00230002 seconds. +>sp|Q9BS26|ERP44_HUMAN Endoplasmic reticulum resident protein 44 OS=Homo sapiens GN=ERP44 PE=1 SV=1 +MHPAVFLSLPDLRCSLLLLVTWVFTPVTTEITSLDTENIDEILNNADVALVNFYADWCRF +SQMLHPIFEEASDVIKEEFPNENQVVFARVDCDQHSDIAQRYRISKYPTLKLFRNGMMMK +REYRGQRSVKALADYIRQQKSDPIQEIRDLAEITTLDRSKRNIIGYFEQKDSDNYRVFER +VANILHDDCAFLSAFGDVSKPERYSGDNIIYKPPGHSAPDMVYLGAMTNFDVTYNWIQDK +CVPLVREITFENGEELTEEGLPFLILFHMKEDTESLEIFQNEVARQLISEKGTINFLHAD +CDKFRHPLLHIQKTPADCPVIAIDSFRHMYVFGDFKDVLIPGKLKQFVFDLHSGKLHREF +HHGPDPTDTAPGEQAQDVASSPPESSFQKLAPSEYRYTLLRDRDEL +>sp|Q9NSY1|BMP2K_HUMAN BMP-2-inducible protein kinase OS=Homo sapiens GN=BMP2K PE=1 SV=2 +MKKFSRMPKSEGGSGGGAAGGGAGGAGAGAGCGSGGSSVGVRVFAVGRHQVTLEESLAEG +GFSTVFLVRTHGGIRCALKRMYVNNMPDLNVCKREITIMKELSGHKNIVGYLDCAVNSIS +DNVWEVLILMEYCRAGQVVNQMNKKLQTGFTEPEVLQIFCDTCEAVARLHQCKTPIIHRD ******** *SNIP * ******** +TVINGSLIINIRGGNNLAAELEANLGLIEEISGYLKIRRSYALVSLSFFRKLRLIRGETL +EIGNYSFYALDNQNLRQLWDWSKHNLTITQGKLFFHYNPKLCLSEIHKMEEVSGTKGRQE +RNDIALKTNGDQASCENELLKFSYIRTSFDKILLRWEPYWPPDFRDLLGFMLFYKEAPYQ +NVTEFDGQDACGSNSWTVVDIDPPLRSNDPKSQNHPGWLMRGLKPWTQYAIFVKTLVTFS +DERRTYGAKSDIIYVQTDATNPSVPLDPISVSNSSSQIILKWKPPSDPNGNITHYLVFWE +RQAEDSELFELDYCLKGLKLPSRTWSPPFESEDSQKHNQSEYEDSAGECCSCPKTDSQIL +KELEESSFRKTFEDYLHNVVFVPRKTSSGTGAEDPRPSRKRRSLGDVGNVTVAVPTVAAF +PNTSSTSVPTSPEEHRPFEKVVNKESLVISGLRHFTGYRIELQACNQDTPEERCSVAAYV +SARTMPEAKADDIVGPVTHEIFENNVVHLMWQEPKEPNGLIVLYEVSYRRYGDEELHLCV +SRKHFALERGCRLRGLSPGNYSVRIRATSLAGNGSWTEPTYFYVTDYLDVPSNIAKIIIG +PLIFVFLFSVVIGSIYLFLRKRQPDGPLGPLYASSNPEYLSASDVFPCSVYVPDEWEVSR +EKITLLRELGQGSFGMVYEGNARDIIKGEAETRVAVKTVNESASLRERIEFLNEASVMKG +FTCHHVVRLLGVVSKGQPTLVVMELMAHGDLKSYLRSLRPEAENNPGRPPPTLQEMIQMA +AEIADGMAYLNAKKFVHRDLAARNCMVAHDFTVKIGDFGMTRDIYETDYYRKGGKGLLPV +RWMAPESLKDGVFTTSSDMWSFGVVLWEITSLAEQPYQGLSNEQVLKFVMDGGYLDQPDN +CPERVTDLMRMCWQFNPKMRPTFLEIVNLLKDDLHPSFPEVSFFHSEENKAPESEELEME +FEDMENVPLDRSSHCQREEAGGRDGGSSLGFKRSYEEHIPYTHMNGGKKNGRILTLPRSN +PS +>sp|P08100|OPSD_HUMAN Rhodopsin OS=Homo sapiens GN=RHO PE=1 SV=1 +MNGTEGPNFYVPFSNATGVVRSPFEYPQYYLAEPWQFSMLAAYMFLLIVLGFPINFLTLY +VTVQHKKLRTPLNYILLNLAVADLFMVLGGFTSTLYTSLHGYFVFGPTGCNLEGFFATLG +GEIALWSLVVLAIERYVVVCKPMSNFRFGENHAIMGVAFTWVMALACAAPPLAGWSRYIP +EGLQCSCGIDYYTLKPEVNNESFVIYMFVVHFTIPMIIIFFCYGQLVFTVKEAAAQQQES +ATTQKAEKEVTRMVIIMVIAFLICWVPYASVAFYIFTHQGSNFGPIFMTIPAFFAKSAAI +YNPVIYIMMNKQFRNCMLTTICCGKNPLGDDEASATVSKTETSQVAPA P.S. The extra space in "*SNIP *" got lost in my emailed patch, https://bitbucket.org/galaxy/galaxy-central/diff/test/base/twilltestcase.py?diff2=15fc8675064e&at=default
On Tue, Feb 25, 2014 at 2:29 PM, John Chilton <jmchilton@gmail.com> wrote:
On Tue, Feb 25, 2014 at 8:08 AM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
I've also just tried this for the masking files - but ran into a problem. Here the masks are optional, and I want the default to be no mask:
https://github.com/peterjc/galaxy_blast/commit/e1bcb24ac24613395a0fe327c54cd...
This works fine from the user interface perspective:
<param name="input_file" type="data" multiple="true" optional="false" format="fasta" label="Input FASTA files(s)" help="One or more FASTA files" />
This requires at least one file, although a confusing red error message is given if none is selected:
"History does not include a dataset of the required format / build"
<param name="mask_data_file" type="data" multiple="true" optional="true" value="" format="maskinfo-asn1,maskinfo-asn1-binary" label="Optional ASN.1 file(s) containing masking data" help="As produced by NCBI masking applications (e.g. dustmasker, segmasker, windowmasker)" />
This does not require a file, and defaults to selecting a new dummy entry "Selection is Optional". This is presumably intended as a visual cue - but it means you can do weird things like picking a read file and the dummy entry, or no entries. Neither allows me to proceed, instead on pressing execute I get a red error on a new page:
Error executing tool: 'NoneType' object has no attribute 'file_name'
galaxy.tools ERROR 2014-02-25 14:02:47,844 Exception caught while attempting tool execution: Traceback (most recent call last): File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/__init__.py", line 1916, in __handle_tool_execute _, out_data = self.execute( trans, incoming=params, history=history, rerun_remap_job_id=rerun_remap_job_id ) File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/__init__.py", line 2424, in execute return self.tool_action.execute( self, trans, incoming=incoming, set_output_hid=set_output_hid, history=history, **kwargs ) File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/actions/__init__.py", line 287, in execute data.name = self.get_output_name( output, data, tool, on_text, trans, incoming, history, wrapped_params.params, job_params ) File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/parameters/wrapped.py", line 26, in params self.wrap_values( self.tool.inputs, params, skip_missing_values=not self.tool.check_values ) File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/parameters/wrapped.py", line 51, in wrap_values name=input.name ) File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/wrappers.py", line 215, in __init__ list.__init__( self, map( to_wrapper, datasets ) ) File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/wrappers.py", line 209, in to_wrapper real_path = dataset.file_name AttributeError: 'NoneType' object has no attribute 'file_name'
Any thoughts? Is zero files a little tested corner of the multiple file support?
Yes it is a corner case I personally had not tested. Are you are running something later than the latest dist but not the latest central? It seems like yes. Because I believe Dan has fixed the bug that produces the particular stack trace you are seeing last week - he has probably fixed the other problems you mentioned also? Can you upgrade and let me know, if not let me know and I can take a look a closer look.
This one on TravisCI (which looks like the latest galaxy-central) fails with: Fatal error: Matched on Error: Error: Ignoring mask file 'None' as it does not exist. https://travis-ci.org/peterjc/galaxy_blast/builds/19577818 Perhaps I can workaround this in the Cheetah template, but ideally the no-files case would be handled nicely in the framework. Peter
Okay, known issue I guess. No inputs evaluates as a list with just "None" in it. This is less than ideal and there is an open Trello card on this (see top item of TODO list) https://trello.com/c/lp5YmA1O. I think vanilla (non-data) select tools work the same - but maybe not. -John On Tue, Feb 25, 2014 at 8:44 AM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
On Tue, Feb 25, 2014 at 2:29 PM, John Chilton <jmchilton@gmail.com> wrote:
On Tue, Feb 25, 2014 at 8:08 AM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
I've also just tried this for the masking files - but ran into a problem. Here the masks are optional, and I want the default to be no mask:
https://github.com/peterjc/galaxy_blast/commit/e1bcb24ac24613395a0fe327c54cd...
This works fine from the user interface perspective:
<param name="input_file" type="data" multiple="true" optional="false" format="fasta" label="Input FASTA files(s)" help="One or more FASTA files" />
This requires at least one file, although a confusing red error message is given if none is selected:
"History does not include a dataset of the required format / build"
<param name="mask_data_file" type="data" multiple="true" optional="true" value="" format="maskinfo-asn1,maskinfo-asn1-binary" label="Optional ASN.1 file(s) containing masking data" help="As produced by NCBI masking applications (e.g. dustmasker, segmasker, windowmasker)" />
This does not require a file, and defaults to selecting a new dummy entry "Selection is Optional". This is presumably intended as a visual cue - but it means you can do weird things like picking a read file and the dummy entry, or no entries. Neither allows me to proceed, instead on pressing execute I get a red error on a new page:
Error executing tool: 'NoneType' object has no attribute 'file_name'
galaxy.tools ERROR 2014-02-25 14:02:47,844 Exception caught while attempting tool execution: Traceback (most recent call last): File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/__init__.py", line 1916, in __handle_tool_execute _, out_data = self.execute( trans, incoming=params, history=history, rerun_remap_job_id=rerun_remap_job_id ) File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/__init__.py", line 2424, in execute return self.tool_action.execute( self, trans, incoming=incoming, set_output_hid=set_output_hid, history=history, **kwargs ) File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/actions/__init__.py", line 287, in execute data.name = self.get_output_name( output, data, tool, on_text, trans, incoming, history, wrapped_params.params, job_params ) File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/parameters/wrapped.py", line 26, in params self.wrap_values( self.tool.inputs, params, skip_missing_values=not self.tool.check_values ) File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/parameters/wrapped.py", line 51, in wrap_values name=input.name ) File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/wrappers.py", line 215, in __init__ list.__init__( self, map( to_wrapper, datasets ) ) File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/wrappers.py", line 209, in to_wrapper real_path = dataset.file_name AttributeError: 'NoneType' object has no attribute 'file_name'
Any thoughts? Is zero files a little tested corner of the multiple file support?
Yes it is a corner case I personally had not tested. Are you are running something later than the latest dist but not the latest central? It seems like yes. Because I believe Dan has fixed the bug that produces the particular stack trace you are seeing last week - he has probably fixed the other problems you mentioned also? Can you upgrade and let me know, if not let me know and I can take a look a closer look.
This one on TravisCI (which looks like the latest galaxy-central) fails with:
Fatal error: Matched on Error: Error: Ignoring mask file 'None' as it does not exist.
https://travis-ci.org/peterjc/galaxy_blast/builds/19577818
Perhaps I can workaround this in the Cheetah template, but ideally the no-files case would be handled nicely in the framework.
Peter
On Tue, Feb 25, 2014 at 2:50 PM, John Chilton <jmchilton@gmail.com> wrote:
Okay, known issue I guess. No inputs evaluates as a list with just "None" in it. This is less than ideal and there is an open Trello card on this (see top item of TODO list) https://trello.com/c/lp5YmA1O.
I think vanilla (non-data) select tools work the same - but maybe not.
-John
Thanks John - the workaround you suggested looks good, https://github.com/peterjc/galaxy_blast/commit/2e5c9e2ea08532a530ac4588a1d01... Tests passed on my local server (now just updated to galaxy-central), waiting for TravisCI now... Peter
participants (2)
-
John Chilton -
Peter Cock