Thanks John, I suggest making this test framework perform this check by default (the twill and API based frameworks) and seeing what - if anything - breaks as a result on the Test Tool Shed. Note that one area of fuzziness is subclasses, e.g. if the tool output was labelled "fastqsanger", but the <test> just said "fastq", I would say the test was broken. On the other hand, if the <test> used a specific datatype like "fastqsanger" but the tool produced a dataset tagged with a more generic datatype like "fastq" I think that is a again a real failure. Peter On Fri, Mar 14, 2014 at 3:15 PM, John Chilton <jmchilton@gmail.com> wrote:
Grepping around the code I think this is the only way a ftype attribute on an output affects the evaluation of test data.
if attributes.get( 'ftype', None ) == 'bam': local_fh, temp_name = self._bam_to_sam( local_name, temp_name ) local_name = local_fh.name
I am not sure it was ever meant as a strict test.
I worry about breaking backward compatibility but it is easy enough to implement this as an actual check when using newer API driven tests. I have opened a pull request for this functionality here:
https://bitbucket.org/galaxy/galaxy-central/pull-request/347/check-ftype-att...
Thoughts?
-John
On Tue, Mar 11, 2014 at 10:30 AM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
That seems to work - thanks Björn :)
This seems to have exposed a bug in the <test> framework, e.g.
<test> <param name="query" value="rhodopsin_nucs.fasta" ftype="fasta" /> <param name="db_opts_selector" value="file" /> <param name="subject" value="three_human_mRNA.fasta" ftype="fasta" /> <param name="database" value="" /> <param name="evalue_cutoff" value="1e-40" /> <param name="out_format" value="5" /> <param name="adv_opts_selector" value="basic" /> <output name="output1" file="blastn_rhodopsin_vs_three_human.xml" ftype="blastxml" /> </test>
This was saying the output should have been "blastxml", but until I just fixed it the output was being tagged as "tabular" (although run_functional_tests.sh did check the content it didn't check the datatype matched).
Dave - do think this is a reasonable enhancement?
Peter