- galaxy-dev - lists.galaxyproject.org

[hg] galaxy 3436: lims:
by Greg Von Kuster 24 Feb '10

24 Feb '10

details: http://www.bx.psu.edu/hg/galaxy/rev/66ba0f2d6d32 changeset: 3436:66ba0f2d6d32 user: rc date: Wed Feb 24 10:42:36 2010 -0500 description: lims: - code cleanup - cleanup tmp files after data transfer diffstat: lib/galaxy/web/controllers/requests_admin.py | 6 ------ scripts/galaxy_messaging/server/data_transfer.py | 2 +- 2 files changed, 1 insertions(+), 7 deletions(-) diffs (28 lines): diff -r c1d024e76c2c -r 66ba0f2d6d32 lib/galaxy/web/controllers/requests_admin.py --- a/lib/galaxy/web/controllers/requests_admin.py Wed Feb 24 10:04:03 2010 -0500 +++ b/lib/galaxy/web/controllers/requests_admin.py Wed Feb 24 10:42:36 2010 -0500 @@ -1589,12 +1589,6 @@ dataset_index=index, cmd=cmd) dtt.start() -# # set the sample state to the last state -# if sample.current_state().id != sample.request.type.states[-1].id: -# event = trans.app.model.SampleEvent(sample, sample.request.type.states[-1], -# 'The dataset is ready and are being transferred to Galaxy') -# trans.sa_session.add( event ) -# trans.sa_session.flush() if error_msg: return trans.response.send_redirect( web.url_for( controller='requests_admin', action='show_datatx_page', diff -r c1d024e76c2c -r 66ba0f2d6d32 scripts/galaxy_messaging/server/data_transfer.py --- a/scripts/galaxy_messaging/server/data_transfer.py Wed Feb 24 10:04:03 2010 -0500 +++ b/scripts/galaxy_messaging/server/data_transfer.py Wed Feb 24 10:42:36 2010 -0500 @@ -94,7 +94,7 @@ # update the data transfer status in the db self.update_status('Complete') # cleanup - #self.cleanup() + self.cleanup() sys.exit(0) def cleanup(self):

1 0

[hg] galaxy 3434: Adding missing sam_merge test files
by Greg Von Kuster 24 Feb '10

24 Feb '10

details: http://www.bx.psu.edu/hg/galaxy/rev/80747406525d changeset: 3434:80747406525d user: Kelly Vincent <kpvincent(a)bx.psu.edu> date: Wed Feb 24 08:44:22 2010 -0500 description: Adding missing sam_merge test files diffstat: test-data/sam_merge_in1.bam | 0 test-data/sam_merge_in2.bam | 0 test-data/sam_merge_in3.bam | 0 test-data/sam_merge_out2.bam | 0 4 files changed, 0 insertions(+), 0 deletions(-) diffs (8 lines): diff -r 5fa69a40e080 -r 80747406525d test-data/sam_merge_in1.bam Binary file test-data/sam_merge_in1.bam has changed diff -r 5fa69a40e080 -r 80747406525d test-data/sam_merge_in2.bam Binary file test-data/sam_merge_in2.bam has changed diff -r 5fa69a40e080 -r 80747406525d test-data/sam_merge_in3.bam Binary file test-data/sam_merge_in3.bam has changed diff -r 5fa69a40e080 -r 80747406525d test-data/sam_merge_out2.bam Binary file test-data/sam_merge_out2.bam has changed

1 0

adding local data
by Roy Weckiewicz 24 Feb '10

24 Feb '10

Hello, Thanks to the help of galaxy-dev, I've been able to migrate into a MySQL backend for my local galaxy instance. My next goal is to add the datasets for different genomes (bacterial, fungal, etc), so I've decided to start with human. I started out following the wiki Data Integration page and downloaded the BigZip files (full data set) for hg19. Next, the guide mentions to check individual tool xml's to see which .location file it uses, which in turn can be used to get the file type and format needed for the file. My first question: Is this the best way to go about adding a genome to a local instance -- Are the database files just referenced by the tools that use them, or is it possible to somehow include them in the MySQL database. Ideally I'd like to be able to select MyGenome from within Galaxy (probably through 'get Data' in the table browser ). The problem I'm having is that I cannot find all of the required files for a lot of the tools that came with the galaxy installation. Are a lot of these files just the resulting output of a script or program? e.g. ENCODE data, sequence alignments, genomic DNA extraction, etc. Best Regards, Roy Weckiewicz -- Roy Weckiewicz Texas A&M University

1 0

[hg] galaxy 3433: Updated Bowtie test files to handle new versio...
by Greg Von Kuster 24 Feb '10

24 Feb '10

details: http://www.bx.psu.edu/hg/galaxy/rev/5fa69a40e080 changeset: 3433:5fa69a40e080 user: Kelly Vincent <kpvincent(a)bx.psu.edu> date: Wed Feb 24 07:11:43 2010 -0500 description: Updated Bowtie test files to handle new version (0.12.3) of Bowtie (bowtie_color_wrapper will fail until new version installed). Updated samtools tools test files. Removed version references from Bowtie and BWA tools. diffstat: test-data/bowtie_in1.fastqsanger | 4 ++-- test-data/bowtie_out1.sam | 2 +- test-data/bowtie_out3.sam | 4 ++-- test-data/bowtie_out5.sam | 2 +- test-data/bowtie_out7.sam | 4 ++-- test-data/sam_to_bam_out1.bam | 0 tools/samtools/sam_merge.xml | 2 +- tools/samtools/sam_pileup.xml | 2 +- tools/samtools/sam_to_bam.xml | 2 +- tools/sr_mapping/bowtie_color_wrapper.xml | 30 ++++++++++++++---------------- tools/sr_mapping/bowtie_wrapper.xml | 30 ++++++++++++++---------------- tools/sr_mapping/bwa_wrapper.xml | 2 -- 12 files changed, 39 insertions(+), 45 deletions(-) diffs (223 lines): diff -r b4dd38c06398 -r 5fa69a40e080 test-data/bowtie_in1.fastqsanger --- a/test-data/bowtie_in1.fastqsanger Tue Feb 23 23:17:09 2010 -0500 +++ b/test-data/bowtie_in1.fastqsanger Wed Feb 24 07:11:43 2010 -0500 @@ -1,4 +1,4 @@ @869_1532_1255/1 -G21022233110003122233210021301222000112122113330022 +G2102223311000312223321002 + -=;8:?@=?;;9:8;=>;5A?;<8><<=:9><;9<=8;96>8<5==:<98; \ No newline at end of file +=;8:?@=?;;9:8;=>;5A?;<8>< diff -r b4dd38c06398 -r 5fa69a40e080 test-data/bowtie_out1.sam --- a/test-data/bowtie_out1.sam Tue Feb 23 23:17:09 2010 -0500 +++ b/test-data/bowtie_out1.sam Wed Feb 24 07:11:43 2010 -0500 @@ -1,1 +1,1 @@ -869_1532_1255/1 16 chrM 3728 255 48M * 0 0 GAAATATGTCTGACAAAAGAGTTACTTTGATAGAGTAAAACATAGAGG "TUVYQPSUSNSRTXTSVYVRVXWYUSVY_UOXZWRQRSUY[\^XQR! XA:i:1 MD:Z:48 NM:i:0 CM:i:2 +869_1532_1255/1 16 chrM 3753 255 23M * 0 0 TTTGATAGAGTAAAACATAGAGG YUSVY_UOXZWRQRSUY[\^XQ! XA:i:1 MD:Z:23 NM:i:0 CM:i:1 diff -r b4dd38c06398 -r 5fa69a40e080 test-data/bowtie_out3.sam --- a/test-data/bowtie_out3.sam Tue Feb 23 23:17:09 2010 -0500 +++ b/test-data/bowtie_out3.sam Wed Feb 24 07:11:43 2010 -0500 @@ -1,2 +1,2 @@ -869_1532_1255 179 chrM 3727 255 25M = 3752 50 GGAAATATGTCTGACAAAAGAGTTA !!RVYVSTXTRSNSUSPQYVUTPR; XA:i:1 MD:Z:25 NM:i:0 CM:i:1 -869_1532_1255 115 chrM 3752 255 25M = 3727 -50 CTTTGATAGAGTAAAACATAGAGGC >USVY_UOXZWRQRSUY[\^XQR!! XA:i:1 MD:Z:25 NM:i:0 CM:i:1 +869_1532_1255 179 chrM 3727 255 25M = 3752 50 GGAAATATGTCTGACAAAAGAGTTA !"VRVYVSTXTRSNSUSPQYVUTP8 XA:i:1 MD:Z:25 NM:i:0 CM:i:1 +869_1532_1255 115 chrM 3752 255 25M = 3727 -50 CTTTGATAGAGTAAAACATAGAGGC <YUSVY_UOXZWRQRSUY[\^XQ!! XA:i:1 MD:Z:25 NM:i:0 CM:i:1 diff -r b4dd38c06398 -r 5fa69a40e080 test-data/bowtie_out5.sam --- a/test-data/bowtie_out5.sam Tue Feb 23 23:17:09 2010 -0500 +++ b/test-data/bowtie_out5.sam Wed Feb 24 07:11:43 2010 -0500 @@ -1,1 +1,1 @@ -869_1532_1255/1 16 chrM 3727 255 50M * 0 0 GGAAATATGTCTGACAAAAGAGTTACTTTGATAGAGTAAAACATAGAGGC !"TUVYQPSUSNSRTXTSVYVRVXWYUSVY_UOXZWRQRSUY[\^XQR!! XA:i:1 MD:Z:50 NM:i:0 CM:i:2 +869_1532_1255/1 16 chrM 3752 255 25M * 0 0 CTTTGATAGAGTAAAACATAGAGGC <YUSVY_UOXZWRQRSUY[\^XQ!! XA:i:1 MD:Z:25 NM:i:0 CM:i:1 diff -r b4dd38c06398 -r 5fa69a40e080 test-data/bowtie_out7.sam --- a/test-data/bowtie_out7.sam Tue Feb 23 23:17:09 2010 -0500 +++ b/test-data/bowtie_out7.sam Wed Feb 24 07:11:43 2010 -0500 @@ -1,2 +1,2 @@ -869_1532_1255 179 chrM 3728 255 23M = 3752 47 GAAATATGTCTGACAAAAGAGTT !RVYVSTXTRSNSUSPQYVUTPR XA:i:1 MD:Z:23 NM:i:0 CM:i:1 -869_1532_1255 115 chrM 3753 255 23M = 3727 -49 TTTGATAGAGTAAAACATAGAGG USVY_UOXZWRQRSUY[\^XQR! XA:i:1 MD:Z:23 NM:i:0 CM:i:1 +869_1532_1255 179 chrM 3728 255 23M = 3752 47 GAAATATGTCTGACAAAAGAGTT "VRVYVSTXTRSNSUSPQYVUTP XA:i:1 MD:Z:23 NM:i:0 CM:i:1 +869_1532_1255 115 chrM 3753 255 23M = 3727 -49 TTTGATAGAGTAAAACATAGAGG YUSVY_UOXZWRQRSUY[\^XQ! XA:i:1 MD:Z:23 NM:i:0 CM:i:1 diff -r b4dd38c06398 -r 5fa69a40e080 test-data/sam_to_bam_out1.bam Binary file test-data/sam_to_bam_out1.bam has changed diff -r b4dd38c06398 -r 5fa69a40e080 tools/samtools/sam_merge.xml --- a/tools/samtools/sam_merge.xml Tue Feb 23 23:17:09 2010 -0500 +++ b/tools/samtools/sam_merge.xml Wed Feb 24 07:11:43 2010 -0500 @@ -1,4 +1,4 @@ -<tool id="sam_merge" name="Merge BAM Files" version="1.0.0"> +<tool id="sam_merge" name="Merge BAM Files" version="1.1.0"> <description>merges BAM files together</description> <command interpreter="python"> sam_merge.py diff -r b4dd38c06398 -r 5fa69a40e080 tools/samtools/sam_pileup.xml --- a/tools/samtools/sam_pileup.xml Tue Feb 23 23:17:09 2010 -0500 +++ b/tools/samtools/sam_pileup.xml Wed Feb 24 07:11:43 2010 -0500 @@ -1,4 +1,4 @@ -<tool id="sam_pileup" name="Generate pileup" version="1.0.0"> +<tool id="sam_pileup" name="Generate pileup" version="1.1.0"> <description>from BAM dataset</description> <command interpreter="python"> sam_pileup.py diff -r b4dd38c06398 -r 5fa69a40e080 tools/samtools/sam_to_bam.xml --- a/tools/samtools/sam_to_bam.xml Tue Feb 23 23:17:09 2010 -0500 +++ b/tools/samtools/sam_to_bam.xml Wed Feb 24 07:11:43 2010 -0500 @@ -1,4 +1,4 @@ -<tool id="sam_to_bam" name="SAM-to-BAM" version="1.0.0"> +<tool id="sam_to_bam" name="SAM-to-BAM" version="1.1.0"> <description>converts SAM format to BAM format</description> <command interpreter="python"> sam_to_bam.py --input1=$source.input1 --dbkey=${input1.metadata.dbkey} diff -r b4dd38c06398 -r 5fa69a40e080 tools/sr_mapping/bowtie_color_wrapper.xml --- a/tools/sr_mapping/bowtie_color_wrapper.xml Tue Feb 23 23:17:09 2010 -0500 +++ b/tools/sr_mapping/bowtie_color_wrapper.xml Wed Feb 24 07:11:43 2010 -0500 @@ -197,12 +197,12 @@ <option value="history">Use one from the history</option> </param> <when value="indexed"> - <param name="index" type="select" label="Select the reference genome" help="if your genome of interest is not listed - contact Galaxy team"> - <options from_file="bowtie_indices_color.loc"> - <column name="value" index="1" /> - <column name="name" index="0" /> - </options> - </param> + <param name="index" type="select" label="Select the reference genome" help="if your genome of interest is not listed - contact Galaxy team"> + <options from_file="bowtie_indices_color.loc"> + <column name="value" index="1" /> + <column name="name" index="0" /> + </options> + </param> </when> <when value="history"> <param name="ownFile" type="data" format="fasta" metadata_name="dbkey" label="Select the reference genome" /> @@ -252,7 +252,7 @@ </when>  </conditional>  </when>  - </conditional>  + </conditional>  <conditional name="singlePaired"> <param name="sPaired" type="select" label="Is this library mate-paired?"> <option value="single">Single-end</option> @@ -262,9 +262,9 @@ <param name="sInput1" type="data" format="fastqsanger" label="FASTQ file" help="Must have Sanger-scaled quality values with ASCII offset 33"/> <conditional name="sParams"> <param name="sSettingsType" type="select" label="Bowtie settings to use" help="For most mapping needs use Commonly used settings. If you want full control use Full parameter list"> - <option value="preSet">Commonly used</option> - <option value="full">Full parameter list</option> - </param> + <option value="preSet">Commonly used</option> + <option value="full">Full parameter list</option> + </param> <when value="preSet" /> <when value="full"> <param name="sSkip" type="integer" value="0" label="Skip the first n reads (-s)" /> @@ -327,9 +327,9 @@ </param> <conditional name="pParams"> <param name="pSettingsType" type="select" label="Bowtie settings to use" help="For most mapping needs use Commonly used settings. If you want full control use Full parameter list"> - <option value="preSet">Commonly used</option> - <option value="full">Full parameter list</option> - </param> + <option value="preSet">Commonly used</option> + <option value="full">Full parameter list</option> + </param> <when value="preSet" /> <when value="full"> <param name="pSkip" type="integer" value="0" label="Skip the first n pairs (-s)" /> @@ -417,7 +417,7 @@  @@ -533,8 +533,6 @@ Bowtie_ is a short read aligner designed to be ultrafast and memory-efficient. It is developed by Ben Langmead and Cole Trapnell. Please cite: Langmead B, Trapnell C, Pop M, Salzberg SL. Ultrafast and memory-efficient alignment of short DNA sequences to the human genome. Genome Biology 10:R25. -This tool uses Bowtie version 0.12.1. - .. _Bowtie: http://bowtie-bio.sourceforge.net/index.shtml ------ diff -r b4dd38c06398 -r 5fa69a40e080 tools/sr_mapping/bowtie_wrapper.xml --- a/tools/sr_mapping/bowtie_wrapper.xml Tue Feb 23 23:17:09 2010 -0500 +++ b/tools/sr_mapping/bowtie_wrapper.xml Wed Feb 24 07:11:43 2010 -0500 @@ -191,13 +191,13 @@ <option value="history">Use one from the history</option> </param> <when value="indexed"> - <param name="index" type="select" label="Select a reference genome" help="if your genome of interest is not listed - contact Galaxy team"> - <options from_file="bowtie_indices.loc"> - <column name="value" index="1" /> - <column name="name" index="0" /> - </options> - </param> - </when> + <param name="index" type="select" label="Select a reference genome" help="if your genome of interest is not listed - contact Galaxy team"> + <options from_file="bowtie_indices.loc"> + <column name="value" index="1" /> + <column name="name" index="0" /> + </options> + </param> + </when> <when value="history"> <param name="ownFile" type="data" format="fasta" metadata_name="dbkey" label="Select the reference genome" /> <conditional name="indexParams"> @@ -246,7 +246,7 @@ </when>  </conditional>  </when>  - </conditional>  + </conditional>  <conditional name="singlePaired"> <param name="sPaired" type="select" label="Is this library mate-paired?"> <option value="single">Single-end</option> @@ -256,9 +256,9 @@ <param name="sInput1" type="data" format="fastqsanger" label="FASTQ file" help="Must have Sanger-scaled quality values with ASCII offset 33"/> <conditional name="sParams"> <param name="sSettingsType" type="select" label="Bowtie settings to use" help="For most mapping needs use Commonly used settings. If you want full control use Full parameter list"> - <option value="preSet">Commonly used</option> - <option value="full">Full parameter list</option> - </param> + <option value="preSet">Commonly used</option> + <option value="full">Full parameter list</option> + </param> <when value="preSet" /> <when value="full"> <param name="sSkip" type="integer" value="0" label="Skip the first n reads (-s)" /> @@ -315,9 +315,9 @@ </param> <conditional name="pParams"> <param name="pSettingsType" type="select" label="Bowtie settings to use" help="For most mapping needs use Commonly used settings. If you want full control use Full parameter list"> - <option value="preSet">Commonly used</option> - <option value="full">Full parameter list</option> - </param> + <option value="preSet">Commonly used</option> + <option value="full">Full parameter list</option> + </param> <when value="preSet" /> <when value="full"> <param name="pSkip" type="integer" value="0" label="Skip the first n pairs (-s)" /> @@ -506,8 +506,6 @@ Bowtie_ is a short read aligner designed to be ultrafast and memory-efficient. It is developed by Ben Langmead and Cole Trapnell. Please cite: Langmead B, Trapnell C, Pop M, Salzberg SL. Ultrafast and memory-efficient alignment of short DNA sequences to the human genome. Genome Biology 10:R25. -This tool uses Bowtie version 0.12.1. - .. _Bowtie: http://bowtie-bio.sourceforge.net/index.shtml ------ diff -r b4dd38c06398 -r 5fa69a40e080 tools/sr_mapping/bwa_wrapper.xml --- a/tools/sr_mapping/bwa_wrapper.xml Tue Feb 23 23:17:09 2010 -0500 +++ b/tools/sr_mapping/bwa_wrapper.xml Wed Feb 24 07:11:43 2010 -0500 @@ -181,8 +181,6 @@ BWA is a fast light-weighted tool that aligns relatively short sequences (queries) to a sequence database (large), such as the human reference genome. It is developed by Heng Li at the Sanger Insitute. Li H. and Durbin R. (2009) Fast and accurate short read alignment with Burrows-Wheeler transform. Bioinformatics, 25, 1754-60. -This tool uses BWA version 0.5.5. - ------ **Know what you are doing**

1 0

[hg] galaxy 3432: FASTQ Manupulation tool no longer considers @ ...
by Greg Von Kuster 24 Feb '10

24 Feb '10

details: http://www.bx.psu.edu/hg/galaxy/rev/b4dd38c06398 changeset: 3432:b4dd38c06398 user: Dan Blankenberg <dan(a)bx.psu.edu> date: Tue Feb 23 23:17:09 2010 -0500 description: FASTQ Manupulation tool no longer considers @ when matching or translating sequence identifiers. diffstat: tools/fastq/fastq_manipulation.xml | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-) diffs (36 lines): diff -r 2b0e2519f7ac -r b4dd38c06398 tools/fastq/fastq_manipulation.xml --- a/tools/fastq/fastq_manipulation.xml Tue Feb 23 19:54:54 2010 -0500 +++ b/tools/fastq/fastq_manipulation.xml Tue Feb 23 23:17:09 2010 -0500 @@ -1,4 +1,4 @@ -<tool id="fastq_manipulation" name="Manipulate FASTQ" version="1.0.0"> +<tool id="fastq_manipulation" name="Manipulate FASTQ" version="1.0.1"> <options sanitize="False" />  <description>reads on various attributes</description> <command interpreter="python">fastq_manipulation.py $input_file $fastq_manipulation_file $output_file $output_file.files_path '${input_file.extension[len( 'fastq' ):]}'</command> @@ -168,7 +168,7 @@ def match_read( fastq_read ): #for $match_block in $match_blocks: #if $match_block['match_type']['match_type_selector'] == 'identifier': - search_target = fastq_read.identifier + search_target = fastq_read.identifier[1:] ##don't include @ #elif $match_block['match_type']['match_type_selector'] == 'sequence': search_target = fastq_read.sequence #elif $match_block['match_type']['match_type_selector'] == 'quality': @@ -186,7 +186,7 @@ #for $manipulate_block in $manipulate_blocks: #if $manipulate_block['manipulation_type']['manipulation_type_selector'] == 'identifier': #if $manipulate_block['manipulation_type']['manipulation']['manipulation_selector'] == 'translate': - new_read.identifier = new_read.identifier.translate( maketrans( binascii.unhexlify( "${ binascii.hexlify( str( manipulate_block['manipulation_type']['manipulation']['from'] ) ) }" ), binascii.unhexlify( "${ binascii.hexlify( str( manipulate_block['manipulation_type']['manipulation']['to'] ) ) }" ) ) ) + new_read.identifier = "@%s" % new_read.identifier[1:].translate( maketrans( binascii.unhexlify( "${ binascii.hexlify( str( manipulate_block['manipulation_type']['manipulation']['from'] ) ) }" ), binascii.unhexlify( "${ binascii.hexlify( str( manipulate_block['manipulation_type']['manipulation']['to'] ) ) }" ) ) ) #end if #elif $manipulate_block['manipulation_type']['manipulation_type_selector'] == 'sequence': #if $manipulate_block['manipulation_type']['manipulation']['manipulation_selector'] == 'translate': @@ -236,6 +236,8 @@ #continue #end if #end for + if new_read.description != "+": + new_read.description = "+%s" % new_read.identifier[1:] ##ensure description is still valid return new_read def match_and_manipulate_read( fastq_read ): new_read = fastq_read

1 0

[hg] galaxy 3431: Add gnuplot requirement tag to boxplot tool.
by Greg Von Kuster 24 Feb '10

24 Feb '10

details: http://www.bx.psu.edu/hg/galaxy/rev/2b0e2519f7ac changeset: 3431:2b0e2519f7ac user: Dan Blankenberg <dan(a)bx.psu.edu> date: Tue Feb 23 19:54:54 2010 -0500 description: Add gnuplot requirement tag to boxplot tool. diffstat: tool_conf.xml.sample | 18 +++++++++--------- tools/plotting/boxplot.xml | 3 +++ 2 files changed, 12 insertions(+), 9 deletions(-) diffs (41 lines): diff -r 25de793f99e5 -r 2b0e2519f7ac tool_conf.xml.sample --- a/tool_conf.xml.sample Tue Feb 23 17:58:44 2010 -0500 +++ b/tool_conf.xml.sample Tue Feb 23 19:54:54 2010 -0500 @@ -176,15 +176,15 @@ </section> <section name="NGS: QC and manipulation" id="cshl_library_information"> <label text="Generic FASTQ data" id="new_fastq" /> - <tool file="fastq/fastq_combiner.xml" /> - <tool file="fastq/fastq_groomer.xml" /> - <tool file="fastq/fastq_paired_end_joiner.xml" /> - <tool file="fastq/fastq_paired_end_splitter.xml" /> - <tool file="fastq/fastq_stats.xml" /> - <tool file="fastq/fastq_filter.xml" /> - <tool file="fastq/fastq_trimmer.xml" /> - <tool file="fastq/fastq_manipulation.xml" /> - <tool file="fastq/fastq_to_fasta.xml" /> + <tool file="fastq/fastq_combiner.xml" /> + <tool file="fastq/fastq_groomer.xml" /> + <tool file="fastq/fastq_paired_end_joiner.xml" /> + <tool file="fastq/fastq_paired_end_splitter.xml" /> + <tool file="fastq/fastq_stats.xml" /> + <tool file="fastq/fastq_filter.xml" /> + <tool file="fastq/fastq_trimmer.xml" /> + <tool file="fastq/fastq_manipulation.xml" /> + <tool file="fastq/fastq_to_fasta.xml" /> <label text="Deprecated: Generic FASTQ data" id="fastq" /> <tool file="next_gen_conversion/fastq_gen_conv.xml" /> <tool file="fastx_toolkit/fastq_quality_converter.xml" /> diff -r 25de793f99e5 -r 2b0e2519f7ac tools/plotting/boxplot.xml --- a/tools/plotting/boxplot.xml Tue Feb 23 17:58:44 2010 -0500 +++ b/tools/plotting/boxplot.xml Tue Feb 23 19:54:54 2010 -0500 @@ -1,6 +1,9 @@ <tool id="qual_stats_boxplot" name="Boxplot" version="1.0.0"> <description>of quality statistics</description> <command>gnuplot < '$gnuplot_commands' > '$output_file'</command> + <requirements> + <requirement type="binary">gnuplot</requirement> + </requirements> <inputs> <param name="input_file" type="data" format="tabular" label="Quality Statistics File"/> <param name="title" type="text" value="Box plot in Galaxy" label="Title for plot" size="50"/>

1 0

[hg] galaxy 3430: Refactor grid_base.mako and Fixes #285
by Greg Von Kuster 24 Feb '10

24 Feb '10

details: http://www.bx.psu.edu/hg/galaxy/rev/25de793f99e5 changeset: 3430:25de793f99e5 user: Kanwei Li <kanwei(a)gmail.com> date: Tue Feb 23 17:58:44 2010 -0500 description: Refactor grid_base.mako and Fixes #285 diffstat: templates/grid_base.mako | 1166 +++++++++++++++++++++------------------------ 1 files changed, 546 insertions(+), 620 deletions(-) diffs (1187 lines): diff -r 9dd990a2fa94 -r 25de793f99e5 templates/grid_base.mako --- a/templates/grid_base.mako Tue Feb 23 16:48:07 2010 -0500 +++ b/templates/grid_base.mako Tue Feb 23 17:58:44 2010 -0500 @@ -41,637 +41,563 @@ </%def> <%def name="grid_javascripts()"> - ${h.js("jquery.autocomplete", "autocomplete_tagging" )} - <script type="text/javascript"> - ## TODO: generalize and move into galaxy.base.js - $(document).ready(function() { - init_grid_elements(); - init_grid_controls(); - - // Initalize filter elements. - $('input[type=text]').each( function() - { - $(this).click( function() { $(this).attr('value', '') } ); - }); - }); - ## Can this be moved into base.mako? - %if refresh_frames: - %if 'masthead' in refresh_frames: - ## Refresh masthead == user changes (backward compatibility) - if ( parent.user_changed ) { - %if trans.user: - parent.user_changed( "${trans.user.email}", ${int( app.config.is_admin_user( trans.user ) )} ); - %else: - parent.user_changed( null, false ); - %endif - } - %endif - %if 'history' in refresh_frames: - if ( parent.frames && parent.frames.galaxy_history ) { - parent.frames.galaxy_history.location.href="${h.url_for( controller='root', action='history')}"; - if ( parent.force_right_panel ) { - parent.force_right_panel( 'show' ); - } - } - %endif - %if 'tools' in refresh_frames: - if ( parent.frames && parent.frames.galaxy_tools ) { - parent.frames.galaxy_tools.location.href="${h.url_for( controller='root', action='tool_menu')}"; - if ( parent.force_left_panel ) { - parent.force_left_panel( 'show' ); - } - } - %endif - %endif - - // - // Code to handle grid operations: filtering, sorting, paging, and operations. - // - - // Operations that are async (AJAX) compatible. - var async_ops = new Object(); - %for operation in grid.operations: - %if operation.async_compatible: - async_ops['${operation.label.lower()}'] = "True"; - %endif - %endfor - - // Initialize grid controls - function init_grid_controls() - { - - // Initialize operation buttons. - $('input[name=operation]:submit').each(function() { - $(this).click( function() { - // Get operation name. - var operation_name = $(this).attr("value"); - - // For some reason, $('input[name=id]:checked').val() does not return all ids for checked boxes. - // The code below performs this function. - var item_ids = new Array() - $('input[name=id]:checked').each(function() { - item_ids[item_ids.length] = $(this).val(); - }); - do_operation(operation_name, item_ids); - }); - }); - - // Initialize submit image elements. - $('.submit-image').each( function() - { - // On mousedown, add class to simulate click. - $(this).mousedown( function() { - $(this).addClass('gray-background'); - }); - - // On mouseup, add class to simulate click. - $(this).mouseup( function() { - $(this).removeClass('gray-background'); - }); - - }); - - // Initialize sort links. - $('.sort-link').each( function() - { - var sort_key = $(this).attr('sort_key'); - $(this).click( function() { - set_sort_condition(sort_key); - return false; - }); - - }); - - // Initialize page links. - $('.page-link > a').each( function() - { - var page_num = $(this).attr('page_num'); - $(this).click( function() { - set_page(page_num); - return false; - }); - - }); - $('#show-all-link').click( function() { - set_page('all'); - return false; - }); - - // Initialize categorical filters. - $('.categorical-filter > a').each( function() - { - $(this).click( function() { - var filter_key = $(this).attr('filter_key'); - var filter_val = $(this).attr('filter_val'); - set_categorical_filter(filter_key, filter_val); + ${h.js("jquery.autocomplete", "autocomplete_tagging" )} + <script type="text/javascript"> + ## TODO: generalize and move into galaxy.base.js + $(document).ready(function() { + init_grid_elements(); + init_grid_controls(); + + // Initialize filter elements. + $('input[type=text]').each( function() { + $(this).click( function() { $(this).attr('value', ''); } ); + }); + }); + ## Can this be moved into base.mako? + %if refresh_frames: + %if 'masthead' in refresh_frames: + ## Refresh masthead == user changes (backward compatibility) + if ( parent.user_changed ) { + %if trans.user: + parent.user_changed( "${trans.user.email}", ${int( app.config.is_admin_user( trans.user ) )} ); + %else: + parent.user_changed( null, false ); + %endif + } + %endif + %if 'history' in refresh_frames: + if ( parent.frames && parent.frames.galaxy_history ) { + parent.frames.galaxy_history.location.href="${h.url_for( controller='root', action='history')}"; + if ( parent.force_right_panel ) { + parent.force_right_panel( 'show' ); + } + } + %endif + %if 'tools' in refresh_frames: + if ( parent.frames && parent.frames.galaxy_tools ) { + parent.frames.galaxy_tools.location.href="${h.url_for( controller='root', action='tool_menu')}"; + if ( parent.force_left_panel ) { + parent.force_left_panel( 'show' ); + } + } + %endif + %endif + + // + // Code to handle grid operations: filtering, sorting, paging, and operations. + // + + // Operations that are async (AJAX) compatible. + var async_ops = {}; + %for operation in grid.operations: + %if operation.async_compatible: + async_ops['${operation.label.lower()}'] = "True"; + %endif + %endfor + + // Initialize grid controls + function init_grid_controls() { + // Initialize operation buttons. + $('input[name=operation]:submit').each(function() { + $(this).click( function() { + // Get operation name. + var operation_name = $(this).attr("value"); + + // For some reason, $('input[name=id]:checked').val() does not return all ids for checked boxes. + // The code below performs this function. + var item_ids = []; + $('input[name=id]:checked').each(function() { + item_ids[item_ids.length] = $(this).val(); + }); + do_operation(operation_name, item_ids); + }); + }); + + // Initialize submit image elements. + $('.submit-image').each( function() { + // On mousedown, add class to simulate click. + $(this).mousedown( function() { + $(this).addClass('gray-background'); + }); + + // On mouseup, add class to simulate click. + $(this).mouseup( function() { + $(this).removeClass('gray-background'); + }); + + }); + + // Initialize sort links. + $('.sort-link').each( function() { + var sort_key = $(this).attr('sort_key'); + $(this).click( function() { + set_sort_condition(sort_key); + return false; + }); + + }); + + // Initialize page links. + $('.page-link > a').each( function() { + var page_num = $(this).attr('page_num'); + $(this).click( function() { + set_page(page_num); + return false; + }); + + }); + $('#show-all-link').click( function() { + set_page('all'); + return false; + }); + + // Initialize categorical filters. + $('.categorical-filter > a').each( function() { + $(this).click( function() { + var filter_key = $(this).attr('filter_key'); + var filter_val = $(this).attr('filter_val'); + set_categorical_filter(filter_key, filter_val); + return false; + }); + }); + + // Initialize text filters. + $('.text-filter-form').each( function() { + $(this).submit( function() { + var column_key = $(this).attr('column_key'); + var text_input_obj = $('#input-' + column_key + '-filter'); + var text_input = text_input_obj.val(); + text_input_obj.val(''); + add_filter_condition(column_key, text_input, true); + return false; + }); + }); + + // Initialize autocomplete for text inputs in search UI. + var t = $("#input-tags-filter"); + if (t.length) { + t.autocomplete( "${h.url_for( controller='tag', action='tag_autocomplete_data', item_class='History' )}", + { selectFirst: false, autoFill: false, highlight: false, mustMatch: false }); + } + + var t2 = $("#input-name-filter"); + if (t2.length) { + t2.autocomplete( "${h.url_for( controller='history', action='name_autocomplete_data' )}", + { selectFirst: false, autoFill: false, highlight: false, mustMatch: false }); + } + + // Initialize advanced search toggles. + $('.advanced-search-toggle').each( function() { + $(this).click( function() { + $('#more-search-options').slideToggle('fast'); return false; - }); - }); - - // Initialize text filters. - $('.text-filter-form').each( function() - { - $(this).submit( function() { - var column_key = $(this).attr('column_key'); - var text_input_obj = $('#input-' + column_key + '-filter'); - var text_input = text_input_obj.val(); - text_input_obj.val(''); - add_filter_condition(column_key, text_input, true); - return false; - }); - }); - - // Initialize autocomplete for text inputs in search UI. - var t = $("#input-tags-filter"); - if (t.length) - { - - var autocomplete_options = - { selectFirst: false, autoFill: false, highlight: false, mustMatch: false }; - - t.autocomplete("${h.url_for( controller='tag', action='tag_autocomplete_data', item_class='History' )}", autocomplete_options); - } - - var t2 = $("#input-name-filter"); - if (t2.length) - { - var autocomplete_options = - { selectFirst: false, autoFill: false, highlight: false, mustMatch: false }; - - t2.autocomplete("${h.url_for( controller='history', action='name_autocomplete_data' )}", autocomplete_options); - } - - // Initialize advanced search toggles. - $('.advanced-search-toggle').each( function() - { - $(this).click( function() { - $('#more-search-options').slideToggle('fast'); - return false; - }); - }); - } - - // Initialize grid elements. - function init_grid_elements() - { - // Initialize grid selection checkboxes. - $(".grid").each( function() { - var grid = this; - var checkboxes = $(this).find("input.grid-row-select-checkbox"); - var update = $(this).find( "span.grid-selected-count" ); - update.text(""); - $(checkboxes).each( function() { - $(this).change( function() { - var n = $(checkboxes).filter("[checked]").size(); - update.text( n ); - }); - }) - }); - - // Initialize item labels. - $(".label").each( function() { - // If href has an operation in it, do operation when clicked. Otherwise do nothing. - var href = $(this).attr('href'); - if ( href.indexOf('operation=') != -1 ) - { - $(this).click( function() { - do_operation_from_href( $(this).attr('href') ); - return false; - }); - } - }); - - // Initialize item menu operations. - make_popup_menus(); - } - - // Filter values for categorical filters. - var categorical_filters = new Object(); - %for column in grid.columns: - %if column.filterable is not None and not isinstance( column, TextColumn ): - var ${column.key}_filters = - { - %for i, filter in enumerate( column.get_accepted_filters() ): - %if i > 0: - , - %endif - ${filter.label} : ${h.to_json_string( filter.args )} - %endfor - }; - categorical_filters['${column.key}'] = ${column.key}_filters; - %endif - %endfor - - // Initialize URL args with filter arguments. - var url_args = ${h.to_json_string( cur_filter_dict )}; - - // Place "f-" in front of all filter arguments. - var arg; - for (arg in url_args) - { - value = url_args[arg]; - delete url_args[arg]; - url_args["f-" + arg] = value; - } - - // Add sort argument to URL args. - url_args['sort'] = "${encoded_sort_key}"; - - // Add async keyword to URL args. - url_args['async'] = true; - - // Add page to URL args. - url_args['page'] = ${cur_page_num}; - - var num_pages = ${num_pages}; - - // Go back to page one; this is useful when a filter is applied. - function go_page_one() - { - // Need to go back to page 1 if not showing all. - var cur_page = url_args['page']; - if (cur_page != null && cur_page != undefined && cur_page != 'all') - url_args['page'] = 1; - } - - // Add tag to grid filter. - function add_tag_to_grid_filter(tag_name, tag_value) - { - // Put tag name and value together. - var tag = tag_name + (tag_value != null && tag_value != "" ? ":" + tag_value : ""); - $('#more-search-options').show('fast'); - add_filter_condition("tags", tag, true); - } - - // Add a condition to the grid filter; this adds the condition and refreshes the grid. - function add_filter_condition(name, value, append) - { - // Do nothing is value is empty. - if (value == "") - return false; - - // Update URL arg with new condition. - if (append) - { - // Update or append value. - var cur_val = url_args["f-" + name]; - var new_val; - if (cur_val == null || cur_val == undefined) - { - new_val = value; - } - else if (typeof(cur_val) == "string") - { - if (cur_val == "All") - new_val = value; - else - { - // Replace string with array. - var values = new Array(); - values[0] = cur_val; - values[1] = value; - new_val = values; - } - } - else { - // Current value is an array. - new_val = cur_val; - new_val[new_val.length] = value; - } - url_args["f-" + name] = new_val; - } - else - { - // Replace value. - url_args["f-" + name] = value; - } - - // Add button that displays filter and provides a button to delete it. - var t = $("<span>" + value + - " <a href='#'><img src='${h.url_for('/static/images/delete_tag_icon_gray.png')}'/></a></span>"); - t.addClass('text-filter-val'); - t.click(function() { - // Remove filter condition. - - // Remove visible element. - $(this).remove(); - - // Remove condition from URL args. - var cur_val = url_args["f-" + name]; - if (cur_val == null || cur_val == undefined) - { - // Unexpected. Throw error? - } - else if (typeof(cur_val) == "string") - { - if (cur_val == "All") - { - // Unexpected. Throw error? - } - else - // Remove condition. - delete url_args["f-" + name]; - } - else { - // Current value is an array. - var conditions = cur_val; - var index; - for (index = 0; index < conditions.length; index++) - if (conditions[index] == value) - { - conditions.splice(index, 1); - break; - } - } - - go_page_one(); - update_grid(); - }); - - var container = $('#' + name + "-filtering-criteria"); - container.append(t); - - go_page_one(); - update_grid(); - } - - // Set sort condition for grid. - function set_sort_condition(col_key) - { - // Set new sort condition. New sort is col_key if sorting new column; if reversing sort on - // currently sorted column, sort is reversed. - var cur_sort = url_args['sort']; - var new_sort = col_key; - if ( cur_sort.indexOf( col_key ) != -1) - { - // Reverse sort. - if ( cur_sort.substring(0,1) != '-' ) - new_sort = '-' + col_key; - else - { - // Sort reversed by using just col_key. - } - } - - // Remove sort arrows elements. - $('.sort-arrow').remove() - - // Add sort arrow element to new sort column. - var sort_arrow = "↑"; - if (new_sort.substring(0,1) != '-') - sort_arrow = "↓"; - var t = $("<span>" + sort_arrow + "</span>").addClass('sort-arrow'); - var th = $("#" + col_key + '-header'); - th.append(t); - - // Need to go back to page 1 if not showing all. - var cur_page = url_args['page']; - if (cur_page != null && cur_page != undefined && cur_page != 'all') - url_args['page'] = 1; - - // Update grid. - url_args['sort'] = new_sort; - go_page_one(); - update_grid(); - } - - // Set new value for categorical filter. - function set_categorical_filter(name, new_value) - { - // Update filter hyperlinks to reflect new filter value. - var category_filter = categorical_filters[name]; - var cur_value = url_args["f-" + name]; - $("." + name + "-filter").each( function() { - var text = $.trim( $(this).text() ); - var filter = category_filter[text]; - var filter_value = filter[name]; - if (filter_value == new_value) - { - // Remove filter link since grid will be using this filter. It is assumed that + }); + }); + } + + // Initialize grid elements. + function init_grid_elements() { + // Initialize grid selection checkboxes. + $(".grid").each( function() { + var checkboxes = $(this).find("input.grid-row-select-checkbox"); + var update = $(this).find( "span.grid-selected-count" ); + update.text(""); + $(checkboxes).each( function() { + $(this).change( function() { + var n = $(checkboxes).filter("[checked]").size(); + update.text( n ); + }); + }); + }); + + // Initialize item labels. + $(".label").each( function() { + // If href has an operation in it, do operation when clicked. Otherwise do nothing. + var href = $(this).attr('href'); + if ( href.indexOf('operation=') != -1 ) { + $(this).click( function() { + do_operation_from_href( $(this).attr('href') ); + return false; + }); + } + }); + + // Initialize item menu operations. + make_popup_menus(); + } + + // Filter values for categorical filters. + var categorical_filters = {}; + %for column in grid.columns: + %if column.filterable is not None and not isinstance( column, TextColumn ): + var ${column.key}_filters = ${ h.to_json_string( dict([ (filter.label, filter.args) for filter in column.get_accepted_filters() ]) ) }; + categorical_filters['${column.key}'] = ${column.key}_filters; + %endif + %endfor + + // Initialize URL args with filter arguments. + var url_args = ${h.to_json_string( cur_filter_dict )}; + + // Place "f-" in front of all filter arguments. + $.map(url_args, function(arg) { + return "f-" + arg; + }); + + // Add sort argument to URL args. + url_args['sort'] = "${encoded_sort_key}"; + + // Add async keyword to URL args. + url_args['async'] = true; + + // Add page to URL args. + url_args['page'] = ${cur_page_num}; + + var num_pages = ${num_pages}; + + // Go back to page one; this is useful when a filter is applied. + function go_page_one() { + // Need to go back to page 1 if not showing all. + var cur_page = url_args['page']; + if (cur_page !== null && cur_page !== undefined && cur_page != 'all') { + url_args['page'] = 1; + } + } + + // Add a condition to the grid filter; this adds the condition and refreshes the grid. + function add_filter_condition(name, value, append) { + // Do nothing is value is empty. + if (value == "") { + return false; + } + + // Update URL arg with new condition. + if (append) { + // Update or append value. + var cur_val = url_args["f-" + name]; + var new_val; + if (cur_val === null || cur_val === undefined) { + new_val = value; + } else if (typeof(cur_val) == "string") { + if (cur_val == "All") { + new_val = value; + } else { + // Replace string with array. + var values = []; + values[0] = cur_val; + values[1] = value; + new_val = values; + } + } else { + // Current value is an array. + new_val = cur_val; + new_val[new_val.length] = value; + } + url_args["f-" + name] = new_val; + } else { + // Replace value. + url_args["f-" + name] = value; + } + + // Add button that displays filter and provides a button to delete it. + var t = $("<span>" + value + + " <a href='#'><img src='${h.url_for('/static/images/delete_tag_icon_gray.png')}'/></a></span>"); + t.addClass('text-filter-val'); + t.click(function() { + // Remove filter condition. + + // Remove visible element. + $(this).remove(); + + // Remove condition from URL args. + var cur_val = url_args["f-" + name]; + if (cur_val === null || cur_val === undefined) { + // Unexpected. Throw error? + } else if (typeof(cur_val) == "string") { + if (cur_val == "All") { + // Unexpected. Throw error? + } else { + // Remove condition. + delete url_args["f-" + name]; + } + } else { + // Current value is an array. + var conditions = cur_val; + for (var index = 0; index < conditions.length; index++) { + if (conditions[index] == value) { + conditions.splice(index, 1); + break; + } + } + } + + go_page_one(); + update_grid(); + }); + + var container = $('#' + name + "-filtering-criteria"); + container.append(t); + + go_page_one(); + update_grid(); + } + + // Add tag to grid filter. + function add_tag_to_grid_filter(tag_name, tag_value) { + // Put tag name and value together. + var tag = tag_name + (tag_value !== null && tag_value != "" ? ":" + tag_value : ""); + $('#more-search-options').show('fast'); + add_filter_condition("tags", tag, true); + } + + // Set sort condition for grid. + function set_sort_condition(col_key) { + // Set new sort condition. New sort is col_key if sorting new column; if reversing sort on + // currently sorted column, sort is reversed. + var cur_sort = url_args['sort']; + var new_sort = col_key; + if ( cur_sort.indexOf( col_key ) != -1) { + // Reverse sort. + if ( cur_sort.substring(0,1) != '-' ) { + new_sort = '-' + col_key; + } else { + // Sort reversed by using just col_key. + } + } + + // Remove sort arrows elements. + $('.sort-arrow').remove(); + + // Add sort arrow element to new sort column. + var sort_arrow = (new_sort.substring(0,1) == '-') ? "↑" : "↓"; + var t = $("<span>" + sort_arrow + "</span>").addClass('sort-arrow'); + var th = $("#" + col_key + '-header'); + th.append(t); + + // Need to go back to page 1 if not showing all. + var cur_page = url_args['page']; + if (cur_page !== null && cur_page !== undefined && cur_page != 'all') { + url_args['page'] = 1; + } + // Update grid. + url_args['sort'] = new_sort; + go_page_one(); + update_grid(); + } + + // Set new value for categorical filter. + function set_categorical_filter(name, new_value) { + // Update filter hyperlinks to reflect new filter value. + var category_filter = categorical_filters[name]; + var cur_value = url_args["f-" + name]; + $("." + name + "-filter").each( function() { + var text = $.trim( $(this).text() ); + var filter = category_filter[text]; + var filter_value = filter[name]; + if (filter_value == new_value) { + // Remove filter link since grid will be using this filter. It is assumed that + // this element has a single child, a hyperlink/anchor with text. + $(this).empty(); + $(this).addClass("current-filter"); + $(this).append(text); + } else if (filter_value == cur_value) { + // Add hyperlink for this filter since grid will no longer be using this filter. It is assumed that + // this element has a single child, a hyperlink/anchor. + $(this).empty(); + var t = $("<a href='#'>" + text + "</a>"); + t.click(function() { + set_categorical_filter( name, filter_value ); + }); + $(this).removeClass("current-filter"); + $(this).append(t); + } + }); + + // Update grid. + url_args["f-" + name] = new_value; + go_page_one(); + update_grid(); + } + + // Set page to view. + function set_page(new_page) { + // Update page hyperlink to reflect new page. + $(".page-link").each( function() { + var id = $(this).attr('id'); + var page_num = parseInt( id.split("-")[2] ); // Id has form 'page-link-<page_num> + var cur_page = url_args['page']; + if (page_num == new_page) { + // Remove link to page since grid will be on this page. It is assumed that // this element has a single child, a hyperlink/anchor with text. + var text = $(this).children().text(); $(this).empty(); - $(this).addClass("current-filter"); - $(this).append(text); - } - else if (filter_value == cur_value) - { - // Add hyperlink for this filter since grid will no longer be using this filter. It is assumed that + $(this).addClass("inactive-link"); + $(this).text(text); + } else if (page_num == cur_page) { + // Add hyperlink to this page since grid will no longer be on this page. It is assumed that // this element has a single child, a hyperlink/anchor. + var text = $(this).text(); $(this).empty(); + $(this).removeClass("inactive-link"); var t = $("<a href='#'>" + text + "</a>"); t.click(function() { - set_categorical_filter( name, filter_value ); + set_page(page_num); }); - $(this).removeClass("current-filter"); $(this).append(t); } - }); - - // Update grid. - url_args["f-" + name] = new_value; - go_page_one(); - update_grid(); - } - - // Set page to view. - function set_page(new_page) - { - // Update page hyperlink to reflect new page. - $(".page-link").each( function() { - var id = $(this).attr('id'); - var page_num = parseInt( id.split("-")[2] ); // Id has form 'page-link-<page_num> - var cur_page = url_args['page']; - if (page_num == new_page) - { - // Remove link to page since grid will be on this page. It is assumed that - // this element has a single child, a hyperlink/anchor with text. - var text = $(this).children().text(); - $(this).empty(); - $(this).addClass("inactive-link"); - $(this).text(text); - } - else if (page_num == cur_page) - { - // Add hyperlink to this page since grid will no longer be on this page. It is assumed that - // this element has a single child, a hyperlink/anchor. - var text = $(this).text(); - $(this).empty(); - $(this).removeClass("inactive-link"); - var t = $("<a href='#'>" + text + "</a>"); - t.click(function() { - set_page(page_num); - }); - $(this).append(t); - } - }); - - var maintain_page_links = true; - if (new_page == "all") - { - url_args['page'] = new_page; - maintain_page_links = false; - } - else - url_args['page'] = parseInt(new_page); - update_grid(maintain_page_links); - } - - // Perform a grid operation. - function do_operation(operation, item_ids) - { - operation = operation.toLowerCase(); - - // Update URL args. - url_args['operation'] = operation; - url_args['id'] = item_ids; - - // If operation cannot be performed asynchronously, redirect to location. Otherwise do operation. - var no_async = ( async_ops[operation] == undefined || async_ops[operation] == null); - if (no_async) - { - go_to_URL(); - } - else - { - update_grid(true); - delete url_args['operation']; - delete url_args['id']; - } - } - - // Perform a hyperlink click that initiates an operation. If there is no operation, ignore click. - function do_operation_from_href(href) - { - // Get operation, id in hyperlink's href. - var href_parts = href.split("?"); - if (href_parts.length > 1) - { - var href_parms_str = href_parts[1]; - var href_parms = href_parms_str.split("&"); - var operation = null; - var id = -1; - for (var index = 0; index < href_parms.length; index++) - { - if (href_parms[index].indexOf('operation') != -1) - { - // Found operation parm; get operation value. - operation = href_parms[index].split('=')[1]; - } - else if (href_parms[index].indexOf('id') != -1) - { - // Found id parm; get id value. - id = href_parms[index].split('=')[1]; - } - } - - // Do operation. - do_operation(operation, id); - return false; - } - - } - - // Navigate window to the URL defined by url_args. This method should be used to short-circuit grid AJAXing. - function go_to_URL() - { - // Not async request. - url_args['async'] = false; - - // Build argument string. - var arg_str = ""; - var arg; - for (arg in url_args) - arg_str = arg_str + arg + "=" + url_args[arg] + "&"; - - // Go. - window.location = encodeURI( "${h.url_for()}?" + arg_str ); - } - - // Update grid. - function update_grid(maintain_page_links) - { - ## If grid is not using async, then go to URL. - %if not grid.use_async: + }); + + var maintain_page_links = true; + if (new_page == "all") { + url_args['page'] = new_page; + maintain_page_links = false; + } else { + url_args['page'] = parseInt(new_page); + } + update_grid(maintain_page_links); + } + + // Perform a grid operation. + function do_operation(operation, item_ids) { + operation = operation.toLowerCase(); + + // Update URL args. + url_args['operation'] = operation; + url_args['id'] = item_ids; + + // If operation cannot be performed asynchronously, redirect to location. Otherwise do operation. + var no_async = ( async_ops[operation] === undefined || async_ops[operation] === null); + if (no_async) { go_to_URL(); - return; - %endif - - // If there's an operation in the args, do POST; otherwise, do GET. - var operation = url_args['operation']; - var method = (operation != null && operation != undefined ? "POST" : "GET" ); - $('.loading-elt-overlay').show(); // Show overlay to indicate loading and prevent user actions. - $.ajax({ - type: method, - url: "${h.url_for()}", - data: url_args, - error: function() { alert( "Grid refresh failed" ) }, - success: function(response_text) { - // HACK: use a simple string to separate the elements in the - // response: (1) table body; (2) number of pages in table; and (3) message. - var parsed_response_text = response_text.split("*****"); - - // Update grid body. - var table_body = parsed_response_text[0]; - $('#grid-table-body').html(table_body); - - // Process grid body. - init_grid_elements(); - make_popup_menus(); - - // Update number of pages. - var num_pages = parseInt( parsed_response_text[1] ); - - // Rebuild page links. - if (!maintain_page_links) - { - // Remove page links. - var page_link_container = $('#page-link-container'); - page_link_container.children().remove(); - - // First page is the current page. - var t = $("<span>1</span>"); - t.addClass('page-link'); - t.addClass('inactive-link'); - t.attr('id', 'page-link-1'); - page_link_container.append(t); - - // Show all link is visible only if there are multiple pages. - var elt = $('#show-all-link-span'); - if (num_pages > 1) - elt.show(); - else - elt.hide(); - - // Subsequent pages are navigable. - for (var i = 2; i <= num_pages; i++) - { - var span = $("<span></span>"); - span.addClass('page-link'); - span.attr('id', 'page-link-' + i); - var t = $("<a href='#'>" + i + "</a>"); - t.attr('page', i); - t.click(function() { - var page = $(this).attr('page'); - set_page(page); - }); - span.append(t) - page_link_container.append(span); - } - } - - // Hide loading overlay. - $('.loading-elt-overlay').hide(); - - // Show message if there is one. - var message = $.trim( parsed_response_text[2] ); - if (message != "") - { - $('#grid-message').html( message ); - setTimeout("$('#grid-message').hide()", 5000); - } - } - }); - } + } else { + update_grid(true); + delete url_args['operation']; + delete url_args['id']; + } + } + + // Perform a hyperlink click that initiates an operation. If there is no operation, ignore click. + function do_operation_from_href(href) { + // Get operation, id in hyperlink's href. + var href_parts = href.split("?"); + if (href_parts.length > 1) { + var href_parms_str = href_parts[1]; + var href_parms = href_parms_str.split("&"); + var operation = null; + var id = -1; + for (var index = 0; index < href_parms.length; index++) { + if (href_parms[index].indexOf('operation') != -1) { + // Found operation parm; get operation value. + operation = href_parms[index].split('=')[1]; + } else if (href_parms[index].indexOf('id') != -1) { + // Found id parm; get id value. + id = href_parms[index].split('=')[1]; + } + } + + // Do operation. + do_operation(operation, id); + return false; + } + + } + + // Navigate window to the URL defined by url_args. This method should be used to short-circuit grid AJAXing. + function go_to_URL() { + // Not async request. + url_args['async'] = false; + + // Build argument string. + var arg_str = ""; + for (var arg in url_args) { + arg_str = arg_str + arg + "=" + url_args[arg] + "&"; + } + + // Go. + window.location = encodeURI( "${h.url_for()}?" + arg_str ); + } + + // Update grid. + function update_grid(maintain_page_links) { + ## If grid is not using async, then go to URL. + %if not grid.use_async: + go_to_URL(); + return; + %endif + + // If there's an operation in the args, do POST; otherwise, do GET. + var operation = url_args['operation']; + var method = (operation !== null && operation !== undefined ? "POST" : "GET" ); + $('.loading-elt-overlay').show(); // Show overlay to indicate loading and prevent user actions. + $.ajax({ + type: method, + url: "${h.url_for()}", + data: url_args, + error: function() { alert( "Grid refresh failed" ); }, + success: function(response_text) { + // HACK: use a simple string to separate the elements in the + // response: (1) table body; (2) number of pages in table; and (3) message. + var parsed_response_text = response_text.split("*****"); + + // Update grid body. + var table_body = parsed_response_text[0]; + $('#grid-table-body').html(table_body); + + // Process grid body. + init_grid_elements(); + make_popup_menus(); + + // Update number of pages. + var num_pages = parseInt( parsed_response_text[1] ); + + // Rebuild page links. + if (!maintain_page_links) { + // Remove page links. + var page_link_container = $('#page-link-container'); + page_link_container.children().remove(); + + // First page is the current page. + var t = $("<span>1</span>"); + t.addClass('page-link'); + t.addClass('inactive-link'); + t.attr('id', 'page-link-1'); + page_link_container.append(t); + + // Show all link is visible only if there are multiple pages. + var elt = $('#show-all-link-span'); + if (num_pages > 1) { + elt.show(); + } else { + elt.hide(); + } + + // Subsequent pages are navigable. + for (var i = 2; i <= num_pages; i++) { + var span = $("<span></span>"); + span.addClass('page-link'); + span.attr('id', 'page-link-' + i); + var t = $("<a href='#'>" + i + "</a>"); + t.attr('page', i); + t.click(function() { + var page = $(this).attr('page'); + set_page(page); + }); + span.append(t); + page_link_container.append(span); + } + } + + // Hide loading overlay. + $('.loading-elt-overlay').hide(); + + // Show message if there is one. + var message = $.trim( parsed_response_text[2] ); + if (message != "") { + $('#grid-message').html( message ); + setTimeout( function() { $('#grid-message').hide(); }, 5000); + } + } + }); + } </script> </%def>

1 0

[hg] galaxy 3429: Add a new FASTQ tool suite. Four FASTQ variant...
by Greg Von Kuster 24 Feb '10

24 Feb '10

details: http://www.bx.psu.edu/hg/galaxy/rev/9dd990a2fa94 changeset: 3429:9dd990a2fa94 user: Dan Blankenberg <dan(a)bx.psu.edu> date: Tue Feb 23 16:48:07 2010 -0500 description: Add a new FASTQ tool suite. Four FASTQ variants are supported: sanger, illumina, solexa and solid. Tools include: FASTQ Groomer convert between various FASTQ quality formats Combine FASTA and QUAL into FASTQ FASTQ joiner on paired end reads FASTQ splitter on joined paired end reads FASTQ to FASTA converter FASTQ Summary Statistics by column Filter FASTQ reads by quality score and length FASTQ Trimmer by column Manipulate FASTQ reads on various attributes Boxplot of quality statistics (Generic, with outliers) diffstat: datatypes_conf.xml.sample | 7 +- lib/galaxy/datatypes/converters/interval_to_bedstrict_converter.py | 2 +- lib/galaxy/datatypes/qualityscore.py | 19 +- lib/galaxy/datatypes/sequence.py | 12 + lib/galaxy_utils/sequence/fastq.py | 702 ++++++++++ lib/galaxy_utils/sequence/sequence.py | 61 + lib/galaxy_utils/sequence/transform.py | 74 + test-data/3.fastqsanger | 1 - test-data/boxplot_summary_statistics_out.png | 0 test-data/combine_phiX_out_1.fastqsolid | 576 ++++++++ test-data/combine_phiX_out_2.fastqsolid | 576 ++++++++ test-data/fastq_combiner_in_1.fasta | 14 + test-data/fastq_combiner_in_1.qual454 | 22 + test-data/fastq_stats_1_out.tabular | 37 + test-data/fastq_to_fasta_python_1.out | 4 + test-data/fastq_to_fasta_python_2.out | 4 + test-data/fastq_trimmer_out1.fastqsanger | 8 + test-data/illumina_full_range_as_sanger.fastqsanger | 8 + test-data/illumina_full_range_as_solexa.fastqsolexa | 8 + test-data/illumina_full_range_as_solid.fastqsolid | 8 + test-data/illumina_full_range_original_illumina.fastqillumina | 8 + test-data/sanger_full_range_as_decimal_sanger.fastqsanger | 8 + test-data/sanger_full_range_as_illumina.fastqillumina | 8 + test-data/sanger_full_range_as_rna.fastqsanger | 8 + test-data/sanger_full_range_as_solexa.fastqsolexa | 8 + test-data/sanger_full_range_as_solid.fastqsolid | 8 + test-data/sanger_full_range_original_sanger.fastqsanger | 8 + test-data/sanger_full_range_rev_comp.fastqsanger | 8 + test-data/sanger_full_range_rev_comp_1_seq.fastqsanger | 8 + test-data/solexa_full_range_as_decimal_solexa.fastqsolexa | 8 + test-data/solexa_full_range_as_illumina.fastqillumina | 8 + test-data/solexa_full_range_as_sanger.fastqsanger | 8 + test-data/solexa_full_range_as_solid.fastqsolid | 8 + test-data/solexa_full_range_original_solexa.fastqsolexa | 8 + test-data/split_pair_reads_1.fastqsanger | 20 + test-data/split_pair_reads_2.fastqsanger | 20 + test-data/split_paired_reads_out_1.fastqsanger | 20 - test-data/split_paired_reads_out_2.fastqsanger | 20 - test-data/wrapping_as_illumina.fastqillumina | 12 + test-data/wrapping_as_sanger.fastqsanger | 12 + test-data/wrapping_as_sanger_decimal.fastqsanger | 12 + test-data/wrapping_as_solexa.fastqsolexa | 12 + test-data/wrapping_original_sanger.fastqsanger | 24 + tool_conf.xml.sample | 13 +- tools/fastq/fastq_combiner.py | 45 + tools/fastq/fastq_combiner.xml | 55 + tools/fastq/fastq_filter.py | 34 + tools/fastq/fastq_filter.xml | 311 ++++ tools/fastq/fastq_groomer.py | 37 + tools/fastq/fastq_groomer.xml | 349 ++++ tools/fastq/fastq_manipulation.py | 37 + tools/fastq/fastq_manipulation.xml | 386 +++++ tools/fastq/fastq_paired_end_joiner.py | 38 + tools/fastq/fastq_paired_end_joiner.xml | 55 + tools/fastq/fastq_paired_end_splitter.py | 33 + tools/fastq/fastq_paired_end_splitter.xml | 56 + tools/fastq/fastq_stats.py | 48 + tools/fastq/fastq_stats.xml | 72 + tools/fastq/fastq_to_fasta.py | 21 + tools/fastq/fastq_to_fasta.xml | 33 + tools/fastq/fastq_trimmer.py | 41 + tools/fastq/fastq_trimmer.xml | 113 + tools/metag_tools/split_paired_reads.xml | 4 +- tools/plotting/boxplot.xml | 72 + 64 files changed, 4210 insertions(+), 50 deletions(-) diffs (truncated from 4591 to 3000 lines): diff -r 03dbef081cf2 -r 9dd990a2fa94 datatypes_conf.xml.sample --- a/datatypes_conf.xml.sample Tue Feb 23 16:12:20 2010 -0500 +++ b/datatypes_conf.xml.sample Tue Feb 23 16:48:07 2010 -0500 @@ -1,6 +1,6 @@ <?xml version="1.0"?> <datatypes> - <registration converters_path="lib/galaxy/datatypes/converters"> + <registration converters_path="lib/galaxy/datatypes/converters" display_path="display_applications"> <datatype extension="ab1" type="galaxy.datatypes.binary:Ab1" mimetype="application/octet-stream" display_in_upload="true"/> <datatype extension="axt" type="galaxy.datatypes.sequence:Axt" display_in_upload="true"/> <datatype extension="bam" type="galaxy.datatypes.binary:Bam" mimetype="application/octet-stream" display_in_upload="true"> @@ -34,6 +34,9 @@ </datatype> <datatype extension="fastq" type="galaxy.datatypes.sequence:Fastq" display_in_upload="true"/> <datatype extension="fastqsanger" type="galaxy.datatypes.sequence:FastqSanger" display_in_upload="true"/> + <datatype extension="fastqsolexa" type="galaxy.datatypes.sequence:FastqSolexa" display_in_upload="true"/> + <datatype extension="fastqsolid" type="galaxy.datatypes.sequence:FastqSolid" display_in_upload="true"/> + <datatype extension="fastqillumina" type="galaxy.datatypes.sequence:FastqIllumina" display_in_upload="true"/> <datatype extension="genetrack" type="galaxy.datatypes.tracks:GeneTrack">  </datatype> @@ -60,7 +63,9 @@ </datatype> <datatype extension="pdf" type="galaxy.datatypes.images:Image" mimetype="application/pdf"/> <datatype extension="png" type="galaxy.datatypes.images:Image" mimetype="image/png"/> + <datatype extension="qual" type="galaxy.datatypes.qualityscore:QualityScore" /> <datatype extension="qualsolexa" type="galaxy.datatypes.qualityscore:QualityScoreSolexa" display_in_upload="true"/> + <datatype extension="qualillumina" type="galaxy.datatypes.qualityscore:QualityScoreIllumina" display_in_upload="true"/> <datatype extension="qualsolid" type="galaxy.datatypes.qualityscore:QualityScoreSOLiD" display_in_upload="true"/> <datatype extension="qual454" type="galaxy.datatypes.qualityscore:QualityScore454" display_in_upload="true"/> <datatype extension="sam" type="galaxy.datatypes.tabular:Sam" display_in_upload="true"/> diff -r 03dbef081cf2 -r 9dd990a2fa94 lib/galaxy/datatypes/converters/interval_to_bedstrict_converter.py --- a/lib/galaxy/datatypes/converters/interval_to_bedstrict_converter.py Tue Feb 23 16:12:20 2010 -0500 +++ b/lib/galaxy/datatypes/converters/interval_to_bedstrict_converter.py Tue Feb 23 16:48:07 2010 -0500 @@ -47,7 +47,7 @@ #does file already conform to bed strict? #if so, we want to keep extended columns, otherwise we'll create a generic 6 column bed file strict_bed = True - if extension == 'bed' and ( chromCol, startCol, endCol, nameCol, strandCol ) == ( 0, 1, 2, 3, 5 ): + if extension in [ 'bed', 'bedstrict' ] and ( chromCol, startCol, endCol, nameCol, strandCol ) == ( 0, 1, 2, 3, 5 ): for count, line in enumerate( open( input_name ) ): line = line.strip() if line == "" or line.startswith("#"): diff -r 03dbef081cf2 -r 9dd990a2fa94 lib/galaxy/datatypes/qualityscore.py --- a/lib/galaxy/datatypes/qualityscore.py Tue Feb 23 16:12:20 2010 -0500 +++ b/lib/galaxy/datatypes/qualityscore.py Tue Feb 23 16:48:07 2010 -0500 @@ -9,7 +9,13 @@ log = logging.getLogger(__name__) -class QualityScoreSOLiD ( data.Text ): +class QualityScore ( data.Text ): + """ + until we know more about quality score formats + """ + file_ext = "qual" + +class QualityScoreSOLiD ( QualityScore ): """ until we know more about quality score formats """ @@ -58,7 +64,7 @@ pass return False -class QualityScore454 ( data.Text ): +class QualityScore454 ( QualityScore ): """ until we know more about quality score formats """ @@ -97,9 +103,14 @@ pass return False -class QualityScoreSolexa ( data.Text ): +class QualityScoreSolexa ( QualityScore ): """ until we know more about quality score formats """ file_ext = "qualsolexa" - \ No newline at end of file + +class QualityScoreIllumina ( QualityScore ): + """ + until we know more about quality score formats + """ + file_ext = "qualillumina" diff -r 03dbef081cf2 -r 9dd990a2fa94 lib/galaxy/datatypes/sequence.py --- a/lib/galaxy/datatypes/sequence.py Tue Feb 23 16:12:20 2010 -0500 +++ b/lib/galaxy/datatypes/sequence.py Tue Feb 23 16:48:07 2010 -0500 @@ -216,6 +216,18 @@ """Class representing a FASTQ sequence ( the Sanger variant )""" file_ext = "fastqsanger" +class FastqSolexa( Fastq ): + """Class representing a FASTQ sequence ( the Solexa variant )""" + file_ext = "fastqsolexa" + +class FastqIllumina( Fastq ): + """Class representing a FASTQ sequence ( the Illumina 1.3+ variant )""" + file_ext = "fastqillumina" + +class FastqSolid( Fastq ): + """Class representing a FASTQ sequence ( the SOLiD (color space) variant )""" + file_ext = "fastqsolid" + try: from galaxy import eggs import pkg_resources; pkg_resources.require( "bx-python" ) diff -r 03dbef081cf2 -r 9dd990a2fa94 lib/galaxy_utils/sequence/fastq.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/lib/galaxy_utils/sequence/fastq.py Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,702 @@ +#Dan Blankenberg +import math +import string +import transform +from sequence import SequencingRead + +class fastqSequencingRead( SequencingRead ): + format = 'sanger' #sanger is default + ascii_min = 33 + ascii_max = 126 + quality_min = 0 + quality_max = 93 + score_system = 'phred' #phred or solexa + sequence_space = 'base' #base or color + @classmethod + def get_class_by_format( cls, format ): + assert format in FASTQ_FORMATS, 'Unknown format type specified: %s' % format + return FASTQ_FORMATS[ format ] + @classmethod + def convert_score_phred_to_solexa( cls, decimal_score_list ): + def phred_to_solexa( score ): + if score <= 0: #can't take log10( 1 - 1 ); make <= 0 into -5 + return -5 + return int( round( 10.0 * math.log10( math.pow( 10.0, ( float( score ) / 10.0 ) ) - 1.0 ) ) ) + return map( phred_to_solexa, decimal_score_list ) + @classmethod + def convert_score_solexa_to_phred( cls, decimal_score_list ): + def solexa_to_phred( score ): + return int( round( 10.0 * math.log10( math.pow( 10.0, ( float( score ) / 10.0 ) ) + 1.0 ) ) ) + return map( solexa_to_phred, decimal_score_list ) + @classmethod + def restrict_scores_to_valid_range( cls, decimal_score_list ): + def restrict_score( score ): + return max( min( score, cls.quality_max ), cls.quality_min ) + return map( restrict_score, decimal_score_list ) + @classmethod + def convert_base_to_color_space( cls, sequence ): + return cls.color_space_converter.to_color_space( sequence ) + @classmethod + def convert_color_to_base_space( cls, sequence ): + return cls.color_space_converter.to_base_space( sequence ) + def is_ascii_encoded( self ): + return ' ' not in self.quality #as per fastq definition only decimal quality strings can have spaces in them (and must have a trailing space) + def get_ascii_quality_scores( self ): + if self.is_ascii_encoded(): + return list( self.quality ) + else: + quality = self.quality.rstrip() #decimal scores should have a trailing space + if quality: + try: + return [ chr( int( val ) + self.ascii_min - self.quality_min ) for val in quality.split( ' ' ) ] + except ValueError, e: + raise ValueError( 'Error Parsing quality String. ASCII quality strings cannot contain spaces (%s): %s' % ( self.quality, e ) ) + else: + return [] + def get_decimal_quality_scores( self ): + if self.is_ascii_encoded(): + return [ ord( val ) - self.ascii_min + self.quality_min for val in self.quality ] + else: + quality = self.quality.rstrip() #decimal scores should have a trailing space + if quality: + return map( int, quality.split( ' ' ) ) + else: + return [] + def convert_read_to_format( self, format, force_quality_encoding = None ): + assert format in FASTQ_FORMATS, 'Unknown format type specified: %s' % format + assert force_quality_encoding in [ None, 'ascii', 'decimal' ], 'Invalid force_quality_encoding: %s' % force_quality_encoding + new_class = FASTQ_FORMATS[ format ] + new_read = new_class() + new_read.identifier = self.identifier + if self.sequence_space == new_class.sequence_space: + new_read.sequence = self.sequence + else: + if self.sequence_space == 'base': + new_read.sequence = self.convert_base_to_color_space( self.sequence ) + else: + new_read.sequence = self.convert_color_to_base_space( self.sequence ) + new_read.description = self.description + if self.score_system != new_read.score_system: + if self.score_system == 'phred': + score_list = self.convert_score_phred_to_solexa( self.get_decimal_quality_scores() ) + else: + score_list = self.convert_score_solexa_to_phred( self.get_decimal_quality_scores() ) + else: + score_list = self.get_decimal_quality_scores() + new_read.quality = "%s " % " ".join( map( str, new_class.restrict_scores_to_valid_range( score_list ) ) ) #need trailing space to be valid decimal fastq + if force_quality_encoding is None: + if self.is_ascii_encoded(): + new_encoding = 'ascii' + else: + new_encoding = 'decimal' + else: + new_encoding = force_quality_encoding + if new_encoding == 'ascii': + new_read.quality = "".join( new_read.get_ascii_quality_scores() ) + return new_read + def get_sequence( self ): + return self.sequence + def slice( self, left_column_offset, right_column_offset ): + new_read = fastqSequencingRead.get_class_by_format( self.format )() + new_read.identifier = self.identifier + new_read.sequence = self.get_sequence()[left_column_offset:right_column_offset] + new_read.description = self.description + if self.is_ascii_encoded(): + new_read.quality = self.quality[left_column_offset:right_column_offset] + else: + quality = map( str, self.get_decimal_quality_scores()[left_column_offset:right_column_offset] ) + if quality: + new_read.quality = "%s " % " ".join( quality ) + else: + new_read.quality = '' + return new_read + def is_valid_format( self ): + if self.is_ascii_encoded(): + for val in self.get_ascii_quality_scores(): + val = ord( val ) + if val < self.ascii_min or val > self.ascii_max: + return False + else: + for val in self.get_decimal_quality_scores(): + if val < self.quality_min or val > self.quality_max: + return False + if not self.is_valid_sequence(): + return False + return True + def is_valid_sequence( self ): + for base in self.get_sequence(): + if base not in self.valid_sequence_list: + return False + return True + def insufficient_quality_length( self ): + return len( self.get_ascii_quality_scores() ) < len( self.sequence ) + def assert_sequence_quality_lengths( self ): + qual_len = len( self.get_ascii_quality_scores() ) + seq_len = len( self.sequence ) + assert qual_len == seq_len, "Invalid FASTQ file: quality score length (%i) does not match sequence length (%i)" % ( qual_len, seq_len ) + def reverse( self, clone = True ): + #need to override how decimal quality scores are reversed + if clone: + rval = self.clone() + else: + rval = self + rval.sequence = transform.reverse( self.sequence ) + if rval.is_ascii_encoded(): + rval.quality = rval.quality[::-1] + else: + rval.quality = reversed( rval.get_decimal_quality_scores() ) + rval.quality = "%s " % " ".join( map( str, rval.quality ) ) + return rval + +class fastqSangerRead( fastqSequencingRead ): + format = 'sanger' + ascii_min = 33 + ascii_max = 126 + quality_min = 0 + quality_max = 93 + score_system = 'phred' + sequence_space = 'base' + +class fastqIlluminaRead( fastqSequencingRead ): + format = 'illumina' + ascii_min = 64 + ascii_max = 126 + quality_min = 0 + quality_max = 62 + score_system = 'phred' + sequence_space = 'base' + +class fastqSolexaRead( fastqSequencingRead ): + format = 'solexa' + ascii_min = 59 + ascii_max = 126 + quality_min = -5 + quality_max = 62 + score_system = 'solexa' + sequence_space = 'base' + +class fastqSolidRead( fastqSequencingRead ): + format = 'solid' #color space + ascii_min = 33 + ascii_max = 126 + quality_min = 0 + quality_max = 93 + score_system = 'phred' + sequence_space = 'color' + valid_sequence_list = map( str, range( 7 ) ) + [ '.' ] + def __len__( self ): + if self.has_adapter_base(): #Adapter base is not counted in length of read + return len( self.sequence ) - 1 + return fastqSequencingRead.__len__( self ) + def has_adapter_base( self ): + if self.sequence and self.sequence[0] in string.letters: #adapter base must be a letter + return True + return False + def insufficient_quality_length( self ): + if self.has_adapter_base(): + return len( self.get_ascii_quality_scores() ) + 1 < len( self.sequence ) + return fastqSequencingRead.insufficient_quality_length( self ) + def assert_sequence_quality_lengths( self ): + if self.has_adapter_base(): + qual_len = len( self.get_ascii_quality_scores() ) + seq_len = len( self.sequence ) + assert qual_len + 1 == seq_len, "Invalid FASTQ file: quality score length (%i) does not match sequence length (%i with adapter base)" % ( qual_len, seq_len ) + else: + return fastqSequencingRead.assert_sequence_quality_lengths( self ) + def get_sequence( self ): + if self.has_adapter_base(): + return self.sequence[1:] + return self.sequence + def reverse( self, clone = True ): + #need to override how color space is reversed + if clone: + rval = self.clone() + else: + rval = self + if rval.has_adapter_base(): + adapter = rval.sequence[0] + #sequence = rval.sequence[1:] + rval.sequence = self.color_space_converter.to_color_space( transform.reverse( self.color_space_converter.to_base_space( rval.sequence ) ), adapter_base = adapter ) + else: + rval.sequence = transform.reverse( rval.sequence ) + + if rval.is_ascii_encoded(): + rval.quality = rval.quality[::-1] + else: + rval.quality = reversed( rval.get_decimal_quality_scores() ) + rval.quality = "%s " % " ".join( map( str, rval.quality ) ) + return rval + def complement( self, clone = True ): + #need to override how color space is complemented + if clone: + rval = self.clone() + else: + rval = self + if rval.has_adapter_base(): #No adapter, color space stays the same + adapter = rval.sequence[0] + sequence = rval.sequence[1:] + if adapter.lower() != 'u': + adapter = transform.DNA_complement( adapter ) + else: + adapter = transform.RNA_complement( adapter ) + rval.sequence = "%s%s" % ( adapter, sequence ) + return rval + def change_adapter( self, new_adapter, clone = True ): + #if new_adapter is empty, remove adapter, otherwise replace with new_adapter + if clone: + rval = self.clone() + else: + rval = self + if rval.has_adapter_base(): + if new_adapter: + if new_adapter != rval.sequence[0]: + rval.sequence = rval.color_space_converter.to_color_space( rval.color_space_converter.to_base_space( rval.sequence ), adapter_base = new_adapter ) + else: + rval.sequence = rval.sequence[1:] + elif new_adapter: + rval.sequence = "%s%s" % ( new_adapter, rval.sequence ) + return rval + + +FASTQ_FORMATS = {} +for format in [ fastqIlluminaRead, fastqSolexaRead, fastqSangerRead, fastqSolidRead ]: + FASTQ_FORMATS[ format.format ] = format + + +class fastqAggregator(): + VALID_FORMATS = FASTQ_FORMATS.keys() + def __init__( self, ): + self.ascii_values_used = [] #quick lookup of all ascii chars used + self.seq_lens = {} #counts of seqs by read len + self.nuc_index_quality = [] #counts of scores by read column + self.nuc_index_base = [] #counts of bases by read column + def consume_read( self, fastq_read ): + #ascii values used + for val in fastq_read.get_ascii_quality_scores(): + if val not in self.ascii_values_used: + self.ascii_values_used.append( val ) + #lengths + seq_len = len( fastq_read ) + self.seq_lens[ seq_len ] = self.seq_lens.get( seq_len, 0 ) + 1 + #decimal qualities by column + for i, val in enumerate( fastq_read.get_decimal_quality_scores() ): + if i == len( self.nuc_index_quality ): + self.nuc_index_quality.append( {} ) + self.nuc_index_quality[ i ][ val ] = self.nuc_index_quality[ i ].get( val, 0 ) + 1 + #bases by column + for i, nuc in enumerate( fastq_read.get_sequence() ): + if i == len( self.nuc_index_base ): + self.nuc_index_base.append( {} ) + nuc = nuc.upper() + self.nuc_index_base[ i ][ nuc ] = self.nuc_index_base[ i ].get( nuc, 0 ) + 1 + def get_valid_formats( self, check_list = None ): + if not check_list: + check_list = self.VALID_FORMATS + rval = [] + sequence = [] + for nuc_dict in self.nuc_index_base: + for nuc in nuc_dict.keys(): + if nuc not in sequence: + sequence.append( nuc ) + sequence = "".join( sequence ) + quality = "".join( self.ascii_values_used ) + for fastq_format in check_list: + fastq_read = fastqSequencingRead.get_class_by_format( fastq_format )() + fastq_read.quality = quality + fastq_read.sequence = sequence + if fastq_read.is_valid_format(): + rval.append( fastq_format ) + return rval + def get_ascii_range( self ): + return ( min( self.ascii_values_used ), max( self.ascii_values_used ) ) + def get_decimal_range( self ): + decimal_values_used = [] + for scores in self.nuc_index_quality: + decimal_values_used.extend( scores.keys() ) + return ( min( decimal_values_used ), max( decimal_values_used ) ) + def get_length_counts( self ): + return self.seq_lens + def get_max_read_length( self ): + return len( self.nuc_index_quality ) + def get_read_count_for_column( self, column ): + if column >= len( self.nuc_index_quality ): + return 0 + return sum( self.nuc_index_quality[ column ].values() ) + def get_read_count( self ): + return self.get_read_count_for_column( 0 ) + def get_base_counts_for_column( self, column ): + return self.nuc_index_base[ column ] + def get_score_list_for_column( self, column ): + return self.nuc_index_quality[ column ].keys() + def get_score_min_for_column( self, column ): + return min( self.nuc_index_quality[ column ].keys() ) + def get_score_max_for_column( self, column ): + return max( self.nuc_index_quality[ column ].keys() ) + def get_score_sum_for_column( self, column ): + return sum( score * count for score, count in self.nuc_index_quality[ column ].iteritems() ) + def get_score_at_position_for_column( self, column, position ): + score_value_dict = self.nuc_index_quality[ column ] + scores = sorted( score_value_dict.keys() ) + for score in scores: + if score_value_dict[ score ] <= position: + position -= score_value_dict[ score ] + else: + return score + def get_summary_statistics_for_column( self, i ): + def _get_med_pos( size ): + halfed = int( size / 2 ) + if size % 2 == 1: + return [ halfed ] + return[ halfed - 1, halfed ] + read_count = self.get_read_count_for_column( i ) + + min_score = self.get_score_min_for_column( i ) + max_score = self.get_score_max_for_column( i ) + sum_score = self.get_score_sum_for_column( i ) + mean_score = float( sum_score ) / float( read_count ) + #get positions + med_pos = _get_med_pos( read_count ) + if 0 in med_pos: + q1_pos = [ 0 ] + q3_pos = [ read_count - 1 ] + else: + q1_pos = _get_med_pos( min( med_pos ) ) + q3_pos = [] + for pos in q1_pos: + q3_pos.append( max( med_pos ) + 1 + pos ) + #get scores at position + med_score = float( sum( [ self.get_score_at_position_for_column( i, pos ) for pos in med_pos ] ) ) / float( len( med_pos ) ) + q1 = float( sum( [ self.get_score_at_position_for_column( i, pos ) for pos in q1_pos ] ) ) / float( len( q1_pos ) ) + q3 = float( sum( [ self.get_score_at_position_for_column( i, pos ) for pos in q3_pos ] ) ) / float( len( q3_pos ) ) + #determine iqr and step + iqr = q3 - q1 + step = 1.5 * iqr + + #Determine whiskers and outliers + outliers = [] + score_list = sorted( self.get_score_list_for_column( i ) ) + left_whisker = q1 - step + for score in score_list: + if left_whisker <= score: + left_whisker = score + break + else: + outliers.append( score ) + + right_whisker = q3 + step + score_list.reverse() + for score in score_list: + if right_whisker >= score: + right_whisker = score + break + else: + outliers.append( score ) + + column_stats = { 'read_count': read_count, + 'min_score': min_score, + 'max_score': max_score, + 'sum_score': sum_score, + 'mean_score': mean_score, + 'q1': q1, + 'med_score': med_score, + 'q3': q3, + 'iqr': iqr, + 'left_whisker': left_whisker, + 'right_whisker': right_whisker, + 'outliers': outliers } + return column_stats + +class fastqReader( object ): + def __init__( self, fh, format = 'sanger' ): + self.file = fh + self.format = format + def close( self ): + return self.file.close() + def next(self): + while True: + fastq_header = self.file.readline() + if not fastq_header: + raise StopIteration + fastq_header = fastq_header.rstrip( '\n\r' ) + #remove empty lines, apparently extra new lines at end of file is common? + if fastq_header: + break + + assert fastq_header.startswith( '@' ), 'Invalid fastq header: %s' % fastq_header + rval = fastqSequencingRead.get_class_by_format( self.format )() + rval.identifier = fastq_header + while True: + line = self.file.readline() + if not line: + raise Exception( 'Invalid FASTQ file: could not parse second instance of sequence identifier.' ) + line = line.rstrip( '\n\r' ) + if line.startswith( '+' ) and ( len( line ) == 1 or line[1:].startswith( fastq_header[1:] ) ): + rval.description = line + break + rval.append_sequence( line ) + while rval.insufficient_quality_length(): + line = self.file.readline() + if not line: + break + rval.append_quality( line ) + rval.assert_sequence_quality_lengths() + return rval + def __iter__( self ): + while True: + yield self.next() + +class fastqNamedReader( object ): + def __init__( self, fh, format = 'sanger' ): + self.file = fh + self.format = format + self.reader = fastqReader( self.file, self.format ) + #self.last_offset = self.file.tell() + self.offset_dict = {} + self.eof = False + def close( self ): + return self.file.close() + def get( self, sequence_id ): + rval = None + if sequence_id in self.offset_dict: + initial_offset = self.file.tell() + seq_offset = self.offset_dict[ sequence_id ].pop( 0 ) + if not self.offset_dict[ sequence_id ]: + del self.offset_dict[ sequence_id ] + self.file.seek( seq_offset ) + rval = self.reader.next() + #assert rval.identifier == sequence_id, 'seq id mismatch' #should be able to remove this + self.file.seek( initial_offset ) + else: + while True: + offset = self.file.tell() + try: + fastq_read = self.reader.next() + except StopIteration: + self.eof = True + break #eof, id not found, will return None + if fastq_read.identifier == sequence_id: + rval = fastq_read + break + else: + if fastq_read.identifier not in self.offset_dict: + self.offset_dict[ fastq_read.identifier ] = [] + self.offset_dict[ fastq_read.identifier ].append( offset ) + return rval + def has_data( self ): + #returns a string representation of remaining data, or empty string (False) if no data remaining + eof = self.eof + count = 0 + rval = '' + if self.offset_dict: + count = sum( map( len, self.offset_dict.values() ) ) + if not eof: + offset = self.file.tell() + try: + fastq_read = self.reader.next() + except StopIteration: + eof = True + self.file.seek( offset ) + if count: + rval = "There were %i known sequence reads not utilized. " + if not eof: + rval = "%s%s" % ( rval, "An additional unknown number of reads exist in the input that were not utilized." ) + return rval + +class fastqWriter( object ): + def __init__( self, fh, format = None, force_quality_encoding = None ): + self.file = fh + self.format = format + self.force_quality_encoding = force_quality_encoding + def write( self, fastq_read ): + if self.format: + fastq_read = fastq_read.convert_read_to_format( self.format, force_quality_encoding = self.force_quality_encoding ) + self.file.write( str( fastq_read ) ) + def close( self ): + return self.file.close() + +class fastaWriter( object ): + def __init__( self, fh ): + self.file = fh + def write( self, fastq_read ): + #this will include SOLiD adapter base if applicable + self.file.write( ">%s\n%s\n" % ( fastq_read.identifier[1:], fastq_read.sequence ) ) + def close( self ): + return self.file.close() + +class fastqJoiner( object ): + def __init__( self, format, force_quality_encoding = None ): + self.format = format + self.force_quality_encoding = force_quality_encoding + def join( self, read1, read2 ): + if read1.identifier.endswith( '/2' ) and read2.identifier.endswith( '/1' ): + #swap 1 and 2 + tmp = read1 + read1 = read2 + read2 = tmp + del tmp + if read1.identifier.endswith( '/1' ) and read2.identifier.endswith( '/2' ): + identifier = read1.identifier[:-2] + else: + identifier = read1.identifier + + #use force quality encoding, if not present force to encoding of first read + force_quality_encoding = self.force_quality_encoding + if not force_quality_encoding: + if read1.is_ascii_encoded(): + force_quality_encoding = 'ascii' + else: + force_quality_encoding = 'decimal' + + new_read1 = read1.convert_read_to_format( self.format, force_quality_encoding = force_quality_encoding ) + new_read2 = read2.convert_read_to_format( self.format, force_quality_encoding = force_quality_encoding ) + rval = FASTQ_FORMATS[ self.format ]() + rval.identifier = identifier + if len( read1.description ) > 1: + rval.description = "+%s" % ( identifier[1:] ) + else: + rval.description = '+' + if rval.sequence_space == 'color': + #need to handle color space joining differently + #convert to nuc space, join, then convert back + rval.sequence = rval.convert_base_to_color_space( new_read1.convert_color_to_base_space( new_read1.sequence ) + new_read2.convert_color_to_base_space( new_read2.sequence ) ) + else: + rval.sequence = new_read1.sequence + new_read2.sequence + if force_quality_encoding == 'ascii': + rval.quality = new_read1.quality + new_read2.quality + else: + rval.quality = "%s %s" % ( new_read1.quality.strip(), new_read2.quality.strip() ) + return rval + def get_paired_identifier( self, fastq_read ): + identifier = fastq_read.identifier + if identifier[-2] == '/': + if identifier[-1] == "1": + identifier = "%s2" % identifier[:-1] + elif identifier[-1] == "2": + identifier = "%s1" % identifier[:-1] + return identifier + +class fastqSplitter( object ): + def split( self, fastq_read ): + length = len( fastq_read ) + #Only reads of even lengths can be split + if length % 2 != 0: + return None, None + half = int( length / 2 ) + read1 = fastq_read.slice( 0, half ) + read1.identifier += "/1" + if len( read1.description ) > 1: + read1.description += "/1" + read2 = fastq_read.slice( half, None ) + read2.identifier += "/2" + if len( read2.description ) > 1: + read2.description += "/2" + return read1, read2 + +class fastaSequence( ): + def __init__( self ): + self.identifier = None + self.sequence = '' #holds raw sequence string: no whitespace + def __len__( self ): + return len( self.sequence ) + def __str__( self ): + return "%s\n%s\n" % ( self.identifier, self.sequence ) + +class fastaReader( object ): + def __init__( self, fh ): + self.file = fh + def close( self ): + return self.file.close() + def next( self ): + line = self.file.readline() + #remove header comment lines + while line and line.startswith( '#' ): + line = self.file.readline() + if not line: + raise StopIteration + assert line.startswith( '>' ), "FASTA headers must start with >" + rval = fastaSequence() + rval.identifier = line.strip() + offset = self.file.tell() + while True: + line = self.file.readline() + if not line or line.startswith( '>' ): + if line: + self.file.seek( offset ) + return rval + #454 qual test data that was used has decimal scores that don't have trailing spaces + #so we'll need to parse and build these sequences not based upon de facto standards + #i.e. in a less than ideal fashion + line = line.rstrip() + if ' ' in rval.sequence or ' ' in line: + rval.sequence = "%s%s " % ( rval.sequence, line ) + else: + rval.sequence += line + offset = self.file.tell() + def __iter__( self ): + while True: + yield self.next() + +class fastaNamedReader( object ): + def __init__( self, fh ): + self.file = fh + self.reader = fastaReader( self.file ) + self.offset_dict = {} + self.eof = False + def close( self ): + return self.file.close() + def get( self, sequence_id ): + rval = None + if sequence_id in self.offset_dict: + initial_offset = self.file.tell() + seq_offset = self.offset_dict[ sequence_id ].pop( 0 ) + if not self.offset_dict[ sequence_id ]: + del self.offset_dict[ sequence_id ] + self.file.seek( seq_offset ) + rval = self.reader.next() + self.file.seek( initial_offset ) + else: + while True: + offset = self.file.tell() + try: + fasta_seq = self.reader.next() + except StopIteration: + self.eof = True + break #eof, id not found, will return None + if fasta_seq.identifier == sequence_id: + rval = fasta_seq + break + else: + if fasta_seq.identifier not in self.offset_dict: + self.offset_dict[ fasta_seq.identifier ] = [] + self.offset_dict[ fasta_seq.identifier ].append( offset ) + return rval + def has_data( self ): + #returns a string representation of remaining data, or empty string (False) if no data remaining + eof = self.eof + count = 0 + rval = '' + if self.offset_dict: + count = sum( map( len, self.offset_dict.values() ) ) + if not eof: + offset = self.file.tell() + try: + fasta_seq = self.reader.next() + except StopIteration: + eof = True + self.file.seek( offset ) + if count: + rval = "There were %i known sequences not utilized. " % count + if not eof: + rval = "%s%s" % ( rval, "An additional unknown number of sequences exist in the input that were not utilized." ) + return rval + +class fastqCombiner( object ): + def __init__( self, format ): + self.format = format + def combine(self, fasta_seq, quality_seq ): + fastq_read = fastqSequencingRead.get_class_by_format( self.format )() + fastq_read.identifier = "@%s" % fasta_seq.identifier[1:] + fastq_read.description = '+' + fastq_read.sequence = fasta_seq.sequence + fastq_read.quality = quality_seq.sequence + return fastq_read diff -r 03dbef081cf2 -r 9dd990a2fa94 lib/galaxy_utils/sequence/sequence.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/lib/galaxy_utils/sequence/sequence.py Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,61 @@ +#Dan Blankenberg +import transform +import string +from copy import deepcopy + +class SequencingRead( object ): + color_space_converter = transform.ColorSpaceConverter() + valid_sequence_list = string.letters + def __init__( self ): + self.identifier = None + self.sequence = '' #holds raw sequence string: no whitespace + self.description = None + self.quality = '' #holds raw quality string: no whitespace, unless this contains decimal scores + def __len__( self ): + return len( self.sequence ) + def __str__( self ): + return "%s\n%s\n%s\n%s\n" % ( self.identifier, self.sequence, self.description, self.quality ) + def append_sequence( self, sequence ): + self.sequence += sequence.rstrip( '\n\r' ) + def append_quality( self, quality ): + self.quality += quality.rstrip( '\n\r' ) + def is_DNA( self ): + return 'u' not in self.sequence.lower() + def clone( self ): + return deepcopy( self ) + def reverse( self, clone = True ): + if clone: + rval = self.clone() + else: + rval = self + rval.sequence = transform.reverse( self.sequence ) + rval.quality = rval.quality[::-1] + return rval + def complement( self, clone = True ): + if clone: + rval = self.clone() + else: + rval = self + if rval.is_DNA(): + rval.sequence = transform.DNA_complement( rval.sequence ) + else: + rval.sequence = transform.RNA_complement( rval.sequence ) + return rval + def reverse_complement( self, clone = True ): + #need to reverse first, then complement + rval = self.reverse( clone = clone ) + return rval.complement( clone = False ) #already working with a clone if requested + def sequence_as_DNA( self, clone = True ): + if clone: + rval = self.clone() + else: + rval = self + rval.sequence = transform.to_DNA( rval.sequence ) + return rval + def sequence_as_RNA( self, clone = True ): + if clone: + rval = self.clone() + else: + rval = self + rval.sequence = transform.to_RNA( rval.sequence ) + return rval diff -r 03dbef081cf2 -r 9dd990a2fa94 lib/galaxy_utils/sequence/transform.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/lib/galaxy_utils/sequence/transform.py Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,74 @@ +#Dan Blankenberg +#Contains methods to tranform sequence strings +import string + +#FIXME: This should Handle ambiguity codes... +#Translation table for reverse Complement +DNA_COMPLEMENT = string.maketrans( "ACGTacgt", "TGCAtgca" ) +RNA_COMPLEMENT = string.maketrans( "ACGUacgu", "UGCAugca" ) +#Translation table for DNA <--> RNA +DNA_TO_RNA = string.maketrans( "Tt", "Uu" ) +RNA_TO_DNA = string.maketrans( "Uu", "Tt" ) + +#reverse sequence string +def reverse( sequence ): + return sequence[::-1] +#complement DNA sequence string +def DNA_complement( sequence ): + return sequence.translate( DNA_COMPLEMENT ) +#complement RNA sequence string +def RNA_complement( sequence ): + return sequence.translate( RNA_COMPLEMENT ) +#returns the reverse complement of the sequence +def DNA_reverse_complement( self, sequence ): + sequence = reverse( sequence ) + return DNA_complement( sequence ) +def RNA_reverse_complement( self, sequence ): + sequence = reverse( sequence ) + return RNA_complement( sequence ) +def to_DNA( sequence ): + return sequence.translate( DNA_TO_RNA ) +def to_RNA( sequence ): + return sequence.translate( RNA_TO_DNA ) + +class ColorSpaceConverter( object ): + unknown_base = 'N' + unknown_color = '.' + color_to_base_dict = {} + color_to_base_dict[ 'A' ] = { '0':'A', '1':'C', '2':'G', '3':'T', '4':'N', '5':'N', '6':'N', '.':'N' } + color_to_base_dict[ 'C' ] = { '0':'C', '1':'A', '2':'T', '3':'G', '4':'N', '5':'N', '6':'N', '.':'N' } + color_to_base_dict[ 'G' ] = { '0':'G', '1':'T', '2':'A', '3':'C', '4':'N', '5':'N', '6':'N', '.':'N' } + color_to_base_dict[ 'T' ] = { '0':'T', '1':'G', '2':'C', '3':'A', '4':'N', '5':'N', '6':'N', '.':'N' } + color_to_base_dict[ 'N' ] = { '0':'N', '1':'N', '2':'N', '3':'N', '4':'N', '5':'N', '6':'N', '.':'N' } + base_to_color_dict = {} + for base, color_dict in color_to_base_dict.iteritems(): + base_to_color_dict[ base ] = {} + for key, value in color_dict.iteritems(): + base_to_color_dict[ base ][ value ] = key + base_to_color_dict[ base ][ 'N' ] = '4' #force ACGT followed by N to be '4', because this is now 'processed' data; we could force to '.' (non-processed data) also + base_to_color_dict[ 'N' ].update( { 'A':'5', 'C':'5', 'G':'5', 'T':'5', 'N':'6' } ) + def __init__( self, fake_adapter_base = 'G' ): + assert fake_adapter_base in self.base_to_color_dict, 'A bad fake adapter base was provided: %s.' % fake_adapter_base + self.fake_adapter_base = fake_adapter_base + def to_color_space( self, sequence, adapter_base = None ): + if adapter_base is None: + adapter_base = self.fake_adapter_base + last_base = adapter_base #we add a fake adapter base so that the sequence can be decoded properly again + rval = last_base + for base in sequence: + rval += self.base_to_color_dict.get( last_base, self.base_to_color_dict[ self.unknown_base ] ).get( base, self.unknown_color ) + last_base = base + return rval + def to_base_space( self, sequence ): + if not isinstance( sequence, list ): + sequence = list( sequence ) + if sequence: + last_base = sequence.pop( 0 ) + else: + last_base = None + assert last_base in self.color_to_base_dict, 'A valid adapter base must be included when converting to base space from color space. Found: %s' % last_base + rval = '' + for color_val in sequence: + last_base = self.color_to_base_dict[ last_base ].get( color_val, self.unknown_base ) + rval += last_base + return rval diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/3.fastqsanger --- a/test-data/3.fastqsanger Tue Feb 23 16:12:20 2010 -0500 +++ b/test-data/3.fastqsanger Tue Feb 23 16:48:07 2010 -0500 @@ -18,4 +18,3 @@ TATCAAAAAAGAATATAATCTGAATCAACACTACAACCTATTAGTGTGTAGAATAGGAAGTAGAGGCCTGCG +HWI-EAS91_1_30788AAXX:7:64:947:234 hhhhhhhhhhhhhhhhhhhhhhhRhhehhahhhhhJhhhhhhhh^hPhWfhhhhThWUhhfhh_hhNIVPUd - diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/boxplot_summary_statistics_out.png Binary file test-data/boxplot_summary_statistics_out.png has changed diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/combine_phiX_out_1.fastqsolid --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/combine_phiX_out_1.fastqsolid Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,576 @@ +@1831_573_1004_F3 +T00030133312212111300011021310132222 ++ +4 29 27 34 5 5 24 24 20 17 10 34 29 20 27 13 30 27 22 24 11 28 19 17 27 17 24 17 25 27 7 24 14 12 22 +@1831_573_1050_F3 +T03330322230322112131010221102122113 ++ +8 26 31 31 16 22 30 31 28 29 22 30 30 31 32 23 30 28 28 31 19 32 30 32 19 8 5 10 13 6 5 10 6 16 11 +@1831_573_1067_F3 +T00023032023103330112220321200200002 ++ +13 10 10 2 4 7 6 11 4 14 3 11 10 5 16 2 5 8 11 8 5 10 6 9 6 9 4 2 3 5 2 4 7 6 3 +@1831_573_1219_F3 +T11211130300300301021212330201121310 ++ +31 31 31 28 19 14 10 8 20 8 19 15 23 30 6 21 21 20 29 9 14 20 30 27 21 16 6 26 27 18 11 25 11 20 12 +@1831_573_1242_F3 +T02132003121011302100130302112221121 ++ +29 29 29 25 25 29 16 22 28 32 20 30 31 31 28 26 22 32 28 26 17 13 21 15 29 23 17 27 23 28 22 19 10 26 26 +@1831_573_1333_F3 +T00200312330110101013212313222303112 ++ +8 28 6 31 2 4 18 28 13 29 8 14 18 19 9 16 16 22 11 11 14 21 12 19 10 13 24 22 19 17 19 20 21 27 8 +@1831_573_1362_F3 +T21203131001102231121211101111321131 ++ +34 33 31 31 30 31 31 30 31 34 31 33 32 30 31 29 31 27 31 23 32 31 30 31 6 13 23 6 30 16 22 25 11 10 30 +@1831_573_1448_F3 +T23101211223113320132212331313312022 ++ +22 22 28 32 8 30 11 13 31 32 31 31 25 31 31 27 28 14 22 8 31 27 12 2 23 21 21 17 4 24 21 16 18 5 10 +@1831_573_1490_F3 +T31312310323301210002210123101021011 ++ +31 31 31 31 30 11 31 30 30 31 3 29 27 28 29 10 4 14 9 29 9 16 11 8 30 17 21 5 16 17 6 24 4 22 13 +@1831_573_1523_F3 +T10322001220012223202202222001230222 ++ +7 11 14 14 9 11 2 23 6 6 13 21 6 6 21 6 17 5 17 22 14 23 15 8 4 5 2 15 6 4 2 9 11 17 6 +@1831_573_1578_F3 +T21202302100010020121100311022120111 ++ +31 30 28 31 27 31 34 31 30 31 31 30 30 28 29 31 26 22 27 27 23 27 21 24 19 8 18 20 16 16 7 10 16 27 13 +@1831_573_1647_F3 +T10222233301013033120132223202022123 ++ +28 30 31 31 27 31 31 30 30 30 26 32 28 28 31 31 26 24 30 31 30 18 28 26 21 13 19 14 26 21 25 28 14 26 19 +@1831_573_1684_F3 +T13310013212312012302121010221231123 ++ +22 29 16 25 23 27 31 18 24 31 31 30 30 24 28 26 31 31 27 29 30 31 31 27 26 27 30 22 29 22 21 30 24 29 30 +@1831_573_1769_F3 +T33220123030232212032021032302233131 ++ +22 20 19 22 19 24 5 5 22 21 22 4 14 22 26 18 3 12 22 26 18 2 11 18 14 14 2 11 19 20 14 2 5 15 21 +@1831_573_1853_F3 +T11000012111222211310103212122102331 ++ +21 19 13 14 33 31 31 22 17 33 10 14 19 30 31 30 22 30 10 31 24 14 10 24 24 13 6 8 17 27 17 5 8 17 5 +@1831_573_1943_F3 +T20300123032210232001222122001132111 ++ +7 29 14 9 16 11 8 11 31 12 8 6 15 9 29 20 6 3 14 30 21 31 14 13 5 30 23 14 7 6 8 32 19 18 11 +@1831_573_1977_F3 +T22212302221310332321002303112011311 ++ +31 31 30 21 31 23 31 31 24 32 31 28 19 21 18 18 32 22 30 24 30 27 18 25 10 22 29 6 31 4 31 30 4 18 17 +@1831_574_109_F3 +T13122332123301331032220222133301033 ++ +15 22 21 20 16 2 19 23 16 22 29 31 21 20 27 17 25 23 19 22 16 21 28 22 23 23 27 16 24 24 20 13 22 30 21 +@1831_574_148_F3 +T01200113123030012202302312200010231 ++ +27 26 12 29 18 26 26 18 29 14 14 29 15 26 26 22 28 11 28 19 12 19 14 10 29 15 17 24 27 24 27 21 17 19 26 +@1831_574_185_F3 +T21123333211302300321312212102123121 ++ +31 27 19 26 4 4 31 22 29 2 9 29 24 22 4 21 26 10 23 3 2 27 15 22 4 16 18 22 17 3 4 27 20 19 2 +@1831_574_243_F3 +T30221011230013102201033131203302330 ++ +2 31 31 31 24 20 31 28 24 30 9 29 27 19 20 3 27 31 27 19 4 31 24 30 22 2 28 30 29 22 5 29 30 24 22 +@1831_574_257_F3 +T00301133110002100302003000000102301 ++ +4 29 29 33 29 8 31 30 29 33 31 33 30 31 16 4 9 4 11 2 10 15 15 13 6 7 10 5 20 5 4 2 3 18 5 +@1831_574_293_F3 +T23213210003000103010211331300320130 ++ +5 6 8 22 6 13 9 4 2 5 3 6 14 13 3 2 2 3 13 3 5 9 8 10 9 3 2 23 4 5 4 2 5 5 4 +@1831_574_389_F3 +T21032213032101122333230212301312020 ++ +8 8 3 29 14 29 23 8 25 17 31 25 17 16 18 26 26 13 16 8 31 18 4 21 19 4 10 8 27 22 8 10 3 24 17 +@1831_574_575_F3 +T33313322100212102033032123311211302 ++ +22 30 31 29 28 30 31 29 23 31 31 31 28 31 21 9 4 5 27 28 6 20 31 27 27 6 11 15 27 28 19 9 20 14 27 +@1831_574_592_F3 +T33103330110123102223122023103310330 ++ +22 18 12 20 8 16 24 11 11 27 31 29 19 15 21 22 27 17 13 12 23 21 19 18 19 21 26 24 27 16 14 26 17 16 17 +@1831_574_617_F3 +T20021031221222021210021322200223211 ++ +31 29 31 30 26 29 30 28 31 28 28 31 29 31 28 20 30 31 10 31 25 31 19 15 31 29 27 25 15 8 6 8 21 19 12 +@1831_574_725_F3 +T32010020322130330333010031120313210 ++ +24 6 29 30 20 17 10 20 28 26 19 3 21 26 27 23 13 16 27 19 22 9 24 10 30 22 7 10 20 26 12 8 13 8 19 +@1831_574_734_F3 +T31132301200020012302210322213222222 ++ +13 12 5 13 3 17 5 27 9 7 12 4 8 25 14 8 4 20 25 8 14 4 22 10 11 24 15 18 19 9 32 13 30 22 15 +@1831_574_824_F3 +T30212100033032123311211302122020013 ++ +26 31 31 31 26 26 8 11 10 31 27 12 27 22 26 27 5 23 24 30 30 5 17 24 30 27 3 17 25 32 27 8 26 23 30 +@1831_574_959_F3 +T11212130220131221111002020123311211 ++ +22 32 28 26 10 30 31 24 26 14 27 31 26 27 23 2 5 5 14 9 4 7 4 13 15 3 15 3 17 8 10 16 3 12 16 +@1831_574_1062_F3 +T30112230030300221001032033012211012 ++ +19 31 23 17 16 27 14 14 25 16 18 21 14 23 27 16 14 20 13 19 17 22 13 26 29 17 23 23 14 15 27 16 3 8 14 +@1831_574_1092_F3 +T02013221200031031212200000111130310 ++ +27 23 30 13 11 24 23 8 15 18 4 2 11 11 11 7 11 10 10 3 6 21 24 6 2 24 6 22 2 6 26 21 13 4 17 +@1831_574_1103_F3 +T20313113203302010303131123021310121 ++ +31 30 30 29 30 31 28 29 31 31 32 27 29 30 28 31 28 28 27 25 30 27 26 20 23 6 5 5 8 25 10 18 20 6 16 +@1831_574_1116_F3 +T21011310123202303021021112021231011 ++ +8 31 10 10 11 8 14 8 30 13 14 8 4 27 8 17 29 13 28 28 25 23 30 5 26 19 19 5 14 5 11 8 14 9 19 +@1831_574_1194_F3 +T23303101033322220312200222013013312 ++ +25 20 29 26 11 30 30 24 31 24 31 30 31 16 31 31 28 31 18 28 31 29 31 19 30 31 31 24 19 30 22 20 14 11 21 +@1831_574_1204_F3 +T21330132231321322010303023221203200 ++ +32 32 20 32 20 33 31 20 25 26 31 31 11 13 24 30 32 16 30 2 13 30 26 18 8 11 29 23 17 4 8 28 6 22 9 +@1831_574_1306_F3 +T10332133020311023221213100301001220 ++ +15 17 13 22 5 4 6 6 13 3 5 2 6 4 2 2 3 2 6 2 4 6 8 6 8 3 6 3 4 3 11 8 6 13 16 +@1831_574_1387_F3 +T12301331310032132101301303230121111 ++ +12 30 32 30 30 25 31 30 31 31 24 30 31 31 30 24 34 26 25 29 5 8 13 11 11 8 26 5 6 27 5 22 7 14 6 +@1831_574_1431_F3 +T12011023331022213001123111301312011 ++ +29 22 30 29 24 33 32 23 24 22 26 10 23 9 28 21 2 21 13 10 20 5 12 2 17 21 3 15 13 11 20 5 6 3 16 +@1831_574_1560_F3 +T32212313302203320020222113111011111 ++ +28 8 8 5 21 20 27 19 28 29 24 24 31 25 20 24 22 7 16 5 26 10 5 7 4 5 19 2 11 2 5 5 5 9 3 +@1831_574_1591_F3 +T23202101330322130221230222201123202 ++ +28 28 27 32 29 27 30 31 22 23 31 27 28 26 31 5 31 21 29 23 25 30 11 29 27 5 5 27 22 29 23 7 23 10 24 +@1831_574_1624_F3 +T20122200222132200313011102302210332 ++ +2 27 21 13 5 7 11 22 12 10 8 8 22 13 13 6 14 26 19 18 13 8 24 17 22 13 7 27 23 20 5 16 18 12 8 +@1831_574_1826_F3 +T13012312120112021233030302313201111 ++ +10 31 27 27 11 8 8 19 20 24 27 30 13 10 20 22 6 12 6 5 21 30 16 8 7 26 16 26 15 5 4 19 11 2 3 +@1831_574_1903_F3 +T30232100103132133321330310210101221 ++ +27 24 20 21 30 31 29 26 22 31 31 31 23 25 31 29 23 29 27 28 24 29 24 29 31 17 28 22 16 30 16 25 21 26 14 +@1831_574_1961_F3 +T02333101331223303300200011100032200 ++ +21 29 12 15 22 12 31 29 7 17 6 31 27 2 15 21 19 31 4 20 4 31 31 21 13 5 27 27 7 24 8 28 25 19 22 +@1831_575_54_F3 +T13331330322230200102132110132013200 ++ +29 28 31 31 21 23 27 17 26 27 23 26 30 31 27 29 10 24 29 22 11 11 21 24 22 17 5 20 22 25 19 5 24 27 23 +@1831_575_80_F3 +T33133322233322221003332230323312313 ++ +9 30 20 29 27 31 30 14 28 20 4 7 5 13 20 8 10 2 14 15 6 14 6 29 2 3 14 6 18 22 2 7 8 21 5 +@1831_575_192_F3 +T30013012111133003301010212123302011 ++ +15 25 23 27 11 24 7 13 28 14 8 2 4 23 3 11 19 6 27 11 25 8 18 14 13 6 13 20 27 11 29 12 26 22 6 +@1831_575_197_F3 +T33312113010133020301131330001310032 ++ +11 12 25 30 19 21 18 31 32 25 23 23 30 21 30 19 18 22 27 28 13 17 5 16 13 26 31 21 22 13 19 31 4 17 11 +@1831_575_223_F3 +T10121010002202131221210302100121020 ++ +31 30 29 30 23 30 31 25 30 26 29 10 3 6 4 5 11 10 2 3 13 12 12 2 4 3 16 6 5 2 12 22 3 5 6 +@1831_575_420_F3 +T31110103220000101310112112001020212 ++ +23 17 30 13 29 30 7 12 6 6 17 10 13 13 31 8 11 3 4 16 31 26 6 8 16 8 30 10 8 22 13 6 8 13 13 +@1831_575_434_F3 +T30312132120223101113223301211113311 ++ +27 31 31 29 27 31 33 31 29 28 31 31 30 29 29 27 31 30 26 8 31 30 26 21 13 30 28 19 11 6 6 26 14 18 8 +@1831_575_444_F3 +T33022120112320220100202132332113320 ++ +27 17 31 31 26 27 13 28 30 31 27 10 28 25 32 27 27 31 30 27 27 14 29 31 29 27 7 27 23 29 19 5 20 26 29 +@1831_575_459_F3 +T31330310210101223330110231120131100 ++ +30 31 31 34 31 30 32 31 31 32 31 30 30 32 31 31 6 21 31 31 33 6 28 26 34 31 6 19 27 28 24 11 31 16 13 +@1831_575_506_F3 +T31210200111210121332321310110132301 ++ +23 30 8 17 9 20 31 18 10 13 22 28 30 32 25 23 31 11 14 5 8 25 31 18 22 14 27 23 24 6 10 19 9 8 9 +@1831_575_569_F3 +T10210201321323001012232322323002203 ++ +14 6 24 12 11 31 17 24 10 19 22 24 4 5 29 24 2 5 15 23 28 29 22 2 24 29 11 21 24 14 10 6 9 18 8 +@1831_575_622_F3 +T01100031122111023002323113231210111 ++ +31 32 23 33 28 31 31 30 27 31 30 27 29 29 28 30 24 21 27 19 24 22 8 27 19 14 27 6 25 19 10 26 22 19 10 +@1831_575_644_F3 +T11332003221203131231202200030110130 ++ +27 27 23 29 13 29 5 11 10 8 23 13 16 15 6 25 20 7 14 12 21 17 5 9 13 28 14 4 7 13 27 5 5 16 23 +@1831_575_663_F3 +T32210013303112103322311101322021210 ++ +31 31 31 31 31 32 27 14 26 32 29 31 21 29 15 22 24 17 27 20 24 21 21 25 30 26 17 20 10 25 29 27 16 11 10 +@1831_575_681_F3 +T23131132033020103031013233200101021 ++ +13 22 17 30 13 8 17 20 26 24 21 12 16 19 6 9 13 15 18 6 2 19 2 10 10 3 8 14 23 10 4 14 7 7 5 +@1831_575_711_F3 +T03032331231101231020121210002332121 ++ +19 4 10 25 17 7 2 6 16 10 13 8 5 9 19 20 12 2 11 27 32 12 4 19 26 29 10 8 8 27 26 2 2 9 5 +@1831_575_730_F3 +T31010102200110302123032330331011111 ++ +11 4 23 27 4 12 6 8 20 11 9 16 5 28 2 8 7 16 9 16 8 12 29 22 3 13 11 19 6 19 6 5 6 9 5 +@1831_575_904_F3 +T20111213300020123200333321131121211 ++ +30 27 29 27 13 21 23 24 31 31 22 31 27 20 19 25 19 28 31 29 21 24 28 25 10 21 12 28 19 17 26 23 27 25 2 +@1831_575_938_F3 +T13103102220022130222233301013033120 ++ +26 25 31 31 25 30 31 26 28 23 26 29 28 26 26 25 28 27 24 18 27 19 13 26 27 21 25 17 27 24 26 19 8 23 22 +@1831_575_970_F3 +T23201311301023133303023011202220221 ++ +28 31 28 31 28 26 31 31 29 27 31 31 25 24 27 25 28 30 26 29 31 28 30 24 29 29 30 31 25 17 28 28 28 27 20 +@1831_575_991_F3 +T33312212031111111012212120321121210 ++ +31 29 29 26 22 30 30 30 14 19 27 25 25 25 11 2 5 9 4 6 3 12 12 20 12 2 7 22 8 5 5 11 12 8 4 +@1831_575_1138_F3 +T23320002011320012120333103233301321 ++ +27 28 15 26 4 16 24 18 21 10 30 24 21 30 11 30 20 27 25 17 29 31 21 31 27 25 26 19 29 13 22 26 9 25 16 +@1831_575_1157_F3 +T13121323330203331222022230133102321 ++ +31 33 31 33 29 30 32 30 32 31 30 24 6 30 26 30 5 23 30 31 31 21 26 27 31 28 10 27 25 24 24 10 24 21 20 +@1831_575_1180_F3 +T32003310122102323303101123331133110 ++ +14 27 33 29 20 30 6 25 14 22 23 30 26 16 26 20 20 25 24 24 22 31 31 29 24 19 28 22 27 16 23 21 12 29 22 +@1831_575_1283_F3 +T02232200301300220130032321323131333 ++ +4 34 31 32 20 19 29 27 30 32 22 23 31 27 30 17 29 25 2 29 28 30 31 17 29 23 15 25 6 30 21 24 13 6 22 +@1831_575_1302_F3 +T01201303312333123130200123201013021 ++ +5 16 17 29 14 2 17 6 5 12 17 5 4 6 22 10 3 4 9 2 3 14 16 7 5 5 16 17 14 3 11 17 11 11 9 +@1831_575_1310_F3 +T31332131312021303211310220101211133 ++ +21 27 27 29 21 21 18 30 24 24 31 7 31 22 28 16 16 31 25 31 31 14 22 30 30 27 31 27 26 29 6 18 12 22 26 +@1831_575_1321_F3 +T11001010233200122122022023000203212 ++ +15 26 33 31 30 31 29 28 32 31 27 32 30 29 31 29 30 21 29 31 20 29 6 19 31 24 30 8 22 31 22 30 14 23 24 +@1831_575_1373_F3 +T21213011223311001221321132013121220 ++ +32 31 31 31 29 31 26 28 24 27 31 30 30 22 31 29 31 25 31 26 31 26 16 22 30 21 18 15 14 22 28 19 11 17 27 +@1831_575_1419_F3 +T33222200303001021230212332001013020 ++ +29 14 14 18 12 29 5 12 13 12 29 4 16 16 14 31 5 14 5 22 25 27 11 12 16 28 11 10 17 27 21 6 4 5 27 +@1831_575_1436_F3 +T13210313021212303321202113301220331 ++ +9 17 31 28 26 18 24 9 18 27 9 19 27 25 29 20 17 29 29 24 6 22 25 27 28 6 15 28 29 16 6 16 18 21 27 +@1831_575_1442_F3 +T33132010022331132101132123132020222 ++ +23 29 20 23 27 30 22 29 10 16 33 21 12 14 24 27 27 25 20 30 29 32 25 16 31 30 20 18 24 13 31 19 14 16 29 +@1831_575_1454_F3 +T11131130011012021120222231313211113 ++ +17 19 28 30 23 28 21 20 23 24 28 15 25 32 28 25 30 29 21 27 26 31 14 11 19 10 12 21 10 14 6 13 11 7 11 +@1831_575_1500_F3 +T11010000223111301132313011130103021 ++ +17 30 31 31 30 32 31 30 23 27 21 27 25 24 23 20 28 24 24 24 24 24 28 28 33 18 27 20 27 27 11 22 30 29 24 +@1831_575_1535_F3 +T21312012030320112110211013300131121 ++ +31 33 28 31 27 31 31 33 30 25 32 31 31 28 29 31 30 21 23 31 28 30 31 20 30 22 26 25 21 27 27 26 29 20 27 +@1831_575_1724_F3 +T33123002323300220213232301000010010 ++ +24 27 33 13 10 31 8 11 17 9 13 4 8 8 8 19 4 17 31 26 22 2 4 7 4 10 3 23 8 8 23 20 4 5 20 +@1831_575_1829_F3 +T21033321320111321230233302313101021 ++ +17 17 7 9 6 4 13 18 3 10 22 8 31 5 4 3 6 18 9 10 9 2 14 2 14 9 10 15 13 28 5 2 8 10 15 +@1831_575_1898_F3 +T31330110303103131001110300102101330 ++ +31 33 29 31 31 5 16 14 8 8 6 19 15 8 4 2 23 14 13 4 2 23 3 7 7 2 26 19 6 3 6 21 18 11 11 +@1831_575_1964_F3 +T22010201103202213200201301300232123 ++ +16 29 18 24 8 31 17 27 17 14 31 10 24 30 17 28 5 8 29 29 31 9 21 17 28 20 5 17 27 19 17 13 6 30 10 +@1831_576_32_F3 +T13012100120333032211330300332022110 ++ +10 27 29 30 29 30 28 22 27 17 8 20 17 17 26 29 27 27 31 19 15 31 29 22 15 19 27 29 20 28 17 18 31 10 5 +@1831_576_74_F3 +T30103313210232220102021223012112100 ++ +23 28 31 31 30 31 29 7 30 30 27 33 20 30 31 31 31 24 27 31 29 27 10 29 27 26 31 6 26 29 21 24 21 16 30 +@1831_576_86_F3 +T10320000121033022010011030032211310 ++ +22 23 16 16 8 17 25 9 13 10 10 20 27 14 25 18 10 19 18 24 17 19 9 8 8 14 25 11 21 5 17 24 8 17 14 +@1831_576_89_F3 +T02132333203332020020220033002121120 ++ +17 8 21 24 10 11 6 13 19 28 12 11 29 14 29 7 9 3 2 8 18 15 18 15 9 6 7 11 4 8 17 2 2 3 11 +@1831_576_266_F3 +T30322223101312011300311121221333223 ++ +27 31 31 30 28 26 27 30 22 28 31 29 24 11 29 31 16 3 5 5 23 24 3 14 25 29 22 6 18 16 22 23 4 5 21 +@1831_576_327_F3 +T22112331301313021321001332120332130 ++ +27 31 30 31 21 31 31 26 27 31 30 29 27 31 29 28 28 18 31 25 28 28 27 18 31 31 29 20 18 27 24 29 27 21 28 +@1831_576_331_F3 +T32012133301311223023011232112333030 ++ +26 12 17 8 10 7 9 13 9 16 14 26 20 4 13 12 24 5 2 14 16 6 10 7 3 9 3 2 2 7 8 4 14 3 13 +@1831_576_387_F3 +T00101211032031120300200222001230022 ++ +14 23 8 13 3 20 2 16 2 9 4 13 3 2 2 13 9 2 3 4 2 2 12 4 11 10 11 16 2 5 4 13 4 8 8 +@1831_576_406_F3 +T00223133010210122221320212103132011 ++ +27 19 27 29 4 4 16 19 25 9 19 21 20 21 8 5 27 17 20 16 5 17 10 18 2 5 16 24 11 21 5 19 29 20 7 +@1831_576_449_F3 +T31312001121222231100020132132100220 ++ +7 10 6 5 6 13 11 11 3 14 10 13 8 3 3 23 5 4 2 10 30 5 11 2 8 12 5 2 2 2 22 11 10 2 9 +@1831_576_519_F3 +T03011321130130133213131202130321131 ++ +28 28 19 21 27 24 22 31 29 17 14 21 30 26 17 27 19 32 23 23 16 29 24 16 17 16 10 27 16 14 19 13 24 10 22 +@1831_576_603_F3 +T21003032313302312320131221001330311 ++ +31 31 19 21 28 31 34 13 30 30 27 32 22 24 31 31 26 12 27 31 31 29 17 24 33 30 29 20 20 27 33 22 20 24 23 +@1831_576_655_F3 +T02001023130302322122200313123123102 ++ +28 31 32 32 24 31 31 27 21 9 29 31 31 20 14 27 31 29 24 6 28 26 29 22 10 31 30 24 29 14 24 26 10 11 4 +@1831_576_677_F3 +T13330131023320301031013230210103022 ++ +21 17 24 15 5 14 9 15 2 5 6 5 11 13 17 6 2 5 9 3 5 7 6 2 12 4 7 3 9 2 4 3 8 2 4 +@1831_576_718_F3 +T31232113331022231333313223132231213 ++ +3 19 2 2 2 3 15 2 2 2 4 7 2 2 2 2 2 2 2 2 2 11 2 2 2 2 4 2 2 2 2 3 2 2 2 +@1831_576_722_F3 +T31230320322120231333030031100313200 ++ +7 9 11 12 11 2 2 13 8 9 5 7 9 16 4 9 7 4 7 12 17 2 10 8 12 2 13 5 12 2 4 4 3 6 8 +@1831_576_754_F3 +T30221231132103120112331303112133020 ++ +13 19 14 14 2 7 13 3 8 6 6 11 29 7 5 2 16 3 27 11 2 2 4 7 7 11 2 20 30 2 15 9 4 16 9 +@1831_576_815_F3 +T23022113203032010120310102321001031 ++ +27 18 29 31 30 24 29 31 30 33 28 29 28 26 29 32 30 30 30 28 29 25 17 20 28 19 13 17 20 30 21 27 20 22 13 +@1831_576_882_F3 +T13230020122320223230022031020110122 ++ +30 17 21 30 23 30 31 25 19 29 31 29 24 21 30 30 27 27 28 20 6 16 27 29 24 23 19 21 28 27 24 27 16 29 23 +@1831_576_898_F3 +T10230132312121033222231132231233213 ++ +28 31 31 34 17 30 34 29 30 27 29 31 23 23 21 33 30 26 30 13 30 30 23 22 28 33 27 23 27 16 20 30 30 28 13 +@1831_576_923_F3 +T21322010320202013210121223010123122 ++ +22 21 13 20 16 28 26 13 21 24 24 27 24 21 29 26 26 19 24 27 26 26 16 16 26 27 31 20 24 25 24 28 21 19 22 +@1831_576_930_F3 +T21322103230123110323102012021020013 ++ +17 2 9 15 13 17 5 13 25 7 7 2 6 16 19 6 2 2 12 8 2 4 3 3 17 4 2 3 14 16 5 2 4 14 2 +@1831_576_1019_F3 +T22032121213231032210312001103122312 ++ +17 27 3 29 3 11 16 11 5 10 10 5 31 11 13 8 6 10 14 10 2 24 6 21 24 14 21 6 17 7 10 12 6 24 12 +@1831_576_1068_F3 +T00020232013101330112220321203220211 ++ +25 29 29 27 28 19 30 26 31 22 28 30 30 24 30 26 24 29 20 31 24 30 25 23 31 25 28 20 6 26 22 26 6 2 19 +@1831_576_1131_F3 +T10233122200222132200313011102302210 ++ +16 15 19 31 2 31 14 31 11 19 29 24 30 17 10 30 16 20 22 16 31 6 11 29 28 26 7 22 20 24 26 9 24 17 27 +@1831_576_1168_F3 +T31013300131121323122002113301002010 ++ +29 31 34 34 21 31 31 17 30 24 28 29 22 30 26 22 21 27 26 19 21 22 31 26 24 11 15 4 17 21 6 6 11 19 3 +@1831_576_1207_F3 +T21001132013000122220301213221213010 ++ +4 3 29 17 11 7 5 30 19 30 7 11 31 25 27 5 11 31 29 30 3 5 25 23 32 4 4 28 15 13 4 11 20 24 22 +@1831_576_1289_F3 +T03021210023110200323310302013121203 ++ +31 24 27 29 16 33 31 31 26 22 31 31 21 21 18 28 28 17 23 11 20 6 25 23 27 27 11 13 28 17 29 29 13 20 15 +@1831_576_1329_F3 +T01100302102020113003022000120002100 ++ +27 29 5 15 11 23 22 15 14 32 31 31 14 20 13 26 28 26 25 6 5 31 18 6 5 3 3 4 11 10 2 7 3 5 17 +@1831_576_1367_F3 +T12231310311233110031222013332011023 ++ +30 30 31 31 27 29 31 31 28 31 27 31 27 28 31 29 31 30 30 24 25 30 27 28 29 28 27 23 26 20 24 31 22 23 22 +@1831_576_1416_F3 +T33021233100123120313103133211203221 ++ +10 6 26 30 13 13 16 13 27 31 6 13 28 10 21 13 20 30 22 27 15 12 30 22 26 7 4 28 29 20 21 13 24 23 31 +@1831_576_1461_F3 +T32022221221112233100210223002100100 ++ +2 16 5 22 19 2 7 2 7 5 2 2 3 2 11 6 2 2 2 4 2 4 4 2 9 2 5 4 4 2 2 4 4 4 6 +@1831_576_1605_F3 +T30232100103132133321330310210101221 ++ +21 27 31 34 31 31 29 16 26 31 25 26 27 27 31 31 24 31 30 30 30 24 27 18 30 20 12 17 16 28 19 23 22 22 11 +@1831_576_1664_F3 +T31212101001312110320301201002011120 ++ +31 31 30 29 30 31 28 32 25 31 31 29 31 28 6 28 27 29 28 8 13 29 28 24 9 23 8 7 22 2 14 10 10 26 8 +@1831_576_1671_F3 +T22313332300211322113223102231322313 ++ +31 31 30 31 20 29 31 31 23 5 23 30 17 20 2 5 8 5 20 5 5 8 16 20 5 5 11 5 19 4 5 11 5 21 4 +@1831_576_1729_F3 +T11233312313010012320101302101023030 ++ +31 30 31 30 27 30 27 31 29 32 30 14 28 28 30 29 30 22 29 30 31 23 27 30 31 29 24 24 26 29 27 10 15 28 29 +@1831_576_1880_F3 +T13032121323320213301001310130212003 ++ +15 20 16 20 14 22 24 17 11 25 11 22 14 4 14 15 20 11 4 3 8 25 10 2 23 4 17 7 16 22 20 19 8 8 18 +@1831_576_1982_F3 +T00032312310201201333221212000011030 ++ +11 31 31 30 31 11 32 29 27 31 24 31 28 30 31 24 33 28 23 27 21 31 31 21 31 17 27 32 30 29 13 22 27 10 31 +@1831_576_1987_F3 +T30022313313231221213220132001011320 ++ +8 2 2 7 8 7 2 5 2 2 8 4 8 14 12 2 4 3 16 16 2 4 18 29 6 2 2 5 3 11 2 3 3 6 10 +@1831_576_2014_F3 +T31123201010100321122111102113021003 ++ +31 31 31 24 11 33 17 24 12 20 29 6 30 11 10 30 22 24 10 14 32 6 26 6 17 31 6 20 5 14 24 11 21 5 16 +@1831_576_2028_F3 +T20131211210311112023201213120201100 ++ +20 27 29 24 29 20 6 17 7 5 22 15 22 13 23 2 5 5 18 24 11 15 4 22 14 2 7 2 11 9 4 5 20 9 10 +@1831_577_40_F3 +T11111212330120012020200031313303003 ++ +24 17 14 19 7 6 13 6 8 7 27 9 30 2 3 8 4 5 27 11 14 18 24 27 7 13 17 11 10 27 28 31 21 16 16 +@1831_577_119_F3 +T33111010021103320103213121313000102 ++ +24 29 30 34 16 16 8 12 16 14 8 2 26 2 14 11 23 20 15 9 10 13 10 3 3 20 20 20 15 10 4 12 13 19 15 +@1831_577_133_F3 +T33213323012231300122223032223331322 ++ +14 19 31 29 29 30 28 16 30 25 10 29 31 15 22 31 31 29 27 29 31 9 27 10 16 31 16 20 8 24 21 6 17 3 8 +@1831_577_255_F3 +T00332022110020300332022020202002232 ++ +22 27 28 23 22 31 29 21 24 4 9 9 2 5 2 12 10 3 13 2 5 5 2 10 3 12 10 4 5 4 13 11 4 20 2 +@1831_577_281_F3 +T03032301231212301013112222111210000 ++ +22 12 25 33 26 23 31 28 26 29 22 31 29 19 30 28 30 10 21 20 24 30 26 20 27 22 30 26 24 31 23 7 29 25 30 +@1831_577_288_F3 +T01031120221303100221230021013201130 ++ +20 26 14 22 24 30 30 5 28 33 25 25 17 24 23 9 21 13 22 14 10 19 5 17 16 11 22 11 21 30 13 22 2 6 21 +@1831_577_322_F3 +T12003213220230103303201000130312202 ++ +24 28 26 29 19 20 15 21 26 17 20 20 19 21 19 12 27 2 22 10 16 24 19 5 17 27 30 21 20 24 21 23 8 22 14 +@1831_577_362_F3 +T31203302330110131230331210121110220 ++ +21 17 33 30 25 31 30 31 27 30 24 20 23 28 11 18 25 11 24 15 25 5 6 12 24 24 11 21 27 20 13 7 3 10 21 +@1831_577_382_F3 +T32312123033111120321303230201332100 ++ +16 17 24 20 22 14 25 16 8 8 28 22 21 7 9 17 19 26 11 18 10 25 27 13 5 13 5 12 28 16 28 17 14 20 9 +@1831_577_464_F3 +T13020221011130013102221333131203302 ++ +18 28 27 22 14 27 23 17 21 8 29 2 13 6 5 19 17 15 19 10 20 2 14 15 19 16 13 22 9 24 16 5 22 20 21 +@1831_577_488_F3 +T13200302330322110200323132101120301 ++ +25 19 23 13 11 9 29 21 20 21 21 27 30 23 28 27 28 27 17 29 21 26 24 19 29 26 28 24 29 31 23 24 17 19 31 +@1831_577_511_F3 +T32232133031023313331312220133230333 ++ +2 22 14 10 5 3 25 27 22 4 21 11 3 3 4 6 4 14 10 8 2 3 22 7 7 5 9 18 29 16 21 6 15 14 10 +@1831_577_545_F3 +T00112131333222303222210031322103233 ++ +29 29 27 27 24 30 27 29 30 32 27 19 23 16 31 27 31 23 28 28 31 22 21 14 21 16 27 24 20 13 20 24 23 23 12 +@1831_577_559_F3 +T32321101303233120102011130022122002 ++ +13 27 31 26 30 30 31 29 31 30 14 17 27 13 31 16 28 29 16 17 28 21 16 14 26 28 30 13 5 17 10 24 17 8 6 +@1831_577_562_F3 +T32331101301233110121000220031120031 ++ +2 7 7 5 9 5 15 4 8 16 4 8 2 7 3 17 12 11 9 9 9 4 14 12 11 11 8 8 5 11 12 13 16 6 9 +@1831_577_637_F3 +T22113312122202103031023120301031110 ++ +23 12 11 19 2 29 25 12 21 10 25 23 11 5 7 20 26 18 28 15 29 22 28 21 23 5 16 14 24 5 6 30 26 19 11 +@1831_577_641_F3 +T13031301101121223221212020032131113 ++ +5 19 22 26 14 6 25 32 26 26 20 30 25 22 17 11 7 28 8 11 2 9 30 10 13 2 5 22 3 23 2 4 22 14 6 +@1831_577_692_F3 +T01122320200330103121202301211100220 ++ +31 33 11 31 30 31 33 7 31 32 30 31 10 31 31 29 31 10 30 28 29 31 6 28 27 31 27 28 27 24 28 30 22 20 27 diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/combine_phiX_out_2.fastqsolid --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/combine_phiX_out_2.fastqsolid Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,576 @@ +@1831_573_1004_F3 +T00030133312212111300011021310132222 ++ +%><C&&9952+C>5<.?<79,=42<292:<(9/-7 +@1831_573_1050_F3 +T03330322230322112131010221102122113 ++ +);@@17?@=>7??@A8?==@4A?A4)&+.'&+'1, +@1831_573_1067_F3 +T00023032023103330112220321200200002 ++ +.++#%(',%/$,+&1#&),)&+'*'*%#$&#%('$ +@1831_573_1219_F3 +T11211130300300301021212330201121310 ++ +@@@=4/+)5)408?'665>*/5?<61';<3,:,5- +@1831_573_1242_F3 +T02132003121011302100130302112221121 ++ +>>>::>17=A5?@@=;7A=;2.60>82<8=74+;; +@1831_573_1333_F3 +T00200312330110101013212313222303112 ++ +)='@#%3=.>)/34*117,,/6-4+.9742456<) +@1831_573_1362_F3 +T21203131001102231121211101111321131 ++ +CB@@?@@?@C@BA?@>@<@8A@?@'.8'?17:,+? +@1831_573_1448_F3 +T23101211223113320132212331313312022 ++ +77=A)?,.@A@@:@@<=/7)@<-#8662%9613&+ +@1831_573_1490_F3 +T31312310323301210002210123101021011 ++ +@@@@?,@??@$><=>+%/*>*1,)?26&12'9%7. +@1831_573_1523_F3 +T10322001220012223202202222001230222 ++ +(,//*,#8''.6''6'2&27/80)%&#0'%#*,2' +@1831_573_1578_F3 +T21202302100010020121100311022120111 ++ +@?=@<@C@?@@??=>@;7<<8<694)3511(+1<. +@1831_573_1647_F3 +T10222233301013033120132223202022123 ++ +=?@@<@@???;A==@@;9?@?3=;6.4/;6:=/;4 +@1831_573_1684_F3 +T13310013212312012302121010221231123 ++ +7>1:8<@39@@??9=;@@<>?@@<;<?7>76?9>? +@1831_573_1769_F3 +T33220123030232212032021032302233131 ++ +754749&&767%/7;3$-7;3#,3//#,45/#&06 +@1831_573_1853_F3 +T11000012111222211310103212122102331 ++ +64./B@@72B+/4?@?7?+@9/+99.')2<2&)2& +@1831_573_1943_F3 +T20300123032210232001222122001132111 ++ +(>/*1,),@-)'0*>5'$/?6(a)/.&?8/(')A43, +@1831_573_1977_F3 +T22212302221310332321002303112011311 ++ +@@?6@8@@9A@=4633A7?9?<3:+7>'@%@?%32 +@1831_574_109_F3 +T13122332123301331032220222133301033 ++ +07651#4817>@65<2:84716=788<1995.7?6 +@1831_574_148_F3 +T01200113123030012202302312200010231 ++ +<;->3;;3>//>0;;7=,=4-4/+>029<9<624; +@1831_574_185_F3 +T21123333211302300321312212102123121 ++ +@<4;%%@7>#*>97%6;+8$#<07%1372$%<54# +@1831_574_243_F3 +T30221011230013102201033131203302330 ++ +#@@@95@=9?*><45$<@<4%@9?7#=?>7&>?97 +@1831_574_257_F3 +T00301133110002100302003000000102301 ++ +%>>B>)@?>B@B?@1%*%,#+00.'(+&5&%#$3& +@1831_574_293_F3 +T23213210003000103010211331300320130 ++ +&')7'.*%#&$'/.$##$.$&*)+*$#8%&%#&&% +@1831_574_389_F3 +T21032213032101122333230212301312020 ++ +))$>/>8):2@:213;;.1)@3%64%+)<7)+$92 +@1831_574_575_F3 +T33313322100212102033032123311211302 ++ +7?@>=?@>8@@@=@6*%&<='5@<<',0<=4*5/< +@1831_574_592_F3 +T33103330110123102223122023103310330 ++ +73-5)19,,<@>4067<2.-864346;9<1/;212 +@1831_574_617_F3 +T20021031221222021210021322200223211 ++ +@>@?;>?=@==@>@=5?@+@:@40@><:0)')64- +@1831_574_725_F3 +T32010020322130330333010031120313210 ++ +9'>?52+5=;4$6;<8.1<47*9+?7(+5;-).)4 +@1831_574_734_F3 +T31132301200020012302210322213222222 ++ +.-&.$2&<*(-%):/)%5:)/%7+,9034*A.?70 +@1831_574_824_F3 +T30212100033032123311211302122020013 ++ +;@@@;;),+@<-<7;<&89??&29?<$2:A<);8? +@1831_574_959_F3 +T11212130220131221111002020123311211 ++ +7A=;+?@9;/<@;<8#&&/*%(%.0$0$2)+1$-1 +@1831_574_1062_F3 +T30112230030300221001032033012211012 ++ +4@821<//:136/8<1/5.427.;>288/0<1$)/ +@1831_574_1092_F3 +T02013221200031031212200000111130310 ++ +<8?.,98)03%#,,,(,++$'69'#9'7#';6.%2 +@1831_574_1103_F3 +T20313113203302010303131123021310121 ++ +@??>?@=>@@A<>?=@==<:?<;58'&&):+35'1 +@1831_574_1116_F3 +T21011310123202303021021112021231011 ++ +)(a)++,)/)?./)%<)2>.==:8?&;44&/&,)/*4 +@1831_574_1194_F3 +T23303101033322220312200222013013312 ++ +:5>;,??9@9@?@1@@=@3=@>@4?@@94?75/,6 +@1831_574_1204_F3 +T21330132231321322010303023221203200 ++ +AA5A5B@5:;@@,.9?A1?#.?;3),>82%)='7* +@1831_574_1306_F3 +T10332133020311023221213100301001220 ++ +02.7&%''.$&#'%##$#'#%')')$'$%$,)'.1 +@1831_574_1387_F3 +T12301331310032132101301303230121111 ++ +-?A??:@?@@9?@@?9C;:>&).,,);&'<&7(/' +@1831_574_1431_F3 +T12011023331022213001123111301312011 ++ +>7?>9BA897;+8*=6#6.+5&-#26$0.,5&'$1 +@1831_574_1560_F3 +T32212313302203320020222113111011111 ++ +=))&65<4=>99@:597(1&;+&(%&4#,#&&&*$ +@1831_574_1591_F3 +T23202101330322130221230222201123202 ++ +==<A><?@78@<=;@&@6>8:?,><&&<7>8(8+9 +@1831_574_1624_F3 +T20122200222132200313011102302210332 ++ +#<6.&(,7-+))7..'/;43.)927.(<85&13-) +@1831_574_1826_F3 +T13012312120112021233030302313201111 ++ ++@<<,))459<?.+57'-'&6?1)(;1;0&%4,#$ +@1831_574_1903_F3 +T30232100103132133321330310210101221 ++ +<956?@>;7@@@8:@>8><=9>9>@2=71?1:6;/ +@1831_574_1961_F3 +T02333101331223303300200011100032200 ++ +6>-07-@>(2'@<#064@%5%@@6.&<<(9)=:47 +@1831_575_54_F3 +T13331330322230200102132110132013200 ++ +>=@@68<2;<8;?@<>+9>7,,6972&57:4&9<8 +@1831_575_80_F3 +T33133322233322221003332230323312313 ++ +*?5><@?/=5%(&.5)+#/0'/'>#$/'37#()6& +@1831_575_192_F3 +T30013012111133003301010212123302011 ++ +0:8<,9(.=/)#%8$,4'<,:)3/.'.5<,>-;7' +@1831_575_197_F3 +T33312113010133020301131330001310032 ++ +,-:?463@A:88?6?437<=.2&1.;@67.4@%2, +@1831_575_223_F3 +T10121010002202131221210302100121020 ++ +@?>?8?@:?;>+$'%&,+#$.--#%$1'&#-7$&' +@1831_575_420_F3 +T31110103220000101310112112001020212 ++ +82?.>?(-''2+..@),$%1@;')1)?+)7.').. +@1831_575_434_F3 +T30312132120223101113223301211113311 ++ +<@@><@B@>=@@?>><@?;)@?;6.?=4,'';/3) +@1831_575_444_F3 +T33022120112320220100202132332113320 ++ +<2@@;<.=?@<+=:A<<@?<</>@><(<8>4&5;> +@1831_575_459_F3 +T31330310210101223330110231120131100 ++ +?@@C@?A@@A@??A@@'6@@B'=;C@'4<=9,@1. +@1831_575_506_F3 +T31210200111210121332321310110132301 ++ +8?)2*5@3+.7=?A:8@,/&):@37/<89'+4*)* +@1831_575_569_F3 +T10210201321323001012232322323002203 ++ +/'9-,@29+479%&>9#&08=>7#9>,69/+'*3) +@1831_575_622_F3 +T01100031122111023002323113231210111 ++ +@A8B=@@?<@?<>>=?96<497)<4/<':4+;74+ +@1831_575_644_F3 +T11332003221203131231202200030110130 ++ +<<8>.>&,+)8.10':5(/-62&*.=/%(.<&&18 +@1831_575_663_F3 +T32210013303112103322311101322021210 ++ +@@@@@A</;A>@6>0792<5966:?;25+:><1,+ +@1831_575_681_F3 +T23131132033020103031013233200101021 ++ +.72?.)25;96-14'*.03'#4#++$)/8+%/((& +@1831_575_711_F3 +T03032331231101231020121210002332121 ++ +4%+:2(#'1+.)&*45-#,<A-%4;>+))<;##*& +@1831_575_730_F3 +T31010102200110302123032330331011111 ++ +,%8<%-')5,*1&=#)(1*1)->7$.,4'4'&'*& +@1831_575_904_F3 +T20111213300020123200333321131121211 ++ +?<><.689@@7@<54:4=@>69=:+6-=42;8<:# +@1831_575_938_F3 +T13103102220022130222233301013033120 ++ +;:@@:?@;=8;>=;;:=<93<4.;<6:2<9;4)87 +@1831_575_970_F3 +T23201311301023133303023011202220221 ++ +=@=@=;@@><@@:9<:=?;>@=?9>>?@:2===<5 +@1831_575_991_F3 +T33312212031111111012212120321121210 ++ +@>>;7???/4<:::,#&*%'$--5-#(7)&&,-)% +@1831_575_1138_F3 +T23320002011320012120333103233301321 ++ +<=0;%1936+?96?,?5<:2>@6@<:;4>.7;*:1 +@1831_575_1157_F3 +T13121323330203331222022230133102321 ++ +@B@B>?A?A@?9'?;?&8?@@6;<@=+<:99+965 +@1831_575_1180_F3 +T32003310122102323303101123331133110 ++ +/<B>5?':/78?;1;55:997@@>94=7<186->7 +@1831_575_1283_F3 +T02232200301300220130032321323131333 ++ +%C@A54><?A78@<?2>:#>=?@2>80:'?69.'7 +@1831_575_1302_F3 +T01201303312333123130200123201013021 ++ +&12>/#2'&-2&%'7+$%*#$/1(&&12/$,2,,* +@1831_575_1310_F3 +T31332131312021303211310220101211133 ++ +6<<>663?99@(@7=11@:@@/7??<@<;>'3-7; +@1831_575_1321_F3 +T11001010233200122122022023000203212 ++ +0;B@?@>=A@<A?>@>?6>@5>'4@9?)7@7?/89 +@1831_575_1373_F3 +T21213011223311001221321132013121220 ++ +A@@@>@;=9<@??7@>@:@;@;17?630/7=4,2< +@1831_575_1419_F3 +T33222200303001021230212332001013020 ++ +>//3->&-.->%11/@&/&7:<,-1=,+2<6'%&< +@1831_575_1436_F3 +T13210313021212303321202113301220331 ++ +*2@=;39*3<*4<:>52>>9'7:<='0=>1'136< +@1831_575_1442_F3 +T33132010022331132101132123132020222 ++ +8>58<?7>+1B6-/9<<:5?>A:1@?539.@4/1> +@1831_575_1454_F3 +T11131130011012021120222231313211113 ++ +24=?8=6589=0:A=:?>6<;@/,4+-6+/'.,(, +@1831_575_1500_F3 +T11010000223111301132313011130103021 ++ +2?@@?A@?8<6<:985=99999==B3<5<<,7?>9 +@1831_575_1535_F3 +T21312012030320112110211013300131121 ++ +@B=@<@@B?:A@@=>@?68@=?@5?7;:6<<;>5< +@1831_575_1724_F3 +T33123002323300220213232301000010010 ++ +9<B.+@),2*.%)))4%2@;7#%(%+$8))85%&5 +@1831_575_1829_F3 +T21033321320111321230233302313101021 ++ +22(*'%.3$+7)@&%$'3*+*#/#/*+0.=&#)+0 +@1831_575_1898_F3 +T31330110303103131001110300102101330 ++ +@B>@@&1/))'40)%#8/.%#8$((#;4'$'63,, +@1831_575_1964_F3 +T22010201103202213200201301300232123 ++ +1>39)@2<2/@+9?2=&)>>@*62=5&2<42.'?+ +@1831_576_32_F3 +T13012100120333032211330300332022110 ++ ++<>?>?=7<2)522;><<@40@>704<>5=23@+& +@1831_576_74_F3 +T30103313210232220102021223012112100 ++ +8=@@?@>(??<B5?@@@9<@><+><;@';>6961? +@1831_576_86_F3 +T10320000121033022010011030032211310 ++ +7811)2:*.++5</:3+43924*))/:,6&29)2/ +@1831_576_89_F3 +T02132333203332020020220033002121120 ++ +2)69+,'.4=-,>/>(*$#)3030*'(,%)2##$, +@1831_576_266_F3 +T30322223101312011300311121221333223 ++ +<@@?=;<?7=@>9,>@1$&&89$/:>7'3178%&6 +@1831_576_327_F3 +T22112331301313021321001332120332130 ++ +<@?@6@@;<@?><@>==3@:==<3@@>53<9><6= +@1831_576_331_F3 +T32012133301311223023011232112333030 ++ +;-2)+(*.*1/;5%.-9&#/1'+($*$##()%/$. +@1831_576_387_F3 +T00101211032031120300200222001230022 ++ +/8).$5#1#*%.$##.*#$%##-%,+,1#&%.%)) +@1831_576_406_F3 +T00223133010210122221320212103132011 ++ +<4<>%%14:*4656)&<251&2+3#&19,6&4>5( +@1831_576_449_F3 +T31312001121222231100020132132100220 ++ +(+'&'.,,$/+.)$$8&%#+?&,#)-&###7,+#* +@1831_576_519_F3 +T03011321130130133213131202130321131 ++ +==46<97@>2/6?;2<4A881>9121+<1/4.9+7 +@1831_576_603_F3 +T21003032313302312320131221001330311 ++ +@@46=@C.??<A79@@;-<@@>29B?>55<B7598 +@1831_576_655_F3 +T02001023130302322122200313123123102 ++ +=@AA9@@<6*>@@5/<@>9'=;>7+@?9>/9;+,% +@1831_576_677_F3 +T13330131023320301031013230210103022 ++ +6290&/*0#&'&,.2'#&*$&('#-%($*#%$)#% +@1831_576_718_F3 +T31232113331022231333313223132231213 ++ +$4###$0###%(#########,####%####$### +@1831_576_722_F3 +T31230320322120231333030031100313200 ++ +(*,-,##.)*&(*1%*(%(-2#+)-#.&-#%%$') +@1831_576_754_F3 +T30221231132103120112331303112133020 ++ +.4//#(.$)'',>(&#1$<,##%((,#5?#0*%1* +@1831_576_815_F3 +T23022113203032010120310102321001031 ++ +<3>@?9>@?B=>=;>A???=>:25=4.25?6<57. +@1831_576_882_F3 +T13230020122320223230022031020110122 ++ +?26?8?@:4>@>96??<<=5'1<>9846=<9<1>8 +@1831_576_898_F3 +T10230132312121033222231132231233213 ++ +=@@C2?C>?<>@886B?;?.??87=B<8<15??=. +@1831_576_923_F3 +T21322010320202013210121223010123122 ++ +76.51=;.699<96>;;49<;;11;<@59:9=647 +@1831_576_930_F3 +T21322103230123110323102012021020013 ++ +2#*0.2&.:((#'14'##-)#%$$2%#$/1&#%/# +@1831_576_1019_F3 +T22032121213231032210312001103122312 ++ +2<$>$,1,&++&@,.)'+/+#9'69/6'2(+-'9- +@1831_576_1068_F3 +T00020232013101330112220321203220211 ++ +:>><=4?;@7=??9?;9>5@9?:8@:=5';7;'#4 +@1831_576_1131_F3 +T10233122200222132200313011102302210 ++ +104@#@/@,4>9?2+?1571@',>=;(759;*92< +@1831_576_1168_F3 +T31013300131121323122002113301002010 ++ +>@CC6@@2?9=>7?;76<;467@;9,0%26'',4$ +@1831_576_1207_F3 +T21001132013000122220301213221213010 ++ +%$>2,(&?4?(,@:<&,@>?$&:8A%%=0.%,597 +@1831_576_1289_F3 +T03021210023110200323310302013121203 ++ +@9<>1B@@;7@@663==28,5':8<<,.=2>>.50 +@1831_576_1329_F3 +T01100302102020113003022000120002100 ++ +<>&0,870/A@@/5.;=;:'&@3'&$$%,+#($&2 +@1831_576_1367_F3 +T12231310311233110031222013332011023 ++ +??@@<>@@=@<@<=@>@??9:?<=>=<8;59@787 +@1831_576_1416_F3 +T33021233100123120313103133211203221 ++ ++';?..1.<@'.=+6.5?7<0-?7;(%=>56.98@ +@1831_576_1461_F3 +T32022221221112233100210223002100100 ++ +#1&74#(#(&##$#,'###%#%%#*#&%%##%%%' +@1831_576_1605_F3 +T30232100103132133321330310210101221 ++ +6<@C@@>1;@:;<<@@9@???9<3?5-21=4877, +@1831_576_1664_F3 +T31212101001312110320301201002011120 ++ +@@?>?@=A:@@>@='=<>=).>=9*8)(7#/++;) +@1831_576_1671_F3 +T22313332300211322113223102231322313 ++ +@@?@5>@@8&8?25#&)&5&&)15&&,&4%&,&6% +@1831_576_1729_F3 +T11233312313010012320101302101023030 ++ +@?@?<?<@>A?/==?>?7>?@8<?@>99;><+0=> +@1831_576_1880_F3 +T13032121323320213301001310130212003 ++ +0515/792,:,7/%/05,%$):+#8%2(1754))3 +@1831_576_1982_F3 +T00032312310201201333221212000011030 ++ +,@@?@,A><@9@=?@9B=8<6@@6@2<A?>.7<+@ +@1831_576_1987_F3 +T30022313313231221213220132001011320 ++ +)##()(#&##)%)/-#%$11#%3>'##&$,#$$'+ +@1831_576_2014_F3 +T31123201010100321122111102113021003 ++ +@@@9,B29-5>'?,+?79+/A';'2@'5&/9,6&1 +@1831_576_2028_F3 +T20131211210311112023201213120201100 ++ +5<>9>5'2(&707.8#&&39,0%7/#(#,*%&5*+ +@1831_577_40_F3 +T11111212330120012020200031313303003 ++ +92/4('.')(<*?#$)%&<,/39<(.2,+<=@611 +@1831_577_119_F3 +T33111010021103320103213121313000102 ++ +9>?C11)-1/)#;#/,850*+.+$$5550+%-.40 +@1831_577_133_F3 +T33213323012231300122223032223331322 ++ +/4@>>?=1?:+>@07@@><>@*<+1@15)96'2$) +@1831_577_255_F3 +T00332022110020300332022020202002232 ++ +7<=87@>69%**#&#-+$.#&&#+$-+%&%.,%5# +@1831_577_281_F3 +T03032301231212301013112222111210000 ++ +7-:B;8@=;>7@>4?=?+659?;5<7?;9@8(>:? +@1831_577_288_F3 +T01031120221303100221230021013201130 ++ +5;/79??&=B::298*6.7/+4&21,7,6?.7#'6 +@1831_577_322_F3 +T12003213220230103303201000130312202 ++ +9=;>4506;255464-<#7+194&2<?65968)7/ +@1831_577_362_F3 +T31203302330110131230331210121110220 ++ +62B?:@?@<?958=,3:,90:&'-99,6<5.($+6 +@1831_577_382_F3 +T32312123033111120321303230201332100 ++ +12957/:1))=76(*24;,3+:<.&.&-=1=2/5* +@1831_577_464_F3 +T13020221011130013102221333131203302 ++ +3=<7/<826)>#.'&4204+5#/041.7*91&756 +@1831_577_488_F3 +T13200302330322110200323132101120301 ++ +:48.,*>6566<?8=<=<2>6;94>;=9>@8924@ +@1831_577_511_F3 +T32232133031023313331312220133230333 ++ +#7/+&$:<7%6,$$%'%/+)#$7((&*3>16'0/+ +@1831_577_545_F3 +T00112131333222303222210031322103233 ++ +>><<9?<>?A<481@<@8==@76/61<95.5988- +@1831_577_559_F3 +T32321101303233120102011130022122002 ++ +.<@;??@>@?/2<.@1=>12=61/;=?.&2+92)' +@1831_577_562_F3 +T32331101301233110121000220031120031 ++ +#((&*&0%)1%)#($2-,***%/-,,))&,-.1'* +@1831_577_637_F3 +T22113312122202103031023120301031110 ++ +8-,4#>:-6+:8,&(5;3=0>7=68&1/9&'?;4, +@1831_577_641_F3 +T13031301101121223221212020032131113 ++ +&47;/':A;;5?:72,(=),#*?+.#&7$8#%7/' +@1831_577_692_F3 +T01122320200330103121202301211100220 ++ +@B,@?@B(@A?@+@@>@+?=>@'=<@<=<9=?75< diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/fastq_combiner_in_1.fasta --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/fastq_combiner_in_1.fasta Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,14 @@ +>SRR014849.50939 EIXKN4201BA2EC length=135 +GAAATTTCAGGGCCACCTTTTTTTT +GATAGAATAATGGAGAAAATTAAAAGCTGTACATATACCAATGAACAATAAATCAATACATAAAAAAGGA +GAAGTTGGAACCGAAAGGGTTTGAATTCAAACCCTTTCGG +>SRR014849.110027 EIXKN4201APUB0 length=131 +CTTCAAATGATTC +CGGGACTGTTGGAACCGAAAGGGTTTGAATTCAAACCC +TTTTCGGTTCCAACTCGCCGTCCGAATAATCCGTTCAAAATCTTGGCCTGTCAAAACGACTTTACGAC +CAGAACGATCCG +>SRR014849.203935 EIXKN4201B4HU6 length=144 +AACCCGTCCCA +TCAAAGATTTTGGTTGGAACCCGAAAGGGTTTTGAATTCAAACC +CCTTTCGGTTCCAACTATTCAATTGTTTAACTTTTTT +TAAATTGATGGTCTGTTGGACCATTTGTAATAATCCCCATCGGAATTTCTTT diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/fastq_combiner_in_1.qual454 --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/fastq_combiner_in_1.qual454 Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,22 @@ +>SRR014849.110027 EIXKN4201APUB0 length=131 +=B::@<':=5A9?7EA0:=<<?6@7<3?5< +@;%D?-B=)::0=4<D? +-EA/D@2";B;B: +B:A9;;=<B;;<B;<B;<B;:A;<A;8FB7 ++=<B;B:A9<1:=FB6(<=<<EA0956;<2 +==A8===:@8= +>SRR014849.203935 EIXKN4201B4HU6 length=144 +A;@;%75?:#<<9EA1;=EA3%B;B;A;B; +@;%9EA1EA1EA3%<B;A;8EA0D@ +3$EA1 +=B;A;B;B;:=:B;:B:A9:EA0A9<FA81 ++&"D?-B;4<::/<;=:A98-5?6=C>+8< +<3;=4:DA3%<;=8-9.A=):B=* +>SRR014849.50939 EIXKN4201BA2EC length=135 +;C?-EA/=<EA/B;<B;D>60,)%"<=:5< +@8<B;=B;<;E +A4'@8FB6*<:=<<===<= +;=B:A9<<B;=B;=EA0:<B:<<=<<FA81 ++$?6;<A9=<3>5@7@8<A<(B=*A=)<<2 +?57B=*B=*D?-:=4 + diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/fastq_stats_1_out.tabular --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/fastq_stats_1_out.tabular Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,37 @@ +#column count min max sum mean Q1 med Q3 IQR lW rW outliers A_Count C_Count G_Count T_Count N_Count other_bases other_base_count +1 9 23 34 288 32.0 33.0 33.0 33.0 0.0 33 33 23,34 3 1 4 1 0 +2 9 28 33 287 31.8888888889 30.5 33.0 33.0 2.5 28 33 3 3 2 1 0 +3 9 13 34 268 29.7777777778 27.5 33.0 33.5 6.0 27 34 13 5 1 0 3 0 +4 9 17 33 261 29.0 24.5 33.0 33.0 8.5 17 33 1 2 3 3 0 +5 9 22 33 269 29.8888888889 26.0 33.0 33.0 7.0 22 33 3 3 3 0 0 +6 9 22 33 277 30.7777777778 29.0 33.0 33.0 4.0 28 33 22 5 3 0 1 0 +7 9 21 33 258 28.6666666667 23.0 33.0 33.0 10.0 21 33 4 1 3 1 0 +8 9 12 33 263 29.2222222222 26.5 33.0 33.0 6.5 21 33 12 2 1 1 5 0 +9 9 29 33 290 32.2222222222 31.5 33.0 33.0 1.5 30 33 29 3 3 2 1 0 +10 9 23 33 277 30.7777777778 28.0 33.0 33.0 5.0 23 33 1 4 2 2 0 +11 9 12 33 245 27.2222222222 21.0 31.0 33.0 12.0 12 33 5 2 1 1 0 +12 9 13 33 214 23.7777777778 14.0 24.0 33.0 19.0 13 33 2 4 2 1 0 +13 9 5 33 249 27.6666666667 26.5 31.0 33.0 6.5 24 33 5 2 1 1 5 0 +14 9 5 33 233 25.8888888889 19.5 33.0 33.0 13.5 5 33 3 3 2 1 0 +15 9 15 33 251 27.8888888889 22.5 33.0 33.0 10.5 15 33 5 1 1 2 0 +16 9 23 34 269 29.8888888889 23.5 33.0 33.0 9.5 23 34 3 1 2 3 0 +17 9 13 34 266 29.5555555556 27.0 33.0 33.0 6.0 21 34 13 2 3 1 3 0 +18 9 21 34 272 30.2222222222 26.0 33.0 33.0 7.0 21 34 0 5 1 3 0 +19 9 5 34 244 27.1111111111 24.0 30.0 33.0 9.0 21 34 5 4 4 1 0 0 +20 9 11 34 241 26.7777777778 17.0 32.0 33.0 16.0 11 34 3 4 2 0 0 +21 9 13 33 240 26.6666666667 22.5 27.0 33.0 10.5 13 33 1 4 0 4 0 +22 9 5 33 190 21.1111111111 9.0 21.0 33.0 24.0 5 33 1 4 0 3 1 +23 9 5 33 205 22.7777777778 14.0 26.0 33.0 19.0 5 33 4 4 1 0 0 +24 9 5 33 247 27.4444444444 24.5 31.0 33.0 8.5 21 33 5 1 5 1 2 0 +25 9 11 34 241 26.7777777778 18.5 33.0 33.0 14.5 11 34 3 4 0 2 0 +26 9 5 33 212 23.5555555556 11.5 31.0 33.0 21.5 5 33 0 6 0 3 0 +27 9 5 33 227 25.2222222222 20.0 26.0 33.0 13.0 5 33 3 4 1 1 0 +28 9 21 33 255 28.3333333333 22.5 31.0 33.0 10.5 21 33 2 4 3 0 0 +29 9 5 33 228 25.3333333333 19.5 30.0 33.0 13.5 5 33 2 4 1 2 0 +30 9 10 33 213 23.6666666667 13.5 28.0 33.0 19.5 10 33 3 4 2 0 0 +31 9 5 33 236 26.2222222222 21.0 31.0 33.0 12.0 5 33 1 4 1 3 0 +32 9 5 33 210 23.3333333333 11.5 29.0 33.0 21.5 5 33 3 3 0 3 0 +33 9 5 33 183 20.3333333333 8.0 21.0 33.0 25.0 5 33 1 4 2 2 0 +34 9 5 33 150 16.6666666667 6.0 17.0 25.5 19.5 5 33 3 4 1 1 0 +35 9 13 33 217 24.1111111111 19.5 24.0 31.0 11.5 13 33 1 4 1 3 0 +36 9 5 33 195 21.6666666667 11.5 21.0 32.5 21.0 5 33 3 2 1 3 0 diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/fastq_to_fasta_python_1.out --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/fastq_to_fasta_python_1.out Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,4 @@ +>FAKE0001 Original version has PHRED scores from 0 to 93 inclusive (in that order) +ACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTAC +>FAKE0002 Original version has PHRED scores from 93 to 0 inclusive (in that order) +CATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCA diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/fastq_to_fasta_python_2.out --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/fastq_to_fasta_python_2.out Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,4 @@ +>FAKE0001 Original version has PHRED scores from 0 to 93 inclusive (in that order) +G2131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131 +>FAKE0002 Original version has PHRED scores from 93 to 0 inclusive (in that order) +G3131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131 diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/fastq_trimmer_out1.fastqsanger --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/fastq_trimmer_out1.fastqsanger Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,8 @@ +@FAKE0001 Original version has PHRED scores from 0 to 93 inclusive (in that order) +CGTA ++ +NOPQ +@FAKE0002 Original version has PHRED scores from 93 to 0 inclusive (in that order) +ATGC ++ +QPON diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/illumina_full_range_as_sanger.fastqsanger --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/illumina_full_range_as_sanger.fastqsanger Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,8 @@ +@FAKE0005 Original version has PHRED scores from 0 to 62 inclusive (in that order) +ACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACG ++ +!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_ +@FAKE0006 Original version has PHRED scores from 62 to 0 inclusive (in that order) +GCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCA ++ +_^]\[ZYXWVUTSRQPONMLKJIHGFEDCBA@?>=<;:9876543210/.-,+*)('&%$#"! diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/illumina_full_range_as_solexa.fastqsolexa --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/illumina_full_range_as_solexa.fastqsolexa Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,8 @@ +@FAKE0005 Original version has PHRED scores from 0 to 62 inclusive (in that order) +ACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACG ++ +;;>@BCEFGHJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~ +@FAKE0006 Original version has PHRED scores from 62 to 0 inclusive (in that order) +GCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCA ++ +~}|{zyxwvutsrqponmlkjihgfedcba`_^]\[ZYXWVUTSRQPONMLKJHGFECB@>;; diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/illumina_full_range_as_solid.fastqsolid --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/illumina_full_range_as_solid.fastqsolid Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,8 @@ +@FAKE0005 Original version has PHRED scores from 0 to 62 inclusive (in that order) +G213131313131313131313131313131313131313131313131313131313131313 ++ +!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_ +@FAKE0006 Original version has PHRED scores from 62 to 0 inclusive (in that order) +G031313131313131313131313131313131313131313131313131313131313131 ++ +_^]\[ZYXWVUTSRQPONMLKJIHGFEDCBA@?>=<;:9876543210/.-,+*)('&%$#"! diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/illumina_full_range_original_illumina.fastqillumina --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/illumina_full_range_original_illumina.fastqillumina Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,8 @@ +@FAKE0005 Original version has PHRED scores from 0 to 62 inclusive (in that order) +ACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACG ++ +@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~ +@FAKE0006 Original version has PHRED scores from 62 to 0 inclusive (in that order) +GCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCA ++ +~}|{zyxwvutsrqponmlkjihgfedcba`_^]\[ZYXWVUTSRQPONMLKJIHGFEDCBA@ diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/sanger_full_range_as_decimal_sanger.fastqsanger --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/sanger_full_range_as_decimal_sanger.fastqsanger Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,8 @@ +@FAKE0001 Original version has PHRED scores from 0 to 93 inclusive (in that order) +ACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTAC ++ +0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 +@FAKE0002 Original version has PHRED scores from 93 to 0 inclusive (in that order) +CATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCA ++ +93 92 91 90 89 88 87 86 85 84 83 82 81 80 79 78 77 76 75 74 73 72 71 70 69 68 67 66 65 64 63 62 61 60 59 58 57 56 55 54 53 52 51 50 49 48 47 46 45 44 43 42 41 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/sanger_full_range_as_illumina.fastqillumina --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/sanger_full_range_as_illumina.fastqillumina Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,8 @@ +@FAKE0001 Original version has PHRED scores from 0 to 93 inclusive (in that order) +ACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTAC ++ +@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +@FAKE0002 Original version has PHRED scores from 93 to 0 inclusive (in that order) +CATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCA ++ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}|{zyxwvutsrqponmlkjihgfedcba`_^]\[ZYXWVUTSRQPONMLKJIHGFEDCBA@ diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/sanger_full_range_as_rna.fastqsanger --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/sanger_full_range_as_rna.fastqsanger Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,8 @@ +@FAKE0001 Original version has PHRED scores from 0 to 93 inclusive (in that order) +ACGUACGUACGUACGUACGUACGUACGUACGUACGUACGUACGUACGUACGUACGUACGUACGUACGUACGUACGUACGUACGUACGUACGUAC ++ +!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~ +@FAKE0002 Original version has PHRED scores from 93 to 0 inclusive (in that order) +CAUGCAUGCAUGCAUGCAUGCAUGCAUGCAUGCAUGCAUGCAUGCAUGCAUGCAUGCAUGCAUGCAUGCAUGCAUGCAUGCAUGCAUGCAUGCA ++ +~}|{zyxwvutsrqponmlkjihgfedcba`_^]\[ZYXWVUTSRQPONMLKJIHGFEDCBA@?>=<;:9876543210/.-,+*)('&%$#"! diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/sanger_full_range_as_solexa.fastqsolexa --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/sanger_full_range_as_solexa.fastqsolexa Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,8 @@ +@FAKE0001 Original version has PHRED scores from 0 to 93 inclusive (in that order) +ACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTAC ++ +;;>@BCEFGHJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +@FAKE0002 Original version has PHRED scores from 93 to 0 inclusive (in that order) +CATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCA ++ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}|{zyxwvutsrqponmlkjihgfedcba`_^]\[ZYXWVUTSRQPONMLKJHGFECB@>;; diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/sanger_full_range_as_solid.fastqsolid --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/sanger_full_range_as_solid.fastqsolid Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,8 @@ +@FAKE0001 Original version has PHRED scores from 0 to 93 inclusive (in that order) +G2131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131 ++ +!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~ +@FAKE0002 Original version has PHRED scores from 93 to 0 inclusive (in that order) +G3131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131 ++ +~}|{zyxwvutsrqponmlkjihgfedcba`_^]\[ZYXWVUTSRQPONMLKJIHGFEDCBA@?>=<;:9876543210/.-,+*)('&%$#"! diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/sanger_full_range_original_sanger.fastqsanger --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/sanger_full_range_original_sanger.fastqsanger Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,8 @@ +@FAKE0001 Original version has PHRED scores from 0 to 93 inclusive (in that order) +ACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTAC ++ +!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~ +@FAKE0002 Original version has PHRED scores from 93 to 0 inclusive (in that order) +CATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCA ++ +~}|{zyxwvutsrqponmlkjihgfedcba`_^]\[ZYXWVUTSRQPONMLKJIHGFEDCBA@?>=<;:9876543210/.-,+*)('&%$#"! diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/sanger_full_range_rev_comp.fastqsanger --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/sanger_full_range_rev_comp.fastqsanger Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,8 @@ +@FAKE0001 Original version has PHRED scores from 0 to 93 inclusive (in that order) +GTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGT ++ +~}|{zyxwvutsrqponmlkjihgfedcba`_^]\[ZYXWVUTSRQPONMLKJIHGFEDCBA@?>=<;:9876543210/.-,+*)('&%$#"! +@FAKE0002 Original version has PHRED scores from 93 to 0 inclusive (in that order) +TGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATG ++ +!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~ diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/sanger_full_range_rev_comp_1_seq.fastqsanger --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/sanger_full_range_rev_comp_1_seq.fastqsanger Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,8 @@ +@FAKE0001 Original version has PHRED scores from 0 to 93 inclusive (in that order) +GTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGT ++ +~}|{zyxwvutsrqponmlkjihgfedcba`_^]\[ZYXWVUTSRQPONMLKJIHGFEDCBA@?>=<;:9876543210/.-,+*)('&%$#"! +@FAKE0002 Original version has PHRED scores from 93 to 0 inclusive (in that order) +CATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCA ++ +~}|{zyxwvutsrqponmlkjihgfedcba`_^]\[ZYXWVUTSRQPONMLKJIHGFEDCBA@?>=<;:9876543210/.-,+*)('&%$#"! diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/solexa_full_range_as_decimal_solexa.fastqsolexa --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/solexa_full_range_as_decimal_solexa.fastqsolexa Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,8 @@ +@FAKE0003 Original version has Solexa scores from -5 to 62 inclusive (in that order) +ACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGT ++ +-5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 +@FAKE0004 Original version has Solexa scores from 62 to -5 inclusive (in that order) +TGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCA ++ +62 61 60 59 58 57 56 55 54 53 52 51 50 49 48 47 46 45 44 43 42 41 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 -1 -2 -3 -4 -5 diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/solexa_full_range_as_illumina.fastqillumina --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/solexa_full_range_as_illumina.fastqillumina Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,8 @@ +@FAKE0003 Original version has Solexa scores from -5 to 62 inclusive (in that order) +ACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGT ++ +AABBCCDDEEFGHIJJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~ +@FAKE0004 Original version has Solexa scores from 62 to -5 inclusive (in that order) +TGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCA ++ +~}|{zyxwvutsrqponmlkjihgfedcba`_^]\[ZYXWVUTSRQPONMLKJJIHGFEEDDCCBBAA diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/solexa_full_range_as_sanger.fastqsanger --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/solexa_full_range_as_sanger.fastqsanger Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,8 @@ +@FAKE0003 Original version has Solexa scores from -5 to 62 inclusive (in that order) +ACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGT ++ +""##$$%%&&'()*++,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_ +@FAKE0004 Original version has Solexa scores from 62 to -5 inclusive (in that order) +TGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCA ++ +_^]\[ZYXWVUTSRQPONMLKJIHGFEDCBA@?>=<;:9876543210/.-,++*)('&&%%$$##"" diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/solexa_full_range_as_solid.fastqsolid --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/solexa_full_range_as_solid.fastqsolid Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,8 @@ +@FAKE0003 Original version has Solexa scores from -5 to 62 inclusive (in that order) +G21313131313131313131313131313131313131313131313131313131313131313131 ++ +""##$$%%&&'()*++,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_ +@FAKE0004 Original version has Solexa scores from 62 to -5 inclusive (in that order) +G11313131313131313131313131313131313131313131313131313131313131313131 ++ +_^]\[ZYXWVUTSRQPONMLKJIHGFEDCBA@?>=<;:9876543210/.-,++*)('&&%%$$##"" diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/solexa_full_range_original_solexa.fastqsolexa --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/solexa_full_range_original_solexa.fastqsolexa Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,8 @@ +@FAKE0003 Original version has Solexa scores from -5 to 62 inclusive (in that order) +ACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGTACGT ++ +;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~ +@FAKE0004 Original version has Solexa scores from 62 to -5 inclusive (in that order) +TGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCATGCA ++ +~}|{zyxwvutsrqponmlkjihgfedcba`_^]\[ZYXWVUTSRQPONMLKJIHGFEDCBA@?>=<; diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/split_pair_reads_1.fastqsanger --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/split_pair_reads_1.fastqsanger Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,20 @@ +@HWI-EAS91_1_30788AAXX:7:21:1542:1758/1 +GTCAATTGTACTGGTCAATACTAAAAGAATAGGATC ++HWI-EAS91_1_30788AAXX:7:21:1542:1758/1 +hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh +@HWI-EAS91_1_30788AAXX:7:22:1621:462/1 +ATAATGGCTATTATTGTGGGGGGGATGATGCTGGAA ++HWI-EAS91_1_30788AAXX:7:22:1621:462/1 +hhhhhhhhhhhhQAhh@hhhhNhhhfhMbCIScC?h +@HWI-EAS91_1_30788AAXX:7:45:408:807/1 +TACCCGATTTTTTGCTTTCCACTTTATCCTACCCTT ++HWI-EAS91_1_30788AAXX:7:45:408:807/1 +hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh +@HWI-EAS91_1_30788AAXX:7:49:654:1439/1 +CTAACTCTATTTATTGTATTTCAACTAAAAATCTCA ++HWI-EAS91_1_30788AAXX:7:49:654:1439/1 +hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh +@HWI-EAS91_1_30788AAXX:7:64:947:234/1 +TATCAAAAAAGAATATAATCTGAATCAACACTACAA ++HWI-EAS91_1_30788AAXX:7:64:947:234/1 +hhhhhhhhhhhhhhhhhhhhhhhRhhehhahhhhhJ diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/split_pair_reads_2.fastqsanger --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/split_pair_reads_2.fastqsanger Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,20 @@ +@HWI-EAS91_1_30788AAXX:7:21:1542:1758/2 +GCTCCTAGCATCTGGAGTCTCTATCACCTGAGCCCA ++HWI-EAS91_1_30788AAXX:7:21:1542:1758/2 +hhhhhhhhhhhhhhhhhhhhhhhh`hfhhVZSWehR +@HWI-EAS91_1_30788AAXX:7:22:1621:462/2 +ACTAGCCCCAATATCAATCCTATATCAAATCTCACC ++HWI-EAS91_1_30788AAXX:7:22:1621:462/2 +hJhhhhChhhJhhhRhhKhePhc\KhhV\KhXhJhh +@HWI-EAS91_1_30788AAXX:7:45:408:807/2 +ATGAGTGCTAGGATCAGGATGGAGAGGATTAGGGCT ++HWI-EAS91_1_30788AAXX:7:45:408:807/2 +hhhhhhhhhhhhhhhhhh`hhhZh`hhhhhRXhhYh +@HWI-EAS91_1_30788AAXX:7:49:654:1439/2 +TAGGTTTATTGATAGTTGTGTTGTTGGTGTAAATGG ++HWI-EAS91_1_30788AAXX:7:49:654:1439/2 +hhhhhhhhhhhhhhhhhhhhhhhhhdhh_hG\XhU@ +@HWI-EAS91_1_30788AAXX:7:64:947:234/2 +CCTATTAGTGTGTAGAATAGGAAGTAGAGGCCTGCG ++HWI-EAS91_1_30788AAXX:7:64:947:234/2 +hhhhhhhh^hPhWfhhhhThWUhhfhh_hhNIVPUd diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/split_paired_reads_out_1.fastqsanger --- a/test-data/split_paired_reads_out_1.fastqsanger Tue Feb 23 16:12:20 2010 -0500 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,20 +0,0 @@ -@HWI-EAS91_1_30788AAXX:7:21:1542:1758/1 -GTCAATTGTACTGGTCAATACTAAAAGAATAGGATC -+HWI-EAS91_1_30788AAXX:7:21:1542:1758/1 -hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh -@HWI-EAS91_1_30788AAXX:7:22:1621:462/1 -ATAATGGCTATTATTGTGGGGGGGATGATGCTGGAA -+HWI-EAS91_1_30788AAXX:7:22:1621:462/1 -hhhhhhhhhhhhQAhh@hhhhNhhhfhMbCIScC?h -@HWI-EAS91_1_30788AAXX:7:45:408:807/1 -TACCCGATTTTTTGCTTTCCACTTTATCCTACCCTT -+HWI-EAS91_1_30788AAXX:7:45:408:807/1 -hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh -@HWI-EAS91_1_30788AAXX:7:49:654:1439/1 -CTAACTCTATTTATTGTATTTCAACTAAAAATCTCA -+HWI-EAS91_1_30788AAXX:7:49:654:1439/1 -hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh -@HWI-EAS91_1_30788AAXX:7:64:947:234/1 -TATCAAAAAAGAATATAATCTGAATCAACACTACAA -+HWI-EAS91_1_30788AAXX:7:64:947:234/1 -hhhhhhhhhhhhhhhhhhhhhhhRhhehhahhhhhJ diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/split_paired_reads_out_2.fastqsanger --- a/test-data/split_paired_reads_out_2.fastqsanger Tue Feb 23 16:12:20 2010 -0500 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,20 +0,0 @@ -@HWI-EAS91_1_30788AAXX:7:21:1542:1758/2 -GCTCCTAGCATCTGGAGTCTCTATCACCTGAGCCCA -+HWI-EAS91_1_30788AAXX:7:21:1542:1758/2 -hhhhhhhhhhhhhhhhhhhhhhhh`hfhhVZSWehR -@HWI-EAS91_1_30788AAXX:7:22:1621:462/2 -ACTAGCCCCAATATCAATCCTATATCAAATCTCACC -+HWI-EAS91_1_30788AAXX:7:22:1621:462/2 -hJhhhhChhhJhhhRhhKhePhc\KhhV\KhXhJhh -@HWI-EAS91_1_30788AAXX:7:45:408:807/2 -ATGAGTGCTAGGATCAGGATGGAGAGGATTAGGGCT -+HWI-EAS91_1_30788AAXX:7:45:408:807/2 -hhhhhhhhhhhhhhhhhh`hhhZh`hhhhhRXhhYh -@HWI-EAS91_1_30788AAXX:7:49:654:1439/2 -TAGGTTTATTGATAGTTGTGTTGTTGGTGTAAATGG -+HWI-EAS91_1_30788AAXX:7:49:654:1439/2 -hhhhhhhhhhhhhhhhhhhhhhhhhdhh_hG\XhU@ -@HWI-EAS91_1_30788AAXX:7:64:947:234/2 -CCTATTAGTGTGTAGAATAGGAAGTAGAGGCCTGCG -+HWI-EAS91_1_30788AAXX:7:64:947:234/2 -hhhhhhhh^hPhWfhhhhThWUhhfhh_hhNIVPUd diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/wrapping_as_illumina.fastqillumina --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/wrapping_as_illumina.fastqillumina Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,12 @@ +(a)SRR014849.50939 EIXKN4201BA2EC length=135 +GAAATTTCAGGGCCACCTTTTTTTTGATAGAATAATGGAGAAAATTAAAAGCTGTACATATACCAATGAACAATAAATCAATACATAAAAAAGGAGAAGTTGGAACCGAAAGGGTTTGAATTCAAACCCTTTCGG ++ +Zb^Ld`N\[d`NaZ[aZc]UOKHDA[\YT[_W[aZ\aZ[Zd`SF_WeaUI[Y\[[\\\[\Z\aY`X[[aZ\aZ\d`OY[aY[[\[[e`WPJC^UZ[`X\[R]T_V_W[`[Ga\I`\H[[Q^TVa\Ia\Ic^LY\S +(a)SRR014849.110027 EIXKN4201APUB0 length=131 +CTTCAAATGATTCCGGGACTGTTGGAACCGAAAGGGTTTGAATTCAAACCCTTTTCGGTTCCAACTCGCCGTCCGAATAATCCGTTCAAAATCTTGGCCTGTCAAAACGACTTTACGACCAGAACGATCCG ++ +\aYY_[FY\T`X^Vd`OY\[[^U_V[R^T[_ZDc^La\HYYO\S[c^Ld`Nc_QAZaZaYaY`XZZ\[aZZ[aZ[aZ[aZY`Z[`ZWeaVJ\[aZaY`X[PY\eaUG[\[[d`OXTUZ[Q\\`W\\\Y_W\ +(a)SRR014849.203935 EIXKN4201B4HU6 length=144 +AACCCGTCCCATCAAAGATTTTGGTTGGAACCCGAAAGGGTTTTGAATTCAAACCCCTTTCGGTTCCAACTATTCAATTGTTTAACTTTTTTTAAATTGATGGTCTGTTGGACCATTTGTAATAATCCCCATCGGAATTTCTTT ++ +`Z_ZDVT^YB[[Xd`PZ\d`RDaZaZ`ZaZ_ZDXd`Pd`Pd`RD[aZ`ZWd`Oc_RCd`P\aZ`ZaZaZY\YaZYaY`XYd`O`X[e`WPJEAc^LaZS[YYN[Z\Y`XWLT^U\b]JW[[RZ\SYc`RD[Z\WLXM`\HYa\I diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/wrapping_as_sanger.fastqsanger --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/wrapping_as_sanger.fastqsanger Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,12 @@ +(a)SRR014849.50939 EIXKN4201BA2EC length=135 +GAAATTTCAGGGCCACCTTTTTTTTGATAGAATAATGGAGAAAATTAAAAGCTGTACATATACCAATGAACAATAAATCAATACATAAAAAAGGAGAAGTTGGAACCGAAAGGGTTTGAATTCAAACCCTTTCGG ++ +;C?-EA/=<EA/B;<B;D>60,)%"<=:5<@8<B;=B;<;EA4'@8FB6*<:=<<===<=;=B:A9<<B;=B;=EA0:<B:<<=<<FA81+$?6;<A9=<3>5@7@8<A<(B=*A=)<<2?57B=*B=*D?-:=4 +(a)SRR014849.110027 EIXKN4201APUB0 length=131 +CTTCAAATGATTCCGGGACTGTTGGAACCGAAAGGGTTTGAATTCAAACCCTTTTCGGTTCCAACTCGCCGTCCGAATAATCCGTTCAAAATCTTGGCCTGTCAAAACGACTTTACGACCAGAACGATCCG ++ +=B::@<':=5A9?7EA0:=<<?6@7<3?5<@;%D?-B=)::0=4<D?-EA/D@2";B;B:B:A9;;=<B;;<B;<B;<B;:A;<A;8FB7+=<B;B:A9<1:=FB6(<=<<EA0956;<2==A8===:@8= +(a)SRR014849.203935 EIXKN4201B4HU6 length=144 +AACCCGTCCCATCAAAGATTTTGGTTGGAACCCGAAAGGGTTTTGAATTCAAACCCCTTTCGGTTCCAACTATTCAATTGTTTAACTTTTTTTAAATTGATGGTCTGTTGGACCATTTGTAATAATCCCCATCGGAATTTCTTT ++ +A;@;%75?:#<<9EA1;=EA3%B;B;A;B;@;%9EA1EA1EA3%<B;A;8EA0D@3$EA1=B;A;B;B;:=:B;:B:A9:EA0A9<FA81+&"D?-B;4<::/<;=:A98-5?6=C>+8<<3;=4:DA3%<;=8-9.A=):B=* diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/wrapping_as_sanger_decimal.fastqsanger --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/wrapping_as_sanger_decimal.fastqsanger Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,12 @@ +(a)SRR014849.50939 EIXKN4201BA2EC length=135 +GAAATTTCAGGGCCACCTTTTTTTTGATAGAATAATGGAGAAAATTAAAAGCTGTACATATACCAATGAACAATAAATCAATACATAAAAAAGGAGAAGTTGGAACCGAAAGGGTTTGAATTCAAACCCTTTCGG ++ +26 34 30 12 36 32 14 28 27 36 32 14 33 26 27 33 26 35 29 21 15 11 8 4 1 27 28 25 20 27 31 23 27 33 26 28 33 26 27 26 36 32 19 6 31 23 37 33 21 9 27 25 28 27 27 28 28 28 27 28 26 28 33 25 32 24 27 27 33 26 28 33 26 28 36 32 15 25 27 33 25 27 27 28 27 27 37 32 23 16 10 3 30 21 26 27 32 24 28 27 18 29 20 31 22 31 23 27 32 27 7 33 28 9 32 28 8 27 27 17 30 20 22 33 28 9 33 28 9 35 30 12 25 28 19 +(a)SRR014849.110027 EIXKN4201APUB0 length=131 +CTTCAAATGATTCCGGGACTGTTGGAACCGAAAGGGTTTGAATTCAAACCCTTTTCGGTTCCAACTCGCCGTCCGAATAATCCGTTCAAAATCTTGGCCTGTCAAAACGACTTTACGACCAGAACGATCCG ++ +28 33 25 25 31 27 6 25 28 20 32 24 30 22 36 32 15 25 28 27 27 30 21 31 22 27 18 30 20 27 31 26 4 35 30 12 33 28 8 25 25 15 28 19 27 35 30 12 36 32 14 35 31 17 1 26 33 26 33 25 33 25 32 24 26 26 28 27 33 26 26 27 33 26 27 33 26 27 33 26 25 32 26 27 32 26 23 37 33 22 10 28 27 33 26 33 25 32 24 27 16 25 28 37 33 21 7 27 28 27 27 36 32 15 24 20 21 26 27 17 28 28 32 23 28 28 28 25 31 23 28 +(a)SRR014849.203935 EIXKN4201B4HU6 length=144 +AACCCGTCCCATCAAAGATTTTGGTTGGAACCCGAAAGGGTTTTGAATTCAAACCCCTTTCGGTTCCAACTATTCAATTGTTTAACTTTTTTTAAATTGATGGTCTGTTGGACCATTTGTAATAATCCCCATCGGAATTTCTTT ++ +32 26 31 26 4 22 20 30 25 2 27 27 24 36 32 16 26 28 36 32 18 4 33 26 33 26 32 26 33 26 31 26 4 24 36 32 16 36 32 16 36 32 18 4 27 33 26 32 26 23 36 32 15 35 31 18 3 36 32 16 28 33 26 32 26 33 26 33 26 25 28 25 33 26 25 33 25 32 24 25 36 32 15 32 24 27 37 32 23 16 10 5 1 35 30 12 33 26 19 27 25 25 14 27 26 28 25 32 24 23 12 20 30 21 28 34 29 10 23 27 27 18 26 28 19 25 35 32 18 4 27 26 28 23 12 24 13 32 28 8 25 33 28 9 diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/wrapping_as_solexa.fastqsolexa --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/wrapping_as_solexa.fastqsolexa Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,12 @@ +(a)SRR014849.50939 EIXKN4201BA2EC length=135 +GAAATTTCAGGGCCACCTTTTTTTTGATAGAATAATGGAGAAAATTAAAAGCTGTACATATACCAATGAACAATAAATCAATACATAAAAAAGGAGAAGTTGGAACCGAAAGGGTTTGAATTCAAACCCTTTCGG ++ +Zb^Ld`N\[d`NaZ[aZc]UOKGB;[\YT[_W[aZ\aZ[Zd`SE_WeaUH[Y\[[\\\[\Z\aY`X[[aZ\aZ\d`OY[aY[[\[[e`WPJ@^UZ[`X\[R]T_V_W[`[Fa\H`\G[[Q^TVa\Ha\Hc^LY\S +(a)SRR014849.110027 EIXKN4201APUB0 length=131 +CTTCAAATGATTCCGGGACTGTTGGAACCGAAAGGGTTTGAATTCAAACCCTTTTCGGTTCCAACTCGCCGTCCGAATAATCCGTTCAAAATCTTGGCCTGTCAAAACGACTTTACGACCAGAACGATCCG ++ +\aYY_[EY\T`X^Vd`OY\[[^U_V[R^T[_ZBc^La\GYYO\S[c^Ld`Nc_Q;ZaZaYaY`XZZ\[aZZ[aZ[aZ[aZY`Z[`ZWeaVJ\[aZaY`X[PY\eaUF[\[[d`OXTUZ[Q\\`W\\\Y_W\ +(a)SRR014849.203935 EIXKN4201B4HU6 length=144 +AACCCGTCCCATCAAAGATTTTGGTTGGAACCCGAAAGGGTTTTGAATTCAAACCCCTTTCGGTTCCAACTATTCAATTGTTTAACTTTTTTTAAATTGATGGTCTGTTGGACCATTTGTAATAATCCCCATCGGAATTTCTTT ++ +`Z_ZBVT^Y>[[Xd`PZ\d`RBaZaZ`ZaZ_ZBXd`Pd`Pd`RB[aZ`ZWd`Oc_R@d`P\aZ`ZaZaZY\YaZYaY`XYd`O`X[e`WPJC;c^LaZS[YYN[Z\Y`XWLT^U\b]JW[[RZ\SYc`RB[Z\WLXM`\GYa\H diff -r 03dbef081cf2 -r 9dd990a2fa94 test-data/wrapping_original_sanger.fastqsanger --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test-data/wrapping_original_sanger.fastqsanger Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,24 @@ +(a)SRR014849.50939 EIXKN4201BA2EC length=135 +GAAATTTCAGGGCCACCTTTTTTTTGATAGAATAATGGAGAAAATTAAAAGCTGTACATATACCAATGAACAATAAATCAATACATAAAAAAGGAGAAGTTGGAACCGAAAGGGTTTGAATTCAAACCCTTTCGG ++ +;C?-EA/=<EA/B;<B;D>60,)%"<=:5< +@8<B;=B;<;EA4'@8FB6*<:=<<===<= +;=B:A9<<B;=B;=EA0:<B:<<=<<FA81 ++$?6;<A9=<3>5@7@8<A<(B=*A=)<<2 +?57B=*B=*D?-:=4 +(a)SRR014849.110027 EIXKN4201APUB0 length=131 +CTTCAAATGATTCCGGGACTGTTGGAACCGAAAGGGTTTGAATTCAAACCCTTTTCGGTTCCAACTCGCCGTCCGAATAATCCGTTCAAAATCTTGGCCTGTCAAAACGACTTTACGACCAGAACGATCCG ++ +=B::@<':=5A9?7EA0:=<<?6@7<3?5< +@;%D?-B=)::0=4<D?-EA/D@2";B;B: +B:A9;;=<B;;<B;<B;<B;:A;<A;8FB7 ++=<B;B:A9<1:=FB6(<=<<EA0956;<2 +==A8===:@8= +(a)SRR014849.203935 EIXKN4201B4HU6 length=144 +AACCCGTCCCATCAAAGATTTTGGTTGGAACCCGAAAGGGTTTTGAATTCAAACCCCTTTCGGTTCCAACTATTCAATTGTTTAACTTTTTTTAAATTGATGGTCTGTTGGACCATTTGTAATAATCCCCATCGGAATTTCTTT ++ +A;@;%75?:#<<9EA1;=EA3%B;B;A;B; +@;%9EA1EA1EA3%<B;A;8EA0D@3$EA1 +=B;A;B;B;:=:B;:B:A9:EA0A9<FA81 ++&"D?-B;4<::/<;=:A98-5?6=C>+8< +<3;=4:DA3%<;=8-9.A=):B=* diff -r 03dbef081cf2 -r 9dd990a2fa94 tool_conf.xml.sample --- a/tool_conf.xml.sample Tue Feb 23 16:12:20 2010 -0500 +++ b/tool_conf.xml.sample Tue Feb 23 16:48:07 2010 -0500 @@ -127,6 +127,7 @@ <tool file="plotting/scatterplot.xml" /> <tool file="plotting/bar_chart.xml" /> <tool file="plotting/xy_plot.xml" /> + <tool file="plotting/boxplot.xml" /> <tool file="visualization/GMAJ.xml" /> <tool file="visualization/LAJ.xml" /> <tool file="visualization/build_ucsc_custom_track.xml" /> @@ -174,7 +175,17 @@ <tool file="fastx_toolkit/fastx_collapser.xml" /> </section> <section name="NGS: QC and manipulation" id="cshl_library_information"> - <label text="Generic FASTQ data" id="fastq" /> + <label text="Generic FASTQ data" id="new_fastq" /> + <tool file="fastq/fastq_combiner.xml" /> + <tool file="fastq/fastq_groomer.xml" /> + <tool file="fastq/fastq_paired_end_joiner.xml" /> + <tool file="fastq/fastq_paired_end_splitter.xml" /> + <tool file="fastq/fastq_stats.xml" /> + <tool file="fastq/fastq_filter.xml" /> + <tool file="fastq/fastq_trimmer.xml" /> + <tool file="fastq/fastq_manipulation.xml" /> + <tool file="fastq/fastq_to_fasta.xml" /> + <label text="Deprecated: Generic FASTQ data" id="fastq" /> <tool file="next_gen_conversion/fastq_gen_conv.xml" /> <tool file="fastx_toolkit/fastq_quality_converter.xml" /> <tool file="fastx_toolkit/fastx_quality_statistics.xml" /> diff -r 03dbef081cf2 -r 9dd990a2fa94 tools/fastq/fastq_combiner.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/tools/fastq/fastq_combiner.py Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,45 @@ +#Dan Blankenberg +import sys, os, shutil +from galaxy_utils.sequence.fastq import fastqWriter, fastaReader, fastaNamedReader, fastqSequencingRead, fastqCombiner + +def main(): + #Read command line arguments + fasta_filename = sys.argv[1] + fasta_type = sys.argv[2] or 'fasta' #should always be fasta or csfasta? what if txt? + qual_filename = sys.argv[3] + qual_type = sys.argv[4] or 'qualsanger' #qual454 qualsolid + output_filename = sys.argv[5] + force_quality_encoding = sys.argv[6] + if force_quality_encoding == 'None': + force_quality_encoding = None + + format = 'sanger' + if fasta_type == 'csfasta' or qual_type == 'qualsolid': + format = 'solid' + elif qual_type == 'qualsolexa': + format = 'solexa' + elif qual_type == 'qualillumina': + format = 'illumina' + + out = fastqWriter( open( output_filename, 'wb' ), format = format, force_quality_encoding = force_quality_encoding ) + qual_input = fastaNamedReader( open( qual_filename, 'rb' ) ) + fastq_combiner = fastqCombiner( format ) + i = None + skip_count = 0 + for i, sequence in enumerate( fastaReader( open( fasta_filename, 'rb' ) ) ): + quality = qual_input.get( sequence.identifier ) + if quality: + fastq_read = fastq_combiner.combine( sequence, quality ) + #Should we check that fastq read is valid? for now, assume groomer will be used to verify + out.write( fastq_read ) + else: + skip_count += 1 + out.close() + if i is None: + print "Your file contains no valid FASTA sequences." + else: + print qual_input.has_data() + print 'Combined %s of %s sequences with quality scores (%.2f%%).' % ( i - skip_count + 1, i + 1, float( i - skip_count + 1 ) / float( i + 1 ) * 100.0 ) + +if __name__ == "__main__": + main() diff -r 03dbef081cf2 -r 9dd990a2fa94 tools/fastq/fastq_combiner.xml --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/tools/fastq/fastq_combiner.xml Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,55 @@ +<tool id="fastq_combiner" name="Combine FASTA and QUAL" version="1.0.0"> + <description>into FASTQ</description> + <command interpreter="python">fastq_combiner.py '$fasta_file' '${fasta_file.extension}' '$qual_file' '${qual_file.extension}' '$output_file' '$force_quality_encoding'</command> + <inputs> + <param name="fasta_file" type="data" format="fasta,csfasta" label="FASTA File" /> + <param name="qual_file" type="data" format="qual" label="Quality Score File" /> + <param name="force_quality_encoding" type="select" label="Force Quality Score encoding"> + <option value="None">Use Source Encoding</option> + <option value="ascii" selected="True">ASCII</option> + <option value="decimal">Decimal</option> + </param> + </inputs> + <outputs> + <data name="output_file" format="fastqsanger"> + <change_format> + <when input_dataset="fasta_file" attribute="extension" value="csfasta" format="fastqsolid" /> + <when input_dataset="qual_file" attribute="extension" value="qualsolid" format="fastqsolid" /> + <when input_dataset="qual_file" attribute="extension" value="qualsolexa" format="fastqsolexa" /> + <when input_dataset="qual_file" attribute="extension" value="qualillumina" format="fastqillumina" /> + </change_format> + </data> + </outputs> + <tests> + <test> + <param name="fasta_file" value="s2fq_phiX.csfasta" ftype="csfasta" /> + <param name="qual_file" value="s2fq_phiX.qualsolid" ftype="qualsolid" /> + <param name="force_quality_encoding" value="None" /> + <output name="output_file" file="combine_phiX_out_1.fastqsolid" /> + </test> + <test> + <param name="fasta_file" value="s2fq_phiX.csfasta" ftype="csfasta" /> + <param name="qual_file" value="s2fq_phiX.qualsolid" ftype="qualsolid" /> + <param name="force_quality_encoding" value="ascii" /> + <output name="output_file" file="combine_phiX_out_2.fastqsolid" /> + </test> + <test> + <param name="fasta_file" value="fastq_combiner_in_1.fasta" ftype="fasta" /> + <param name="qual_file" value="fastq_combiner_in_1.qual454" ftype="qual454" /> + <param name="force_quality_encoding" value="None" /> + <output name="output_file" file="wrapping_as_sanger.fastqsanger" /> + </test> + <test> + <param name="fasta_file" value="fastq_combiner_in_1.fasta" ftype="fasta" /> + <param name="qual_file" value="fastq_combiner_in_1.qual454" ftype="qual454" /> + <param name="force_quality_encoding" value="decimal" /> + <output name="output_file" file="wrapping_as_sanger_decimal.fastqsanger" /> + </test> + </tests> + <help> +**What it does** + +This tool joins a FASTA file to a Quality Score file, creating a single FASTQ block for each read. + + </help> +</tool> diff -r 03dbef081cf2 -r 9dd990a2fa94 tools/fastq/fastq_filter.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/tools/fastq/fastq_filter.py Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,34 @@ +#Dan Blankenberg +import sys, os, shutil +from galaxy_utils.sequence.fastq import fastqReader, fastqWriter + +def main(): + #Read command line arguments + input_filename = sys.argv[1] + script_filename = sys.argv[2] + output_filename = sys.argv[3] + additional_files_path = sys.argv[4] + input_type = sys.argv[5] or 'sanger' + + #Save script file for debuging/verification info later + os.mkdir( additional_files_path ) + shutil.copy( script_filename, os.path.join( additional_files_path, 'debug.txt' ) ) + + out = fastqWriter( open( output_filename, 'wb' ), format = input_type ) + + i = None + reads_kept = 0 + for i, fastq_read in enumerate( fastqReader( open( input_filename ), format = input_type ) ): + local = {'fastq_read':fastq_read, 'ret_val':False} + execfile( script_filename, {}, local ) + if local['ret_val']: + out.write( fastq_read ) + reads_kept += 1 + out.close() + if i is None: + print "Your file contains no valid fastq reads." + else: + print 'Kept %s of %s reads (%.2f%%).' % ( reads_kept, i + 1, float( reads_kept ) / float( i + 1 ) * 100.0 ) + +if __name__ == "__main__": + main() diff -r 03dbef081cf2 -r 9dd990a2fa94 tools/fastq/fastq_filter.xml --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/tools/fastq/fastq_filter.xml Tue Feb 23 16:48:07 2010 -0500 @@ -0,0 +1,311 @@ +<tool id="fastq_filter" name="Filter FASTQ" version="1.0.0"> + <description>reads by quality score and length</description> + <command interpreter="python">fastq_filter.py $input_file $fastq_filter_file $output_file $output_file.files_path '${input_file.extension[len( 'fastq' ):]}'</command> + <inputs> + <page> + <param name="input_file" type="data" format="fastqsanger,fastqillumina,fastqsolexa,fastqsolid" label="FASTQ File" help="Requires groomed data: if your data does not appear here try using the FASTQ groomer."/> + <param name="min_size" label="Minimum Size" value="0" type="integer"> + <validator type="in_range" message="Minimum size must be positive" min="0"/> + </param> + <param name="max_size" label="Maximum Size" value="0" type="integer" help="A maximum size less than 1 indicates no limit."/> + <param name="min_quality" label="Minimum Quality" value="0" type="float"/> + <param name="max_quality" label="Maximum Quality" value="0" type="float" help="A maximum quality less than 1 indicates no limit."/> + <param name="max_num_deviants" label="Maximum number of bases allowed outside of quality range" value="0" type="integer"> + <validator type="in_range" message="Maximum number of deviate bases must be positive" min="0"/> + </param> + <param name="paired_end" label="This is paired end data" type="boolean" truevalue="paired_end" falsevalue="single_end" checked="False"/> + <repeat name="fastq_filters" title="Quality Filter on a Range of Bases" help="The above settings do not apply to these filters."> + <conditional name="offset_type"> + <param name="base_offset_type" type="select" label="Define Base Offsets as"> + <option value="offsets_absolute" selected="true">Absolute Values</option> + <option value="offsets_percent">Percentage of Read Length</option> + </param> + <when value="offsets_absolute"> + <param name="left_column_offset" label="Absolute Left Base Offset" value="0" type="integer" help="Values start at 0, increasing from the left"> + <validator type="in_range" message="Base Offsets must be positive" min="0" max="inf"/> + <validator type="expression" message="An integer is required.">int( float( value ) ) == float( value )</validator> + </param> + <param name="right_column_offset" label="Absolute Right Base Offset" value="0" type="integer" help="Values start at 0, increasing from the right"> + <validator type="in_range" message="Base Offsets must be positive" min="0" max="inf"/> + <validator type="expression" message="An integer is required.">int( float( value ) ) == float( value )</validator> + </param> + </when> + <when value="offsets_percent"> + <param name="left_column_offset" label="Percentage Left Base Offset" value="0" type="float"> + <validator type="in_range" message="Base Offsets must be between 0 and 100" min="0" max="100"/> + </param> + <param name="right_column_offset" label="Percentage Right Base Offset" value="0" type="float"> + <validator type="in_range" message="Base Offsets must be between 0 and 100" min="0" max="100"/> + </param> + </when> + </conditional> + <param name="score_operation" type="select" label="Aggregate read score for specified range"> + <option value="min" selected="True">min score</option> + <option value="max">max score</option> + <option value="sum">sum of scores</option> + <option value="mean">mean of scores</option> + </param> + <param name="score_comparison" type="select" label="Keep read when aggregate score is"> + <option value=">">></option> + <option value=">=" selected="true">>=</option> + <option value="==">==</option> + <option value="<"><</option> + <option value="<="><=</option> + <sanitizer sanitize="False"/> + </param> + <param name="score" label="Quality Score" value="0" type="float" /> + </repeat> + </page> + </inputs> + <configfiles> + <configfile name="fastq_filter_file"> +def fastq_read_pass_filter( fastq_read ): + def mean( score_list ): + return float( sum( score_list ) ) / float( len( score_list ) ) + if len( fastq_read ) < $min_size: + return False + if $max_size > 0 and len( fastq_read ) > $max_size: + return False + num_deviates = $max_num_deviants + qual_scores = fastq_read.get_decimal_quality_scores() + for qual_score in qual_scores: + if qual_score < $min_quality or ( $max_quality > 0 and qual_score > $max_quality ): + if num_deviates == 0: + return False + else: + num_deviates -= 1 +#if $paired_end.value == 'single_end': + qual_scores_split = [ qual_scores ] +#else: + qual_scores_split = [ qual_scores[ 0:int( len( qual_scores ) / 2 ) ], qual_scores[ int( len( qual_scores ) / 2 ): ] ] +#end if +#for $fastq_filter in $fastq_filters: + for split_scores in qual_scores_split: + left_column_offset = $fastq_filter[ 'offset_type' ][ 'left_column_offset' ] + right_column_offset = $fastq_filter[ 'offset_type' ][ 'right_column_offset' ] +#if $fastq_filter[ 'offset_type' ]['base_offset_type'] == 'offsets_percent': + left_column_offset = int( round( float( left_column_offset ) / 100.0 * float( len( split_scores ) ) ) ) + right_column_offset = int( round( float( right_column_offset ) / 100.0 * float( len( split_scores ) ) ) ) +#end if + if right_column_offset > 0: + split_scores = split_scores[ left_column_offset:-right_column_offset] + else: + split_scores = split_scores[ left_column_offset:] + if split_scores: ##if a read doesn't have enough columns, it passes by default + if not ( ${fastq_filter[ 'score_operation' ]}( split_scores ) $fastq_filter[ 'score_comparison' ] $fastq_filter[ 'score' ] ): + return False +#end for + return True +ret_val = fastq_read_pass_filter( fastq_read ) +</configfile> + </configfiles> + <outputs> + <data format="input" name="output_file" /> + </outputs> + <tests> +  + <test> + <param name="input_file" value="sanger_full_range_original_sanger.fastqsanger" ftype="fastqsanger"/> + <param name="min_size" value="0"/> + <param name="max_size" value="0"/> + <param name="min_quality" value="0"/> + <param name="max_quality" value="0"/> + <param name="max_num_deviants" value="0"/> + <param name="paired_end" value="single_end"/> + <param name="base_offset_type" value="offsets_absolute"/> + <param name="left_column_offset" value="0"/> + <param name="right_column_offset" value="0"/> + <param name="score_operation" value="min"/> + <param name="score_comparison" value=">="/> + <param name="score" value="0"/> + <output name="out_file1" file="sanger_full_range_original_sanger.fastqsanger"/> + </test> + <test> + <param name="input_file" value="solexa_full_range_original_solexa.fastqsolexa" ftype="fastqsolexa"/> + <param name="min_size" value="0"/> + <param name="max_size" value="0"/> + <param name="min_quality" value="-5"/> + <param name="max_quality" value="0"/> + <param name="max_num_deviants" value="0"/> + <param name="paired_end" value="single_end"/> + <param name="base_offset_type" value="offsets_absolute"/> + <param name="left_column_offset" value="0"/> + <param name="right_column_offset" value="0"/> + <param name="score_operation" value="min"/> + <param name="score_comparison" value=">="/> + <param name="score" value="-5"/> + <output name="out_file1" file="solexa_full_range_original_solexa.fastqsolexa"/> + </test> +  + <test> + <param name="input_file" value="sanger_full_range_as_solid.fastqsolid" ftype="fastqsolid"/> + <param name="min_size" value="0"/> + <param name="max_size" value="0"/> + <param name="min_quality" value="0"/> + <param name="max_quality" value="0"/> + <param name="max_num_deviants" value="0"/> + <param name="paired_end" value="single_end"/> + <param name="base_offset_type" value="offsets_absolute"/> + <param name="left_column_offset" value="0"/> + <param name="right_column_offset" value="0"/> + <param name="score_operation" value="min"/> + <param name="score_comparison" value=">="/> + <param name="score" value="0"/> + <output name="out_file1" file="sanger_full_range_as_solid.fastqsolid"/> + </test> +  + <test> + <param name="input_file" value="sanger_full_range_original_sanger.fastqsanger" ftype="fastqsanger"/> + <param name="min_size" value="0"/> + <param name="max_size" value="0"/> + <param name="min_quality" value="1"/> + <param name="max_quality" value="0"/> + <param name="max_num_deviants" value="0"/> + <param name="paired_end" value="single_end"/> + <param name="base_offset_type" value="offsets_absolute"/> + <param name="left_column_offset" value="0"/> + <param name="right_column_offset" value="0"/> + <param name="score_operation" value="min"/> + <param name="score_comparison" value=">="/> + <param name="score" value="0"/> + <output name="out_file1" file="empty_file.dat"/> + </test> + <test> + <param name="input_file" value="solexa_full_range_original_solexa.fastqsolexa" ftype="fastqsolexa"/> + <param name="min_size" value="0"/> + <param name="max_size" value="0"/> + <param name="min_quality" value="-4"/> + <param name="max_quality" value="0"/> + <param name="max_num_deviants" value="0"/> + <param name="paired_end" value="single_end"/> + <param name="base_offset_type" value="offsets_absolute"/> + <param name="left_column_offset" value="0"/> + <param name="right_column_offset" value="0"/> + <param name="score_operation" value="min"/> + <param name="score_comparison" value=">="/> + <param name="score" value="-5"/>

1 0

[hg] galaxy 3428: update buildbot_setup.sh for new data location
by Greg Von Kuster 24 Feb '10

24 Feb '10

details: http://www.bx.psu.edu/hg/galaxy/rev/03dbef081cf2 changeset: 3428:03dbef081cf2 user: Nate Coraor <nate(a)bx.psu.edu> date: Tue Feb 23 16:12:20 2010 -0500 description: update buildbot_setup.sh for new data location diffstat: buildbot_setup.sh | 47 ++++++++++++++++++++++++++--------------------- 1 files changed, 26 insertions(+), 21 deletions(-) diffs (73 lines): diff -r 6dbe31bbbf02 -r 03dbef081cf2 buildbot_setup.sh --- a/buildbot_setup.sh Tue Feb 23 15:14:28 2010 -0500 +++ b/buildbot_setup.sh Tue Feb 23 16:12:20 2010 -0500 @@ -19,31 +19,34 @@ done [ ! -d "$HYPHY" ] && unset HYPHY ;; + solaris2.10) + HYPHY="/galaxy/software/linux2.6-x86_64/hyphy" + ;; esac LINKS=" -/depot/data2/galaxy/alignseq.loc -/depot/data2/galaxy/annotation_profiler/annotation_profiler.loc -/depot/data2/galaxy/annotation_profiler/annotation_profiler_options.xml -/depot/data2/galaxy/annotation_profiler/annotation_profiler_valid_builds.txt -/depot/data2/galaxy/binned_scores.loc -/depot/data2/galaxy/blastdb.loc -/depot/data2/galaxy/bowtie_indices.loc -/depot/data2/galaxy/bowtie_indices_color.loc -/depot/data2/galaxy/bwa_index.loc -/depot/data2/galaxy/encode_datasets.loc +/galaxy/data/location/alignseq.loc +/galaxy/data/annotation_profiler/annotation_profiler.loc +/galaxy/data/annotation_profiler/annotation_profiler_options.xml +/galaxy/data/annotation_profiler/annotation_profiler_valid_builds.txt +/galaxy/data/location/binned_scores.loc +/galaxy/data/location/blastdb.loc +/galaxy/data/location/bowtie_indices.loc +/galaxy/data/location/bowtie_indices_color.loc +/galaxy/data/location/bwa_index.loc +/galaxy/data/location/encode_datasets.loc /galaxy/home/universe/encode_feature_partitions -/depot/data2/galaxy/lastz_seqs.loc -/depot/data2/galaxy/liftOver.loc -/depot/data2/galaxy/maf_index.loc -/depot/data2/galaxy/maf_pairwise.loc -/depot/data2/galaxy/microbes/microbial_data.loc -/depot/data2/galaxy/phastOdds.loc -/depot/data2/galaxy/quality_scores.loc -/depot/data2/galaxy/regions.loc -/depot/data2/galaxy/sam_fa_indices.loc -/depot/data2/galaxy/taxonomy -/depot/data2/galaxy/twobit.loc +/galaxy/data/location/lastz_seqs.loc +/galaxy/data/location/liftOver.loc +/galaxy/data/location/maf_index.loc +/galaxy/data/location/maf_pairwise.loc +/galaxy/data/location/microbes/microbial_data.loc +/galaxy/data/location/phastOdds.loc +/galaxy/data/location/quality_scores.loc +/galaxy/data/location/regions.loc +/galaxy/data/location/sam_fa_indices.loc +/galaxy/data/location/taxonomy +/galaxy/data/location/twobit.loc " SAMPLES=" @@ -63,11 +66,13 @@ for link in $LINKS; do echo "Linking $link" + rm -f tool-data/`basename $link` ln -sf $link tool-data done if [ -d "$HYPHY" ]; then echo "Linking $HYPHY" + rm -f tool-data/HYPHY ln -sf $HYPHY tool-data/HYPHY fi

1 0

[hg] galaxy 3427: Modify functional_tests.py to allow for testin...
by Greg Von Kuster 24 Feb '10

24 Feb '10

details: http://www.bx.psu.edu/hg/galaxy/rev/6dbe31bbbf02 changeset: 3427:6dbe31bbbf02 user: Nate Coraor <nate(a)bx.psu.edu> date: Tue Feb 23 15:14:28 2010 -0500 description: Modify functional_tests.py to allow for testing of an environment nearly identical to Galaxy Test/Main via buildbot diffstat: scripts/functional_tests.py | 132 +++++++++++++++++++++++++++++++++----------- 1 files changed, 99 insertions(+), 33 deletions(-) diffs (206 lines): diff -r 1f05b0c4071f -r 6dbe31bbbf02 scripts/functional_tests.py --- a/scripts/functional_tests.py Tue Feb 23 10:40:19 2010 -0500 +++ b/scripts/functional_tests.py Tue Feb 23 15:14:28 2010 -0500 @@ -1,6 +1,6 @@ #!/usr/bin/env python -import os, sys +import os, sys, shutil # Assume we are run from the galaxy root directory, add lib to the python path cwd = os.getcwd() @@ -52,32 +52,68 @@ if not os.path.isabs( galaxy_test_file_dir ): galaxy_test_file_dir = os.path.join( os.getcwd(), galaxy_test_file_dir ) start_server = 'GALAXY_TEST_EXTERNAL' not in os.environ + tool_path = os.environ.get( 'GALAXY_TEST_TOOL_PATH', 'tools' ) if start_server: - if 'GALAXY_TEST_DBPATH' in os.environ: - db_path = os.environ['GALAXY_TEST_DBPATH'] - else: - tempdir = tempfile.mkdtemp() - db_path = os.path.join( tempdir, 'database' ) - file_path = os.path.join( db_path, 'files' ) - try: - os.makedirs( file_path ) - except OSError: - pass - if 'GALAXY_TEST_DBURI' in os.environ: - database_connection = os.environ['GALAXY_TEST_DBURI'] + psu_production = False + galaxy_test_proxy_port = None + if 'GALAXY_TEST_PSU_PRODUCTION' in os.environ: + if not galaxy_test_port: + raise Exception( 'Please set GALAXY_TEST_PORT to the port to which the proxy server will proxy' ) + galaxy_test_proxy_port = os.environ.get( 'GALAXY_TEST_PROXY_PORT', None ) + if not galaxy_test_proxy_port: + raise Exception( 'Please set GALAXY_TEST_PROXY_PORT to the port on which the proxy server is listening' ) + base_file_path = os.environ.get( 'GALAXY_TEST_BASE_FILE_PATH', None ) + if not base_file_path: + raise Exception( 'Please set GALAXY_TEST_BASE_FILE_PATH to the directory which will contain the dataset files directory' ) + base_new_file_path = os.environ.get( 'GALAXY_TEST_BASE_NEW_FILE_PATH', None ) + if not base_new_file_path: + raise Exception( 'Please set GALAXY_TEST_BASE_NEW_FILE_PATH to the directory which will contain the temporary directory' ) + database_connection = os.environ.get( 'GALAXY_TEST_DBURI', None ) + if not database_connection: + raise Exception( 'Please set GALAXY_TEST_DBURI to the URI of the database to be used for tests' ) + nginx_upload_store = os.environ.get( 'GALAXY_TEST_NGINX_UPLOAD_STORE', None ) + if not nginx_upload_store: + raise Exception( 'Please set GALAXY_TEST_NGINX_UPLOAD_STORE to the path where the nginx upload module places uploaded files' ) + file_path = tempfile.mkdtemp( dir=base_file_path ) + new_file_path = tempfile.mkdtemp( dir=base_new_file_path ) + cluster_files_directory = os.path.join( new_file_path, 'pbs' ) + job_working_directory = os.path.join( new_file_path, 'job_working_directory' ) + os.mkdir( cluster_files_directory ) + os.mkdir( job_working_directory ) + kwargs = dict( database_engine_option_pool_size = '10', + database_engine_option_max_overflow = '20', + database_engine_option_strategy = 'threadlocal', + nginx_x_accel_redirect_base = '/_x_accel_redirect', + nginx_upload_store = nginx_upload_store, + nginx_upload_path = '/_upload', + cluster_files_directory = cluster_files_directory, + job_working_directory = job_working_directory, + outputs_to_working_directory = 'True', + set_metadata_externally = 'True', + static_enabled = 'False', + debug = 'False', + track_jobs_in_database = 'True', + job_scheduler_policy = 'FIFO', + start_job_runners = 'pbs', + default_cluster_job_runner = 'pbs:///', ) + psu_production = True else: - database_connection = 'sqlite:///' + os.path.join( db_path, 'universe.sqlite' ) - if 'GALAXY_TEST_RUNNERS' in os.environ: - start_job_runners = os.environ['GALAXY_TEST_RUNNERS'] - else: - start_job_runners = None - if 'GALAXY_TEST_DEF_RUNNER' in os.environ: - default_cluster_job_runner = os.environ['GALAXY_TEST_DEF_RUNNER'] - else: - default_cluster_job_runner = 'local:///' - set_metadata_externally = False - if 'GALAXY_SET_METADATA_EXTERNALLY' in os.environ: - set_metadata_externally = True + if 'GALAXY_TEST_DBPATH' in os.environ: + db_path = os.environ['GALAXY_TEST_DBPATH'] + else: + tempdir = tempfile.mkdtemp() + db_path = os.path.join( tempdir, 'database' ) + file_path = os.path.join( db_path, 'files' ) + new_file_path = os.path.join( db_path, 'tmp' ) + if 'GALAXY_TEST_DBURI' in os.environ: + database_connection = os.environ['GALAXY_TEST_DBURI'] + else: + database_connection = 'sqlite:///' + os.path.join( db_path, 'universe.sqlite' ) + try: + os.makedirs( file_path ) + except OSError: + pass + kwargs = {} print "Database connection:", database_connection @@ -91,17 +127,20 @@ if start_server: + global_conf = { '__file__' : 'universe_wsgi.ini.sample' } + if psu_production: + global_conf = None + # Build the Universe Application app = UniverseApplication( job_queue_workers = 5, - start_job_runners = start_job_runners, - default_cluster_job_runner = default_cluster_job_runner, id_secret = 'changethisinproductiontoo', template_path = "templates", database_connection = database_connection, file_path = file_path, + new_file_path = new_file_path, + tool_path = tool_path, tool_config_file = "tool_conf.xml.sample", datatype_converters_config_file = "datatype_converters_conf.xml.sample", - tool_path = "tools", tool_parse_help = False, test_conf = "test.conf", log_destination = "stdout", @@ -111,8 +150,8 @@ admin_users = 'test(a)bx.psu.edu', library_import_dir = galaxy_test_file_dir, user_library_import_dir = os.path.join( galaxy_test_file_dir, 'users' ), - set_metadata_externally = set_metadata_externally, - global_conf = { "__file__": "universe_wsgi.ini.sample" } ) + global_conf = global_conf, + **kwargs ) log.info( "Embedded Universe application started" ); @@ -140,14 +179,17 @@ raise else: raise Exception( "Unable to open a port between %s and %s to start Galaxy server" % ( default_galaxy_test_port_min, default_galaxy_test_port_max ) ) - os.environ['GALAXY_TEST_PORT'] = galaxy_test_port + if galaxy_test_proxy_port: + os.environ['GALAXY_TEST_PORT'] = galaxy_test_proxy_port + else: + os.environ['GALAXY_TEST_PORT'] = galaxy_test_port t = threading.Thread( target=server.serve_forever ) t.start() # Test if the server is up for i in range( 10 ): - conn = httplib.HTTPConnection( galaxy_test_host, galaxy_test_port ) + conn = httplib.HTTPConnection( galaxy_test_host, galaxy_test_port ) # directly test the app, not the proxy conn.request( "GET", "/" ) if conn.getresponse().status == 200: break @@ -155,6 +197,13 @@ else: raise Exception( "Test HTTP server did not return '200 OK' after 10 tries" ) + # Test if the proxy server is up + if psu_production: + conn = httplib.HTTPConnection( galaxy_test_host, galaxy_test_proxy_port ) # directly test the app, not the proxy + conn.request( "GET", "/" ) + if not conn.getresponse().status == 200: + raise Exception( "Test HTTP proxy server did not return '200 OK'" ) + log.info( "Embedded web server started" ) @@ -179,7 +228,10 @@ # ---- Find tests --------------------------------------------------------- - log.info( "Functional tests will be run against %s:%s" % ( galaxy_test_host, galaxy_test_port ) ) + if galaxy_test_proxy_port: + log.info( "Functional tests will be run against %s:%s" % ( galaxy_test_host, galaxy_test_proxy_port ) ) + else: + log.info( "Functional tests will be run against %s:%s" % ( galaxy_test_host, galaxy_test_port ) ) success = False @@ -231,6 +283,20 @@ app.shutdown() app = None log.info( "Embedded Universe application stopped" ) + try: + if os.path.exists( tempdir ) and 'GALAXY_TEST_NO_CLEANUP' not in os.environ: + log.info( "Cleaning up temporary files in %s" % tempdir ) + shutil.rmtree( tempdir ) + except: + pass + if psu_production and 'GALAXY_TEST_NO_CLEANUP' not in os.environ: + for dir in ( file_path, new_file_path ): + try: + if os.path.exists( dir ): + log.info( 'Cleaning up temporary files in %s' % dir ) + shutil.rmtree( dir ) + except: + pass if success: return 0

1 0