October 2009 - galaxy-dev - lists.galaxyproject.org

[hg] galaxy 2847: Added 'Median' operation to grouping tool
by Greg Von Kuster 08 Oct '09

08 Oct '09

details: http://www.bx.psu.edu/hg/galaxy/rev/39ccea195b8e changeset: 2847:39ccea195b8e user: gua110 date: Thu Oct 08 10:37:26 2009 -0400 description: Added 'Median' operation to grouping tool 1 file(s) affected in this change: tools/stats/grouping.xml diffs (17 lines): diff -r f0d565ab7028 -r 39ccea195b8e tools/stats/grouping.xml --- a/tools/stats/grouping.xml Thu Oct 08 10:07:35 2009 -0400 +++ b/tools/stats/grouping.xml Thu Oct 08 10:37:26 2009 -0400 @@ -1,4 +1,4 @@ -<tool id="Grouping1" name="Group" version="1.8.0"> +<tool id="Grouping1" name="Group" version="1.9.0"> <description>data by a column and perform aggregate operation on other columns.</description> <command interpreter="python"> grouping.py @@ -21,6 +21,7 @@ <repeat name="operations" title="Operation"> <param name="optype" type="select" label="Type"> <option value="mean">Mean</option> + <option value="median">Median</option> <option value="max">Maximum</option> <option value="min">Minimum</option> <option value="sum">Sum</option>

1 0

[hg] galaxy 2849: Refactored the form validation code
by Greg Von Kuster 08 Oct '09

08 Oct '09

details: http://www.bx.psu.edu/hg/galaxy/rev/674bb364697a changeset: 2849:674bb364697a user: rc date: Thu Oct 08 11:14:04 2009 -0400 description: Refactored the form validation code 1 file(s) affected in this change: lib/galaxy/web/controllers/forms.py diffs (43 lines): diff -r a105810a5da8 -r 674bb364697a lib/galaxy/web/controllers/forms.py --- a/lib/galaxy/web/controllers/forms.py Thu Oct 08 10:45:31 2009 -0400 +++ b/lib/galaxy/web/controllers/forms.py Thu Oct 08 11:14:04 2009 -0400 @@ -91,8 +91,8 @@ self.current_form = {} self.current_form[ 'name' ] = 'New Form' self.current_form[ 'desc' ] = '' - self.current_form['type'] = 'none' - self.current_form[ 'layout' ] = ['Main'] + self.current_form[ 'type' ] = params.get( 'form_type', 'none' ) + self.current_form[ 'layout' ] = [ 'Main' ] self.current_form[ 'fields' ] = [] inputs = [ ( 'Name', TextField( 'name', 40, self.current_form[ 'name' ] ) ), ( 'Description', TextField( 'description', 40, self.current_form[ 'desc' ] ) ), @@ -286,9 +286,9 @@ if util.restore_text( params.form_type_selectbox ) == 'none': return None, 'Form type must be selected.' # fields - for i in range( len(self.current_form['fields']) ): - if not util.restore_text(params.get( 'field_name_%i' % i, None )): - return None, "All the field label(s) must be completed." +# for i in range( len(self.current_form['fields']) ): +# if not util.restore_text(params.get( 'field_name_%i' % i, None )): +# return None, "All the field label(s) must be completed." return True, '' def __get_form(self, trans, **kwd): params = util.Params( kwd ) @@ -365,8 +365,13 @@ flag, msg = self.__validate_form(**kwd) if not flag: return None, msg - fd = trans.app.model.FormDefinition() - fd.name, fd.desc, fd.type, fd.layout, fd.fields = self.__get_form(trans, **kwd) + name, desc, form_type, layout, fields = self.__get_form(trans, **kwd) + # validate fields + for field in fields: + if not field[ 'label' ]: + return None, "All the field label(s) must be completed." + fd = trans.app.model.FormDefinition(name, desc, fields, current_form=None, + form_type=form_type, layout=layout ) if fdc_id: # save changes to the existing form # change the pointer in the form_definition_current table to point # to this new record

1 0

[hg] galaxy 2848: Removed sniff method from fastqsanger datatype
by Greg Von Kuster 08 Oct '09

08 Oct '09

details: http://www.bx.psu.edu/hg/galaxy/rev/a105810a5da8 changeset: 2848:a105810a5da8 user: Kelly Vincent <kpvincent(a)bx.psu.edu> date: Thu Oct 08 10:45:31 2009 -0400 description: Removed sniff method from fastqsanger datatype 4 file(s) affected in this change: datatypes_conf.xml.sample lib/galaxy/datatypes/registry.py lib/galaxy/datatypes/sequence.py test/functional/test_sniffing_and_metadata_settings.py diffs (113 lines): diff -r 39ccea195b8e -r a105810a5da8 datatypes_conf.xml.sample --- a/datatypes_conf.xml.sample Thu Oct 08 10:37:26 2009 -0400 +++ b/datatypes_conf.xml.sample Thu Oct 08 10:45:31 2009 -0400 @@ -23,7 +23,7 @@ <converter file="fasta_to_tabular_converter.xml" target_datatype="tabular"/> </datatype> <datatype extension="fastq" type="galaxy.datatypes.sequence:Fastq" display_in_upload="true"/> - <datatype extension="fastqsanger" type="galaxy.datatypes.sequence:FastqSanger" display_in_upload="true"/> + <datatype extension="fastqsanger" type="galaxy.datatypes.sequence:FastqSanger"/> <datatype extension="genetrack" type="galaxy.datatypes.tracks:GeneTrack"/> <datatype extension="gff" type="galaxy.datatypes.interval:Gff" display_in_upload="true"> <converter file="gff_to_bed_converter.xml" target_datatype="bed"/> @@ -197,7 +197,6 @@ <sniffer type="galaxy.datatypes.qualityscore:QualityScoreSOLiD"/> <sniffer type="galaxy.datatypes.qualityscore:QualityScore454"/> <sniffer type="galaxy.datatypes.sequence:Fasta"/> - <sniffer type="galaxy.datatypes.sequence:FastqSanger"/> <sniffer type="galaxy.datatypes.sequence:Fastq"/> <sniffer type="galaxy.datatypes.interval:Wiggle"/> <sniffer type="galaxy.datatypes.images:Html"/> diff -r 39ccea195b8e -r a105810a5da8 lib/galaxy/datatypes/registry.py --- a/lib/galaxy/datatypes/registry.py Thu Oct 08 10:37:26 2009 -0400 +++ b/lib/galaxy/datatypes/registry.py Thu Oct 08 10:45:31 2009 -0400 @@ -179,7 +179,6 @@ qualityscore.QualityScoreSOLiD(), qualityscore.QualityScore454(), sequence.Fasta(), - sequence.FastqSanger(), sequence.Fastq(), interval.Wiggle(), images.Html(), diff -r 39ccea195b8e -r a105810a5da8 lib/galaxy/datatypes/sequence.py --- a/lib/galaxy/datatypes/sequence.py Thu Oct 08 10:37:26 2009 -0400 +++ b/lib/galaxy/datatypes/sequence.py Thu Oct 08 10:45:31 2009 -0400 @@ -174,58 +174,10 @@ except: return False - class FastqSanger( Fastq ): """Class representing a FASTQ sequence ( the Sanger variant )""" file_ext = "fastqsanger" - def sniff( self, filename ): - """ - Determines whether the file is in fastqsanger format (Sanger Variant) - For details, see http://maq.sourceforge.net/fastq.shtml - - Note: There are three kinds of FASTQ files, known as "Sanger" (sometimes called "Standard"), Solexa, and Illumina - These differ in the representation of the quality scores - - >>> fname = get_test_fname( '1.fastqsanger' ) - >>> FastqSanger().sniff( fname ) - True - >>> fname = get_test_fname( '2.fastqsanger' ) - >>> FastqSanger().sniff( fname ) - True - """ - headers = get_headers( filename, None ) - bases_regexp = re.compile( "^[NGTAC]*$" ) - try: - if len( headers ) >= 4 and headers[0][0] and headers[0][0][0] == "@" and headers[2][0] and headers[2][0][0] == "+" and headers[1][0]: - # look through first 20 blocks and make sure bases valid and qualities valid - for i in range( 1, 80, 4 ): - try: - # check that bases are legitimate - if not bases_regexp.match( headers[i][0] ): - return False - # check length of qualities (matching bases) - if len( headers[i+2][0] ) != len( headers[1][0] ): - return False - # check qualities within fastqsanger range - if not self.check_qual_values_within_range( headers[i+2][0] ): - return False - except IndexError: - pass - return True - return False - except: - return False - def check_qual_values_within_range( self, qual_seq ): - under59 = False - for val in qual_seq: - if ord(val) < 33 or ord(val) > 126: - return False - if not under59 and ord(val) < 59: - under59 = True - if under59: - return True - return False try: from galaxy import eggs diff -r 39ccea195b8e -r a105810a5da8 test/functional/test_sniffing_and_metadata_settings.py --- a/test/functional/test_sniffing_and_metadata_settings.py Thu Oct 08 10:37:26 2009 -0400 +++ b/test/functional/test_sniffing_and_metadata_settings.py Thu Oct 08 10:45:31 2009 -0400 @@ -206,16 +206,6 @@ assert latest_hda is not None, "Problem retrieving wig hda from the database" if not latest_hda.name == '1.wig' and not latest_hda.extension == 'wig': raise AssertionError, "wig data type was not correctly sniffed." - def test_085_fastqsanger_datatype( self ): - """Testing correctly sniffing fastqsanger ( the Sanger variant ) data type upon upload""" - self.upload_file( '1.fastqsanger' ) - self.verify_dataset_correctness( '1.fastqsanger' ) - self.check_history_for_string( '1.fastqsanger format: <span class="fastqsanger">fastqsanger</span>, database: \? Info: uploaded fastqsanger file' ) - latest_hda = galaxy.model.HistoryDatasetAssociation.query() \ - .order_by( desc( galaxy.model.HistoryDatasetAssociation.table.c.create_time ) ).first() - assert latest_hda is not None, "Problem retrieving fastqsanger hda from the database" - if not latest_hda.name == '1.fastqsanger' and not latest_hda.extension == 'fastqsanger': - raise AssertionError, "fastqsanger data type was not correctly sniffed." def test_090_sam_datatype( self ): """Testing correctly sniffing sam format upon upload""" self.upload_file( '1.sam' )

1 0

[hg] galaxy 2850: Fixed ascending/descending bug on Sort, and ad...
by Greg Von Kuster 08 Oct '09

08 Oct '09

details: http://www.bx.psu.edu/hg/galaxy/rev/d146ca99434f changeset: 2850:d146ca99434f user: Kelly Vincent <kpvincent(a)bx.psu.edu> date: Thu Oct 08 11:33:43 2009 -0400 description: Fixed ascending/descending bug on Sort, and added ability to specify ascending or descending on each column. 2 file(s) affected in this change: tools/filters/sorter.py tools/filters/sorter.xml diffs (100 lines): diff -r 674bb364697a -r d146ca99434f tools/filters/sorter.py --- a/tools/filters/sorter.py Thu Oct 08 11:14:04 2009 -0400 +++ b/tools/filters/sorter.py Thu Oct 08 11:33:43 2009 -0400 @@ -26,19 +26,20 @@ try: inputfile = options.input outputfile = '-o %s' % options.out_file1 - order = ('', '-r')[options.order == 'DESC'] columns = [options.column] styles = [('','n')[options.style == 'num']] - col_styles = sys.argv[6:] - if len(col_styles) > 1: - columns.extend([col_styles[i] for i in range(0,len(col_styles),2)]) - styles.extend([('','n')[col_styles[i] == 'num'] for i in range(1,len(col_styles),2)]) - cols = [ '-k%s,%s%s'%(columns[i], columns[i], styles[i]) for i in range(len(columns)) ] + orders = [('','r')[options.order == 'DESC']] + col_style_orders = sys.argv[6:] + if len(col_style_orders) > 1: + columns.extend([col_style_orders[i] for i in range(0,len(col_style_orders),3)]) + styles.extend([('','n')[col_style_orders[i] == 'num'] for i in range(1,len(col_style_orders),3)]) + orders.extend([('','r')[col_style_orders[i] == 'DESC'] for i in range(2,len(col_style_orders),3)]) + cols = [ '-k%s,%s%s%s'%(columns[i], columns[i], styles[i], orders[i]) for i in range(len(columns)) ] except Exception, ex: stop_err('Error parsing input parameters\n' + str(ex)) # Launch sort. - cmd = "sort -f -t $'\t' %s %s %s %s" % (order, ' '.join(cols), outputfile, inputfile) + cmd = "sort -f -t $'\t' %s %s %s" % (' '.join(cols), outputfile, inputfile) try: os.system(cmd) except Exception, ex: diff -r 674bb364697a -r d146ca99434f tools/filters/sorter.xml --- a/tools/filters/sorter.xml Thu Oct 08 11:14:04 2009 -0400 +++ b/tools/filters/sorter.xml Thu Oct 08 11:33:43 2009 -0400 @@ -10,6 +10,7 @@ #for $col in $column_set: ${col.other_column} ${col.other_style} + ${col.other_order} #end for </command> <inputs> @@ -19,17 +20,21 @@ <option value="num">Numerical sort</option> <option value="alpha">Alphabetical sort</option> </param> + <param name="order" type="select" label="everything in"> + <option value="DESC">Descending order</option> + <option value="ASC">Ascending order</option> + </param> <repeat name="column_set" title="Column selection"> <param name="other_column" label="on column" type="data_column" data_ref="input" accept_default="true" /> <param name="other_style" type="select" label="with flavor"> <option value="num">Numerical sort</option> <option value="alpha">Alphabetical sort</option> </param> + <param name="other_order" type="select" label="everything in"> + <option value="DESC">Descending order</option> + <option value="ASC">Ascending order</option> + </param> </repeat> - <param name="order" type="select" label="everything in"> - <option value="DESC">Descending order</option> - <option value="ASC">Ascending order</option> - </param> </inputs> <outputs> <data format="input" name="out_file1" metadata_source="input"/> @@ -39,18 +44,20 @@ <param name="input" value="sort_in1.bed"/> <param name="column" value="1"/> <param name="style" value="num"/> + <param name="order" value="ASC"/> <param name="other_column" value="3"/> <param name="other_style" value="num"/> - <param name="order" value="ASC"/> + <param name="other_order" value="ASC"/> <output name="out_file1" file="sort_out1.bed"/> </test> <test> <param name="input" value="sort_in1.bed"/> <param name="column" value="3"/> <param name="style" value="alpha"/> + <param name="order" value="ASC"/> <param name="other_column" value="1"/> <param name="other_style" value="alpha"/> - <param name="order" value="ASC"/> + <param name="other_order" value="ASC"/> <output name="out_file1" file="sort_out2.bed"/> </test> </tests> @@ -64,7 +71,7 @@ **Syntax** -This tool sorts the dataset on any number of columns in either ascending or descending order (all columns will be ascending or descending). +This tool sorts the dataset on any number of columns in either ascending or descending order. * Numerical sort orders numbers by their magnitude, ignores all characters besides numbers, and evaluates a string of numbers to the value they signify. * Alphabetical sort is a phonebook type sort based on the conventional order of letters in an alphabet. Each nth letter is compared with the nth letter of other words in the list, starting at the first letter of each word and advancing to the second, third, fourth, and so on, until the order is established. Therefore, in an alphabetical sort, 2 comes after 100 (1 < 2).

1 0

[hg] galaxy 2851: More interface updates
by Greg Von Kuster 08 Oct '09

08 Oct '09

details: http://www.bx.psu.edu/hg/galaxy/rev/c8ba2a78e696 changeset: 2851:c8ba2a78e696 user: Anton Nekrutenko <anton(a)bx.psu.edu> date: Thu Oct 08 13:33:54 2009 -0400 description: More interface updates 2 file(s) affected in this change: tools/sr_mapping/bowtie_wrapper.xml tools/sr_mapping/bwa_wrapper.xml diffs (46 lines): diff -r d146ca99434f -r c8ba2a78e696 tools/sr_mapping/bowtie_wrapper.xml --- a/tools/sr_mapping/bowtie_wrapper.xml Thu Oct 08 11:33:43 2009 -0400 +++ b/tools/sr_mapping/bowtie_wrapper.xml Thu Oct 08 13:33:54 2009 -0400 @@ -426,9 +426,19 @@ **What it does** -Bowtie_ is a short read aligner designed to be ultrafast and memory-efficient. Reads can be as long as 1024 base pairs, though shorter is better. Bowtie produces a specific output format which is converted to SAM by this tool. +Bowtie_ is a short read aligner designed to be ultrafast and memory-efficient. It is developed by Ben Langmead and Cole Trapnell. .. _Bowtie: http://bowtie-bio.sourceforge.net/index.shtml + +------ + +**Know what you are doing** + +.. class:: warningmark + +There is no such thing (yet) as automated gearshift in short read mapping. It is all like stick-shift driving in San Francisco. In other words = running this tool with default parameters will probably not give you meaningful results. A way to deal with this is to **understand** the parameters by carefully reading `documentation`__ and experimenting. Fortunaly, Galaxy makes experimenting easy. + + .. __: http://bowtie-bio.sourceforge.net/index.shtml ------ diff -r d146ca99434f -r c8ba2a78e696 tools/sr_mapping/bwa_wrapper.xml --- a/tools/sr_mapping/bwa_wrapper.xml Thu Oct 08 11:33:43 2009 -0400 +++ b/tools/sr_mapping/bwa_wrapper.xml Thu Oct 08 13:33:54 2009 -0400 @@ -291,7 +291,17 @@ **What it does** -**BWA** is a high performance sequence aligner that succeeds MAQ. It is based on BWT-SW but uses a completely different algorithm and is aimed towards short read alignments. It is fast--it can map the human genome in only 15-25 minutes. Heng Li of the Sanger Institute wrote the majority of the code, with contributions by Chi-Kwong Wong at the University of Hong Kong, Nong Ge at Sun Yat-Sen University, and Yuta Mori. +BWA is a fast light-weighted tool that aligns relatively short sequences (queries) to a sequence database (large), such as the human reference genome. It is developed by Heng Li at the Sanger Insitute. + +------ + +**Know what you are doing** + +.. class:: warningmark + +There is no such thing (yet) as automated gearshift in short read mapping. It is all like stick-shift driving in San Francisco. In other words = running this tool with default parameters will probably not give you meaningful results. A way to deal with this is to **understand** the parameters by carefully reading `documentation`__ and experimenting. Fortunaly, Galaxy makes experimenting easy. + + .. __: http://bio-bwa.sourceforge.net/ ------

1 0

[hg] galaxy 2843: Updated several sample index loc files to make...
by Greg Von Kuster 08 Oct '09

08 Oct '09

details: http://www.bx.psu.edu/hg/galaxy/rev/ebe3e881ac25 changeset: 2843:ebe3e881ac25 user: Kelly Vincent <kpvincent(a)bx.psu.edu> date: Wed Oct 07 15:31:18 2009 -0400 description: Updated several sample index loc files to make it clearer how the actual loc files should appear 4 file(s) affected in this change: tool-data/bowtie_indices.loc.sample tool-data/sam_fa_indices.loc.sample tool-data/sequence_index_base.loc.sample tool-data/sequence_index_color.loc.sample diffs (89 lines): diff -r 31c577c6fd49 -r ebe3e881ac25 tool-data/bowtie_indices.loc.sample --- a/tool-data/bowtie_indices.loc.sample Wed Oct 07 15:25:16 2009 -0400 +++ b/tool-data/bowtie_indices.loc.sample Wed Oct 07 15:31:18 2009 -0400 @@ -1,8 +1,8 @@ #This is a sample file distributed with Galaxy that enables tools #to use a directory of Bowtie indexed sequences data files. You will need #to create these data files and then create a bowtie_indices.loc file -#similar to this one (store it in this directory ) that points to -#the directories in which those files are stored. The bowtie_indices.loc +#similar to this one (store it in this directory) that points to +#the directories in which those files are stored. The bowtie_indices.loc #file has this format (white space characters are TAB characters): # #<build> <file_base> @@ -26,3 +26,4 @@ #exist, but it is the prefix for the actual index files. For example: # #hg18 /depot/data2/galaxy/bowtie/hg18/hg18 +#hg19 /depot/data2/galaxy/bowtie/hg19/hg19 diff -r 31c577c6fd49 -r ebe3e881ac25 tool-data/sam_fa_indices.loc.sample --- a/tool-data/sam_fa_indices.loc.sample Wed Oct 07 15:25:16 2009 -0400 +++ b/tool-data/sam_fa_indices.loc.sample Wed Oct 07 15:31:18 2009 -0400 @@ -1,17 +1,17 @@ #This is a sample file distributed with Galaxy that enables tools #to use a directory of Samtools indexed sequences data files. You will need #to create these data files and then create a sam_fa_indices.loc file -#similar to this one (store it in this directory ) that points to -#the directories in which those files are stored. The sam_fa_indices.loc +#similar to this one (store it in this directory) that points to +#the directories in which those files are stored. The sam_fa_indices.loc #file has this format (white space characters are TAB characters): # -#<index> <seq> <location> +#index <seq> <location> # #So, for example, if you had hg18 indexed stored in #/depot/data2/galaxy/sam/, #then the sam_fa_indices.loc entry would look like this: # -#hg18 /depot/data2/galaxy/sam/hg18.fa +#index hg18 /depot/data2/galaxy/sam/hg18.fa # #and your /depot/data2/galaxy/sam/ directory #would contain hg18.fa and hg18.fa.fai files: @@ -24,4 +24,5 @@ #exist, but it should never be directly used. Instead, the name serves #as a prefix for the index file. For example: # -#hg18 /depot/data2/galaxy/sam/hg18.fa +#index hg18 /depot/data2/galaxy/sam/hg18.fa +#index hg19 /depot/data2/galaxy/sam/hg19.fa diff -r 31c577c6fd49 -r ebe3e881ac25 tool-data/sequence_index_base.loc.sample --- a/tool-data/sequence_index_base.loc.sample Wed Oct 07 15:25:16 2009 -0400 +++ b/tool-data/sequence_index_base.loc.sample Wed Oct 07 15:31:18 2009 -0400 @@ -1,8 +1,8 @@ #This is a sample file distributed with Galaxy that enables tools #to use a directory of BWA indexed sequences data files. You will need #to create these data files and then create a sequence_index_base.loc file -#similar to this one (store it in this directory ) that points to -#the directories in which those files are stored. The sequence_index_base.loc +#similar to this one (store it in this directory) that points to +#the directories in which those files are stored. The sequence_index_base.loc #file has this format (white space characters are TAB characters): # #<build> <file_base> @@ -26,3 +26,4 @@ #exist, but it is the prefix for the actual index files. For example: # #phiX /depot/data2/galaxy/phiX/base/phiX.fa +#hg18 /depot/data2/galaxy/hg18/base/hg18.fa diff -r 31c577c6fd49 -r ebe3e881ac25 tool-data/sequence_index_color.loc.sample --- a/tool-data/sequence_index_color.loc.sample Wed Oct 07 15:25:16 2009 -0400 +++ b/tool-data/sequence_index_color.loc.sample Wed Oct 07 15:31:18 2009 -0400 @@ -1,8 +1,8 @@ #This is a sample file distributed with Galaxy that enables tools #to use a directory of BWA indexed sequences data files. You will need #to create these data files and then create a sequence_index_color.loc file -#similar to this one (store it in this directory ) that points to -#the directories in which those files are stored. The sequence_index_color.loc +#similar to this one (store it in this directory) that points to +#the directories in which those files are stored. The sequence_index_color.loc #file has this format (white space characters are TAB characters): # #<build> <file_base> @@ -26,3 +26,4 @@ #exist, but it is the prefix for the actual index files. For example: # #phiX /depot/data2/galaxy/phiX/color/phiX.fa +#hg18 /depot/data2/galaxy/hg18/color/hg18.fa

1 0

[hg] galaxy 2844: Incorporate code to provide UCSC and Gbrowse i...
by Greg Von Kuster 08 Oct '09

08 Oct '09

details: http://www.bx.psu.edu/hg/galaxy/rev/e73efc9387ee changeset: 2844:e73efc9387ee user: Greg Von Kuster <greg(a)bx.psu.edu> date: Wed Oct 07 16:37:48 2009 -0400 description: Incorporate code to provide UCSC and Gbrowse integration for wiggle files contributed by Brad Chapman - handles ticket # 134. 3 file(s) affected in this change: lib/galaxy/datatypes/genetics.py lib/galaxy/datatypes/interval.py lib/galaxy/datatypes/tabular.py diffs (263 lines): diff -r ebe3e881ac25 -r e73efc9387ee lib/galaxy/datatypes/genetics.py --- a/lib/galaxy/datatypes/genetics.py Wed Oct 07 15:31:18 2009 -0400 +++ b/lib/galaxy/datatypes/genetics.py Wed Oct 07 16:37:48 2009 -0400 @@ -56,10 +56,6 @@ def get_estimated_display_viewport( self, dataset ): """Return a chrom, start, stop tuple for viewing a file.""" raise notImplemented - - def as_ucsc_display_file( self, dataset, **kwd ): - """Returns file""" - return file(dataset.file_name,'r') def ucsc_links( self, dataset, type, app, base_url ): """ from the ever-helpful angie hinrichs angie(a)soe.ucsc.edu diff -r ebe3e881ac25 -r e73efc9387ee lib/galaxy/datatypes/interval.py --- a/lib/galaxy/datatypes/interval.py Wed Oct 07 15:31:18 2009 -0400 +++ b/lib/galaxy/datatypes/interval.py Wed Oct 07 16:37:48 2009 -0400 @@ -493,7 +493,6 @@ """Initialize datatype, by adding GBrowse display app""" Tabular.__init__(self, **kwd) self.add_display_app ( 'c_elegans', 'display in Wormbase', 'as_gbrowse_display_file', 'gbrowse_links' ) - def set_meta( self, dataset, overwrite = True, **kwd ): i = 0 for i, line in enumerate( file ( dataset.file_name ) ): @@ -508,7 +507,6 @@ except: pass Tabular.set_meta( self, dataset, overwrite = overwrite, skip = i ) - def make_html_table( self, dataset, skipchars=[] ): """Create HTML table, used for displaying peek""" out = ['<table cellspacing="0" cellpadding="3">'] @@ -524,11 +522,6 @@ except Exception, exc: out = "Can't create peek %s" % exc return out - - def as_gbrowse_display_file( self, dataset, **kwd ): - """Returns file contents that can be displayed in GBrowse apps.""" - return open( dataset.file_name ) - def get_estimated_display_viewport( self, dataset ): """ Return a chrom, start, stop tuple for viewing a file. There are slight differences between gff 2 and gff 3 @@ -568,7 +561,6 @@ return ( seqid, str( start ), str( stop ) ) else: return ( '', '', '' ) - def gbrowse_links( self, dataset, type, app, base_url ): ret_val = [] if dataset.has_data: @@ -582,7 +574,6 @@ link = "%s?start=%s&stop=%s&ref=%s&dbkey=%s" % ( site_url, start, stop, seqid, dataset.dbkey ) ret_val.append( ( site_name, link ) ) return ret_val - def sniff( self, filename ): """ Determines whether the file is in gff format @@ -639,7 +630,6 @@ def __init__(self, **kwd): """Initialize datatype, by adding GBrowse display app""" Gff.__init__(self, **kwd) - def set_meta( self, dataset, overwrite = True, **kwd ): i = 0 for i, line in enumerate( file ( dataset.file_name ) ): @@ -666,7 +656,6 @@ if valid_start and valid_end and start < end and strand in self.valid_gff3_strand and phase in self.valid_gff3_phase: break Tabular.set_meta( self, dataset, overwrite = overwrite, skip = i ) - def sniff( self, filename ): """ Determines whether the file is in gff version 3 format @@ -740,9 +729,70 @@ MetadataElement( name="columns", default=3, desc="Number of columns", readonly=True, visible=False ) + def __init__( self, **kwd ): + Tabular.__init__( self, **kwd ) + self.add_display_app( 'ucsc', 'display at UCSC', 'as_ucsc_display_file', 'ucsc_links' ) + self.add_display_app( 'gbrowse', 'display in Gbrowse', 'as_gbrowse_display_file', 'gbrowse_links' ) + def get_estimated_display_viewport( self, dataset ): + value = ( "", "", "" ) + num_check_lines = 100 # only check up to this many non empty lines + for i, line in enumerate( file( dataset.file_name ) ): + line = line.rstrip( '\r\n' ) + if line and line.startswith( "browser" ): + chr_info = line.split()[-1] + wig_chr, coords = chr_info.split( ":" ) + start, end = coords.split( "-" ) + value = ( wig_chr, start, end ) + break + if i > num_check_lines: + break + return value + def _get_remote_call_url( self, redirect_url, site_name, dataset, type, app, base_url ): + """Retrieve the URL to call out to an external site and retrieve data. + This routes our external URL through a local galaxy instance which makes + the data available, followed by redirecting to the remote site with a + link back to the available information. + """ + internal_url = "%s" % url_for( controller='dataset', dataset_id=dataset.id, action='display_at', filename='%s_%s' % ( type, site_name ) ) + base_url = app.config.get( "display_at_callback", base_url ) + if base_url.startswith( 'https://' ): + base_url = base_url.replace( 'https', 'http', 1 ) + display_url = urllib.quote_plus( "%s%s/display_as?id=%i&display_app=%s&authz_method=display_at" % \ + ( base_url, url_for( controller='root' ), dataset.id, type ) ) + link = '%s?redirect_url=%s&display_url=%s' % ( internal_url, redirect_url, display_url ) + return link + def _get_viewer_range( self, dataset ): + """Retrieve the chromosome, start, end for an external viewer.""" + if dataset.has_data: + viewport_tuple = self.get_estimated_display_viewport( dataset ) + if viewport_tuple: + chrom = viewport_tuple[0] + start = viewport_tuple[1] + stop = viewport_tuple[2] + return ( chrom, start, stop ) + return ( None, None, None ) + def gbrowse_links( self, dataset, type, app, base_url ): + ret_val = [] + chrom, start, stop = self._get_viewer_range( dataset ) + if chrom is not None: + for site_name, site_url in util.get_gbrowse_sites_by_build( dataset.dbkey ): + if site_name in app.config.gbrowse_display_sites: + redirect_url = urllib.quote_plus( "%s%s/?ref=%s&start=%s&stop=%s&eurl=%%s" % ( site_url, dataset.dbkey, chrom, start, stop ) ) + link = self._get_remote_call_url( redirect_url, site_name, dataset, type, app, base_url ) + ret_val.append( ( site_name, link ) ) + return ret_val + def ucsc_links( self, dataset, type, app, base_url ): + ret_val = [] + chrom, start, stop = self._get_viewer_range( dataset ) + if chrom is not None: + for site_name, site_url in util.get_ucsc_by_build( dataset.dbkey ): + if site_name in app.config.ucsc_display_sites: + redirect_url = urllib.quote_plus( "%sdb=%s&position=%s:%s-%s&hgt.customText=%%s" % ( site_url, dataset.dbkey, chrom, start, stop ) ) + link = self._get_remote_call_url( redirect_url, site_name, dataset, type, app, base_url ) + ret_val.append( ( site_name, link ) ) + return ret_val def make_html_table( self, dataset ): return Tabular.make_html_table( self, dataset, skipchars=['track', '#'] ) - def set_meta( self, dataset, overwrite = True, **kwd ): i = 0 for i, line in enumerate( file ( dataset.file_name ) ): @@ -761,7 +811,6 @@ if do_break: break Tabular.set_meta( self, dataset, overwrite = overwrite, skip = i ) - def sniff( self, filename ): """ Determines wether the file is in wiggle format @@ -792,7 +841,6 @@ return False except: return False - def get_track_window(self, dataset, data, start, end): """ Assumes we have a numpy file. @@ -817,7 +865,6 @@ y = data[ t_start : t_end ] return zip(x.tolist(), y.tolist()) - def get_track_resolution( self, dataset, start, end): range = end - start # Determine appropriate resolution to plot ~1000 points @@ -826,7 +873,6 @@ resolution = min( resolution, 100000 ) resolution = max( resolution, 1 ) return resolution - def get_track_type( self ): return "LineTrack" @@ -882,8 +928,6 @@ except: #return "." return ('', '', '') - def as_ucsc_display_file( self, dataset ): - return open(dataset.file_name) def ucsc_links( self, dataset, type, app, base_url ): ret_val = [] if dataset.has_data: @@ -948,58 +992,6 @@ return False return True -class GBrowseTrack ( Tabular ): - """GMOD GBrowseTrack""" - file_ext = "gbrowsetrack" - - def __init__(self, **kwd): - """Initialize datatype, by adding GBrowse display app""" - Tabular.__init__(self, **kwd) - self.add_display_app ('c_elegans', 'display in Wormbase', 'as_gbrowse_display_file', 'gbrowse_links' ) - - def set_readonly_meta( self, dataset, skip=1, **kwd ): - """Resets the values of readonly metadata elements.""" - Tabular.set_readonly_meta( self, dataset, skip = skip, **kwd ) - - def set_meta( self, dataset, overwrite = True, **kwd ): - Tabular.set_meta( self, dataset, overwrite = overwrite, skip = 1 ) - - def make_html_table( self, dataset ): - return Tabular.make_html_table( self, dataset, skipchars=['track', '#'] ) - - def get_estimated_display_viewport( self, dataset ): - #TODO: fix me... - return ('', '', '') - - def gbrowse_links( self, dataset, type, app, base_url ): - ret_val = [] - if dataset.has_data: - viewport_tuple = self.get_estimated_display_viewport(dataset) - if viewport_tuple: - chrom = viewport_tuple[0] - start = viewport_tuple[1] - stop = viewport_tuple[2] - for site_name, site_url in util.get_gbrowse_sites_by_build(dataset.dbkey): - if site_name in app.config.gbrowse_display_sites: - display_url = urllib.quote_plus( "%s%s/display_as?id=%i&display_app=%s" % (base_url, url_for( controller='root' ), dataset.id, type) ) - link = "%sname=%s&ref=%s:%s..%s&eurl=%s" % (site_url, dataset.dbkey, chrom, start, stop, display_url ) - ret_val.append( (site_name, link) ) - return ret_val - - def as_gbrowse_display_file( self, dataset, **kwd ): - """Returns file contents that can be displayed in GBrowse apps.""" - #TODO: fix me... - return open(dataset.file_name) - - def sniff( self, filename ): - """ - Determines whether the file is in gbrowsetrack format. - - GBrowseTrack files are built within Galaxy. - TODO: Not yet sure what this file will look like. Fix this sniffer and add some unit tests here as soon as we know. - """ - return False - if __name__ == '__main__': import doctest, sys doctest.testmod(sys.modules[__name__]) diff -r ebe3e881ac25 -r e73efc9387ee lib/galaxy/datatypes/tabular.py --- a/lib/galaxy/datatypes/tabular.py Wed Oct 07 15:31:18 2009 -0400 +++ b/lib/galaxy/datatypes/tabular.py Wed Oct 07 16:37:48 2009 -0400 @@ -205,6 +205,10 @@ def display_peek( self, dataset ): """Returns formatted html of peek""" return self.make_html_table( dataset ) + def as_gbrowse_display_file( self, dataset, **kwd ): + return open( dataset.file_name ) + def as_ucsc_display_file( self, dataset, **kwd ): + return open( dataset.file_name ) class Taxonomy( Tabular ): def __init__(self, **kwd):

1 0

[hg] galaxy 2845: added space to paste tool, removed old solexa ...
by Greg Von Kuster 08 Oct '09

08 Oct '09

details: http://www.bx.psu.edu/hg/galaxy/rev/1fa410dd1aa1 changeset: 2845:1fa410dd1aa1 user: Anton Nekrutenko <anton(a)bx.psu.edu> date: Thu Oct 08 09:01:19 2009 -0400 description: added space to paste tool, removed old solexa tool, fixes fastq stat test 4 file(s) affected in this change: tool_conf.xml.sample tools/fastx_toolkit/fastx_quality_statistics.xml tools/filters/pasteWrapper.pl tools/filters/pasteWrapper.xml diffs (69 lines): diff -r e73efc9387ee -r 1fa410dd1aa1 tool_conf.xml.sample --- a/tool_conf.xml.sample Wed Oct 07 16:37:48 2009 -0400 +++ b/tool_conf.xml.sample Thu Oct 08 09:01:19 2009 -0400 @@ -64,7 +64,6 @@ <tool file="filters/axt_to_lav.xml" /> <tool file="filters/bed2gff.xml" /> <tool file="fasta_tools/fasta_to_tabular.xml" /> - <tool file="metag_tools/fastqsolexa_to_fasta_qual.xml" /> <tool file="filters/gff2bed.xml" /> <tool file="filters/lav_to_bed.xml" /> <tool file="maf/maf_to_bed.xml" /> diff -r e73efc9387ee -r 1fa410dd1aa1 tools/fastx_toolkit/fastx_quality_statistics.xml --- a/tools/fastx_toolkit/fastx_quality_statistics.xml Wed Oct 07 16:37:48 2009 -0400 +++ b/tools/fastx_toolkit/fastx_quality_statistics.xml Thu Oct 08 09:01:19 2009 -0400 @@ -9,7 +9,7 @@ <tests> <test> - <param name="input" value="fastq_stats1.fastq" /> + <param name="input" value="fastq_stats1.fastq" ftype="fastq"/> <param name="offset" value="64" /> <output name="output" file="fastq_stats1.out" /> </test> diff -r e73efc9387ee -r 1fa410dd1aa1 tools/filters/pasteWrapper.pl --- a/tools/filters/pasteWrapper.pl Wed Oct 07 16:37:48 2009 -0400 +++ b/tools/filters/pasteWrapper.pl Thu Oct 08 09:01:19 2009 -0400 @@ -4,7 +4,7 @@ use warnings; my $command = ""; # a wrapper for paste for use in galaxy -# lessWrapper.pl [filename1] [filename2] [delimiter] [output] +# pasteWrapper.pl [filename1] [filename2] [delimiter] [output] die "Check arguments" unless @ARGV == 4; @@ -20,18 +20,12 @@ $command = "paste -d \"|\" $ARGV[0] $ARGV[1]"; } elsif ($ARGV[2] eq 'Dt') { $command = "paste -d \".\" $ARGV[0] $ARGV[1]"; +} elsif ($ARGV[2] eq 'Sp') { + $command = "paste -d \" \" $ARGV[0] $ARGV[1]"; } - - open (OUT, ">$ARGV[3]") or die "Cannot create $ARGV[2]:$!\n"; open (PASTE, "$command |") or die "Cannot run paste:$!\n"; - - - - - - while (<PASTE>) { print OUT; diff -r e73efc9387ee -r 1fa410dd1aa1 tools/filters/pasteWrapper.xml --- a/tools/filters/pasteWrapper.xml Wed Oct 07 16:37:48 2009 -0400 +++ b/tools/filters/pasteWrapper.xml Thu Oct 08 09:01:19 2009 -0400 @@ -11,7 +11,8 @@ <option value="C">Comma</option> <option value="D">Dash</option> <option value="U">Underscore</option> - <option value="P">Pipe</option> + <option value="P">Pipe</option> + <option value="Sp">Space</option> </param> </inputs> <outputs>

1 0

[hg] galaxy 2840: Reorganization and rewording of history Option...
by Greg Von Kuster 08 Oct '09

08 Oct '09

details: http://www.bx.psu.edu/hg/galaxy/rev/ecb6d86a5a9c changeset: 2840:ecb6d86a5a9c user: jeremy goecks <jeremy.goecks at emory.edu> date: Wed Oct 07 11:48:30 2009 -0400 description: Reorganization and rewording of history Options menu. 1 file(s) affected in this change: templates/root/index.mako diffs (56 lines): diff -r 3ad620871b25 -r ecb6d86a5a9c templates/root/index.mako --- a/templates/root/index.mako Wed Oct 07 11:18:14 2009 -0400 +++ b/templates/root/index.mako Wed Oct 07 11:48:30 2009 -0400 @@ -6,12 +6,18 @@ $(function(){ $("#history-options-button").css( "position", "relative" ); make_popupmenu( $("#history-options-button"), { - "List your histories": null, - "Stored by you": function() { + "History Lists": null, + "My Saved Histories": function() { galaxy_main.location = "${h.url_for( controller='history', action='list')}"; }, + "My Shared Histories": function() { + galaxy_main.location = "${h.url_for( controller='history', action='list', operation='sharing' )}"; + }, + "Histories Shared with Me": function() { + galaxy_main.location = "${h.url_for( controller='history', action='list_shared')}"; + }, "Current History": null, - "Create new": function() { + "Create New": function() { galaxy_history.location = "${h.url_for( controller='root', action='history_new' )}"; }, "Clone": function() { @@ -20,13 +26,13 @@ "Share": function() { galaxy_main.location = "${h.url_for( controller='history', action='share' )}"; }, - "Extract workflow": function() { + "Extract Workflow": function() { galaxy_main.location = "${h.url_for( controller='workflow', action='build_from_current_history' )}"; }, - "Dataset security": function() { + "Dataset Security": function() { galaxy_main.location = "${h.url_for( controller='root', action='history_set_default_permissions' )}"; }, - "Show deleted datasets": function() { + "Show Deleted Datasets": function() { galaxy_history.location = "${h.url_for( controller='root', action='history', show_deleted=True)}"; }, "Delete": function() @@ -35,13 +41,6 @@ { galaxy_main.location = "${h.url_for( controller='history', action='delete_current' )}"; } - }, - "Manage shared histories": null, - "Shared by you": function() { - galaxy_main.location = "${h.url_for( controller='history', action='list', operation='sharing' )}"; - }, - "Shared with you": function() { - galaxy_main.location = "${h.url_for( controller='history', action='list_shared')}"; } }); });

1 0

[hg] galaxy 2842: Fixed bug in bed_to_interval_index converter.
by Greg Von Kuster 08 Oct '09

08 Oct '09

details: http://www.bx.psu.edu/hg/galaxy/rev/31c577c6fd49 changeset: 2842:31c577c6fd49 user: guru date: Wed Oct 07 15:25:16 2009 -0400 description: Fixed bug in bed_to_interval_index converter. 1 file(s) affected in this change: lib/galaxy/datatypes/converters/bed_to_interval_index_converter.py diffs (14 lines): diff -r 6252781aa157 -r 31c577c6fd49 lib/galaxy/datatypes/converters/bed_to_interval_index_converter.py --- a/lib/galaxy/datatypes/converters/bed_to_interval_index_converter.py Wed Oct 07 12:43:15 2009 -0400 +++ b/lib/galaxy/datatypes/converters/bed_to_interval_index_converter.py Wed Oct 07 15:25:16 2009 -0400 @@ -15,8 +15,8 @@ offset = 0 for line in open(input_fname, "r"): - feature = line.split() - if not feature or feature[0] == "track" or feature[0] == "#": + feature = line.strip().split() + if not feature or feature[0].startswith("track") or feature[0].startswith("#"): offset += len(line) continue chrom = feature[0]

1 0