September 2009 - galaxy-dev - lists.galaxyproject.org

[hg] galaxy 2739: galaxy amqp listener now runs as a daemon
by Nate Coraor 25 Sep '09

25 Sep '09

details: http://www.bx.psu.edu/hg/galaxy/rev/df1f5b739a53 changeset: 2739:df1f5b739a53 user: rc date: Mon Sep 21 14:45:56 2009 -0400 description: galaxy amqp listener now runs as a daemon 1 file(s) affected in this change: run_galaxy_listener.sh diffs (11 lines): diff -r 2b730844747b -r df1f5b739a53 run_galaxy_listener.sh --- a/run_galaxy_listener.sh Mon Sep 21 12:41:39 2009 -0400 +++ b/run_galaxy_listener.sh Mon Sep 21 14:45:56 2009 -0400 @@ -1,3 +1,4 @@ #!/bin/sh -python scripts/galaxy_messaging/amqp_consumer.py \ No newline at end of file +cd `dirname $0` +python scripts/galaxy_messaging/amqp_consumer.py >> galaxy_listener.log 2>&1 \ No newline at end of file

1 0

[hg] galaxy 2740: Tweaks plus reducing size of Assaf's icons
by Nate Coraor 25 Sep '09

25 Sep '09

details: http://www.bx.psu.edu/hg/galaxy/rev/ad4f25a155f0 changeset: 2740:ad4f25a155f0 user: Anton Nekrutenko <anton(a)bx.psu.edu> date: Mon Sep 21 15:40:22 2009 -0400 description: Tweaks plus reducing size of Assaf's icons 13 file(s) affected in this change: static/fastx_icons/barcode_splitter_output_example.png static/fastx_icons/fasta_clipping_histogram_1.png static/fastx_icons/fasta_clipping_histogram_2.png static/fastx_icons/fastq_nucleotides_distribution_1.png static/fastx_icons/fastq_nucleotides_distribution_2.png static/fastx_icons/fastq_nucleotides_distribution_3.png static/fastx_icons/fastq_nucleotides_distribution_4.png static/fastx_icons/fastq_quality_boxplot_1.png static/fastx_icons/fastq_quality_boxplot_2.png static/fastx_icons/fastq_quality_boxplot_3.png static/fastx_icons/fastx_clipper_example.png tool_conf.xml.sample tools/samtools/sam2interval.xml diffs (59 lines): diff -r df1f5b739a53 -r ad4f25a155f0 static/fastx_icons/barcode_splitter_output_example.png Binary file static/fastx_icons/barcode_splitter_output_example.png has changed diff -r df1f5b739a53 -r ad4f25a155f0 static/fastx_icons/fasta_clipping_histogram_1.png Binary file static/fastx_icons/fasta_clipping_histogram_1.png has changed diff -r df1f5b739a53 -r ad4f25a155f0 static/fastx_icons/fasta_clipping_histogram_2.png Binary file static/fastx_icons/fasta_clipping_histogram_2.png has changed diff -r df1f5b739a53 -r ad4f25a155f0 static/fastx_icons/fastq_nucleotides_distribution_1.png Binary file static/fastx_icons/fastq_nucleotides_distribution_1.png has changed diff -r df1f5b739a53 -r ad4f25a155f0 static/fastx_icons/fastq_nucleotides_distribution_2.png Binary file static/fastx_icons/fastq_nucleotides_distribution_2.png has changed diff -r df1f5b739a53 -r ad4f25a155f0 static/fastx_icons/fastq_nucleotides_distribution_3.png Binary file static/fastx_icons/fastq_nucleotides_distribution_3.png has changed diff -r df1f5b739a53 -r ad4f25a155f0 static/fastx_icons/fastq_nucleotides_distribution_4.png Binary file static/fastx_icons/fastq_nucleotides_distribution_4.png has changed diff -r df1f5b739a53 -r ad4f25a155f0 static/fastx_icons/fastq_quality_boxplot_1.png Binary file static/fastx_icons/fastq_quality_boxplot_1.png has changed diff -r df1f5b739a53 -r ad4f25a155f0 static/fastx_icons/fastq_quality_boxplot_2.png Binary file static/fastx_icons/fastq_quality_boxplot_2.png has changed diff -r df1f5b739a53 -r ad4f25a155f0 static/fastx_icons/fastq_quality_boxplot_3.png Binary file static/fastx_icons/fastq_quality_boxplot_3.png has changed diff -r df1f5b739a53 -r ad4f25a155f0 static/fastx_icons/fastx_clipper_example.png Binary file static/fastx_icons/fastx_clipper_example.png has changed diff -r df1f5b739a53 -r ad4f25a155f0 tool_conf.xml.sample --- a/tool_conf.xml.sample Mon Sep 21 14:45:56 2009 -0400 +++ b/tool_conf.xml.sample Mon Sep 21 15:40:22 2009 -0400 @@ -170,7 +170,7 @@ <tool file="fastx_toolkit/fastx_collapser.xml" /> </section> <section name="NGS: QC and manipulation" id="cshl_library_information"> - <label text="Genetic FASTQ data" id="fastq" /> + <label text="Generic FASTQ data" id="fastq" /> <tool file="fastx_toolkit/fastx_quality_statistics.xml" /> <tool file="fastx_toolkit/fastq_quality_boxplot.xml" /> <tool file="fastx_toolkit/fastx_nucleotides_distribution.xml" /> @@ -185,10 +185,10 @@ <tool file="fastx_toolkit/fastq_quality_filter.xml" />  <tool file="metag_tools/split_paired_reads.xml" /> - <label text="Roche-454 Specific" id="454" /> + <label text="Roche-454 data" id="454" /> <tool file="metag_tools/short_reads_figure_score.xml" /> <tool file="metag_tools/short_reads_trim_seq.xml" /> - <label text="AB-SOLiD Specific" id="solid" /> + <label text="AB-SOLiD data" id="solid" /> <tool file="solid_tools/solid_qual_stats.xml" /> <tool file="solid_tools/solid_qual_boxplot.xml" /> </section> diff -r df1f5b739a53 -r ad4f25a155f0 tools/samtools/sam2interval.xml --- a/tools/samtools/sam2interval.xml Mon Sep 21 14:45:56 2009 -0400 +++ b/tools/samtools/sam2interval.xml Mon Sep 21 15:40:22 2009 -0400 @@ -55,7 +55,7 @@ ref 28 33 - r003 16 ref 29 30 6H5M * 0 0 TAGGC * NM:i:0 ref 36 45 - r001 83 ref 37 30 9M = 7 -39 CAGCGCCAT * -Setting *Print all?* is set to **No** will generate the following:: +Setting *Print all?* to **No** will generate the following:: ref 6 22 + r001 ref 8 19 + r002

1 0

[hg] galaxy 2737: Galaxy AMQP messaging now uses amqp & sqlalche...
by Nate Coraor 25 Sep '09

25 Sep '09

details: http://www.bx.psu.edu/hg/galaxy/rev/7a4e6f19aa17 changeset: 2737:7a4e6f19aa17 user: rc date: Mon Sep 21 12:06:33 2009 -0400 description: Galaxy AMQP messaging now uses amqp & sqlalchemy eggs 2 file(s) affected in this change: scripts/galaxy_messaging/amqp_consumer.py scripts/galaxy_messaging/galaxydb_interface.py diffs (58 lines): diff -r f2e5b1725b50 -r 7a4e6f19aa17 scripts/galaxy_messaging/amqp_consumer.py --- a/scripts/galaxy_messaging/amqp_consumer.py Mon Sep 21 11:21:35 2009 -0400 +++ b/scripts/galaxy_messaging/amqp_consumer.py Mon Sep 21 12:06:33 2009 -0400 @@ -9,13 +9,23 @@ this script can be run to receive messages and update the Galaxy database accordingly ''' - -from amqplib import client_0_8 as amqp import ConfigParser -import sys +import sys, os import optparse import xml.dom.minidom from galaxydb_interface import GalaxyDbInterface + +assert sys.version_info[:2] >= ( 2, 4 ) +new_path = [ os.path.join( os.getcwd(), "lib" ) ] +new_path.extend( sys.path[1:] ) # remove scripts/ from the path +sys.path = new_path + +from galaxy import eggs +import pkg_resources +pkg_resources.require( "amqplib" ) + +from amqplib import client_0_8 as amqp + galaxy_config_file = 'universe_wsgi.ini' global dbconnstr diff -r f2e5b1725b50 -r 7a4e6f19aa17 scripts/galaxy_messaging/galaxydb_interface.py --- a/scripts/galaxy_messaging/galaxydb_interface.py Mon Sep 21 11:21:35 2009 -0400 +++ b/scripts/galaxy_messaging/galaxydb_interface.py Mon Sep 21 12:06:33 2009 -0400 @@ -1,13 +1,22 @@ #/usr/bin/python -from sqlalchemy import * -from sqlalchemy.orm import sessionmaker from datetime import datetime, timedelta import sys import optparse import os import time import logging + +assert sys.version_info[:2] >= ( 2, 4 ) +new_path = [ os.path.join( os.getcwd(), "lib" ) ] +new_path.extend( sys.path[1:] ) # remove scripts/ from the path +sys.path = new_path +from galaxy import eggs +import pkg_resources +#pkg_resources.require( "psycopg2 >= 2.0.6" ) +pkg_resources.require( "SQLAlchemy >= 0.4" ) +from sqlalchemy import * +from sqlalchemy.orm import sessionmaker logging.basicConfig(level=logging.DEBUG) log = logging.getLogger( 'GalaxyDbInterface' )

1 0

[hg] galaxy 2736: Add amqplib egg
by Nate Coraor 25 Sep '09

25 Sep '09

details: http://www.bx.psu.edu/hg/galaxy/rev/f2e5b1725b50 changeset: 2736:f2e5b1725b50 user: Nate Coraor <nate(a)bx.psu.edu> date: Mon Sep 21 11:21:35 2009 -0400 description: Add amqplib egg 1 file(s) affected in this change: eggs.ini diffs (19 lines): diff -r 12e6192f6201 -r f2e5b1725b50 eggs.ini --- a/eggs.ini Mon Sep 21 10:49:17 2009 -0400 +++ b/eggs.ini Mon Sep 21 11:21:35 2009 -0400 @@ -25,6 +25,7 @@ guppy = 0.1.8 [eggs:noplatform] +amqplib = 0.6.1 Beaker = 1.4 docutils = 0.4 elementtree = 1.2.6_20050316 @@ -73,6 +74,7 @@ python_lzo = http://www.oberhumer.com/opensource/lzo/download/LZO-v1/python-lzo-1.08.tar… http://www.oberhumer.com/opensource/lzo/download/LZO-v1/lzo-1.08.tar.gz threadframe = http://www.majid.info/python/threadframe/threadframe-0.2.tar.gz guppy = http://pypi.python.org/packages/source/g/guppy/guppy-0.1.8.tar.gz +amqplib = http://py-amqplib.googlecode.com/files/amqplib-0.6.1.tgz Beaker = http://cheeseshop.python.org/packages/source/B/Beaker/Beaker-1.4.tar.gz docutils = http://downloads.sourceforge.net/docutils/docutils-0.4.tar.gz elementtree = http://effbot.org/downloads/elementtree-1.2.6-20050316.tar.gz

1 0

[hg] galaxy 2738: added psycopg2 import stmt in galaxy_interface.py
by Nate Coraor 25 Sep '09

25 Sep '09

details: http://www.bx.psu.edu/hg/galaxy/rev/2b730844747b changeset: 2738:2b730844747b user: rc date: Mon Sep 21 12:41:39 2009 -0400 description: added psycopg2 import stmt in galaxy_interface.py 1 file(s) affected in this change: scripts/galaxy_messaging/galaxydb_interface.py diffs (13 lines): diff -r 7a4e6f19aa17 -r 2b730844747b scripts/galaxy_messaging/galaxydb_interface.py --- a/scripts/galaxy_messaging/galaxydb_interface.py Mon Sep 21 12:06:33 2009 -0400 +++ b/scripts/galaxy_messaging/galaxydb_interface.py Mon Sep 21 12:41:39 2009 -0400 @@ -13,7 +13,8 @@ sys.path = new_path from galaxy import eggs import pkg_resources -#pkg_resources.require( "psycopg2 >= 2.0.6" ) +pkg_resources.require( "psycopg2" ) +import psycopg2 pkg_resources.require( "SQLAlchemy >= 0.4" ) from sqlalchemy import * from sqlalchemy.orm import sessionmaker

1 0

[hg] galaxy 2732: tags made available in history grid
by Nate Coraor 25 Sep '09

25 Sep '09

details: http://www.bx.psu.edu/hg/galaxy/rev/c2e3183756da changeset: 2732:c2e3183756da user: jeremy goecks <jeremy.goecks(a)emory.edu> date: Mon Sep 21 10:39:58 2009 -0400 description: tags made available in history grid 3 file(s) affected in this change: lib/galaxy/web/controllers/history.py static/scripts/autocomplete_tagging.js templates/tagging_common.mako diffs (91 lines): diff -r ebf68a725c09 -r c2e3183756da lib/galaxy/web/controllers/history.py --- a/lib/galaxy/web/controllers/history.py Sun Sep 20 11:48:51 2009 -0400 +++ b/lib/galaxy/web/controllers/history.py Mon Sep 21 10:39:58 2009 -0400 @@ -45,9 +45,10 @@ def get_value( self, trans, grid, history ): self.tag_elt_id_gen += 1 - return trans.fill_template( "/tagging_common.mako", trans=trans, - tagged_item=history, - elt_id="tagging-elt" + str(self.tag_elt_id_gen) ) + elt_id="tagging-elt" + str( self.tag_elt_id_gen ) + div_elt = "<div id=%s></div>" % elt_id + return div_elt + trans.fill_template( "/tagging_common.mako", trans=trans, + tagged_item=history, elt_id = elt_id, in_form="true", input_size="20" ) # Grid definition title = "Stored histories" @@ -59,7 +60,7 @@ link=( lambda item: iff( item.deleted, None, dict( operation="switch", id=item.id ) ) ), attach_popup=True ), DatasetsByStateColumn( "Datasets (by state)", ncells=4 ), - #TagsColumn( "Tags" ), + TagsColumn( "Tags"), StatusColumn( "Status", attach_popup=False ), grids.GridColumn( "Created", key="create_time", format=time_ago ), grids.GridColumn( "Last Updated", key="update_time", format=time_ago ), diff -r ebf68a725c09 -r c2e3183756da static/scripts/autocomplete_tagging.js --- a/static/scripts/autocomplete_tagging.js Sun Sep 20 11:48:51 2009 -0400 +++ b/static/scripts/autocomplete_tagging.js Mon Sep 21 10:39:58 2009 -0400 @@ -23,7 +23,7 @@ text = "Add tags"; return text; }, - tag_click_fn : function (tag) { }, + tag_click_fn : function (name, value) { }, input_size: 20, in_form: false, tags : {}, @@ -240,10 +240,11 @@ // Build tag button. var tag_name_elt = $("<span>").text(tag_str).addClass("tag-name"); tag_name_elt.click( function() - { - settings.tag_click_fn(tag_str); - return true; - }); + { + tag_name_and_value = tag_str.split(":") + settings.tag_click_fn(tag_name_and_value[0], tag_name_and_value[1]); + return true; + }); var tag_button = $("<span></span>").addClass("tag-button"); tag_button.append(tag_name_elt); diff -r ebf68a725c09 -r c2e3183756da templates/tagging_common.mako --- a/templates/tagging_common.mako Sun Sep 20 11:48:51 2009 -0400 +++ b/templates/tagging_common.mako Mon Sep 21 10:39:58 2009 -0400 @@ -1,3 +1,8 @@ +## Render a tagging element if there is a tagged_item. +%if tagged_item is not None and elt_id is not None: + ${render_tagging_element(tagged_item, elt_id=elt_id, in_form=in_form, input_size=input_size)} +%endif + ## Render the tags 'tags' as an autocomplete element. <%def name="render_tagging_element(tagged_item, elt_id, use_toggle_link='true', in_form='false', input_size='15')"> <script type="text/javascript"> @@ -73,11 +78,23 @@ return text; }; + // + // Function to handle a tag click. + // + var tag_click_fn = function(tag_name, tag_value) + { + alert(tag_name); + + // Do URL request to get histories tag. + self.location = "http://www.yahoo.com"; + }; + var options = { tags : ${h.to_json_string(tag_names_and_values)}, get_toggle_link_text_fn: get_toggle_link_text, - tag_click_fn: function(tag) { /* Do nothing. */ }, + tag_click_fn: tag_click_fn, + ##tag_click_fn: function(name, value) { /* Do nothing. */ }, <% tagged_item_id = trans.security.encode_id(tagged_item.id) %> ajax_autocomplete_tag_url: "${h.url_for( controller='tag', action='tag_autocomplete_data', id=tagged_item_id, item_class=tagged_item.__class__.__name__ )}", ajax_add_tag_url: "${h.url_for( controller='tag', action='add_tag_async', id=tagged_item_id, item_class=tagged_item.__class__.__name__ )}",

1 0

[hg] galaxy 2733: merge
by Nate Coraor 25 Sep '09

25 Sep '09

details: http://www.bx.psu.edu/hg/galaxy/rev/2f848dc3e35c changeset: 2733:2f848dc3e35c user: jeremy goecks <jeremy.goecks(a)emory.edu> date: Mon Sep 21 10:40:25 2009 -0400 description: merge 0 file(s) affected in this change: diffs (196 lines): diff -r c2e3183756da -r 2f848dc3e35c tool_conf.xml.sample --- a/tool_conf.xml.sample Mon Sep 21 10:39:58 2009 -0400 +++ b/tool_conf.xml.sample Mon Sep 21 10:40:25 2009 -0400 @@ -169,29 +169,28 @@ <tool file="fastx_toolkit/fasta_nucleotide_changer.xml" /> <tool file="fastx_toolkit/fastx_collapser.xml" /> </section> - <section name="NGS: FASTQ QC and manipulation" id="cshl_library_information"> - <tool file="fastx_toolkit/fastx_quality_statistics.xml" /> - <tool file="fastx_toolkit/fastq_quality_boxplot.xml" /> - <tool file="fastx_toolkit/fastx_nucleotides_distribution.xml" /> -  - <tool file="fastx_toolkit/fastq_to_fasta.xml" /> - <tool file="fastx_toolkit/fastq_quality_converter.xml" /> -  - <tool file="fastx_toolkit/fastx_trimmer.xml" /> - <tool file="fastx_toolkit/fastx_renamer.xml" /> - <tool file="fastx_toolkit/fastx_reverse_complement.xml" /> - <tool file="fastx_toolkit/fastx_artifacts_filter.xml" /> - <tool file="fastx_toolkit/fastq_quality_filter.xml" /> -  - <tool file="metag_tools/split_paired_reads.xml" /> - </section> - <section name="454: QC" id="short_read_analysis"> - <tool file="metag_tools/short_reads_figure_score.xml" /> - <tool file="metag_tools/short_reads_trim_seq.xml" /> - </section> - <section name="SOLiD: QC" id="solid_tools"> - <tool file="solid_tools/solid_qual_stats.xml" /> - <tool file="solid_tools/solid_qual_boxplot.xml" /> + <section name="NGS: QC and manipulation" id="cshl_library_information"> + <label text="Genetic FASTQ data" id="fastq" /> + <tool file="fastx_toolkit/fastx_quality_statistics.xml" /> + <tool file="fastx_toolkit/fastq_quality_boxplot.xml" /> + <tool file="fastx_toolkit/fastx_nucleotides_distribution.xml" /> +  + <tool file="fastx_toolkit/fastq_to_fasta.xml" /> + <tool file="fastx_toolkit/fastq_quality_converter.xml" /> +  + <tool file="fastx_toolkit/fastx_trimmer.xml" /> + <tool file="fastx_toolkit/fastx_renamer.xml" /> + <tool file="fastx_toolkit/fastx_reverse_complement.xml" /> + <tool file="fastx_toolkit/fastx_artifacts_filter.xml" /> + <tool file="fastx_toolkit/fastq_quality_filter.xml" /> +  + <tool file="metag_tools/split_paired_reads.xml" /> + <label text="Roche-454 Specific" id="454" /> + <tool file="metag_tools/short_reads_figure_score.xml" /> + <tool file="metag_tools/short_reads_trim_seq.xml" /> + <label text="AB-SOLiD Specific" id="solid" /> + <tool file="solid_tools/solid_qual_stats.xml" /> + <tool file="solid_tools/solid_qual_boxplot.xml" /> </section> <section name="NGS: Mapping" id="solexa_tools">  diff -r c2e3183756da -r 2f848dc3e35c tools/samtools/pileup_parser.pl --- a/tools/samtools/pileup_parser.pl Mon Sep 21 10:39:58 2009 -0400 +++ b/tools/samtools/pileup_parser.pl Mon Sep 21 10:40:25 2009 -0400 @@ -1,6 +1,8 @@ #! /usr/bin/perl -w use strict; +use POSIX; + die "Usage: pileup_parser.pl <in_file> <ref_base_column> <read_bases_column> <base_quality_column> <coverage column> <qv cutoff> <coverage cutoff> <SNPs only?> <output bed?> <coord_column> <out_file>\n" unless @ARGV == 11; @@ -28,10 +30,12 @@ while (<IN>) { chop; + next if m/^\#/; my @fields = split /\t/; next if $fields[ $ref_base_column ] eq "*"; # skip indel lines - next if $fields[ $cvrg_column ] < $cvrg_cutoff; # skip low coverage lines - my $read_bases = $fields[ $read_bases_column ]; + my $read_bases = $fields[ $read_bases_column ]; + die "Coverage column" . ($cvrg_column+1) . " contains non-numeric values. Check your input parameters as well as format of input dataset." if ( not isdigit $fields[ $cvrg_column ] ); + next if $fields[ $cvrg_column ] < $cvrg_cutoff; my $base_quality = $fields[ $base_quality_column ]; if ($read_bases =~ m/[\$\^\+-]/) { @@ -42,11 +46,8 @@ $read_bases =~ s/[\+-]{1}$indel_len.{$indel_len}//; # remove indel info from read base field } } - if ( length($read_bases) != length($base_quality) ) { - - $first_skipped_line = $_ if $invalid_line_counter == 0; - ++$invalid_line_counter; - } + die "Error parsing read bases and qualities in line $.. Last processed line conatined these values: " . join("\t", @fields) . "\n" if ( length($read_bases) != length($base_quality) ); + # after removing read block and indel data the length of read_base # field should identical to the length of base_quality field @@ -95,4 +96,4 @@ print STDERR "Could not parse $invalid_line_counter line(s) beginning with: $first_skipped_line\n" if $invalid_line_counter > 0; close IN; -close OUT; \ No newline at end of file +close OUT; diff -r c2e3183756da -r 2f848dc3e35c tools/samtools/sam2interval.py --- a/tools/samtools/sam2interval.py Mon Sep 21 10:39:58 2009 -0400 +++ b/tools/samtools/sam2interval.py Mon Sep 21 10:40:25 2009 -0400 @@ -78,7 +78,7 @@ for line in infile: line = line.rstrip( '\r\n' ) - if line and not line.startswith( '#,@' ): + if line and not line.startswith( '#' ) and not line.startswith( '@' ) : fields = line.split( '\t' ) start = int( fields[ int( options.start_col ) - 1 ] ) - 1 end = 0 diff -r c2e3183756da -r 2f848dc3e35c tools/samtools/sam_bitwise_flag_filter.py --- a/tools/samtools/sam_bitwise_flag_filter.py Mon Sep 21 10:39:58 2009 -0400 +++ b/tools/samtools/sam_bitwise_flag_filter.py Mon Sep 21 10:40:25 2009 -0400 @@ -135,7 +135,7 @@ for line in infile: line = line.rstrip( '\r\n' ) - if line and not line.startswith( '#' ): + if line and not line.startswith( '#' ) and not line.startswith( '@' ) : fields = line.split( '\t' ) sam_states = [] sam_states.append( bool( int( fields[ int( options.flag_col ) - 1 ] ) & 0x0001 ) ) diff -r c2e3183756da -r 2f848dc3e35c tools/sr_mapping/bowtie_wrapper.xml --- a/tools/sr_mapping/bowtie_wrapper.xml Mon Sep 21 10:39:58 2009 -0400 +++ b/tools/sr_mapping/bowtie_wrapper.xml Mon Sep 21 10:40:25 2009 -0400 @@ -336,7 +336,7 @@ </param> </when> </conditional> - <param name="offrate" type="integer" value="-1" label="Override the offrate of the index to n -o)" help="-1 for default" /> + <param name="offrate" type="integer" value="-1" label="Override the offrate of the index to n (-o)" help="-1 for default" /> <param name="seed" type="integer" value="-1" label="Seed for pseudo-random number generator (--seed)" help="-1 for default" /> </when>  </conditional>  @@ -443,31 +443,34 @@ The output is in SAM format, and has the following columns:: - 1 QNAME - Query (pair) NAME - 2 FLAG - bitwise FLAG - 3 RNAME - Reference sequence NAME - 4 POS - 1-based leftmost POSition/coordinate of clipped sequence - 5 MAPQ - MAPping Quality (Phred-scaled) - 6 CIGAR - extended CIGAR string - 7 MRNM - Mate Reference sequence NaMe ('=' if same as RNAME) - 8 MPOS - 1-based Mate POSition - 9 ISIZE - Inferred insert SIZE - 10 SEQ - query SEQuence on the same strand as the reference - 11 QUAL - query QUALity (ASCII-33 gives the Phred base quality) - 12 OPT - variable OPTional fields in the format TAG:VTYPE:VALU + Column Description + -------- -------------------------------------------------------- + 1 QNAME Query (pair) NAME + 2 FLAG bitwise FLAG + 3 RNAME Reference sequence NAME + 4 POS 1-based leftmost POSition/coordinate of clipped sequence + 5 MAPQ MAPping Quality (Phred-scaled) + 6 CIGAR extended CIGAR string + 7 MRNM Mate Reference sequence NaMe ('=' if same as RNAME) + 8 MPOS 1-based Mate POSition + 9 ISIZE Inferred insert SIZE + 10 SEQ query SEQuence on the same strand as the reference + 11 QUAL query QUALity (ASCII-33 gives the Phred base quality) + 12 OPT variable OPTional fields in the format TAG:VTYPE:VALUE The flags are as follows:: - Flag - Description - 0x0001 - the read is paired in sequencing - 0x0002 - the read is mapped in a proper pair - 0x0004 - the query sequence itself is unmapped - 0x0008 - the mate is unmapped - 0x0010 - strand of the query (1 for reverse) - 0x0020 - strand of the mate - 0x0040 - the read is the first read in a pair - 0x0080 - the read is the second read in a pair - 0x0100 - the alignment is not primary + Flag Description + ------ ------------------------------------- + 0x0001 the read is paired in sequencing + 0x0002 the read is mapped in a proper pair + 0x0004 the query sequence itself is unmapped + 0x0008 the mate is unmapped + 0x0010 strand of the query (1 for reverse) + 0x0020 strand of the mate + 0x0040 the read is the first read in a pair + 0x0080 the read is the second read in a pair + 0x0100 the alignment is not primary It looks like this (scroll sideways to see the entire example)::

1 0

[hg] galaxy 2734: Quick fix to disable partial functionality in ...
by Nate Coraor 25 Sep '09

25 Sep '09

details: http://www.bx.psu.edu/hg/galaxy/rev/63e37852f658 changeset: 2734:63e37852f658 user: jeremy goecks <jeremy.goecks(a)emory.edu> date: Mon Sep 21 10:46:36 2009 -0400 description: Quick fix to disable partial functionality in tagging 1 file(s) affected in this change: templates/tagging_common.mako diffs (16 lines): diff -r 2f848dc3e35c -r 63e37852f658 templates/tagging_common.mako --- a/templates/tagging_common.mako Mon Sep 21 10:40:25 2009 -0400 +++ b/templates/tagging_common.mako Mon Sep 21 10:46:36 2009 -0400 @@ -83,10 +83,12 @@ // var tag_click_fn = function(tag_name, tag_value) { + /* alert(tag_name); // Do URL request to get histories tag. self.location = "http://www.yahoo.com"; + */ }; var options =

1 0

[hg] galaxy 2735: Fixed a bug in creating requests as an admin
by Nate Coraor 25 Sep '09

25 Sep '09

details: http://www.bx.psu.edu/hg/galaxy/rev/12e6192f6201 changeset: 2735:12e6192f6201 user: rc date: Mon Sep 21 10:49:17 2009 -0400 description: Fixed a bug in creating requests as an admin 2 file(s) affected in this change: lib/galaxy/web/form_builder.py scripts/galaxy_messaging/amqp_consumer.py diffs (62 lines): diff -r 63e37852f658 -r 12e6192f6201 lib/galaxy/web/form_builder.py --- a/lib/galaxy/web/form_builder.py Mon Sep 21 10:46:36 2009 -0400 +++ b/lib/galaxy/web/form_builder.py Mon Sep 21 10:49:17 2009 -0400 @@ -384,8 +384,9 @@ from galaxy import util address_html = '' add_ids = ['none'] - for a in self.user.addresses: - add_ids.append(str(a.id)) + if self.user: + for a in self.user.addresses: + add_ids.append(str(a.id)) add_ids.append('new') self.select_address = SelectField(self.name, refresh_on_change=True, @@ -394,16 +395,17 @@ self.select_address.add_option('Select one', 'none', selected=True) else: self.select_address.add_option('Select one', 'none') - for a in self.user.addresses: - if not a.deleted: - if self.value == str(a.id): - self.select_address.add_option(a.desc, str(a.id), selected=True) - # display this address - address_html = '''<div class="form-row"> - %s - </div>''' % a.get_html() - else: - self.select_address.add_option(a.desc, str(a.id)) + if self.user: + for a in self.user.addresses: + if not a.deleted: + if self.value == str(a.id): + self.select_address.add_option(a.desc, str(a.id), selected=True) + # display this address + address_html = '''<div class="form-row"> + %s + </div>''' % a.get_html() + else: + self.select_address.add_option(a.desc, str(a.id)) if self.value == 'new': self.select_address.add_option('Add a new address', 'new', selected=True) for field_name, label in self.fields(): diff -r 63e37852f658 -r 12e6192f6201 scripts/galaxy_messaging/amqp_consumer.py --- a/scripts/galaxy_messaging/amqp_consumer.py Mon Sep 21 10:46:36 2009 -0400 +++ b/scripts/galaxy_messaging/amqp_consumer.py Mon Sep 21 10:49:17 2009 -0400 @@ -1,3 +1,15 @@ +''' +Galaxy Messaging with AMQP (RabbitMQ) +Galaxy uses AMQ protocol to receive messages from external sources like +bar code scanners. Galaxy has been tested against RabbitMQ AMQP implementation. +For Galaxy to receive messages from a message queue the RabbitMQ server has +to be set up with a user account and other parameters listed in the [galaxy:amq] +section in the universe_wsgi.ini config file +Once the RabbitMQ server has been setup and started with the given parameters, +this script can be run to receive messages and update the Galaxy database accordingly +''' + + from amqplib import client_0_8 as amqp import ConfigParser import sys

1 0

[hg] galaxy 2731: More SR changes
by Nate Coraor 25 Sep '09

25 Sep '09

details: http://www.bx.psu.edu/hg/galaxy/rev/8255ed4330e9 changeset: 2731:8255ed4330e9 user: Anton Nekrutenko <anton(a)bx.psu.edu> date: Mon Sep 21 10:32:48 2009 -0400 description: More SR changes 5 file(s) affected in this change: tool_conf.xml.sample tools/samtools/pileup_parser.pl tools/samtools/sam2interval.py tools/samtools/sam_bitwise_flag_filter.py tools/sr_mapping/bowtie_wrapper.xml diffs (196 lines): diff -r ebf68a725c09 -r 8255ed4330e9 tool_conf.xml.sample --- a/tool_conf.xml.sample Sun Sep 20 11:48:51 2009 -0400 +++ b/tool_conf.xml.sample Mon Sep 21 10:32:48 2009 -0400 @@ -169,29 +169,28 @@ <tool file="fastx_toolkit/fasta_nucleotide_changer.xml" /> <tool file="fastx_toolkit/fastx_collapser.xml" /> </section> - <section name="NGS: FASTQ QC and manipulation" id="cshl_library_information"> - <tool file="fastx_toolkit/fastx_quality_statistics.xml" /> - <tool file="fastx_toolkit/fastq_quality_boxplot.xml" /> - <tool file="fastx_toolkit/fastx_nucleotides_distribution.xml" /> -  - <tool file="fastx_toolkit/fastq_to_fasta.xml" /> - <tool file="fastx_toolkit/fastq_quality_converter.xml" /> -  - <tool file="fastx_toolkit/fastx_trimmer.xml" /> - <tool file="fastx_toolkit/fastx_renamer.xml" /> - <tool file="fastx_toolkit/fastx_reverse_complement.xml" /> - <tool file="fastx_toolkit/fastx_artifacts_filter.xml" /> - <tool file="fastx_toolkit/fastq_quality_filter.xml" /> -  - <tool file="metag_tools/split_paired_reads.xml" /> - </section> - <section name="454: QC" id="short_read_analysis"> - <tool file="metag_tools/short_reads_figure_score.xml" /> - <tool file="metag_tools/short_reads_trim_seq.xml" /> - </section> - <section name="SOLiD: QC" id="solid_tools"> - <tool file="solid_tools/solid_qual_stats.xml" /> - <tool file="solid_tools/solid_qual_boxplot.xml" /> + <section name="NGS: QC and manipulation" id="cshl_library_information"> + <label text="Genetic FASTQ data" id="fastq" /> + <tool file="fastx_toolkit/fastx_quality_statistics.xml" /> + <tool file="fastx_toolkit/fastq_quality_boxplot.xml" /> + <tool file="fastx_toolkit/fastx_nucleotides_distribution.xml" /> +  + <tool file="fastx_toolkit/fastq_to_fasta.xml" /> + <tool file="fastx_toolkit/fastq_quality_converter.xml" /> +  + <tool file="fastx_toolkit/fastx_trimmer.xml" /> + <tool file="fastx_toolkit/fastx_renamer.xml" /> + <tool file="fastx_toolkit/fastx_reverse_complement.xml" /> + <tool file="fastx_toolkit/fastx_artifacts_filter.xml" /> + <tool file="fastx_toolkit/fastq_quality_filter.xml" /> +  + <tool file="metag_tools/split_paired_reads.xml" /> + <label text="Roche-454 Specific" id="454" /> + <tool file="metag_tools/short_reads_figure_score.xml" /> + <tool file="metag_tools/short_reads_trim_seq.xml" /> + <label text="AB-SOLiD Specific" id="solid" /> + <tool file="solid_tools/solid_qual_stats.xml" /> + <tool file="solid_tools/solid_qual_boxplot.xml" /> </section> <section name="NGS: Mapping" id="solexa_tools">  diff -r ebf68a725c09 -r 8255ed4330e9 tools/samtools/pileup_parser.pl --- a/tools/samtools/pileup_parser.pl Sun Sep 20 11:48:51 2009 -0400 +++ b/tools/samtools/pileup_parser.pl Mon Sep 21 10:32:48 2009 -0400 @@ -1,6 +1,8 @@ #! /usr/bin/perl -w use strict; +use POSIX; + die "Usage: pileup_parser.pl <in_file> <ref_base_column> <read_bases_column> <base_quality_column> <coverage column> <qv cutoff> <coverage cutoff> <SNPs only?> <output bed?> <coord_column> <out_file>\n" unless @ARGV == 11; @@ -28,10 +30,12 @@ while (<IN>) { chop; + next if m/^\#/; my @fields = split /\t/; next if $fields[ $ref_base_column ] eq "*"; # skip indel lines - next if $fields[ $cvrg_column ] < $cvrg_cutoff; # skip low coverage lines - my $read_bases = $fields[ $read_bases_column ]; + my $read_bases = $fields[ $read_bases_column ]; + die "Coverage column" . ($cvrg_column+1) . " contains non-numeric values. Check your input parameters as well as format of input dataset." if ( not isdigit $fields[ $cvrg_column ] ); + next if $fields[ $cvrg_column ] < $cvrg_cutoff; my $base_quality = $fields[ $base_quality_column ]; if ($read_bases =~ m/[\$\^\+-]/) { @@ -42,11 +46,8 @@ $read_bases =~ s/[\+-]{1}$indel_len.{$indel_len}//; # remove indel info from read base field } } - if ( length($read_bases) != length($base_quality) ) { - - $first_skipped_line = $_ if $invalid_line_counter == 0; - ++$invalid_line_counter; - } + die "Error parsing read bases and qualities in line $.. Last processed line conatined these values: " . join("\t", @fields) . "\n" if ( length($read_bases) != length($base_quality) ); + # after removing read block and indel data the length of read_base # field should identical to the length of base_quality field @@ -95,4 +96,4 @@ print STDERR "Could not parse $invalid_line_counter line(s) beginning with: $first_skipped_line\n" if $invalid_line_counter > 0; close IN; -close OUT; \ No newline at end of file +close OUT; diff -r ebf68a725c09 -r 8255ed4330e9 tools/samtools/sam2interval.py --- a/tools/samtools/sam2interval.py Sun Sep 20 11:48:51 2009 -0400 +++ b/tools/samtools/sam2interval.py Mon Sep 21 10:32:48 2009 -0400 @@ -78,7 +78,7 @@ for line in infile: line = line.rstrip( '\r\n' ) - if line and not line.startswith( '#,@' ): + if line and not line.startswith( '#' ) and not line.startswith( '@' ) : fields = line.split( '\t' ) start = int( fields[ int( options.start_col ) - 1 ] ) - 1 end = 0 diff -r ebf68a725c09 -r 8255ed4330e9 tools/samtools/sam_bitwise_flag_filter.py --- a/tools/samtools/sam_bitwise_flag_filter.py Sun Sep 20 11:48:51 2009 -0400 +++ b/tools/samtools/sam_bitwise_flag_filter.py Mon Sep 21 10:32:48 2009 -0400 @@ -135,7 +135,7 @@ for line in infile: line = line.rstrip( '\r\n' ) - if line and not line.startswith( '#' ): + if line and not line.startswith( '#' ) and not line.startswith( '@' ) : fields = line.split( '\t' ) sam_states = [] sam_states.append( bool( int( fields[ int( options.flag_col ) - 1 ] ) & 0x0001 ) ) diff -r ebf68a725c09 -r 8255ed4330e9 tools/sr_mapping/bowtie_wrapper.xml --- a/tools/sr_mapping/bowtie_wrapper.xml Sun Sep 20 11:48:51 2009 -0400 +++ b/tools/sr_mapping/bowtie_wrapper.xml Mon Sep 21 10:32:48 2009 -0400 @@ -336,7 +336,7 @@ </param> </when> </conditional> - <param name="offrate" type="integer" value="-1" label="Override the offrate of the index to n -o)" help="-1 for default" /> + <param name="offrate" type="integer" value="-1" label="Override the offrate of the index to n (-o)" help="-1 for default" /> <param name="seed" type="integer" value="-1" label="Seed for pseudo-random number generator (--seed)" help="-1 for default" /> </when>  </conditional>  @@ -443,31 +443,34 @@ The output is in SAM format, and has the following columns:: - 1 QNAME - Query (pair) NAME - 2 FLAG - bitwise FLAG - 3 RNAME - Reference sequence NAME - 4 POS - 1-based leftmost POSition/coordinate of clipped sequence - 5 MAPQ - MAPping Quality (Phred-scaled) - 6 CIGAR - extended CIGAR string - 7 MRNM - Mate Reference sequence NaMe ('=' if same as RNAME) - 8 MPOS - 1-based Mate POSition - 9 ISIZE - Inferred insert SIZE - 10 SEQ - query SEQuence on the same strand as the reference - 11 QUAL - query QUALity (ASCII-33 gives the Phred base quality) - 12 OPT - variable OPTional fields in the format TAG:VTYPE:VALU + Column Description + -------- -------------------------------------------------------- + 1 QNAME Query (pair) NAME + 2 FLAG bitwise FLAG + 3 RNAME Reference sequence NAME + 4 POS 1-based leftmost POSition/coordinate of clipped sequence + 5 MAPQ MAPping Quality (Phred-scaled) + 6 CIGAR extended CIGAR string + 7 MRNM Mate Reference sequence NaMe ('=' if same as RNAME) + 8 MPOS 1-based Mate POSition + 9 ISIZE Inferred insert SIZE + 10 SEQ query SEQuence on the same strand as the reference + 11 QUAL query QUALity (ASCII-33 gives the Phred base quality) + 12 OPT variable OPTional fields in the format TAG:VTYPE:VALUE The flags are as follows:: - Flag - Description - 0x0001 - the read is paired in sequencing - 0x0002 - the read is mapped in a proper pair - 0x0004 - the query sequence itself is unmapped - 0x0008 - the mate is unmapped - 0x0010 - strand of the query (1 for reverse) - 0x0020 - strand of the mate - 0x0040 - the read is the first read in a pair - 0x0080 - the read is the second read in a pair - 0x0100 - the alignment is not primary + Flag Description + ------ ------------------------------------- + 0x0001 the read is paired in sequencing + 0x0002 the read is mapped in a proper pair + 0x0004 the query sequence itself is unmapped + 0x0008 the mate is unmapped + 0x0010 strand of the query (1 for reverse) + 0x0020 strand of the mate + 0x0040 the read is the first read in a pair + 0x0080 the read is the second read in a pair + 0x0100 the alignment is not primary It looks like this (scroll sideways to see the entire example)::

1 0