galaxy-commits February 2012

galaxy-commits@lists.galaxyproject.org

2 participants
113 discussions

commit/galaxy-central: dan: Remove bam index metadata validator from GATK tools. The GATK wrapper will now build missing bam indexes on the fly as needed.
by Bitbucket 27 Feb '12

27 Feb '12

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/changeset/167fddae9646/ changeset: 167fddae9646 user: dan date: 2012-02-27 18:49:30 summary: Remove bam index metadata validator from GATK tools. The GATK wrapper will now build missing bam indexes on the fly as needed. affected #: 10 files diff -r 17caba3f5b808bd62a96ab3076934779dd01fb77 -r 167fddae9646f761a87c99a15984f52a605cc498 tools/gatk/analyze_covariates.xml --- a/tools/gatk/analyze_covariates.xml +++ b/tools/gatk/analyze_covariates.xml @@ -74,7 +74,7 @@ **Outputs** -The output is in and HTML file with links to PDF graphs and a data files. +The output is in CSV and HTML files with links to PDF graphs and a data files. Go `here <http://www.broadinstitute.org/gsa/wiki/index.php/Input_files_for_the_GATK>`_ for details on GATK file formats. diff -r 17caba3f5b808bd62a96ab3076934779dd01fb77 -r 167fddae9646f761a87c99a15984f52a605cc498 tools/gatk/count_covariates.xml --- a/tools/gatk/count_covariates.xml +++ b/tools/gatk/count_covariates.xml @@ -2,12 +2,15 @@ <description>on BAM files</description><requirements><requirement type="package" version="1.4">gatk</requirement> + <requirement type="package">samtools</requirement></requirements><command interpreter="python">gatk_wrapper.py --max_jvm_heap_fraction "1" --stdout "${output_log}" -d "-I" "${reference_source.input_bam}" "${reference_source.input_bam.ext}" "gatk_input" - -d "" "${reference_source.input_bam.metadata.bam_index}" "bam_index" "gatk_input" ##hardcode galaxy ext type as bam_index + #if str( $reference_source.input_bam.metadata.bam_index ) != "None": + -d "" "${reference_source.input_bam.metadata.bam_index}" "bam_index" "gatk_input" ##hardcode galaxy ext type as bam_index + #end if -p 'java -jar "${GALAXY_DATA_INDEX_DIR}/shared/jars/gatk/GenomeAnalysisTK.jar" -T "CountCovariates" @@ -143,7 +146,6 @@ <when value="cached"><param name="input_bam" type="data" format="bam" label="BAM file"><validator type="unspecified_build" /> - <validator type="metadata" check="bam_index" message="Metadata missing, click the pencil icon in the history item and use the auto-detect feature to correct this issue."/><validator type="dataset_metadata_in_data_table" table_name="gatk_picard_indexes" metadata_name="dbkey" metadata_column="dbkey" message="Sequences are not currently available for the specified build." /></param><param name="ref_file" type="select" label="Using reference genome"> @@ -155,7 +157,6 @@ </when><when value="history"><param name="input_bam" type="data" format="bam" label="BAM file" > - <validator type="metadata" check="bam_index" message="Metadata missing, click the pencil icon in the history item and use the auto-detect feature to correct this issue."/></param><param name="ref_file" type="data" format="fasta" label="Using reference file" /></when> diff -r 17caba3f5b808bd62a96ab3076934779dd01fb77 -r 167fddae9646f761a87c99a15984f52a605cc498 tools/gatk/depth_of_coverage.xml --- a/tools/gatk/depth_of_coverage.xml +++ b/tools/gatk/depth_of_coverage.xml @@ -2,13 +2,16 @@ <description>on BAM files</description><requirements><requirement type="package" version="1.4">gatk</requirement> + <requirement type="package">samtools</requirement></requirements><command interpreter="python">gatk_wrapper.py --max_jvm_heap_fraction "1" --stdout "${output_log}" #for $i, $input_bam in enumerate( $reference_source.input_bams ): -d "-I" "${input_bam.input_bam}" "${input_bam.input_bam.ext}" "gatk_input_${i}" - -d "" "${input_bam.input_bam.metadata.bam_index}" "bam_index" "gatk_input_${i}" ##hardcode galaxy ext type as bam_index + #if str( $input_bam.input_bam.metadata.bam_index ) != "None": + -d "" "${input_bam.input_bam.metadata.bam_index}" "bam_index" "gatk_input_${i}" ##hardcode galaxy ext type as bam_index + #end if #end for -p 'java -jar "${GALAXY_DATA_INDEX_DIR}/shared/jars/gatk/GenomeAnalysisTK.jar" @@ -190,7 +193,6 @@ <repeat name="input_bams" title="BAM file" min="1"><param name="input_bam" type="data" format="bam" label="BAM file"><validator type="unspecified_build" /> - <validator type="metadata" check="bam_index" message="Metadata missing, click the pencil icon in the history item and use the auto-detect feature to correct this issue."/><validator type="dataset_metadata_in_data_table" table_name="gatk_picard_indexes" metadata_name="dbkey" metadata_column="dbkey" message="Sequences are not currently available for the specified build." /></param></repeat> @@ -204,7 +206,6 @@ <when value="history"><repeat name="input_bams" title="BAM file" min="1"><param name="input_bam" type="data" format="bam" label="BAM file" > - <validator type="metadata" check="bam_index" message="Metadata missing, click the pencil icon in the history item and use the auto-detect feature to correct this issue."/></param></repeat><param name="ref_file" type="data" format="fasta" label="Using reference file" /> diff -r 17caba3f5b808bd62a96ab3076934779dd01fb77 -r 167fddae9646f761a87c99a15984f52a605cc498 tools/gatk/gatk_wrapper.py --- a/tools/gatk/gatk_wrapper.py +++ b/tools/gatk/gatk_wrapper.py @@ -43,6 +43,23 @@ html_out.write( '<li><a href="%s">%s</a></li>\n' % ( fname, fname ) ) html_out.write( '</ul>\n</body>\n</html>\n' ) +def index_bam_files( bam_filenames, tmp_dir ): + for bam_filename in bam_filenames: + bam_index_filename = "%s.bai" % bam_filename + if not os.path.exists( bam_index_filename ): + #need to index this bam file + stderr_name = tempfile.NamedTemporaryFile( prefix = "bam_index_stderr" ).name + command = 'samtools index %s %s' % ( bam_filename, bam_index_filename ) + proc = subprocess.Popen( args=command, shell=True, stderr=open( stderr_name, 'wb' ) ) + return_code = proc.wait() + if return_code: + for line in open( stderr_name ): + print >> sys.stderr, line + os.unlink( stderr_name ) #clean up + cleanup_before_exit( tmp_dir ) + raise Exception( "Error indexing BAM file" ) + os.unlink( stderr_name ) #clean up + def __main__(): #Parse Command Line parser = optparse.OptionParser() @@ -67,11 +84,15 @@ cmd = cmd.replace( 'java ', 'java -Xmx%s ' % ( options.max_jvm_heap ), 1 ) elif options.max_jvm_heap_fraction is not None: cmd = cmd.replace( 'java ', 'java -XX:DefaultMaxRAMFraction=%s -XX:+UseParallelGC ' % ( options.max_jvm_heap_fraction ), 1 ) + bam_filenames = [] if options.datasets: for ( dataset_arg, filename, galaxy_ext, prefix ) in options.datasets: gatk_filename = gatk_filename_from_galaxy( filename, galaxy_ext, target_dir = tmp_dir, prefix = prefix ) if dataset_arg: cmd = '%s %s "%s"' % ( cmd, gatk_filetype_argument_substitution( dataset_arg, galaxy_ext ), gatk_filename ) + if galaxy_ext == "bam": + bam_filenames.append( gatk_filename ) + index_bam_files( bam_filenames, tmp_dir ) #set up stdout and stderr output options stdout = open_file_from_option( options.stdout, mode = 'wb' ) stderr = open_file_from_option( options.stderr, mode = 'wb' ) diff -r 17caba3f5b808bd62a96ab3076934779dd01fb77 -r 167fddae9646f761a87c99a15984f52a605cc498 tools/gatk/indel_realigner.xml --- a/tools/gatk/indel_realigner.xml +++ b/tools/gatk/indel_realigner.xml @@ -2,12 +2,15 @@ <description>- perform local realignment</description><requirements><requirement type="package" version="1.4">gatk</requirement> + <requirement type="package">samtools</requirement></requirements><command interpreter="python">gatk_wrapper.py --max_jvm_heap_fraction "1" --stdout "${output_log}" -d "-I" "${reference_source.input_bam}" "${reference_source.input_bam.ext}" "gatk_input" - -d "" "${reference_source.input_bam.metadata.bam_index}" "bam_index" "gatk_input" ##hardcode galaxy ext type as bam_index + #if str( $reference_source.input_bam.metadata.bam_index ) != "None": + -d "" "${reference_source.input_bam.metadata.bam_index}" "bam_index" "gatk_input" ##hardcode galaxy ext type as bam_index + #end if -p 'java -jar "${GALAXY_DATA_INDEX_DIR}/shared/jars/gatk/GenomeAnalysisTK.jar" -T "IndelRealigner" @@ -121,7 +124,6 @@ <when value="cached"><param name="input_bam" type="data" format="bam" label="BAM file"><validator type="unspecified_build" /> - <validator type="metadata" check="bam_index" message="Metadata missing, click the pencil icon in the history item and use the auto-detect feature to correct this issue."/><validator type="dataset_metadata_in_data_table" table_name="gatk_picard_indexes" metadata_name="dbkey" metadata_column="dbkey" message="Sequences are not currently available for the specified build." /></param><param name="ref_file" type="select" label="Using reference genome"> @@ -133,7 +135,6 @@ </when><when value="history"><param name="input_bam" type="data" format="bam" label="BAM file" > - <validator type="metadata" check="bam_index" message="Metadata missing, click the pencil icon in the history item and use the auto-detect feature to correct this issue."/></param><param name="ref_file" type="data" format="fasta" label="Using reference file"><options> diff -r 17caba3f5b808bd62a96ab3076934779dd01fb77 -r 167fddae9646f761a87c99a15984f52a605cc498 tools/gatk/print_reads.xml --- a/tools/gatk/print_reads.xml +++ b/tools/gatk/print_reads.xml @@ -2,13 +2,16 @@ <description>from BAM files</description><requirements><requirement type="package" version="1.4">gatk</requirement> + <requirement type="package">samtools</requirement></requirements><command interpreter="python">gatk_wrapper.py --max_jvm_heap_fraction "1" --stdout "${output_log}" #for $i, $input_bam in enumerate( $reference_source.input_bams ): -d "-I" "${input_bam.input_bam}" "${input_bam.input_bam.ext}" "gatk_input_${i}" - -d "" "${input_bam.input_bam.metadata.bam_index}" "bam_index" "gatk_input_${i}" ##hardcode galaxy ext type as bam_index + #if str( $input_bam.input_bam.metadata.bam_index ) != "None": + -d "" "${input_bam.input_bam.metadata.bam_index}" "bam_index" "gatk_input_${i}" ##hardcode galaxy ext type as bam_index + #end if #end for -p 'java -jar "${GALAXY_DATA_INDEX_DIR}/shared/jars/gatk/GenomeAnalysisTK.jar" @@ -106,7 +109,6 @@ <repeat name="input_bams" title="Sample BAM file" min="1"><param name="input_bam" type="data" format="bam" label="BAM file"><validator type="unspecified_build" /> - <validator type="metadata" check="bam_index" message="Metadata missing, click the pencil icon in the history item and use the auto-detect feature to correct this issue."/><validator type="dataset_metadata_in_data_table" table_name="gatk_picard_indexes" metadata_name="dbkey" metadata_column="dbkey" message="Sequences are not currently available for the specified build." /></param></repeat> @@ -120,7 +122,6 @@ <when value="history"><repeat name="input_bams" title="Sample BAM file" min="1"><param name="input_bam" type="data" format="bam" label="BAM file" > - <validator type="metadata" check="bam_index" message="Metadata missing, click the pencil icon in the history item and use the auto-detect feature to correct this issue."/></param></repeat><param name="ref_file" type="data" format="fasta" label="Using reference file" /> diff -r 17caba3f5b808bd62a96ab3076934779dd01fb77 -r 167fddae9646f761a87c99a15984f52a605cc498 tools/gatk/realigner_target_creator.xml --- a/tools/gatk/realigner_target_creator.xml +++ b/tools/gatk/realigner_target_creator.xml @@ -2,12 +2,15 @@ <description>for use in local realignment</description><requirements><requirement type="package" version="1.3">gatk</requirement> + <requirement type="package">samtools</requirement></requirements><command interpreter="python">gatk_wrapper.py --max_jvm_heap_fraction "1" --stdout "${output_log}" -d "-I" "${reference_source.input_bam}" "${reference_source.input_bam.ext}" "gatk_input" - -d "" "${reference_source.input_bam.metadata.bam_index}" "bam_index" "gatk_input" ##hardcode galaxy ext type as bam_index + #if str( $reference_source.input_bam.metadata.bam_index ) != "None": + -d "" "${reference_source.input_bam.metadata.bam_index}" "bam_index" "gatk_input" ##hardcode galaxy ext type as bam_index + #end if -p 'java -jar "${GALAXY_DATA_INDEX_DIR}/shared/jars/gatk/GenomeAnalysisTK.jar" -T "RealignerTargetCreator" @@ -109,7 +112,6 @@ <when value="cached"><param name="input_bam" type="data" format="bam" label="BAM file"><validator type="unspecified_build" /> - <validator type="metadata" check="bam_index" message="Metadata missing, click the pencil icon in the history item and use the auto-detect feature to correct this issue."/><validator type="dataset_metadata_in_data_table" table_name="gatk_picard_indexes" metadata_name="dbkey" metadata_column="dbkey" message="Sequences are not currently available for the specified build." /></param><param name="ref_file" type="select" label="Using reference genome"> @@ -121,7 +123,6 @@ </when><when value="history"><param name="input_bam" type="data" format="bam" label="BAM file" > - <validator type="metadata" check="bam_index" message="Metadata missing, click the pencil icon in the history item and use the auto-detect feature to correct this issue."/></param><param name="ref_file" type="data" format="fasta" label="Using reference file"><options> diff -r 17caba3f5b808bd62a96ab3076934779dd01fb77 -r 167fddae9646f761a87c99a15984f52a605cc498 tools/gatk/table_recalibration.xml --- a/tools/gatk/table_recalibration.xml +++ b/tools/gatk/table_recalibration.xml @@ -2,12 +2,15 @@ <description>on BAM files</description><requirements><requirement type="package" version="1.4">gatk</requirement> + <requirement type="package">samtools</requirement></requirements><command interpreter="python">gatk_wrapper.py --max_jvm_heap_fraction "1" --stdout "${output_log}" -d "-I" "${reference_source.input_bam}" "${reference_source.input_bam.ext}" "gatk_input" - -d "" "${reference_source.input_bam.metadata.bam_index}" "bam_index" "gatk_input" ##hardcode galaxy ext type as bam_index + #if str( $reference_source.input_bam.metadata.bam_index ) != "None": + -d "" "${reference_source.input_bam.metadata.bam_index}" "bam_index" "gatk_input" ##hardcode galaxy ext type as bam_index + #end if -p 'java -jar "${GALAXY_DATA_INDEX_DIR}/shared/jars/gatk/GenomeAnalysisTK.jar" -T "TableRecalibration" @@ -126,7 +129,6 @@ <when value="cached"><param name="input_bam" type="data" format="bam" label="BAM file"><validator type="unspecified_build" /> - <validator type="metadata" check="bam_index" message="Metadata missing, click the pencil icon in the history item and use the auto-detect feature to correct this issue."/><validator type="dataset_metadata_in_data_table" table_name="gatk_picard_indexes" metadata_name="dbkey" metadata_column="dbkey" message="Sequences are not currently available for the specified build." /></param><param name="ref_file" type="select" label="Using reference genome"> @@ -138,7 +140,6 @@ </when><when value="history"><param name="input_bam" type="data" format="bam" label="BAM file" > - <validator type="metadata" check="bam_index" message="Metadata missing, click the pencil icon in the history item and use the auto-detect feature to correct this issue."/></param><param name="ref_file" type="data" format="fasta" label="Using reference file" /></when> diff -r 17caba3f5b808bd62a96ab3076934779dd01fb77 -r 167fddae9646f761a87c99a15984f52a605cc498 tools/gatk/unified_genotyper.xml --- a/tools/gatk/unified_genotyper.xml +++ b/tools/gatk/unified_genotyper.xml @@ -2,13 +2,16 @@ <description>SNP and indel caller</description><requirements><requirement type="package" version="1.4">gatk</requirement> + <requirement type="package">samtools</requirement></requirements><command interpreter="python">gatk_wrapper.py --max_jvm_heap_fraction "1" --stdout "${output_log}" #for $i, $input_bam in enumerate( $reference_source.input_bams ): -d "-I" "${input_bam.input_bam}" "${input_bam.input_bam.ext}" "gatk_input_${i}" - -d "" "${input_bam.input_bam.metadata.bam_index}" "bam_index" "gatk_input_${i}" ##hardcode galaxy ext type as bam_index + #if str( $input_bam.input_bam.metadata.bam_index ) != "None": + -d "" "${input_bam.input_bam.metadata.bam_index}" "bam_index" "gatk_input_${i}" ##hardcode galaxy ext type as bam_index + #end if #end for -p 'java -jar "${GALAXY_DATA_INDEX_DIR}/shared/jars/gatk/GenomeAnalysisTK.jar" @@ -155,7 +158,6 @@ <repeat name="input_bams" title="Sample BAM file" min="1"><param name="input_bam" type="data" format="bam" label="BAM file"><validator type="unspecified_build" /> - <validator type="metadata" check="bam_index" message="Metadata missing, click the pencil icon in the history item and use the auto-detect feature to correct this issue."/><validator type="dataset_metadata_in_data_table" table_name="gatk_picard_indexes" metadata_name="dbkey" metadata_column="dbkey" message="Sequences are not currently available for the specified build." /></param></repeat> @@ -169,7 +171,6 @@ <when value="history"><repeat name="input_bams" title="Sample BAM file" min="1"><param name="input_bam" type="data" format="bam" label="BAM file" > - <validator type="metadata" check="bam_index" message="Metadata missing, click the pencil icon in the history item and use the auto-detect feature to correct this issue."/></param></repeat><param name="ref_file" type="data" format="fasta" label="Using reference file" /> diff -r 17caba3f5b808bd62a96ab3076934779dd01fb77 -r 167fddae9646f761a87c99a15984f52a605cc498 tools/gatk/variant_annotator.xml --- a/tools/gatk/variant_annotator.xml +++ b/tools/gatk/variant_annotator.xml @@ -2,13 +2,16 @@ <description></description><requirements><requirement type="package" version="1.4">gatk</requirement> + <requirement type="package">samtools</requirement></requirements><command interpreter="python">gatk_wrapper.py --max_jvm_heap_fraction "1" --stdout "${output_log}" #if str( $reference_source.input_bam ) != "None": -d "-I" "${reference_source.input_bam}" "${reference_source.input_bam.ext}" "gatk_input" - -d "" "${reference_source.input_bam.metadata.bam_index}" "bam_index" "gatk_input" ##hardcode galaxy ext type as bam_index + #if str( $reference_source.input_bam.metadata.bam_index ) != "None": + -d "" "${reference_source.input_bam.metadata.bam_index}" "bam_index" "gatk_input" ##hardcode galaxy ext type as bam_index + #end if #end if -d "--variant" "${reference_source.input_variant}" "${reference_source.input_variant.ext}" "input_variant" -p 'java @@ -151,7 +154,6 @@ <param name="input_variant_bti" type="boolean" truevalue="-BTI variant" falsevalue="" label="Increase efficiency for small variant files." /><param name="input_bam" type="data" format="bam" label="BAM file" optional="True" help="Not needed for all annotations." ><validator type="unspecified_build" /> - <validator type="metadata" check="bam_index" message="Metadata missing, click the pencil icon in the history item and use the auto-detect feature to correct this issue."/><validator type="dataset_metadata_in_data_table" table_name="gatk_picard_indexes" metadata_name="dbkey" metadata_column="dbkey" message="Sequences are not currently available for the specified build." /></param><param name="ref_file" type="select" label="Using reference genome"> @@ -165,7 +167,6 @@ <param name="input_variant" type="data" format="vcf" label="Variant file to annotate" /><param name="input_variant_bti" type="boolean" truevalue="-BTI variant" falsevalue="" label="Increase efficiency for small variant files." /><param name="input_bam" type="data" format="bam" label="BAM file" optional="True" > - <validator type="metadata" check="bam_index" message="Metadata missing, click the pencil icon in the history item and use the auto-detect feature to correct this issue."/></param><param name="ref_file" type="data" format="fasta" label="Using reference file" /></when> Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: dannon: Preliminary autopacking for javascript.
by Bitbucket 24 Feb '12

24 Feb '12

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/changeset/17caba3f5b80/ changeset: 17caba3f5b80 user: dannon date: 2012-02-24 23:47:47 summary: Preliminary autopacking for javascript. Disabled by default (pack_scripts in universe_wsgi.ini) and the packed scripts are still in the distribution, to be removed at some point. affected #: 2 files diff -r ad69e258128ae3dc5a9cfe2d13b53680279b48e8 -r 17caba3f5b808bd62a96ab3076934779dd01fb77 lib/galaxy/web/buildapp.py --- a/lib/galaxy/web/buildapp.py +++ b/lib/galaxy/web/buildapp.py @@ -140,6 +140,8 @@ webapp = wrap_in_middleware( webapp, global_conf, **kwargs ) if asbool( kwargs.get( 'static_enabled', True ) ): webapp = wrap_in_static( webapp, global_conf, **kwargs ) + if asbool(kwargs.get('pack_scripts', False)): + pack_scripts() # Close any pooled database connections before forking try: galaxy.model.mapping.metadata.engine.connection_provider._pool.dispose() @@ -147,7 +149,21 @@ pass # Return return webapp - + +def pack_scripts(): + from glob import glob + from subprocess import call + cmd = "java -jar scripts/yuicompressor.jar --type js static/scripts/%(fname)s -o static/scripts/packed/%(fname)s" + raw_js= [os.path.basename(g) for g in glob( "static/scripts/*.js" )] + for fname in raw_js: + if os.path.exists('static/scripts/packed/%s' % fname): + if os.path.getmtime('static/scripts/packed/%s' % fname) > os.path.getmtime('static/scripts/%s' % fname): + continue # Skip, packed is newer than source. + d = dict( fname=fname ) + log.info("%(fname)s --> packed/%(fname)s" % d) + call( cmd % d, shell=True ) + + def wrap_in_middleware( app, global_conf, **local_conf ): """ Based on the configuration wrap `app` in a set of common and useful diff -r ad69e258128ae3dc5a9cfe2d13b53680279b48e8 -r 17caba3f5b808bd62a96ab3076934779dd01fb77 universe_wsgi.ini.sample --- a/universe_wsgi.ini.sample +++ b/universe_wsgi.ini.sample @@ -261,6 +261,12 @@ static_style_dir = %(here)s/static/june_2007_style/blue static_robots_txt = %(here)s/static/robots.txt +# Pack javascript at launch (/static/scripts/*.js) +# This only happens if the modified timestamp of the source .js is newer +# than the version (if it exists) in /static/scripts/packed/ +# Note that this requires java > 1.4 for executing yuicompressor.jar +#pack_scripts = False + # -- Advanced proxy features # For help on configuring the Advanced proxy features, see: @@ -525,7 +531,6 @@ # be used for each "Set at Runtime" input, independent of others in the Workflow #enable_unique_workflow_defaults = False - # -- Job Execution # If running multiple Galaxy processes, one can be designated as the job @@ -673,4 +678,5 @@ #queue = galaxy_queue #exchange = galaxy_exchange #routing_key = bar_code_scanner -#rabbitmqctl_path = /path/to/rabbitmqctl +#rabbitmqctl_path = /path/to/rabbitmqctl + Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: fubar: remove validator from input genome - as Jeremy pointed out it will prevent the use of an unspecified build local genome.
by Bitbucket 24 Feb '12

24 Feb '12

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/changeset/ad69e258128a/ changeset: ad69e258128a user: fubar date: 2012-02-24 22:48:03 summary: remove validator from input genome - as Jeremy pointed out it will prevent the use of an unspecified build local genome. affected #: 1 file diff -r 71f34e63440125da6b2e572ea360b2032cfa79fc -r ad69e258128ae3dc5a9cfe2d13b53680279b48e8 tools/extract/extract_genomic_dna.xml --- a/tools/extract/extract_genomic_dna.xml +++ b/tools/extract/extract_genomic_dna.xml @@ -23,9 +23,7 @@ #end if </command><inputs> - <param format="interval,gff" name="input" type="data" label="Fetch sequences for intervals in"> - <validator type="unspecified_build" /> - </param> + <param format="interval,gff" name="input" type="data" label="Fetch sequences for intervals in"/><param name="interpret_features" type="select" label="Interpret features when possible" help="Only meaningful for GFF, GTF datasets."><option value="yes">Yes</option><option value="no">No</option> Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: natefoo: Include the job id in the error message when a job operation raises an exception in the PBS runner.
by Bitbucket 24 Feb '12

24 Feb '12

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/changeset/71f34e634401/ changeset: 71f34e634401 user: natefoo date: 2012-02-24 21:49:54 summary: Include the job id in the error message when a job operation raises an exception in the PBS runner. affected #: 1 file diff -r 4f8164a1a9c5504152decfb89d54a171d54de988 -r 71f34e63440125da6b2e572ea360b2032cfa79fc lib/galaxy/jobs/runners/pbs.py --- a/lib/galaxy/jobs/runners/pbs.py +++ b/lib/galaxy/jobs/runners/pbs.py @@ -193,7 +193,7 @@ elif op == 'fail': self.fail_job( obj ) except: - log.exception( "Uncaught exception %sing job" % op ) + log.exception( "(%s) Uncaught exception %sing job" % ( getattr( obj, 'job_id', None ), op ) ) def queue_job( self, job_wrapper ): """Create PBS script for a job and submit it to the PBS queue""" Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: dannon: boto 2.2.2 - update eggs.ini
by Bitbucket 24 Feb '12

24 Feb '12

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/changeset/4f8164a1a9c5/ changeset: 4f8164a1a9c5 user: dannon date: 2012-02-24 19:58:01 summary: boto 2.2.2 - update eggs.ini affected #: 1 file diff -r a9e1f0c2f2398661152ef221193e944e50cb71a1 -r 4f8164a1a9c5504152decfb89d54a171d54de988 eggs.ini --- a/eggs.ini +++ b/eggs.ini @@ -32,7 +32,7 @@ [eggs:noplatform] amqplib = 0.6.1 Beaker = 1.4 -boto = 1.8d +boto = 2.2.2 decorator = 3.1.2 docutils = 0.7 drmaa = 0.4b3 Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: dannon: FTP Upload - Normalize filesystem and input filename encodings prior to comparison.
by Bitbucket 24 Feb '12

24 Feb '12

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/changeset/a9e1f0c2f239/ changeset: a9e1f0c2f239 user: dannon date: 2012-02-24 19:37:57 summary: FTP Upload - Normalize filesystem and input filename encodings prior to comparison. Umlauts, etc will now behave properly in FTP upload dialog and will no longer default to "No data was entered in the upload form...". affected #: 1 file diff -r 9a5a2a6bbec2b74f888b907aff05ebcfb274aee4 -r a9e1f0c2f2398661152ef221193e944e50cb71a1 lib/galaxy/tools/parameters/grouping.py --- a/lib/galaxy/tools/parameters/grouping.py +++ b/lib/galaxy/tools/parameters/grouping.py @@ -2,18 +2,19 @@ Constructs for grouping tool parameters """ -from basic import ToolParameter -from galaxy.util.expressions import ExpressionContext - import logging log = logging.getLogger( __name__ ) -import StringIO, os, urllib +import os +import StringIO +import unicodedata +from basic import ToolParameter from galaxy.datatypes import sniff +from galaxy.util import inflector +from galaxy.util import relpath +from galaxy.util import sanitize_for_filename from galaxy.util.bunch import Bunch -from galaxy.util.odict import odict -from galaxy.util import json, relpath, sanitize_for_filename -from galaxy.util import inflector +from galaxy.util.expressions import ExpressionContext class Group( object ): def __init__( self ): @@ -306,6 +307,9 @@ # look for files uploaded via FTP valid_files = [] if ftp_files is not None: + # Normalize input paths to ensure utf-8 encoding is normal form c. + # This allows for comparison when the filesystem uses a different encoding than the browser. + ftp_files = [unicodedata.normalize('NFC', f) for f in ftp_files] if trans.user is None: log.warning( 'Anonymous user passed values in ftp_files: %s' % ftp_files ) ftp_files = [] @@ -316,7 +320,8 @@ for filename in filenames: path = relpath( os.path.join( dirpath, filename ), user_ftp_dir ) if not os.path.islink( os.path.join( dirpath, filename ) ): - valid_files.append( path ) + # Normalize filesystem paths + valid_files.append( unicodedata.normalize('NFC', path )) else: ftp_files = [] for ftp_file in ftp_files: Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: fubar: Add a validator type="unspecified_build"
by Bitbucket 24 Feb '12

24 Feb '12

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/changeset/9a5a2a6bbec2/ changeset: 9a5a2a6bbec2 user: fubar date: 2012-02-24 08:27:34 summary: Add a validator type="unspecified_build" to extract genomic dna tool affected #: 1 file diff -r f4389a047276b4a465b01cc33e1a8c316466a3cc -r 9a5a2a6bbec2b74f888b907aff05ebcfb274aee4 tools/extract/extract_genomic_dna.xml --- a/tools/extract/extract_genomic_dna.xml +++ b/tools/extract/extract_genomic_dna.xml @@ -23,7 +23,9 @@ #end if </command><inputs> - <param format="interval,gff" name="input" type="data" label="Fetch sequences for intervals in"/> + <param format="interval,gff" name="input" type="data" label="Fetch sequences for intervals in"> + <validator type="unspecified_build" /> + </param><param name="interpret_features" type="select" label="Interpret features when possible" help="Only meaningful for GFF, GTF datasets."><option value="yes">Yes</option><option value="no">No</option> Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: jgoecks: Put Trackster stylesheets in display base so that they are available for embedded visualizations.
by Bitbucket 23 Feb '12

23 Feb '12

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/changeset/f4389a047276/ changeset: f4389a047276 user: jgoecks date: 2012-02-23 20:47:08 summary: Put Trackster stylesheets in display base so that they are available for embedded visualizations. affected #: 2 files diff -r 8cf1bb73a24748aea056a0f1e218783dcaac6e1b -r f4389a047276b4a465b01cc33e1a8c316466a3cc templates/display_base.mako --- a/templates/display_base.mako +++ b/templates/display_base.mako @@ -112,7 +112,7 @@ <%def name="stylesheets()"> ${parent.stylesheets()} - ${h.css( "autocomplete_tagging", "embed_item", "trackster", "jquery.rating" )} + ${h.css( "autocomplete_tagging", "embed_item", "trackster", "jquery.rating", "overcast/jquery-ui-1.8.5.custom" )} <style type="text/css"> .page-body { diff -r 8cf1bb73a24748aea056a0f1e218783dcaac6e1b -r f4389a047276b4a465b01cc33e1a8c316466a3cc templates/visualization/display.mako --- a/templates/visualization/display.mako +++ b/templates/visualization/display.mako @@ -19,8 +19,6 @@ } </style> - ## For visualization (TODO: copied from browser.mako): - ${h.css( "trackster", "overcast/jquery-ui-1.8.5.custom" )} <style type="text/css"> #browser-container { overflow: none; Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: dan: Correct typo in GATK EvalVariant tool.
by Bitbucket 23 Feb '12

23 Feb '12

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/changeset/8cf1bb73a247/ changeset: 8cf1bb73a247 user: dan date: 2012-02-23 19:48:09 summary: Correct typo in GATK EvalVariant tool. affected #: 3 files diff -r 981e31193e25dc5db1923ef2d77c88b99347d670 -r 8cf1bb73a24748aea056a0f1e218783dcaac6e1b test-data/gatk/gatk_variant_eval/gatk_variant_eval_out_1.gatk_report --- /dev/null +++ b/test-data/gatk/gatk_variant_eval/gatk_variant_eval_out_1.gatk_report @@ -0,0 +1,30 @@ +##:GATKReport.v0.2 CompOverlap : The overlap between eval and comp sites +CompOverlap CompRod EvalRod JexlExpression Novelty nEvalVariants novelSites nVariantsAtComp compRate nConcordant concordantRate +CompOverlap dbsnp input_0 none all 0 0 0 0.00 0 0.00 +CompOverlap dbsnp input_0 none known 0 0 0 0.00 0 0.00 +CompOverlap dbsnp input_0 none novel 0 0 0 0.00 0 0.00 + +##:GATKReport.v0.2 CountVariants : Counts different classes of variants in the sample +CountVariants CompRod EvalRod JexlExpression Novelty nProcessedLoci nCalledLoci nRefLoci nVariantLoci variantRate variantRatePerBp nSNPs nMNPs nInsertions nDeletions nComplex nSymbolic nMixed nNoCalls nHets nHomRef nHomVar nSingletons nHomDerived heterozygosity heterozygosityPerBp hetHomRatio indelRate indelRatePerBp deletionInsertionRatio +CountVariants dbsnp input_0 none all 5386 1 1 0 0.00000000 0.00000000 0 0 0 0 0 0 0 1 0 0 0 0 0 0.00e+00 0.00 0.00 0.00e+00 0.00 0.00 +CountVariants dbsnp input_0 none known 5386 1 1 0 0.00000000 0.00000000 0 0 0 0 0 0 0 1 0 0 0 0 0 0.00e+00 0.00 0.00 0.00e+00 0.00 0.00 +CountVariants dbsnp input_0 none novel 5386 0 0 0 0.00000000 0.00000000 0 0 0 0 0 0 0 0 0 0 0 0 0 0.00e+00 0.00 0.00 0.00e+00 0.00 0.00 + +##:GATKReport.v0.2 TiTvVariantEvaluator : Ti/Tv Variant Evaluator +TiTvVariantEvaluator CompRod EvalRod JexlExpression Novelty nTi nTv tiTvRatio nTiInComp nTvInComp TiTvRatioStandard nTiDerived nTvDerived tiTvDerivedRatio +TiTvVariantEvaluator dbsnp input_0 none all 0 0 0.00 0 0 0.00 0 0 0.00 +TiTvVariantEvaluator dbsnp input_0 none known 0 0 0.00 0 0 0.00 0 0 0.00 +TiTvVariantEvaluator dbsnp input_0 none novel 0 0 0.00 0 0 0.00 0 0 0.00 + +##:GATKReport.v0.2 ValidationReport : Assess site accuracy and sensitivity of callset against follow-up validation assay +ValidationReport CompRod EvalRod JexlExpression Novelty nComp TP FP FN TN sensitivity specificity PPV FDR CompMonoEvalNoCall CompMonoEvalFiltered CompMonoEvalMono CompMonoEvalPoly CompPolyEvalNoCall CompPolyEvalFiltered CompPolyEvalMono CompPolyEvalPoly CompFiltered nDifferentAlleleSites +ValidationReport dbsnp input_0 none all 43 0 0 0 43 NaN 100.00 NaN NaN 42 0 1 0 0 0 0 0 0 0 +ValidationReport dbsnp input_0 none known 1 0 0 0 1 NaN 100.00 NaN NaN 0 0 1 0 0 0 0 0 0 0 +ValidationReport dbsnp input_0 none novel 42 0 0 0 42 NaN 100.00 NaN NaN 42 0 0 0 0 0 0 0 0 0 + +##:GATKReport.v0.2 VariantSummary : 1000 Genomes Phase I summary of variants table +VariantSummary CompRod EvalRod JexlExpression Novelty nSamples nProcessedLoci nSNPs TiTvRatio SNPNoveltyRate nSNPsPerSample TiTvRatioPerSample SNPDPPerSample nIndels IndelNoveltyRate nIndelsPerSample IndelDPPerSample nSVs SVNoveltyRate nSVsPerSample +VariantSummary dbsnp input_0 none all 1 5386 0 0.00 NA 0 0.00 0.0 0 NA 0 0.0 0 NA 0 +VariantSummary dbsnp input_0 none known 1 5386 0 0.00 NA 0 0.00 0.0 0 NA 0 0.0 0 NA 0 +VariantSummary dbsnp input_0 none novel 1 5386 0 0.00 NA 0 0.00 0.0 0 NA 0 0.0 0 NA 0 + diff -r 981e31193e25dc5db1923ef2d77c88b99347d670 -r 8cf1bb73a24748aea056a0f1e218783dcaac6e1b test-data/gatk/gatk_variant_eval/gatk_variant_eval_out_1.tabular --- a/test-data/gatk/gatk_variant_eval/gatk_variant_eval_out_1.tabular +++ /dev/null @@ -1,30 +0,0 @@ -##:GATKReport.v0.2 CompOverlap : The overlap between eval and comp sites -CompOverlap CompRod EvalRod JexlExpression Novelty nEvalVariants novelSites nVariantsAtComp compRate nConcordant concordantRate -CompOverlap dbsnp input_0 none all 0 0 0 0.00 0 0.00 -CompOverlap dbsnp input_0 none known 0 0 0 0.00 0 0.00 -CompOverlap dbsnp input_0 none novel 0 0 0 0.00 0 0.00 - -##:GATKReport.v0.2 CountVariants : Counts different classes of variants in the sample -CountVariants CompRod EvalRod JexlExpression Novelty nProcessedLoci nCalledLoci nRefLoci nVariantLoci variantRate variantRatePerBp nSNPs nMNPs nInsertions nDeletions nComplex nSymbolic nMixed nNoCalls nHets nHomRef nHomVar nSingletons nHomDerived heterozygosity heterozygosityPerBp hetHomRatio indelRate indelRatePerBp deletionInsertionRatio -CountVariants dbsnp input_0 none all 5386 1 1 0 0.00000000 0.00000000 0 0 0 0 0 0 0 1 0 0 0 0 0 0.00e+00 0.00 0.00 0.00e+00 0.00 0.00 -CountVariants dbsnp input_0 none known 5386 1 1 0 0.00000000 0.00000000 0 0 0 0 0 0 0 1 0 0 0 0 0 0.00e+00 0.00 0.00 0.00e+00 0.00 0.00 -CountVariants dbsnp input_0 none novel 5386 0 0 0 0.00000000 0.00000000 0 0 0 0 0 0 0 0 0 0 0 0 0 0.00e+00 0.00 0.00 0.00e+00 0.00 0.00 - -##:GATKReport.v0.2 TiTvVariantEvaluator : Ti/Tv Variant Evaluator -TiTvVariantEvaluator CompRod EvalRod JexlExpression Novelty nTi nTv tiTvRatio nTiInComp nTvInComp TiTvRatioStandard nTiDerived nTvDerived tiTvDerivedRatio -TiTvVariantEvaluator dbsnp input_0 none all 0 0 0.00 0 0 0.00 0 0 0.00 -TiTvVariantEvaluator dbsnp input_0 none known 0 0 0.00 0 0 0.00 0 0 0.00 -TiTvVariantEvaluator dbsnp input_0 none novel 0 0 0.00 0 0 0.00 0 0 0.00 - -##:GATKReport.v0.2 ValidationReport : Assess site accuracy and sensitivity of callset against follow-up validation assay -ValidationReport CompRod EvalRod JexlExpression Novelty nComp TP FP FN TN sensitivity specificity PPV FDR CompMonoEvalNoCall CompMonoEvalFiltered CompMonoEvalMono CompMonoEvalPoly CompPolyEvalNoCall CompPolyEvalFiltered CompPolyEvalMono CompPolyEvalPoly CompFiltered nDifferentAlleleSites -ValidationReport dbsnp input_0 none all 43 0 0 0 43 NaN 100.00 NaN NaN 42 0 1 0 0 0 0 0 0 0 -ValidationReport dbsnp input_0 none known 1 0 0 0 1 NaN 100.00 NaN NaN 0 0 1 0 0 0 0 0 0 0 -ValidationReport dbsnp input_0 none novel 42 0 0 0 42 NaN 100.00 NaN NaN 42 0 0 0 0 0 0 0 0 0 - -##:GATKReport.v0.2 VariantSummary : 1000 Genomes Phase I summary of variants table -VariantSummary CompRod EvalRod JexlExpression Novelty nSamples nProcessedLoci nSNPs TiTvRatio SNPNoveltyRate nSNPsPerSample TiTvRatioPerSample SNPDPPerSample nIndels IndelNoveltyRate nIndelsPerSample IndelDPPerSample nSVs SVNoveltyRate nSVsPerSample -VariantSummary dbsnp input_0 none all 1 5386 0 0.00 NA 0 0.00 0.0 0 NA 0 0.0 0 NA 0 -VariantSummary dbsnp input_0 none known 1 5386 0 0.00 NA 0 0.00 0.0 0 NA 0 0.0 0 NA 0 -VariantSummary dbsnp input_0 none novel 1 5386 0 0.00 NA 0 0.00 0.0 0 NA 0 0.0 0 NA 0 - diff -r 981e31193e25dc5db1923ef2d77c88b99347d670 -r 8cf1bb73a24748aea056a0f1e218783dcaac6e1b tools/gatk/variant_eval.xml --- a/tools/gatk/variant_eval.xml +++ b/tools/gatk/variant_eval.xml @@ -1,4 +1,4 @@ -<tool id="gatk_variant_eval" name="Eval Variants" version="0.0.7"> +<tool id="gatk_variant_eval" name="Eval Variants" version="0.0.8"><description></description><requirements><requirement type="package" version="1.4">gatk</requirement> @@ -13,7 +13,7 @@ -p 'java -jar "${GALAXY_DATA_INDEX_DIR}/shared/jars/gatk/GenomeAnalysisTK.jar" -T "VariantEval" - --out "${output_table}" + --out "${output_report}" --num_threads 4 ##hard coded, for now -et "NO_ET" ##ET no phone home ##-log "${output_log}" ##don't use this to log to file, instead directly capture stdout @@ -495,7 +495,7 @@ </inputs><outputs> - <data format="gatkreport" name="output_table" label="${tool.name} on ${on_string} (table)" /> + <data format="gatk_report" name="output_report" label="${tool.name} on ${on_string} (report)" /><data format="txt" name="output_log" label="${tool.name} on ${on_string} (log)" /></outputs><tests> @@ -509,7 +509,7 @@ <param name="comp_rod_bind" value="0" /><param name="gatk_param_type_selector" value="basic" /><param name="analysis_param_type_selector" value="basic" /> - <output name="output_table" file="gatk/gatk_variant_eval/gatk_variant_eval_out_1.tabular" /> + <output name="output_report" file="gatk/gatk_variant_eval/gatk_variant_eval_out_1.gatk_report" /><output name="output_log" file="gatk/gatk_variant_eval/gatk_variant_eval_out_1.log.contains" compare="contains" /></test></tests> Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: guru: Added apiMel3 to manual builds.
by Bitbucket 23 Feb '12

23 Feb '12

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/changeset/981e31193e25/ changeset: 981e31193e25 user: guru date: 2012-02-23 19:38:01 summary: Added apiMel3 to manual builds. affected #: 1 file diff -r 96a30bc9711e4019e31e3f597b305953b2444a72 -r 981e31193e25dc5db1923ef2d77c88b99347d670 tool-data/shared/ucsc/manual_builds.txt --- a/tool-data/shared/ucsc/manual_builds.txt +++ b/tool-data/shared/ucsc/manual_builds.txt @@ -705,3 +705,4 @@ Arabidopsis_thaliana_TAIR10 Arabidopsis thaliana TAIR10 (Arabidopsis_thaliana_TAIR10) chr1=30427671,chr2=19698289,chr3=23459830,chr4=18585056,chr5=26975502,chrM=366924,chrC=154478 dp4 D. pseudoobscura (dp4) Tcacao_1.0 Theobroma cacao Dec 2010 (CIRAD/Theobroma cacao Belizian Criollo) Tc01=31268538,Tc02=27754001,Tc03=25475297,Tc04=23504306,Tc05=25651337,Tc06=15484475,Tc07=14169093,Tc08=11535834,Tc09=28459094,Tc10=15164258,Tc00=108886888 +apiMel3 Honeybee (Apis mellifera): apiMel3 Group10=11440700,Group11=12576330,Group12=9182753,Group13=8929068,Group14=8318479,Group15=7856270,Group16=5631066,Group1=25854376,Group2=14465785,Group3=12341916,Group4=10796202,Group5=13386189,Group6=14581788,Group7=9974240,Group8=11452794,Group9=10282195,GroupUn=399230636 \ No newline at end of file Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

galaxy-commits February 2012