1 new changeset in galaxy-central:
http://bitbucket.org/galaxy/galaxy-central/changeset/39a034c36baa/ changeset: r5568:39a034c36baa user: kellyv date: 2011-05-17 07:57:04 summary: Numerous updates to Picard tools: Cleaned up and standardized wrapper's Python code, removing redundant variables and using the same ones in different segment of code when possible; Changed sam/bam outputs to be bam by default and use change_format if user requests sam and removed redundant output parameters; Got tests passing for AddOrReplaceReadGroups; Relaxed the matching constraints for output for BamIndexStats and InsertSize wrappers Standardized on the parameter output-format rather than newformat to indicate sam preference over bam affected #: 14 files (2.5 KB)
--- a/test-data/picard_ARRG_output1.sam Mon May 16 17:08:06 2011 -0400 +++ b/test-data/picard_ARRG_output1.sam Tue May 17 01:57:04 2011 -0400 @@ -3,22 +3,22 @@ @SQ SN:chr2 LN:100001 @SQ SN:chr3 LN:10001 @SQ SN:chr4 LN:1001 -@RG ID:1 PL:illumina PU:plat LB:lib SM:sam1 -bar:record:4 77 chr1 1 0 * * 0 0 AAAAAAAAAAAAA 1111111111111 RG:Z:1 -bar:record:6 77 chr1 1 0 * * 0 0 AAAAAAAAAAAAA 1111111111111 RG:Z:1 -bar:record:1 77 chr1 10 0 * * 0 0 AAAAAAAAAAAAA 1111111111111 RG:Z:1 -bar:record:3 77 chr1 10 0 * * 0 0 AAAAAAAAAAAAA 1111111111111 RG:Z:1 -bar:record:1 141 chr1 20 0 * * 0 0 CCCCCCCCCCCCC 2222222222222 RG:Z:1 -bar:record:7 77 chr1 20 0 * * 0 0 AAAAAAAAAAAAA 1111111111111 RG:Z:1 -bar:record:8 77 chr1 30 0 * * 0 0 AAAAAAAAAAAAA 1111111111111 RG:Z:1 -bar:record:4 141 chr1 40 0 * * 0 0 CCCCCCCCCCCCC 2222222222222 RG:Z:1 -bar:record:5 77 chr1 40 0 * * 0 0 AAAAAAAAAAAAA 1111111111111 RG:Z:1 -bar:record:6 141 chr1 50 0 * * 0 0 CCCCCCCCCCCCC 2222222222222 RG:Z:1 -bar:record:2 77 chr2 10 0 * * 0 0 AAAAAAAAAAAAA 1111111111111 RG:Z:1 -bar:record:2 141 chr2 30 0 * * 0 0 CCCCCCCCCCCCC 2222222222222 RG:Z:1 -bar:record:3 141 chr3 20 0 * * 0 0 CCCCCCCCCCCCC 2222222222222 RG:Z:1 -bar:record:8 141 chr3 20 0 * * 0 0 CCCCCCCCCCCCC 2222222222222 RG:Z:1 -bar:record:5 141 chr3 40 0 * * 0 0 CCCCCCCCCCCCC 2222222222222 RG:Z:1 -bar:record:9 77 chr4 10 0 * * 0 0 AAAAAAAAAAAAA 1111111111111 RG:Z:1 -bar:record:7 141 chr4 20 0 * * 0 0 CCCCCCCCCCCCC 2222222222222 RG:Z:1 -bar:record:9 141 chr4 60 0 * * 0 0 CCCCCCCCCCCCC 2222222222222 RG:Z:1 +@RG ID:one PL:illumina PU:peaewe LB:lib SM:sam1 +bar:record:4 77 chr1 1 0 * * 0 0 AAAAAAAAAAAAA 1111111111111 RG:Z:one +bar:record:6 77 chr1 1 0 * * 0 0 AAAAAAAAAAAAA 1111111111111 RG:Z:one +bar:record:1 77 chr1 10 0 * * 0 0 AAAAAAAAAAAAA 1111111111111 RG:Z:one +bar:record:3 77 chr1 10 0 * * 0 0 AAAAAAAAAAAAA 1111111111111 RG:Z:one +bar:record:1 141 chr1 20 0 * * 0 0 CCCCCCCCCCCCC 2222222222222 RG:Z:one +bar:record:7 77 chr1 20 0 * * 0 0 AAAAAAAAAAAAA 1111111111111 RG:Z:one +bar:record:8 77 chr1 30 0 * * 0 0 AAAAAAAAAAAAA 1111111111111 RG:Z:one +bar:record:4 141 chr1 40 0 * * 0 0 CCCCCCCCCCCCC 2222222222222 RG:Z:one +bar:record:5 77 chr1 40 0 * * 0 0 AAAAAAAAAAAAA 1111111111111 RG:Z:one +bar:record:6 141 chr1 50 0 * * 0 0 CCCCCCCCCCCCC 2222222222222 RG:Z:one +bar:record:2 77 chr2 10 0 * * 0 0 AAAAAAAAAAAAA 1111111111111 RG:Z:one +bar:record:2 141 chr2 30 0 * * 0 0 CCCCCCCCCCCCC 2222222222222 RG:Z:one +bar:record:3 141 chr3 20 0 * * 0 0 CCCCCCCCCCCCC 2222222222222 RG:Z:one +bar:record:8 141 chr3 20 0 * * 0 0 CCCCCCCCCCCCC 2222222222222 RG:Z:one +bar:record:5 141 chr3 40 0 * * 0 0 CCCCCCCCCCCCC 2222222222222 RG:Z:one +bar:record:9 77 chr4 10 0 * * 0 0 AAAAAAAAAAAAA 1111111111111 RG:Z:one +bar:record:7 141 chr4 20 0 * * 0 0 CCCCCCCCCCCCC 2222222222222 RG:Z:one +bar:record:9 141 chr4 60 0 * * 0 0 CCCCCCCCCCCCC 2222222222222 RG:Z:one
--- a/test-data/picard_ARRG_output2.sam Mon May 16 17:08:06 2011 -0400 +++ b/test-data/picard_ARRG_output2.sam Tue May 17 01:57:04 2011 -0400 @@ -2,14 +2,23 @@ @SQ SN:chr1 LN:10001 @SQ SN:chr2 LN:100001 @SQ SN:chr3 LN:10001 +@SQ SN:chr4 LN:1001 @RG ID:M5 PL:IL PU:PLAT LB:LIB DS:description with spaces SM:smp CN:FamousCenter bar:record:4 77 chr1 1 0 * * 0 0 AAAAAAAAAAAAA 1111111111111 RG:Z:M5 +bar:record:6 77 chr1 1 0 * * 0 0 AAAAAAAAAAAAA 1111111111111 RG:Z:M5 bar:record:1 77 chr1 10 0 * * 0 0 AAAAAAAAAAAAA 1111111111111 RG:Z:M5 bar:record:3 77 chr1 10 0 * * 0 0 AAAAAAAAAAAAA 1111111111111 RG:Z:M5 bar:record:1 141 chr1 20 0 * * 0 0 CCCCCCCCCCCCC 2222222222222 RG:Z:M5 +bar:record:7 77 chr1 20 0 * * 0 0 AAAAAAAAAAAAA 1111111111111 RG:Z:M5 +bar:record:8 77 chr1 30 0 * * 0 0 AAAAAAAAAAAAA 1111111111111 RG:Z:M5 bar:record:4 141 chr1 40 0 * * 0 0 CCCCCCCCCCCCC 2222222222222 RG:Z:M5 bar:record:5 77 chr1 40 0 * * 0 0 AAAAAAAAAAAAA 1111111111111 RG:Z:M5 +bar:record:6 141 chr1 50 0 * * 0 0 CCCCCCCCCCCCC 2222222222222 RG:Z:M5 bar:record:2 77 chr2 10 0 * * 0 0 AAAAAAAAAAAAA 1111111111111 RG:Z:M5 bar:record:2 141 chr2 30 0 * * 0 0 CCCCCCCCCCCCC 2222222222222 RG:Z:M5 bar:record:3 141 chr3 20 0 * * 0 0 CCCCCCCCCCCCC 2222222222222 RG:Z:M5 +bar:record:8 141 chr3 20 0 * * 0 0 CCCCCCCCCCCCC 2222222222222 RG:Z:M5 bar:record:5 141 chr3 40 0 * * 0 0 CCCCCCCCCCCCC 2222222222222 RG:Z:M5 +bar:record:9 77 chr4 10 0 * * 0 0 AAAAAAAAAAAAA 1111111111111 RG:Z:M5 +bar:record:7 141 chr4 20 0 * * 0 0 CCCCCCCCCCCCC 2222222222222 RG:Z:M5 +bar:record:9 141 chr4 60 0 * * 0 0 CCCCCCCCCCCCC 2222222222222 RG:Z:M5
--- a/test-data/picard_BIS_output1.txt Mon May 16 17:08:06 2011 -0400 +++ b/test-data/picard_BIS_output1.txt Tue May 17 01:57:04 2011 -0400 @@ -13,8 +13,8 @@ <body><div class="document"> Galaxy tool BamIndexStats run at 12/05/2011 14:18:06</b><br/><b>The following output files were created (click the filename to view/download a copy):</b><hr/><table> +<tr><td><a href="BamIndexStats.log">BamIndexStats.log</a></td></tr><tr><td><a href="BamIndexStats.metrics.txt">BamIndexStats.metrics.txt</a></td></tr> -<tr><td><a href="BamIndexStats.log">BamIndexStats.log</a></td></tr></table><p/><b>Picard on line resources</b><ul><li><a href="http://picard.sourceforge.net/index.shtml">Click here for Picard Documentation</a></li>
--- a/test-data/picard_BIS_output2.txt Mon May 16 17:08:06 2011 -0400 +++ b/test-data/picard_BIS_output2.txt Tue May 17 01:57:04 2011 -0400 @@ -13,8 +13,8 @@ <body><div class="document"> Galaxy tool BamIndexStats run at 12/05/2011 14:18:22</b><br/><b>The following output files were created (click the filename to view/download a copy):</b><hr/><table> +<tr><td><a href="BamIndexStats.log">BamIndexStats.log</a></td></tr><tr><td><a href="BamIndexStats.metrics.txt">BamIndexStats.metrics.txt</a></td></tr> -<tr><td><a href="BamIndexStats.log">BamIndexStats.log</a></td></tr></table><p/><b>Picard on line resources</b><ul><li><a href="http://picard.sourceforge.net/index.shtml">Click here for Picard Documentation</a></li>
--- a/tools/picard/picard_AddOrReplaceReadGroups.xml Mon May 16 17:08:06 2011 -0400 +++ b/tools/picard/picard_AddOrReplaceReadGroups.xml Tue May 17 01:57:04 2011 -0400 @@ -1,4 +1,4 @@ -<tool name="Add or Replace Groups:" id="picard_ARRG" version="0.01"> +<tool name="Add or Replace Groups:" id="picard_ARRG" version="0.2.0"><requirements><requirement type="package">picard</requirement></requirements><description>(Picard)</description><command interpreter="python"> @@ -8,18 +8,14 @@ --rg-pl="$rgpl" --rg-pu="$rgpu" --rg-sm="$rgsm" + --rg-id="$rgid" --rg-opts=${readGroupOpts.rgOpts} #if $readGroupOpts.rgOpts == "full" - --rg-id="$readGroupOpts.rgid" --rg-cn="$readGroupOpts.rgcn" --rg-ds="$readGroupOpts.rgds" #end if --output-format=$outputFormat - #if str( $outputFormat ) == "sam" - --output-sam=$outFileSam - #else if str( $outputFormat ) == "bam" - --output-sam=$outFileBam - #end if + --output=$outFile -j "${GALAXY_DATA_INDEX_DIR}/shared/jars/AddOrReplaceReadGroups.jar" </command><inputs> @@ -29,14 +25,14 @@ <param name="rgpl" value="" type="text" label="Read group platform" help="illumina, solid, etc." /><param name="rgpu" value="" type="text" label="Read group platform unit" help="like run barcode, etc." /><param name="rgsm" value="" type="text" label="Read group sample name" /> + <param name="rgid" value="1" type="text" label="Read group ID" help="Picard will use a value of '1' if nothing provided" /><conditional name="readGroupOpts"> - <param name="rgOpts" type="select" label="Specify additional (optional) arguments" help="Allows you to set RGID, RGCN, and RGDS"> + <param name="rgOpts" type="select" label="Specify additional (optional) arguments" help="Allows you to set RGCN and RGDS."><option value="preSet">Use pre-set defaults</option><option value="full">Set optional arguments</option></param><when value="preSet" /><when value="full"> - <param name="rgid" value="" type="text" label="Read group ID" /><param name="rgcn" value="" type="text" label="Read group sequencing center name" help="Leave set to <null> for default (none)" /><param name="rgds" value="" type="text" label="Read group description" help="Leave set to <null> for default (none)" /></when> @@ -44,70 +40,58 @@ <param name="outputFormat" type="boolean" checked="True" truevalue="bam" falsevalue="sam" label="Output bam instead of sam" help="Uncheck for sam output" /></inputs><outputs> - <data name="outFileSam" format="sam" label="${tool.name} on ${on_string}: sam with read groups replaced"> - <filter>outputFormat is False</filter> - </data> - <data name="outFileBam" format="bam" label="${tool.name} on ${on_string}: bam with read groups replaced"> - <filter>outputFormat is True</filter> + <data name="outFile" format="bam" label="${tool.name} on ${on_string}: ${outputFormat} with read groups replaced"> + <change_format> + <when input="outputFormat" value="sam" format="sam" /> + </change_format></data></outputs><tests><test> - <!-- Command: - java -jar AddOrReplaceReadGroups.jar VALIDATION_STRINGENCY=LENIENT I=picard_input_input1.sam O=picard_ARG_output1.sam RGLB=lib RGPL=illumina RGPU=plat RGSM=sam1 + <!-- Command for replacing read groups in bam: + java -jar AddOrReplaceReadGroups.jar VALIDATION_STRINGENCY=LENIENT I=test-data/picard_ARRG_input1.bam O=picard_ARRG_output1.sam RGID=one RGLB=lib RGPL=illumina RGPU=peaewe RGSM=sam1 + --> + <param name="inputFile" value="picard_ARRG_input1.bam" /> + <param name="rglb" value="lib" /> + <param name="rgpl" value="illumina" /> + <param name="rgpu" value="peaewe" /> + <param name="rgsm" value="sam1" /> + <param name="rgid" value="one" /> + <param name="rgOpts" value="preSet" /> + <param name="outputFormat" value="False" /> + <output name="outFile" file="picard_ARRG_output1.sam" ftype="sam" /> + </test> + <test> + <!-- Command for replacing read groups in sam: + java -jar AddOrReplaceReadGroups.jar VALIDATION_STRINGENCY=LENIENT I=test-data/picard_ARRG_input1.sam O=picard_ARRG_output2.sam RGLB=LIB RGPL=IL RGPU=PLAT RGSM=smp RGID=M5 RGCN=FamousCenter RGDS="description with spaces" + picard_ARRG_input1.bam can be created from picard_ARRG_input1.sam --><param name="inputFile" value="picard_ARRG_input1.sam" /> - <param name="rglb" value="lib" /> - <param name="rgpl" value="illumina" /> - <param name="rgpu" value="plat" /> - <param name="rgsm" value="sam1" /> - <param name="rgOpts" value="preSet" /> - <param name="outputFormat" value="sam" /> - <output name="outFileSam" file="picard_ARRG_output1.sam" /> - </test> - <!-- Functional tests with Picard bam outputs currently aren't working - <test> - --> - <!-- Command: - java -jar AddOrReplaceReadGroups.jar VALIDATION_STRINGENCY=LENIENT I=picard_ARRG_input1.bam O=picard_ARRG_output2.sam RGLB=LIB RGPL=IL RGPU=PLAT RGSM=smp RGID=M5 RGCN=FamousCenter RGDS="description with spaces" - picard_ARRG_input1.bam can be created from picard_ARRG_input1.sam - --> - <!-- - <param name="inputFile" value="picard_ARRG_input1.bam" /> - <param name="sortOrder" value="" /><param name="rglb" value="LIB" /><param name="rgpl" value="IL" /><param name="rgpu" value="PLAT" /><param name="rgsm" value="smp" /> + <param name="rgid" value="M5" /><param name="rgOpts" value="full" /> - <param name="rgid" value="M5" /><param name="rgcn" value="FamousCenter" /><param name="rgds" value="description with spaces" /> - <param name="outputFormat" value="sam" /> - <output name="outFileSam" ftype="picard_ARRG_output2.sam" /> + <param name="outputFormat" value="False" /> + <output name="outFile" file="picard_ARRG_output2.sam" ftype="sam" /></test> - --> - <!-- Functional tests with Picard bam outputs currently aren't working <test> - --> - <!-- Command: - java -jar AddOrReplaceReadGroups.jar VALIDATION_STRINGENCY=LENIENT I=picard_ARRG_input1.sam O=picard_ARRG_output2.bam RGLB=LIB RGPL=IL RGPU=PLAT RGSM=smp RGID=M5 RGCN=FamousCenter RGDS="description with spaces" + <!-- Command for adding read groups in sam: + java -jar AddOrReplaceReadGroups.jar VALIDATION_STRINGENCY=LENIENT I=test-data/picard_ARRG_input2.sam O=picard_ARRG_output3.bam RGID=M6 RGLB=LIB RGPL=IL RGPU=PLAT RGSM=smp1 --> - <!-- - <param name="inputFile" value="picard_ARRG_input1.sam" /> - <param name="sortOrder" value="" /> + <param name="inputFile" value="picard_ARRG_input2.sam" /><param name="rglb" value="LIB" /><param name="rgpl" value="IL" /><param name="rgpu" value="PLAT" /> - <param name="rgsm" value="smp" /> - <param name="rgOpts" value="full" /> - <param name="rgid" value="M5" /> - <param name="rgcn" value="FamousCenter" /> - <param name="rgds" value="description with spaces" /> - <param name="outputFormat" value="bam" /> - <output name="outFileBam" ftype="picard_ARRG_output2.bam" /> + <param name="rgsm" value="smp1" /> + <param name="rgid" value="M6" /> + <param name="rgOpts" value="preSet" /> + <param name="outputFormat" value="True" /> + <output name="outFile" file="picard_ARRG_output3.bam" ftype="bam" /></test> - --></tests><help>
--- a/tools/picard/picard_BamIndexStats.xml Mon May 16 17:08:06 2011 -0400 +++ b/tools/picard/picard_BamIndexStats.xml Tue May 17 01:57:04 2011 -0400 @@ -1,4 +1,4 @@ -<tool name="BAM Index Statistics:" id="picard_BamIndexStats" version="0.01"> +<tool name="BAM Index Statistics:" id="picard_BamIndexStats" version="0.2.0"><requirements><requirement type="package">picard</requirement></requirements><description>(Picard)</description><command interpreter="python"> @@ -23,7 +23,7 @@ picard_input_tiny_coord.bam can be created from picard_input_tiny_coord.sam --><param name="input_file" value="picard_input_tiny_coord.bam" ftype="bam" /> - <output name="htmlfile" file="picard_BIS_output1.txt" ftype="html" lines_diff="12"/> + <output name="htmlfile" file="picard_BIS_output1.txt" ftype="html" compare="contains" lines_diff="12"/></test><test><!-- Command @@ -31,7 +31,7 @@ picard_BIS_input1.bam can be created from picard_BIS_input1.sam --><param name="input_file" value="picard_BIS_input1.bam" ftype="bam" /> - <output name="htmlfile" file="picard_BIS_output2.txt" ftype="html" lines_diff="12" /> + <output name="htmlfile" file="picard_BIS_output2.txt" ftype="html" compare="contains" lines_diff="12" /></test></tests><help>
--- a/tools/picard/picard_ReorderSam.xml Mon May 16 17:08:06 2011 -0400 +++ b/tools/picard/picard_ReorderSam.xml Tue May 17 01:57:04 2011 -0400 @@ -1,22 +1,22 @@ -<tool name="Reorder SAM:" id="picard_ReorderSam" version="0.02"> +<tool name="Reorder SAM:" id="picard_ReorderSam" version="0.3.0"><requirements><requirement type="package">picard</requirement></requirements><description>(Picard)</description><command interpreter="python"> - picard_wrapper.py -d "$html_file.files_path" -t "$html_file" --input=$inputFile + picard_wrapper.py + --input=$inputFile #if $source.indexSource == "built-in" --ref="${ filter( lambda x: str( x[0] ) == str( $source.ref ), $__app__.tool_data_tables[ 'picard_indexes' ].get_fields() )[0][-1] }" #else - --ref-file=$source.refFile + --ref-file=$refFile --species-name=$source.speciesName --build-name=$source.buildName --trunc-names=$source.truncateSeqNames #end if --allow-inc-dict-concord=$allowIncDictConcord --allow-contig-len-discord=$allowContigLenDiscord - --newformat=$outputFormat - --output="$out_file" + --output-format=$outputFormat + --output=$outFile -j "${GALAXY_DATA_INDEX_DIR}/shared/jars/ReorderSam.jar" - --picard-cmd="ReorderSam" </command><inputs><param format="bam,sam" name="inputFile" type="data" label="The sam or bam file in your current history whose header you want to replace" @@ -42,25 +42,21 @@ <param name="allowContigLenDiscord" type="boolean" checked="False" truevalue="true" falsevalue="false" label="Allow contig length discordance?" help="This is dangerous--don't check it unless you know exactly what you're doing!" /><param name="outputFormat" type="boolean" checked="True" truevalue="bam" falsevalue="sam" label="Output bam instead of sam" help="Uncheck for sam output" /></inputs> - <outputs> - <data format="html" name="html_file" label="ReorderBam_on_${on_string}.html" /> - <data format="bam" name="out_file" label="ReorderBam_on_${on_string}.${outputFormat}"> - <change_format> - <when input="outputFormat" value="sam" format="sam" /> - </change_format> + <outputs> + <data name="outFile" format="bam" label="${tool.name} on ${on_string}: reordered ${outputFormat}"> + <change_format> + <when input="outputFormat" value="sam" format="sam" /> + </change_format></data></outputs><tests> - <!-- Functional tests with Picard bam outputs currently aren't working <test> - --><!-- Commands: cp test-data/phiX.fasta . samtools faidx phiX.fasta java -jar CreateSequenceDictionary.jar R=phiX.fasta O=phiX.dict URI=phiX.fasta TRUNCATE_NAMES_AT_WHITESPACE=false SPECIES=phiX174 java -jar ReorderSam.jar VALIDATION_STRINGENCY=LENIENT I=test-data/picard_RS_input1.bam O=picard_RS_output1.bam REFERENCE=phiX.fasta ALLOW_INCOMPLETE_DICT_CONCORDANCE=false ALLOW_CONTIG_LENGTH_DISCORDANCE=false --> - <!-- <param name="inputFile" value="picard_RS_input1.bam" /><param name="indexSource" value="history" /><param name="refFile" value="phiX.fasta" /> @@ -69,10 +65,9 @@ <param name="truncateSeqNames" value="false" /><param name="allowIncDictConcord" value="false" /><param name="allowContigLenDiscord" value="false" /> - <param name="outputFormat" value="bam" /> - <output name="outFileBam" file="picard_RS_output1.bam" ftype="bam" lines_diff="2" /> + <param name="outputFormat" value="True" /> + <output name="outFile" file="picard_RS_output1.bam" ftype="bam" lines_diff="4" compare="contains" /></test> - --><test><!-- Command: java -jar ReorderSam.jar VALIDATION_STRINGENCY=LENIENT I=test-data/picard_RS_input2.sam O=picard_RS_output2.sam REFERENCE=/path/to/phiX/picard_index/phiX.fa ALLOW_INCOMPLETE_DICT_CONCORDANCE=false ALLOW_CONTIG_LENGTH_DISCORDANCE=false @@ -83,8 +78,8 @@ <param name="ref" value="phiX" /><param name="allowIncDictConcord" value="false" /><param name="allowContigLenDiscord" value="false" /> - <param name="outputFormat" value="sam" /> - <output name="outFileSam" file="picard_RS_output2.sam" ftype="sam" lines_diff="2" /> + <param name="outputFormat" value="False" /> + <output name="outFile" file="picard_RS_output2.sam" ftype="sam" lines_diff="4" sort="True" /></test><test><!-- Commands: @@ -102,8 +97,8 @@ <param name="truncateSeqNames" value="true" /><param name="allowIncDictConcord" value="true" /><param name="allowContigLenDiscord" value="false" /> - <param name="outputFormat" value="sam" /> - <output name="outFileSam" file="picard_RS_output3.sam" ftype="sam" lines_diff="12" sort="True" /> + <param name="outputFormat" value="False" /> + <output name="outFile" file="picard_RS_output3.sam" ftype="sam" lines_diff="12" sort="True" /></test></tests><help>
--- a/tools/picard/picard_ReplaceSamHeader.xml Mon May 16 17:08:06 2011 -0400 +++ b/tools/picard/picard_ReplaceSamHeader.xml Tue May 17 01:57:04 2011 -0400 @@ -1,14 +1,10 @@ -<tool name="Replace Sam Header:" id="picard_ReplaceSamHeader" version="0.0.1"> +<tool name="Replace Sam Header:" id="picard_ReplaceSamHeader" version="0.2.0"><requirements><requirement type="package">picard</requirement></requirements><description>(Picard)</description><command interpreter="python"> picard_wrapper.py --input "$inputFile" - #if str( $outputFormat ) == "sam" - -o "$outFileSam" - #else if str( $outputFormat ) == "bam" - -o "$outFileBam" - #end if + -o $outFile --header-file $headerFile --output-format $outputFormat -j "${GALAXY_DATA_INDEX_DIR}/shared/jars/ReplaceSamHeader.jar" @@ -19,17 +15,13 @@ help="If the select list is empty, you need to upload or import some aligned short read data from a shared library" /><param format="bam,sam" name="headerFile" type="data" label="sam or bam file from which header will be read" help="If the select list is empty, you need to upload or import some aligned short read data from a shared library" /> - <param name="outputFormat" type="select" label="Output format - use bam to save space please"> - <option value="bam" selected="true">Binary (compressed) bam</option> - <option value="sam">Uncompressed, space hungry sam</option> - </param> + <param name="outputFormat" type="boolean" checked="True" truevalue="bam" falsevalue="sam" label="Output bam instead of sam" help="Uncheck for sam output" /></inputs><outputs> - <data name="outFileSam" format="sam" label="${tool.name} on ${on_string}: sam with replaced header"> - <filter>outputFormat == 'sam'</filter> - </data> - <data name="outFileBam" format="bam" label="${tool.name} on ${on_string}: bam with replaced header"> - <filter>outputFormat == 'bam'</filter> + <data name="outFile" format="bam" label="${tool.name} on ${on_string}: ${outputFormat} with replaced header"> + <change_format> + <when input="outputFormat" value="sam" format="sam" /> + </change_format></data></outputs><tests> @@ -40,8 +32,8 @@ --><param name="inputFile" value="picard_input_tiny_coord.bam" ftype="bam" /><param name="headerFile" value="picard_RSH_input1.bam" ftype="bam" /> - <param name="outputFormat" value="sam" /> - <output name="outFileSam" file="picard_RSH_output1.sam" ftype="sam" /> + <param name="outputFormat" value="False" /> + <output name="outFile" file="picard_RSH_output1.sam" ftype="sam" /></test><test><!-- Command: @@ -50,22 +42,18 @@ --><param name="inputFile" value="picard_input_tiny_coord.sam" ftype="sam" /><param name="headerFile" value="picard_RSH_input1.bam" ftype="bam" /> - <param name="outputFormat" value="sam" /> - <output name="outFileSam" file="picard_RSH_output2.sam" ftype="sam" /> + <param name="outputFormat" value="False" /> + <output name="outFile" file="picard_RSH_output2.sam" ftype="sam" /></test> - <!-- Functional tests with Picard bam outputs currently aren't working <test> - --><!-- Command: java -jar ReplaceSamHeader.jar VALIDATION_STRINGENCY=LENIENT I=test-data/picard_input_tiny_coord.sam HEADER=test-data/picard_RSH_input1.sam O=picard_RSH_output2.bam --> - <!-- <param name="inputFile" value="picard_input_tiny_coord.sam" ftype="sam" /><param name="headerFile" value="picard_RSH_input1.sam" ftype="sam" /> - <param name="outputFormat" value="bam" /> - <output name="outFileBam" file="picard_RSH_output2.bam" ftype="bam" /> + <param name="outputFormat" value="True" /> + <output name="outFile" file="picard_RSH_output2.bam" ftype="bam" /></test> - --></tests><help>
--- a/tools/picard/picard_wrapper.py Mon May 16 17:08:06 2011 -0400 +++ b/tools/picard/picard_wrapper.py Tue May 17 01:57:04 2011 -0400 @@ -36,7 +36,7 @@ """ return time.strftime('%d/%m/%Y %H:%M:%S', time.localtime(time.time()))
- + class PicardBase(): """ simple base class with some utilities for Picard @@ -84,7 +84,7 @@
def readLarge(self,fname=None): - """ read a potentially huge file.. + """ read a potentially huge file. """ try: # get stderr, allowing for case where it's very large @@ -152,15 +152,15 @@ cl = ['samtools view -h -b -S -o ',tempbam,infile] tlog,stdouts = self.runCL(cl,outdir) return tlog,tempbam - - def bamToSam(self,infile=None,outdir=None): - """ - use samtools view to convert bam to sam - """ - fd,tempsam = tempfile.mkstemp(dir=outdir,suffix='rgutilsTemp.sam') - cl = ['samtools view -h -o ',tempsam,infile] - tlog,stdouts = self.runCL(cl,outdir) - return tlog,tempsam + + #def bamToSam(self,infile=None,outdir=None): + # """ + # use samtools view to convert bam to sam + # """ + # fd,tempsam = tempfile.mkstemp(dir=outdir,suffix='rgutilsTemp.sam') + # cl = ['samtools view -h -o ',tempsam,infile] + # tlog,stdouts = self.runCL(cl,outdir) + # return tlog,tempsam
def sortSam(self, infile=None,outfile=None,outdir=None): """ @@ -264,7 +264,7 @@ if llen > maxloglines: n = min(50,int(maxloglines/2)) rlog += l[:n] - rlog.append('------------ ## %d rows deleted ## --------------\n' % (llen-maxloglines)) + rlog.append('------------ ## %d rows deleted ## --------------\n' % (llen-maxloglines)) rlog += l[-n:] else: rlog += l @@ -303,8 +303,7 @@ o.write(''.join(head)) o.write(''.join(bed)) o.close() - return outf - + return outf
def cleanSam(self, insam=None, newsam=None, picardErrors=[],outformat=None): """ @@ -369,9 +368,7 @@ op.add_option('-j','--jar',default='') op.add_option('','--picard-cmd',default=None) # Many tools - op.add_option( '', '--output-txt', dest='output_txt', help='Output file in text format' ) op.add_option( '', '--output-format', dest='output_format', help='Output format' ) - op.add_option( '', '--output-sam', dest='output_sam', help='Output file in SAM or BAM format' ) op.add_option( '', '--bai-file', dest='bai_file', help='The path to the index file for the input bam file' ) op.add_option( '', '--ref', dest='ref', help='Built-in reference with fasta and dict file', default=None ) # CreateSequenceDictionary @@ -390,7 +387,6 @@ op.add_option('', '--maxinsert', default="20") op.add_option('', '--adaptors', action='append', type="string") # FixMateInformation and validate - op.add_option('','--newformat', default='bam') # CollectGcBiasMetrics op.add_option('', '--windowsize', default='100') op.add_option('', '--mingenomefrac', default='0.00001') @@ -437,10 +433,10 @@
# set ref and dict files to use (create if necessary) ref_file_name = opts.ref - if opts.ref_file <> None: + if opts.ref_file <> None: csd = 'CreateSequenceDictionary' realjarpath = os.path.split(opts.jar)[0] - jarpath = os.path.join(realjarpath,'%s.jar' % csd) # for refseq + jarpath = os.path.join(realjarpath,'%s.jar' % csd) # for refseq tmp_ref_fd, tmp_ref_name = tempfile.mkstemp( dir=opts.tmpdir , prefix = pic.picname) ref_file_name = '%s.fasta' % tmp_ref_name # build dict @@ -460,6 +456,15 @@ s = pic.runPic(jarpath, cl) # run relevant command(s)
+ # define temporary output + # if output is sam, it must have that extension, otherwise bam will be produced + # specify sam or bam file with extension + if opts.output_format == 'sam': + suff = '.sam' + else: + suff = '' + tmp_fd, tempout = tempfile.mkstemp( dir=opts.tmpdir, suffix=suff ) + cl = ['VALIDATION_STRINGENCY=LENIENT',]
if pic.picname == 'AddOrReplaceReadGroups': @@ -468,22 +473,22 @@ # input cl.append('INPUT=%s' % opts.input) # outputs - cl.append('OUTPUT=%s' % opts.output) + cl.append('OUTPUT=%s' % tempout) # required read groups cl.append('RGLB="%s"' % opts.rg_library) cl.append('RGPL="%s"' % opts.rg_platform) cl.append('RGPU="%s"' % opts.rg_plat_unit) cl.append('RGSM="%s"' % opts.rg_sample) + if opts.rg_id: + cl.append('RGID="%s"' % opts.rg_id) # optional read groups - if opts.rg_opts == 'full': - if opts.rg_id: - cl.append('RGID="%s"' % opts.rg_id) - if opts.rg_seq_center: - cl.append('RGCN="%s"' % opts.rg_seq_center) - if opts.rg_desc: - cl.append('RGDS="%s"' % opts.rg_desc) - s = pic.runPic(opts.jar, cl) - + if opts.rg_seq_center: + cl.append('RGCN="%s"' % opts.rg_seq_center) + if opts.rg_desc: + cl.append('RGDS="%s"' % opts.rg_desc) + pic.runPic(opts.jar, cl) + haveTempout = True + elif pic.picname == 'BamIndexStats': tmp_fd, tmp_name = tempfile.mkstemp( dir=tmp_dir ) tmp_bam_name = '%s.bam' % tmp_name @@ -494,7 +499,7 @@ pic.delme.append(tmp_bam_name) pic.delme.append(tmp_bai_name) pic.delme.append(tmp_name) - s = pic.runPic( opts.jar, cl ) + s = pic.runPic( opts.jar, cl ) f = open(pic.metricsOut,'a') f.write(s) # got this on stdout from runCl f.write('\n') @@ -525,7 +530,7 @@ except: s = '## unable to symlink %s to %s - different devices? May need to replace with shutil.copy' info = s - shutil.copy(ref_file_name,fakefasta) + shutil.copy(ref_file_name,fakefasta) pic.delme.append(fakefasta) cl.append('ASSUME_SORTED=%s' % opts.assumesorted) adaptorseqs = ''.join([' ADAPTER_SEQUENCE=%s' % x for x in opts.adaptors]) @@ -555,14 +560,14 @@ except: s = '## unable to symlink %s to %s - different devices? May need to replace with shutil.copy' info = s - shutil.copy(ref_file_name,fakefasta) + shutil.copy(ref_file_name,fakefasta) pic.delme.append(fakefasta) x = 'rgPicardGCBiasMetrics' pdfname = '%s.pdf' % x jpgname = '%s.jpg' % x tempout = os.path.join(opts.outdir,'rgPicardGCBiasMetrics.out') temppdf = os.path.join(opts.outdir,pdfname) - cl.append('R=%s' % fakefasta) + cl.append('R=%s' % fakefasta) cl.append('WINDOW_SIZE=%s' % opts.windowsize) cl.append('MINIMUM_GENOME_FRACTION=%s' % opts.mingenomefrac) cl.append('INPUT=%s' % opts.input) @@ -587,7 +592,7 @@ histpdf = 'InsertSizeHist.pdf' cl.append('I=%s' % opts.input) cl.append('O=%s' % pic.metricsOut) - cl.append('HISTOGRAM_FILE=%s' % histpdf) + cl.append('HISTOGRAM_FILE=%s' % histpdf) if opts.taillimit <> '0': cl.append('TAIL_LIMIT=%s' % opts.taillimit) if opts.histwidth <> '0': @@ -623,9 +628,8 @@ # maximum offset between two duplicate clusters cl.append('OPTICAL_DUPLICATE_PIXEL_DISTANCE=%s' % opts.optdupdist) pic.runPic(opts.jar, cl) - + elif pic.picname == 'FixMateInformation': - tmp_fd, tempout = tempfile.mkstemp( dir=opts.tmpdir,prefix='FixMateTempOut') cl.append('I=%s' % opts.input) cl.append('O=%s' % tempout) cl.append('SORT_ORDER=%s' % opts.sortorder) @@ -633,7 +637,6 @@ haveTempout = True
elif pic.picname == 'ReorderSam': - tmp_fd, tempout = tempfile.mkstemp( dir=opts.tmpdir,prefix='ReOrderTempOut') # input cl.append('INPUT=%s' % opts.input) # output @@ -650,18 +653,13 @@ haveTempout = True
elif pic.picname == 'ReplaceSamHeader': - tmp_fd, tempout = tempfile.mkstemp( dir=opts.tmpdir,prefix='RSHTempOut') cl.append('INPUT=%s' % opts.input) cl.append('OUTPUT=%s' % tempout) cl.append('HEADER=%s' % opts.header_file) - s = pic.runPic(opts.jar, cl) - if opts.output_format == 'sam': - tlog,newsam = pic.bamToSam(tempout,opts.tmpdir) - shutil.move(newsam,opts.output) - else: - shutil.move(tempout,opts.output) - - elif pic.picname == "CalculateHsMetrics": + pic.runPic(opts.jar, cl) + haveTempout = True + + elif pic.picname == 'CalculateHsMetrics': maxloglines = 100 baitfname = os.path.join(opts.outdir,'rgPicardHsMetrics.bait') targetfname = os.path.join(opts.outdir,'rgPicardHsMetrics.target') @@ -675,9 +673,9 @@ cl.append('INPUT=%s' % os.path.abspath(opts.input)) cl.append('OUTPUT=%s' % pic.metricsOut) cl.append('TMP_DIR=%s' % opts.tmpdir) - pic.runPic(opts.jar,cl) + pic.runPic(opts.jar,cl)
- elif pic.picname == "ValidateSamFile": + elif pic.picname == 'ValidateSamFile': import pysam doTranspose = False sortedfile = os.path.join(opts.outdir,'rgValidate.sorted') @@ -710,14 +708,12 @@ if opts.bisulphite.lower() <> 'false': cl.append('IS_BISULFITE_SEQUENCED=true') if opts.ref <> None or opts.ref_file <> None: - cl.append('R=%s' % ref_file_name) - pic.runPic(opts.jar,cl) - if opts.datatype == 'sam': + cl.append('R=%s' % ref_file_name) + pic.runPic(opts.jar,cl) + if opts.datatype == 'sam': pic.delme.append(tempbam) newsam = opts.output - outformat = 'bam' - if opts.newformat == 'sam': - outformat = 'sam' + outformat = 'bam' pe = open(pic.metricsOut,'r').readlines() pic.cleanSam(insam=sortedfile, newsam=newsam, picardErrors=pe,outformat=outformat) pic.delme.append(sortedfile) # not wanted @@ -729,11 +725,7 @@ if haveTempout: # Some Picard tools produced a potentially intermediate bam file. # Either just move to final location or create sam - if opts.newformat == 'sam': - tlog, tempsam = pic.bamToSam( tempout, opts.outdir ) - shutil.move(tempsam,os.path.abspath(opts.output)) - else: - shutil.move(tempout, os.path.abspath(opts.output)) + shutil.move(tempout, os.path.abspath(opts.output))
if opts.htmlout <> None or doFix: # return a pretty html page pic.fixPicardOutputs(transpose=doTranspose,maxloglines=maxloglines)
--- a/tools/picard/rgPicardFixMate.xml Mon May 16 17:08:06 2011 -0400 +++ b/tools/picard/rgPicardFixMate.xml Tue May 17 01:57:04 2011 -0400 @@ -1,8 +1,8 @@ -<tool name="Paired Read Mate Fixer:" id="rgPicFixMate" version="0.01"> +<tool name="Paired Read Mate Fixer:" id="rgPicFixMate" version="0.2.0"><description>for paired data</description><command interpreter="python"> picard_wrapper.py -i "$input_file" -o "$out_file" --tmpdir "${__new_file_path__}" -n "$out_prefix" - --newformat "$newformat" -j "${GALAXY_DATA_INDEX_DIR}/shared/jars/FixMateInformation.jar" --sortorder "$sortOrder" + --output-format "$outputFormat" -j "${GALAXY_DATA_INDEX_DIR}/shared/jars/FixMateInformation.jar" --sortorder "$sortOrder" </command><requirements><requirement type="package">picard</requirement></requirements><inputs> @@ -16,15 +16,12 @@ </param><param name="out_prefix" value="Fix Mate" type="text" label="Title for the output file - use this remind you what the job was for" size="80" /> - <param name="newformat" type="select" label="Output file format" size="2"> - <option value="bam" selected="true">Bam format</option> - <option value="sam">Sam format</option> - </param> + <param name="outputFormat" type="boolean" checked="True" truevalue="bam" falsevalue="sam" label="Output bam instead of sam" help="Uncheck for sam output" /></inputs><outputs> - <data format="bam" name="out_file" label="SortBam_${out_prefix}.${newformat}"> + <data format="bam" name="out_file" label="${tool.name} on ${on_string}: ${outputFormat} with fixed mates"><change_format> - <when input="newformat" value="sam" format="sam" /> + <when input="outputFormat" value="sam" format="sam" /></change_format></data></outputs> @@ -32,14 +29,14 @@ <test><param name="input_file" value="picard_input_sorted_pair.sam" /><param name="sortOrder" value="coordinate" /> - <param name="newformat" value="bam" /> + <param name="outputFormat" value="True" /><param name="out_prefix" value="Test FixMate" /><output name="out_file" file="picard_output_fixmate_sorted_pair.bam" ftype="bam" /></test><test><param name="input_file" value="picard_input_sorted_pair.sam" /><param name="sortOrder" value="coordinate" /> - <param name="newformat" value="sam" /> + <param name="outputFormat" value="False" /><param name="out_prefix" value="Test FixMate" /><output name="out_file" file="picard_output_fixmate_sorted_pair.sam" ftype="sam" /></test>
--- a/tools/picard/rgPicardInsertSize.xml Mon May 16 17:08:06 2011 -0400 +++ b/tools/picard/rgPicardInsertSize.xml Tue May 17 01:57:04 2011 -0400 @@ -1,4 +1,4 @@ -<tool name="Insertion size metrics:" id="PicardInsertSize" version="0.02"> +<tool name="Insertion size metrics:" id="PicardInsertSize" version="0.3.0"><description>for PAIRED data</description><requirements><requirement type="package">picard</requirement></requirements><command interpreter="python"> @@ -31,7 +31,7 @@ <param name="tailLimit" value="10000" /><param name="histWidth" value="0" /><param name="minPct" value="0.01" /> - <output name="html_file" file="picard_output_insertsize_tinysam.html" ftype="html" lines_diff="35" /> + <output name="html_file" file="picard_output_insertsize_tinysam.html" ftype="html" compare="contains" lines_diff="40" /></test></tests><help>
Repository URL: https://bitbucket.org/galaxy/galaxy-central/
--
This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.