April 2011 - galaxy-user - lists.galaxyproject.org

too many list digests
by Yury Bukhman 14 Apr '11

14 Apr '11

Hi, I have been receiving multiple digests of this list every day, as many as 8 on April 6! Is it possible to make the digests less frequent? One per day would really be enough for me. Thanks. Yury -- Yury V. Bukhman, Ph.D. Associate Scientist, Bioinformatics Great Lakes Bioenergy Research Center University of Wisconsin - Madison 445 Henry Mall, Rm. 513 Madison, WI 53706, USA Phone: 608-890-2680 Fax: 608-890-2427 Email: ybukhman(a)glbrc.wisc.edu

4 3

Re: [galaxy-user] RNA seq analysis and GTF files
by Jeremy Goecks 14 Apr '11

14 Apr '11

David, Your analysis looks reasonable. In fact, in your isoform tracking FPKM file you get nearest_ref_id, so that's promising. What I think is needed is the addition of an attribute called gene_name to your reference file; you can use whatever value you want for gene name, and using the same value as gene_id probably makes sense. Rerun your analysis with the further-patched GTF file, and let us know if this doesn't solve the problem. Also note that even using this attribute, some gene name/ids and some nearest_ref_id columns will not be populated in some cuffdiff files. See the post from Howie in this thread for an explanation from a Cufflinks developer: http://seqanswers.com/forums/showthread.php?t=6288 Best, J. On Apr 7, 2011, at 5:00 PM, David K Crossman wrote: > Jeremy, > > I’ve shared it with you using your email address. > > Thanks, > David > > > From: Jeremy Goecks [mailto:jeremy.goecks@emory.edu] > Sent: Thursday, April 07, 2011 3:42 PM > To: David K Crossman > Cc: galaxy-user > Subject: Re: [galaxy-user] RNA seq analysis and GTF files > > David, can you please share your history with me and I'll take a look (History Options --> Share/Publish --> Share with User --> my email? > > Thanks, > J. > > On Apr 7, 2011, at 3:23 PM, David K Crossman wrote: > > > Hello! > > I would like to ask a question related to this thread below. I ran into the same issues as below and was unaware of having to swap some columns around in the GTF file. So, after 'swapping the gene name from the complete table (name2 value, column 12) into the GFT file's gene_id value (which by default is the same as transcript_id)," I uploaded this "patched" file (mm9) into Galaxy and ran Cufflinks, CuffCompare and CuffDiff using this "patched" GTF file as the reference annotation. For both Cufflinks and CuffCompare, the gene_id was present in their respective columns. The problem I have encountered now is that in all of the output files in CuffDiff, the gene_id column is blank (contains a "-"; highlighted in yellow below). This example is from the CuffDiff gene expression output file: > > test_id > gene > locus > sample_1 > sample_2 > status > value_1 > value_2 > ln(fold_change) > test_stat > p_value > significant > XLOC_000001 > - > chr1:4797973-4836816 > q1 > q2 > OK > 73.1908 > 82.1567 > 0.115559 > -0.71896 > 0.472168 > no > XLOC_000002 > - > chr1:4847774-4887990 > q1 > q2 > OK > 81.7264 > 53.1165 > -0.43089 > 2.44474 > 0.014496 > no > XLOC_000003 > - > chr1:5073253-5152630 > q1 > q2 > OK > 408.289 > 333.749 > -0.20159 > 2.73173 > 0.0063 > no > XLOC_000004 > - > chr1:5578573-5596214 > q1 > q2 > NOTEST > 2.34764 > 4.79772 > 0.71473 > -0.89735 > 0.369532 > no > > What am I doing wrong? I am interested in the differentially expressed genes in this RNA-Seq dataset (as well as calling variants, which is my next step, but want to get this answered first before moving on). Any info, suggestions or help would be greatly appreciated. > > Thanks, > David > > > -----Original Message----- > From: galaxy-user-bounces(a)lists.bx.psu.edu [mailto:galaxy-user-bounces@lists.bx.psu.edu] On Behalf Of Jeremy Goecks > Sent: Friday, April 01, 2011 8:47 AM > To: <ssassi(a)CCIB.MGH.HARVARD.EDU> > Cc: galaxy-user > Subject: Re: [galaxy-user] RNA seq analysis and GTF files > > > > On Mar 31, 2011, at 12:30 PM, <ssassi(a)CCIB.MGH.HARVARD.EDU> <ssassi(a)CCIB.MGH.HARVARD.EDU> wrote: > > > Hi Jeremy, > > I used your exercise to perform an RNA-seq analysis. First I encountered a problem where the gene IDs were missing from the results. Jen from the Galaxy team suggested this: > > > > "Yes, the team has taken a look and there are a few things going on. > > > > The first is that when running the Cuffcompare program, a reference annotation file in GTF format should be used in order to obtain the same results as in Jeremy's exercise. This seemed to be missing from your runs, which resulted in badly formatted output that later resulted in a poor result when Cuffdiff was used. > > > > The second has to do with the reference GTF file itself. For the best results, the GTF file must have the "gene_id" attribute defined in the 9th column of the file and the chromosome names must be in the same format as the genome native to Galaxy. Depending on the source of the reference GTF, one of these may need to be adjusted. Chromosome names can be adjusted using Galaxy's "Text Manipulation" tools. The gene_id attribute would need to be adjusted prior to loading into Galaxy. > > > > For mm9, using the "Get Data -> UCSC Main table browser" tool can help you to obtain all of the raw data necessary to create a complete GTF file with a gene_id identifier. Extract data from the track "RefSeq Genes" and output the primary data table "refGene" twice - first in GTF format, then again as the complete table in tabular format (not BED). Then, using your own tools, swap in the gene name from the complete table (name2 value, column 12) into the GTF file's gene_id value (which by default is the same as transcript_id). Upload and the tools will function as intended. > > > > The team is aware of the issues associated with GTF source files and is discussing solutions. Any changes to native data content will be reported to the mailing list in a News Brief or other communications. > > > > Our apologies for the inconvenience! Thanks for using Galaxy and > > please let us know if we can help again, > > > > Best, > > > > Jen > > Galaxy team" > > > > > > I followed the directions (or at least I think I did) and things seemed to work better but there is one more issue for example in file: > > Galaxy287-[Cuffdiff_on_data_197,_data_197,_and_data_274__isoform_FPKM_ > > tracking].tabular.txt The column gene_short_name does not have any > > names in it. nearest_ref_id does have the gene ID info so I can still interpret the data, but I was wondering if there remains another problem that I'm not aware of with the GTF file. > > Slim, > > Please send questions to the galaxy-user mailing list (cc'd) rather than individual Galaxy team members; there are many people on the list that may be able to address your question, and discussions are archived for future use as well. Without seeing your analysis, I'd suggest trying two things: > > (1) Provide gene annotation reference file to Cufflinks as well as Cuffcompare and Cuffdiff; in other words, you'll want to do guided assembly. > (2) Try using an Ensembl GTF, which has the gene name in the attributes. > > I think (2) is more likely to generate the results you want, but there are the many known problems in using Ensembl GTFs with Cufflinks/compare/diff. > > Good luck, > J. > ___________________________________________________________ > The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ >

2 3

Nucleotide analysis - GC percentage
by Peter Cock 14 Apr '11

14 Apr '11

Hi all, Are there any built in Galaxy tools that I have missed to do with GC percentage (or indeed, AT percentage)? I'm thinking of a tool to calculate the GC percentage (and perhaps related statistics like counts/percentages of A, C, G, T), and perhaps a related tool to filter on GC. Possible use cases include filtering NGS reads to remove high/low GC reads from a contaminate. Slightly more complicated, right now I want to calculate the GC (or in fact AT) percentage from the first and last ~20 (configurable) bases. In this case I am looking for (and filtering on) AT rich ends of contigs which may be indicative of viral sequences. A very similar task would be looking for (and filtering on) poly A tails of mRNA, or if sequenced from the reverse strand, a poly T start. Peter

8 11

View Running Job from Multiple Computers?
by Mike Dufault 14 Apr '11

14 Apr '11

Hi, I have an instance of Galaxy running on AWS. I would like to view it's progress throughout the day, but I am not always at my home computer. Is it possible to enter the public DNS from any computer to view the progress? I am hesitant to try anything which may interfere with the job that is currently running. Thanks, Mike

2 1

extraction of intervals
by pande 14 Apr '11

14 Apr '11

Dear Galaxy, I have two ChIP reads and want to extract the regions in between them . For example chr1:66585377-66585400 and chr1:66585616-66585639 and now I would like to extract the region beginning from 665855401 up to 66585615. Can I get help from Galaxy tools. regards, Amit

2 1

Re: [galaxy-user] cufflinks FPKM problem
by 李世勇 11 Apr '11

11 Apr '11

Hi,Paul Korir: Thank you for yours help.I have known the reason,But I also I have a little problem about to solve the question. if I want to add a XS tag ,what should I do ,can you tell me in detail(like that ,dose it only have two value ,such as XS:A:-,XS:A:+ ,not have XS:B（[B-Z]）:+ ? Best wishes ----- 原始邮件 ----- 发件人: "Paul Korir" <polariseke(a)gmail.com> 收件人: "lishiyong" <lishiyong(a)genomics.org.cn> 抄送: "tophat.cufflinks" <tophat.cufflinks(a)gmail.com>, "galaxy-user" <galaxy-user(a)lists.bx.psu.edu>, "高欢" <gaohuan(a)genomics.org.cn> 发送时间: 星期一, 2011年 4 月 11日下午 11:10:56 主题: Re: [galaxy-user] cufflinks FPKM problem Hi Li, Tophat includes a custom tag 'XS' at the end of spliced read alignments which your pipeline is not aware about. The following is taken from http://cufflinks.cbcb.umd.edu/manual.html "Cufflinks takes a text file of SAM alignments as input. For more details on the SAM format, see the specification . The RNA-Seq read mapper TopHat produces output in this format, and is recommended for use with Cufflinks. However Cufflinks will accept SAM alignments generated by any read mapper. Here's an example of an alignment Cufflinks will accept: s6.25mer.txt-913508 16 chr1 4482736 255 14M431N11M * 0 0 \ CAAGATGCTAGGCAAGTCTTGGAAG IIIIIIIIIIIIIIIIIIIIIIIII NM:i:0 XS:A:- Note the use of the custom tag XS . This attribute, which must have a value of "+" or "-", indicates which strand the RNA that produced this read came from. While this tag can be applied to any alignment, including unspliced ones, it must be present for all spliced alignment records (those with a 'N' operation in the CIGAR string)." Kind regards, Paul 2011/4/11 lishiyong < lishiyong(a)genomics.org.cn > Hi： I use the solid PE sequencing data and mapped with the bioscope tools(AB company supported) ，which is better for solid data mapping ,so I don't use the bowtie to map . Ｉgain the BAM file!　Now ,I want use the cufflinks to calculate the gene expression. But there is a error. [15:08:06] Inspecting reads and determining fragment length distribution. BAM record error: found spliced alignment without XS attribute BAM record error: found spliced alignment without XS attribute the BAM file : 323_358_2010 73 chr1 343 0 45M5H * 0 0 CCCTAACCCTACCCTAACCCTAACCCTAACCCTAACCCTAACCCT IIIIIIIIIII))C/1<DE''@DAHD379AID1;7BI+'7))I?3 RG:Z:20110328192522421 NH:i:0 CM:i:4 SM:i:2 CQ:Z:A=ABA<<>@?<4)='))415'-4118-'1)9>'+1'<6+'1)85+)-+6- CS:Z:T20023010023110230100030100230100230100030000200000 423_236_1955 81 chr1 550 0 8H42M = 699451 698945 GTGCAGAGGAGAACGCAGCTCCGCCCTCGCGGTGCTCTCCGG GF>IIII%%III))8IIII?IIII%%IIIIIIIIIIIIIIII RG:Z:20110328192522421 NH:i:2 CM:i:5 SM:i:3 CQ:Z:9BA<AAB>;?AB:55;A%9?AB,4:@@*/)7>2<%5@<:3,;-.%8.*;5 CS:Z:T20302222311033322303302232133302223222131122330223 298_1884_1495 113 chr1 562 0 7H43M chr3 199392032 0 ACGCAGCTCCGCCCTCGCGGTGCTCTCCGGGTCTGTGCTGAGG 5AI;6:>AIIII>?I7FIEIIIIIIIIIIIIIIIIIIIIIIII RG:Z:20110328192522421 NH:i:2 CM:i:0 SM:i:3 CQ:Z:BB@7<AB8@ABA=2;=>82:?A388.A&28(77;64.1*-/<&0:9/%3? CS:Z:T20221231112210030222231103332200330223213312222022 62_1428_1954 89 chr1 562 1 50M * 0 0 ACGCAGCTCCGCCCTCGCGGTGCTCTCCGGGTCTGTGCTGAGGAGAATGC *=AIII4/CII=%%I((=EIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII RG:Z:20110328192522421 NH:i:0 CM:i:4 SM:i:0 CQ:Z:@B@BABB=ABBB?@A=B>>@@?<;?>B>=<??'7(;A%&849+%0:@.4* CS:Z:T13130222022123111221003022223110331222033022321331 I have sorted the bam file and the gtf file. cufflinks -G refGene_hg18.gtf -p 3 -r human_hg18.fa -o test test.pe.bam (the version of cufflinks is v0.9.2 ) Who know the reason ,and what shoud I do! best wishes! Shiyong Li 2011-04-11 lishiyong ___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org . Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Paul Korir www.paulkorir.com

1 0

cufflinks FPKM problem
by lishiyong 11 Apr '11

11 Apr '11

Hi： I use the solid PE sequencing data and mapped with the bioscope tools(AB company supported) ，which is better for solid data mapping ,so I don't use the bowtie to map . Ｉgain the BAM file!　Now ,I want use the cufflinks to calculate the gene expression. But there is a error. [15:08:06] Inspecting reads and determining fragment length distribution. BAM record error: found spliced alignment without XS attribute BAM record error: found spliced alignment without XS attribute the BAM file : 323_358_2010 73 chr1 343 0 45M5H * 0 0 CCCTAACCCTACCCTAACCCTAACCCTAACCCTAACCCTAACCCT IIIIIIIIIII))C/1<DE''@DAHD379AID1;7BI+'7))I?3 RG:Z:20110328192522421 NH:i:0 CM:i:4 SM:i:2 CQ:Z:A=ABA<<>@?<4)='))415'-4118-'1)9>'+1'<6+'1)85+)-+6- CS:Z:T20023010023110230100030100230100230100030000200000 423_236_1955 81 chr1 550 0 8H42M = 699451 698945 GTGCAGAGGAGAACGCAGCTCCGCCCTCGCGGTGCTCTCCGG GF>IIII%%III))8IIII?IIII%%IIIIIIIIIIIIIIII RG:Z:20110328192522421 NH:i:2 CM:i:5 SM:i:3 CQ:Z:9BA<AAB>;?AB:55;A%9?AB,4:@@*/)7>2<%5@<:3,;-.%8.*;5 CS:Z:T20302222311033322303302232133302223222131122330223 298_1884_1495 113 chr1 562 0 7H43M chr3 199392032 0 ACGCAGCTCCGCCCTCGCGGTGCTCTCCGGGTCTGTGCTGAGG 5AI;6:>AIIII>?I7FIEIIIIIIIIIIIIIIIIIIIIIIII RG:Z:20110328192522421 NH:i:2 CM:i:0 SM:i:3 CQ:Z:BB@7<AB8@ABA=2;=>82:?A388.A&28(77;64.1*-/<&0:9/%3? CS:Z:T20221231112210030222231103332200330223213312222022 62_1428_1954 89 chr1 562 1 50M * 0 0 ACGCAGCTCCGCCCTCGCGGTGCTCTCCGGGTCTGTGCTGAGGAGAATGC *=AIII4/CII=%%I((=EIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII RG:Z:20110328192522421 NH:i:0 CM:i:4 SM:i:0 CQ:Z:@B@BABB=ABBB?@A=B>>@@?<;?>B>=<??'7(;A%&849+%0:@.4* CS:Z:T13130222022123111221003022223110331222033022321331 I have sorted the bam file and the gtf file. cufflinks -G refGene_hg18.gtf -p 3 -r human_hg18.fa -o test test.pe.bam (the version of cufflinks is v0.9.2 ) Who know the reason ,and what shoud I do! best wishes! Shiyong Li 2011-04-11 lishiyong

5 4

Re: [galaxy-user] downstream analysis of cuffdiff out put
by Jeremy Goecks 11 Apr '11

11 Apr '11

> On Thu, Mar 10, 2011 at 7:55 AM, Jeremy Goecks <jeremy.goecks(a)emory.edu> wrote: > Jagat, > Just like any mRNA-seq experiment to achieve following objectives: >> 1. Reconstruct all transcripts of a particular gene and corresponding Cuffdiff significantly expressed transcripts as called by cuffdiff. >> 2. What are different isoforms >> 3. Location of splicing >> >> From various output files which unique ID can be matched from one file say Cuffdiff.expr (transcript/ isoform/Splicing) to other file - transcript.gtf corresponding to each sample or combined GTF file. >> > I've got a script that does this for the cuffdiff isoform expression testing file and a GTF file; I'll wrap it up and add it to Galaxy in the next couple weeks. It would probably be useful to have similar scripts for the other expression testing files as well. Also, it would be nice to be able to take the FPKM values generated by Cuffdiff and attach them to their respective transcripts as attributes. Hello all, I've added a tool called 'Filter GTF file by attribute values list' to the galaxy-central code repository. This tool is available on our test server ( http://test.g2.bx.psu.edu/ ) at Filter and Sort --> GFF --> Filter GTF data by attribute values list and will be available on our main server in the next few weeks. As expected, this tool filters a GTF file based on a list of attribute values--or filters using a tabular file where attribute values are first column, as is the case for Cuffdiff output files. Potential attributes that can be filtered on include transcript_id, gene_id, tss_id, and p_id; conveniently, these are the IDs that Cuffdiff uses in its output files. Here's an example workflow: (1) Run Cufflinks/compare/diff (2) Filter Cufflinks isoform differential expression file for transcripts that are differentially expressed; in other words, filter for c12=='yes' (2) Use 'Filter GTF data by attribute values list' to filter Cuffcompare combined transcripts using the filtered file from step (2) as the attribute values list and, voila, you have a GTF file of the differentially expressed transcripts that you can view in your favorite genome browser. Hope this helps; feedback is always welcome. Best, J.

1 0

2011 Galaxy Community Conference, May 25-26, Lunteren, The Netherlands
by Dave Clements 11 Apr '11

11 Apr '11

Hello all, We are pleased to announce the *2011 Galaxy Community Conference*, being held *May 25-26 in Lunteren, The Netherlands*. The meeting will feature two full days of presentations and discussion on extending Galaxy to use new tools and data sources, deploying Galaxy at your organization, and best practices for using Galaxy to further your own and your community's research. *Link: http://galaxy.psu.edu/gcc2011/ * *Overview *This event aims to engage a broader community of developers, data producers, tool creators, and core facility and other research hub staff to become an active part of the Galaxy community. We'll cover defining resources in the Galaxy framework, increasing their visibility and making them easier to use and integrate with other resources, how to extend Galaxy to use custom data sources and custom tools, and best practices for using Galaxy in your organization. Additional topics include, but are not limited to: * Talks submitted by the Galaxy community * Integration of tools (including NGS analysis tools) and distributed job management * Deployment of Galaxy instances on local resources and on the Cloud * Management of large datasets with the Galaxy Library System * Using the Galaxy LIMS functionality at NGS sequencing facilities * Visualizing Data without leaving Galaxy * Performing reproducible research * Performing and sharing complex analyses with Workflows * An "Introduction to Galaxy" session, offered on May 24, for Galaxy newcomers. *Registration *The conference fee is €100 on or before April 24, and €120 after that. The meeting is being held at the Conference Centre De Werelt in Lunteren, The Netherlands, which is also the conference hotel. You are encouraged to register early, as space at the hotel (and at the "Intro to Galaxy" session) is limited and is likely to fill up before the conference itself does. *Link: http://galaxy.psu.edu/gcc2011/Register.html ** Abstract Submission *Abstracts are now being accepted for short oral presentations. Proposals on any topic of interest to the Galaxy community are welcome and encouraged. The abstract submission deadline is the end of February 28. *Link: http://galaxy.psu.edu/gcc2011/Abstracts.html * *Sponsors *The 2011 Galaxy Community Conference is co-sponsored by the US National Science Foundation (NSF), and the Netherlands Bioinformatics Centre (NBIC). NBIC is a collaborative institute of the bioinformatics groups in the Netherlands. Together, these groups perform cutting-edge research, develop novel tools and support platforms, create an e-science infrastructure and educate the next generations of bioinformaticians. *Links: http://www.nbic.nl/ and http://www.nsf.gov/ * We are looking forward to a great conference and hope to see you in the Netherlands! The Galaxy and NBIC Teams -- http://galaxy.psu.edu/gcc2011/ http://getgalaxy.org http://usegalaxy.org/

5 7

Mapping Illumina reads using LASTZ
by Kamila Knapik 10 Apr '11

10 Apr '11

Hello, I am trying to map metagenomic Illumina reads to the reference genomes. I used Galaxy LASTZ tool to align Illumina reads to bacteriophage B3, that genomic sequence I downloaded from NCBI website. LASTZ produced an output file in SAM format, the file size is 1.8 MB and contain 16,000 lines. Then, I used Galaxy SAM tools to converted SAM to BAM, and now I am trying to visualize the BAM file in e.g. BamView or Integrated Genome Browser, but without any success. Can anybody tell me what I am doing wrong or how to visualize the alignment, please? Thank you, Kamila

2 1