- galaxy-user - lists.galaxyproject.org

Re: [galaxy-user] Metagenomic filtering
by Scott Tighe 16 Oct '13

16 Oct '13

Jing If I have a Galaxy dataset, do you think it is possible to develop a pipeline that can Megablast from Shotgun data for: DNA gyrase only Ribosomal ITSinternal transcribed spacer only Cytochrome RecA Pol As well as filter all model organisms? Have you worked with the Galaxy Toolshed? Scott Scott Tighe Senior Core Laboratory Research Staff Advanced Genome Technologies Core University of Vermont Vermont Cancer Center 149 Beaumont ave Health Science Research Facility 303/305 Burlington Vermont 05405 802-656-2557 On 10/6/2013 9:59 AM, Jing Yu wrote: > Dear Scott, > > I think what you propose is doable. > > You may > 1. use a 16s or gyrase DNA sequence as feeds to blast against your > data to get the relative sequences, > 2. and then use the sequences as feeds to blast against your > nucleotide database with appropriate filters. > > There are several ways to make the steps. For example, you may already > have the 16s sequence from assembly against a reference genome. > And for Step 2, if you are not blasting thousands of times a day, and > believe in the recent stability of NCBI, then a simple web_blast code > will do the trick. Otherwise, since the local blast+ toolkit doesn't > provide the equivalent organism filters, you'll have to work a wit bit > on it: > > Make a nucleotide database for Prokaryotes. > Search txid561[ORGN] on http://www.ncbi.nlm.nih.gov/nuccore (this is > for Escherichia as an example), > Send to 'File' -> Format ->GI List > When Blast, use this GI list as the value of this argument: > -negative_gilist > Then parse the Blast result. > > Most of these can be automated with some code, but I don't know how to > incorporate it into Galaxy. > > Regards, > Jing > On 4 Oct 2013, at 23:52, Scott Tighe <scott.tighe(a)uvm.edu > <mailto:scott.tighe@uvm.edu>> wrote: > >> Dear Jing >> >> What you have outlined below is perfect. >> >> I wonder how hard it would be to design a few filters that only look >> a certain genes and or filter model organisms out of the dataset. >> >> For example, say you want only data for 16s or only gyrase, but no >> /E.coli/ and no /Pseudomanas aeroginosa/ >> >> Scott >> Scott Tighe >> Senior Core Laboratory Research Staff >> Advanced Genome Technologies Core >> University of Vermont >> Vermont Cancer Center >> 149 Beaumont ave >> Health Science Research Facility 303/305 >> Burlington Vermont 05405 >> 802-656-2557 >> On 9/25/2013 12:06 AM, Jing Yu wrote: >>> Hi Scott, >>> >>> My first thought is: >>> >>> 1. Remove rDNA sequences (and/or other well known highly-conserved >>> sequences to reduce the workload in step 2). >>> 2. Blast, then remove sequences with > (say 99%) match to > (say 5) >>> genus. (Optional if step 1 is already good enough) >>> >>> >>> For step 1: >>> Build a fasta file of the chosen highly conserved sequences, and >>> use it as a feed to blast against your MiSeq result. >>> Remove positive hits. >>> For step 2: >>> Blast remaining MiSeq sequences against NCBI (or whatever) database. >>> Remove if it hits more than n genus. >>> >>> Jing >>> On 24 Sep 2013, at 22:17, Scott Tighe <scott.tighe(a)uvm.edu >>> <mailto:scott.tighe@uvm.edu>> wrote: >>> >>>> Jing et al >>>> >>>> Thank you for the offer to write some code to help advance the >>>> metagenomics arena. It is certainly needed. >>>> >>>> So the problem is well known with megablast and shotgun >>>> metagenomics and without proper understanding and correct software >>>> will yield very misleading and in many cases incorrect data. For >>>> those of us who wish NOT to move to a protein level of comparison >>>> for specific reasons, we are stuck. >>>> >>>> *The Problem:* >>>> >>>> If I megablast 50 million sequences from a HiSeq run, millions of >>>> rRNA sequences will have a 99% match to all microbes rRNA genbank >>>> deposits. Not surprizing since the rRNA is highly conserved. The >>>> difference between E.coli and Shigella is 1 to 2 bases for the full >>>> 1540 bp 16s. So 16s is not useful for Genus level, and certainly >>>> not Species >>>> >>>> *So what happens:* >>>> >>>> The returned matches will have many hits to whatever model organism >>>> is in Genbank. For example E coli has 13000 entries for rRNA and >>>> Sphearotilus has 3 entries for rRNA. If the blasted sequence >>>> matches both, the results will mislead the investigator to think >>>> they have 13000 hits to E coli, EVEN if the microbe is Sphearotilus. >>>> >>>> *The cure?:* >>>> >>>> If there was a way to filter/ remove all hits ? Let say, for >>>> example, that a result has a first match (say E. coli) at >99% a >>>> second match (say Pseudomanas) at >99% and a third , forth and >>>> fifth match >99 for three other organisms. This sequence _must_ be >>>> discarded because it is a conserve sequence. >>>> >>>> Basically conserved sequence is the enemy and invalidates the >>>> entire result. >>>> * >>>> **Another problem:* >>>> >>>> If you have a reference sample with 19 non-model microbes, and you >>>> run that by HiSeq Shotgun for metagenomics and then megablast, what >>>> do you think you get? If E coli is not in the reference sample, >>>> how many hits do you think you get? Yes, 10,000 of thousands. So >>>> without removing conserved sequences, your data is wrong and you >>>> are much better served by culturing and running a Biolog metabolic >>>> panel and comparing to the sequence result. >>>> >>>> So where do we start? I have some shotgun metagenomics data from >>>> the reference sample which included the 19 microbes. That was data >>>> from a MiSeq. >>>> >>>> Scott >>>> Scott Tighe >>>> Senior Core Laboratory Research Staff >>>> Advanced Genome Technologies Core >>>> University of Vermont >>>> Vermont Cancer Center >>>> 149 Beaumont ave >>>> Health Science Research Facility 303/305 >>>> Burlington Vermont 05405 >>>> 802-656-2557 >>>> On 9/20/2013 9:17 PM, Jing Yu wrote: >>>>> Hi Scott, >>>>> >>>>> I can do some perl programming, such as local/remote blasting. Can >>>>> you specify your problem a little bit clearer, so that maybe I can >>>>> write a program to do just that? >>>>> >>>>> Regards, >>>>> Jing >>>>> >>>>> >>>>> >>>>> >>>>> Gerald >>>>> >>>>> 16s is basically useless for identification to genus. Since I >>>>> started sequencing 16s in 1992, I have come to realize that >>>>> without sequencing the full 1540 bases, it is generally >>>>> misleading, and even than, it is not accurate enough to nail genus >>>>> on more than 1/2 the cases. However, what is your feeling on >>>>> ITS and gyrase, They seem to be far more discriminating but those >>>>> databases have been decommissioned sometime ago. >>>>> >>>>> The desirable thing would be that Galaxy or NCBI add a "filter >>>>> conserved genes" [ ie any hit with a second choice greater than 3% >>>>> distance]. Something such as that. >>>>> >>>>> If you (or others) are aware of such a thing, I'd love the here >>>>> about it. >>>>> >>>>> Sincerely >>>>> Scott >>>> >>> >> >

1 0

Free Training (Boston): Analyzing NGS Data with Galaxy
by Anushka Brownley 15 Oct '13

15 Oct '13

Galaxy Community, Based on popular demand, we are pleased to announce another Galaxy training event for this year. BioTeam and the Galaxy Team are offering a training session that will focus on analyzing NGS Data using Galaxy. In addition to learning about the NGS related Galaxy features, you will also learn about what it takes to host your own Galaxy instance. This event is kindly hosted by Dana-Farber Cancer Institute. For more information about this training please visit the event website at http://bioteam.net/slipstream/galaxy-edition/galaxy-training/. Date: Tuesday October 22nd, 2013 Time: 9:00am – 12:00pm Location: Dana Farber Cancer Institute, Boston, MA, USA Cost: Free Space is limited so please register for this free event soon. REGISTER NOW: http://bioteam.net/slipstream/galaxy-edition/galaxy-training/galaxy-trainin… Anushka Brownley Senior Scientific Consultant www.bioteam.net SlipStream Appliance Galaxy Edition http://bioteam.net/slipstream/galaxy-edition/ SlipStream Appliance White Paper http://bioteam.net/wp-content/uploads/2013/03/SlipStream_Galaxy-White_Paper…

1 1

Importing reference genome into Galaxy for Tophat2
by Stanislas Werfel 15 Oct '13

15 Oct '13

Hi, I am new to the field of RNA-Seq analysis. I'd be very greatful if somene could help me with the following issue: I'm using the Galaxy Main plattform and would like to run Tophat2 on RNA-Seq data from rat. However in Main under Tophat2 there are only built-in human and mouse genomes, while a lot more genomes are vailable under Tophat for Illumina. I tried to import the rat genome sequence using the built-in UCSC Table browser, however I only managed to get a full genome as one FASTA file with all the different sequences. After mapping to this genome I can't see any hits when I click "display at UCSC main" on accepted hits. So I assume that the mapped reads do not conform to the official reference rat genome (rn5). So how can one solve this isse? -Is there a way to import a proper genome for use with Tophat2? Ideally directly indo Galaxy without prior download. -If not, how does Tophat2 differ from "Tophat for Illumina". Can one also analyze non-Illumina data with that? As I understand it Tophat2 is faster than the previous version? Thanks in advance for any help.

2 1

miRNA-seq help
by Gabriel Calvin 14 Oct '13

14 Oct '13

Hi, I'm new to Galaxy and am trying to view several miRNA datasets as a differential expression. The pipeline I'm using is Bowtie for Illumina (paired-end run) > SAM-to-BAM > ? > xls. The references I used with Bowtie are a mature miRNA fasta and a piRNA fasta and the reads are 30nt in length. So, my questions are: Is this the proper pipeline? How do I go about converting the BAM into a xls file viewable in Excel? Thanks!

4 6

(no subject)
by Stanislas w 12 Oct '13

12 Oct '13

set delivery on

1 0

New cuffdiff- no result files
by Johanna Sandgren 11 Oct '13

11 Oct '13

Hi, I have this week used the new wrapper version of Cuffdiff that you provide that should now include replicate information in output. But there is no resulting files created. Why? I get all the results in the right panel as usual and they are green (no error message) but the they are all empty.. I use bam-files and cuffmerge files that I have gotten result from before (both in spring and during summer- when you upgraded cuffdiff version). I attach below info for one resulting output. Kind regards, Johanna Tool: Cuffdiff Name: Cuffdiff on data 684, data 384, and others: transcript FPKM tracking Created: Oct 09, 2013 Filesize: 0 bytes Dbkey: hg19 Format: tabular Galaxy Tool Version: 0.0.6 Tool Version: cuffdiff v2.1.1 (4046M) Tool Standard Output: stdout<https://usegalaxy.org/datasets/bbd44e69cb8906b581bffe7ea9e0cd97/stdout> Tool Standard Error: stderr<https://usegalaxy.org/datasets/bbd44e69cb8906b581bffe7ea9e0cd97/stderr> Tool Exit Code: 0 API ID: bbd44e69cb8906b581bffe7ea9e0cd97 Input Parameter Value Note for rerun Transcripts 684: Cuffmerge on data 559, data 618, and others: merged transcripts Name MBs Add replicate 261: MarkDups_Dupes Marked s101_ok.bam Add replicate 348: s102_MarkDups_Dupes Marked on 256.bam Add replicate 412: s103MarkDups_Dupes Marked 357.bam Add replicate 378: s104MarkDups_Dupes Marked 347.bam Add replicate 425: s105MarkDups_Dupes Marked.bam Name Ctrls Add replicate 701: MarkDups_Dupes Marked688.bam Add replicate 671: SRR112675MarkDups_Dupes Marked417.bam Add replicate 433: SRR112673MarkDups_Dupes Marked 343.bam Add replicate 427: SRR111937MarkDups_Dupes Marked 376.bam Add replicate 382: SRR112601MarkDups_Dupes Marked 316.bam Add replicate 384: SRR111936MarkDups_Dupes Marked 328.bam Library normalization method geometric Dispersion estimation method pooled False Discovery Rate 0.05 Min Alignment Count 5 Use multi-read correct Yes Perform Bias Correction Yes Reference sequence data cached Include Read Group Datasets Yes Set Additional Parameters? (not recommended for paired-end reads) No ...................................................................................................................................................... Johanna Sandgren, PhD Department of Oncology-Pathology CCK, Karolinska Institutet SE-171 76 Stockholm, Sweden +46-8-517 721 35 (office), +46-8- 321047(fax), +46-708 388476 (mobile)

2 2

unable to import run or save-to-file published workflow after galaxy upgrade
by McCulloch, Alan 11 Oct '13

11 Oct '13

dear all, we've just upgraded our Galaxy server (Galaxy revision 7148:17d57db9a7c0, upgraded to revision 10422:a886bc3ae924 ), and have found that an NGS training workflow that one of us set up and published is no longer accessible - we now get a stack trace when we try to run it (screenshot attached) ; if we try to save to file, we "get page not found" from the browser, but no errors reported in web logs, or Galaxy log as far as we can see Is there any way we can recover the workflow ? How to avoid this happening in future upgrades ? This time it is not a major problem, however we would not want this to happen to production workflows. Should we save-to-file on all workflows as a precaution before an upgrade ? (Would that help ? ) The workflow included steps as below. Grateful for any suggestions. Cheers Alan McC First Part : checking GC content *Upload the sampling.fasta file or get it from the shared data in Galaxy. *Use geecee from EMBOSS *Remove beginning (Text manipulation), first line *Convert as tabular (Text manipulation) or cut (Text manipulation) *Histogram (Graph/Display Data)on col2 *Compute data (Text manipulation) (data4 convert; col2>0.2) *Count data (statistics on col3) If it is worth it we can remove the sequences which have a too low GC content 3 Second part : sequence length *Compute Sequence length (FASTA manipulation) *Summary Statistics (Statistics)on col2 *Filter by length(FASTA manipulation) ; 800 *Line/world/character count(Text manipulation) *Blastn against fungi db : yeast, only one hit to show (advanced options ) *Count the lines to know how many sequences have a hit . (Text manipulation) 4 Bonus track : removing sequences with low GC content *Select (filter and sort) on data 6 where matching = True *FASTA-to-Tabular (FASTA manipulation) the input file sampling.fasta (2 col for the title) *Join two datasets(Join, Subtract and Group) on the columns c1 *Tabular-to-FASTA (fasta manipulation) of the previous results. *You can see how many sequences you took away (click on the dataset name in History) and it should correspond to the number of True in Count . 5

3 2

Tophat for Illumina - built-in dog genome mislabelled
by Court, Michael 10 Oct '13

10 Oct '13

Hi, I minor annoyance that I had found with the current implementation of "Tophat for Illumina (version 1.5.0)" in the public usegalaxy.org site: When you submit sequences for alignment, the dropdown list of available genomes gives 2 dog genome choices - BUT both are labeled the same - "Dog(Canis lupus familiaris): canFam2". After trial and error I found that the second (lower on the list) choice is actually "Dog(Canis lupus familiaris): canFam3.1". Hopefully this can be corrected at some point - maybe when the next release comes out from the Broad. Michael [cid:image002.png@01CEC5C0.75D35560] Michael H. Court, BVSc, PhD, Diplomate ACVA Professor and William R. Jones Endowed Chair Individualized Medicine Program Pharmacogenomics Laboratory Department of Veterinary Clinical Sciences College of Veterinary Medicine, Washington State University 100 Grimes Way, Pullman, WA 99164-6610 Office: 509-335-0817 Cell: 774-287-7082 Fax: 509-335-0880 michael.court(a)vetmed.wsu.edu<mailto:michael.court@vetmed.wsu.edu> www.vetmed.wsu.edu<http://www.vetmed.wsu.edu/>

2 1

Trouble viewing History from a particular laptop
by Olivia Choudhury 10 Oct '13

10 Oct '13

Hello, I am a graduate student in the University of Notre Dame. I have installed a personal instance of Galaxy and have been developing tools there as a part of my research. Everything worked well on my laptop until recently, when I could not view the 'History' panel, no matter which browser (Firefox, Chrome, Safari) I used on my laptop (Windows 7 OS). Everything works absolutely fine and I can view 'History' when I am using a desktop or any other laptop. I have checked this across different operating systems and also ensured that no changes were made with the Firewall setup of my laptop. I have attached a print screen of what the Galaxy page looks like. It would be great if I could get some help with this. Thanks a lot in advance, Olivia

2 1

Problems processing histories existing on galaxy before the changeover
by Elwood Linney 10 Oct '13

10 Oct '13

Is anyone who had developed histories on Galaxy online before the changeover getting red error messages after the changeover for tophat or cuffdiff processing? I have now tried a few times to move forward without luck and will remove at least one of my histories and start all over again just in case there is some kind of mismatch in the processing. Before I remove them all and start them all over, I would appreciate knowing if anyone else has identified where the block is. Elwood Linney

2 1