- galaxy-user - lists.galaxyproject.org

Re: [galaxy-user] your video of using cloudman
by Anton Nekrutenko 24 Sep '13

24 Sep '13

Elwood: With large fastq files your best option will be either using public server (which is having difficulties today) or running a local instance of Galaxy. We are very much aware that the public site is not behaving well. It will be back on-line today and in the coming weeks we will be switching underlying infrastructure brining much needed relief to all Galaxy users at once. The best venue for posting such questions is out galaxy-user mailing list (I CC'ed it here). Thank you and sorry for the main site troubles. anton On Tue, Sep 24, 2013 at 11:29 AM, Elwood Linney <ellinney(a)gmail.com> wrote: > Hello, > Because of the problems with Galaxy online, I am looking into using Galaxy > on the cloud but even with all the information it sometimes is confusing > for individuals who wish to use existing Galaxy in a specific manner. > > In your video of transferring datasets to the cloud you used URLs for > relatively small files, but under normal circumstances with Galaxy online, > this does not work for large files. I generally work with fastq files of > about 16gb in size. Would I be able to transfer this size of file via URL > to the cloud or would I have to use some other means (for galaxy online I > use fetch)? > > Rather that bother you with this question, is there a specific address > that I should send this type of question to (like the user list--though it > seems like there is so much disfunctional with Galaxy online that it takes > a few weeks to get a repy from that)? > > Elwood Linney > Professor of Molecular Genetics and Microbiology > Duke University Medical Center > -- Anton Nekrutenko Associate Professor Dept. of Biochemistry and Molecular Biology www.galaxyproject.org (814) 826-3051

1 0

Secure file upload gateway for Galaxy
by Georgios Magklaras 24 Sep '13

24 Sep '13

Hi, Our team at the University of Oslo is building a Life Science Portal based on Galaxy. We operate several standalone instances and we have the necessary sysadmin experience, but we really need to implement a more secure file upload mechanism than FTP (we do not like to send cleartext password credentials in the open ), and we understand that Galaxy does not integrate an upload method other than FTP with reference to this screencast: http://screencast.g2.bx.psu.edu/quickie_17_ftp_upload/flow.html One possible solution for this is to setup an SFTP upload server with a huge scratch space, that runs the SFTP upload gateway on one end and an IP restricted FTP server on the other, so that users can then upload/index the SFTP uploaded data into their Galaxy session via the URL upload field. This two step process might be a bit cumbersome for some of our users and we are looking for ways to simplify it. Do you have best recipes for SFTP/Aspera upload gateway integration to Galaxy? We would welcome advise on that matter. GM Best regards, -- -- George Magklaras PhD RHCE no: 805008309135525 Head of IT/Senior Systems Engineer Biotechnology Center of Oslo and the Norwegian Center for Molecular Medicine/ Vitenskapelig Databehandling (VD) - Research Computing Services - USIT EMBnet TMPC Chair http://folk.uio.no/georgios http://www.uio.no/english/services/it/research/hpc/abel/ Tel: +47 22840535

3 2

non-coding RNA annotation
by Hoang, Thanh 23 Sep '13

23 Sep '13

Hi all, I am analyzing my small RNA sequencing data on mouse tissue. Does anyone know where to download annotation file for non-coding RNA? Thanks Thanh

2 1

Galaxy Main - Files not Found, Server Error
by Yardley, Nathan 23 Sep '13

23 Sep '13

Hello, I am getting a resource not found error when I try to view data on the Galaxy Main Server. For some of my FASTQ Groomed files I get: The resource could not be found. File Not Found (/galaxy/main_pool/pool7/files/006/764/dataset_6764692.dat). [[Note the number before .dat changes depending on the file requested for viewing]] For some of my Map with Bowtie for Illumina files I get: Internal Server Error Galaxy was unable to sucessfully complete your request An error occurred. This may be an intermittent problem due to load or other unpredictable factors, reloading the page may address the problem. The error has been logged to our team. If you want to contact us about this error, please reference the following GURU MEDITATION: #dc57d31266564204baf90d698880401a I also have two mapping processes that have not been processed since Friday afternoon (It is Monday Morning as of this post). Any help on this matter would be great Thank you very much for your help. Nathan

2 1

History will not load?
by Maria Hoffman 23 Sep '13

23 Sep '13

Hello, I have been having trouble with Galaxy since yesterday. I have been trying to run cuffdiff which has been waiting to run since Friday afternoon and now my history will not even load (red error message shows up). Any incite would be much appreciated. Maria

2 1

RNA-DNA converter in Fasta format
by Hoang, Thanh 22 Sep '13

22 Sep '13

Hi all, I want to to map my sequencing reads to miRNA reference database . Anyone know how to convert RNA to DNA in FASTA format ( U to T) . Thanks Thanh

2 2

Re: [galaxy-user] Metagenomic filtering
by Scott Tighe 20 Sep '13

20 Sep '13

Gerald 16s is basically useless for identification to genus. Since I started sequencing 16s in 1992, I have come to realize that without sequencing the full 1540 bases, it is generally misleading, and even than, it is not accurate enough to nail genus on more than 1/2 the cases. However, what is your feeling on ITS and gyrase, They seem to be far more discriminating but those databases have been decommissioned sometime ago. The desirable thing would be that Galaxy or NCBI add a "filter conserved genes" [ ie any hit with a second choice greater than 3% distance]. Something such as that. If you (or others) are aware of such a thing, I'd love the here about it. Sincerely Scott Scott Tighe Senior Core Laboratory Research Staff Advanced Genome Technologies Core University of Vermont Vermont Cancer Center 149 Beaumont ave Health Science Research Facility 303/305 Burlington Vermont 05405 802-656-2557 On 9/18/2013 2:05 PM, Gerald Bothe wrote: > Removing model organisms may not be enough, you may have the same > problem with, say, a Clostridium cluster IV anaerobe. I think a > solution would be to > first: compare to a collection of genes, e.g. get all the hits for 16S > rRNA genes, RNA polymerases (conserved to quite conserved), and to > e.g. ion channels and cell surface proteins. > then: once a read or contig is identified as belonging to a gene > family, gene, or protein domain, check within that group for species > identities. Then you compare apples to apples in terms of gene > conservation level > Does anybody know a program that would do this efficiently from > metagenomic data? > Gerald Bothe > > *From:* Scott W. Tighe <Scott.Tighe(a)uvm.edu> > *To:* galaxy-user(a)lists.bx.psu.edu > *Sent:* Wednesday, September 18, 2013 10:03 AM > *Subject:* Re: [galaxy-user] Metagenomic filtering > > Dear Galaxy > > When running HiSeq shot metagenomics sample from the environment > against megablast and taxonomic representation, How do I > filter/remove all the 16s and other conserved sequences. > > The problem if blasting a single organism that has a fraction of > conserved sequence, the results will align with E.coli 10,000 > times more then the possible target organism. This data would be > wrong and misleading. For example a 100mg sample that was negative > for e coli using MUG test, give thousands of hits with galaxy. > > 1) Is there a "filter conserved sequences" setting? > > > > 2) Is there a "remove model organisms" setting? > > > Scott Tighe > --Core Laboratory Research Staff > Advanced Genome Technologies Core > Deep Sequencing (MPS) Facility > Vermont Cancer Center > 149 Beaumont Ave > University of Vermont HSRF 303 > Burlington Vermont USA 05045 > 802-656-AGTC > 802-999-6666 (cell) > > > > Quoting Jennifer Jackson <jen(a)bx.psu.edu <mailto:jen@bx.psu.edu>>: > > > Hello Elwood, > > > > Are you still having connection issues today? Or is this resolved? > > > > Best, > > > > Jen > > Galaxy team > > > > On 9/13/13 11:36 AM, Elwood Linney wrote: > >> A message sent earlier this week by me indicated that I could > not connect to Galaxy via Fetch to download data. > >> > >> A reply indicated a glitch was fixed. > >> > >> I then could connect with Fetch and I tried to transfer 4 x > 16gb files and the connection disconnected about 4 times. > >> > >> Now, once again, I cannot connect with Galaxy online to > transfer data. > >> > >> Is this a problem that can be solved-either at my end or at Galaxy? > >> > >> Elwood Linney > >> > >> > >> ___________________________________________________________ > >> The Galaxy User list should be used for the discussion of > >> Galaxy analysis and other features on the public server > >> at usegalaxy.org <http://usegalaxy.org/>. Please keep all > replies on the list by > >> using "reply all" in your mail client. For discussion of > >> local Galaxy instances and the Galaxy source code, please > >> use the Galaxy Development list: > >> > >> http://lists.bx.psu.edu/listinfo/galaxy-dev > >> > >> To manage your subscriptions to this and other Galaxy lists, > >> please use the interface at: > >> > >> http://lists.bx.psu.edu/ > >> > >> To search Galaxy mailing lists use the unified search at: > >> > >> http://galaxyproject.org/search/mailinglists/ > > > > --Jennifer Hillman-Jackson > > http://galaxyproject.org > > > > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > > http://galaxyproject.org/search/mailinglists/ > >

2 1

Re: [galaxy-user] 3' adapter trimming using FASTX-toolkit clipper
by Jennifer Jackson 20 Sep '13

20 Sep '13

Hi Thanh, Questions are fine, that is what this mailing list is for. But please do try to cc the mailing list and start a new thread for new topics when possible. To generate a length distribution (among other stats), the tool "NGS: QC and manipulation -> FastQC" is a quick method. Take care, Jen Galaxy team On 9/20/13 9:11 AM, Hoang, Thanh wrote: > Hi, > Thank you very much Jenny. > You are right. The FASTX-toolkit does some tolerances. > Now I got the output file after clipping full-length 3' adapter > sequence. Is there any tool in Galaxy where I can draw some kind of > graph or statistics showing distribution of the length versus number > of the reads in the output file? > Sorry for asking many basic questions > Thanh > > > > On Thu, Sep 19, 2013 at 9:31 PM, Jennifer Jackson <jen(a)bx.psu.edu > <mailto:jen@bx.psu.edu>> wrote: > > Thanh, > > To hopefully be clearer, the part matched is clipped (whole or > partial, and there is even some tolerance for low-frequency > mismatches). > > I would suggest taking a few sequences out and running the tool on > them to try it out. You could test for both length and mismatch > constraints this way. (Perhaps even using constructed sequences > that are modified to have specific adapter lengths and/or mismatch > counts). This is a great way to get a feel for new tools in general. > > If you need more details about exactly how the algorithm works, > you can read the original documentation and then if you still need > help, try contacting the tool author (links at bottom of tool > form). But this is a very popular, commonly used tool and what I > have shared is how it is behaves to my knowledge & experience. > There may not be much more to it. > > Best, > > Jen > Galaxy Team > > > On Sep 19, 2013, at 5:57 PM, "Hoang, Thanh" <hoangtv(a)miamioh.edu > <mailto:hoangtv@miamioh.edu>> wrote: > >> Hi Jenny, >> Thank you. >> When you put the whole 3' adapter sequence into the Clipper, what >> will happen to the reads that only contains part of the adapter? >> Are they considered as not containing the adapter and >> subsequently non-clipped reads? >> Thanh >> >> >> On Thu, Sep 19, 2013 at 8:46 PM, Jennifer Jackson <jen(a)bx.psu.edu >> <mailto:jen@bx.psu.edu>> wrote: >> >> Hi Thanh, >> >> Just enter the whole adapter sequence. The tool will match >> what is found in the input sequence and clip. The help >> graphic on the Clip form itself illustrates this - only one >> adapter is entered (can be entered) but a variable length is >> clipped from the input to produce the output. >> >> Thanks for posting this new question to the mailing list. >> This greatly helps us to track & provide the speediest replies. >> >> Best, >> >> Jen >> Galaxy team >> >> >> On 9/19/13 4:15 PM, Hoang, Thanh wrote: >>> Hi all, >>> I am analyzing miRNA sequencing now. My data is 51bp, single >>> -ended and ~5 M reads. I want to remove the adapter >>> sequences from the reads before mapping to the genomes/known >>> miRNA database. >>> My 3' adapter sequence is : 5-AGATCGGAAGAGCACACGTCT-3. I >>> found that many reads only contain part of the 3' adapter >>> sequence. I am using FASTX-toolkit to clip it off. How many >>> bases should I put in the " Enter custom clipping sequence" >>> ? Because in the output files, I end up with more reads when >>> putting the whole 3 adapter sequence than putting only first >>> 8 nt. >>> Also, miRNA is about 17-25 nt long, I guess that the rest of >>> the reads (51-21=30bp) must contain part or whole 5's >>> adapter sequence or the by-product of mRNA/tRNA degradation. >>> So I think that I have to trim the 5' adapter as well. >>> Any suggestion will be highly appreciated >>> Thanh >>> >>> >>> >>> ___________________________________________________________ >>> The Galaxy User list should be used for the discussion of >>> Galaxy analysis and other features on the public server >>> atusegalaxy.org <http://usegalaxy.org>. Please keep all replies on the list by >>> using "reply all" in your mail client. For discussion of >>> local Galaxy instances and the Galaxy source code, please >>> use the Galaxy Development list: >>> >>> http://lists.bx.psu.edu/listinfo/galaxy-dev >>> >>> To manage your subscriptions to this and other Galaxy lists, >>> please use the interface at: >>> >>> http://lists.bx.psu.edu/ >>> >>> To search Galaxy mailing lists use the unified search at: >>> >>> http://galaxyproject.org/search/mailinglists/ >> -- Jennifer Hillman-Jackson http://galaxyproject.org

1 0

Re: [galaxy-user] 3' adapter trimming using FASTX-toolkit clipper
by Jennifer Jackson 19 Sep '13

19 Sep '13

Thanh, To hopefully be clearer, the part matched is clipped (whole or partial, and there is even some tolerance for low-frequency mismatches). I would suggest taking a few sequences out and running the tool on them to try it out. You could test for both length and mismatch constraints this way. (Perhaps even using constructed sequences that are modified to have specific adapter lengths and/or mismatch counts). This is a great way to get a feel for new tools in general. If you need more details about exactly how the algorithm works, you can read the original documentation and then if you still need help, try contacting the tool author (links at bottom of tool form). But this is a very popular, commonly used tool and what I have shared is how it is behaves to my knowledge & experience. There may not be much more to it. Best, Jen Galaxy Team On Sep 19, 2013, at 5:57 PM, "Hoang, Thanh" <hoangtv(a)miamioh.edu> wrote: > Hi Jenny, > Thank you. > When you put the whole 3' adapter sequence into the Clipper, what will happen to the reads that only contains part of the adapter? Are they considered as not containing the adapter and subsequently non-clipped reads? > Thanh > > > On Thu, Sep 19, 2013 at 8:46 PM, Jennifer Jackson <jen(a)bx.psu.edu> wrote: >> Hi Thanh, >> >> Just enter the whole adapter sequence. The tool will match what is found in the input sequence and clip. The help graphic on the Clip form itself illustrates this - only one adapter is entered (can be entered) but a variable length is clipped from the input to produce the output. >> >> Thanks for posting this new question to the mailing list. This greatly helps us to track & provide the speediest replies. >> >> Best, >> >> Jen >> Galaxy team >> >> >> On 9/19/13 4:15 PM, Hoang, Thanh wrote: >>> Hi all, >>> I am analyzing miRNA sequencing now. My data is 51bp, single -ended and ~5 M reads. I want to remove the adapter sequences from the reads before mapping to the genomes/known miRNA database. >>> My 3' adapter sequence is : 5-AGATCGGAAGAGCACACGTCT-3. I found that many reads only contain part of the 3' adapter sequence. I am using FASTX-toolkit to clip it off. How many bases should I put in the " Enter custom clipping sequence" ? Because in the output files, I end up with more reads when putting the whole 3 adapter sequence than putting only first 8 nt. >>> Also, miRNA is about 17-25 nt long, I guess that the rest of the reads (51-21=30bp) must contain part or whole 5's adapter sequence or the by-product of mRNA/tRNA degradation. So I think that I have to trim the 5' adapter as well. >>> Any suggestion will be highly appreciated >>> Thanh >>> >>> >>> >>> ___________________________________________________________ >>> The Galaxy User list should be used for the discussion of >>> Galaxy analysis and other features on the public server >>> at usegalaxy.org. Please keep all replies on the list by >>> using "reply all" in your mail client. For discussion of >>> local Galaxy instances and the Galaxy source code, please >>> use the Galaxy Development list: >>> >>> http://lists.bx.psu.edu/listinfo/galaxy-dev >>> >>> To manage your subscriptions to this and other Galaxy lists, >>> please use the interface at: >>> >>> http://lists.bx.psu.edu/ >>> >>> To search Galaxy mailing lists use the unified search at: >>> >>> http://galaxyproject.org/search/mailinglists/ >> >> -- >> Jennifer Hillman-Jackson >> http://galaxyproject.org >

1 0

3' adapter trimming using FASTX-toolkit clipper
by Hoang, Thanh 19 Sep '13

19 Sep '13

Hi all, I am analyzing miRNA sequencing now. My data is 51bp, single -ended and ~5 M reads. I want to remove the adapter sequences from the reads before mapping to the genomes/known miRNA database. My 3' adapter sequence is : 5-AGATCGGAAGAGCACACGTCT-3. I found that many reads only contain part of the 3' adapter sequence. I am using FASTX-toolkit to clip it off. How many bases should I put in the " Enter custom clipping sequence" ? Because in the output files, I end up with more reads when putting the whole 3 adapter sequence than putting only first 8 nt. Also, miRNA is about 17-25 nt long, I guess that the rest of the reads (51-21=30bp) must contain part or whole 5's adapter sequence or the by-product of mRNA/tRNA degradation. So I think that I have to trim the 5' adapter as well. Any suggestion will be highly appreciated Thanh

2 1