- galaxy-user - lists.galaxyproject.org

[Fwd: Re: error occurred when converting the SOLID output to fastq]
by anton＠bx.psu.edu 09 Jun '11

09 Jun '11

---------------------------- Original Message ---------------------------- Subject: Re: [galaxy-user] error occurred when converting the SOLID output to fastq From: "Jia-Xing Yue" <yjx(a)rice.edu> Date: Wed, June 8, 2011 5:48 pm To: "Anton Nekrutenko" <anton(a)bx.psu.edu> -------------------------------------------------------------------------- Dear Anton, Thank you very much! I re-uploaded the input file this Monday and galaxy now can work well with the data. I guess the corrupted input file is due to some problem when I uploaded them via FTP software at the first time. :) Best regards, Sincerely, Jia-Xing -- Jia-Xing Yue Graduate Student Putnam Lab Ecology & Evolutionary Bio. -MS 170 Rice University 6100 Main Street Houston TX 77005 Phone: 1-832-360-6228 E-mail: yjx(a)rice.edu Blog: http://bestrok.blogspot.com/ On Jun 8, 2011, at 3:40 PM, Anton Nekrutenko wrote: > Jia-Xing: > > Your input datasets are corrupted and result in this error. Specifically, you _F3 files are out of sync. One of the reads in csfasta file: > > #line number #file content > 21035992 >905_1278_862_F3 > 21035993 T21231001202131323300311210111233230100333003123120 > > appears twice in the corresponding QUAL file: > > #line number #file content > 21035992 >905_1278_862_F3 > 21035993 31 32 32 29 32 33 31 32 30 33 32 32 32 29 32 33 31 33 32 30 33 30 29 32 22 32 32 31 31 26 33 32 30 30 30 28 26 28 28 28 33 25 24 31 13 30 21 7 14 30 > -- > 21037350 >905_1278_862_F3 > 21037351 31 32 32 29 32 33 31 32 30 33 32 32 32 29 32 33 31 33 32 30 33 30 29 32 22 32 32 31 31 26 33 32 30 30 30 28 26 28 28 28 33 25 24 31 13 30 21 7 14 30 > > which should not happen. As a result this file cannot be easily converted into fastq without explicit identifier matching. Please, check that your input csfasta and QUAL files have exactly the same number of non-comment lines. > > Thanks for using Galaxy, > > anton > galaxy team > > PS Your _R3 files are fine. > > Anton Nekrutenko > http://nekrut.bx.psu.edu > http://usegalaxy.org > > > > On May 26, 2011, at 10:50 AM, Jia-Xing Yue wrote: > >> Hi, when I converted the solid output to fastq, the galaxy always complained the following error. Does anybody know why>> >> ------------- >> An error occurred running this job:Traceback (most recent call last): >> File "/galaxy/home/g2main/galaxy_main/tools/next_gen_conversion/solid2fastq.py", line 207, in <module> >> main() >> File "/galaxy/home/g2main/galaxy_main/tools/next_gen_conversion/solid2fastq.py", line 188, in main >> >> >> Traceback (most recent call last): >> File "/galaxy/home/g2main/galaxy_main/tools/next_gen_conversion/solid2fastq.py", line 207, in >> >> main() >> File "/galaxy/home/g2main/galaxy_main/tools/next_gen_conversion/solid2fastq.py", line 188, in main >> merge_reads_qual( fr, fq, con, trim_name=options.trim_name, out='db', double_encode=options.de, trim_first_base=options.trim_first_base, min_qual=options.min_qual, table_name="f3" ) >> File "/galaxy/home/g2main/galaxy_main/tools/next_gen_conversion/solid2fastq.py", line 83, in merge_reads_qual >> cursor.execute('insert into %s values("%s","%s","%s")' % (table_name, defline, lines[0], qual ) ) >> sqlite3.OperationalError: near "?3": syntax error >> >> ----------- >> Thanks! >> >> Jia-Xing >> >> -- >> Jia-Xing Yue >> >> Graduate Student >> Ecology & Evolutionary Bio. -MS 170 >> Rice University >> 6100 Main Street >> Houston TX 77005 >> >> Phone: 1-832-360-6228 >> E-mail: yjx(a)rice.edu >> Blog: http://bestrok.blogspot.com/ >> >> >> >> >> ___________________________________________________________ >> The Galaxy User list should be used for the discussion of >> Galaxy analysis and other features on the public server >> at usegalaxy.org. Please keep all replies on the list by >> using "reply all" in your mail client. For discussion of >> local Galaxy instances and the Galaxy source code, please >> use the Galaxy Development list: >> >> http://lists.bx.psu.edu/listinfo/galaxy-dev >> >> To manage your subscriptions to this and other Galaxy lists, >> please use the interface at: >> >> http://lists.bx.psu.edu/ > >

1 0

Re: [galaxy-user] UCSC->EMBOSS/fuzznuc->UCSC workflow?
by Jennifer Jackson 07 Jun '11

07 Jun '11

Hello Curtis, The BED extraction data can be resolved in Galaxy. Pull out the whole gene and then modify the coordinates in Galaxy to be 10k upstream. To be clear - this coordinate data is going to be used to transform the coordinates in your current fuzznuc output that is transcript-based to be genome-based. The coordinates are not input for fuzznuc - the are used after fuzznuc is run on the fasta file, in order to covert the result coordinates only. This page in the UCSC wiki has a good description of how the UCSC coordinates are organized. http://genomewiki.ucsc.edu/index.php/Coordinate_Transforms The output format for fuzznuc is documented in the tool's help - the last line on the tool form has a link. Hopefully this helps to clear up the suggested processing, Thanks, Jen Galaxy team On 5/17/11 2:08 PM, Robert Curtis Hendrickson wrote: > Jennifer, > > I tried getting data from UCSC as .BED - two issues: > > 1. Unlike "get sequence", I can no longer specify how far upstream I > want - it's EITHER "whole gene" (what's the definition of that!!!) OR > #bp_upstream OR exons OR introns -- with get seq those are not mutually > exclusive - I happen to want the genomic region (5'UTR, exons, introns > 3'UTR + 10kbp upstream of 5'UTR) > > 2. fuzznuc does not recognize BED as a valid input format. So, I can't > run fuzznuc because my BED file doesn’t' show up in the pulldown. > > Indeed, BED files are just annotation, they don’t carry any sequence. > > Have I mis-understood your directions? > > Regards, > > Curtis > > -----Original Message----- > From: Jennifer Jackson [mailto:jen@bx.psu.edu] > Sent: Tuesday, May 17, 2011 11:23 AM > To: Robert Curtis Hendrickson > Cc: 'galaxy-user(a)lists.bx.psu.edu' > Subject: Re: [galaxy-user] UCSC->EMBOSS/fuzznuc->UCSC workflow? > > Hello Curtis, > > No need to use the fasta headers from your original fasta file. > > To obtain the coordinates in BED format: using "Get Data -> UCSC main" > > again to link to the UCSC Table browser, set the same selection criteria > > as for the original fasta sequence, only change the output type to be > > "BED" (instead of "sequence"). Once in your Galaxy history, this format > > will be easier to work with. > > Best, > > Jen > > Galaxy team > > On 5/16/11 9:04 PM, Robert Curtis Hendrickson wrote: > > > Jennifer, > > > > > > Thanks for the outline. I'll try that approach. > > > > > > However, it seems rather painful to have to join the fuzznuc output > back to the original fasta to get at the header information that really > should have been passed along. It would see that there must be a way to > get the data out of UCSC without that space in the fasta header, so that > the chromosome& genomic location get correctly preserved in the fuzznuc > output. Failing that, is there an easy text manipulation that would > convert that fasta header space to a "|"? > > > > > > Regards, > > > Curtis > > > > > > > > > -----Original Message----- > > > From: Jennifer Jackson [mailto:jen@bx.psu.edu] > > > Sent: Monday, May 16, 2011 6:50 PM > > > To: Robert Curtis Hendrickson > > > Cc: 'galaxy-user(a)lists.bx.psu.edu' > > > Subject: Re: [galaxy-user] UCSC->EMBOSS/fuzznuc->UCSC workflow? > > > > > > Hello Curtis, > > > > > > The coordinates of your match are with respect to the fasta sequence, > > > not with respect to the reference genome. Only data mapped to the > > > reference genome can be viewed in the UCSC Browser > > > > > > You will need to calculate from the position of the match in the fasta > > > sequence back through to the reference genome. > > > > > > One suggested way to do this: > > > > > > a) Merge together the original genomic coordinates of the 2kb regions > > > with each line of output from fuzznuc. Use the original source fasta > > > sequence name as the common key for the merge. If both data are in BED > > > format, that would be ideal and make the following steps possible. You > > > may need to split the file based on whether the original fasta sequence > > > came from the positive or negative strand to run "b" and "c" below > > > separately. > > > b) Use "Text Manipulation -> Compute an expression on every row" to > > > create new coordinates. For example, if your data is on the positive > > > strand, and base 1 in your fasta file was genomic coordinate 100, and > > > the alignment from fuzznuc started at base 5 (local coordinate == "4" if > > > in BED format with a zero-based start), then the new genomic start > > > coordinate would be [100 + 4)] = 104. Do this for both start and stop. > > > c) Adjust the logic for "b" if any of your original fasta sequences are > > > from the negative strand, on the negative strand portion of your data > > > ("b" would be run on just the positive strand portion of your data). > > > d) arrange/cut the resulting file down into a standard BED format to > > > remove the local coordinates and keep the genomic coordinates, using the > > > original chromosome names. > > > e) once the logic for the calculations is worked out, save the process > > > into a workflow for use again. > > > > > > Hopefully this helps, > > > > > > Best, > > > > > > Jen > > > Galaxy team > > > > > > On 5/13/11 9:32 AM, Robert Curtis Hendrickson wrote: > > >> Folks, > > >> > > >> I wanted to scan the 2kb upstream of a list of human gene isoforms > for TFBS using fuzznuc. I was able to > > >> "Get Data"> "UCSC Main"> "As sequence" and get my sequences > > >> "EMBOSS"> fuzznuc ran fine, and output the hits > > >> > > >> HOWEVER, fuzznuc lost the genomic position information that UCSC has > put after a space in the sequence headers of the FASTA file. It only > provided offsets within the fasta. > > >> > > >> http://main.g2.bx.psu.edu/u/curtish-uab/h/ucsc-fuzznuc-ucsc-broken > > >> > > >> Thus, when I converted the fuzznuc output back to a BED file and > tried to visualize the hits in UCSC browser, it failed with "invalid BED > File". > > >> I tried fuzznuc with output: seqtable, feattable and gff3, but in > all cases the genomic position was missing, and being a bit of Galaxy > novice, I couldn't figure out how to get the output back to UCSC to > visualize the hits. > > >> > > >> Can anyone tell me how to link up these tools correctly, or share a > history with some other tool set that accomplishes this goal? > > >> > > >> Regards, > > >> Curtis > > >> > > >> Research Associate > > >> Center for Clinical and Translational Science > > >> University of Alabama at Birmingham > > >> > > >> ___________________________________________________________ > > >> The Galaxy User list should be used for the discussion of > > >> Galaxy analysis and other features on the public server > > >> at usegalaxy.org. Please keep all replies on the list by > > >> using "reply all" in your mail client. For discussion of > > >> local Galaxy instances and the Galaxy source code, please > > >> use the Galaxy Development list: > > >> > > >> http://lists.bx.psu.edu/listinfo/galaxy-dev > > >> > > >> To manage your subscriptions to this and other Galaxy lists, > > >> please use the interface at: > > >> > > >> http://lists.bx.psu.edu/ > > > > > -- > > Jennifer Jackson > > http://usegalaxy.org > > http://galaxyproject.org > -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org

2 3

Re: [galaxy-user] Loading files to local Galaxy
by Paul-Michael Agapow 07 Jun '11

07 Jun '11

> From: Tilahun Abebe <Tilahun.Abebe(a)uni.edu> > > We are trying to load Illumina data to our local Galaxy instance. The > files are between 700 MB and 2.2 GB. Files below 2 GB load in less than > 5 minutes. Files larger than 2 GB don't upload at all. We installed > Galaxy locally because we thought loading files will be faster than the > server version. Any suggestions to solve this problem is highly appreciated. While (from others experience) Galaxy _should_ be able to upload files that large, we've had some problems with our local installation too. Investigation didn't reveal any cause, so we put it down to the quality of our network. You might what to look at the webserver or proxy that you have in front of Galaxy - from memory, both Apache and nginx can be configured to impose file size limits, so that _may_ be the problem. In any event, you might want to configure your server to handle uploads and downloads directly as per <https://bitbucket.org/galaxy/galaxy-central/wiki/Config/ProductionServe r>. Finally you can pass an url to the upload dialog to get Galaxy to pull the file from an ftp server - for example - and that may prove more robust. ---- Paul Agapow (paul-michael.agapow(a)hpa.org.uk) Bioinformatics, Centre for Infections, Health Protection Agency ----------------------------------------- ************************************************************************** The information contained in the EMail and any attachments is confidential and intended solely and for the attention and use of the named addressee(s). It may not be disclosed to any other person without the express authority of the HPA, or the intended recipient, or both. If you are not the intended recipient, you must not disclose, copy, distribute or retain this message or any part of it. This footnote also confirms that this EMail has been swept for computer viruses, but please re-sweep any attachments before opening or saving. HTTP://www.HPA.org.uk **************************************************************************

3 3

Re: [galaxy-user] Most Popular Linux flavour for Galaxy
by Enis Afgan 07 Jun '11

07 Jun '11

Hi Ron, Yes, it's the 64-bit Ubuntu and we've opted to go with the 10.04 version because it's LTS version, although I don't think there were any changes in the more recent versions that will break things in the scripts. Enis On Tue, Jun 7, 2011 at 10:38 AM, Ronald Worthington <rworthi(a)siue.edu>wrote: > Hi Enis, > > Is that 64 bit Ubuntu, and why not Ubuntu 11.04 - are there stability > issues with later versions? I know Bio-linux uses 64 bit Ubuntu 10.04 as > well. > > - Ron > > On Jun 7, 2011, at 10:15 AM, Enis Afgan wrote: > > Hi Matloob, > Others should contribute their experiences as well but for the Galaxy VM > and cloud, we use Ubuntu 10.04. Also, to automate the machine setup, we have > a scripts that perform all of the required setup steps; the scripts are > usable by anyone for a range of machines and are available here: > https://bitbucket.org/afgane/mi-deployment/ > > Good luck, > Enis > > On Tue, Jun 7, 2011 at 8:00 AM, Matloob Khushi < > matloob.khushi(a)sydney.edu.au> wrote: > >> Hello Galaxy Users >> >> We are anticipating setting up a local instance of Galaxy and wondering >> what is the most popular (or best) flavour of Linux people out there using. >> >> I wonder which OS on UseGalaxy.org has been used. Thanks for your help. >> >> Regards >> Matloob >> PhD Candidate >> >> ___________________________________________________________ >> The Galaxy User list should be used for the discussion of >> Galaxy analysis and other features on the public server >> at usegalaxy.org. Please keep all replies on the list by >> using "reply all" in your mail client. For discussion of >> local Galaxy instances and the Galaxy source code, please >> use the Galaxy Development list: >> >> http://lists.bx.psu.edu/listinfo/galaxy-dev >> >> To manage your subscriptions to this and other Galaxy lists, >> please use the interface at: >> >> http://lists.bx.psu.edu/ >> > > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/ > > >

1 0

TSS list with annotation for Hg19
by shamsher jagat 03 Jun '11

03 Jun '11

I want to extract TSS ids with correspodning location for Hg19. Once I extract such TSS coordinates I want to have there annotation either with enetrz gene id or some other unique Ids. I am sure Galaxy can do this, But I dont know where to start, or which workflow to use. Can some one please dircet me to right dircetion. Appreciate your help.

2 2

Flagstat on BAM files
by Slim Sassi 03 Jun '11

03 Jun '11

Hello, I tried to use NGS: SAM Tools ->flagstat on a BAM files for basic stats, but I got results like you see below. It doesn't seem to be working. Any suggestions? 26584869 in total 0 QC failure 0 duplicates 26584869 mapped (100.00%) 0 paired in sequencing 0 read1 0 read2 0 properly paired (-nan%) 0 with itself and mate mapped 0 singletons (-nan%) 0 with mate mapped to a different chr 0 with mate mapped to a different chr (mapQ>=5) Thanks Slim The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

6 11

biomart error
by Christophe Klopp 31 May '11

31 May '11

Hi, I've tried a simple query in biomart : retrieve all gene locations for the Danio rerio in Ensembl 62 and I get an error : An error occurred running this job: /failure preparing job / The same happens with the test site? Cheers Christophe/ / -- ==================================================================== = Christophe KLOPP BIA INRA Toulouse 31326 Castanet-Tolosan = = Tel: 33 5 61 28 50 36 Email: christophe.klopp(a)toulouse.inra.fr = = http://www.sigenae.org/ = = http://bioinfo.genotoul.fr/ = ====================================================================

5 10

Mycoplasma pneumoniae M129 and FH reference genome
by David K Crossman 31 May '11

31 May '11

Hello! I noticed that Mycoplasma pneumonia M129 and FH are not found in the reference genome in Galaxy. Would it be possible to have both of those in there? Thanks, David

2 2

Concatenate fasta tool problem
by Paul McAdam 31 May '11

31 May '11

Hi, I'm trying to use the concatenate fasta tool at http://main.g2.bx.psu.edu/root?tool_id=fasta_concatenate0 to concatenate an xmfa file output from mauve (headers formatted to following standard: >1:517-24215(+)|0402981.fa but am receiving the following error. Traceback (most recent call last): File "/galaxy/home/g2main/galaxy_main/tools/fasta_tools/fasta_concatenate_by_species.py", line 40, in if __name__ == "__main__" : __main__() File "/galaxy/home/g2main/galaxy_main/tools/fasta_tools/fasta_concatenate_by_species.py", line 23, in __main__ species[component.species] = tempfile.TemporaryFile() File "/usr/lib/python2.6/tempfile.py", line 475, in TemporaryFile (fd, name) = _mkstemp_inner(dir, prefix, suffix, flags) File "/usr/lib/python2.6/tempfile.py", line 228, in _mkstemp_inner fd = _os.open(file, flags, 0600) OSError: [Errno 24] Too many open files: '/space/g2main/tmpotY_hX' Does anyone have any ideas what the issue may be? Regards Paul -- Laboratory for Bacterial Evolution and Pathogenesis Centre for Infectious Diseases and The Roslin Institute University of Edinburgh Easter Bush Campus Midlothian EH25 9RG Tel: +44 (0)131 6519231 email: p.mcadam(a)sms.ed.ac.uk

2 1

Re: [galaxy-user] (no subject)
by Bossers, Alex 24 May '11

24 May '11

Aaron, Please do check for contaminants.... our experience with service providers and QC....I can write a book probably :(. The FastQC suite is a good place to start (also a galaxy wrapper is available for that). Even for 454 (not having fixed base positions and fixed lengths) it's quite informative (kmer overrepresentation and such). In addition...check for contaminating sequences (ie Coli or Mycoplasma sequences not expected when sequencing human cells.... but you better check ....experience). In the MIRA documentation you will find some info on this prior to assembly filtering as well if I remember correctly. Please keep us posted on your progress. @Peter; hope you manage to take a flight and join the conference. A pity I won't be there but it looks very promising... Alex Van: Aaron Jex [mailto:ajex@unimelb.edu.au] Verzonden: dinsdag 24 mei 2011 8:52 Aan: Bossers, Alex Onderwerp: RE: [galaxy-user] (no subject) Hi Alex, Thanks for the email. I will have to have a closer read of the MIRA documentation I think. I know that it definitely makes use of the quality data to some extent, but I hadn't considered whether it ignores low quality data or not (perhaps there's a threshold setting I could use - I'll check that). I'm not too worried about adaptor sequence at the moment as these "should" be trimmed by our sequencing service, and I clip the ends on the reads when I extract the qual and fasta files from the original sff files anyways. Best regards, Aaron Aaron Jex, BSc, PhD Senior Research Officer, Department of Veterinary Science, The University of Melbourne, 250 Princes Highway, Werribee, Victoria, 3030 tel: +61 3 9731 2294 From: Bossers, Alex [mailto:Alex.Bossers@wur.nl] Sent: Tuesday, 24 May 2011 4:44 PM To: Aaron Jex; galaxy-user(a)bx.psu.edu Subject: RE: [galaxy-user] (no subject) Aaron, As far as I remember MIRA....isn't MIRA taking into account the low/high quality bases anyway? So no need to filter there right? Only filtering needed is for contaminating sequences.....(incl adapters and such). You can/have to check the MIRA website to be sure though. The high qual segments I have used as in the metagenomics example but indeed you loose the exact qual info....but that is already above the provided threshold (default above 20 in Sanger quality score range). Alex Van: galaxy-user-bounces(a)lists.bx.psu.edu [mailto:galaxy-user-bounces@lists.bx.psu.edu] Namens Aaron Jex Verzonden: dinsdag 24 mei 2011 1:40 Aan: galaxy-user(a)bx.psu.edu Onderwerp: [galaxy-user] (no subject) Hi, Can't seem to find an answer to this on your wiki site and it's not in the tutorial. I would like to filter my 454 reads for high quality regions, rename the resulting sequence fragments AND relink the new reads (fragments) to the original quality data so that I can take these filtered reads and assembly them using MIRA. Is there a way to do this with Galaxy? So basically all I want to do is take the new read fragments I get from converting the tabular file to the fasta file as shown in your metagenomics tutorial, and generate a corresponding qual file for these 'new' reads. Best regards, Aaron Aaron Jex, BSc, PhD Senior Research Officer, Department of Veterinary Science, The University of Melbourne, 250 Princes Highway, Werribee, Victoria, 3030 tel: +61 3 9731 2294

1 0