June 2011 - galaxy-user - lists.galaxyproject.org

FastQ Groomer and Compute Quality Statistics
by John David Osborne 09 Jun '11

09 Jun '11

I noticed that for our new Ilumina data (which generate Sanger format) the FastQ groomer output is identical to the Ilumina FastQ input file. I was hoping to go ahead and just use the raw FastQ files as input (saving disk space) for computing quality statistics to look at box plots, but it appears that the tool "Compute Quality Statistics" appears to require that the data have been run through FastQ Groomer first. Is there a way to get around this and is this a bug? I assuming this is some sort of safety measure built into this tool? -John

4 5

LASTZ arguments in "Full parameter list"-mode
by Appelt, Uwe 09 Jun '11

09 Jun '11

Hi @All, short version: Is it possible to specify more arguments in "Full parameter list"-mode, particularly those that're implicitely described in YASRA-mode (match/mismatch rewards and step-size (Z/step))? long version: I'm dealing with 454 reads and it's crucial to the scenario to have the 5' ends aligned properly (in terms of sensitivity), so all the YASRA-templates comprising heavy gap penalties perform fairly poor as soon as there's a gap nearby the 5' end. However, heading to "Full parameter list"-mode leads to excessive CPU-use, because (presumably) the step-size (default=1?) is left unset. Also, it'd be great to be able to alter not only gap penalties, but match/mismatch rewards as well. (Obviously a mismatch would cause the same problem). Examples: AAAAAAAAAA (target) AAAAATAAAA (query) => I need the alignment boundary to be found at position 1, rather than behind the mismatched T and this cleary doesn't work as long as the mismatch penalty is too large (same applies for gap-open/extend penalties). Thanks in advance and Cheers, Uwe

7 7

Galaxy tool error report from johnwu@ibms.sinica.edu.tw
by "mailto:galaxy-user"＠lists.bx.psu.edu 09 Jun '11

09 Jun '11

GALAXY TOOL ERROR REPORT ------------------------ This error report was sent from the Galaxy instance hosted on the server "192.168.0.36:8080" ----------------------------------------------------------------------------- This is in reference to dataset id 316 from history id 2 ----------------------------------------------------------------------------- You should be able to view the history containing the related history item 121: Cufflinks on data 92: gene expression by logging in as a Galaxy admin user to the Galaxy instance referenced above and pointing your browser to the following link. 192.168.0.36:8080/history/view?id=f597429621d6eb2b ----------------------------------------------------------------------------- The user 'johnwu(a)ibms.sinica.edu.tw' provided the following information: Hi galaxy dev team: I just installed galaxy on my ubuntu box and tried to do RNA-seq analysis according to Jeremy's excellent tutorial http://main.g2.bx.psu.edu/u/jeremy/p/galaxy-rna-seq-analysis-exercise however, I encountered following error messages when I was trying to execute Cufflinks in galaxy.. ----------------------------------------------------------------------------- job id: 106 tool id: cufflinks ----------------------------------------------------------------------------- job command line: python /home/galaxy/software/galaxy_dist/tools/ngs_rna/cufflinks_wrapper.py --input=/home/galaxy/software/galaxy_dist/database/files/000/dataset_287.dat --assembled-isoforms-output=/home/galaxy/software/galaxy_dist/database/files/000/dataset_318.dat --num-threads="4" -I 300000 -F 0.05 -j 0.05 -b --ref_file="None" --dbkey=hg19 --index_dir=/home/galaxy/software/galaxy_dist/tool-data ----------------------------------------------------------------------------- job stderr: Error running cufflinks. [Errno 2] No such file or directory: 'transcripts.gtf' ----------------------------------------------------------------------------- job stdout: cufflinks v1.0.3 cufflinks -q --no-update-check -I 300000 -F 0.050000 -j 0.050000 -p 4 -b ----------------------------------------------------------------------------- job info: None ----------------------------------------------------------------------------- job traceback: None ----------------------------------------------------------------------------- (This is an automated message).

1 0

[Fwd: Re: error occurred when converting the SOLID output to fastq]
by anton＠bx.psu.edu 09 Jun '11

09 Jun '11

---------------------------- Original Message ---------------------------- Subject: Re: [galaxy-user] error occurred when converting the SOLID output to fastq From: "Jia-Xing Yue" <yjx(a)rice.edu> Date: Wed, June 8, 2011 5:48 pm To: "Anton Nekrutenko" <anton(a)bx.psu.edu> -------------------------------------------------------------------------- Dear Anton, Thank you very much! I re-uploaded the input file this Monday and galaxy now can work well with the data. I guess the corrupted input file is due to some problem when I uploaded them via FTP software at the first time. :) Best regards, Sincerely, Jia-Xing -- Jia-Xing Yue Graduate Student Putnam Lab Ecology & Evolutionary Bio. -MS 170 Rice University 6100 Main Street Houston TX 77005 Phone: 1-832-360-6228 E-mail: yjx(a)rice.edu Blog: http://bestrok.blogspot.com/ On Jun 8, 2011, at 3:40 PM, Anton Nekrutenko wrote: > Jia-Xing: > > Your input datasets are corrupted and result in this error. Specifically, you _F3 files are out of sync. One of the reads in csfasta file: > > #line number #file content > 21035992 >905_1278_862_F3 > 21035993 T21231001202131323300311210111233230100333003123120 > > appears twice in the corresponding QUAL file: > > #line number #file content > 21035992 >905_1278_862_F3 > 21035993 31 32 32 29 32 33 31 32 30 33 32 32 32 29 32 33 31 33 32 30 33 30 29 32 22 32 32 31 31 26 33 32 30 30 30 28 26 28 28 28 33 25 24 31 13 30 21 7 14 30 > -- > 21037350 >905_1278_862_F3 > 21037351 31 32 32 29 32 33 31 32 30 33 32 32 32 29 32 33 31 33 32 30 33 30 29 32 22 32 32 31 31 26 33 32 30 30 30 28 26 28 28 28 33 25 24 31 13 30 21 7 14 30 > > which should not happen. As a result this file cannot be easily converted into fastq without explicit identifier matching. Please, check that your input csfasta and QUAL files have exactly the same number of non-comment lines. > > Thanks for using Galaxy, > > anton > galaxy team > > PS Your _R3 files are fine. > > Anton Nekrutenko > http://nekrut.bx.psu.edu > http://usegalaxy.org > > > > On May 26, 2011, at 10:50 AM, Jia-Xing Yue wrote: > >> Hi, when I converted the solid output to fastq, the galaxy always complained the following error. Does anybody know why>> >> ------------- >> An error occurred running this job:Traceback (most recent call last): >> File "/galaxy/home/g2main/galaxy_main/tools/next_gen_conversion/solid2fastq.py", line 207, in <module> >> main() >> File "/galaxy/home/g2main/galaxy_main/tools/next_gen_conversion/solid2fastq.py", line 188, in main >> >> >> Traceback (most recent call last): >> File "/galaxy/home/g2main/galaxy_main/tools/next_gen_conversion/solid2fastq.py", line 207, in >> >> main() >> File "/galaxy/home/g2main/galaxy_main/tools/next_gen_conversion/solid2fastq.py", line 188, in main >> merge_reads_qual( fr, fq, con, trim_name=options.trim_name, out='db', double_encode=options.de, trim_first_base=options.trim_first_base, min_qual=options.min_qual, table_name="f3" ) >> File "/galaxy/home/g2main/galaxy_main/tools/next_gen_conversion/solid2fastq.py", line 83, in merge_reads_qual >> cursor.execute('insert into %s values("%s","%s","%s")' % (table_name, defline, lines[0], qual ) ) >> sqlite3.OperationalError: near "?3": syntax error >> >> ----------- >> Thanks! >> >> Jia-Xing >> >> -- >> Jia-Xing Yue >> >> Graduate Student >> Ecology & Evolutionary Bio. -MS 170 >> Rice University >> 6100 Main Street >> Houston TX 77005 >> >> Phone: 1-832-360-6228 >> E-mail: yjx(a)rice.edu >> Blog: http://bestrok.blogspot.com/ >> >> >> >> >> ___________________________________________________________ >> The Galaxy User list should be used for the discussion of >> Galaxy analysis and other features on the public server >> at usegalaxy.org. Please keep all replies on the list by >> using "reply all" in your mail client. For discussion of >> local Galaxy instances and the Galaxy source code, please >> use the Galaxy Development list: >> >> http://lists.bx.psu.edu/listinfo/galaxy-dev >> >> To manage your subscriptions to this and other Galaxy lists, >> please use the interface at: >> >> http://lists.bx.psu.edu/ > >

1 0

Re: [galaxy-user] UCSC->EMBOSS/fuzznuc->UCSC workflow?
by Jennifer Jackson 07 Jun '11

07 Jun '11

Hello Curtis, The BED extraction data can be resolved in Galaxy. Pull out the whole gene and then modify the coordinates in Galaxy to be 10k upstream. To be clear - this coordinate data is going to be used to transform the coordinates in your current fuzznuc output that is transcript-based to be genome-based. The coordinates are not input for fuzznuc - the are used after fuzznuc is run on the fasta file, in order to covert the result coordinates only. This page in the UCSC wiki has a good description of how the UCSC coordinates are organized. http://genomewiki.ucsc.edu/index.php/Coordinate_Transforms The output format for fuzznuc is documented in the tool's help - the last line on the tool form has a link. Hopefully this helps to clear up the suggested processing, Thanks, Jen Galaxy team On 5/17/11 2:08 PM, Robert Curtis Hendrickson wrote: > Jennifer, > > I tried getting data from UCSC as .BED - two issues: > > 1. Unlike "get sequence", I can no longer specify how far upstream I > want - it's EITHER "whole gene" (what's the definition of that!!!) OR > #bp_upstream OR exons OR introns -- with get seq those are not mutually > exclusive - I happen to want the genomic region (5'UTR, exons, introns > 3'UTR + 10kbp upstream of 5'UTR) > > 2. fuzznuc does not recognize BED as a valid input format. So, I can't > run fuzznuc because my BED file doesn’t' show up in the pulldown. > > Indeed, BED files are just annotation, they don’t carry any sequence. > > Have I mis-understood your directions? > > Regards, > > Curtis > > -----Original Message----- > From: Jennifer Jackson [mailto:jen@bx.psu.edu] > Sent: Tuesday, May 17, 2011 11:23 AM > To: Robert Curtis Hendrickson > Cc: 'galaxy-user(a)lists.bx.psu.edu' > Subject: Re: [galaxy-user] UCSC->EMBOSS/fuzznuc->UCSC workflow? > > Hello Curtis, > > No need to use the fasta headers from your original fasta file. > > To obtain the coordinates in BED format: using "Get Data -> UCSC main" > > again to link to the UCSC Table browser, set the same selection criteria > > as for the original fasta sequence, only change the output type to be > > "BED" (instead of "sequence"). Once in your Galaxy history, this format > > will be easier to work with. > > Best, > > Jen > > Galaxy team > > On 5/16/11 9:04 PM, Robert Curtis Hendrickson wrote: > > > Jennifer, > > > > > > Thanks for the outline. I'll try that approach. > > > > > > However, it seems rather painful to have to join the fuzznuc output > back to the original fasta to get at the header information that really > should have been passed along. It would see that there must be a way to > get the data out of UCSC without that space in the fasta header, so that > the chromosome& genomic location get correctly preserved in the fuzznuc > output. Failing that, is there an easy text manipulation that would > convert that fasta header space to a "|"? > > > > > > Regards, > > > Curtis > > > > > > > > > -----Original Message----- > > > From: Jennifer Jackson [mailto:jen@bx.psu.edu] > > > Sent: Monday, May 16, 2011 6:50 PM > > > To: Robert Curtis Hendrickson > > > Cc: 'galaxy-user(a)lists.bx.psu.edu' > > > Subject: Re: [galaxy-user] UCSC->EMBOSS/fuzznuc->UCSC workflow? > > > > > > Hello Curtis, > > > > > > The coordinates of your match are with respect to the fasta sequence, > > > not with respect to the reference genome. Only data mapped to the > > > reference genome can be viewed in the UCSC Browser > > > > > > You will need to calculate from the position of the match in the fasta > > > sequence back through to the reference genome. > > > > > > One suggested way to do this: > > > > > > a) Merge together the original genomic coordinates of the 2kb regions > > > with each line of output from fuzznuc. Use the original source fasta > > > sequence name as the common key for the merge. If both data are in BED > > > format, that would be ideal and make the following steps possible. You > > > may need to split the file based on whether the original fasta sequence > > > came from the positive or negative strand to run "b" and "c" below > > > separately. > > > b) Use "Text Manipulation -> Compute an expression on every row" to > > > create new coordinates. For example, if your data is on the positive > > > strand, and base 1 in your fasta file was genomic coordinate 100, and > > > the alignment from fuzznuc started at base 5 (local coordinate == "4" if > > > in BED format with a zero-based start), then the new genomic start > > > coordinate would be [100 + 4)] = 104. Do this for both start and stop. > > > c) Adjust the logic for "b" if any of your original fasta sequences are > > > from the negative strand, on the negative strand portion of your data > > > ("b" would be run on just the positive strand portion of your data). > > > d) arrange/cut the resulting file down into a standard BED format to > > > remove the local coordinates and keep the genomic coordinates, using the > > > original chromosome names. > > > e) once the logic for the calculations is worked out, save the process > > > into a workflow for use again. > > > > > > Hopefully this helps, > > > > > > Best, > > > > > > Jen > > > Galaxy team > > > > > > On 5/13/11 9:32 AM, Robert Curtis Hendrickson wrote: > > >> Folks, > > >> > > >> I wanted to scan the 2kb upstream of a list of human gene isoforms > for TFBS using fuzznuc. I was able to > > >> "Get Data"> "UCSC Main"> "As sequence" and get my sequences > > >> "EMBOSS"> fuzznuc ran fine, and output the hits > > >> > > >> HOWEVER, fuzznuc lost the genomic position information that UCSC has > put after a space in the sequence headers of the FASTA file. It only > provided offsets within the fasta. > > >> > > >> http://main.g2.bx.psu.edu/u/curtish-uab/h/ucsc-fuzznuc-ucsc-broken > > >> > > >> Thus, when I converted the fuzznuc output back to a BED file and > tried to visualize the hits in UCSC browser, it failed with "invalid BED > File". > > >> I tried fuzznuc with output: seqtable, feattable and gff3, but in > all cases the genomic position was missing, and being a bit of Galaxy > novice, I couldn't figure out how to get the output back to UCSC to > visualize the hits. > > >> > > >> Can anyone tell me how to link up these tools correctly, or share a > history with some other tool set that accomplishes this goal? > > >> > > >> Regards, > > >> Curtis > > >> > > >> Research Associate > > >> Center for Clinical and Translational Science > > >> University of Alabama at Birmingham > > >> > > >> ___________________________________________________________ > > >> The Galaxy User list should be used for the discussion of > > >> Galaxy analysis and other features on the public server > > >> at usegalaxy.org. Please keep all replies on the list by > > >> using "reply all" in your mail client. For discussion of > > >> local Galaxy instances and the Galaxy source code, please > > >> use the Galaxy Development list: > > >> > > >> http://lists.bx.psu.edu/listinfo/galaxy-dev > > >> > > >> To manage your subscriptions to this and other Galaxy lists, > > >> please use the interface at: > > >> > > >> http://lists.bx.psu.edu/ > > > > > -- > > Jennifer Jackson > > http://usegalaxy.org > > http://galaxyproject.org > -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org

2 3

Re: [galaxy-user] Loading files to local Galaxy
by Paul-Michael Agapow 07 Jun '11

07 Jun '11

> From: Tilahun Abebe <Tilahun.Abebe(a)uni.edu> > > We are trying to load Illumina data to our local Galaxy instance. The > files are between 700 MB and 2.2 GB. Files below 2 GB load in less than > 5 minutes. Files larger than 2 GB don't upload at all. We installed > Galaxy locally because we thought loading files will be faster than the > server version. Any suggestions to solve this problem is highly appreciated. While (from others experience) Galaxy _should_ be able to upload files that large, we've had some problems with our local installation too. Investigation didn't reveal any cause, so we put it down to the quality of our network. You might what to look at the webserver or proxy that you have in front of Galaxy - from memory, both Apache and nginx can be configured to impose file size limits, so that _may_ be the problem. In any event, you might want to configure your server to handle uploads and downloads directly as per <https://bitbucket.org/galaxy/galaxy-central/wiki/Config/ProductionServe r>. Finally you can pass an url to the upload dialog to get Galaxy to pull the file from an ftp server - for example - and that may prove more robust. ---- Paul Agapow (paul-michael.agapow(a)hpa.org.uk) Bioinformatics, Centre for Infections, Health Protection Agency ----------------------------------------- ************************************************************************** The information contained in the EMail and any attachments is confidential and intended solely and for the attention and use of the named addressee(s). It may not be disclosed to any other person without the express authority of the HPA, or the intended recipient, or both. If you are not the intended recipient, you must not disclose, copy, distribute or retain this message or any part of it. This footnote also confirms that this EMail has been swept for computer viruses, but please re-sweep any attachments before opening or saving. HTTP://www.HPA.org.uk **************************************************************************

3 3

Re: [galaxy-user] Most Popular Linux flavour for Galaxy
by Enis Afgan 07 Jun '11

07 Jun '11

Hi Ron, Yes, it's the 64-bit Ubuntu and we've opted to go with the 10.04 version because it's LTS version, although I don't think there were any changes in the more recent versions that will break things in the scripts. Enis On Tue, Jun 7, 2011 at 10:38 AM, Ronald Worthington <rworthi(a)siue.edu>wrote: > Hi Enis, > > Is that 64 bit Ubuntu, and why not Ubuntu 11.04 - are there stability > issues with later versions? I know Bio-linux uses 64 bit Ubuntu 10.04 as > well. > > - Ron > > On Jun 7, 2011, at 10:15 AM, Enis Afgan wrote: > > Hi Matloob, > Others should contribute their experiences as well but for the Galaxy VM > and cloud, we use Ubuntu 10.04. Also, to automate the machine setup, we have > a scripts that perform all of the required setup steps; the scripts are > usable by anyone for a range of machines and are available here: > https://bitbucket.org/afgane/mi-deployment/ > > Good luck, > Enis > > On Tue, Jun 7, 2011 at 8:00 AM, Matloob Khushi < > matloob.khushi(a)sydney.edu.au> wrote: > >> Hello Galaxy Users >> >> We are anticipating setting up a local instance of Galaxy and wondering >> what is the most popular (or best) flavour of Linux people out there using. >> >> I wonder which OS on UseGalaxy.org has been used. Thanks for your help. >> >> Regards >> Matloob >> PhD Candidate >> >> ___________________________________________________________ >> The Galaxy User list should be used for the discussion of >> Galaxy analysis and other features on the public server >> at usegalaxy.org. Please keep all replies on the list by >> using "reply all" in your mail client. For discussion of >> local Galaxy instances and the Galaxy source code, please >> use the Galaxy Development list: >> >> http://lists.bx.psu.edu/listinfo/galaxy-dev >> >> To manage your subscriptions to this and other Galaxy lists, >> please use the interface at: >> >> http://lists.bx.psu.edu/ >> > > ___________________________________________________________ > The Galaxy User list should be used for the discussion of > Galaxy analysis and other features on the public server > at usegalaxy.org. Please keep all replies on the list by > using "reply all" in your mail client. For discussion of > local Galaxy instances and the Galaxy source code, please > use the Galaxy Development list: > > http://lists.bx.psu.edu/listinfo/galaxy-dev > > To manage your subscriptions to this and other Galaxy lists, > please use the interface at: > > http://lists.bx.psu.edu/ > > >

1 0

TSS list with annotation for Hg19
by shamsher jagat 03 Jun '11

03 Jun '11

I want to extract TSS ids with correspodning location for Hg19. Once I extract such TSS coordinates I want to have there annotation either with enetrz gene id or some other unique Ids. I am sure Galaxy can do this, But I dont know where to start, or which workflow to use. Can some one please dircet me to right dircetion. Appreciate your help.

2 2

Flagstat on BAM files
by Slim Sassi 03 Jun '11

03 Jun '11

Hello, I tried to use NGS: SAM Tools ->flagstat on a BAM files for basic stats, but I got results like you see below. It doesn't seem to be working. Any suggestions? 26584869 in total 0 QC failure 0 duplicates 26584869 mapped (100.00%) 0 paired in sequencing 0 read1 0 read2 0 properly paired (-nan%) 0 with itself and mate mapped 0 singletons (-nan%) 0 with mate mapped to a different chr 0 with mate mapped to a different chr (mapQ>=5) Thanks Slim The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.

6 11