- galaxy-user - lists.galaxyproject.org

How to replace ensembl gene ID with gene names in Cuffdiff output?
by Wei Liao 24 Apr '12

24 Apr '12

Hi all, I had the following Cuffdiff output from genes defferetial expression testing: test_id gene_id gene locus sample_1 sample_2 status value_1 value_2 log2(fold_change) test_stat p_value q_value XLOC_000001 XLOC_000001 ENST00000450305,ENST00000456328,ENST00000515242,ENST00000518655 chr1:11868-31109 NORMAL 1 OK 0.558797 0.84004 0.588134 -0.44598 0.655611 0.767628 There are four transcript IDs belong to gene DDXL11L1, my question is how to replace these ID with official gene names? Thanks, -- Wei Liao Research Scientist, Brentwood Biomedical Research Institute 16111 Plummer St. Bldg 7, Rm D-122 North Hills, CA 91343 818-891-7711 ext 7645

2 1

Creating a galaxy tool in R - "You must not use 8-bit bytestrings"
by Dan Tenenbaum 24 Apr '12

24 Apr '12

Hello, I'm a galaxy newbie and running into several issues trying to adapt an R script to be a galaxy tool. I'm looking at the XY plotting tool for guidance (tools/plot/xy_plot.xml), but I decided not to embed my script in XML, but instead have it in a separate script file, that way I can still run it from the command line and make sure it works as I make incremental changes. (So my script starts with args <- commandArgs(TRUE)). Also, if it doesn't work, this suggests to me that there is a problem with my galaxy configuration. First, I tried using the r_wrapper.sh script that comes with the XY plotting tool, but it threw away my arguments: An error occurred running this job: ARGUMENT '/Users/dtenenba/dev/galaxy-dist/database/files/000/dataset_4.dat' __ignored__ ARGUMENT '/Users/dtenenba/dev/galaxy-dist/database/files/000/dataset_3.dat' __ignored__ ARGUMENT 'Fly' __ignored__ ARGUMENT 'Tagwise' __ignored__ etc. So then I tried just switching to Rscript: <command interpreter="bash">Rscript RNASeq.R $countsTsv $designTsv "$organism" $dispersion $minimumCountsPerMillion $minimumSamplesPerTranscript $out_file1 $out_file2</command> (My script produces as output a csv file and a pdf file. The final two arguments I'm passing are the names of those files.) But then I get an error that Rscript can't be found. So I wrote a little wrapper script, Rscript_wrapper.sh: #!/bin/sh Rscript $* And called that: <command interpreter="bash">Rscript_wrapper.sh RNASeq.R $countsTsv $designTsv "$organism" $dispersion $minimumCountsPerMillion $minimumSamplesPerTranscript $out_file1 $out_file2</command> Then I got an error that RNASeq.R could not be found. So then I added the absolute path to my R script to the <command> tag. This seemed to work (that is, it got me further, to the next error), but I'm not sure why I had to do this; in all the other tools I'm looking at, the directory to the script to run does not have to be specified; I assumed that the command would run in the appropriate directory. So now I've specified the full path to my R script: <command interpreter="bash">Rscript_wrapper.sh /Users/dtenenba/dev/galaxy-dist/tools/bioc/RNASeq.R $countsTsv $designTsv "$organism" $dispersion $minimumCountsPerMillion $minimumSamplesPerTranscript $out_file1 $out_file2</command> And I get the following long error, which includes all of the output of my R script: Traceback (most recent call last): File "/Users/dtenenba/dev/galaxy-dist/lib/galaxy/jobs/runners/local.py", line 133, in run_job job_wrapper.finish( stdout, stderr ) File "/Users/dtenenba/dev/galaxy-dist/lib/galaxy/jobs/__init__.py", line 725, in finish self.sa_session.flush() File "/Users/dtenenba/dev/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/scoping.py", line 127, in do return getattr(self.registry(), name)(*args, **kwargs) File "/Users/dtenenba/dev/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/session.py", line 1356, in flush self._flush(objects) File "/Users/dtenenba/dev/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/session.py", line 1434, in _flush flush_context.execute() File "/Users/dtenenba/dev/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/unitofwork.py", line 261, in execute UOWExecutor().execute(self, tasks) File "/Users/dtenenba/dev/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/unitofwork.py", line 753, in execute self.execute_save_steps(trans, task) File "/Users/dtenenba/dev/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/unitofwork.py", line 768, in execute_save_steps self.save_objects(trans, task) File "/Users/dtenenba/dev/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/unitofwork.py", line 759, in save_objects task.mapper._save_obj(task.polymorphic_tosave_objects, trans) File "/Users/dtenenba/dev/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/mapper.py", line 1413, in _save_obj c = connection.execute(statement.values(value_params), params) File "/Users/dtenenba/dev/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py", line 824, in execute return Connection.executors[c](self, object, multiparams, params) File "/Users/dtenenba/dev/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py", line 874, in _execute_clauseelement return self.__execute_context(context) File "/Users/dtenenba/dev/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py", line 896, in __execute_context self._cursor_execute(context.cursor, context.statement, context.parameters[0], context=context) File "/Users/dtenenba/dev/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py", line 950, in _cursor_execute self._handle_dbapi_exception(e, statement, parameters, cursor, context) File "/Users/dtenenba/dev/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py", line 931, in _handle_dbapi_exception raise exc.DBAPIError.instance(statement, parameters, e, connection_invalidated=is_disconnect) ProgrammingError: (ProgrammingError) You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings. u'UPDATE job SET update_time=?, stdout=?, stderr=? WHERE job.id = ?' ['2012-04-24 18:55:45.791417', '', 'BiocInstaller version 1.5.7, ?biocLite for help\nWarning message:\nNAs introduced by coercion \nLoading required package: methods\nLoading required package: limma\nLoading required package: BiasedUrn\nLoading required package: geneLenDataBase\nLoading required package: org.Dm.eg.db\nLoading required package: AnnotationDbi\nLoading required package: BiocGenerics\n\nAttaching package: \xe2\x80\x98BiocGenerics\xe2\x80\x99\n\nThe following object(s) are masked from \xe2\x80\x98package:stats\xe2\x80\x99:\n\n xtabs\n\nThe following object(s) are masked from \xe2\x80\x98package:base\xe2\x80\x99:\n\n anyDuplicated, cbind, colnames, duplicated, eval, Filter, Find,\n get, intersect, lapply, Map, mapply, mget, order, paste, pmax,\n pmax.int, pmin, pmin.int, Position, rbind, Reduce, rep.int,\n rownames, sapply, setdiff, table, tapply, union, unique\n\nLoading required package: Biobase\nWelcome to Bioconductor\n\n Vignettes contain introductory material; view with\n \'browseVignettes()\'. To cite Bioconductor, see\n \'citation("Biobase")\', and for packages \'citation("pkgname")\'.\n\nLoading required package: DBI\n\nCalculating library sizes from column totals.\nError in matrix(u, nrow = nrows, byrow = TRUE) : \n negative extents to matrix\nCalls: plotMDS.DGEList ... equalizeLibSizes -> splitIntoGroups -> lapply -> FUN -> matrix\nExecution halted\n', 15] Note that if I run my script from the command line: ./Rscript_wrapper.sh RNASeq.R /Users/dtenenba/dev/galaxy-dist/database/files/000/dataset_4.dat /Users/dtenenba/dev/galaxy-dist/database/files/000/dataset_3.dat Fly 1 1 Tagwise MDSPlot.pdf outputs.csv It works fine and does not produce a warning about "NAs introduced by coercion", nor does it fail with the "Error in matrix" above. So, can anyone tell me what is going wrong here? Why does R behave differently in galaxy than it does on the command line? (I'm using the same instance of R, same machine, for my galaxy and command-line efforts). Is this 8-bit bytestring error a red herring? Can I filter it so that galaxy is happy? Finally, one other curiosity. Every time I hit "Execute" in galaxy to run my tool, it is run twice--two jobs are created (which each fail in the same way). Why is this? My R script: https://gist.github.com/2482783 My XML file: https://gist.github.com/2482792 I can share more data (such as sample input files) if necessary. Thanks for your help. Dan

1 0

Data upload...
by Gregory Miles 23 Apr '12

23 Apr '12

We have large files that cannot be uploaded using the "file upload" command and instead would need to be uploaded using a URL. Unfortunately, we are using a "local" install on a non-local machine, so setting up an FTP server on this machine is a security issue. The files are located on this computer already anyhow, and Galaxy would simply be copying from one folder to another in order to perform the "get data" step. Is there a simple way to have a "pointer" of some sort such that galaxy knows where this file is and: 1) Would not have to copy it and could simply refer to the file location. 2) Could perform data analysis steps on this file and push the output to the usual location (not the location of the data files). Any help would be greatly appreciated. Thanks. Dr. Gregory Miles Bioinformatics Specialist Cancer Institute of New Jersey @ UMDNJ Office: (732) 235 8817 ----------------------------------------------------------------------------------------------------------------------------- CONFIDENTIALITY NOTICE: This email communication may contain private, confidential, or legally privileged information intended for the sole use of the designated and/or duly authorized recipient(s). If you are not the intended recipient or have received this email in error, please notify the sender immediately by email and permanently delete all copies of this email including all attachments without reading them. If you are the intended recipient, secure the contents in a manner that conforms to all applicable state and/or federal requirements related to privacy and confidentiality of such information.

2 5

Re: [galaxy-user] 1. cuffcompare or cuffmerge (???)
by Wei Liao 23 Apr '12

23 Apr '12

Hi, JiWen I thought about this before, here is the answer from Cole Trapnell from Seqanswer website: " I can shed some light on this. We have an upcoming protocol paper that describes our recommended workflow for TopHat and Cufflinks that discusses some of these issues. As turnersd outlined, there are three strategies: 1) merge bams and assemble in a single run of Cufflinks 2) assemble each bam and cuffcompare them to get a combined.gtf 3) assemble each bam and cuffmerge them to get a merged.gtf All three options work a little differently depending on whether you're also trying to integrate reference transcripts from UCSC or another annotation source. #1 is quite different from #2 and #3, so I'll discuss its pros and cons first. The advantage here is simplicity of workflow. It's one Cufflinks run, so no need to worry about the details of the other programs. As turnersd mentions, you might also think this maximizes the accuracy of the resulting assembly, and that might be the case, but it also might not (for technical reasons that I don't want to get into right now). The disadvantage of this approach is that your computer might not be powerful enough to run it. More data and more isoforms means substantially more memory and running time. I haven't actually tried this on something like the human body map, but I would be very impressed and surprised if Cufflinks can deal with all of that on a machine owned by mere mortals. #2 and #3 are very similar - both are designed to gracefully merge full-length and partial transcript assemblies without ever merging transfrags that disagree on splicing structure. Consider two transfrags, A and B, each with a couple exons. If A and B overlap, and they don't disagree on splicing structure, we can (and according to Cufflinks' assembly philosophy, we should) merge them. The difference between Cuffcompare and Cuffmerge is that Cuffcompare will only merge them if A is "contained" in B, or vice versa. That is, only if one of the transfrags is essentially redundant. Otherwise, they both get included. Cuffmerge on the other hand, will merge them if they overlap, and agree on splicing, and are in the same orientiation. As turnersd noted, this is done by converting the transfrags into SAM alignments and running Cufflinks on them. The other thing that distinguishes these two options is how they deal with a reference annotation. You can read on our website how the Cufflinks Reference Annotation Based Transcript assembler (RABT) works. Cuffcompare doesn't do any RABT assembly, it just includes the reference annotation in the combined.gtf and discards partial transfrags that are contained and compatible with the reference. Cuffmerge actually runs RABT when you provide a reference, and this happens during the step where transfrags are converted into SAM alignments and assembled. We do this to improve quantification accuracy and reduce errors downstream. I should also say that Cuffmerge runs cuffcompare in order annotate the merged assembly with certain helpful features for use later on. So we recommend #3 for a number of reasons, because it is the closest in spirit to #1 while still being reasonably fast. For reasons that I don't want to get into here (pretty arcane details about the Cufflinks assembler) I also feel that option #3 is actually the most accurate in most experimental settings. " Hope this helps. Wei Liao Research Scientist, Brentwood Biomedical Research Institute 16111 Plummer St. Bldg 7, Rm D-122 North Hills, CA 91343 818-891-7711 ext 7645 > Today's Topics: > > 1. cuffcompare or cuffmerge (???) > 2. Data upload... (Gregory Miles) > > > ---------- Forwarded message ---------- > From: 杨继文 <jiwenyang0605(a)126.com> > To: galaxy-user(a)lists.bx.psu.edu > Cc: > Date: Mon, 23 Apr 2012 20:32:28 +0800 (CST) > Subject: [galaxy-user] cuffcompare or cuffmerge > Hi all, > I read one paper "Differential gene and transcript expression analysis of > RNA-seq experiments with TopHat and Cufflinks". > They say the procedure for RNA-Seq analysis is *Tophat-->cufflinks--> > cuffmerge-->cuffdiff* > > But what I normally do in Galaxy is *Tophat-->cufflinks-->cuffcompare-->cuffdiff. > I have six samples, which means I will generate 6 assembled transcript > files by cufflinks. Then I run cuffcompare using all six assembled > transcript files as input. The resulting "combined transcript" is the input > for cufflinks. * > ** > *I don't know why I shoud use cuffmerge. Actually I don't understand the > function of cuffmerge.* > ** > *Did I miss something?? * > Please let me know your opinions. > Jiwen > > > ------------------------------ > -- > Wei Liao Research Scientist, Brentwood Biomedical Research Institute 16111 Plummer St. Bldg 7, Rm D-122 North Hills, CA 91343 818-891-7711 ext 7645

1 0

cuffcompare or cuffmerge
by 杨继文 23 Apr '12

23 Apr '12

Hi all, I read one paper "Differential gene and transcript expression analysis of RNA-seq experiments with TopHat and Cufflinks". They say the procedure for RNA-Seq analysis is Tophat-->cufflinks-->cuffmerge-->cuffdiff But what I normally do in Galaxy is Tophat-->cufflinks-->cuffcompare-->cuffdiff. I have six samples, which means I will generate 6 assembled transcript files by cufflinks. Then I run cuffcompare using all six assembled transcript files as input. The resulting "combined transcript" is the input for cufflinks. I don't know why I shoud use cuffmerge. Actually I don't understand the function of cuffmerge. Did I miss something?? Please let me know your opinions. Jiwen

1 0

GATK tools: other genome available?
by Jean JAUBERT 21 Apr '12

21 Apr '12

Hi, The NGS: GATK Tools on main are so far only for human genome. Do you have an idea when other genomes (such as my favorite, i.e mice and mm9) would be accessible ? Otherwise, is there a way to easily transfer "mm9 genome that is system-installed on main" directly in a "Custom build genome" that could be used in such a case ? Or is this a bad idea ? Thanks for your help, Best. Jean Jaubert

2 1

Annotation
by hsharm03＠students.poly.edu 21 Apr '12

21 Apr '12

Dear galaxy users, This might be a very basic question to most of you. But I was hopimg I could get better understanding of this concept by asking you all. How exactly can we accomplish annotation of our reads? The combination of Tophat and cufflinks does annotate genes right? . I am a bit confused regarding this topic. Any help will be much appreciated. Thanks.

3 3

Re: [galaxy-user] how to do an enrichment analysis in Galaxy
by Anton Nekrutenko 19 Apr '12

19 Apr '12

Binnaz: Please, in the future forward such questions to galaxy-user(a)bx.psu.edu mailing list. This is the most reliable way to get your queries answered. If you carefully go through http://usegalaxy.org/galaxy101 you will see how to count occurrences of features (such as LINES) within genomic ranges (such as (ChrX:144,000,000-146,500,000)). Thanks! anton Anton Nekrutenko http://nekrut.bx.psu.edu http://usegalaxy.org On Apr 19, 2012, at 12:53 PM, Binnaz Yalcin wrote: > Dear Anton, > > Can I do an anrichment analysis in Galaxy? if so, how? > > I want to know whether a region on mouse chromosome X (ChrX:144,000,000-146,500,000 bp) is enriched with LINE elements, compared to the rest of the chromosome X. > > Any advice would be much more appreciated. > > Thanks > > Best, > Binnaz YALCIN >

1 0

Re: [galaxy-user] [galaxy-dev] Create and Transfer Galaxy Page
by Jeremy Goecks 19 Apr '12

19 Apr '12

Hi Todd, > [Not sure if this is better suited to galaxy-dev or -user, so I'm sending to both]. galaxy-user is most appropriate for this question because it related to usage of Galaxy; galaxy-dev is for local installation and tool development questions. > My question is - can I create a Galaxy 'Published Page' from my local Galaxy instance/histories, and then transfer that page to the main Galaxy instance? Not currently, though this is in our long-term plan. > The reason is that I cannot make my local Galaxy instance public, as I am using a campus resource to host our galaxy. If this is possible, how can I do that? If not, any other ideas? It is possible to move datasets and workflows relatively easily between instances, so I'd recommend that: (a) you move your data and workflows to our public instance; (b) rerun your analyses on the public instance to create the required; (c) create and host the Page on our public instance. You can be assured that we will maintain our public server over the coming years and your Page will remain available and have a stable URL. > Also, are there any tutorials/pages on how to create Published Pages in Galaxy in the first place? Not yet, though the idea is for the Page editor to be self explanatory. Here's how to get started with Pages: (a) from User menu, go to Saved Pages; (b) create a Page; (c) edit the Page using the Web-based editor; there are menus for inserting embedded datasets, workflows, histories, and visualizations as well as performing standard word-processing operations. Let us know if you have problems/questions and we'll start a guide for creating Pages. Best, J.

4 4

"Server Error"on Main when viewing or downloading FastQC results
by Davis, Wade 18 Apr '12

18 Apr '12

I've run FastQC successfully on several different files, but I can not download them or view them. In either. case I receive the following message in my browser (Firefox): Server Error An error occurred. See the error logs for more information. (Turn debug on to display exception reports here) If I go into a history and try to view other FastQC reports that I have viewed before, I get the same error. So I don't think this is a data format or tools issue. I searched the archives, and the server error message has been reported before, and was fixed but no details were given. Thanks for your help. Wade

2 1