September 2011 - galaxy-user - lists.galaxyproject.org

Accessing Galaxy on the cloud from http
by Colleen Doherty 27 Sep '11

27 Sep '11

Thanks for the suggestions! The ami I've been mainly using is ami-da58aab3 (although I've tried a few others). The user data may be where I am messing up: I have a few questions I'm not clear on, I think I've tried may options here.. 1. Is cluster name just something I make up? 2. Are the <> characters part of the yaml structure? (I've tried both). 3. Do I need to specify the default information? I've tried: cluster_name: galaxy_cloud_cluster password: apassword access_key: mypublickeyblahblahblahblab secret_key: myprivatekeyblahblabhalbahblahblah I've also tried with the <> cluster_name: <galaxy_cloud_cluster> password: <apassword> I've also tried adding the other information... boot_script_name: [cm_boot.py] boot_script_path: [/tmp/cm] bucket_cluster: cm-<hash> bucket_default: [cloudman] cloudman_home: /mnt/cm [master_ip: <*exists only if worker*; master internal IP address>] role:<master or worker> [testflag: False] On apache: I totally understand it's not part of any of this, I was just trying that to see if I could access through http (I was just trying things to narrow down where the problem is). Here's an attempt at an output from py.log, I hope I have it all. Thanks for working through this with me! ubuntu@ip-10-204-103-174:/opt/galaxy/pkg$ cat ec2autorun.py.log [INFO] ec2autorun:38 2011-09-27 07:09:50,922: Getting user data from 'http://169 .254.169.254/latest/user-data', attempt 0 [DEBUG] ec2autorun:42 2011-09-27 07:09:50,938: Saving user data in its original format to file '/tmp/cm/original_userData.yaml' [DEBUG] ec2autorun:46 2011-09-27 07:09:50,938: Got user data [INFO] ec2autorun:284 2011-09-27 07:09:50,938: Handling user data in yaml format [DEBUG] ec2autorun:317 2011-09-27 07:09:50,940: The provided user data does not contain bucket_default field; setting it to 'cloudman'. [DEBUG] ec2autorun:337 2011-09-27 07:09:50,940: Composed user data: {'access_key ': '<accesskeyblah>', 'cloudman_home': '/mnt/cm', 'cluster_name': '<galax y_cloud_cluster>', 'bucket_default': 'cloudman', 'role': 'master', 'bucket_clust er': 'cm-6df419acea85fd768bdf345a980ab112', 'boot_script_path': '/tmp/cm', 'boot _script_name': 'cm_boot.py', 'secret_key': '<blahblahblahblah>', 'password': '<somepassword>'} [DEBUG] ec2autorun:75 2011-09-27 07:09:51,120: Establishing boto S3 connection [DEBUG] ec2autorun:78 2011-09-27 07:09:51,121: Got boto S3 connection. [DEBUG] ec2autorun:98 2011-09-27 07:09:52,528: Cluster bucket 'cm-6df419acea85fd 768bdf345a980ab112' not found. [DEBUG] ec2autorun:195 2011-09-27 07:09:52,528: Using default bucket 'cloudman' [DEBUG] ec2autorun:214 2011-09-27 07:09:52,528: Could not get boot script 'cm_bo ot.py' from cluster bucket 'cm-6df419acea85fd768bdf345a980ab112'; retrieving pub lic one from bucket url 'http://s3.amazonaws.com/cloudman/cm_boot.py' [INFO] ec2autorun:169 2011-09-27 07:09:52,529: Getting boot script from 'http:// s3.amazonaws.com/cloudman/cm_boot.py' and saving it locally to '/tmp/cm/cm_boot. py' [DEBUG] ec2autorun:176 2011-09-27 07:09:52,565: Got boot script from 'http://s3. amazonaws.com/cloudman/cm_boot.py' [DEBUG] ec2autorun:217 2011-09-27 07:09:52,565: Saved boot script to '/tmp/cm/cm _boot.py' [DEBUG] ec2autorun:75 2011-09-27 07:09:52,565: Establishing boto S3 connection [DEBUG] ec2autorun:78 2011-09-27 07:09:52,565: Got boto S3 connection. [DEBUG] ec2autorun:98 2011-09-27 07:09:52,589: Cluster bucket 'cm-6df419acea85fd 768bdf345a980ab112' not found. [INFO] ec2autorun:229 2011-09-27 07:09:52,590: Running boot script '/tmp/cm/cm_b oot.py' [DEBUG] ec2autorun:233 2011-09-27 07:10:04,836: Successfully ran boot script '/t mp/cm/cm_boot.py' [INFO] ec2autorun:373 2011-09-27 07:10:04,836: ---> /opt/galaxy/pkg/ec2autorun.p y done <--- On Sep 26, 2011 10:47 PM, "Enis Afgan" <eafgan(a)emory.edu> wrote:

2 2

How to analyze Genes Differential Expression using Galaxy? Thanks.
by Binbin You 27 Sep '11

27 Sep '11

Hi All, Now I have two Paired-End samples data ( they are Next Generation Sequence Data and fastq format) generated from Illumina GA II. Let's say Data Lane1 and Data Lane2. After Quality trimming and Adapter trimming, I want to check the Genes Differential Expression between these two samples. How could I do that in Penn State version Galaxy or other versions with only having Reference File? Thanks a lot in advance! Best wishes, Bin

2 1

Question
by dtrejo＠ira.cinvestav.mx 27 Sep '11

27 Sep '11

Hi, mi name is Diana Trejo and I am working with galaxy for first time. I would like to analyze millions of sequences in galaxy but I don´t know if it is possible because my file is 330 MB. Are there any MB restriction for using galaxy? Thanks for your help Diana Trejo -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.

3 2

Re: [galaxy-user] mm9 reference GTF file for Cuffcompare (shamsher jagat)
by Lozach, Jean 27 Sep '11

27 Sep '11

------------------------------ Message: 4 >Date: Mon, 26 Sep 2011 16:13:54 -0700 >From: shamsher jagat <kanwarjag(a)gmail.com> >To: galaxy-user <galaxy-user(a)lists.bx.psu.edu> >Subject: [galaxy-user] mm9 reference GTF file for Cuffcompare >Message-ID: > <CAN_Q6i72ALsjHKgbQNRFo=kTywAB24BwAtJsEE-1NPa3z64DKA(a)mail.gmail.com> >Content-Type: text/plain; charset="iso-8859-1" > >I wonder if some has mouse genome GTF file compatible with Tophat/ >Cuffcompare. The contig names on Ensembel and that of Tophat GTF file are >not same. >Thanks ------------------------------ Illumina provides a ftp site where you can download all the files you need to use TopHat and Cufflinks: Bowtie index and gtf files are present for about 20 species. For most species, we do covered different data sources (mainly NCBI, Ensembl and UCSC) as well as different builds. This resource is called iGenomes. You can find all the links here http://cufflinks.cbcb.umd.edu/igenomes.html and more information about iGenomes here ftp://ftp.illumina.com/README.txt The annotations are few months old but we will have a full update in 2 weeks that will also include BWA Index files. Regards Jean Lozach Manager, Bioinformatics Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 ------------------------------

1 0

Galaxy Architecture Documentation
by Oren Livne 27 Sep '11

27 Sep '11

Dear All, I am a software developer, new to Galaxy. Is there a detailed document about the Galaxy code architecture, API, what runners are, and how to write a runner? I am especially interested in integrating Galaxy with an external scheduler software (I am aware of Galaxy's scheduling capabilities). I'd like to intercept Galaxy workflows and send their object model to my scheduler. I could not find this information on the Galaxy Wiki. Thank you so much in advance, Oren Livne

4 8

Accessing Galaxy on the cloud from http
by Colleen Doherty 26 Sep '11

26 Sep '11

Hello, I am having trouble setting up access to the galaxy cloud instance and may have missed something in the tutorial (I am following this one:http://wiki.g2.bx.psu.edu/Admin/Cloud) I am able to launch the instance I can access it through ssh, but when I try to access it through firefox or chrome it just times out. I am using the public DNS provided by amazon I DO add the /cloud I do have the http port added in the security group for "GalaxyWeb" that I set up (I also tried to add https just in case) Would someone be willing to work through this with me? I suspect I have incorrectly set up my security group, but I have tried many versions of the options. I have been through the amazon security faqs and I don't see what the issue could be (other than maybe IPv4 vs IPv6). Any advice or links for further help would be greatly appreciated Thank you!

3 4

mm9 reference GTF file for Cuffcompare
by shamsher jagat 26 Sep '11

26 Sep '11

I wonder if some has mouse genome GTF file compatible with Tophat/ Cuffcompare. The contig names on Ensembel and that of Tophat GTF file are not same. Thanks

1 0

fasta-Cuffcompare
by dongdong zhaoweiming 26 Sep '11

26 Sep '11

hi, I want to use Cuffcompare to compare my assembled transcripts to a reference annotation. My assembled transcripts is a Fasta format produced by trinity, but Cuffcompare requried GTF file , so what should I do? Thank you! weimin zhao

2 1

Chip-seq
by shamsher jagat 21 Sep '11

21 Sep '11

Can I analyze two bed files from Chip seq experiemnt in Galaxy? I have one file of input and other of sample. Both these files have peak locations. Any suggestion of a work flow in Galaxy?

2 1

Re: [galaxy-user] Simulating sequencing and removing redundant sequences
by Kevin Lam 20 Sep '11

20 Sep '11

No I believe there isn't sth like that. It should be a simple perl script to get that info or linux grep would help On 20 September 2011 17:43, Daniel Sher <dsher(a)sci.haifa.ac.il> wrote: > Thanks Kevin - that might work - collapse sequences and then grab the > sequence names using "select" (under filter and sort tools). Is there a way > to "loop" such a search in order to go through the lines of a long file? I > assume there must be something in the workflows but I have never used that > feature. > > Thanks > > Daniel > > On 20/09/2011 11:46, Kevin Lam wrote: > > Hi Daniel, > You would have multiple names for each sequence and that would be quite > hard to display. I am sure someone thought through this. Since the sequence > is the same, you can use the sequence to look back in the fastq file for > read name. Although I am not sure how that would help you? > > Cheers > Kevin > > > On 20 September 2011 13:43, Daniel Sher <dsher(a)sci.haifa.ac.il> wrote: > >> Thanks Kevin. However, the collapse sequences replaces the original >> name of the sequences with a numerical code, and I need to keep the original >> names. Any other suggestions? >> >> Thanks >> >> Daniel >> On 20/09/2011 05:32, Kevin Lam wrote: >> >> Hi Daniel >> for 2) you may use the tools under NGS QC and manipulation >> FASTQ to FASTA<http://main.g2.bx.psu.edu/tool_runner?tool_id=cshl_fastq_to_fasta>converter >> >> followed by >> >> Collapse<http://main.g2.bx.psu.edu/tool_runner?tool_id=cshl_fastx_collapser>sequences >> >> >> On 19 September 2011 09:54, Kevin Lam <kevin(a)aitbiotech.com> wrote: >> >>> For 1) you may refer to Simulated Dataset of Solexa - SEQanswers<http://seqanswers.com/forums/showthread.php?t=806> >>> >>> >>> Has anyone replied you for 2) ? >>> >>> >>> >>> On 18 September 2011 21:12, Daniel Sher <dsher(a)sci.haifa.ac.il> wrote: >>> >>>> Hello, >>>> >>>> I have two questions - I apologize if they are trivial.. >>>> >>>> 1) I want to simulate the amount of Illumina sequencing needed to >>>> sequence and assemble a known genome. Is there a way to randomly pick >>>> sequences of a specific length from a genome (either one available online or >>>> one I upload)? Something like "pick 100bp randomly (either strand), move >>>> 400-500bp forward and pick another 100bp?" >>>> >>>> 2) Is there a way to remove redundant sequences from a FASTA file >>>> without losing the original sequence names (as happens with "collapse")? >>>> >>>> Thanks >>>> >>>> Daniel >>>> >>>> >>>> -- >>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >>>> Daniel Sher, PhD >>>> Department of Marine Biology >>>> Leon H. Charney School of Marine Sciences >>>> University of Haifa, Mt. Carmel 31905, Haifa, Israel >>>> >>>> Office +972-4-8240731 >>>> Lab +972-4-8288961 >>>> email: dsher(a)sci.haifa.ac.il >>>> >>>> >>>> ___________________________________________________________ >>>> The Galaxy User list should be used for the discussion of >>>> Galaxy analysis and other features on the public server >>>> at usegalaxy.org. Please keep all replies on the list by >>>> using "reply all" in your mail client. For discussion of >>>> local Galaxy instances and the Galaxy source code, please >>>> use the Galaxy Development list: >>>> >>>> http://lists.bx.psu.edu/listinfo/galaxy-dev >>>> >>>> To manage your subscriptions to this and other Galaxy lists, >>>> please use the interface at: >>>> >>>> http://lists.bx.psu.edu/ >>>> >>> >>> >> >> -- >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> Daniel Sher, PhD >> Department of Marine Biology >> Leon H. Charney School of Marine Sciences >> University of Haifa, Mt. Carmel 31905, Haifa, Israel >> >> Office +972-4-8240731 >> Lab +972-4-8288961 >> email: dsher(a)sci.haifa.ac.il >> >> > > -- > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > Daniel Sher, PhD > Department of Marine Biology > Leon H. Charney School of Marine Sciences > University of Haifa, Mt. Carmel 31905, Haifa, Israel > > Office +972-4-8240731 > Lab +972-4-8288961 > email: dsher(a)sci.haifa.ac.il > >

1 0