Hi Andrew,

We do not have any estimates posted on the wiki currently for example usage on the cloud, but this is a good idea and the team is discussing the best way to add in something like this. The difficulty comes with how variable actual job run-times can be, but there still are some ways to break this down.

These examples are based on how *long* an instance would be up and center on two primary costs: the type of instance and the size of the EBS volume. The details at Amazon are on this link: aws.amazon.com/ec2/pricing

1 extra large high memory instance capable of RNA-seq w/ 200GB storage:   $25/day.
 + 1 worker instance, $10/day/each.

1 basic instance capable of general text manipulation w/ 50GB storage:  $10/day

I am not sure if you will be using GATK or SAM Tools for your processing, but running any variant analysis pipeline would be somewhat similar to an RNA-seq pipeline since it would involve mapping, large data file manipulations, etc. For you particular case, the data storage would be larger than the estimate above, so using the table at Amazon should help you to calculate a figure that reflects your storage needs. It is difficult to say how long any job will run purely based on the size of the inputs, as content and parameter settings have a significant effect on run time, but after the first job, or first time through a complete workflow, if the data is somewhat homogenous, you may be able to estimate a total from there for future runs. Although I or almost anyone else can tell you that these sorts of experiments can pop out with surprises now and then!

Others on the list using a cloud are welcomed to post comments to this thread. Once we get the initial wiki table posted, it will be open to community input, so that this type of actual usage data can be captured. If you or anyone else also wants to send back results meanwhile (post to thread and/or ticket, with experiment & instance detail) please do, here is the new development -> https://trello.com/c/pMbri7QI

Hopefully this helps a little bit! Apologies for not being able to give more detail, this is a tough question to answer with precision for a complete workflow! A pool of case examples is probably the best way to get a bead on this data, so that's part of the goal now.

Jen
Galaxy team

On 4/10/13 9:20 AM, Thompson, Andrew wrote:
Dear Jen.
Yes, that was my problem, I skipped some steps by relying too much on the screencast and ignoring the text.

Now I am reluctant to launch the AMI as I having trouble estimating my usage and costs on AWS - as a new user I have little
idea what to set many of the parameters in the usage calculator. Are there any examples of typical parameters and costs for running Galaxy on the cloud?
My first task is to map about 80 gbp of total paired end reads from genomic DNA from two accessions to a 900 mbp reference genome and
then find SNPs and INDELs. A ball-park figure would be reassuring!

regards
  Andrew
________________________________
From: Jennifer Jackson [jen@bx.psu.edu]
Sent: 09 April 2013 15:11
To: Thompson, Andrew
Cc: 'galaxy-user@lists.bx.psu.edu'
Subject: Re: [galaxy-user] merging fastq files

Hi Andrew,

My first guess is that perhaps the region is not set correctly?
http://wiki.galaxyproject.org/CloudMan/AWS/GettingStarted

See " Step 1: One Time Amazon Setup", subsection 2, where region is " set your AWS Region to US East (Virginia)".
The image in the wiki for step 1.2 is slightly outdated, instead it will look like this:

[cid:part2.09010203.02020409@bx.psu.edu]

Please give this a try and let us know if you continue to have issues.

Thanks!

Jen
Galaxy team

On 4/8/13 3:56 PM, Thompson, Andrew wrote:

Dear Jen
Thanks. I have merged the files and end up with 4 x 47 G fastq files for read mapping to the reference.
It seems this is too much data to analyse on the public main instance if the size limit is 250 G?
So I tried to set up the cloud option following the screencast (http://screencast.g2.bx.psu.edu/cloud/), but when I search for the current AMI name (861460482541/galaxy-cloudman-2011-03-22) it is not found in the list of community AMIs under Amazon's EC2 Management Console. Any ideas why this is not working?
regards
   Andrew




--
Jennifer Hillman-Jackson
Galaxy Support and Training
http://galaxyproject.org


-- 
Jennifer Hillman-Jackson
Galaxy Support and Training
http://galaxyproject.org