Hi Andrew,
We do not have any estimates posted on the wiki currently for
example usage on the cloud, but this is a good idea and the team is
discussing the best way to add in something like this. The
difficulty comes with how variable actual job run-times can be, but
there still are some ways to break this down.
These examples are based on how *long* an instance would be up and
center on two primary costs: the type of instance and the size of
the EBS volume. The details at Amazon are on this link: aws.amazon.com/ec2/pricing
1 extra large high memory instance capable of RNA-seq
w/ 200GB storage: $25/day.
+ 1 worker instance, $10/day/each.
1 basic instance capable of general text manipulation
w/ 50GB storage: $10/day
I am not sure if you will be using GATK or SAM Tools for your
processing, but running any variant analysis pipeline would be
somewhat similar to an RNA-seq pipeline since it would involve
mapping, large data file manipulations, etc. For you particular
case, the data storage would be larger than the estimate above, so
using the table at Amazon should help you to calculate a figure that
reflects your storage needs. It is difficult to say how long any job
will run purely based on the size of the inputs, as content and
parameter settings have a significant effect on run time, but after
the first job, or first time through a complete workflow, if the
data is somewhat homogenous, you may be able to estimate a total
from there for future runs. Although I or almost anyone else can
tell you that these sorts of experiments can pop out with surprises
now and then!
Others on the list using a cloud are welcomed to post comments to
this thread. Once we get the initial wiki table posted, it will be
open to community input, so that this type of actual usage data can
be captured. If you or anyone else also wants to send back results
meanwhile (post to thread and/or ticket, with experiment &
instance detail) please do, here is the new development ->
https://trello.com/c/pMbri7QI
Hopefully this helps a little bit! Apologies for not being able to
give more detail, this is a tough question to answer with precision
for a complete workflow! A pool of case examples is probably the
best way to get a bead on this data, so that's part of the goal now.
Jen
Galaxy team
On 4/10/13 9:20 AM, Thompson, Andrew
wrote:
Dear Jen.
Yes, that was my problem, I skipped some steps by relying too much on the screencast and ignoring the text.
Now I am reluctant to launch the AMI as I having trouble estimating my usage and costs on AWS - as a new user I have little
idea what to set many of the parameters in the usage calculator. Are there any examples of typical parameters and costs for running Galaxy on the cloud?
My first task is to map about 80 gbp of total paired end reads from genomic DNA from two accessions to a 900 mbp reference genome and
then find SNPs and INDELs. A ball-park figure would be reassuring!
regards
Andrew
________________________________
From: Jennifer Jackson [jen@bx.psu.edu]
Sent: 09 April 2013 15:11
To: Thompson, Andrew
Cc: 'galaxy-user@lists.bx.psu.edu'
Subject: Re: [galaxy-user] merging fastq files
Hi Andrew,
My first guess is that perhaps the region is not set correctly?
http://wiki.galaxyproject.org/CloudMan/AWS/GettingStarted
See " Step 1: One Time Amazon Setup", subsection 2, where region is " set your AWS Region to US East (Virginia)".
The image in the wiki for step 1.2 is slightly outdated, instead it will look like this:
[cid:part2.09010203.02020409@bx.psu.edu]
Please give this a try and let us know if you continue to have issues.
Thanks!
Jen
Galaxy team
On 4/8/13 3:56 PM, Thompson, Andrew wrote:
Dear Jen
Thanks. I have merged the files and end up with 4 x 47 G fastq files for read mapping to the reference.
It seems this is too much data to analyse on the public main instance if the size limit is 250 G?
So I tried to set up the cloud option following the screencast (http://screencast.g2.bx.psu.edu/cloud/), but when I search for the current AMI name (861460482541/galaxy-cloudman-2011-03-22) it is not found in the list of community AMIs under Amazon's EC2 Management Console. Any ideas why this is not working?
regards
Andrew
--
Jennifer Hillman-Jackson
Galaxy Support and Training
http://galaxyproject.org
--
Jennifer Hillman-Jackson
Galaxy Support and Training
http://galaxyproject.org