On Mon, Jan 26, 2015 at 12:28 PM, Brad Chapman <chapmanb@fastmail.com> wrote:

Enis, John and all;
I spotted ansible-cloudman on GitHub today which reminded me I've been
meaning to write about the approach we setup late last year to run bcbio
on AWS. It uses elasticluster (https://github.com/gc3-uzh-ch/elasticluster)
which has the advantage of being all Ansible scripts and bootstrapping
from standard images -- so no more making AMIs. It also uses SLURM
instead of SGE, which is a nice change.

We wrote an interface that automates all of the stuff you need to setup
on AWS: IAM users, VPCs and what not. It is a pretty streamlined process
from the command line, including specifying the cluster size and
stopping/starting it:

https://bcbio-nextgen.readthedocs.org/en/latest/contents/cloud.html#aws-setup

I also wrote up some benchmarking work using to give an idea of using
it in practice:

http://bcb.io/2014/12/19/awsbench/

All of the code is here:

https://github.com/chapmanb/bcbio-nextgen-vm

As always, happy to overlap/share with whatever y'all decide to
do. We could make bcbio specific stuff optional as needed, although it
is pretty lightweight -- just the driver scripts and a Docker image of
bcbio. It's basically a ready to use cluster with this little extra
added so hopefully could be useful for future plans with CloudMan.

Hope this is useful,
Brad
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/