Hi Marius,
thanks for shifting the discussion over to the mailinglist.
Am 19.08.2014 um 13:13 schrieb Marius van den Beek:
Hi everyone,
it’s my first post to the list, so forgive me if I miss something obvious, I tried to get all the information I could so if there was already a discussion I’d be glad if you can point me towards this.
https://trello.com/c/fYB8fMY4/43-docker-in-galaxy-in-docker https://trello.com/c/tm1fPvyq https://trello.com/c/8k55T24F https://github.com/bgruening/galaxy-ipython https://github.com/bgruening/docker-galaxy-stable
The galaxy-docker-stable image can be used to generate self-defined set of Tools with a full working Galaxy instance.
I’m posting to start a discussion about how galaxy is going to handle interactions with Docker.
I know there are already some tools and resources out there that make use of Docker. As I understood, we can already specify Docker as a job destination in the tool xml, along with an image to use (See https://github.com/apetkau/galaxy-hackathon-2014/tree/master/smalt for a detailed example). This is cool, and allows for the controlling of resources, but I think we could take the integration to the next step if we allowed the generation of Docker images in galaxy.
The Tool Shed can, since the latest release, generate you "ready to go" Dockerfiles with all your favorite tools.
I was thinking if there is a trusted baseimage (ie without root access), we could let users install
Can you please elaborate on this a little bit more?
all the packages they need, commit after the install, and let them use this new docker image. This could be beneficial for generating new tools with a sandboxed version of the Galaxy Tool Factory ( http://www.ncbi.nlm.nih.gov/pubmed/23024011).
I am currently working on this ( https://bitbucket.org/mvdbeek/dockertoolfactory), and I would like to have your opinion on how to manage these user-generated Docker images.
Some questions that I came across:
Do we store the committed images in the user’s history (they could potentially become very big!)?
How to display available images to the user? Something like a per-user image registry?
How to identify available images (dataset_id as the tag?)
How to transfer user-generated images to the toolshed?
Is it a good idea at all to store user-generated images in the toolshed?
What do we do if the security of the baseimage is compromised? Obviously
we can blacklist execution of images, but what if somebody installed a dangerous image from the toolshed, and is not aware of this?
To answer these questions it would be good to have some use-cases for your docker tool images. For example I think most of the time an IPython instance would be sufficient. You will only store notebook files in your history (small) and execute them on new datasets.
I would be really interested in your use-cases.
Imho, all this should be taken with care, in the end we would like to have proper tools and images in the toolshed and reusable for many others. I consider IPython Integration and the Toolfactory as bonus and tools mostly for developers. Nothing a biologist should deal with. And we should take care to not over complicate the tool setup/usage.
Ultimately I think this could be a way to have advanced users run their own scripts inside galaxy, to generate their own tools and tool-dependencies inside galaxy, and why not, even have “user-space tools”.
It would bridge the gap between galaxy users and galaxy tool-developers,
so I’m curious what you’re thinking about this.
Absolutely, thanks for bringing this up! Bjoern
Cheers,
Marius
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/