Hello, my name is Stavros Sachtouris and I'm developing an object storage backend for Synnefo [1] Object Store (aka Pithos+ [2]). In the context of the OpenMinTeD (Open MINing for TExt and Data) project [3], we are trying to use Galaxy as a workflow engine and we need to set it up on to use our Object Store (Pithos+). Therefore, we need a Pithos+ backend for Galaxy.
Pithos+ API claims compatibility with OpenStack Swift API, but in practice this rarely works. That's why we have developed our own python API library (kamaki [5]). In order to provide Pithos+ access to Galaxy, I had to develop a Pithos+ backend using kamaki. We then deployed a testing Galaxy and patched it with our code and it seems to work well.
I'm sending this message for two reasons: a. to let the Galaxy developers know about our interest in Galaxy and start a cooperation which is hopefully going to be beneficial for both parties. b. to ask for advice and guidelines on how to prepare my first patch and how to ship it for review. Although the patch seems to work, we feel like we are missing further testing.
Bellow you can find more information on who we are, what cloud we are providing, what is OpenMinTeD and how is it related to Galaxy.
Who are we ------------ I work for the Greek Research Network of Technology (GRNET S.A. [3]) as a software developer. The last few years I'm been involved in many cloud-related projects like Synnefo, ~okeanos and OpenMinTeD (more on them later).
Synnefo, Pithos+, ~okeanos --------------------------- Synnefo [1] is an open source cloud software developed and supported by GRNET. It features an OpenStack REST API, many interesting features and it is around for more than 5 years already. Pithos+ is the object storage service of Synnefo, providing object storage with de-duplication, sharing and more. ~okeanos is the largest Synnefo deployment providing thousands of virtual machines and terabytes of storage to academic users in Greece and all over Europe. ~okeanos is also operated and supported by GRNET.
OpenMinTeD ------------- OpenMinTeD is a collaborative project aiming in deploying text and data mining tools to academic communities. Galaxy ahs been selected as the main tool to support workflows. The plan is to run Galaxy on GRNET ~okeanos infrastructure, use Pithos+ object storage as a storage backend and make it collaborate with docker-oriented tools like Mesos/Marathon or Docker Swarm on ~okeanos provisioned VMs.
[1] https://www.synnefo.org [2] https://www.synnefo.org/docs/synnefo/latest/pithos.html [3] https://www.openminted.eu [4] https://www.grnet.gr [5] https://www.synnefo.org/docs/kamaki/latest/
Stavros Sachtouris Software Engineer @ www.grnet.gr