Galaxy instance with multiple partitions
Dear all, We are in the middle of setting up a local Galaxy instance on CentOS, but we have a problem with disk space. In particular, we do not have a single large disk partition but, instead, many medium sized ext4 partitions. Thus, we were wondering if it would be possible for Galaxy to use multiple disk partitions instead of a single one for data and intermediate files. Ideally, it would be great if we could allocate users to various groups, which in turn are allocated separate disk partitions. Less ideal, but still something we could work with, is to ask users to copy/move to/from the other partitions while one remains the "working one" for Galaxy. Another idea which I think wouldn't work since it would interfere with Galaxy is to move files (i.e., via a cron job) to another partition when a job is complete and then add a symbolic link to it from the "working" partition. Are there any suggestions on how I can get around this? Or are we stuck with either a single partition or somehow reformatting it and merging the partitions (using an xfs file system, etc.) Thank you! Ray
Hi Raymond, Galaxy as a very sophisticated support for this called ObjectStore. Please have a look at the API and the Wiki (the wiki can be improved so much on this). https://wiki.galaxyproject.org/ObjectStore http://galaxy.readthedocs.org/en/stable/lib/galaxy.objectstore.html I can also send you our configuration using a hierarchical object store. Cheers, Bjoern
Hi Bjoern, Thank you for the prompt reply! I didn't know what to look for, but now that you mention it, I can search for "ObjectStore" in the configuration file... Looking at object_store_conf.xml.sample, it looks like I can specify various partitions and when data is stored. I presume when it comes time to store the data, the location is determined and some look-up table is updated? That's indeed what I was looking for! Is "object store" in Galaxy's documentation not the same as "ObjectStore"? I just searched for this term on Google and it seems to be a commercial database [https://en.wikipedia.org/wiki/ObjectStore]. Thank you for the offer to send me your configuration! Let me play around with it a bit and see how hard it is, first... Thanks! Ray On Tue, Aug 11, 2015 at 4:20 PM, Björn Grüning <bjoern.gruening@gmail.com> wrote:
Hi!
Hi Bjoern,
Thank you for the prompt reply!
Your welcome!
Yes!
This is different than the Galaxy term. "Galaxy" has always trouble with google and naming ;)
Thank you for the offer to send me your configuration! Let me play around with it a bit and see how hard it is, first...
Ciao, Bjoern
participants (2)
-
Björn Grüning
-
Raymond Wan