On Tue, May 12, 2015 at 5:12 AM, Pablo Escobar <pescobar001@gmail.com> wrote:
Hi,
We have a test galaxy instance integrated with our SGE cluster. The filesystem is shared in the galaxy server and all the cluster compute nodes. Now we are submitting all the jobs as galaxy user and all the data is stored in the home folder for galaxy user.
We plan to configure galaxy to submit the jobs to SGE as the real user as explained in docs https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#Submitting_J... and we would like that each user store their galaxy data in their personal home folder and not in the galaxy user home folder so each user can keep their private data not accesible to other users in the cluster and also to be able to do proper accounting of how much storage each user is using. Our main concern is about users keeping their data private, the storage usage accounting is something that would be nice but it's not so critical.
Is this setup possible? Any advice or documentation about how to achieve this setup?
This setup would not be doable without significant modifications to Galaxy. The way Galaxy does sharing histories and datasets is that it just keeps one copy of the dataset and uses the database to track who owns what and who has it marked as deleted, purged, etc.... This is not readily compatible with the way you are describing storing data on per user basis. It is definitely on the devteam's radar - this flexibility is needed for instance to enable users to attach S3 buckets to accounts on usegalaxy.org - but I suspect this will be a lot of work. When I was at MSI - we dealt with this limitation by setting up scripts that would look at Galaxy's database and fetch user quotas on a nightly basis and adjust their quota on the filesystem accordingly. If the desire to store data this way is not motivated by usage tracking but instead for enabling convenient filesystem access to Galaxy data - it might be worth looking at the Galaxy FUSE layer maintained by Clare Sloggett here https://github.com/claresloggett/gvl_commandline_utilities. -John
thanks in advance for any help or suggestion. regards, Pablo.
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/