Peter van Heusden wrote:
Hi there
At the moment Galaxy's choice of Sun Grid Engine settings are done on a per-tool basis - that is, you can define different queues and projects on a per-tool basis. However, I've got two use cases that currently aren't supported.
- Per-user settings. The SGEJobRunner potentially has access to user
information (via the job.session_id), and while all jobs get run as user galaxy, per-user settings could be simulated by mapping Galaxy users to SGE projects. Then the SGE admin can do load-balancing, etc on a per-project basis. For a site with a relatively small and static set of users this could work. Alternately you need to map Galaxy users to SGE users - this is much trickier, requiring elevated privileges and thus probably a separate job runner daemon.
Hi Peter,
There's an issue in our tracker to implement functionality allowing jobs to run on the cluster as real users instead of the Galaxy user:
http://bitbucket.org/galaxy/galaxy-central/issue/106
Once implemented, you could then define which resources used by which users directly in Grid Engine. I think this would be the cleanest way to do #1.
- Queue selection. Again this could be done on a per-tool basis, but
usage patterns don't always support that. So for example, our site has two queues - a long running one and a short running one. Certain resources are reserved for only the short running queue's usage. How to select a different queue for jobs raises a bit of a design problem for me - effectively you're telling Galaxy "I want to run this workflow, but with this extra parameter" - its almost like having a "meta parameter" since clearly you don't want to have queue selection as an input for any particular tool.
So I'd be interested in the Galaxy team's input as to how best to address these two use cases.
This has been a pretty difficult one to define, since it's almost impossible to look at a job and decide how long it's going to run before you run it.
Your solution of giving the users the choice is interesting, although would be a pretty site-specific implementation, since that choice may be a queue, a project, a different cell, or just different qsub parameters.
--nate
Thanks, Peter
galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev