Hi there
At the moment Galaxy's choice of Sun Grid Engine settings are done on a
per-tool basis - that is, you can define different queues and projects
on a per-tool basis. However, I've got two use cases that currently
aren't supported.
1) Per-user settings. The SGEJobRunner potentially has access to user
information (via the job.session_id), and while all jobs get run as user
galaxy, per-user settings could be simulated by mapping Galaxy users to
SGE projects. Then the SGE admin can do load-balancing, etc on a
per-project basis. For a site with a relatively small and static set of
users this could work. Alternately you need to map Galaxy users to SGE
users - this is much trickier, requiring elevated privileges and thus
probably a separate job runner daemon.
2) Queue selection. Again this could be done on a per-tool basis, but
usage patterns don't always support that. So for example, our site has
two queues - a long running one and a short running one. Certain
resources are reserved for only the short running queue's usage. How to
select a different queue for jobs raises a bit of a design problem for
me - effectively you're telling Galaxy "I want to run this workflow, but
with this extra parameter" - its almost like having a "meta parameter"
since clearly you don't want to have queue selection as an input for any
particular tool.
So I'd be interested in the Galaxy team's input as to how best to
address these two use cases.
Thanks,
Peter