On Thu, Apr 21, 2011 at 5:43 PM, Assaf Gordon <gordon@cshl.edu> wrote:
Hi Peter,
Peter Cock wrote, On 04/21/2011 11:00 AM:
Is there any built in way to control the Unix priority level (e.g. nice or ionice) used to run tasks? I don't see anything on here, but perhaps I'm looking in the wrong place: https://bitbucket.org/galaxy/galaxy-central/wiki/Config/WebApplicationScalin...
I've been experimenting with 'nice' as well (for other servers), and it seems the consensus is that 'nice' is broken.
Linus puts it in a colorful way here: https://lwn.net/Articles/418739/ "[...] Seriously. Nobody _ever_ does "nice make", unless they are seriously repressed beta-males (eg MIS people who get shouted at when they do system maintenance unless they hide in dark corners and don't get discovered). It just doesn't happen."
I know nice isn't perfect, but in the case of the sys admin setting up Galaxy, we don't have the human laziness problem to overcome: We could ensure all job tasks get run with nice 10 automatically, without the Galaxy users having to do anything special.
One recommended solution is to use "cgroups", which can control CPU, Disk I/O, Memory usage and network load, as explained here: http://broadcast.oreilly.com/2009/06/manage-your-performance-with-cgroups-an...
With "cgroups" there will be no need to change anything in galaxy (just apply a cgroup to the galaxy user, or something similar).
Yes, but I want to treat the Galaxy webserver differently from the compute jobs it launches.
Unfortunately, "cgroups" requires a recent kernel, so it might not be applicable in your case.
-gordon
Are you using cgroups on your Galaxy? Peter