Galaxy tools' priority level (nice)
Hi all, Our Galaxy is running on a shared server which will sometimes be running other computationally demanding jobs (outside of Galaxy). In some cases I'd like these to have priority, perhaps by having Galaxy run the tool child processes at a nice level of 10 (say). Is there any built in way to control the Unix priority level (e.g. nice or ionice) used to run tasks? I don't see anything on here, but perhaps I'm looking in the wrong place: https://bitbucket.org/galaxy/galaxy-central/wiki/Config/WebApplicationScalin... Thanks, Peter
Hi Peter, Peter Cock wrote, On 04/21/2011 11:00 AM:
Is there any built in way to control the Unix priority level (e.g. nice or ionice) used to run tasks? I don't see anything on here, but perhaps I'm looking in the wrong place: https://bitbucket.org/galaxy/galaxy-central/wiki/Config/WebApplicationScalin...
I've been experimenting with 'nice' as well (for other servers), and it seems the consensus is that 'nice' is broken. Linus puts it in a colorful way here: https://lwn.net/Articles/418739/ "[...] Seriously. Nobody _ever_ does "nice make", unless they are seriously repressed beta-males (eg MIS people who get shouted at when they do system maintenance unless they hide in dark corners and don't get discovered). It just doesn't happen." One recommended solution is to use "cgroups", which can control CPU, Disk I/O, Memory usage and network load, as explained here: http://broadcast.oreilly.com/2009/06/manage-your-performance-with-cgroups-an... With "cgroups" there will be no need to change anything in galaxy (just apply a cgroup to the galaxy user, or something similar). Unfortunately, "cgroups" requires a recent kernel, so it might not be applicable in your case. -gordon
On Thu, Apr 21, 2011 at 5:43 PM, Assaf Gordon <gordon@cshl.edu> wrote:
Hi Peter,
Peter Cock wrote, On 04/21/2011 11:00 AM:
Is there any built in way to control the Unix priority level (e.g. nice or ionice) used to run tasks? I don't see anything on here, but perhaps I'm looking in the wrong place: https://bitbucket.org/galaxy/galaxy-central/wiki/Config/WebApplicationScalin...
I've been experimenting with 'nice' as well (for other servers), and it seems the consensus is that 'nice' is broken.
Linus puts it in a colorful way here: https://lwn.net/Articles/418739/ "[...] Seriously. Nobody _ever_ does "nice make", unless they are seriously repressed beta-males (eg MIS people who get shouted at when they do system maintenance unless they hide in dark corners and don't get discovered). It just doesn't happen."
I know nice isn't perfect, but in the case of the sys admin setting up Galaxy, we don't have the human laziness problem to overcome: We could ensure all job tasks get run with nice 10 automatically, without the Galaxy users having to do anything special.
One recommended solution is to use "cgroups", which can control CPU, Disk I/O, Memory usage and network load, as explained here: http://broadcast.oreilly.com/2009/06/manage-your-performance-with-cgroups-an...
With "cgroups" there will be no need to change anything in galaxy (just apply a cgroup to the galaxy user, or something similar).
Yes, but I want to treat the Galaxy webserver differently from the compute jobs it launches.
Unfortunately, "cgroups" requires a recent kernel, so it might not be applicable in your case.
-gordon
Are you using cgroups on your Galaxy? Peter
Peter Cock wrote, On 04/21/2011 12:59 PM:
On Thu, Apr 21, 2011 at 5:43 PM, Assaf Gordon <gordon@cshl.edu> wrote:
Hi Peter,
Peter Cock wrote, On 04/21/2011 11:00 AM:
Is there any built in way to control the Unix priority level (e.g. nice or ionice) used to run tasks? I don't see anything on here, but perhaps I'm looking in the wrong place: https://bitbucket.org/galaxy/galaxy-central/wiki/Config/WebApplicationScalin...
I've been experimenting with 'nice' as well (for other servers), and it seems the consensus is that 'nice' is broken.
Linus puts it in a colorful way here: https://lwn.net/Articles/418739/ "[...] Seriously. Nobody _ever_ does "nice make", unless they are seriously repressed beta-males (eg MIS people who get shouted at when they do system maintenance unless they hide in dark corners and don't get discovered). It just doesn't happen."
I know nice isn't perfect, but in the case of the sys admin setting up Galaxy, we don't have the human laziness problem to overcome: We could ensure all job tasks get run with nice 10 automatically, without the Galaxy users having to do anything special.
The article goes to explain further why "nice" is not the best option, the above quote was just a jest. But you are right that it's better than nothing.
One recommended solution is to use "cgroups", which can control CPU, Disk I/O, Memory usage and network load, as explained here: http://broadcast.oreilly.com/2009/06/manage-your-performance-with-cgroups-an...
With "cgroups" there will be no need to change anything in galaxy (just apply a cgroup to the galaxy user, or something similar).
Yes, but I want to treat the Galaxy webserver differently from the compute jobs it launches.
I'm certain there's a way to set limits per processes, not just per users - but I haven't experimented with it.
Unfortunately, "cgroups" requires a recent kernel, so it might not be applicable in your case.
Are you using cgroups on your Galaxy?
Only on my development system - unfortunately my production serve uses CentOS 5 with an old kernel and no cgroup support. -gordon
On Thu, Apr 21, 2011 at 9:27 PM, Assaf Gordon <gordon@cshl.edu> wrote:
Peter Cock wrote, On 04/21/2011 12:59 PM:
On Thu, Apr 21, 2011 at 5:43 PM, Assaf Gordon <gordon@cshl.edu> wrote:
I know nice isn't perfect, but in the case of the sys admin setting up Galaxy, we don't have the human laziness problem to overcome: We could ensure all job tasks get run with nice 10 automatically, without the Galaxy users having to do anything special.
The article goes to explain further why "nice" is not the best option, the above quote was just a jest. But you are right that it's better than nothing.
:)
Unfortunately, "cgroups" requires a recent kernel, so it might not be applicable in your case.
Are you using cgroups on your Galaxy?
Only on my development system - unfortunately my production serve uses CentOS 5 with an old kernel and no cgroup support.
Our production server is on CentOS 5.4, which is occasionally annoying. Peter
participants (2)
-
Assaf Gordon -
Peter Cock