On Mon, Jun 07, 2010 at 03:57:29PM +1000, Dennis Gascoigne wrote:
I know it must be possible to make Galaxy use more of the resources available to it but I haven't been able to figure out how. If galaxy is calling python scripts, it only seems to use a single processor to do so. We have an 8 core machine and if other binaries are spawned then they get other processors no problems, but any of the python operations though seem to operate on the same processor no matter how many of these 'python based' jobs are running i.e. If i run 5 tools simultaneously, I would expect allocation to make use of all available processing resources.
Is there some config to make this happen? I am sure I am missing something basic.
There was a great presentation about just how to do this at dev con 2010: https://docs.google.com/viewer?url=http://bitbucket.org/galaxy/galaxy-centra... In short, you run multiple galaxy 'runner' instances that federate with a single 'web' instance. This is required as the local runner uses the same python VM to run tasks and python VMs have a single-thread chokepoint. -- Ry4an Brase 612-626-6575 University of Minnesota Supercomputing Institute for Advanced Computational Research http://www.msi.umn.edu