On Tue, Jun 8, 2010 at 7:38 AM, Dennis Gascoigne <dennis.gascoigne@gmail.com> wrote:

I am seeing two python processes. One is my runner - maxing at appr. 100%, the other is my web process just ticking along at about 5%. There are 5 instances of SAM to Interval. I would expect to see 5 processes, preferably taking multiple processors.

On Mon, Jun 7, 2010 at 11:35 PM, Nate Coraor <nate@bx.psu.edu> wrote:

James Taylor wrote:

Nate, shouldn't python tools run as different processes scale out to multiple processors without any special configuration though? E.g. running 5 gops jobs in parallel should use multiple processes. Is there something that prevents this?

Oh, yes, I suppose I failed to read this email properly.  It should definitely be starting separate processes, which would remove the GIL's one core limitation.  Dennis, are you seeing multiple python processes when you run these tools?

--nate

On Jun 7, 2010, at 9:29 AM, Nate Coraor wrote:

Dennis Gascoigne wrote:

I know it must be possible to make Galaxy use more of the resources available to it but I haven't been able to figure out how. If galaxy is calling python scripts, it only seems to use a single processor to do so. We have an 8 core machine and if other binaries are spawned then they get other processors no problems, but any of the python operations though seem to operate on the same processor no matter how many of these 'python based' jobs are running i.e. If i run 5 tools simultaneously, I would expect allocation to make use of all available processing resources.
Is there some config to make this happen? I am sure I am missing something basic.

Hi Dennis,

This is something I've referred to on the ProductionServer wiki page, but have not yet documented.  I also covered it in my Developer Conference talk (slides: http://usegalaxy.org/dev2010 ).

However, probably the best place to find it right now is the conversation on the list from last month between Davide Cittaro and I:

http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-May/002634.html

I'd suggest copying universe_wsgi.ini in both instances rather than moving in one, as I originally suggested.  Otherwise, egg-checking routines will fail because of the missing config file.

--nate

Cheers
------------------------------------------------------------------------
_______________________________________________
galaxy-dev mailing list
galaxy-dev@lists.bx.psu.edu
http://lists.bx.psu.edu/listinfo/galaxy-dev

_______________________________________________
galaxy-dev mailing list
galaxy-dev@lists.bx.psu.edu
http://lists.bx.psu.edu/listinfo/galaxy-dev

--
Dennis Gascoigne
0407 639 995
dennis.gascoigne@gmail.com