It worked like a charm Thanks --R On Fri, Aug 9, 2013 at 12:32 PM, Nate Coraor <nate@bx.psu.edu> wrote:
On Aug 9, 2013, at 1:16 PM, Rodolfo Aramayo wrote:
Dear Nate,
Thanks
Just to be sure I understand this correctly
I now have in my 'universe' file the following:
##### # ---- HTTP Server ----------------------------------------------------------
# Configuration of the internal HTTP server.
[server:main]
# job_manager = manager <I ASSUME THIS IS CORRECT because I get errors when it is not commented
################################################################################
[server:manager] use = egg:Paste#http port = 8200 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5
################################################################################
[server:handler0] use = egg:Paste#http port = 8300 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5
[server:handler1] use = egg:Paste#http port = 8301 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5
...snip...
[server:handler19] use = egg:Paste#http port = 8319 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5
# [server:handler20] # use = egg:Paste#http # port = 8320 # host = 127.0.0.1 # use_threadpool = true # threadpool_workers = 5
################################################################################ # The internal HTTP server to use. Currently only Paste is provided. This # option is required. use = egg:Paste#http
# The port on which to listen. #port = 8080
# The address on which to listen. By default, only listen to localhost (Galaxy # will not be accessible over the network). Use '0.0.0.0' to listen on all # available network interfaces. #host = 127.0.0.1 host = 0.0.0.0
# Use a threadpool for the web server instead of creating a thread for each # request. use_threadpool = True
# Number of threads in the web server thread pool. threadpool_workers = 200
################################################################################ [server:web0] use = egg:Paste#http port = 8400 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10
[server:web1] use = egg:Paste#http port = 8401 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10
...snip...
[server:web11] use = egg:Paste#http port = 8411 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10
[server:web12] use = egg:Paste#http port = 8412 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10
################################################################################ # ---- Filters --------------------------------------------------------------
# Filters sit between Galaxy and the HTTP server. ...
#####
And in my job_conf.xml:
<handler id="handler0" tags="handlers"/> <handler id="handler1" tags="handlers"/> ...snip... <handler id="handler18" tags="handlers"/> <handler id="handler19" tags="handlers"/>
#####
Is this correct?
Rodolfo,
Some of this confusion might be because you have the (documented) options from the original [server:main] section orphaned underneath [server:handler19], which already has those options set. In an ini file, sections begin with [name] and all key/value pairs underneath that heading are assigned to [name] until another section heading is encountered. Removing all comments, that means your config would look like this:
[server:main]
[server:manager] use = egg:Paste#http port = 8200 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5
[server:handler0] use = egg:Paste#http port = 8300 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5
[server:handler1] use = egg:Paste#http port = 8301 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5
[server:handler19] use = egg:Paste#http port = 8319 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 use = egg:Paste#http host = 0.0.0.0 use_threadpool = True threadpool_workers = 200
[server:web0] use = egg:Paste#http port = 8400 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10
[server:web1] use = egg:Paste#http port = 8401 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10
[server:web11] use = egg:Paste#http port = 8411 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10
[server:web12] use = egg:Paste#http port = 8412 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10
-----------------------------
Remove [server:main], the entire [server:manager] section, and those extra options under [server:handler19], and you should be good:
[server:handler0] use = egg:Paste#http port = 8300 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5
[server:handler1] use = egg:Paste#http port = 8301 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5
[server:handler19] use = egg:Paste#http port = 8319 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5
[server:web0] use = egg:Paste#http port = 8400 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10
[server:web1] use = egg:Paste#http port = 8401 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10
[server:web11] use = egg:Paste#http port = 8411 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10
[server:web12] use = egg:Paste#http port = 8412 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10
-----------------------------
FWIW, this seems like a lot of processes to start, we don't run nearly this many for the Galaxy public site.
Also, I guess the part I do not understand is what is the magic formula between the number of threadpool_workers as configured here:
# Use a threadpool for the web server instead of creating a thread for each # request. use_threadpool = True
# Number of threads in the web server thread pool. threadpool_workers = 200
and those configured on the web servers and on the server handlers?
Hopefully this is clearer now due to my reply above. Each threadpool_workers option applies only to the [server:...] section that it is defined beneath.
--nate
Thanks
--Rodolfo