need help on extra servers
Hi again, I'm trying to implement what has been showed at the devcon, the extra servers stuff... There's only a slide about that (in the scalable galaxy talk), but seems should be pretty easy by adding [server:ROLE_ID] port = XXXX where role is... web or runner? Is this the correct syntax: [server:web_0] port = 8081 [server:web_1] port = 8082 [server:runner_0] port = 8100 [server:runner_1] port = 8101 [server:runner_2] port = 8102 [server:runner_3] port = 8103 [server:runner_4] port = 8104 Anyway, added those to the universe file, netstat says there are not additional ports opened... Thanks d /* Davide Cittaro Cogentech - Consortium for Genomic Technologies via adamello, 16 20139 Milano Italy tel.: +39(02)574303007 e-mail: davide.cittaro@ifom-ieo-campus.it */
Davide Cittaro wrote:
Hi again, I'm trying to implement what has been showed at the devcon, the extra servers stuff...
Hi! Great meeting you at the conference.
There's only a slide about that (in the scalable galaxy talk), but seems should be pretty easy by adding
[server:ROLE_ID] port = XXXX
where role is... web or runner? Is this the correct syntax:
[server:web_0] port = 8081 [server:web_1] port = 8082
[server:runner_0] port = 8100 [server:runner_1] port = 8101 [server:runner_2] port = 8102 [server:runner_3] port = 8103 [server:runner_4] port = 8104
Anyway, added those to the universe file, netstat says there are not additional ports opened...
Those were slimmed down examples for slide-friendliness. You'll need to make two copies of your config file. I do this as: % cp universe_wsgi.ini universe_wsgi.webapp.ini % mv universe_wsgi.ini universe_wsgi.runner.ini Then edit universe_wsgi.webapp.ini, and add as many web servers as you need: [server:web0] use = egg:Paste#http port = 9980 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 [server:web1] use = egg:Paste#http port = 9981 host = 127.0.0.1 use_threadpool = true threadpool_workers = 10 These should completely replace the existing [server:main] section. Then set the following in the [app:main] section: enable_job_running = False Save and exit. Next, open universe_wsgi.runner.ini and set the server name and port: [server:runner0] use = egg:Paste#http port = 9880 host = 127.0.0.1 use_threadpool = true threadpool_workers = 5 Then under [app:main], set: enable_job_running = True track_jobs_in_database = True Save and exit. Now that it's configured, you need to start and stop each server. This is best done with a script, but the individual commands for the examples shown above will look like this: python ./scripts/paster.py serve universe_wsgi.webapp.ini --server-name=test_web0 --pid-file=test_web0.pid --log-file=test_web0.log --daemon python ./scripts/paster.py serve universe_wsgi.webapp.ini --server-name=test_web1 --pid-file=test_web1.pid --log-file=test_web1.log --daemon python ./scripts/paster.py serve universe_wsgi.runner.ini --server-name=test_runner0 --pid-file=test_runner0.pid --log-file=test_runner0.log --daemon To make this easier, I just committed a 'multiprocess.sh' in Galaxy's contrib/ subdirectory that should be copied, modified for the number of servers and names you used, and run like 'run.sh'. That's the short version, longer documentation in the wiki is forthcoming. --nate
Thanks
d /* Davide Cittaro
Cogentech - Consortium for Genomic Technologies via adamello, 16 20139 Milano Italy
tel.: +39(02)574303007 e-mail: davide.cittaro@ifom-ieo-campus.it <mailto:davide.cittaro@ifom-ieo-campus.it> */
------------------------------------------------------------------------
_______________________________________________ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
Hi Nate,
Those were slimmed down examples for slide-friendliness. You'll need to make two copies of your config file. I do this as:
% cp universe_wsgi.ini universe_wsgi.webapp.ini % mv universe_wsgi.ini universe_wsgi.runner.ini
Doh! I forgot you told me that during the last session...
To make this easier, I just committed a 'multiprocess.sh' in Galaxy's contrib/ subdirectory that should be copied, modified for the number of servers and names you used, and run like 'run.sh'.
That's the short version, longer documentation in the wiki is forthcoming.
Great, thanks, trying it ASAP. d /* Davide Cittaro Cogentech - Consortium for Genomic Technologies via adamello, 16 20139 Milano Italy tel.: +39(02)574303007 e-mail: davide.cittaro@ifom-ieo-campus.it */
On May 21, 2010, at 9:40 AM, Davide Cittaro wrote:
Great, thanks, trying it ASAP.
Doesn't work as it looks for universe_wgsi.ini while initialization in lib/galaxy/eggs/__init__.py: class GalaxyConfig( object ): config_file = os.path.join( galaxy_dir, "universe_wsgi.ini" ) always_conditional = ( 'GeneTrack', 'pysam' ) def __init__( self ): self.config = ConfigParser.ConfigParser() if self.config.read( GalaxyConfig.config_file ) == []: raise Exception( "error: unable to read Galaxy config from %s" % GalaxyConfig.config_file ) I'm now looking where and how the config file is passed to galaxy from command line... d
d
/* Davide Cittaro
Cogentech - Consortium for Genomic Technologies via adamello, 16 20139 Milano Italy
tel.: +39(02)574303007 e-mail: davide.cittaro@ifom-ieo-campus.it */
_______________________________________________ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
/* Davide Cittaro Cogentech - Consortium for Genomic Technologies via adamello, 16 20139 Milano Italy tel.: +39(02)574303007 e-mail: davide.cittaro@ifom-ieo-campus.it */
On May 21, 2010, at 10:13 AM, Davide Cittaro wrote:
I'm now looking where and how the config file is passed to galaxy from command line...
It works after this: $ hg diff lib/galaxy/eggs/__init__.py diff -r 48432330228e lib/galaxy/eggs/__init__.py --- a/lib/galaxy/eggs/__init__.py Fri May 14 10:37:36 2010 -0400 +++ b/lib/galaxy/eggs/__init__.py Fri May 21 10:30:59 2010 +0200 @@ -288,7 +288,7 @@ raise EggNotFetchable( missing ) class GalaxyConfig( object ): - config_file = os.path.join( galaxy_dir, "universe_wsgi.ini" ) + config_file = os.path.join( galaxy_dir, sys.argv[2] ) always_conditional = ( 'GeneTrack', 'pysam' ) def __init__( self ): self.config = ConfigParser.ConfigParser() Of course, this assumes that the config file is always the second argument (which is seems mandatory to me...) d /* Davide Cittaro Cogentech - Consortium for Genomic Technologies via adamello, 16 20139 Milano Italy tel.: +39(02)574303007 e-mail: davide.cittaro@ifom-ieo-campus.it */
participants (2)
-
Davide Cittaro
-
Nate Coraor