Thanks I think that helps!
Cheers Jochen
On 09.07.2018 17:36, Martin Čech wrote:
Another recently updated resource for uwsgi is the training slides:
https://galaxyproject.github.io/dagobah-training/2018-gccbosc/10-uwsgi/uw...
M.
On Mon, Jul 9, 2018 at 11:31 AM Peter Briggs <peter.briggs(a)manchester.ac.uk>
wrote:
> Hello Jochen
>
> I'm not aware of a tutorial but here are some observations from my (very)
> recent experiences upgrading a local paste-based 17.09 instance to a
> uwsgi-based 18.05 configuration. It's not comprehensive because the
> configuration will depend on decisions you need to make based on your local
> setup, but it might be helpful as a starting point.
>
> Before beginning:
>
> -- Commit to migrating from galaxy.ini to galaxy.yml (if not done
> previously)
>
> Even though its possible to stick with the old galaxy.ini file, it seems
> best to move to the new format - my feeling is, probably the migration will
> have to happen at some point anyway, and also the documentation is more
> focussed on the newer galaxy.yml version.
>
> -- Decide on how you're going to handle job submission (essentially, how
> you're going to replace the old job handler processes).
>
> There are a number of possibilities outlined in the documentation here:
>
>
>
https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#deploy...
>
> It seems important to figure this out at the beginning because it impacts
> a lot of the configuration specifics that follow. For our setup case "uwsgi
> + mules" looked like the most appropriate:
>
>
>
https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#uwsgi-...
>
> Then once you're ready to start (nb backing up the database etc before
> starting are implicitly assumed!):
>
> -- Update the Galaxy source code
>
> The standard recipe for this seems to work okay i.e.
>
> git fetch origin && git checkout release_18.05 && git pull --ff-only
> origin release_18.05
>
> (You'll probably also need to run scripts/common_startup.sh to update the
> Python packages, and scripts/manage_db.py to update the database schema.)
>
> -- Create galaxy.yml and configure the "galaxy" section:
>
> You can make a galaxy.yml from the sample version and go through the
> "galaxy" section replicating the settings from galaxy.ini. (The settings
in
> the "uwsgi" section can wait at this stage.)
> Don't forget to move galaxy.ini out of the way once you're done.
>
> -- Configure the "uwsgi" section of galaxy.yml:
>
> The options for various scenarios are given in the "Deployment/uwsgi"
> section:
>
>
https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#uwsgi
>
> but the required settings seem to be spread out between different sections
> so (for me) it felt like it took some piecing together to get it to work
> when I first tried this.
>
> (Also I was thrown a little by the "socket" parameter not having a
> placeholder in sample galaxy.yml.)
>
> The core uwsgi options are here:
>
>
>
https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#config...
>
> then you need to set other options dependent on:
>
> 1. how job submission is handled (e.g. options for the "uwsgi + mules"
> setup in my case)
> 2. how you're proxying Galaxy (for Apache or nginx it looks like it's just
> setting the uwsgi "socket" option, unless you're serving Galaxy from a
URL
> prefix in which case there are a few more options needed - these are
> covered in the documentation.)
>
> -- Fix your job_conf.xml file
>
> If you have an existing job_conf.xml file in your config directory then
> you might need to update that too (specifically the "handlers" section)
> depending on the job submission handling you've decided to use.
>
> -- Use supervisor to manage your uwsgi processes
>
> The docs for this are here:
>
>
>
https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#starti...
>
> Essentially supervisor replaces "run.sh", so this becomes how you start
> and stop Galaxy (e.g. "supervisorctl start galaxy:" replaces
> "GALAXY_RUN_ALL=1 sh run.sh --daemon".)
>
> I think this is quite straightforward from the docs but again you need to
> tailor it for how you're handling job submission (e.g the "uwsgi +
mules"
> doesn't need any handlers defining, but it looks like other strategies
> needs extra stuff defining here).
>
> -- Set up the proxying
>
> Assuming you're proxing via either Apache or nginx (we're using nginx),
> these are covered in the relevant sections:
>
>
https://docs.galaxyproject.org/en/release_18.05/admin/apache.html
>
https://docs.galaxyproject.org/en/release_18.05/admin/nginx.html
>
> The main difference I noticed for nginx is that you no longer need to use
> the "upstream"/"proxy_pass" directives, as nginx can talk
directly to uwsgi
> using a "uwsgi_pass" directive that connects to the socket defined in the
> "uwsgi" section of galaxy.yml previously.
>
> (There's also a "gotcha" if you're nginx configuration also tries
to serve
> "static/scripts/packed", which needs to be removed - this is noted in the
> "breaking changes" for the 18.05 docs but is easy to miss.)
>
> Hopefully this is some help. There's lots of documentation, which is a
> good thing but which can sometimes make it harder to see where to start.
>
> Good luck with the update,
>
> Best wishes
>
> Peter
>
> --
> Peter Briggs peter.briggs(a)manchester.ac.uk
> Bioinformatics Core Facility University of Manchester
> B.1083 Michael Smith Bldg Tel: (0161) 2751482
>
>
> ________________________________________
> From: galaxy-dev [galaxy-dev-bounces(a)lists.galaxyproject.org] on behalf
> of Jochen Bick [jochen.bick(a)usys.ethz.ch]
> Sent: Monday, July 09, 2018 12:49 PM
> To: galaxy-dev(a)lists.galaxyproject.org
> Subject: [galaxy-dev] From Paste to uWSGI
>
> Hi all,
>
> we would like to update to 18.05 and thereby also update our Paste Server.
>
>
https://docs.galaxyproject.org/en/latest/admin/scaling.html
>
> Is there any tutorial on how to update from Paste to uWSGI?
>
>
> Cheers Jochen
>
> --
> ETH Zurich
> *Jochen Bick*
> Animal Physiology
> Institute of Agricultural Sciences
> Postal address: Universitätstrasse 2
>
<
https://maps.google.com/?q=Universit%C3%A4tstrasse+2&entry=gmail&...
> / LFW B 58.1
> 8092 Zurich, Switzerland
> Office: Eschikon 27
> 8315 Lindau, Switzerland
>
> Phone +41 52 354 92 06 <+41%2052%20354%2092%2006>
> jochen.bick(a)usys.ethz.ch <mailto:jochen.bick@usys.ethz.ch>
>
www.ap.ethz.ch
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client. To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>
https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>
http://galaxyproject.org/search/
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client. To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>
https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>
http://galaxyproject.org/search/
--
ETH Zurich
*Jochen Bick*
Animal Physiology
Institute of Agricultural Sciences
Postal address: Universitätstrasse 2 / LFW B 58.1
Office: Tannenstrasse 1 / TAN D 6.2
8092 Zurich, Switzerland
Phone +41 44 632 28 25
jochen.bick(a)usys.ethz.ch <mailto:jochen.bick@usys.ethz.ch>