Hello Jochen

I'm not aware of a tutorial but here are some observations from my (very) recent experiences upgrading a local paste-based 17.09 instance to a uwsgi-based 18.05 configuration. It's not comprehensive because the configuration will depend on decisions you need to make based on your local setup, but it might be helpful as a starting point.

Before beginning:

-- Commit to migrating from galaxy.ini to galaxy.yml (if not done previously)

Even though its possible to stick with the old galaxy.ini file, it seems best to move to the new format - my feeling is, probably the migration will have to happen at some point anyway, and also the documentation is more focussed on the newer galaxy.yml version.

-- Decide on how you're going to handle job submission (essentially, how you're going to replace the old job handler processes).

There are a number of possibilities outlined in the documentation here:

https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#deployment-options

It seems important to figure this out at the beginning because it impacts a lot of the configuration specifics that follow. For our setup case "uwsgi + mules" looked like the most appropriate:

https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#uwsgi-for-web-serving-with-mules-as-job-handlers

Then once you're ready to start (nb backing up the database etc before starting are implicitly assumed!):

-- Update the Galaxy source code

The standard recipe for this seems to work okay i.e.

git fetch origin && git checkout release_18.05 && git pull --ff-only origin release_18.05

(You'll probably also need to run scripts/common_startup.sh to update the Python packages, and scripts/manage_db.py to update the database schema.)

-- Create galaxy.yml and configure the "galaxy" section:

You can make a galaxy.yml from the sample version and go through the "galaxy" section replicating the settings from galaxy.ini. (The settings in the "uwsgi" section can wait at this stage.)
Don't forget to move galaxy.ini out of the way once you're done.

-- Configure the "uwsgi" section of galaxy.yml:

The options for various scenarios are given in the "Deployment/uwsgi" section:

https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#uwsgi

but the required settings seem to be spread out between different sections so (for me) it felt like it took some piecing together to get it to work when I first tried this.

(Also I was thrown a little by the "socket" parameter not having a placeholder in sample galaxy.yml.)

The core uwsgi options are here:

https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#configuration-common-to-all-uwsgi-deployment-styles

then you need to set other options dependent on:

1. how job submission is handled (e.g. options for the "uwsgi + mules" setup in my case)
2. how you're proxying Galaxy (for Apache or nginx it looks like it's just setting the uwsgi "socket" option, unless you're serving Galaxy from a URL prefix in which case there are a few more options needed - these are covered in the documentation.)

-- Fix your job_conf.xml file

If you have an existing job_conf.xml file in your config directory then you might need to update that too (specifically the "handlers" section) depending on the job submission handling you've decided to use.

-- Use supervisor to manage your uwsgi processes

The docs for this are here:

https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#starting-and-stopping

Essentially supervisor replaces "run.sh", so this becomes how you start and stop Galaxy (e.g. "supervisorctl start galaxy:" replaces "GALAXY_RUN_ALL=1 sh run.sh --daemon".)

I think this is quite straightforward from the docs but again you need to tailor it for how you're handling job submission (e.g the "uwsgi + mules" doesn't need any handlers defining, but it looks like other strategies needs extra stuff defining here).

-- Set up the proxying

Assuming you're proxing via either Apache or nginx (we're using nginx), these are covered in the relevant sections:

https://docs.galaxyproject.org/en/release_18.05/admin/apache.html
https://docs.galaxyproject.org/en/release_18.05/admin/nginx.html

The main difference I noticed for nginx is that you no longer need to use the "upstream"/"proxy_pass" directives, as nginx can talk directly to uwsgi using a "uwsgi_pass" directive that connects to the socket defined in the "uwsgi" section of galaxy.yml previously.

(There's also a "gotcha" if you're nginx configuration also tries to serve "static/scripts/packed", which needs to be removed - this is noted in the "breaking changes" for the 18.05 docs but is easy to miss.)

Hopefully this is some help. There's lots of documentation, which is a good thing but which can sometimes make it harder to see where to start.

Good luck with the update,

Best wishes

Peter

--
Peter Briggs peter.briggs@manchester.ac.uk
Bioinformatics Core Facility University of Manchester
B.1083 Michael Smith Bldg Tel: (0161) 2751482


________________________________________
From: galaxy-dev [galaxy-dev-bounces@lists.galaxyproject.org] on behalf of Jochen Bick [jochen.bick@usys.ethz.ch]
Sent: Monday, July 09, 2018 12:49 PM
To: galaxy-dev@lists.galaxyproject.org
Subject: [galaxy-dev] From Paste to uWSGI

Hi all,

we would like to update to 18.05 and thereby also update our Paste Server.

https://docs.galaxyproject.org/en/latest/admin/scaling.html

Is there any tutorial on how to update from Paste to uWSGI?


Cheers Jochen

--
ETH Zurich
*Jochen Bick*
Animal Physiology
Institute of Agricultural Sciences
Postal address: Universitätstrasse 2 / LFW B 58.1
8092 Zurich, Switzerland
Office: Eschikon 27
8315 Lindau, Switzerland

Phone +41 52 354 92 06
jochen.bick@usys.ethz.ch <mailto:jochen.bick@usys.ethz.ch>
www.ap.ethz.ch

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/