Dear Galaxy Community,
is there any possibility to expose the command line call, generated within galaxy?
We searched the net and found this discussions: https://github.com/galaxyproject/galaxy/issues/3252 https://github.com/galaxyproject/galaxy/issues/2954
But both are on the exposure to non-admin users. Within our instance we see the command line call nor for admin, neither for normal users!
The problem, behind the question is the error message: "An invalid option was selected for database, 'myCoolDatabaseName', please verify." In Blast (toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_blastn_wrapper/0.2.01)
Local (directly from command line, without Galaxy in between) the blast works fine.
We are using the 18.05 Docker Galaxy Version.
Thanks in advance!
Mit freundlichen Grüßen
Matthias Enders -----------------------------------------------------------------------
GERMAN SEED ALLIANCE GmbH c/o Norddeutsche Pflanzenzucht Hans-Georg Lembke KG Hohenlieth, 24363 Holtsee Tel.: +49 (0)4351/ 736-189 Fax: + 49 (0)4351/ 736-271 Mobil: +49 (0)151/ 14247360
Email: m.enders@german-seed-alliance.de
Firmensitz Köln Amtsgericht Köln, HRB 73844
-----Ursprüngliche Nachricht----- Von: galaxy-dev galaxy-dev-bounces@lists.galaxyproject.org Im Auftrag von galaxy-dev-request@lists.galaxyproject.org Gesendet: Montag, 9. Juli 2018 18:00 An: galaxy-dev@lists.galaxyproject.org Betreff: galaxy-dev Digest, Vol 145, Issue 3
Send galaxy-dev mailing list submissions to galaxy-dev@lists.galaxyproject.org
To subscribe or unsubscribe via the World Wide Web, visit https://lists.galaxyproject.org/listinfo/galaxy-dev or, via email, send a message with subject or body 'help' to galaxy-dev-request@lists.galaxyproject.org
You can reach the person managing the list at galaxy-dev-owner@lists.galaxyproject.org
When replying, please edit your Subject line so it is more specific than "Re: Contents of galaxy-dev digest..."
HEY! This is important! If you reply to a thread in a digest, please 1. Change the subject of your response from "Galaxy-dev Digest Vol ..." to the original subject for the thread. 2. Strip out everything else in the digest that is not part of the thread you are responding to.
Why? 1. This will keep the subject meaningful. People will have some idea from the subject line if they should read it or not. 2. Not doing this greatly increases the number of emails that match search queries, but that aren't actually informative.
Today's Topics:
1. From Paste to uWSGI (Jochen Bick) 2. Re: From Paste to uWSGI (Peter Briggs) 3. Re: From Paste to uWSGI (Martin Čech)
----------------------------------------------------------------------
Message: 1 Date: Mon, 9 Jul 2018 13:49:23 +0200 From: Jochen Bick jochen.bick@usys.ethz.ch To: "galaxy-dev@lists.galaxyproject.org" galaxy-dev@lists.galaxyproject.org Subject: [galaxy-dev] From Paste to uWSGI Message-ID: 1a0f33ad-938c-64d5-8db8-28914a5ca07e@usys.ethz.ch Content-Type: text/plain; charset="utf-8"; format=flowed
Hi all,
we would like to update to 18.05 and thereby also update our Paste Server.
https://docs.galaxyproject.org/en/latest/admin/scaling.html
Is there any tutorial on how to update from Paste to uWSGI?
Cheers Jochen
-- ETH Zurich *Jochen Bick* Animal Physiology Institute of Agricultural Sciences Postal address: Universitätstrasse 2 / LFW B 58.1 8092 Zurich, Switzerland Office: Eschikon 27 8315 Lindau, Switzerland
Phone +41 52 354 92 06 jochen.bick@usys.ethz.ch mailto:jochen.bick@usys.ethz.ch www.ap.ethz.ch
------------------------------
Message: 2 Date: Mon, 9 Jul 2018 15:31:16 +0000 From: Peter Briggs peter.briggs@manchester.ac.uk To: Jochen Bick jochen.bick@usys.ethz.ch, "galaxy-dev@lists.galaxyproject.org" galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] From Paste to uWSGI Message-ID: E20AB71CD352854B9E24B5830826BC43C8586657@MBXP15.ds.man.ac.uk Content-Type: text/plain; charset="iso-8859-1"
Hello Jochen
I'm not aware of a tutorial but here are some observations from my (very) recent experiences upgrading a local paste-based 17.09 instance to a uwsgi-based 18.05 configuration. It's not comprehensive because the configuration will depend on decisions you need to make based on your local setup, but it might be helpful as a starting point.
Before beginning:
-- Commit to migrating from galaxy.ini to galaxy.yml (if not done previously)
Even though its possible to stick with the old galaxy.ini file, it seems best to move to the new format - my feeling is, probably the migration will have to happen at some point anyway, and also the documentation is more focussed on the newer galaxy.yml version.
-- Decide on how you're going to handle job submission (essentially, how you're going to replace the old job handler processes).
There are a number of possibilities outlined in the documentation here:
https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#deploymen...
It seems important to figure this out at the beginning because it impacts a lot of the configuration specifics that follow. For our setup case "uwsgi + mules" looked like the most appropriate:
https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#uwsgi-for...
Then once you're ready to start (nb backing up the database etc before starting are implicitly assumed!):
-- Update the Galaxy source code
The standard recipe for this seems to work okay i.e.
git fetch origin && git checkout release_18.05 && git pull --ff-only origin release_18.05
(You'll probably also need to run scripts/common_startup.sh to update the Python packages, and scripts/manage_db.py to update the database schema.)
-- Create galaxy.yml and configure the "galaxy" section:
You can make a galaxy.yml from the sample version and go through the "galaxy" section replicating the settings from galaxy.ini. (The settings in the "uwsgi" section can wait at this stage.) Don't forget to move galaxy.ini out of the way once you're done.
-- Configure the "uwsgi" section of galaxy.yml:
The options for various scenarios are given in the "Deployment/uwsgi" section:
https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#uwsgi
but the required settings seem to be spread out between different sections so (for me) it felt like it took some piecing together to get it to work when I first tried this.
(Also I was thrown a little by the "socket" parameter not having a placeholder in sample galaxy.yml.)
The core uwsgi options are here:
https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#configura...
then you need to set other options dependent on:
1. how job submission is handled (e.g. options for the "uwsgi + mules" setup in my case) 2. how you're proxying Galaxy (for Apache or nginx it looks like it's just setting the uwsgi "socket" option, unless you're serving Galaxy from a URL prefix in which case there are a few more options needed - these are covered in the documentation.)
-- Fix your job_conf.xml file
If you have an existing job_conf.xml file in your config directory then you might need to update that too (specifically the "handlers" section) depending on the job submission handling you've decided to use.
-- Use supervisor to manage your uwsgi processes
The docs for this are here:
https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#starting-...
Essentially supervisor replaces "run.sh", so this becomes how you start and stop Galaxy (e.g. "supervisorctl start galaxy:" replaces "GALAXY_RUN_ALL=1 sh run.sh --daemon".)
I think this is quite straightforward from the docs but again you need to tailor it for how you're handling job submission (e.g the "uwsgi + mules" doesn't need any handlers defining, but it looks like other strategies needs extra stuff defining here).
-- Set up the proxying
Assuming you're proxing via either Apache or nginx (we're using nginx), these are covered in the relevant sections:
https://docs.galaxyproject.org/en/release_18.05/admin/apache.html https://docs.galaxyproject.org/en/release_18.05/admin/nginx.html
The main difference I noticed for nginx is that you no longer need to use the "upstream"/"proxy_pass" directives, as nginx can talk directly to uwsgi using a "uwsgi_pass" directive that connects to the socket defined in the "uwsgi" section of galaxy.yml previously.
(There's also a "gotcha" if you're nginx configuration also tries to serve "static/scripts/packed", which needs to be removed - this is noted in the "breaking changes" for the 18.05 docs but is easy to miss.)
Hopefully this is some help. There's lots of documentation, which is a good thing but which can sometimes make it harder to see where to start.
Good luck with the update,
Best wishes
Peter
-- Peter Briggs peter.briggs@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482
________________________________________ From: galaxy-dev [galaxy-dev-bounces@lists.galaxyproject.org] on behalf of Jochen Bick [jochen.bick@usys.ethz.ch] Sent: Monday, July 09, 2018 12:49 PM To: galaxy-dev@lists.galaxyproject.org Subject: [galaxy-dev] From Paste to uWSGI
Hi all,
we would like to update to 18.05 and thereby also update our Paste Server.
https://docs.galaxyproject.org/en/latest/admin/scaling.html
Is there any tutorial on how to update from Paste to uWSGI?
Cheers Jochen
-- ETH Zurich *Jochen Bick* Animal Physiology Institute of Agricultural Sciences Postal address: Universitätstrasse 2 / LFW B 58.1 8092 Zurich, Switzerland Office: Eschikon 27 8315 Lindau, Switzerland
Phone +41 52 354 92 06 jochen.bick@usys.ethz.ch mailto:jochen.bick@usys.ethz.ch www.ap.ethz.ch
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
------------------------------
Message: 3 Date: Mon, 9 Jul 2018 11:36:27 -0400 From: Martin Čech marten@bx.psu.edu To: Peter Briggs peter.briggs@manchester.ac.uk Cc: "galaxy-dev@lists.galaxyproject.org" galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] From Paste to uWSGI Message-ID: CAHKBquGXMTog-MUgLbPWD2wgmXvpOxiQGRa4+QCWk5EAHGK3=g@mail.gmail.com Content-Type: text/plain; charset="utf-8"
Another recently updated resource for uwsgi is the training slides: https://galaxyproject.github.io/dagobah-training/2018-gccbosc/10-uwsgi/uwsgi...
M.
On Mon, Jul 9, 2018 at 11:31 AM Peter Briggs peter.briggs@manchester.ac.uk wrote:
Hello Jochen
I'm not aware of a tutorial but here are some observations from my (very) recent experiences upgrading a local paste-based 17.09 instance to a uwsgi-based 18.05 configuration. It's not comprehensive because the configuration will depend on decisions you need to make based on your local setup, but it might be helpful as a starting point.
Before beginning:
-- Commit to migrating from galaxy.ini to galaxy.yml (if not done previously)
Even though its possible to stick with the old galaxy.ini file, it seems best to move to the new format - my feeling is, probably the migration will have to happen at some point anyway, and also the documentation is more focussed on the newer galaxy.yml version.
-- Decide on how you're going to handle job submission (essentially, how you're going to replace the old job handler processes).
There are a number of possibilities outlined in the documentation here:
https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#dep loyment-options
It seems important to figure this out at the beginning because it impacts a lot of the configuration specifics that follow. For our setup case "uwsgi
- mules" looked like the most appropriate:
https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#uws gi-for-web-serving-with-mules-as-job-handlers
Then once you're ready to start (nb backing up the database etc before starting are implicitly assumed!):
-- Update the Galaxy source code
The standard recipe for this seems to work okay i.e.
git fetch origin && git checkout release_18.05 && git pull --ff-only origin release_18.05
(You'll probably also need to run scripts/common_startup.sh to update the Python packages, and scripts/manage_db.py to update the database schema.)
-- Create galaxy.yml and configure the "galaxy" section:
You can make a galaxy.yml from the sample version and go through the "galaxy" section replicating the settings from galaxy.ini. (The settings in the "uwsgi" section can wait at this stage.) Don't forget to move galaxy.ini out of the way once you're done.
-- Configure the "uwsgi" section of galaxy.yml:
The options for various scenarios are given in the "Deployment/uwsgi" section:
https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#uws gi
but the required settings seem to be spread out between different sections so (for me) it felt like it took some piecing together to get it to work when I first tried this.
(Also I was thrown a little by the "socket" parameter not having a placeholder in sample galaxy.yml.)
The core uwsgi options are here:
https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#con figuration-common-to-all-uwsgi-deployment-styles
then you need to set other options dependent on:
- how job submission is handled (e.g. options for the "uwsgi + mules"
setup in my case) 2. how you're proxying Galaxy (for Apache or nginx it looks like it's just setting the uwsgi "socket" option, unless you're serving Galaxy from a URL prefix in which case there are a few more options needed - these are covered in the documentation.)
-- Fix your job_conf.xml file
If you have an existing job_conf.xml file in your config directory then you might need to update that too (specifically the "handlers" section) depending on the job submission handling you've decided to use.
-- Use supervisor to manage your uwsgi processes
The docs for this are here:
https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#sta rting-and-stopping
Essentially supervisor replaces "run.sh", so this becomes how you start and stop Galaxy (e.g. "supervisorctl start galaxy:" replaces "GALAXY_RUN_ALL=1 sh run.sh --daemon".)
I think this is quite straightforward from the docs but again you need to tailor it for how you're handling job submission (e.g the "uwsgi + mules" doesn't need any handlers defining, but it looks like other strategies needs extra stuff defining here).
-- Set up the proxying
Assuming you're proxing via either Apache or nginx (we're using nginx), these are covered in the relevant sections:
https://docs.galaxyproject.org/en/release_18.05/admin/apache.html https://docs.galaxyproject.org/en/release_18.05/admin/nginx.html
The main difference I noticed for nginx is that you no longer need to use the "upstream"/"proxy_pass" directives, as nginx can talk directly to uwsgi using a "uwsgi_pass" directive that connects to the socket defined in the "uwsgi" section of galaxy.yml previously.
(There's also a "gotcha" if you're nginx configuration also tries to serve "static/scripts/packed", which needs to be removed - this is noted in the "breaking changes" for the 18.05 docs but is easy to miss.)
Hopefully this is some help. There's lots of documentation, which is a good thing but which can sometimes make it harder to see where to start.
Good luck with the update,
Best wishes
Peter
-- Peter Briggs peter.briggs@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482
From: galaxy-dev [galaxy-dev-bounces@lists.galaxyproject.org] on behalf of Jochen Bick [jochen.bick@usys.ethz.ch] Sent: Monday, July 09, 2018 12:49 PM To: galaxy-dev@lists.galaxyproject.org Subject: [galaxy-dev] From Paste to uWSGI
Hi all,
we would like to update to 18.05 and thereby also update our Paste Server.
https://docs.galaxyproject.org/en/latest/admin/scaling.html
Is there any tutorial on how to update from Paste to uWSGI?
Cheers Jochen
-- ETH Zurich *Jochen Bick* Animal Physiology Institute of Agricultural Sciences Postal address: Universitätstrasse 2 https://maps.google.com/?q=Universit%C3%A4tstrasse+2&entry=gmail&sour ce=g / LFW B 58.1 8092 Zurich, Switzerland Office: Eschikon 27 8315 Lindau, Switzerland
Phone +41 52 354 92 06 <+41%2052%20354%2092%2006> jochen.bick@usys.ethz.ch mailto:jochen.bick@usys.ethz.ch www.ap.ethz.ch
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/ ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
-------------- next part -------------- An HTML attachment was scrubbed... URL: https://lists.galaxyproject.org/pipermail/galaxy-dev/attachments/20180709/a7d44c29/attachment-0001.html
------------------------------
Subject: Digest Footer
_______________________________________________ galaxy-dev mailing list galaxy-dev@lists.galaxyproject.org https://lists.galaxyproject.org/listinfo/galaxy-dev
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
------------------------------
End of galaxy-dev Digest, Vol 145, Issue 3 ******************************************