galaxy-dev July 2018

galaxy-dev@lists.galaxyproject.org

15 participants
12 discussions

Monitoring Dashboard for Galaxy
by evan clark 03 Sep '18

03 Sep '18

I remember there previously being an additional tool that ran on a different port from galaxy that allowed for monitoring of performance and jobs running. Is this tool still packaged with galaxy and if so how can it be activated.

3 3

Unable to install tools from toolshed/mercurial issue in release_18.05 instance
by Peter Briggs 16 Aug '18

16 Aug '18

Dear devs I've encountered what appears to be a subtle bug with release 18.05, which breaks the installation of tools from the toolshed, and appears to be a result of not having mercurial (hg) available in /usr/bin on the system that Galaxy is installed on (in this case Scientific Linux 6.5). When attempting to install a tool (e.g. devteam/fastqc) from the main toolshed via the admin interface, after clicking "install" the tool installation status goes immediately to "Error". The tool repository isn't cloned to "shed_tools" and no dependencies are installed. I've been unable to find any error messages in the logs. However, attempting to install via the API does return the message: Error cloning repository: [Errno 2] No such file or directory which comes from the "clone_repository" function in lib/tool_shed/util/hg_util.py (when something goes wrong with the "hg clone ..." command). Installing Mercurial 1.3 via yum on the server and attempting tool installation again gives a slightly different error via the API: Error cloning repository: Command '['hg', 'clone', '-r', u'17', u'https://toolshed.g2.bx.psu.edu/repos/devteam/fastqc', u'/XXXXXXXXXXXXXX/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/fastqc/c15237684a01/fastqc']' returned non-zero exit status 255 Output was: abort: No such file or directory: /XXXXXXXXXXXXXXXXXX/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/fastqc/c15237684a01/fastqc Uninstalling the system Mercurial and instead installing version 3.7.3 and making a link from /usr/bin/hg seems to fix the problem, and tools can be installed without problems. I couldn't find any evidence of this being reported before, and I don't know if I've missed some configuration detail which means that Galaxy isn't picking up hg from its virtualenv instead of /usr/bin. Has anyone else encountered this problem? and is there a fix/workaround for it (other than horrible links to /usr/bin/hg)? Any advice gratefully received! Best wishes Peter -- Peter Briggs peter.briggs(a)manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482

3 3

Hierarchical upload API documentation and example
by Lance Parsons 01 Aug '18

01 Aug '18

Hi all, I’ve been looking at the new hierarchical upload API and I’d like to try and use it to integrate our existing sequence data repository a bit better with Galaxy. Currently I use a tool to post data to Galaxy, which works, but seems a bit clunky. This seems more built in and future proof. I’ve seen the discussion on galaxyproject/galaxy#5220 <https://github.com/galaxyproject/galaxy/pull/5220> but I was wondering if there was any other, more formal, documentation. Or possibly anyone who has actually implemented (or started to implement) anything using this. Thanks! I’ve also seen the tutorial linked in the release notes <https://github.com/jmchilton/training-material/blob/rules/topics/introducti…> but I was wondering where that is actually hosted for public consumption. I’m a bit lost with the various places for Galaxy documentation and training these days… -- Lance Parsons - Scientific Programmer Carl C. Icahn Laboratory - Room 136 Lewis-Sigler Institute for Integrative Genomics Princeton University

2 1

Local Galaxy Instance Error: "enable third-party cookies" they're enabled already!
by David Halvorsen 30 Jul '18

30 Jul '18

Hello, I am attempting to install the "join" tool (Join the intervals of two datasets side-by-side) from Galaxy Tool Shed. However, I get the following error "Repository installation is not possible due to an invalid Galaxy URL: *None*. You may need to enable third-party cookies in your browser." I am running a local instance of Galaxy on Ubuntu 16.04 LTS. I have attempted to install this tool on both Google Chrome and Firefox. Google Chrome is set to "Allow sites to save and read cookie data (recommended) and "Block third-party cookies" is turned off. Firefox is set to "Accept cookies and site data from websites (recommended)" and "Accept third-party cookies and site data Always". I tried restarted my Galaxy instance after ensuring that those were my broswer settings. My config.yml file has my email address set as an admin user. I have also tried restarting my computer and attempting all of those steps again. The "join" tool works fine under "Operate on Genomic Intervals" when I use the "usegalaxy.org" instance. This is my first time trying to run a local Galaxy instance. *Does anyone have any ideas on how to fix this?* Best, David

1 0

Installing Tool Dependencies
by Tom Misilo 18 Jul '18

18 Jul '18

Hello, I am trying to help a research here at Florida Tech setup Galaxy locally to process their data. However, I am running into a problem installing some tool dependencies was on this page (https://moin.galaxyproject.org/Admin/Tools/ToolDependencies). For example, I am trying to install BEAM, LPS, PASS, and GPASS. However, I cannot find in the documentation for either the tools themselves or on Galaxy on how to set these tools up for use in Galaxy. Would you be able to assist in this matter? Thank you! Tom Thomas Misilo Network Engineer Florida Institute of Technology Evans Library 150 W. University Blvd Melbourne, FL 32901 Phone: 321-674-8831 Fax: 321-724-2559 misilot(a)fit.edu<mailto:misilot@fit.edu>

4 6

Expose Command Line in Galaxy
by Matthias Enders 11 Jul '18

11 Jul '18

Dear Galaxy Community, is there any possibility to expose the command line call, generated within galaxy? We searched the net and found this discussions: https://github.com/galaxyproject/galaxy/issues/3252 https://github.com/galaxyproject/galaxy/issues/2954 But both are on the exposure to non-admin users. Within our instance we see the command line call nor for admin, neither for normal users! The problem, behind the question is the error message: "An invalid option was selected for database, 'myCoolDatabaseName', please verify." In Blast (toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_blastn_wrapper/0.…) Local (directly from command line, without Galaxy in between) the blast works fine. We are using the 18.05 Docker Galaxy Version. Thanks in advance! Mit freundlichen Grüßen Matthias Enders ----------------------------------------------------------------------- GERMAN SEED ALLIANCE GmbH c/o Norddeutsche Pflanzenzucht Hans-Georg Lembke KG Hohenlieth, 24363 Holtsee Tel.: +49 (0)4351/ 736-189 Fax: + 49 (0)4351/ 736-271 Mobil: +49 (0)151/ 14247360 Email: m.enders(a)german-seed-alliance.de Firmensitz Köln Amtsgericht Köln, HRB 73844 -----Ursprüngliche Nachricht----- Von: galaxy-dev <galaxy-dev-bounces(a)lists.galaxyproject.org> Im Auftrag von galaxy-dev-request(a)lists.galaxyproject.org Gesendet: Montag, 9. Juli 2018 18:00 An: galaxy-dev(a)lists.galaxyproject.org Betreff: galaxy-dev Digest, Vol 145, Issue 3 Send galaxy-dev mailing list submissions to galaxy-dev(a)lists.galaxyproject.org To subscribe or unsubscribe via the World Wide Web, visit https://lists.galaxyproject.org/listinfo/galaxy-dev or, via email, send a message with subject or body 'help' to galaxy-dev-request(a)lists.galaxyproject.org You can reach the person managing the list at galaxy-dev-owner(a)lists.galaxyproject.org When replying, please edit your Subject line so it is more specific than "Re: Contents of galaxy-dev digest..." HEY! This is important! If you reply to a thread in a digest, please 1. Change the subject of your response from "Galaxy-dev Digest Vol ..." to the original subject for the thread. 2. Strip out everything else in the digest that is not part of the thread you are responding to. Why? 1. This will keep the subject meaningful. People will have some idea from the subject line if they should read it or not. 2. Not doing this greatly increases the number of emails that match search queries, but that aren't actually informative. Today's Topics: 1. From Paste to uWSGI (Jochen Bick) 2. Re: From Paste to uWSGI (Peter Briggs) 3. Re: From Paste to uWSGI (Martin Čech) ---------------------------------------------------------------------- Message: 1 Date: Mon, 9 Jul 2018 13:49:23 +0200 From: Jochen Bick <jochen.bick(a)usys.ethz.ch> To: "galaxy-dev(a)lists.galaxyproject.org" <galaxy-dev(a)lists.galaxyproject.org> Subject: [galaxy-dev] From Paste to uWSGI Message-ID: <1a0f33ad-938c-64d5-8db8-28914a5ca07e(a)usys.ethz.ch> Content-Type: text/plain; charset="utf-8"; format=flowed Hi all, we would like to update to 18.05 and thereby also update our Paste Server. https://docs.galaxyproject.org/en/latest/admin/scaling.html Is there any tutorial on how to update from Paste to uWSGI? Cheers Jochen -- ETH Zurich *Jochen Bick* Animal Physiology Institute of Agricultural Sciences Postal address: Universitätstrasse 2 / LFW B 58.1 8092 Zurich, Switzerland Office: Eschikon 27 8315 Lindau, Switzerland Phone +41 52 354 92 06 jochen.bick(a)usys.ethz.ch <mailto:jochen.bick@usys.ethz.ch> www.ap.ethz.ch ------------------------------ Message: 2 Date: Mon, 9 Jul 2018 15:31:16 +0000 From: Peter Briggs <peter.briggs(a)manchester.ac.uk> To: Jochen Bick <jochen.bick(a)usys.ethz.ch>, "galaxy-dev(a)lists.galaxyproject.org" <galaxy-dev(a)lists.galaxyproject.org> Subject: Re: [galaxy-dev] From Paste to uWSGI Message-ID: <E20AB71CD352854B9E24B5830826BC43C8586657(a)MBXP15.ds.man.ac.uk> Content-Type: text/plain; charset="iso-8859-1" Hello Jochen I'm not aware of a tutorial but here are some observations from my (very) recent experiences upgrading a local paste-based 17.09 instance to a uwsgi-based 18.05 configuration. It's not comprehensive because the configuration will depend on decisions you need to make based on your local setup, but it might be helpful as a starting point. Before beginning: -- Commit to migrating from galaxy.ini to galaxy.yml (if not done previously) Even though its possible to stick with the old galaxy.ini file, it seems best to move to the new format - my feeling is, probably the migration will have to happen at some point anyway, and also the documentation is more focussed on the newer galaxy.yml version. -- Decide on how you're going to handle job submission (essentially, how you're going to replace the old job handler processes). There are a number of possibilities outlined in the documentation here: https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#deployme… It seems important to figure this out at the beginning because it impacts a lot of the configuration specifics that follow. For our setup case "uwsgi + mules" looked like the most appropriate: https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#uwsgi-fo… Then once you're ready to start (nb backing up the database etc before starting are implicitly assumed!): -- Update the Galaxy source code The standard recipe for this seems to work okay i.e. git fetch origin && git checkout release_18.05 && git pull --ff-only origin release_18.05 (You'll probably also need to run scripts/common_startup.sh to update the Python packages, and scripts/manage_db.py to update the database schema.) -- Create galaxy.yml and configure the "galaxy" section: You can make a galaxy.yml from the sample version and go through the "galaxy" section replicating the settings from galaxy.ini. (The settings in the "uwsgi" section can wait at this stage.) Don't forget to move galaxy.ini out of the way once you're done. -- Configure the "uwsgi" section of galaxy.yml: The options for various scenarios are given in the "Deployment/uwsgi" section: https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#uwsgi but the required settings seem to be spread out between different sections so (for me) it felt like it took some piecing together to get it to work when I first tried this. (Also I was thrown a little by the "socket" parameter not having a placeholder in sample galaxy.yml.) The core uwsgi options are here: https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#configur… then you need to set other options dependent on: 1. how job submission is handled (e.g. options for the "uwsgi + mules" setup in my case) 2. how you're proxying Galaxy (for Apache or nginx it looks like it's just setting the uwsgi "socket" option, unless you're serving Galaxy from a URL prefix in which case there are a few more options needed - these are covered in the documentation.) -- Fix your job_conf.xml file If you have an existing job_conf.xml file in your config directory then you might need to update that too (specifically the "handlers" section) depending on the job submission handling you've decided to use. -- Use supervisor to manage your uwsgi processes The docs for this are here: https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#starting… Essentially supervisor replaces "run.sh", so this becomes how you start and stop Galaxy (e.g. "supervisorctl start galaxy:" replaces "GALAXY_RUN_ALL=1 sh run.sh --daemon".) I think this is quite straightforward from the docs but again you need to tailor it for how you're handling job submission (e.g the "uwsgi + mules" doesn't need any handlers defining, but it looks like other strategies needs extra stuff defining here). -- Set up the proxying Assuming you're proxing via either Apache or nginx (we're using nginx), these are covered in the relevant sections: https://docs.galaxyproject.org/en/release_18.05/admin/apache.html https://docs.galaxyproject.org/en/release_18.05/admin/nginx.html The main difference I noticed for nginx is that you no longer need to use the "upstream"/"proxy_pass" directives, as nginx can talk directly to uwsgi using a "uwsgi_pass" directive that connects to the socket defined in the "uwsgi" section of galaxy.yml previously. (There's also a "gotcha" if you're nginx configuration also tries to serve "static/scripts/packed", which needs to be removed - this is noted in the "breaking changes" for the 18.05 docs but is easy to miss.) Hopefully this is some help. There's lots of documentation, which is a good thing but which can sometimes make it harder to see where to start. Good luck with the update, Best wishes Peter -- Peter Briggs peter.briggs(a)manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482 ________________________________________ From: galaxy-dev [galaxy-dev-bounces(a)lists.galaxyproject.org] on behalf of Jochen Bick [jochen.bick(a)usys.ethz.ch] Sent: Monday, July 09, 2018 12:49 PM To: galaxy-dev(a)lists.galaxyproject.org Subject: [galaxy-dev] From Paste to uWSGI Hi all, we would like to update to 18.05 and thereby also update our Paste Server. https://docs.galaxyproject.org/en/latest/admin/scaling.html Is there any tutorial on how to update from Paste to uWSGI? Cheers Jochen -- ETH Zurich *Jochen Bick* Animal Physiology Institute of Agricultural Sciences Postal address: Universitätstrasse 2 / LFW B 58.1 8092 Zurich, Switzerland Office: Eschikon 27 8315 Lindau, Switzerland Phone +41 52 354 92 06 jochen.bick(a)usys.ethz.ch <mailto:jochen.bick@usys.ethz.ch> www.ap.ethz.ch ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/ ------------------------------ Message: 3 Date: Mon, 9 Jul 2018 11:36:27 -0400 From: Martin Čech <marten(a)bx.psu.edu> To: Peter Briggs <peter.briggs(a)manchester.ac.uk> Cc: "galaxy-dev(a)lists.galaxyproject.org" <galaxy-dev(a)lists.galaxyproject.org> Subject: Re: [galaxy-dev] From Paste to uWSGI Message-ID: <CAHKBquGXMTog-MUgLbPWD2wgmXvpOxiQGRa4+QCWk5EAHGK3=g(a)mail.gmail.com> Content-Type: text/plain; charset="utf-8" Another recently updated resource for uwsgi is the training slides: https://galaxyproject.github.io/dagobah-training/2018-gccbosc/10-uwsgi/uwsg… M. On Mon, Jul 9, 2018 at 11:31 AM Peter Briggs <peter.briggs(a)manchester.ac.uk> wrote: > Hello Jochen > > I'm not aware of a tutorial but here are some observations from my > (very) recent experiences upgrading a local paste-based 17.09 instance > to a uwsgi-based 18.05 configuration. It's not comprehensive because > the configuration will depend on decisions you need to make based on > your local setup, but it might be helpful as a starting point. > > Before beginning: > > -- Commit to migrating from galaxy.ini to galaxy.yml (if not done > previously) > > Even though its possible to stick with the old galaxy.ini file, it > seems best to move to the new format - my feeling is, probably the > migration will have to happen at some point anyway, and also the > documentation is more focussed on the newer galaxy.yml version. > > -- Decide on how you're going to handle job submission (essentially, > how you're going to replace the old job handler processes). > > There are a number of possibilities outlined in the documentation here: > > > https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#dep > loyment-options > > It seems important to figure this out at the beginning because it > impacts a lot of the configuration specifics that follow. For our > setup case "uwsgi > + mules" looked like the most appropriate: > > > https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#uws > gi-for-web-serving-with-mules-as-job-handlers > > Then once you're ready to start (nb backing up the database etc before > starting are implicitly assumed!): > > -- Update the Galaxy source code > > The standard recipe for this seems to work okay i.e. > > git fetch origin && git checkout release_18.05 && git pull --ff-only > origin release_18.05 > > (You'll probably also need to run scripts/common_startup.sh to update > the Python packages, and scripts/manage_db.py to update the database > schema.) > > -- Create galaxy.yml and configure the "galaxy" section: > > You can make a galaxy.yml from the sample version and go through the > "galaxy" section replicating the settings from galaxy.ini. (The > settings in the "uwsgi" section can wait at this stage.) Don't forget > to move galaxy.ini out of the way once you're done. > > -- Configure the "uwsgi" section of galaxy.yml: > > The options for various scenarios are given in the "Deployment/uwsgi" > section: > > https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#uws > gi > > but the required settings seem to be spread out between different > sections so (for me) it felt like it took some piecing together to get > it to work when I first tried this. > > (Also I was thrown a little by the "socket" parameter not having a > placeholder in sample galaxy.yml.) > > The core uwsgi options are here: > > > https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#con > figuration-common-to-all-uwsgi-deployment-styles > > then you need to set other options dependent on: > > 1. how job submission is handled (e.g. options for the "uwsgi + mules" > setup in my case) > 2. how you're proxying Galaxy (for Apache or nginx it looks like it's > just setting the uwsgi "socket" option, unless you're serving Galaxy > from a URL prefix in which case there are a few more options needed - > these are covered in the documentation.) > > -- Fix your job_conf.xml file > > If you have an existing job_conf.xml file in your config directory > then you might need to update that too (specifically the "handlers" > section) depending on the job submission handling you've decided to use. > > -- Use supervisor to manage your uwsgi processes > > The docs for this are here: > > > https://docs.galaxyproject.org/en/release_18.05/admin/scaling.html#sta > rting-and-stopping > > Essentially supervisor replaces "run.sh", so this becomes how you > start and stop Galaxy (e.g. "supervisorctl start galaxy:" replaces > "GALAXY_RUN_ALL=1 sh run.sh --daemon".) > > I think this is quite straightforward from the docs but again you need > to tailor it for how you're handling job submission (e.g the "uwsgi + mules" > doesn't need any handlers defining, but it looks like other strategies > needs extra stuff defining here). > > -- Set up the proxying > > Assuming you're proxing via either Apache or nginx (we're using > nginx), these are covered in the relevant sections: > > https://docs.galaxyproject.org/en/release_18.05/admin/apache.html > https://docs.galaxyproject.org/en/release_18.05/admin/nginx.html > > The main difference I noticed for nginx is that you no longer need to > use the "upstream"/"proxy_pass" directives, as nginx can talk directly > to uwsgi using a "uwsgi_pass" directive that connects to the socket > defined in the "uwsgi" section of galaxy.yml previously. > > (There's also a "gotcha" if you're nginx configuration also tries to > serve "static/scripts/packed", which needs to be removed - this is > noted in the "breaking changes" for the 18.05 docs but is easy to > miss.) > > Hopefully this is some help. There's lots of documentation, which is a > good thing but which can sometimes make it harder to see where to start. > > Good luck with the update, > > Best wishes > > Peter > > -- > Peter Briggs peter.briggs(a)manchester.ac.uk Bioinformatics Core > Facility University of Manchester > B.1083 Michael Smith Bldg Tel: (0161) 2751482 > > > ________________________________________ > From: galaxy-dev [galaxy-dev-bounces(a)lists.galaxyproject.org] on > behalf of Jochen Bick [jochen.bick(a)usys.ethz.ch] > Sent: Monday, July 09, 2018 12:49 PM > To: galaxy-dev(a)lists.galaxyproject.org > Subject: [galaxy-dev] From Paste to uWSGI > > Hi all, > > we would like to update to 18.05 and thereby also update our Paste Server. > > https://docs.galaxyproject.org/en/latest/admin/scaling.html > > Is there any tutorial on how to update from Paste to uWSGI? > > > Cheers Jochen > > -- > ETH Zurich > *Jochen Bick* > Animal Physiology > Institute of Agricultural Sciences > Postal address: Universitätstrasse 2 > <https://maps.google.com/?q=Universit%C3%A4tstrasse+2&entry=gmail&sour > ce=g> > / LFW B 58.1 > 8092 Zurich, Switzerland > Office: Eschikon 27 > 8315 Lindau, Switzerland > > Phone +41 52 354 92 06 <+41%2052%20354%2092%2006> > jochen.bick(a)usys.ethz.ch <mailto:jochen.bick@usys.ethz.ch> > www.ap.ethz.ch > > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this and other > Galaxy lists, please use the interface at: > https://lists.galaxyproject.org/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/ > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this and other > Galaxy lists, please use the interface at: > https://lists.galaxyproject.org/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://lists.galaxyproject.org/pipermail/galaxy-dev/attachments/20180709/a…> ------------------------------ Subject: Digest Footer _______________________________________________ galaxy-dev mailing list galaxy-dev(a)lists.galaxyproject.org https://lists.galaxyproject.org/listinfo/galaxy-dev To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ------------------------------ End of galaxy-dev Digest, Vol 145, Issue 3 ******************************************

4 13

Conda problems: Failed to activate conda environment
by Van Wageningen, GERHARD [gerhardv＠sun.ac.za] 11 Jul '18

11 Jul '18

Hi All I have recently started learning the Galaxy setup starting with an upgrade. Made some headway but I was wondering if someone could comment on a problem that has emerged. The system is a Galaxy 18.05 installation (upgraded from 17.01) on an Altair PBS scheduler with drmaa setup (working) Conda problem: Failed to activate conda environment. Conda works during tool installation or reinstallation. Galaxy tools have been updated/reinstalled so they all look "green" in manage tools. In Manage Dependencies there are a number of "Dependency resolved but version not found" warnings, otherwise the deps look clean. When running tools in Galaxy errors occur, after the jobs have correctly deployed on the cluster. The galaxy application raised errors like the two below. "Failed to activate conda environment! Error was: Fatal Python error: Py_Initialize: Unable to get the locale encoding File "/home/.galaxy/tooldeps/python/2.7/iuc/package_python_2_7/8b09fe018cac/lib/python2.7/encodings/__init__.py", line 123 raise C" " Failed to activate conda environment! Error was: CondaEnvironmentError: Environment error: Cannot activate environment bash. User does not have write access for conda symlinks. " Galaxy started and stopped using supervisord with config below: Thanks, hopefully someone has seen this before ... regards, Gerhard [program:web] command = uwsgi --plugin python --virtualenv /home/.galaxy/.venv --ini-paste /home/.galaxy/config/galaxy.ini directory = /home/.galaxy umask = 022 autostart = true autorestart = true startsecs = 10 user = galaxy numprocs = 1 stopsignal = INT [program:handler] command = python ./scripts/galaxy-main -c /home/.galaxy/config/galaxy.ini --server-name=handler%(process_num)s directory = /home/.galaxy process_name = handler%(process_num)s numprocs = 1 umask = 022 autostart = true autorestart = true startsecs = 10 user = galaxy environment = VIRTUAL_ENV="/home/.galaxy/.venv",PATH="/home/.galaxy/.venv/bin:%(ENV_PATH)s",http_proxy="http://storage2.eth.hpc:3128/",https_proxy="http://storage2.eth.hpc:3128/" stdout_logfile = /home/.galaxy/handler%(process_num)s.log redirect_stderr = true [group:galaxy] programs = handler, web [http://cdn.sun.ac.za/100/ProductionFooter.jpg]<http://www.sun.ac.za/english/Pages/Water-crisis.aspx> The integrity and confidentiality of this email is governed by these terms. Disclaimer<http://www.sun.ac.za/emaildisclaimer> Die integriteit en vertroulikheid van hierdie e-pos word deur die volgende bepalings gereël. Vrywaringsklousule<http://www.sun.ac.za/emaildisclaimer>

2 8

AMD EPYC and Galaxy
by Van Wageningen, GERHARD [gerhardv＠sun.ac.za] 10 Jul '18

10 Jul '18

Hi All - Just a quick ask - anyone have anything BAD to say about running Galaxy on a CENTOS 7.4/Scientific Linux with AMD EPYC chipset? Its a bit new ... but looks good on paper and we would like to grow the Galaxy environment here. Thanks Gerhard Stellenbosch IT [http://cdn.sun.ac.za/100/ProductionFooter.jpg]<http://www.sun.ac.za/english/Pages/Water-crisis.aspx> The integrity and confidentiality of this email is governed by these terms. Disclaimer<http://www.sun.ac.za/emaildisclaimer> Die integriteit en vertroulikheid van hierdie e-pos word deur die volgende bepalings gereël. Vrywaringsklousule<http://www.sun.ac.za/emaildisclaimer>

1 0

Error with SAMTools Mpileup v1.2
by Aarthi Mohan 10 Jul '18

10 Jul '18

Hi all, I have a galaxy instance running 18.05 version. I recently installed 'suite_samtools_1_2' (revision: 5b673ccc8747). When I used 'Mpileup' tool, I ran into the below error, and the jobs fail. Error message from UI: This job was killed when Galaxy was restarted. Please retry the job. Error report from UI: python: /home/aarthi/GALAXY/galaxy_18/galaxy_18.05/database/jobs_directory/000/33/galaxy_33.sh: line 61: 27021 Segmentation fault (core dumped) python "/home/aarthi/GALAXY/galaxy_18/galaxy_18.05/database/jobs_directory/000/33/set_metadata_16ppzR.py" "/home/aarthi/GALAXY/galaxy_18/galaxy_18.05/database/jobs_directory/000/33/registry.xml" "/home/aarthi/GALAXY/galaxy_18/galaxy_18.05/database/jobs_directory/000/33/working/galaxy.json" "/home/aarthi/GALAXY/galaxy_18/galaxy_18.05/database/jobs_directory/000/33/metadata_in_HistoryDatasetAssociation_74_dGoEXq,/home/aarthi/GALAXY/galaxy_18/galaxy_18.05/database/jobs_directory/000/33/metadata_kwds_HistoryDatasetAssociation_74_ZgPen_,/home/aarthi/GALAXY/galaxy_18/galaxy_18.05/database/jobs_directory/000/33/metadata_out_HistoryDatasetAssociation_74_OsySK1,/home/aarthi/GALAXY/galaxy_18/galaxy_18.05/database/jobs_directory/000/33/metadata_results_HistoryDatasetAssociation_74_1NfJC0,/home/aarthi/GALAXY/galaxy_18/galaxy_18.05/database/files/000/dataset_74.dat,/home/aarthi/GALAXY/galaxy_18/galaxy_18.05/database/jobs_directory/000/33/metadata_override_HistoryDatasetAssociation_74_vZTPWp" "/home/aarthi/GALAXY/galaxy_18/galaxy_18.05/database/jobs_directory/000/33/metadata_in_HistoryDatasetAssociation_75_AoWnZx,/home/aarthi/GALAXY/galaxy_18/galaxy_18.05/database/jobs_directory/000/33/metadata_kwds_HistoryDatasetAssociation_75_J8fJUV,/home/aarthi/GALAXY/galaxy_18/galaxy_18.05/database/jobs_directory/000/33/metadata_out_HistoryDatasetAssociation_75_CXBh3u,/home/aarthi/GALAXY/galaxy_18/galaxy_18.05/database/jobs_directory/000/33/metadata_results_HistoryDatasetAssociation_75_T_tzxr,/home/aarthi/GALAXY/galaxy_18/galaxy_18.05/database/files/000/dataset_75.dat,/home/aarthi/GALAXY/galaxy_18/galaxy_18.05/database/jobs_directory/000/33/metadata_override_HistoryDatasetAssociation_75_9fiQnm" 5242880 >From galaxy.log file: [LocalRunner.work_thread-1] execution finished: /home/aarthi/GALAXY/galaxy_18/galaxy_18.05/database/jobs_directory/000/33/galaxy_33.sh galaxy.model.metadata DEBUG 2018-07-10 17:31:51,417 [p:26910,w:1,m:0] [LocalRunner.work_thread-1] setting metadata externally failed for HistoryDatasetAssociation 74: External set_meta() not called /home/aarthi/GALAXY/galaxy_18/galaxy_18.05/.venv/bin/python: !!! uWSGI process 26910 got Segmentation Fault !!! *** backtrace of 26910 *** /home/aarthi/GALAXY/galaxy_18/galaxy_18.05/.venv/lib/python2.7/site-packages/uwsgi.so(uwsgi_backtrace+0x2e) [0x7fc5c385897e] /home/aarthi/GALAXY/galaxy_18/galaxy_18.05/.venv/lib/python2.7/site-packages/uwsgi.so(uwsgi_segfault+0x2f) [0x7fc5c3858d4f] /lib64/libc.so.6(+0x35270) [0x7fc5ca0ba270] /lib64/libc.so.6(strchrnul+0x1a) [0x7fc5ca11522a] /lib64/libc.so.6(_IO_vfprintf+0x92) [0x7fc5ca0cc212] /lib64/libc.so.6(+0x4c431) [0x7fc5ca0d1431] /lib64/libc.so.6(_IO_vfprintf+0x1ee) [0x7fc5ca0cc36e] /lib64/libc.so.6(+0xf6155) [0x7fc5ca17b155] /lib64/libc.so.6(error+0xcd) [0x7fc5ca17b2ad] /home/aarthi/GALAXY/galaxy_18/galaxy_18.05/.venv/lib/python2.7/site-packages/pysam/./libcbcftools.so(main_vcfindex+0x504) [0x7fc5a9183bc4] /home/aarthi/GALAXY/galaxy_18/galaxy_18.05/.venv/lib/python2.7/site-packages/pysam/libcutils.so(+0x14271) [0x7fc5a96da271] /lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x6722) [0x7fc5cae514e2] /lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed) [0x7fc5cae53efd] /lib64/libpython2.7.so.1.0(+0x70858) [0x7fc5caddd858] /lib64/libpython2.7.so.1.0(PyObject_Call+0x43) [0x7fc5cadb89a3] /lib64/libpython2.7.so.1.0(+0x5a995) [0x7fc5cadc7995] /lib64/libpython2.7.so.1.0(PyObject_Call+0x43) [0x7fc5cadb89a3] /lib64/libpython2.7.so.1.0(+0xa2ce7) [0x7fc5cae0fce7] /lib64/libpython2.7.so.1.0(PyObject_Call+0x43) [0x7fc5cadb89a3] /lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x2336) [0x7fc5cae4d0f6] /lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed) [0x7fc5cae53efd] /lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x663c) [0x7fc5cae513fc] /lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed) [0x7fc5cae53efd] /lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x663c) [0x7fc5cae513fc] /lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x67bd) [0x7fc5cae5157d] /lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x67bd) [0x7fc5cae5157d] /lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed) [0x7fc5cae53efd] /lib64/libpython2.7.so.1.0(+0x7094d) [0x7fc5caddd94d] /lib64/libpython2.7.so.1.0(PyObject_Call+0x43) [0x7fc5cadb89a3] /lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x17fd) [0x7fc5cae4c5bd] /lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x67bd) [0x7fc5cae5157d] /lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x67bd) [0x7fc5cae5157d] /lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x7ed) [0x7fc5cae53efd] /lib64/libpython2.7.so.1.0(+0x70858) [0x7fc5caddd858] /lib64/libpython2.7.so.1.0(PyObject_Call+0x43) [0x7fc5cadb89a3] /lib64/libpython2.7.so.1.0(+0x5a995) [0x7fc5cadc7995] /lib64/libpython2.7.so.1.0(PyObject_Call+0x43) [0x7fc5cadb89a3] /lib64/libpython2.7.so.1.0(PyEval_CallObjectWithKeywords+0x47) [0x7fc5cae4a7b7] /lib64/libpython2.7.so.1.0(+0x1156e2) [0x7fc5cae826e2] /lib64/libpthread.so.0(+0x7e25) [0x7fc5cab58e25] /lib64/libc.so.6(clone+0x6d) [0x7fc5ca17d34d] *** end of backtrace *** I am testing it inside a virtual box, and currently running galaxy in sudo mode. I saw that galaxy requires 'samtools' to be found in the PATH so, I tried running galaxy like with this command 'sudo env "PATH=$PATH" sh run.sh', still I get the same error. I will appreciate any help or suggestion. Thanks in advance! Thanks, Aarthi ________________________________ Important: This email is confidential and may be privileged. If you are not the intended recipient, please delete it and notify us immediately; you should not copy or use it for any purpose, nor disclose its contents to any other person. Thank you.

1 0

uWSGI threading issue
by Durdevic, Marija 10 Jul '18

10 Jul '18

Dear developers, Since last Galaxy upgrade some issue with uwsgi and threading arise. I used Galaxy documentation<https://docs.galaxyproject.org/en/master/admin/scaling.html> as a guideline to configure galaxy.yml: uwsgi: socket: 127.0.0.1:4001 http: 0.0.0.0:8080 buffer-size: 16384 processes: 4 threads: 8 offload-threads: 2 static-map: /static/style=static/style/blue static-map: /static=static master: true virtualenv: .venv pythonpath: lib module: galaxy.webapps.galaxy.buildapp:uwsgi_app() thunder-lock: true die-on-term: true hook-master-start: unix_signal:2 gracefully_kill_them_all hook-master-start: unix_signal:15 gracefully_kill_them_all py-call-osafterfork: true enable-threads: true mule: lib/galaxy/main.py mule: lib/galaxy/main.py mule: lib/galaxy/main.py farm: job-handlers:1,2,3 When I run Galaxy, I got error message - error: can't start new thread Also if I run some job which require a lot of threads, I always get error message - RuntimeError: can't start new thread I tried to increase number of processes in /etc/security/limits.conf for galaxy user but it didn’t help. What can be solution? Do you have any idea what could be wrong? Thanks a lot for your support. Best, Marija Marija Đurđević PhD, Diagnostic & Research Center for Molecular BioMedicine Institute of Pathology Bioinformatician, Core Facility Computational Bioanalytics Center for Medical Research Medical University of Graz Stiftingtalstraße 24, A-8010 Graz Austria Phone: +43 316/385-73024 Fax:+43 316/385-73009 Email: marija.durdevic(a)medunigraz.at<mailto:marija.durdevic@medunigraz.at> Email: marija.djurdjevic(a)klinikum-graz.at<mailto:marija.djurdjevic@klinikum-graz.at> Web: https://zmf.medunigraz.at<https://zmf.medunigraz.at/>

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

galaxy-dev July 2018