galaxy-dev December 2018

galaxy-dev@lists.galaxyproject.org

12 participants
14 discussions

URL prefix doesn't work properly in Galaxy v18.09
by Ping Luo 20 May '19

20 May '19

I configured URL prefix in galaxy 18.09. It has problems of resolving some of the links, including: workflow visualize -> create visualization shared data -> workflows shared data -> visualizations I will use workflow as an example to illustrate what happens. When the mouse cursor is put on workflow, the URL shown in at the left bottom of the browser shows https://xxx.xxx.tamu.edu/galaxy/workflows/list But after I clicked 'workflow', the real URL shown in the address bar is actually https://xxx.xxxx.tamu.edu/galaxy/galaxy//workflows/list?__identifer=sk0xsyw… Of course, there is no such a page and the browser shows a blank page. If I manually remove one galaxy in the URL, the page is correctly shown. Beside the four mentioned above, other links work fine. If galaxy is configured without prefix, then there is no problem. I also tested v18.05. With all the configuration being the same, this version doesn't have the problem with prefix like v18.09 does. So I think this is a bug in v18.09 with URL prefix. Ping

2 3

Nginx+Shibboleth+UCSC
by Martin Demko 18 Apr '19

18 Apr '19

Dear list, I have been trying to manage an access to UCSC genome browser from our Galaxy instance which uses Nginx as a proxy with external authentication via Shibboleth and IdP service. Not even the configuration of nginx+shibboleth was quite complicated to achieve (but I succeeded after a week or so) but now during a testing phase, we have found out that a cooperation with external UCSC servers is forbidden due to our auth. configuration. I have found a way how to do that but only for Apache (https://docs.galaxyproject.org/en/master/admin/special_topics/apache.html#d…) so I tried to do something similar in Nginx but only with IP addresses as Nginx doesn't support using of named domains and defined a location: location /display_as { satisfy any; deny all; allow 128.114.119.131; allow 128.114.119.132; allow 128.114.119.133; allow 128.114.119.134; allow 128.114.119.135; allow 128.114.119.136; } Used IP addresses should be the right ones for UCSC servers according to few sources such as (https://genome.soe.ucsc.narkive.com/sll2JSk2/ucsc-ip-address): hgw1.cse.ucsc.edu hgw2.cse.ucsc.edu hgw3.cse.ucsc.edu hgw4.cse.ucsc.edu hgw5.cse.ucsc.edu hgw6.cse.ucsc.edu I tried to change the location also to "/display_application" but didn't help (actually, I don't know why in the documentation there is used as the location that "/display_as" as I have never encountered such location in Galaxy which is my first question). In galaxy.ini I set: display_servers = hgw1.cse.ucsc.edu,hgw2.cse.ucsc.edu,hgw3.cse.ucsc.edu,hgw4.cse.ucsc.edu,hgw5.cse.ucsc.edu,hgw6.cse.ucsc.edu,hgw7.cse.ucsc.edu,hgw8.cse.ucsc.edu,lowepub.cse.ucsc.edu,128.114.119.131,128.114.119.132,128.114.119.133,128.114.119.134,128.114.119.135,128.114.119.136 Just for case I used both, the domains and also IP addresses. Also I have tried to use only the domains or only the addresses but nothing helped. Then I found out after reading of the following (https://www.switch.ch/aai/guides/sp/access-rules/) that the main obstacle should be actually shibboleth so I defined in /etc/shibboleth/shibboleth2.xml an unrestricted path: <RequestMapper type="XML"> <RequestMap> <Host name="our.galaxy.something" authType="shibboleth" requireSession="true" redirectToSSL="443" > <Path name="display_application" requireSession="false" redirectToSSL="443" />  </Host> </RequestMap> </RequestMapper> Again, I have tried to use both locations "display_as" and "display_application", but it haven't solved my problem entirely. I moved forward I suppose because now the requests should be going around the authentication but UCSC stil warns me that: Expected 200 https://our.galaxy.something/display_application/e1304269a2f56a52/ucsc_bigw…: 403 Forbidden Before that it was a different message containing the address of our idp server so I believe I'am on the right way here. The presence of attribute 'authType="shibboleth"' seams to be useless as 'requireSession="false"' is used. Then I tried to teach Nginx to use also the domain names with the using of 3rd party nginx-http-rdns module from (https://github.com/flant/nginx-http-rdns), so my nginx.conf looked like: ....only the part of nginx.conf.... ############## Shibboleth authentication conf ################# # FastCGI authorizer for Shibboleth Auth Request module location = /shibauthorizer { internal; include fastcgi_params; fastcgi_pass unix:/var/run/shibboleth/shibauthorizer.sock; } # FastCGI responder for SSO location /Shibboleth.sso { include fastcgi_params; fastcgi_pass unix:/var/run/shibboleth/shibresponder.sock; } ############### Entry point for Galaxy ############### # Location secured by Shibboleth location / { shib_request /shibauthorizer; more_clear_input_headers 'Variable-*' 'Shib-*' 'Remote-User' 'REMOTE_USER' 'Auth-Type' 'AUTH_TYPE' 'GX_SECRET'; include uwsgi_params; shib_request_set $shib_id $upstream_http_variable_eppn; uwsgi_param HTTP_REMOTE_USER $shib_id; uwsgi_param HTTP_GX_SECRET $our_secret; uwsgi_pass 127.0.0.1:4001; # resolver $correct_resolver_acc_to_resolv.conf; # rdns on; # satisfy any; # rdns_allow genome\.ucsc\.edu; # rdns_allow hgw1\.cse\.ucsc\.edu; # rdns_allow hgw2\.cse\.ucsc\.edu; # rdns_allow hgw3\.cse\.ucsc\.edu; # rdns_allow hgw4\.cse\.ucsc\.edu; # rdns_allow hgw5\.cse\.ucsc\.edu; # rdns_allow hgw6\.cse\.ucsc\.edu; # rdns_allow hgw7\.cse\.ucsc\.edu; # rdns_allow hgw8\.cse\.ucsc\.edu; } # location /display_as { # resolver $correct_resolver_acc_to_resolv.conf; # rdns on; # satisfy any; # rdns_allow genome\.ucsc\.edu; # rdns_allow hgw1\.cse\.ucsc\.edu; # rdns_allow hgw2\.cse\.ucsc\.edu; # rdns_allow hgw3\.cse\.ucsc\.edu; # rdns_allow hgw4\.cse\.ucsc\.edu; # rdns_allow hgw5\.cse\.ucsc\.edu; # rdns_allow hgw6\.cse\.ucsc\.edu; # rdns_allow hgw7\.cse\.ucsc\.edu; # rdns_allow hgw8\.cse\.ucsc\.edu; # } # location /display_application { # resolver $correct_resolver_acc_to_resolv.conf; # rdns on; # satisfy any; # rdns_allow genome\.ucsc\.edu; # rdns_allow hgw1\.cse\.ucsc\.edu; # rdns_allow hgw2\.cse\.ucsc\.edu; # rdns_allow hgw3\.cse\.ucsc\.edu; # rdns_allow hgw4\.cse\.ucsc\.edu; # rdns_allow hgw5\.cse\.ucsc\.edu; # rdns_allow hgw6\.cse\.ucsc\.edu; # rdns_allow hgw7\.cse\.ucsc\.edu; # rdns_allow hgw8\.cse\.ucsc\.edu; # } ....END of the part of nginx.conf.... Everything commented out, I tried to use together or one at the time but nothing helped. But there are differences: 1) when uncommented only location /display_application - I will get just a blank page with: 404 Not Found. 2) when uncommented only location /display_as - nothing changes. 3) when uncommented part inside the location "/" - I will get a new message: Error unexpected end of input reading http header on https://our.galaxy.something/display_application/e1304269a2f56a52/ucsc_bigw… And now I'm out of ideas. I would really appreciate any help. PS: I stil don't get why in the documentation for Apache configuration (https://docs.galaxyproject.org/en/master/admin/special_topics/apache.html#d…) there is used the /display_as location. Can somebody explain, please? Thank you in advance, Martin Demko

2 1

End of 2018 Reminders: Admin Training, GCC2019 Training Topic Nominations, and GalaxyAdmins meetups topics
by Dave Clements 26 Dec '18

26 Dec '18

Happy Holidays all! A couple of reminders before 2019 arrives: - *Galaxy Admin Training <https://galaxyproject.org/events/2019-admin-training/>*, January 28-February 1. Learn best practices for installing and supporting your own Galaxy service. Split into a 2-day intro session, and a 3 day advanced session. *Advance registration ends December 31, and then rates go up by, um, a lot. Register by Monday.* - *GCC2019 Training Topic Nominations <https://galaxyproject.org/news/2018-12-gcc2019-training-nomination/> end 31 December! * Nominated topics will be voted on by the community in January. Topics nominated thus far are here <https://galaxyproject.org/events/gcc2019/training/#training-topic-nominatio…>. Please nominate new topics here <https://docs.google.com/forms/d/1fBPjEmw7Td52rhBnC0SvflnQCD7Ue5IMJ9RkM0SjnY…>, and if you want to expand on existing nominations <https://galaxyproject.org/events/gcc2019/training/#training-topic-nominatio…> you can edit the hub directly, or send the edits to me. (Note: I suspect, but don't know, that we'll extend this deadline into early January.) - *GalaxyAdmins meetups <https://galaxyproject.org/news/2018-12-galaxy-admins/>* will resume in 2019 <https://galaxyproject.org/news/2018-12-galaxy-admins/>. If you have ideas for meetup topics, please nominate them <https://docs.google.com/forms/d/e/1FAIpQLSfv257YcdQQO7gp3K4USoBzqkd-UDEblst…> . - We need your project ideas for *Google Summer of Code 2019* <https://docs.google.com/forms/d/e/1FAIpQLSfv257YcdQQO7gp3K4USoBzqkd-UDEblst…>*. *Please add your project ideas to the proposal, either directly on this page <http://gmod.org/wiki/GSOC_Project_Ideas_2019> (requires an account), of via this online form <http://bit.ly/gsoc-2019-ogi-propose>. Finally, for those of you not on Twitter, I can't resist sharing this post from Martin Čech <https://twitter.com/martenson?ref_src=twsrc%5Etfw%7Ctwcamp%5Eembeddedtimeli…> . See you in the new year, Dave C [image: 2018-12-26_12-01-57-martins-tree.png] -- https://galaxyproject.org/ https://usegalaxy.org/

1 0

Galaxy Administrator Training, Jan 28 - Feb 1, Penn State University
by Dave Clements 20 Dec '18

20 Dec '18

Hello all, We are pleased to announce that* the 2019 Galaxy Admin Training <https://galaxyproject.org/events/2019-admin-training/> will be offered January 28 through February 1 at Penn State University* (where the Galaxy Project got started over ten years ago). The workshop offers a 2 day introductory session <https://github.com/galaxyproject/dagobah-training#basic-sessions> followed by a 3 day advanced topics session <https://github.com/galaxyproject/dagobah-training#advanced-sessions>. Participants can register for one or both sessions. This workshop will cover what you need to know to set up your own high-performance and multi-user production Galaxy instance. Sessions will be *intensive and hands-on*, and taught by experienced instructors from the Galaxy Community. Participants will learn how to install, configure, customize, and extend their own Galaxy servers. Topics include tool configuration, authentication and user management, using heterogeneous storage and compute services, and many other topics that will enable you to get your own Galaxy server up and running, performing well, and used by your community. Registration <https://galaxyproject.org/events/2019-admin-training/#registration> is now open and starts at $40 / day for participants from non-profits and academia. Early registration ends October 31. However, the 2016 and 2018 admin training were both full, so you are strongly encouraged to *register now*. *The workshop does have prerequisites <https://galaxyproject.org/events/2019-admin-training/#prerequisites>. Please read them before you register.* Cheers, Dave C. *About Galaxy * Galaxy is an open web based platform for biomedical data integration and analysis. It is deployed at large and small organizations around the world and used in a broad range of biomedical research domains, and across the tree of life. -- https://galaxyproject.org/ https://usegalaxy.org/

1 2

tools gone after switching to galaxy.yml
by Matthias Bernt 19 Dec '18

19 Dec '18

Dear list, just tried to switch to galaxy.yml, which worked in general, but now tools that have been installed from the TS disappeared. In the logs I get for each tool: ``` galaxy.tools.toolbox.base ERROR 2018-12-19 13:28:15,815 [p:22877,w:0,m:0] [MainThread] Error reading tool from path: toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/74a8bef53a00/text_pr… Traceback (most recent call last): File "lib/galaxy/tools/toolbox/base.py", line 563, in _load_tool_tag_set tool_shed_repository = self.get_tool_repository_from_xml_item(item, path) File "lib/galaxy/tools/toolbox/base.py", line 629, in get_tool_repository_from_xml_item raise Exception(msg) Exception: Attempted to load tool shed tool, but the repository with name 'text_processing' from owner 'bgruening' was not found in database ``` Any idea what could be wrong here? The connection to the DB seems to be functional, since I can see the users in the admin Panel. Cheers, Matthias -- ------------------------------------------- Matthias Bernt Bioinformatics Service Molekulare Systembiologie (MOLSYB) Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/ Helmholtz Centre for Environmental Research GmbH - UFZ Permoserstraße 15, 04318 Leipzig, Germany Phone +49 341 235 482296, m.bernt(a)ufz.de, www.ufz.de Sitz der Gesellschaft/Registered Office: Leipzig Registergericht/Registration Office: Amtsgericht Leipzig Handelsregister Nr./Trade Register Nr.: B 4703 Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig Wilfried Kraus Wissenschaftlicher Geschäftsführer/Scientific Managing Director: Prof. Dr. Dr. h.c. Georg Teutsch -------------------------------------------

2 2

Galaxy and Google Summer of Code 2019
by Dave Clements 18 Dec '18

18 Dec '18

Hello all, Galaxy will be applying to participate in the 2019 Google Summer of Code <https://summerofcode.withgoogle.com/>. We will again be participating as part of the *Open Genome Informatics Group <http://gmod.org/wiki/GSoC>* that also includes Bioconda <https://bioconda.github.io/>, GMOD <http://gmod.org/>, Reactome <https://reactome.org/>, MGI <http://www.informatics.jax.org/>, OICR <https://oicr.on.ca/research-portfolio/>, and WormBase <http://www.wormbase.org/>. We are currently building our group's proposal. If you have an idea for a project that you would be interested in mentoring someone for, then please add it to the proposal, either directly on this page <http://gmod.org/wiki/GSOC_Project_Ideas_2019> (requires an account), of via this online form <http://bit.ly/gsoc-2019-ogi-propose>. The deadline for submitting your projects is February 1. Thanks, Dave C "Google Summer of Code is a global program that offers student developers stipends to write code for various open source software projects. We work with many open source, free software, and technology-related groups to identify and fund projects over a three month period. Since its inception in 2005, the program has brought together over 14,000 successful student participants from 118 countries, 651 open source organizations, and over 35 million lines of code. Through Google Summer of Code, accepted student applicants are paired with a mentor or mentors from the participating projects, thus gaining exposure to real-world software development scenarios and the opportunity for employment in areas related to their academic pursuits. In turn, the participating projects are able to more easily identify and bring in new developers. Best of all, more source code is created and released for the use and benefit of all." -- https://galaxyproject.org/ https://usegalaxy.org/

1 0

how to configure sub-uri in galaxy.yml
by Ping Luo 14 Dec '18

14 Dec '18

I am trying to configure Galaxy v18.09 wtih sub-uri. In prior 18.01, I use this configuration in conjunction with Apache rewrite rules and it works well: [filter:proxy-prefix] use = egg:PasteDeploy#prefix prefix = /msgalaxy [app:main] filter-with = proxy-prefix cookie_path = '/galaxy' In galaxy.yml, I know I need to uncomment and add the following filter-with: proxy-prefix cookie_path: '/galaxy' However, I don't know where and how to add filter:proxy-prefix. The comment in config.yml says: # If running behind a proxy server and Galaxy is served from a # subdirectory, enable the proxy-prefix filter and set the prefix in # the [filter:proxy-prefix] section above. which is for galaxy.ini file. An example on how to do it would be much appreciated. Ping

2 3

Conda issue on a server using a sshfs mounting point
by Estelle Ancelet 13 Dec '18

13 Dec '18

Hi Galaxy team, I have a Galaxy server sharing a folder with a cluster through sshfs. The Galaxy folder is cloned on the cluster : /home/galaxy-user/galaxy. On the server, I have the same directory but empty : /home/galaxy-user/galaxy. On the server side, I use the following mounting command: sshfs -o cache=no galaxy-user@clusterfr:galaxy galaxy When I run galaxy (sh ./run.sh from galaxy-user in the VM), everything seems to go well but in fact, the _conda directory doesn’t contain any binary. It only contains the pkg folder. The log message I get is “cannot install conda because conda_prefix /home/galaxy-user/galaxy/database/dependencies/_conda exists and is not empty”. In the UI, if I try to install a tool with dependencies from the toolshed, I get the same log message. I’ve also tried sshfs -o cache=no,idmap=user,allow_other. But then, I cannot launch Galaxy. I have an arror (add error message). Any clue on how to make conda work with this infrastructure? Thanks in advance Estelle

1 2

GalaxyAdmins is returning to a device near you in February 2019, and we need your help
by Dave Clements 12 Dec '18

12 Dec '18

Hi All After a two year hiatus, the bimonthly online GalaxyAdmins meetups <https://galaxyproject.org/community/galaxy-admins/meetups/> will return on February 28 <https://galaxyproject.org/community/galaxy-admins/meetups/2019-02-28/>. GalaxyAdmins <https://galaxyproject.org/community/galaxy-admins/> is a discussion group for Galaxy community members who are responsible for Galaxy installations. Our online meetups are around an hour long and feature a presentation followed by an open discussion. It's a great place to catch up on what your fellow admins are thinking about. *But, before we start meeting next year, we need to know what you care about <https://docs.google.com/forms/d/e/1FAIpQLSfv257YcdQQO7gp3K4USoBzqkd-UDEblst…>.* What do you want to learn? Who do you want to hear from? What's causing you trouble? Or, what have you worked that your fellow GalaxyAdmins would benefit from knowing? Got some ideas? *Nominate them <https://docs.google.com/forms/d/e/1FAIpQLSfv257YcdQQO7gp3K4USoBzqkd-UDEblst…>* We hope to see you in February, Helena Rasche <https://galaxyproject.org/people/helena-rasche/> and Dave Clements <https://galaxyproject.org/people/dave-clements/> -- https://galaxyproject.org/ https://usegalaxy.org/

1 0

_remove_dead_weakref error even after deleting .venv (Galaxy 18.09)
by Peter van Heusden 12 Dec '18

12 Dec '18

Hi there I'm trying to run Windowmasker in Galaxy and receiving the following error: "computing the genome length pass 1 pass 2 optimizing the data structure Traceback (most recent call last): File "/tools/software/galaxy/galaxy1/database/jobs/006/6476/set_metadata_bUlLkt.py", line 1, in <module> from galaxy_ext.metadata.set_metadata import set_metadata; set_metadata() File "/tools/software/galaxy/galaxy1/lib/galaxy_ext/metadata/set_metadata.py", line 14, in <module> import logging File "/usr/lib/python2.7/logging/__init__.py", line 26, in <module> import sys, os, time, cStringIO, traceback, warnings, weakref, collections File "/usr/lib/python2.7/weakref.py", line 14, in <module> from _weakref import ( ImportError: cannot import name _remove_dead_weakref" The usual suggested fix for this is to remove the Galaxy virtual environment (.venv) and recreated it. I have done that but the error persists. Galaxy is running here on Ubuntu 16.04 with Python 2.7.12. Any advice? Thanks! Peter

1 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

galaxy-dev December 2018