- galaxy-dev - lists.galaxyproject.org

Nginx+Shibboleth+UCSC
by Martin Demko 18 Apr '19

18 Apr '19

Dear list, I have been trying to manage an access to UCSC genome browser from our Galaxy instance which uses Nginx as a proxy with external authentication via Shibboleth and IdP service. Not even the configuration of nginx+shibboleth was quite complicated to achieve (but I succeeded after a week or so) but now during a testing phase, we have found out that a cooperation with external UCSC servers is forbidden due to our auth. configuration. I have found a way how to do that but only for Apache (https://docs.galaxyproject.org/en/master/admin/special_topics/apache.html#d…) so I tried to do something similar in Nginx but only with IP addresses as Nginx doesn't support using of named domains and defined a location: location /display_as { satisfy any; deny all; allow 128.114.119.131; allow 128.114.119.132; allow 128.114.119.133; allow 128.114.119.134; allow 128.114.119.135; allow 128.114.119.136; } Used IP addresses should be the right ones for UCSC servers according to few sources such as (https://genome.soe.ucsc.narkive.com/sll2JSk2/ucsc-ip-address) hgw1.cse.ucsc.edu hgw2.cse.ucsc.edu hgw3.cse.ucsc.edu hgw4.cse.ucsc.edu hgw5.cse.ucsc.edu hgw6.cse.ucsc.edu I tried to change the location also to "/display_application" but didn't help (actually, I don't know why in the documentation there is used as the location that "/display_as" as I have never encountered such location in Galaxy which is my first question). In galaxy.ini I set: display_servers = hgw1.cse.ucsc.edu,hgw2.cse.ucsc.edu,hgw3.cse.ucsc.edu,hgw4.cse.ucsc.edu,hgw5.cse.ucsc.edu,hgw6.cse.ucsc.edu,hgw7.cse.ucsc.edu,hgw8.cse.ucsc.edu,lowepub.cse.ucsc.edu,128.114.119.131,128.114.119.132,128.114.119.133,128.114.119.134,128.114.119.135,128.114.119.136 Just for case I used both, the domains and also IP addresses. Also I have tried to use only the domains or only the addresses but nothing helped. Then I found out after reading of the following (https://www.switch.ch/aai/guides/sp/access-rules/) that the main obstacle should be actually shibboleth so I defined in /etc/shibboleth/shibboleth2.xml an unrestricted path: <RequestMapper type="XML"> <RequestMap> <Host name="our.galaxy.something" authType="shibboleth" requireSession="true" redirectToSSL="443" > <Path name="display_application" requireSession="false" redirectToSSL="443" />  </Host> </RequestMap> </RequestMapper> Again, I have tried to use both locations "display_as" and "display_application", but it haven't solved my problem entirely. I moved forward I suppose because now the requests should be going around the authentication but UCSC stil warns me that: Expected 200 https://our.galaxy.something/display_application/e1304269a2f56a52/ucsc_bigw…: 403 Forbidden Before that it was a different message containing the address of our idp server so I believe I'am on the right way here. The presence of attribute 'authType="shibboleth"' seams to be useless as 'requireSession="false"' is used. Then I tried to teach Nginx to use also the domain names with the using of 3rd party nginx-http-rdns module from (https://github.com/flant/nginx-http-rdns) so my nginx.conf looked like: ....only the part of nginx.conf.... ############## Shibboleth authentication conf ################# # FastCGI authorizer for Shibboleth Auth Request module location = /shibauthorizer { internal; include fastcgi_params; fastcgi_pass unix:/var/run/shibboleth/shibauthorizer.sock; } # FastCGI responder for SSO location /Shibboleth.sso { include fastcgi_params; fastcgi_pass unix:/var/run/shibboleth/shibresponder.sock; } ############### Entry point for Galaxy ############### # Location secured by Shibboleth location / { shib_request /shibauthorizer; more_clear_input_headers 'Variable-*' 'Shib-*' 'Remote-User' 'REMOTE_USER' 'Auth-Type' 'AUTH_TYPE' 'GX_SECRET'; include uwsgi_params; shib_request_set $shib_id $upstream_http_variable_eppn; uwsgi_param HTTP_REMOTE_USER $shib_id; uwsgi_param HTTP_GX_SECRET $our_secret; uwsgi_pass 127.0.0.1:4001; # resolver $correct_resolver_acc_to_resolv.conf; # rdns on; # satisfy any; # rdns_allow genome\.ucsc\.edu; # rdns_allow hgw1\.cse\.ucsc\.edu; # rdns_allow hgw2\.cse\.ucsc\.edu; # rdns_allow hgw3\.cse\.ucsc\.edu; # rdns_allow hgw4\.cse\.ucsc\.edu; # rdns_allow hgw5\.cse\.ucsc\.edu; # rdns_allow hgw6\.cse\.ucsc\.edu; # rdns_allow hgw7\.cse\.ucsc\.edu; # rdns_allow hgw8\.cse\.ucsc\.edu; } # location /display_as { # resolver $correct_resolver_acc_to_resolv.conf; # rdns on; # satisfy any; # rdns_allow genome\.ucsc\.edu; # rdns_allow hgw1\.cse\.ucsc\.edu; # rdns_allow hgw2\.cse\.ucsc\.edu; # rdns_allow hgw3\.cse\.ucsc\.edu; # rdns_allow hgw4\.cse\.ucsc\.edu; # rdns_allow hgw5\.cse\.ucsc\.edu; # rdns_allow hgw6\.cse\.ucsc\.edu; # rdns_allow hgw7\.cse\.ucsc\.edu; # rdns_allow hgw8\.cse\.ucsc\.edu; # } # location /display_application { # resolver $correct_resolver_acc_to_resolv.conf; # rdns on; # satisfy any; # rdns_allow genome\.ucsc\.edu; # rdns_allow hgw1\.cse\.ucsc\.edu; # rdns_allow hgw2\.cse\.ucsc\.edu; # rdns_allow hgw3\.cse\.ucsc\.edu; # rdns_allow hgw4\.cse\.ucsc\.edu; # rdns_allow hgw5\.cse\.ucsc\.edu; # rdns_allow hgw6\.cse\.ucsc\.edu; # rdns_allow hgw7\.cse\.ucsc\.edu; # rdns_allow hgw8\.cse\.ucsc\.edu; # } ....END of the part of nginx.conf.... Everything commented out, I tried to use together or one at the time but nothing helped. But there are differences: 1) when uncommented only location /display_application - I will get just a blank page with: 404 Not Found. 2) when uncommented only location /display_as - nothing changes. 3) when uncommented part inside the location "/" - I will get a new message: Error unexpected end of input reading http header on https://our.galaxy.something/display_application/e1304269a2f56a52/ucsc_bigw… And now I'm out of ideas. I would really appreciate any help. PS: I stil don't get why in the documentation for Apache configuration (https://docs.galaxyproject.org/en/master/admin/special_topics/apache.html#d…) there is used the /display_as location. Can somebody explain, please? Thank you in advance, Martin Demko

2 1

GCC2019 Talk Abstracts due Monday, 22, April
by Dave Clements 17 Apr '19

17 Apr '19

Hello All, This is just a reminder that oral presentation and lightning talk submissions for the 2019 Galaxy Community Conference are due on Monday, 22 April. And, in case you missed these items: - There will be childcare - Travel fellowships are available (application review starts 23 April) Finally, please redistribute this in any of your communities where it's relevant. Thanks, Dave C ------------------ 2019 Galaxy Community Conference (GCC2019) 1-8 July, Freiburg, Germany https://galaxyproject.org/gcc2019 ------------------ Deadlines 22 April: Oral Presentation and Lightning Talk abstracts due 17 May: Early registration ends The 2019 Galaxy Community Conference (GCC2019) will be held 1-8 July in Freiburg, Germany. GCC brings together hundreds of faculty, clinicians, researchers, and students, all working in and supporting data intensive science that is accessible, sharable and reproducible. GCC2019 features oral presentations, lightning talks, posters, demos, birds-of-a-feather gatherings (BoFs), training, a CollaborationFest, and plenty of opportunities for networking. Presentations will cover the full spectrum of Galaxy applications, enhancements and deployments. If you are working in data intensive science then GCC2019 is an ideal conference for sharing your work, learning from others, and finding new collaborators. Present your work! Abstract submission for talks, lightning talks, demos and posters is now open. If you work in data-intensive science then please consider presenting your work at GCC2019. This is a chance to present to 200+ researchers, all addressing challenges in data intensive science. Review of oral presentations starts on 22 April. Submit an abstract (or two) now! Registration Early registration starts at €49/day for students and postdocs, and €79 / day for other academics and non-profit researchers. Childcare is available and travel fellowships are available as well. Early registration ends 17 May, when rates go up by 60%. So, register early. About Galaxy Galaxy (https://galaxyproject.org/) is a platform for data integration and analysis in the life sciences. It enables researchers to build, run, share, and repeat their own complex computational analyses using only a web browser and without having to first learn system administration and command line interfaces. The Galaxy Project is driven by a vibrant community who publish workflows and analyses, wrap new tools, maintain and enhance the source code, provide support, and write documentation and training materials. Galaxy is open-source and freely available, and is deployed in hundreds of organizations, running on everything from laptops through supercomputers to public and private clouds. Over 150 of these platforms are publicly available and can be used with little or no setup. Thousands of tools have been ported to Galaxy ("wrapped") and are deployable from the Galaxy Tool Shed. Galaxy was developed to support life science research, but the software is domain agnostic and is now used in domains as diverse as natural language processing, constructive solid geometry, and social science. We hope to see you in Freiburg! GCC2019 Organizers -- https://galaxyproject.org/gcc2019/ https://galaxyproject.org/ https://usegalaxy.org/

1 0

user login / registration problem
by Matthias Bernt 10 Apr '19

10 Apr '19

Hi, on our ldap-autoregistration Galaxy-dev instance (details below) no new users can be auto-registered since recently because of https://github.com/galaxyproject/galaxy/issues/7617 to me it seems that the function ``` message, status, user, success = self.__register(trans, **kwd) ``` called in `lib/galaxy/webapps/galaxy/controllers/user.py` (L90) got lost between 19.01 and now. Would be great to get some feedback. I could help with debugging or contributing, but I'm wondering how, since I would need to purge a user in order test this (creation of a new ldap user involves a lot of administrative work) .. and this seems impossible at the moment. Cheers, Matthias P.S. Instance details if of interest: we have a dev instance that uses ldap for authentication, ie users can not create accounts on their own, but the account gets created if they are in ldap. So I have in galaxy.yml `allow_user_creation: false` and in auth_conf ``` <allow-register>Challenge</allow-register> <auto-register>True</auto-register> ``` The configuration worked till recently (LDAP authentication still works according to the logs) -- ------------------------------------------- Matthias Bernt Bioinformatics Service Molekulare Systembiologie (MOLSYB) Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/ Helmholtz Centre for Environmental Research GmbH - UFZ Permoserstraße 15, 04318 Leipzig, Germany Phone +49 341 235 482296, m.bernt(a)ufz.de, www.ufz.de Helmholtz-Zentrum für Umweltforschung GmbH - UFZ Helmholtz Centre for Environmental Research GmbH - UFZ Permoserstraße 15, 04318 Leipzig, Germany Phone +49 341 235 1269, Fax +49 341 235 1468 (optional) max.mustermann(a)ufz.de, www.ufz.de Sitz der Gesellschaft/Registered Office: Leipzig Registergericht/Registration Office: Amtsgericht Leipzig Handelsregister Nr./Trade Register Nr.: B 4703 Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig Wilfried Kraus Wissenschaftlicher Geschäftsführer/Scientific Managing Director: Prof. Dr. Georg Teutsch Administrative Geschäftsführerin/Administrative Managing Director: Dr. Sabine König

1 0

April 2019 Galaxy News: GCC2019 registration and abstract submission are open, and more!
by Mo Heydarian 03 Apr '19

03 Apr '19

Hello All, The *April 2019 Galactic News <https://galaxyproject.org/news/2019-04-galaxy-update/>* is here! This is a summary of what is going on in the Galaxy community. - Event News - GCC2019 registration and abstract submission are open! <https://galaxyproject.org/news/2019-04-galaxy-update/#gcc2019-registration-…> - 120 new publications <https://galaxyproject.org/news/2019-04-galaxy-update/#publications>, great resources lead to great insight. - Some most excellent Galaxy Platform News <https://galaxyproject.org/news/2019-04-galaxy-update/#galaxy-platforms-news>, including ways to investigate unmapped RNA-seq reads, language analysis, and RNA structure tools! - A new entry to The Galactic Blog <https://galaxyproject.org/news/2019-04-galaxy-update/#new-galactic-blog-post>, about the upcomming GCC. - At least 9 Open positions <https://galaxyproject.org/news/2019-04-galaxy-update/#whos-hiring> in three countries on two continents. - Updates to training materials <https://galaxyproject.org/news/2019-04-galaxy-update/#doc-hub-and-training-…> . - ToolShed contributions <https://galaxyproject.org/news/2019-04-galaxy-update/#toolshed-contributions> . - CloudBridge 2.0 released <https://galaxyproject.org/news/2019-04-galaxy-update/##cloudbridge-2.0-rele…> . - Galaxy status page <https://galaxyproject.org/news/2019-04-galaxy-update/##galaxy-status> is live! - And a bunch of other news <https://galaxyproject.org/news/2019-04-galaxy-update/#other-news> too. If you have anything to add to next month's newsletter, then please send it to outreach(a)galaxyproject.org. Cheers, Mo Heydarian, on behalf of everyone

1 0

tool installation problem
by Matthias Bernt 27 Mar '19

27 Mar '19

Dear list, I have a problem during installing a tool version that I previously uninstalled. I get the message: `Revision 0696db066a5b of repository deseq2 owned by iuc has already been installed.` It seems to be uninstalled on the file system. Are there any entries that might be left in the DB? Thanks in advance. Cheers, Matthias -- ------------------------------------------- Matthias Bernt Bioinformatics Service Molekulare Systembiologie (MOLSYB) Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/ Helmholtz Centre for Environmental Research GmbH - UFZ Permoserstraße 15, 04318 Leipzig, Germany Phone +49 341 235 482296, m.bernt(a)ufz.de, www.ufz.de Helmholtz-Zentrum für Umweltforschung GmbH - UFZ Helmholtz Centre for Environmental Research GmbH - UFZ Permoserstraße 15, 04318 Leipzig, Germany Phone +49 341 235 1269, Fax +49 341 235 1468 (optional) max.mustermann(a)ufz.de, www.ufz.de Sitz der Gesellschaft/Registered Office: Leipzig Registergericht/Registration Office: Amtsgericht Leipzig Handelsregister Nr./Trade Register Nr.: B 4703 Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig Wilfried Kraus Wissenschaftlicher Geschäftsführer/Scientific Managing Director: Prof. Dr. Georg Teutsch Administrative Geschäftsführerin/Administrative Managing Director: Dr. Sabine König

2 3

GalaxyAdmins online meetup this Thursday
by Dave Clements 21 Mar '19

21 Mar '19

Hi All, The GalaxyAdmins group is back (thanks to a rejuvenation by Helena Rasche), and will have its first online meetup in over two year this Thursday. *When:* 21 March 2019, 15:00 GMT (see your local time <https://www.timeanddate.com/worldclock/fixedtime.html?msg=March+2019+Galaxy…> ) *Where: *On Zoom <https://zoom.us/>. If you don't already have the Zoom client installed, then you'll need to download it <https://support.zoom.us/hc/en-us/articles/207373866-Zoom-Installers> before the meetup. *We will publish the Zoom room link here <https://galaxyproject.org/community/galaxy-admins/meetups/2019-03-21/> and on Twitter <https://twitter.com/galaxyproject> before the meeting.* *What:* We will have a general discussion / give and take about Galaxy administration and a presentation by Pablo Moreno <https://galaxyproject.org/blog/2018-08-cotm-pablo-moreno/> on *Running Galaxy on Kubernetes.* - Interested? It would be great to have you join us. Thanks and hope to see you Thursday, Dave C (channeling Helena) -- https://galaxyproject.org/ https://usegalaxy.org/

1 1

Serious issues with picard-tools after upgrade to Galaxy version 19.01
by Previti 20 Mar '19

20 Mar '19

Dear all, I'm having serious issues with picard tools (latest Galaxy version 2.18.2.1 by devteam) after upgrading our galaxy to version 19.01 I've been running fastqtoSam on paired-end data: _JAVA_OPTIONS=${_JAVA_OPTIONS:-'-Xmx200g -Xms256m'} && export _JAVA_OPTIONS && picard FastqToSam FASTQ="/opt/galaxy/galaxy/database/files/039/dataset_39081.dat" FASTQ2="/opt/galaxy/galaxy/database/files/039/dataset_39082.dat" QUALITY_FORMAT="Standard" OUTPUT="/opt/galaxy/galaxy/database/files/039/dataset_39441.dat" READ_GROUP_NAME="A" SAMPLE_NAME="sample-a" MIN_Q="0" MAX_Q="93" STRIP_UNPAIRED_MATE_NUMBER="false" ALLOW_AND_IGNORE_EMPTY_LINES="false" SORT_ORDER=coordinate VALIDATION_STRINGENCY="LENIENT" QUIET=true VERBOSITY=ERROR `if [ -n "$TMPDIR" ] ; then echo 'TMP_DIR=$TMPDIR' ; else if [ -n "$TEMP" ] ; then echo 'TMP_DIR=$TEMP' ; fi ; fi` But the bam file I get is not readable and I also get following message: Picked up _JAVA_OPTIONS: -Xmx200g -Xms256m 14:16:29.629 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/opt/galaxy/galaxy/database/dependencies/_conda/envs/__picard@2.18.2/share/picard-2.18.2-0/picard.jar!/com/intel/gkl/native/libgkl_compression.so does anybody know what's going on? The output "bam" file is the correct size (>20GB) but not readable... Thanks and best regards, Christopher Previti *Dr. Christopher Previti* Genomics and Proteomics Core Facility High Throughput Sequencing (W190) Bioinformatician German Cancer Research Center (DKFZ) Foundation under Public Law Im Neuenheimer Feld 580 69120 Heidelberg Germany Room: B2.102 (INF580/TP3) Phone: +49 6221 42-4661 christopher.previti(a)dkfz.de <http://www.dkfz.de/> www.dkfz.de <http://www.dkfz.de/> Management Board: Prof. Dr. Michael Baumann, Prof. Dr. Josef Puchta VAT-ID No.: DE143293537 Vertraulichkeitshinweis: Diese Nachricht ist ausschließlich für die Personen bestimmt, an die sie adressiert ist. Sie kann vertrauliche und/oder nur für den/die Empfänger bestimmte Informationen enthalten. Sollten Sie nicht der bestimmungsgemäße Empfänger sein, kontaktieren Sie bitte den Absender und löschen Sie die Mitteilung. Jegliche unbefugte Verwendung der Informationen in dieser Nachricht ist untersagt.

1 1

GCC2019 Registration & Abstract Submission are Open
by Dave Clements 19 Mar '19

19 Mar '19

*We are pleased to announce that registration <https://galaxyproject.org/events/gcc2019/registration/> and abstract submission <https://galaxyproject.org/events/gcc2019/abstracts/> for the 2019 Galaxy Community Conference (GCC2019) <https://galaxyproject.org/events/gcc2019/> are now open.* GCC2019 will be held 1-8 July in Freiburg, Germany. The *tenth* GCC <https://galaxyproject.org/gcc/> will have many familiar features from earlier years, including accepted and lightning talks, posters and demos, birds-of-a-feather gatherings (BoFs), training, and a CollaborationFest. 2019 also brings the most significant conference program update <https://gcc2019.sched.com/> in several years: - some training will be integrated with the main conference, - three days of conference instead of two, - several parallel sessions, - and a strong emphasis on organizing content by domain. GCC2019, like every GCC before it, will be built around *community*. Training topics are nominated, selected, and presented by the community. Presentations will feature the full spectrum of Galaxy applications, enhancements and deployments from the community as well. If you are working in data intensive life science research then there will not be a better place to share your work, learn from others, and find new collaborators. *Registration* Early registration <https://galaxyproject.org/events/gcc2019/registration/> starts at €49/day for students and postdocs, and €79 / day for other academics and non-profit researchers. Early registration ends 17 May, *when rates go up by 60%. So, register early <https://galaxyproject.org/events/gcc2019/registration/>.* *Present your work!* Abstract submission <https://galaxyproject.org/events/gcc2019/abstracts/> for talks, lightning talks, demos and posters is now open. If you work in data-intensive life science (or in other fields using Galaxy) then please consider presenting your work at GCC2019. This is an ideal chance to present to 200+ researchers, all addressing common challenges in data intensive science. Review of oral presentations starts on 22 April, in a little over a month. *Submit an abstract (or two) now <https://galaxyproject.org/events/gcc2019/abstracts/>!* *We hope to see you in Freiburg!* GCC2019 Organizers <https://galaxyproject.org/events/gcc2019/organizers/> -- https://galaxyproject.org/ https://usegalaxy.org/

1 0

Upload bigger 2GB
by Jochen Bick 13 Mar '19

13 Mar '19

Hi, since some months we have the problem that users are allowed to upload files bigger 2GB. Is there some kind of options in the galaxy config that can stop this? Our data storage is not able to handle such big data duplication, we used to use Shared Data->Data libraries to reduce data duplication. Thanks in advance. Cheers Jochen -- ETH Zurich *Jochen Bick* Animal Physiology Institute of Agricultural Sciences Postal address: Universitätstrasse 2 / LFW B 58.1 8092 Zurich, Switzerland Office: Eschikon 27 8315 Lindau, Switzerland Phone +41 52 354 92 06 jochen.bick(a)usys.ethz.ch <mailto:jochen.bick@usys.ethz.ch> www.ap.ethz.ch

2 2

Gateways 2019 Call for Participation (Abstracts due April 29; Full submissions due May 6)
by Katherine Lawrence 08 Mar '19

08 Mar '19

* Call for Participation: Gateways 2019(September 23–25, San Diego, California) is now accepting submissions of papers, demos, tutorials, and panels (2–4 pages) on the topic of gateways for science, engineering, or other disciplines. Gateways are user-friendly interfaces to scientific computing, data, and other domain-specific resources to support research and education. Topics may include their design, use, impact, development processes, sustainability, best practices, or any other aspect that you think fellow gateway creators or users will find interesting to learn. We also welcome educational topics directed toward the next generation of gateway creators. In addition, this year Gateways 2019 is co-located with the eScience <https://escience2019.sdsc.edu/>conference. The two conferences will offer shared sessions and connected registration.Abstracts are due April 29, and full submissions are due May 6, 2019.A poster session deadline (open to all attendees) will be August 15. Read more details in the Call for Participation:http://sciencegateways.org/gateways2019/call*

1 0