January 2019 - galaxy-dev - lists.galaxyproject.org

URL prefix doesn't work properly in Galaxy v18.09
by Ping Luo 20 May '19

20 May '19

I configured URL prefix in galaxy 18.09. It has problems of resolving some of the links, including: workflow visualize -> create visualization shared data -> workflows shared data -> visualizations I will use workflow as an example to illustrate what happens. When the mouse cursor is put on workflow, the URL shown in at the left bottom of the browser shows https://xxx.xxx.tamu.edu/galaxy/workflows/list But after I clicked 'workflow', the real URL shown in the address bar is actually https://xxx.xxxx.tamu.edu/galaxy/galaxy//workflows/list?__identifer=sk0xsyw… Of course, there is no such a page and the browser shows a blank page. If I manually remove one galaxy in the URL, the page is correctly shown. Beside the four mentioned above, other links work fine. If galaxy is configured without prefix, then there is no problem. I also tested v18.05. With all the configuration being the same, this version doesn't have the problem with prefix like v18.09 does. So I think this is a bug in v18.09 with URL prefix. Ping

2 3

Nginx+Shibboleth+UCSC
by Martin Demko 19 Apr '19

19 Apr '19

Dear list, I have been trying to manage an access to UCSC genome browser from our Galaxy instance which uses Nginx as a proxy with external authentication via Shibboleth and IdP service. Not even the configuration of nginx+shibboleth was quite complicated to achieve (but I succeeded after a week or so) but now during a testing phase, we have found out that a cooperation with external UCSC servers is forbidden due to our auth. configuration. I have found a way how to do that but only for Apache (https://docs.galaxyproject.org/en/master/admin/special_topics/apache.html#d…) so I tried to do something similar in Nginx but only with IP addresses as Nginx doesn't support using of named domains and defined a location: location /display_as { satisfy any; deny all; allow 128.114.119.131; allow 128.114.119.132; allow 128.114.119.133; allow 128.114.119.134; allow 128.114.119.135; allow 128.114.119.136; } Used IP addresses should be the right ones for UCSC servers according to few sources such as (https://genome.soe.ucsc.narkive.com/sll2JSk2/ucsc-ip-address) hgw1.cse.ucsc.edu hgw2.cse.ucsc.edu hgw3.cse.ucsc.edu hgw4.cse.ucsc.edu hgw5.cse.ucsc.edu hgw6.cse.ucsc.edu I tried to change the location also to "/display_application" but didn't help (actually, I don't know why in the documentation there is used as the location that "/display_as" as I have never encountered such location in Galaxy which is my first question). In galaxy.ini I set: display_servers = hgw1.cse.ucsc.edu,hgw2.cse.ucsc.edu,hgw3.cse.ucsc.edu,hgw4.cse.ucsc.edu,hgw5.cse.ucsc.edu,hgw6.cse.ucsc.edu,hgw7.cse.ucsc.edu,hgw8.cse.ucsc.edu,lowepub.cse.ucsc.edu,128.114.119.131,128.114.119.132,128.114.119.133,128.114.119.134,128.114.119.135,128.114.119.136 Just for case I used both, the domains and also IP addresses. Also I have tried to use only the domains or only the addresses but nothing helped. Then I found out after reading of the following (https://www.switch.ch/aai/guides/sp/access-rules/) that the main obstacle should be actually shibboleth so I defined in /etc/shibboleth/shibboleth2.xml an unrestricted path: <RequestMapper type="XML"> <RequestMap> <Host name="our.galaxy.something" authType="shibboleth" requireSession="true" redirectToSSL="443" > <Path name="display_application" requireSession="false" redirectToSSL="443" />  </Host> </RequestMap> </RequestMapper> Again, I have tried to use both locations "display_as" and "display_application", but it haven't solved my problem entirely. I moved forward I suppose because now the requests should be going around the authentication but UCSC stil warns me that: Expected 200 https://our.galaxy.something/display_application/e1304269a2f56a52/ucsc_bigw…: 403 Forbidden Before that it was a different message containing the address of our idp server so I believe I'am on the right way here. The presence of attribute 'authType="shibboleth"' seams to be useless as 'requireSession="false"' is used. Then I tried to teach Nginx to use also the domain names with the using of 3rd party nginx-http-rdns module from (https://github.com/flant/nginx-http-rdns) so my nginx.conf looked like: ....only the part of nginx.conf.... ############## Shibboleth authentication conf ################# # FastCGI authorizer for Shibboleth Auth Request module location = /shibauthorizer { internal; include fastcgi_params; fastcgi_pass unix:/var/run/shibboleth/shibauthorizer.sock; } # FastCGI responder for SSO location /Shibboleth.sso { include fastcgi_params; fastcgi_pass unix:/var/run/shibboleth/shibresponder.sock; } ############### Entry point for Galaxy ############### # Location secured by Shibboleth location / { shib_request /shibauthorizer; more_clear_input_headers 'Variable-*' 'Shib-*' 'Remote-User' 'REMOTE_USER' 'Auth-Type' 'AUTH_TYPE' 'GX_SECRET'; include uwsgi_params; shib_request_set $shib_id $upstream_http_variable_eppn; uwsgi_param HTTP_REMOTE_USER $shib_id; uwsgi_param HTTP_GX_SECRET $our_secret; uwsgi_pass 127.0.0.1:4001; # resolver $correct_resolver_acc_to_resolv.conf; # rdns on; # satisfy any; # rdns_allow genome\.ucsc\.edu; # rdns_allow hgw1\.cse\.ucsc\.edu; # rdns_allow hgw2\.cse\.ucsc\.edu; # rdns_allow hgw3\.cse\.ucsc\.edu; # rdns_allow hgw4\.cse\.ucsc\.edu; # rdns_allow hgw5\.cse\.ucsc\.edu; # rdns_allow hgw6\.cse\.ucsc\.edu; # rdns_allow hgw7\.cse\.ucsc\.edu; # rdns_allow hgw8\.cse\.ucsc\.edu; } # location /display_as { # resolver $correct_resolver_acc_to_resolv.conf; # rdns on; # satisfy any; # rdns_allow genome\.ucsc\.edu; # rdns_allow hgw1\.cse\.ucsc\.edu; # rdns_allow hgw2\.cse\.ucsc\.edu; # rdns_allow hgw3\.cse\.ucsc\.edu; # rdns_allow hgw4\.cse\.ucsc\.edu; # rdns_allow hgw5\.cse\.ucsc\.edu; # rdns_allow hgw6\.cse\.ucsc\.edu; # rdns_allow hgw7\.cse\.ucsc\.edu; # rdns_allow hgw8\.cse\.ucsc\.edu; # } # location /display_application { # resolver $correct_resolver_acc_to_resolv.conf; # rdns on; # satisfy any; # rdns_allow genome\.ucsc\.edu; # rdns_allow hgw1\.cse\.ucsc\.edu; # rdns_allow hgw2\.cse\.ucsc\.edu; # rdns_allow hgw3\.cse\.ucsc\.edu; # rdns_allow hgw4\.cse\.ucsc\.edu; # rdns_allow hgw5\.cse\.ucsc\.edu; # rdns_allow hgw6\.cse\.ucsc\.edu; # rdns_allow hgw7\.cse\.ucsc\.edu; # rdns_allow hgw8\.cse\.ucsc\.edu; # } ....END of the part of nginx.conf.... Everything commented out, I tried to use together or one at the time but nothing helped. But there are differences: 1) when uncommented only location /display_application - I will get just a blank page with: 404 Not Found. 2) when uncommented only location /display_as - nothing changes. 3) when uncommented part inside the location "/" - I will get a new message: Error unexpected end of input reading http header on https://our.galaxy.something/display_application/e1304269a2f56a52/ucsc_bigw… And now I'm out of ideas. I would really appreciate any help. PS: I stil don't get why in the documentation for Apache configuration (https://docs.galaxyproject.org/en/master/admin/special_topics/apache.html#d…) there is used the /display_as location. Can somebody explain, please? Thank you in advance, Martin Demko

2 1

[GSoC 2019] [Open Genome Informatics] Google Summer of Code 2019 Participation
by Robin Haw 11 Feb '19

11 Feb '19

Dear All, The Open Genome Informatics team serves as an “umbrella" organization to support the efforts of many open access open-source bioinformatics projects for Google Summer of Code (GSoC)<https://summerofcode.withgoogle.com/>. Among this list of projects are GMOD and its software projects -- GBrowse, JBrowse; Galaxy; Reactome; WormBase; DockStore; Bioconda; and others. Call for 2019 Project Ideas and Mentors: We are seeking project ideas to post and attract talented students to this year’s Summer of Code competition. If you have a project idea for which you would like to mentor a student, please contact Robin Haw, Marc Gillespie, Dave Clements, Dannon Baker, and Scott Cain (emails above). You can also submit your ideas here<http://gmod.org/wiki/GSOC_Project_Ideas_2019> or this online form<https://docs.google.com/forms/d/1uQlpsbPCnhNwO5OFKAB-IiY9DLWlmaHZATnaeP6KHhw>. For more information please refer to the Open Genome Informatics page<http://gmod.org/wiki/GSoC>. The mentoring organization application deadline with GSoC is February 6th, 2019 at 3 pm EST. So, if you are interested in taking part with the team please let us know as soon as possible. Please forward this to others who might be interested in taking part. If you have any questions please let us know. Thanks, Robin, Marc, Dave, Dannon, and Scott.

1 1

Question using sftp to upload file to galaxy
by Rui Wang 06 Feb '19

06 Feb '19

Hey Folks, I'm looking at the instructions of using ftp with proftpd. There is a section talking about extending it to use sftp. However, the sample config isn't comprehensive. I'm wondering if anyone has a working config for reference? What's the setting of user and group? It says it should match the one in the SQLNamedQuery, what does it mean exactly? I start proftpd as root, but start galaxy as bioinfoadmin(normal user with sudo). Just fyi, my proftpd config module and config file are pasted below. I'm working it out on a trial and error fashion, please feel free to point out if anything is wrong! Cheers, Rui modules: $ sbin/proftpd -l Compiled-in modules: mod_core.c mod_xfer.c mod_rlimit.c mod_auth_unix.c mod_auth.c mod_ls.c mod_log.c mod_site.c mod_delay.c mod_facts.c mod_sql.c mod_sql_postgres.c mod_sql_passwd.c mod_sftp.c mod_cap.c etc/proftpd.conf ServerType standalone # You must put this in a virtual host if you want it to listen on its own port. VHost != Apache Vhost. <VirtualHost 10.3.17.42> Port 2222 SFTPEngine on AuthOrder mod_auth_unix.c mod_sql.c # If you don't do this you will get weird disconnects SFTPHostKey /etc/ssh/ssh_host_rsa_key RequireValidShell no MaxLoginAttempts 6 ServerName "Galaxy SFTP" DefaultServer on Umask 077 User bioinfoadmin Group bioinfoadmin UseFtpUsers off DefaultRoot ~ AllowOverwrite on AllowStoreRestart on SQLEngine on SQLGroupInfo sftp_groups name id members # Do not authenticate against real (system) users <IfModule mod_auth_pam.c> AuthPAM off </IfModule> # Common SQL authentication options SQLPasswordEngine on SQLBackend postgres SQLConnectInfo galaxy@galaxy.my.org:5432 bioinfoadmin dbpwd SQLAuthenticate users # Configuration that handles PBKDF2 encryption # Set up mod_sql to authenticate against the Galaxy database SQLAuthTypes PBKDF2 SQLPasswordPBKDF2 SHA256 10000 24 SQLPasswordEncoding base64 SQLPasswordUserSalt sql:/GetUserSalt # Define a custom query for lookup that returns a passwd-like entry. Replace 512s with the UID and GID of the user running the Galaxy server SQLUserInfo custom:/LookupGalaxyUser SQLNamedQuery LookupGalaxyUser SELECT "email, (CASE WHEN substring(password from 1 for 6) = 'PBKDF2' THEN substring(password from 38 for 69) ELSE password END) AS password2,512,512,'/media/galaxy/galaxy/database/ftp/%U','/bin/bash' FROM galaxy_user WHERE email='%U'" # Define custom query to fetch the password salt SQLNamedQuery GetUserSalt SELECT "(CASE WHEN SUBSTRING (password from 1 for 6) = 'PBKDF2' THEN SUBSTRING (password from 21 for 16) END) AS salt FROM galaxy_user WHERE email='%U'" </VirtualHost> # Don't use IPv6 support by default. UseIPv6 off MaxInstances 30 # To cause every FTP user to be "jailed" (chrooted) into their home # directory, uncomment this line. # Bar use of SITE CHMOD by default <Limit SITE_CHMOD> DenyAll </Limit> # Bar use of RETR (download) since this is not a public file drop <Limit RETR> DenyAll </Limit> ~

2 4

Re: [galaxy-dev] [External] Copying files to FTP upload directory
by Keith Suderman 01 Feb '19

01 Feb '19

I have been creating directories for the users, but putting files in (for example) /home/galaxy/database/ftp/user(a)example.com <mailto:home/galaxy/database/ftp/user@example.com> does not make them available for the user(a)example.com <mailto:user@example.com> user in Galaxy. Also for the list admins... I never did see my message appear on the list and the message Cicada replied to appears to be from galaxy-dev-bounces(a)lists.galaxyproject.org <mailto:galaxy-dev-bounces@lists.galaxyproject.org>. Any ideas why my message(s) appear to be bouncing. I am pretty sure I am sending from the correct address. Cheers, Keith > On Jan 25, 2019, at 4:36 PM, Dennis, H. E. Cicada Brokaw <hbrokaw(a)iu.edu> wrote: > > I think you have to create the user's directories under /home/galaxy/database/ftp. I don't think Galaxy will create them if they are not there. > Cicada > > ________________________________________ > From: galaxy-dev <galaxy-dev-bounces(a)lists.galaxyproject.org> on behalf of Keith Suderman <suderman(a)cs.vassar.edu> > Sent: Friday, January 25, 2019 4:01 PM > To: Galaxy Dev List > Subject: [External] [galaxy-dev] Copying files to FTP upload directory > > Hello, > > I am trying to (ab)use the FTP file upload functionality to get data from an external service into Galaxy. From my reading of the docs I simply need to configure the ftp_upload_dir and ftp_upload_site in my galaxy.ini (haven't migrated to yaml yet) and Galaxy will expect to find subdirectories named after the user's account (email) in the ftp_upload_dir. > > I have configured the ftp_upload_dir (/home/galaxy/database/ftp in our case) and the "Choose FTP File" button is now available in the Upload dialog, however regardless of where I put files, or what I name them, Galaxy always reports that "Your FTP directory does not contain any files." For example, if I have the user account user(a)example.org I would expect to be able to place files in /home/galaxy/database/ftp/user(a)example.org and they will be available in the Upload dialog for the user(a)example.org user. However, this is not working; am I missing something? > > Thanks, > Keith > > > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > https://lists.galaxyproject.org/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/

2 3

Bugfix for CollectRNAmetrics (Picardtools)
by Previti 29 Jan '19

29 Jan '19

Dear all, I just fixed a bug that prevented CollectRNAmetrics (a Picard-Tool, galaxy version 2.7.1) from working with GTF annotation files: The pre-processing step that converts gtf to the genepred format needs two extra parameters (marked red): gtfToGenePred -genePredExt -geneNameAsName2 'test.gtf' refFlat.tab.raw The format of my GTF is: Seqname Source Feature Start End Score Strand Frame Attributes 3R FlyBase gene 722370 722621 . - . gene_id "FBgn0085804"; gene_name "CR41571"; gene_source "FlyBase"; gene_biotype "pseudogene"; I'm assuming this is the "standard" GTF format (according to ensembl) How's the best way to make this type of thing known...? Cheers, Christopher -- *Dr. Christopher Previti* Genomics and Proteomics Core Facility High Throughput Sequencing (W190) Bioinformatician German Cancer Research Center (DKFZ) Foundation under Public Law Im Neuenheimer Feld 580 69120 Heidelberg Germany Room: B2.102 (INF580/TP3) Phone: +49 6221 42-4661 christopher.previti(a)dkfz.de <http://www.dkfz.de/> www.dkfz.de <http://www.dkfz.de/> Management Board: Prof. Dr. Michael Baumann, Prof. Dr. Josef Puchta VAT-ID No.: DE143293537 Vertraulichkeitshinweis: Diese Nachricht ist ausschließlich für die Personen bestimmt, an die sie adressiert ist. Sie kann vertrauliche und/oder nur für den/die Empfänger bestimmte Informationen enthalten. Sollten Sie nicht der bestimmungsgemäße Empfänger sein, kontaktieren Sie bitte den Absender und löschen Sie die Mitteilung. Jegliche unbefugte Verwendung der Informationen in dieser Nachricht ist untersagt.

2 2

Vote Now: GCC2019 Training Topics
by Dave Clements 28 Jan '19

28 Jan '19

Hello all, The next 2019 Galaxy Community Conference (GCC2019) <https://galaxyproject.org/events/gcc2019/> will be in Freiburg (Germany) in July. This year the program is specifically aimed at drawing in more Galaxy end-users, including for training. We are really interested it what you would like to learn. *Please take a few minutes to decide <https://galaxyproject.org/events/gcc2019/training/#training-topic-votes> and then vote which training should be offered during GCC2019 <https://goo.gl/forms/iaY5AezdFpyJ0ETh2>! *If there is *any* chance that you will attend then we want to hear from you. *Topic voting closes 4 February.* The Training schedule, including instructors, will be published before early registration opens. GCC2019 <https://galaxyproject.org/events/gcc2019/> will be held 1-6 July in Freiburg, Germany. If you work in data-intensive biomedical research, there is no better place than GCC2019 to present your work and to learn from others. Thanks, and see you in Freiburg The GCC2019 Organizers <https://galaxyproject.org/events/gcc2019/organizers/> -- https://galaxyproject.org/ https://usegalaxy.org/

1 0

Copying files to FTP upload directory
by Keith Suderman 25 Jan '19

25 Jan '19

Hello, I am trying to (ab)use the FTP file upload functionality to get data from an external service into Galaxy. From my reading of the docs I simply need to configure the ftp_upload_dir and ftp_upload_site in my galaxy.ini (haven't migrated to yaml yet) and Galaxy will expect to find subdirectories named after the user's account (email) in the ftp_upload_dir. I have configured the ftp_upload_dir (/home/galaxy/database/ftp in our case) and the "Choose FTP File" button is now available in the Upload dialog, however regardless of where I put files, or what I name them, Galaxy always reports that "Your FTP directory does not contain any files." For example, if I have the user account user(a)example.org I would expect to be able to place files in /home/galaxy/database/ftp/user(a)example.org and they will be available in the Upload dialog for the user(a)example.org user. However, this is not working; am I missing something? Thanks, Keith

1 0

Input paired dataset collection in workflow
by Christopher Previti 24 Jan '19

24 Jan '19

Dear Galaxy Community, Is there a way to input a paired dataset collection into a workflow? I would like to build a workflow using cutadapt with a paired dataset collection but...I'm missing something. I have seen workflows (at usegalaxy.eu for example: 1_CLIPseq-Explorer_demulti_PEAKachu_iCLIP_hg19) where this is possible. But how is this done practically? Thanks and best regards, Christopher Genomics and Proteomics Core Facility High Throughput Sequencing (W190) Bioinformatician German Cancer Research Center (DKFZ) Foundation under Public Law Im Neuenheimer Feld 580 69120 Heidelberg Germany Room: B2.102 (INF580/TP3) Phone: +49 6221 42-4434 christopher.previti(a)dkfz.de <http://www.dkfz.de/> www.dkfz.de <http://www.dkfz.de/> logo <http://www.dkfz.de> Management Board: Prof. Dr. Michael Baumann, Prof. Dr. Josef Puchta VAT-ID No.: DE143293537 Vertraulichkeitshinweis: Diese Nachricht ist ausschließlich für die Personen bestimmt, an die sie adressiert ist. Sie kann vertrauliche und/oder nur für den/die Empfänger bestimmte Informationen enthalten. Sollten Sie nicht der bestimmungsgemäße Empfänger sein, kontaktieren Sie bitte den Absender und löschen Sie die Mitteilung. Jegliche unbefugte Verwendung der Informationen in dieser Nachricht ist untersagt.

1 1

usegalaxy.eu is hiring!
by Björn Grüning 13 Jan '19

13 Jan '19

Dear Friends, we have a wide variety of active projects and are searching for a software engineer (100%), system analysts/administrators (100%), data analyst (100%), and a community and/or research manager (100%). Read more at https://galaxyproject.eu/posts/2019/01/10/openpositions/ and join our team in Freiburg (Germany). Talk to you soon :) Björn, on behalf of the Freiburg Galaxy team

1 0