URL prefix doesn't work properly in Galaxy v18.09
by Ping Luo
I configured URL prefix in galaxy 18.09. It has problems of resolving some
of the links, including:
workflow
visualize -> create visualization
shared data -> workflows
shared data -> visualizations
I will use workflow as an example to illustrate what happens. When the
mouse cursor is put on workflow, the URL shown in at the left bottom of the
browser shows
https://xxx.xxx.tamu.edu/galaxy/workflows/list
But after I clicked 'workflow', the real URL shown in the address bar is
actually
https://xxx.xxxx.tamu.edu/galaxy/galaxy//workflows/list?__identifer=sk0xs...
Of course, there is no such a page and the browser shows a blank page. If I
manually remove one galaxy in the URL, the page is correctly shown.
Beside the four mentioned above, other links work fine. If galaxy is
configured without prefix, then there is no problem.
I also tested v18.05. With all the configuration being the same, this
version doesn't have the problem with prefix like v18.09 does. So I think
this is a bug in v18.09 with URL prefix.
Ping
6 months
Nginx+Shibboleth+UCSC
by Martin Demko
Dear list,
I have been trying to manage an access to UCSC genome browser from our Galaxy
instance which uses Nginx as a proxy with external authentication via
Shibboleth and IdP service.
Not even the configuration of nginx+shibboleth was quite complicated to
achieve (but I succeeded after a week or so) but now during a testing phase, we
have found out that a cooperation with external UCSC servers is forbidden due
to our auth. configuration.
I have found a way how to do that but only for Apache (https://docs.galaxyproject.org/en/master/admin/special_topics/apache.html...) so I tried to do something similar in Nginx but only with IP addresses as Nginx doesn't support using of named domains and defined a location:
location /display_as {
satisfy any;
deny all;
allow 128.114.119.131;
allow 128.114.119.132;
allow 128.114.119.133;
allow 128.114.119.134;
allow 128.114.119.135;
allow 128.114.119.136;
}
Used IP addresses should be the right ones for UCSC servers according to few
sources such as (https://genome.soe.ucsc.narkive.com/sll2JSk2/ucsc-ip-address):
hgw1.cse.ucsc.edu
hgw2.cse.ucsc.edu
hgw3.cse.ucsc.edu
hgw4.cse.ucsc.edu
hgw5.cse.ucsc.edu
hgw6.cse.ucsc.edu
I tried to change the location also to "/display_application" but didn't help
(actually, I don't know why in the documentation there is used as the location
that "/display_as" as I have never encountered such location in Galaxy which is
my first question). In galaxy.ini I set:
display_servers = hgw1.cse.ucsc.edu,hgw2.cse.ucsc.edu,hgw3.cse.ucsc.edu,hgw4.cse.ucsc.edu,hgw5.cse.ucsc.edu,hgw6.cse.ucsc.edu,hgw7.cse.ucsc.edu,hgw8.cse.ucsc.edu,lowepub.cse.ucsc.edu,128.114.119.131,128.114.119.132,128.114.119.133,128.114.119.134,128.114.119.135,128.114.119.136
Just for case I used both, the domains and also IP addresses. Also I have
tried to use only the domains or only the addresses but nothing helped.
Then I found out after reading of the following (https://www.switch.ch/aai/guides/sp/access-rules/) that the main obstacle should be actually shibboleth so I defined in /etc/shibboleth/shibboleth2.xml an unrestricted path:
<RequestMapper type="XML">
<RequestMap>
<Host name="our.galaxy.something"
authType="shibboleth"
requireSession="true"
redirectToSSL="443" >
<Path name="display_application" requireSession="false" redirectToSSL="443" />
<!--Path name="display_as" authType="shibboleth" requireSession="false"
redirectToSSL="443" /-->
</Host>
</RequestMap>
</RequestMapper>
Again, I have tried to use both locations "display_as" and "display_application", but it
haven't solved my problem entirely. I moved forward I suppose because now the
requests
should be going around the authentication but UCSC stil warns me that:
Expected 200 https://our.galaxy.something/display_application/e1304269a2f56a52/ucsc_bi...: 403 Forbidden
Before that it was a different message containing the address of our idp
server so I believe I'am on the right way here. The presence of attribute
'authType="shibboleth"' seams to be useless as 'requireSession="false"' is
used.
Then I tried to teach Nginx to use also the domain names with the using of 3rd
party nginx-http-rdns module from (https://github.com/flant/nginx-http-rdns),
so my nginx.conf looked like:
....only the part of nginx.conf....
############## Shibboleth authentication conf #################
# FastCGI authorizer for Shibboleth Auth Request module
location = /shibauthorizer {
internal;
include fastcgi_params;
fastcgi_pass unix:/var/run/shibboleth/shibauthorizer.sock;
}
# FastCGI responder for SSO
location /Shibboleth.sso {
include fastcgi_params;
fastcgi_pass unix:/var/run/shibboleth/shibresponder.sock;
}
############### Entry point for Galaxy ###############
# Location secured by Shibboleth
location / {
shib_request /shibauthorizer;
more_clear_input_headers 'Variable-*' 'Shib-*' 'Remote-User' 'REMOTE_USER'
'Auth-Type' 'AUTH_TYPE' 'GX_SECRET';
include uwsgi_params;
shib_request_set $shib_id $upstream_http_variable_eppn;
uwsgi_param HTTP_REMOTE_USER $shib_id;
uwsgi_param HTTP_GX_SECRET $our_secret;
uwsgi_pass 127.0.0.1:4001;
# resolver $correct_resolver_acc_to_resolv.conf;
# rdns on;
# satisfy any;
# rdns_allow genome\.ucsc\.edu;
# rdns_allow hgw1\.cse\.ucsc\.edu;
# rdns_allow hgw2\.cse\.ucsc\.edu;
# rdns_allow hgw3\.cse\.ucsc\.edu;
# rdns_allow hgw4\.cse\.ucsc\.edu;
# rdns_allow hgw5\.cse\.ucsc\.edu;
# rdns_allow hgw6\.cse\.ucsc\.edu;
# rdns_allow hgw7\.cse\.ucsc\.edu;
# rdns_allow hgw8\.cse\.ucsc\.edu;
}
# location /display_as {
# resolver $correct_resolver_acc_to_resolv.conf;
# rdns on;
# satisfy any;
# rdns_allow genome\.ucsc\.edu;
# rdns_allow hgw1\.cse\.ucsc\.edu;
# rdns_allow hgw2\.cse\.ucsc\.edu;
# rdns_allow hgw3\.cse\.ucsc\.edu;
# rdns_allow hgw4\.cse\.ucsc\.edu;
# rdns_allow hgw5\.cse\.ucsc\.edu;
# rdns_allow hgw6\.cse\.ucsc\.edu;
# rdns_allow hgw7\.cse\.ucsc\.edu;
# rdns_allow hgw8\.cse\.ucsc\.edu;
# }
# location /display_application {
# resolver $correct_resolver_acc_to_resolv.conf;
# rdns on;
# satisfy any;
# rdns_allow genome\.ucsc\.edu;
# rdns_allow hgw1\.cse\.ucsc\.edu;
# rdns_allow hgw2\.cse\.ucsc\.edu;
# rdns_allow hgw3\.cse\.ucsc\.edu;
# rdns_allow hgw4\.cse\.ucsc\.edu;
# rdns_allow hgw5\.cse\.ucsc\.edu;
# rdns_allow hgw6\.cse\.ucsc\.edu;
# rdns_allow hgw7\.cse\.ucsc\.edu;
# rdns_allow hgw8\.cse\.ucsc\.edu;
# }
....END of the part of nginx.conf....
Everything commented out, I tried to use together or one at the time but
nothing helped. But there are differences:
1) when uncommented only location /display_application - I will get just a
blank page with: 404 Not Found.
2) when uncommented only location /display_as - nothing changes.
3) when uncommented part inside the location "/" - I will get a new message:
Error unexpected end of input reading http header on https://our.galaxy.something/display_application/e1304269a2f56a52/ucsc_bi...
And now I'm out of ideas. I would really appreciate any help.
PS: I stil don't get why in the documentation for Apache configuration
(https://docs.galaxyproject.org/en/master/admin/special_topics/apache.html...)
there is used the /display_as location. Can somebody explain, please?
Thank you in advance,
Martin Demko
7 months
[GSoC 2019] [Open Genome Informatics] Google Summer of Code 2019 Participation
by Robin Haw
Dear All,
The Open Genome Informatics team serves as an “umbrella" organization to support the efforts of many open access open-source bioinformatics projects for Google Summer of Code (GSoC)<https://summerofcode.withgoogle.com/>. Among this list of projects are GMOD and its software projects -- GBrowse, JBrowse; Galaxy; Reactome; WormBase; DockStore; Bioconda; and others.
Call for 2019 Project Ideas and Mentors: We are seeking project ideas to post and attract talented students to this year’s Summer of Code competition. If you have a project idea for which you would like to mentor a student, please contact Robin Haw, Marc Gillespie, Dave Clements, Dannon Baker, and Scott Cain (emails above). You can also submit your ideas here<http://gmod.org/wiki/GSOC_Project_Ideas_2019> or this online form<https://docs.google.com/forms/d/1uQlpsbPCnhNwO5OFKAB-IiY9DLWlmaHZATnaeP6KHhw>.
For more information please refer to the Open Genome Informatics page<http://gmod.org/wiki/GSoC>.
The mentoring organization application deadline with GSoC is February 6th, 2019 at 3 pm EST. So, if you are interested in taking part with the team please let us know as soon as possible.
Please forward this to others who might be interested in taking part.
If you have any questions please let us know.
Thanks,
Robin, Marc, Dave, Dannon, and Scott.
9 months, 1 week
Question using sftp to upload file to galaxy
by Rui Wang
Hey Folks,
I'm looking at the instructions of using ftp with proftpd. There is a
section talking about extending it to use sftp. However, the sample config
isn't comprehensive. I'm wondering if anyone has a working config for
reference?
What's the setting of user and group? It says it should match the one in
the SQLNamedQuery, what does it mean exactly? I start proftpd as root, but
start galaxy as bioinfoadmin(normal user with sudo).
Just fyi, my proftpd config module and config file are pasted below. I'm
working it out on a trial and error fashion, please feel free to point out
if anything is wrong!
Cheers,
Rui
modules:
$ sbin/proftpd -l
Compiled-in modules:
mod_core.c
mod_xfer.c
mod_rlimit.c
mod_auth_unix.c
mod_auth.c
mod_ls.c
mod_log.c
mod_site.c
mod_delay.c
mod_facts.c
mod_sql.c
mod_sql_postgres.c
mod_sql_passwd.c
mod_sftp.c
mod_cap.c
etc/proftpd.conf
ServerType standalone
# You must put this in a virtual host if you want it to listen on its own
port. VHost != Apache Vhost.
<VirtualHost 10.3.17.42>
Port 2222
SFTPEngine on
AuthOrder mod_auth_unix.c mod_sql.c # If you don't do this you will get
weird disconnects
SFTPHostKey /etc/ssh/ssh_host_rsa_key
RequireValidShell no
MaxLoginAttempts 6
ServerName "Galaxy SFTP"
DefaultServer on
Umask 077
User bioinfoadmin
Group bioinfoadmin
UseFtpUsers off
DefaultRoot ~
AllowOverwrite on
AllowStoreRestart on
SQLEngine on
SQLGroupInfo sftp_groups name id members
# Do not authenticate against real (system) users
<IfModule mod_auth_pam.c>
AuthPAM off
</IfModule>
# Common SQL authentication options
SQLPasswordEngine on
SQLBackend postgres
SQLConnectInfo galaxy@galaxy.my.org:5432 bioinfoadmin dbpwd
SQLAuthenticate users
# Configuration that handles PBKDF2 encryption
# Set up mod_sql to authenticate against the Galaxy database
SQLAuthTypes PBKDF2
SQLPasswordPBKDF2 SHA256 10000 24
SQLPasswordEncoding base64
SQLPasswordUserSalt sql:/GetUserSalt
# Define a custom query for lookup that returns a passwd-like entry.
Replace 512s with the UID and GID of the user running the Galaxy server
SQLUserInfo custom:/LookupGalaxyUser
SQLNamedQuery LookupGalaxyUser SELECT "email, (CASE WHEN
substring(password from 1 for 6) = 'PBKDF2' THEN substring(password from 38
for 69) ELSE password END) AS
password2,512,512,'/media/galaxy/galaxy/database/ftp/%U','/bin/bash' FROM
galaxy_user WHERE email='%U'"
# Define custom query to fetch the password salt
SQLNamedQuery GetUserSalt SELECT "(CASE WHEN SUBSTRING
(password from 1 for 6) = 'PBKDF2' THEN SUBSTRING (password from 21 for 16)
END) AS salt FROM galaxy_user WHERE email='%U'"
</VirtualHost>
# Don't use IPv6 support by default.
UseIPv6 off
MaxInstances 30
# To cause every FTP user to be "jailed" (chrooted) into their home
# directory, uncomment this line.
# Bar use of SITE CHMOD by default
<Limit SITE_CHMOD>
DenyAll
</Limit>
# Bar use of RETR (download) since this is not a public file drop
<Limit RETR>
DenyAll
</Limit>
~
9 months, 2 weeks
Re: [galaxy-dev] [External] Copying files to FTP upload directory
by Keith Suderman
I have been creating directories for the users, but putting files in (for example) /home/galaxy/database/ftp/user(a)example.com <mailto:home/galaxy/database/ftp/user@example.com> does not make them available for the user(a)example.com <mailto:user@example.com> user in Galaxy.
Also for the list admins... I never did see my message appear on the list and the message Cicada replied to appears to be from galaxy-dev-bounces(a)lists.galaxyproject.org <mailto:galaxy-dev-bounces@lists.galaxyproject.org>. Any ideas why my message(s) appear to be bouncing. I am pretty sure I am sending from the correct address.
Cheers,
Keith
> On Jan 25, 2019, at 4:36 PM, Dennis, H. E. Cicada Brokaw <hbrokaw(a)iu.edu> wrote:
>
> I think you have to create the user's directories under /home/galaxy/database/ftp. I don't think Galaxy will create them if they are not there.
> Cicada
>
> ________________________________________
> From: galaxy-dev <galaxy-dev-bounces(a)lists.galaxyproject.org> on behalf of Keith Suderman <suderman(a)cs.vassar.edu>
> Sent: Friday, January 25, 2019 4:01 PM
> To: Galaxy Dev List
> Subject: [External] [galaxy-dev] Copying files to FTP upload directory
>
> Hello,
>
> I am trying to (ab)use the FTP file upload functionality to get data from an external service into Galaxy. From my reading of the docs I simply need to configure the ftp_upload_dir and ftp_upload_site in my galaxy.ini (haven't migrated to yaml yet) and Galaxy will expect to find subdirectories named after the user's account (email) in the ftp_upload_dir.
>
> I have configured the ftp_upload_dir (/home/galaxy/database/ftp in our case) and the "Choose FTP File" button is now available in the Upload dialog, however regardless of where I put files, or what I name them, Galaxy always reports that "Your FTP directory does not contain any files." For example, if I have the user account user(a)example.org I would expect to be able to place files in /home/galaxy/database/ftp/user(a)example.org and they will be available in the Upload dialog for the user(a)example.org user. However, this is not working; am I missing something?
>
> Thanks,
> Keith
>
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client. To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
> http://galaxyproject.org/search/
9 months, 2 weeks
Bugfix for CollectRNAmetrics (Picardtools)
by Previti
Dear all,
I just fixed a bug that prevented CollectRNAmetrics (a Picard-Tool,
galaxy version 2.7.1) from working with GTF annotation files:
The pre-processing step that converts gtf to the genepred format needs
two extra parameters (marked red):
gtfToGenePred -genePredExt -geneNameAsName2 'test.gtf' refFlat.tab.raw
The format of my GTF is:
Seqname Source Feature Start End Score Strand Frame Attributes
3R FlyBase gene 722370 722621 . - . gene_id "FBgn0085804";
gene_name "CR41571"; gene_source "FlyBase"; gene_biotype "pseudogene";
I'm assuming this is the "standard" GTF format (according to ensembl)
How's the best way to make this type of thing known...?
Cheers,
Christopher
--
*Dr. Christopher Previti*
Genomics and Proteomics Core Facility
High Throughput Sequencing (W190)
Bioinformatician
German Cancer Research Center (DKFZ)
Foundation under Public Law
Im Neuenheimer Feld 580
69120 Heidelberg
Germany
Room: B2.102 (INF580/TP3)
Phone: +49 6221 42-4661
christopher.previti(a)dkfz.de <http://www.dkfz.de/>
www.dkfz.de <http://www.dkfz.de/>
Management Board: Prof. Dr. Michael Baumann, Prof. Dr. Josef Puchta
VAT-ID No.: DE143293537
Vertraulichkeitshinweis: Diese Nachricht ist ausschließlich für die
Personen bestimmt, an die sie adressiert ist.
Sie kann vertrauliche und/oder nur für den/die Empfänger bestimmte
Informationen enthalten. Sollten Sie nicht
der bestimmungsgemäße Empfänger sein, kontaktieren Sie bitte den
Absender und löschen Sie die Mitteilung.
Jegliche unbefugte Verwendung der Informationen in dieser Nachricht ist
untersagt.
9 months, 3 weeks
Copying files to FTP upload directory
by Keith Suderman
Hello,
I am trying to (ab)use the FTP file upload functionality to get data from an external service into Galaxy. From my reading of the docs I simply need to configure the ftp_upload_dir and ftp_upload_site in my galaxy.ini (haven't migrated to yaml yet) and Galaxy will expect to find subdirectories named after the user's account (email) in the ftp_upload_dir.
I have configured the ftp_upload_dir (/home/galaxy/database/ftp in our case) and the "Choose FTP File" button is now available in the Upload dialog, however regardless of where I put files, or what I name them, Galaxy always reports that "Your FTP directory does not contain any files." For example, if I have the user account user(a)example.org I would expect to be able to place files in /home/galaxy/database/ftp/user(a)example.org and they will be available in the Upload dialog for the user(a)example.org user. However, this is not working; am I missing something?
Thanks,
Keith
9 months, 3 weeks
Input paired dataset collection in workflow
by Christopher Previti
Dear Galaxy Community,
Is there a way to input a paired dataset collection into a workflow?
I would like to build a workflow using cutadapt with a paired dataset
collection but...I'm missing something.
I have seen workflows (at usegalaxy.eu for example:
1_CLIPseq-Explorer_demulti_PEAKachu_iCLIP_hg19) where this is possible.
But how is this done practically?
Thanks and best regards,
Christopher
Genomics and Proteomics Core Facility
High Throughput Sequencing (W190)
Bioinformatician
German Cancer Research Center (DKFZ)
Foundation under Public Law
Im Neuenheimer Feld 580
69120 Heidelberg
Germany
Room: B2.102 (INF580/TP3)
Phone: +49 6221 42-4434
christopher.previti(a)dkfz.de <http://www.dkfz.de/>
www.dkfz.de <http://www.dkfz.de/>
logo <http://www.dkfz.de>
Management Board: Prof. Dr. Michael Baumann, Prof. Dr. Josef Puchta
VAT-ID No.: DE143293537
Vertraulichkeitshinweis: Diese Nachricht ist ausschließlich für die
Personen bestimmt, an die sie adressiert ist.
Sie kann vertrauliche und/oder nur für den/die Empfänger bestimmte
Informationen enthalten. Sollten Sie nicht
der bestimmungsgemäße Empfänger sein, kontaktieren Sie bitte den
Absender und löschen Sie die Mitteilung.
Jegliche unbefugte Verwendung der Informationen in dieser Nachricht ist
untersagt.
9 months, 4 weeks
usegalaxy.eu is hiring!
by Björn Grüning
Dear Friends,
we have a wide variety of active projects and are searching for a
software engineer (100%), system analysts/administrators (100%), data
analyst (100%), and a community and/or research manager (100%).
Read more at https://galaxyproject.eu/posts/2019/01/10/openpositions/
and join our team in Freiburg (Germany).
Talk to you soon :)
Björn, on behalf of the Freiburg Galaxy team
10 months, 1 week
