URL prefix doesn't work properly in Galaxy v18.09
by Ping Luo
I configured URL prefix in galaxy 18.09. It has problems of resolving some
of the links, including:
workflow
visualize -> create visualization
shared data -> workflows
shared data -> visualizations
I will use workflow as an example to illustrate what happens. When the
mouse cursor is put on workflow, the URL shown in at the left bottom of the
browser shows
https://xxx.xxx.tamu.edu/galaxy/workflows/list
But after I clicked 'workflow', the real URL shown in the address bar is
actually
https://xxx.xxxx.tamu.edu/galaxy/galaxy//workflows/list?__identifer=sk0xs...
Of course, there is no such a page and the browser shows a blank page. If I
manually remove one galaxy in the URL, the page is correctly shown.
Beside the four mentioned above, other links work fine. If galaxy is
configured without prefix, then there is no problem.
I also tested v18.05. With all the configuration being the same, this
version doesn't have the problem with prefix like v18.09 does. So I think
this is a bug in v18.09 with URL prefix.
Ping
3 years, 8 months
Nginx+Shibboleth+UCSC
by Martin Demko
Dear list,
I have been trying to manage an access to UCSC genome browser from our Galaxy
instance which uses Nginx as a proxy with external authentication via
Shibboleth and IdP service.
Not even the configuration of nginx+shibboleth was quite complicated to
achieve (but I succeeded after a week or so) but now during a testing phase, we
have found out that a cooperation with external UCSC servers is forbidden due
to our auth. configuration.
I have found a way how to do that but only for Apache (https://docs.galaxyproject.org/en/master/admin/special_topics/apache.html...) so I tried to do something similar in Nginx but only with IP addresses as Nginx doesn't support using of named domains and defined a location:
location /display_as {
satisfy any;
deny all;
allow 128.114.119.131;
allow 128.114.119.132;
allow 128.114.119.133;
allow 128.114.119.134;
allow 128.114.119.135;
allow 128.114.119.136;
}
Used IP addresses should be the right ones for UCSC servers according to few
sources such as (https://genome.soe.ucsc.narkive.com/sll2JSk2/ucsc-ip-address):
hgw1.cse.ucsc.edu
hgw2.cse.ucsc.edu
hgw3.cse.ucsc.edu
hgw4.cse.ucsc.edu
hgw5.cse.ucsc.edu
hgw6.cse.ucsc.edu
I tried to change the location also to "/display_application" but didn't help
(actually, I don't know why in the documentation there is used as the location
that "/display_as" as I have never encountered such location in Galaxy which is
my first question). In galaxy.ini I set:
display_servers = hgw1.cse.ucsc.edu,hgw2.cse.ucsc.edu,hgw3.cse.ucsc.edu,hgw4.cse.ucsc.edu,hgw5.cse.ucsc.edu,hgw6.cse.ucsc.edu,hgw7.cse.ucsc.edu,hgw8.cse.ucsc.edu,lowepub.cse.ucsc.edu,128.114.119.131,128.114.119.132,128.114.119.133,128.114.119.134,128.114.119.135,128.114.119.136
Just for case I used both, the domains and also IP addresses. Also I have
tried to use only the domains or only the addresses but nothing helped.
Then I found out after reading of the following (https://www.switch.ch/aai/guides/sp/access-rules/) that the main obstacle should be actually shibboleth so I defined in /etc/shibboleth/shibboleth2.xml an unrestricted path:
<RequestMapper type="XML">
<RequestMap>
<Host name="our.galaxy.something"
authType="shibboleth"
requireSession="true"
redirectToSSL="443" >
<Path name="display_application" requireSession="false" redirectToSSL="443" />
<!--Path name="display_as" authType="shibboleth" requireSession="false"
redirectToSSL="443" /-->
</Host>
</RequestMap>
</RequestMapper>
Again, I have tried to use both locations "display_as" and "display_application", but it
haven't solved my problem entirely. I moved forward I suppose because now the
requests
should be going around the authentication but UCSC stil warns me that:
Expected 200 https://our.galaxy.something/display_application/e1304269a2f56a52/ucsc_bi...: 403 Forbidden
Before that it was a different message containing the address of our idp
server so I believe I'am on the right way here. The presence of attribute
'authType="shibboleth"' seams to be useless as 'requireSession="false"' is
used.
Then I tried to teach Nginx to use also the domain names with the using of 3rd
party nginx-http-rdns module from (https://github.com/flant/nginx-http-rdns),
so my nginx.conf looked like:
....only the part of nginx.conf....
############## Shibboleth authentication conf #################
# FastCGI authorizer for Shibboleth Auth Request module
location = /shibauthorizer {
internal;
include fastcgi_params;
fastcgi_pass unix:/var/run/shibboleth/shibauthorizer.sock;
}
# FastCGI responder for SSO
location /Shibboleth.sso {
include fastcgi_params;
fastcgi_pass unix:/var/run/shibboleth/shibresponder.sock;
}
############### Entry point for Galaxy ###############
# Location secured by Shibboleth
location / {
shib_request /shibauthorizer;
more_clear_input_headers 'Variable-*' 'Shib-*' 'Remote-User' 'REMOTE_USER'
'Auth-Type' 'AUTH_TYPE' 'GX_SECRET';
include uwsgi_params;
shib_request_set $shib_id $upstream_http_variable_eppn;
uwsgi_param HTTP_REMOTE_USER $shib_id;
uwsgi_param HTTP_GX_SECRET $our_secret;
uwsgi_pass 127.0.0.1:4001;
# resolver $correct_resolver_acc_to_resolv.conf;
# rdns on;
# satisfy any;
# rdns_allow genome\.ucsc\.edu;
# rdns_allow hgw1\.cse\.ucsc\.edu;
# rdns_allow hgw2\.cse\.ucsc\.edu;
# rdns_allow hgw3\.cse\.ucsc\.edu;
# rdns_allow hgw4\.cse\.ucsc\.edu;
# rdns_allow hgw5\.cse\.ucsc\.edu;
# rdns_allow hgw6\.cse\.ucsc\.edu;
# rdns_allow hgw7\.cse\.ucsc\.edu;
# rdns_allow hgw8\.cse\.ucsc\.edu;
}
# location /display_as {
# resolver $correct_resolver_acc_to_resolv.conf;
# rdns on;
# satisfy any;
# rdns_allow genome\.ucsc\.edu;
# rdns_allow hgw1\.cse\.ucsc\.edu;
# rdns_allow hgw2\.cse\.ucsc\.edu;
# rdns_allow hgw3\.cse\.ucsc\.edu;
# rdns_allow hgw4\.cse\.ucsc\.edu;
# rdns_allow hgw5\.cse\.ucsc\.edu;
# rdns_allow hgw6\.cse\.ucsc\.edu;
# rdns_allow hgw7\.cse\.ucsc\.edu;
# rdns_allow hgw8\.cse\.ucsc\.edu;
# }
# location /display_application {
# resolver $correct_resolver_acc_to_resolv.conf;
# rdns on;
# satisfy any;
# rdns_allow genome\.ucsc\.edu;
# rdns_allow hgw1\.cse\.ucsc\.edu;
# rdns_allow hgw2\.cse\.ucsc\.edu;
# rdns_allow hgw3\.cse\.ucsc\.edu;
# rdns_allow hgw4\.cse\.ucsc\.edu;
# rdns_allow hgw5\.cse\.ucsc\.edu;
# rdns_allow hgw6\.cse\.ucsc\.edu;
# rdns_allow hgw7\.cse\.ucsc\.edu;
# rdns_allow hgw8\.cse\.ucsc\.edu;
# }
....END of the part of nginx.conf....
Everything commented out, I tried to use together or one at the time but
nothing helped. But there are differences:
1) when uncommented only location /display_application - I will get just a
blank page with: 404 Not Found.
2) when uncommented only location /display_as - nothing changes.
3) when uncommented part inside the location "/" - I will get a new message:
Error unexpected end of input reading http header on https://our.galaxy.something/display_application/e1304269a2f56a52/ucsc_bi...
And now I'm out of ideas. I would really appreciate any help.
PS: I stil don't get why in the documentation for Apache configuration
(https://docs.galaxyproject.org/en/master/admin/special_topics/apache.html...)
there is used the /display_as location. Can somebody explain, please?
Thank you in advance,
Martin Demko
3 years, 9 months
A question about Galaxy
by Abdullah El Kurdi
Dear all,
I would like to ask if is there any way that I could copy my galaxy docker instance from a server to another including the "export" directory, without having to re-install all the tools again.
Thanks in advance.
3 years, 11 months
[GSoC 2019] [Open Genome Informatics] Google Summer of Code 2019 Participation
by Robin Haw
Dear All,
The Open Genome Informatics team serves as an “umbrella" organization to support the efforts of many open access open-source bioinformatics projects for Google Summer of Code (GSoC)<https://summerofcode.withgoogle.com/>. Among this list of projects are GMOD and its software projects -- GBrowse, JBrowse; Galaxy; Reactome; WormBase; DockStore; Bioconda; and others.
Call for 2019 Project Ideas and Mentors: We are seeking project ideas to post and attract talented students to this year’s Summer of Code competition. If you have a project idea for which you would like to mentor a student, please contact Robin Haw, Marc Gillespie, Dave Clements, Dannon Baker, and Scott Cain (emails above). You can also submit your ideas here<http://gmod.org/wiki/GSOC_Project_Ideas_2019> or this online form<https://docs.google.com/forms/d/1uQlpsbPCnhNwO5OFKAB-IiY9DLWlmaHZATnaeP6KHhw>.
For more information please refer to the Open Genome Informatics page<http://gmod.org/wiki/GSoC>.
The mentoring organization application deadline with GSoC is February 6th, 2019 at 3 pm EST. So, if you are interested in taking part with the team please let us know as soon as possible.
Please forward this to others who might be interested in taking part.
If you have any questions please let us know.
Thanks,
Robin, Marc, Dave, Dannon, and Scott.
3 years, 12 months
Uncaught exception in exposed API method:
by Jochen Bick
Hi,
I'm having this error since some minutes. The error message is coming in
the moment I click to any tool (e.g.: featurecounts) that provides data
collections. I think it has to do with a collection in my history. Here
some detail of my log file:
galaxy.web.framework.decorators ERROR 2019-02-06 15:50:09,026 Uncaught
exception in exposed API method:
Traceback (most recent call last):
File
"/opt/galaxy/production/galaxy-dist/lib/galaxy/web/framework/decorators.py",
line 283, in decorator
rval = func(self, trans, *args, **kwargs)
File
"/opt/galaxy/production/galaxy-dist/lib/galaxy/webapps/galaxy/api/tools.py",
line 113, in build
return tool.to_json(trans, kwd.get('inputs', kwd))
File
"/opt/galaxy/production/galaxy-dist/lib/galaxy/tools/__init__.py", line
1903, in to_json
self.populate_model(request_context, self.inputs, state_inputs,
tool_model['inputs'])
File
"/opt/galaxy/production/galaxy-dist/lib/galaxy/tools/__init__.py", line
1951, in populate_model
tool_dict = input.to_dict(request_context)
File
"/opt/galaxy/production/galaxy-dist/lib/galaxy/tools/parameters/grouping.py",
line 701, in to_dict
cond_dict["cases"] = list(map(nested_to_dict, self.cases))
File
"/opt/galaxy/production/galaxy-dist/lib/galaxy/tools/parameters/grouping.py",
line 699, in nested_to_dict
return input.to_dict(trans)
File
"/opt/galaxy/production/galaxy-dist/lib/galaxy/tools/parameters/grouping.py",
line 719, in to_dict
when_dict["inputs"] = list(map(input_to_dict, self.inputs.values()))
File
"/opt/galaxy/production/galaxy-dist/lib/galaxy/tools/parameters/grouping.py",
line 717, in input_to_dict
return input.to_dict(trans)
File
"/opt/galaxy/production/galaxy-dist/lib/galaxy/tools/parameters/basic.py",
line 1966, in to_dict
match = dataset_collection_matcher.hdca_match(hdca)
File
"/opt/galaxy/production/galaxy-dist/lib/galaxy/tools/parameters/dataset_matcher.py",
line 226, in hdca_match
converted_ext, _ =
datatypes_registry.find_conversion_destination_for_dataset_by_extensions(extension,
formats)
File
"/opt/galaxy/production/galaxy-dist/lib/galaxy/datatypes/registry.py",
line 892, in find_conversion_destination_for_dataset_by_extensions
self.log.warning("Datatype class not found for extension '%s',
which is used as target for conversion from datatype '%s'" %
(convert_ext, dataset.ext))
AttributeError: 'NoneType' object has no attribute 'ext'
82.130.95.243 - - [06/Feb/2019:15:50:08 +0200] "GET
/api/tools/toolshed.g2.bx.psu.edu/repos/iuc/featurecounts/featurecounts/1.6.3%2Bgalaxy2/build?version=1.6.3%2Bgalaxy2&__identifer=ku6ydfn1c8&tool_version=1.6.3%252Bgalaxy2
HTTP/1.1" 500 -
"http://green-l-bio.ethz.ch:8090/?tool_id=toolshed.g2.bx.psu.edu%2Frepos%2..."
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/72.0.3626.81 Safari/537.36"
Any ideas? Errors only occur if I'm in that history. I can also provide
more information. This is running on a Galaxy 18.09.
The data collections contains 8 fastq files.
Cheers Jochen
--
ETH Zurich
*Jochen Bick*
Animal Physiology
Institute of Agricultural Sciences
Postal address: Universitätstrasse 2 / LFW B 58.1
8092 Zurich, Switzerland
Office: Eschikon 27
8315 Lindau, Switzerland
Phone +41 52 354 92 06
jochen.bick(a)usys.ethz.ch <mailto:jochen.bick@usys.ethz.ch>
www.ap.ethz.ch
4 years
Question using sftp to upload file to galaxy
by Rui Wang
Hey Folks,
I'm looking at the instructions of using ftp with proftpd. There is a
section talking about extending it to use sftp. However, the sample config
isn't comprehensive. I'm wondering if anyone has a working config for
reference?
What's the setting of user and group? It says it should match the one in
the SQLNamedQuery, what does it mean exactly? I start proftpd as root, but
start galaxy as bioinfoadmin(normal user with sudo).
Just fyi, my proftpd config module and config file are pasted below. I'm
working it out on a trial and error fashion, please feel free to point out
if anything is wrong!
Cheers,
Rui
modules:
$ sbin/proftpd -l
Compiled-in modules:
mod_core.c
mod_xfer.c
mod_rlimit.c
mod_auth_unix.c
mod_auth.c
mod_ls.c
mod_log.c
mod_site.c
mod_delay.c
mod_facts.c
mod_sql.c
mod_sql_postgres.c
mod_sql_passwd.c
mod_sftp.c
mod_cap.c
etc/proftpd.conf
ServerType standalone
# You must put this in a virtual host if you want it to listen on its own
port. VHost != Apache Vhost.
<VirtualHost 10.3.17.42>
Port 2222
SFTPEngine on
AuthOrder mod_auth_unix.c mod_sql.c # If you don't do this you will get
weird disconnects
SFTPHostKey /etc/ssh/ssh_host_rsa_key
RequireValidShell no
MaxLoginAttempts 6
ServerName "Galaxy SFTP"
DefaultServer on
Umask 077
User bioinfoadmin
Group bioinfoadmin
UseFtpUsers off
DefaultRoot ~
AllowOverwrite on
AllowStoreRestart on
SQLEngine on
SQLGroupInfo sftp_groups name id members
# Do not authenticate against real (system) users
<IfModule mod_auth_pam.c>
AuthPAM off
</IfModule>
# Common SQL authentication options
SQLPasswordEngine on
SQLBackend postgres
SQLConnectInfo galaxy@galaxy.my.org:5432 bioinfoadmin dbpwd
SQLAuthenticate users
# Configuration that handles PBKDF2 encryption
# Set up mod_sql to authenticate against the Galaxy database
SQLAuthTypes PBKDF2
SQLPasswordPBKDF2 SHA256 10000 24
SQLPasswordEncoding base64
SQLPasswordUserSalt sql:/GetUserSalt
# Define a custom query for lookup that returns a passwd-like entry.
Replace 512s with the UID and GID of the user running the Galaxy server
SQLUserInfo custom:/LookupGalaxyUser
SQLNamedQuery LookupGalaxyUser SELECT "email, (CASE WHEN
substring(password from 1 for 6) = 'PBKDF2' THEN substring(password from 38
for 69) ELSE password END) AS
password2,512,512,'/media/galaxy/galaxy/database/ftp/%U','/bin/bash' FROM
galaxy_user WHERE email='%U'"
# Define custom query to fetch the password salt
SQLNamedQuery GetUserSalt SELECT "(CASE WHEN SUBSTRING
(password from 1 for 6) = 'PBKDF2' THEN SUBSTRING (password from 21 for 16)
END) AS salt FROM galaxy_user WHERE email='%U'"
</VirtualHost>
# Don't use IPv6 support by default.
UseIPv6 off
MaxInstances 30
# To cause every FTP user to be "jailed" (chrooted) into their home
# directory, uncomment this line.
# Bar use of SITE CHMOD by default
<Limit SITE_CHMOD>
DenyAll
</Limit>
# Bar use of RETR (download) since this is not a public file drop
<Limit RETR>
DenyAll
</Limit>
~
4 years
featureCounts with GFF
by Jochen Bick
Hi,
our group would like to switch to featureCounts
<http://green-l-bio.ethz.ch:8090/tool_runner?tool_id=toolshed.g2.bx.psu.ed...>
but we are using only GFF from NCBI. It looks like featureCounts
<http://green-l-bio.ethz.ch:8090/tool_runner?tool_id=toolshed.g2.bx.psu.ed...>
provides GFF but out of the box it does not provide a proper count
table. The output looks like this:
Status HISAT2 on data 33
Assigned 5186527
Unassigned_Unmapped 518985
Unassigned_MappingQuality 1353602
Unassigned_Chimera 0
Unassigned_FragmentLength 0
Unassigned_Duplicate 0
Unassigned_MultiMapping 0
Unassigned_Secondary 0
Unassigned_Nonjunction 0
Unassigned_NoFeatures 7515020
Unassigned_Overlapping_Length 0
Unassigned_Ambiguity 0
Geneid HISAT2 on data 33
5186527
I guess I need some advanced options? Which are different to the default
run settings?
Does anyone have a tutorial or example on how to configure the tool
correctly with a standard GFF file, or does anyone also use GFF files?
Or is there a better tool to use?
Cheers Jochen
--
ETH Zurich
*Jochen Bick*
Animal Physiology
Institute of Agricultural Sciences
Postal address: Universitätstrasse 2 / LFW B 58.1
8092 Zurich, Switzerland
Office: Eschikon 27
8315 Lindau, Switzerland
Phone +41 52 354 92 06
jochen.bick(a)usys.ethz.ch <mailto:jochen.bick@usys.ethz.ch>
www.ap.ethz.ch
4 years
Re: [galaxy-dev] [External] Copying files to FTP upload directory
by Keith Suderman
I have been creating directories for the users, but putting files in (for example) /home/galaxy/database/ftp/user(a)example.com <mailto:home/galaxy/database/ftp/user@example.com> does not make them available for the user(a)example.com <mailto:user@example.com> user in Galaxy.
Also for the list admins... I never did see my message appear on the list and the message Cicada replied to appears to be from galaxy-dev-bounces(a)lists.galaxyproject.org <mailto:galaxy-dev-bounces@lists.galaxyproject.org>. Any ideas why my message(s) appear to be bouncing. I am pretty sure I am sending from the correct address.
Cheers,
Keith
> On Jan 25, 2019, at 4:36 PM, Dennis, H. E. Cicada Brokaw <hbrokaw(a)iu.edu> wrote:
>
> I think you have to create the user's directories under /home/galaxy/database/ftp. I don't think Galaxy will create them if they are not there.
> Cicada
>
> ________________________________________
> From: galaxy-dev <galaxy-dev-bounces(a)lists.galaxyproject.org> on behalf of Keith Suderman <suderman(a)cs.vassar.edu>
> Sent: Friday, January 25, 2019 4:01 PM
> To: Galaxy Dev List
> Subject: [External] [galaxy-dev] Copying files to FTP upload directory
>
> Hello,
>
> I am trying to (ab)use the FTP file upload functionality to get data from an external service into Galaxy. From my reading of the docs I simply need to configure the ftp_upload_dir and ftp_upload_site in my galaxy.ini (haven't migrated to yaml yet) and Galaxy will expect to find subdirectories named after the user's account (email) in the ftp_upload_dir.
>
> I have configured the ftp_upload_dir (/home/galaxy/database/ftp in our case) and the "Choose FTP File" button is now available in the Upload dialog, however regardless of where I put files, or what I name them, Galaxy always reports that "Your FTP directory does not contain any files." For example, if I have the user account user(a)example.org I would expect to be able to place files in /home/galaxy/database/ftp/user(a)example.org and they will be available in the Upload dialog for the user(a)example.org user. However, this is not working; am I missing something?
>
> Thanks,
> Keith
>
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client. To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
> http://galaxyproject.org/search/
4 years
