[galaxy-dev] Nginx+Shibboleth+UCSC

2 Nov 2018

      Dear list,

I have been trying to manage an access to UCSC genome browser from our Galaxy
instance which uses Nginx as a proxy with external authentication via
Shibboleth and IdP service.

Not even the configuration of nginx+shibboleth was quite complicated to
achieve (but I succeeded after a week or so) but now during a testing phase, we
have found out that a cooperation with external UCSC servers is forbidden due
to our auth. configuration.

I have found a way how to do that but only for Apache (https://docs.galaxyproject.org/en/master/admin/special_topics/apache.html#di...) so I tried to do something similar in Nginx but only with IP addresses as Nginx doesn't support using of named domains and defined a location:

location /display_as {
  satisfy any;
  deny all;
  allow 128.114.119.131;
  allow 128.114.119.132;
  allow 128.114.119.133;
  allow 128.114.119.134;
  allow 128.114.119.135;
  allow 128.114.119.136;
}

Used IP addresses should be the right ones for UCSC servers according to few
sources such as (https://genome.soe.ucsc.narkive.com/sll2JSk2/ucsc-ip-address):

hgw1.cse.ucsc.edu
hgw2.cse.ucsc.edu
hgw3.cse.ucsc.edu
hgw4.cse.ucsc.edu
hgw5.cse.ucsc.edu
hgw6.cse.ucsc.edu

I tried to change the location also to "/display_application" but didn't help
(actually, I don't know why in the documentation there is used as the location
that "/display_as" as I have never encountered such location in Galaxy which is
my first question). In galaxy.ini I set:

display_servers = hgw1.cse.ucsc.edu,hgw2.cse.ucsc.edu,hgw3.cse.ucsc.edu,hgw4.cse.ucsc.edu,hgw5.cse.ucsc.edu,hgw6.cse.ucsc.edu,hgw7.cse.ucsc.edu,hgw8.cse.ucsc.edu,lowepub.cse.ucsc.edu,128.114.119.131,128.114.119.132,128.114.119.133,128.114.119.134,128.114.119.135,128.114.119.136

Just for case I used both, the domains and also IP addresses. Also I have
tried to use only the domains or only the addresses but nothing helped.

Then I found out after reading of the following (https://www.switch.ch/aai/guides/sp/access-rules/) that the main obstacle should be actually shibboleth so I defined in /etc/shibboleth/shibboleth2.xml an unrestricted path:

<RequestMapper type="XML">
  <RequestMap>
    <Host name="our.galaxy.something"
          authType="shibboleth"
          requireSession="true"
          redirectToSSL="443" >
      <Path name="display_application" requireSession="false" redirectToSSL="443" />
      <!--Path name="display_as" authType="shibboleth" requireSession="false"
redirectToSSL="443" /-->
    </Host>
  </RequestMap>
</RequestMapper>

Again, I have tried to use both locations "display_as" and "display_application", but it
haven't solved my problem entirely. I moved forward I suppose because now the
requests
should be going around the authentication but UCSC stil warns me that:

Expected 200 https://our.galaxy.something/display_application/e1304269a2f56a52/ucsc_bigwi...: 403 Forbidden

Before that it was a different message containing the address of our idp
server so I believe I'am on the right way here. The presence of attribute
'authType="shibboleth"' seams to be useless as 'requireSession="false"' is
used.

Then I tried to teach Nginx to use also the domain names with the using of 3rd
party nginx-http-rdns module from (https://github.com/flant/nginx-http-rdns),
so my nginx.conf looked like:

....only the part of nginx.conf....
############## Shibboleth authentication conf #################

# FastCGI authorizer for Shibboleth Auth Request module
location = /shibauthorizer {
internal;
include fastcgi_params;
fastcgi_pass unix:/var/run/shibboleth/shibauthorizer.sock;
}

# FastCGI responder for SSO
location /Shibboleth.sso {
include fastcgi_params;
fastcgi_pass unix:/var/run/shibboleth/shibresponder.sock;
}

############### Entry point for Galaxy ###############
# Location secured by Shibboleth
location / {
shib_request /shibauthorizer;
more_clear_input_headers 'Variable-*' 'Shib-*' 'Remote-User' 'REMOTE_USER'
'Auth-Type' 'AUTH_TYPE' 'GX_SECRET';
include uwsgi_params;
shib_request_set $shib_id $upstream_http_variable_eppn;
uwsgi_param HTTP_REMOTE_USER $shib_id;
uwsgi_param HTTP_GX_SECRET $our_secret;
uwsgi_pass 127.0.0.1:4001;

# resolver $correct_resolver_acc_to_resolv.conf;
# rdns on;
# satisfy any;
# rdns_allow genome\.ucsc\.edu;
# rdns_allow hgw1\.cse\.ucsc\.edu;
# rdns_allow hgw2\.cse\.ucsc\.edu;
# rdns_allow hgw3\.cse\.ucsc\.edu;
# rdns_allow hgw4\.cse\.ucsc\.edu;
# rdns_allow hgw5\.cse\.ucsc\.edu;
# rdns_allow hgw6\.cse\.ucsc\.edu;
# rdns_allow hgw7\.cse\.ucsc\.edu;
# rdns_allow hgw8\.cse\.ucsc\.edu;
}

# location /display_as {
# resolver $correct_resolver_acc_to_resolv.conf;
# rdns on;
# satisfy any;
# rdns_allow genome\.ucsc\.edu;
# rdns_allow hgw1\.cse\.ucsc\.edu;
# rdns_allow hgw2\.cse\.ucsc\.edu;
# rdns_allow hgw3\.cse\.ucsc\.edu;
# rdns_allow hgw4\.cse\.ucsc\.edu;
# rdns_allow hgw5\.cse\.ucsc\.edu;
# rdns_allow hgw6\.cse\.ucsc\.edu;
# rdns_allow hgw7\.cse\.ucsc\.edu;
# rdns_allow hgw8\.cse\.ucsc\.edu;
# }

# location /display_application {
# resolver $correct_resolver_acc_to_resolv.conf;
# rdns on;
# satisfy any;
# rdns_allow genome\.ucsc\.edu;
# rdns_allow hgw1\.cse\.ucsc\.edu;
# rdns_allow hgw2\.cse\.ucsc\.edu;
# rdns_allow hgw3\.cse\.ucsc\.edu;
# rdns_allow hgw4\.cse\.ucsc\.edu;
# rdns_allow hgw5\.cse\.ucsc\.edu;
# rdns_allow hgw6\.cse\.ucsc\.edu;
# rdns_allow hgw7\.cse\.ucsc\.edu;
# rdns_allow hgw8\.cse\.ucsc\.edu;
# }
....END of the part of nginx.conf....

Everything commented out, I tried to use together or one at the time but
nothing helped. But there are differences:

1) when uncommented only location /display_application - I will get just a
blank page with: 404 Not Found.

2) when uncommented only location /display_as - nothing changes.

3) when uncommented part inside the location "/" - I will get a new message:
Error unexpected end of input reading http header on https://our.galaxy.something/display_application/e1304269a2f56a52/ucsc_bigwi...

And now I'm out of ideas. I would really appreciate any help.

PS: I stil don't get why in the documentation for Apache configuration
(https://docs.galaxyproject.org/en/master/admin/special_topics/apache.html#di...)
there is used the /display_as location. Can somebody explain, please?

Thank you in advance,
Martin Demko