Re: [galaxy-dev] galaxy config from 17.05 to 18.05

4 Sep 2018

      Hey gents,

I would really appreciate some help with this.
If you can find some kind of error in my configuration.

I believe I have found all the configuration to run galaxy on port 7112 and have an apache proxy on 8081.
Also I would like two handlers one for local and one for drama.

I however still getting the following error in the apache logs:
[Tue Sep 04 12:22:57 2018] [debug] mod_proxy_http.c(56): proxy: HTTP: canonicalising URL //localhost:7112/
[Tue Sep 04 12:22:57 2018] [debug] proxy_util.c(1544): [client 10.33.158.247] proxy: *: found reverse proxy worker for http://localhost:7112/
[Tue Sep 04 12:22:57 2018] [debug] mod_proxy.c(1026): Running scheme http handler (attempt 0)
[Tue Sep 04 12:22:57 2018] [debug] mod_proxy_http.c(2018): proxy: HTTP: serving URL http://localhost:7112/
[Tue Sep 04 12:22:57 2018] [debug] proxy_util.c(2102): proxy: HTTP: has acquired connection for (*)
[Tue Sep 04 12:22:57 2018] [debug] proxy_util.c(2158): proxy: connecting http://localhost:7112/ to localhost:7112
[Tue Sep 04 12:22:57 2018] [debug] proxy_util.c(2289): proxy: connected / to localhost:7112
[Tue Sep 04 12:22:57 2018] [debug] proxy_util.c(2540): proxy: HTTP: fam 2 socket created to connect to *
[Tue Sep 04 12:22:57 2018] [debug] proxy_util.c(2672): proxy: HTTP: connection complete to 127.0.0.1:7112 (localhost)
[Tue Sep 04 12:23:01 2018] [error] [client 10.33.158.247] (20014)Internal error: proxy: error reading status line from remote server localhost
[Tue Sep 04 12:23:01 2018] [debug] mod_proxy_http.c(1499): [client 10.33.158.247] proxy: NOT Closing connection to client although reading from backend server localhost failed.
[Tue Sep 04 12:23:01 2018] [error] [client 10.33.158.247] proxy: Error reading from remote server returned by /galaxy-prod/
[Tue Sep 04 12:23:01 2018] [debug] proxy_util.c(2120): proxy: HTTP: has released connection for (*)
[Tue Sep 04 12:23:01 2018] [debug] mod_deflate.c(687): [client 10.33.158.247] Zlib: Compressed 497 to 333 : URL /galaxy-prod/

Hers is some of my config:

diff galaxy.yml.sample galaxy.yml
19a20
...
socket: 127.0.0.1:7112
24c25
<   http: 127.0.0.1:8080

...
#http: 127.0.0.1:8080
30c31
<   buffer-size: 16384

...
buffer-size: 32768
35c36
<   processes: 1

...
processes: 2
53c54
<   master: false

...
master: true
68c69
<   thunder-lock: false

...
thunder-lock: true
87a89,92
  mule: lib/galaxy/main.py
  mule: lib/galaxy/main.py
  farm: job-handlers:1,2
88a94
...
server_name: main
93c99
<   #filter-with: proxy-prefix

...
filter-with: proxy-prefix
100c106
<   #cookie_path: null

...
cookie_path: /galaxy-prod
107a114
  database_connection: mysql://galaxy:qaz1wsx2@localhost/galaxy_db_prod?unix_socket=/var/lib/mysql/mysql.sock
122a130
  database_engine_option_pool_recycle: 7200
168a177
  file_path: /home/galaxy/galaxy-prod/database/files
171a181
  new_file_path: /home/galaxy/galaxy-prod/database/tmp
177a188
  tool_config_file: '/home/galaxy/galaxy-prod/config/tool_conf.xml,/home/galaxy/galaxy-prod/config/shed_tool_conf.xml'
192a204
  migrated_tools_config: /home/galaxy/galaxy-prod/config/migrated_tools_conf.xml
199a212
  integrated_tool_panel_config: /home/galaxy/galaxy-prod/config/integrated_tool_panel.xml
214a228
  tool_dependency_dir: /home/galaxy/galaxy-prod/database/dependencies
281a296
  tool_sheds_config_file: /home/galaxy/galaxy-prod/config/tool_sheds_conf.xml
342a358
  tool_data_table_config_path: /home/galaxy/galaxy-prod/config/tool_data_table_conf.xml
351a368
  shed_tool_data_table_config: /home/galaxy/galaxy-prod/config/shed_tool_data_table_conf.xml
441a459
  job_working_directory: /home/galaxy/galaxy-prod/database/jobs_directory
445a464
  cluster_files_directory: /home/galaxy/galaxy-prod/database/pbs
508a528
  smtp_server: smtp.umontreal.ca:25
522a543
  smtp_ssl: false
535a557
  error_email_to: edgar.fernandez@umontreal.ca
542a565
  email_from: Galaxy ESI Project <esi-galaxy-no-reply@umontreal.ca>
564a588
  user_activation_on: true
775a800
  apache_xsendfile: true
786a812
  upstream_gzip: false
989a1016
  use_interactive: false
1204a1232
  id_secret: 3s1G@l@xyPr0j3ct
1256a1285
  admin_users: rootbac@esi.umontreal.ca
1259a1289
  require_login: true
1263a1294
  show_welcome_with_login: true
1267a1299
  allow_user_creation: false
1270a1303
  allow_user_deletion: true
1273a1307
  allow_user_impersonation: true
1282a1317
  allow_user_dataset_purge: true
1288a1324
  new_user_dataset_access_role_default_private: true
1493a1530
  enable_quotas: true
1499a1537
  expose_dataset_path: true
1532a1571
  job_config_file: /home/galaxy/galaxy-prod/config/job_conf.xml
1622a1662
  cleanup_job: onsuccess
Also,
cat job_conf.xml
<?xml version="1.0"?>
<job_conf>
  <plugins>
    <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner"/>
    <plugin id="drmaa" type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner"/>
  </plugins>
  <handlers default="main.job-handlers.1">
    <handler id="main.web.1" tags="handlers"><plugin id="local"/></handler>
    <handler id="main.job-handlers.1" tags="handlers"><plugin id="drmaa"/></handler>
  </handlers>
  <destinations default="torque">
    <destination id="local" runner="local"></destination>
    <destination id="torque" runner="drmaa"><param id="native_specification">-l nodes=1:ppn=8</param></destination>
  </destinations>
  <tools>
    <tool id="upload1" handler="main" destination="local"/>
  </tools>
</job_conf>

Finally,
cat /etc/httpd/vhosts.d/galaxy_prod.conf
User galaxy
Group users

Listen 8081

NameVirtualHost *:8081

<VirtualHost *:8081>
  # don't decode encoded slashes in path info
  AllowEncodedSlashes NoDecode

  # enable compression on all relevant types
  AddOutputFilterByType DEFLATE text/html text/plain text/xml
  AddOutputFilterByType DEFLATE text/css
  AddOutputFilterByType DEFLATE application/x-javascript application/javascript application/ecmascript
  AddOutputFilterByType DEFLATE application/rss+xml
  AddOutputFilterByType DEFLATE application/xml
  AddOutputFilterByType DEFLATE application/json

  # allow access to static content
  <Directory "/home/galaxy/galaxy-prod/static">
    AllowOverride None
    Require all granted
  </Directory>

  # allow up to 3 minutes for Galaxy to respond to slow requests before timing out
  ProxyTimeout 180

  # uWSGI on a TCP socket
  ProxyPass /galaxy-prod/ uwsgi://127.0.0.1:7112/

  # serve framework static content
  RewriteEngine On
  RewriteRule ^/galaxy-prod$ /home/galaxy/galaxy-prod/ [R]
  RewriteRule ^/galaxy-prod/static/style/(.*) /home/galaxy/galaxy-prod/static/style/blue/$1 [L]
  RewriteRule ^/galaxy-prod/static/(.*) /home/galaxy/galaxy-prod/static/$1 [L]
  RewriteRule ^/galaxy-prod/favicon.ico /home/galaxy/galaxy-prod/static/favicon.ico [L]
  RewriteRule ^/galaxy-prod/robots.txt /home/galaxy/galaxy-prod/static/robots.txt [L]
  RewriteRule ^/galaxy-prod(.*) http://localhost:7112$1 [P]

  # enable caching on static content
  <Location "/static">
    ExpiresActive On
    ExpiresDefault "access plus 24 hours"
  </Location>

  # serve visualization and interactive environment plugin static content
  <Directory "/home/galaxy/galaxy-prod/config/plugins/(.+)/(.+)/static">
    AllowOverride None
    Require all granted
  </Directory>
  RewriteRule ^/galaxy-prod/plugins/(.+)/(.+)/static/(.*)$ /home/galaxy/galaxy-prod/config/plugins/$1/$2/static/$3 [L]

  <Location "/">
    XSendFile on
    XSendFilePath /
  </Location>

  # Log levels
  LogLevel debug
  ErrorLog /var/log/httpd/galaxy-prod-error_log
  CustomLog /var/log/httpd/galaxy-prod-access_log common
</VirtualHost>

Please let me know what you guys think...

Thank you,

Edgar Fernandez
Administrateur Système (Linux)
Technologies de l'Information
Université de Montréal

PAVILLON ROGER-GAUDRY, bureau X-210
*  Bur. : 1-514-343-6111 poste 16568

From: Fernandez Edgar
Sent: August 31, 2018 10:52 AM
To: 'galaxy-dev@bx.psu.edu' <galaxy-dev@bx.psu.edu>
Subject: galaxy config from 17.05 to 18.05

Hello gents,

I would greatly appreciate some help configuring my new installation of galaxy 18.05.

I found most of the configuration I needed but I'm missing the following configuration in green.
Here is my galaxy 17.05:
[galaxy@esilbac3a ~]$ diff galaxy-17.05/config/galaxy.ini  galaxy-17.05/config/galaxy.ini.sample
32c32
< port = 7112
---
...
#port = 8080
37c37
< host = 0.0.0.0

...
#host = 127.0.0.1
50,59d49
< # ---- HTTP PBS DRMAA HANDLER -----------------------------------------------
<
< # Configuration of the pbs drmaa handler
<
< [server:handler0]
< use = egg:Paste#http
< port = 8088
< use_threadpool = True
< threadpool_workers = 10
<
74c64
< prefix = /galaxy-prod

...
prefix = /galaxy
95c85
< filter-with = proxy-prefix

...
#filter-with = proxy-prefix
109c99
< database_connection = mysql://galaxy:qaz1wsx2@localhost/galaxy_db_prod2?unix_socket=/var/lib/mysql/mysql.sock

...
#database_connection = sqlite:///./database/universe.sqlite?isolation_level=IMMEDIATE
119c109
< database_engine_option_pool_recycle = 7200

...
#database_engine_option_pool_recycle = -1
152c142
< file_path = /home/galaxy/galaxy-prod/database/files

...
#file_path = database/files
155c145
< new_file_path = /home/galaxy/galaxy-prod/database/tmp

...
#new_file_path = database/tmp
161c151
< tool_config_file = /home/galaxy/galaxy-prod/config/tool_conf.xml,/home/galaxy/galaxy-prod/config/shed_tool_conf.xml

...
#tool_config_file = config/tool_conf.xml,config/shed_tool_conf.xml
197c187
< tool_dependency_dir = /home/galaxy/galaxy-prod/tool-data/toolshed.dependency.dir

...
#tool_dependency_dir = database/dependencies
401c391
< job_working_directory = /home/galaxy/galaxy-prod/database/jobs_directory

...
#job_working_directory = database/jobs_directory
466c456
< smtp_server = smtp.umontreal.ca:25

...
#smtp_server = None
475c465
< smtp_ssl = False

...
#smtp_ssl = False
486c476
< error_email_to = edgar.fernandez@umontreal.ca<mailto:edgar.fernandez@umontreal.ca>

...
#error_email_to = None
493c483
< email_from = Galaxy ESI Project <rootbac@esi.umontreal.ca<mailto:rootbac@esi.umontreal.ca>>

...
#email_from = None
679c669
< apache_xsendfile = True

...
#apache_xsendfile = False
693c683
< upstream_gzip = False

...
#upstream_gzip = False
848c838
< debug = False

...
#debug = False
861c851
< use_interactive = False

...
use_interactive = True
989c979
< id_secret = 3s1G@l@xyPr0j3ct

...
#id_secret = USING THE DEFAULT IS NOT SECURE!
1038c1028
< admin_users = rootbac@esi.umontreal.ca<mailto:rootbac@esi.umontreal.ca>

...
#admin_users = None
1041c1031
< require_login = True

...
#require_login = False
1049c1039
< allow_user_creation = False

...
#allow_user_creation = True
1052c1042
< allow_user_deletion = True

...
#allow_user_deletion = False
1055c1045
< allow_user_impersonation = True

...
#allow_user_impersonation = False
1060c1050
< allow_user_dataset_purge = True

...
#allow_user_dataset_purge = True
1066c1056
< new_user_dataset_access_role_default_private = True

...
#new_user_dataset_access_role_default_private = False
1200c1190
< enable_quotas = True

...
#enable_quotas = False
1234c1224
< job_config_file = /home/galaxy/galaxy-prod/config/job_conf.xml

...
#job_config_file = config/job_conf.xml
1292c1282
< retry_job_output_collection = 20

...
#retry_job_output_collection = 0
1310c1300
< cleanup_job = onsuccess

...
#cleanup_job = always
Please help me find what I am missing.

Best regards,

Edgar Fernandez
Administrateur Système (Linux)
Technologies de l'Information
Université de Montréal

PAVILLON ROGER-GAUDRY, bureau X-210
*  Bur. : 1-514-343-6111 poste 16568

Fernandez Edgar

tags

participants (1)