Galaxy Administrator Training, Jan 28 - Feb 1, Penn State University
by Dave Clements
Hello all,
We are pleased to announce that* the 2019 Galaxy Admin Training
<https://galaxyproject.org/events/2019-admin-training/> will be offered
January 28 through February 1 at Penn State University* (where the Galaxy
Project got started over ten years ago). The workshop offers a 2 day
introductory session
<https://github.com/galaxyproject/dagobah-training#basic-sessions> followed
by a 3 day advanced topics session
<https://github.com/galaxyproject/dagobah-training#advanced-sessions>.
Participants can register for one or both sessions.
This workshop will cover what you need to know to set up your own
high-performance and multi-user production Galaxy instance. Sessions will
be *intensive and hands-on*, and taught by experienced instructors from the
Galaxy Community. Participants will learn how to install, configure,
customize, and extend their own Galaxy servers. Topics include tool
configuration, authentication and user management, using heterogeneous
storage and compute services, and many other topics that will enable you to
get your own Galaxy server up and running, performing well, and used by
your community.
Registration
<https://galaxyproject.org/events/2019-admin-training/#registration> is now
open and starts at $40 / day for participants from non-profits and
academia. Early registration ends October 31. However, the 2016 and 2018
admin training were both full, so you are strongly encouraged to *register
now*.
*The workshop does have prerequisites
<https://galaxyproject.org/events/2019-admin-training/#prerequisites>.
Please read them before you register.*
Cheers,
Dave C.
*About Galaxy *
Galaxy is an open web based platform for biomedical data integration and
analysis. It is deployed at large and small organizations around the world
and used in a broad range of biomedical research domains, and across the
tree of life.
--
https://galaxyproject.org/
https://usegalaxy.org/
2 years, 7 months
Using Singularity images to resolve dependences in Galaxy 18.05
by Peter van Heusden
Hi there
Is it possible to use Singularity images to resolve dependencies in Galaxy
18.05? If so is there documentation somewhere?
I have a Galaxy server where for at least some of the dependencies conda
will not be a good option so I would like to be able to use singularity if
at all possible.
Thanks!
Peter
2 years, 10 months
Issue with deseq2
by Previti
Dear all,
I have been trying to get deseq2 (latest version 17, 2018-09-05 from
iuc; we are still running on galaxy version 17.09) to work properly but
to no avail. Older versions have other errors, I have focused on the
latest version.
I get this cryptic error when I run it:
/opt/galaxy/galaxy/database/dependencies/_conda/envs/mulled-v1-dbc1e05d0db06d983572e7eca0df68af225d8f398b6e11dd4d586075518c4101/lib/R/bin/exec/R: symbol lookup error:
/opt/galaxy/galaxy/database/dependencies/_conda/envs/mulled-v1-dbc1e05d0db06d983572e7eca0df68af225d8f398b6e11dd4d586075518c4101/lib/R/bin/exec/../../lib/../../libreadline.so.6: undefined symbol: PC
/opt/galaxy/galaxy/database/dependencies/_conda/envs/mulled-v1-dbc1e05d0db06d983572e7eca0df68af225d8f398b6e11dd4d586075518c4101/lib/R/bin/exec/R: symbol lookup error:
/opt/galaxy/galaxy/database/dependencies/_conda/envs/mulled-v1-dbc1e05d0db06d983572e7eca0df68af225d8f398b6e11dd4d586075518c4101/lib/R/bin/exec/../../lib/../../libreadline.so.6: undefined symbol: PC
Does anybody know what to do about this?
Thanks and best regards,
Christopher
--
*Dr. Christopher Previti*
Genomics and Proteomics Core Facility
High Throughput Sequencing (W190)
Bioinformatician
German Cancer Research Center (DKFZ)
Foundation under Public Law
Im Neuenheimer Feld 580
69120 Heidelberg
Germany
Room: B2.102 (INF580/TP3)
Phone: +49 6221 42-4661
christopher.previti(a)dkfz.de <http://www.dkfz.de/>
www.dkfz.de <http://www.dkfz.de/>
Management Board: Prof. Dr. Michael Baumann, Prof. Dr. Josef Puchta
VAT-ID No.: DE143293537
Vertraulichkeitshinweis: Diese Nachricht ist ausschließlich für die
Personen bestimmt, an die sie adressiert ist.
Sie kann vertrauliche und/oder nur für den/die Empfänger bestimmte
Informationen enthalten. Sollten Sie nicht
der bestimmungsgemäße Empfänger sein, kontaktieren Sie bitte den
Absender und löschen Sie die Mitteilung.
Jegliche unbefugte Verwendung der Informationen in dieser Nachricht ist
untersagt.
2 years, 10 months
"Merging tabular data tables with non-matching columns is not allowed"
by Peter van Heusden
Hi there
I'm working on a data manager to keep track of the reference data
collection used by the SHAPEIT2 tool. I had to add a column to the data
manager but when I try and do a `planemo shed_update` I get the error:
Unexpected HTTP status code: 500: {"err_msg": "Error attempting to parse
file tool_data_table_conf.xml.sample: Merging tabular data tables with non
matching columns is not allowed: shapeit_ref:{'path': 2,
'reference_prefix': 3, 'name': 1, 'value': 0, 'map_prefix': 4} !=
shapeit_ref:{'name': 1, 'value': 0, 'sample_prefix': 5, 'path': 2,
'reference_prefix': 3, 'map_prefix': 4}"}
This is for data_manager_shapeit_reference in testtoolshed, which is on
github at
https://github.com/pvanheus/data_manager_shapeit_reference
As far as I can tell the columns do accord. Is there any way to see which
files are being merged to help try and debug this error?
Thanks!
Peter
2 years, 10 months
Update from 17.10 to 18.05
by Jochen Bick
Hi,
today we tried to update from 17.10 to 18.05
your server is running but with out any interface, its just white after
loading?
<title>
Galaxy
| AP - Group
</title>
<link rel="index" href="/"/>
<link href="/static/style/jquery.rating.css?v=1537264325"
media="screen" rel="stylesheet" type="text/css" />
<link
href="/static/style/jquery-ui/smoothness/jquery-ui.css?v=1537264325"
media="screen" rel="stylesheet" type="text/css" />
<link href="/static/style/base.css?v=1537264325" media="screen"
rel="stylesheet" type="text/css" />
<link href="/static/style/bootstrap-tour.css?v=1537264325"
media="screen" rel="stylesheet" type="text/css" />
# tail ./galaxy-dist/dsmerror.log
09/05/2018 11:18:21 ANS0361I DIAG: TransErrno: Unexpected error from
GetFSInfo:stat, errno = 6
09/05/2018 11:18:22 ANS0361I DIAG: TransErrno: Unexpected error from
GetFSInfo:stat, errno = 6
09/05/2018 11:57:19 ANS0361I DIAG: TransErrno: Unexpected error from
GetFSInfo:stat, errno = 6
09/05/2018 11:57:39 ANS0361I DIAG: TransErrno: Unexpected error from
GetFSInfo:stat, errno = 6
09/05/2018 11:57:39 ANS0361I DIAG: TransErrno: Unexpected error from
GetFSInfo:stat, errno = 6
09/05/2018 11:57:40 ANS0361I DIAG: TransErrno: Unexpected error from
lstat, errno = 6
09/05/2018 11:57:40 ANS0361I DIAG: TransErrno: Unexpected error from
GetFSInfo:stat, errno = 6
09/05/2018 11:57:40 ANS1999E Incremental processing of '/' stopped.
Cheers Jochen
2 years, 10 months
good practice for Galaxy admin and updates ?
by julie dubois
Hi Galaxy Community,
I manage a Galaxy local instance on a standalone station since 6 years.
Regarding the numerous troubleshoots I have encountered (and the very
long time I spent) during these years with Galaxy updates trying to
minimize impact for my users, trying in the same time to keep safe
our customization of this instance. I'm wondering about the best solution
to manage my instance and to test my own dev and also Galaxy updates
before pushing them on
my production server.
So I want to test a new way and I'd appreciate community feedback on this idea
or let me know if other practices could be more appropriate in my situation.
So my basic idea is:
First install a new Galaxy from scratch, not by cloning the stable branch of
Galaxy but by forking it. And use the master branch of this fork as
production server.
Then to manage an update of the Galaxy stable, my idea is to make a
test branch on my
fork, then pull Galaxy update on this test branch and run this test instance as
a dev server in another station to be sure that update will not override our
customization or any crucial tools/functionality for my users.
Finally when I'll be sure this update is safe, I'll merge the test branch with
the master branch of my forked Galaxy.
Is anyone already acting in this way? Is it a good (the best) way to manage
local instance of Galaxy? Have you other good practices?
Thanks a lot for your help.
Julie
2 years, 11 months
Salmon references and data manager
by Previti
Dear Björn,
I just installed Salmon on our Galaxy instance and I have a couple of
basic questions.
Currently the reference transcriptomes are put in the same data table as
the genomes, would it be of interest to separate this and give the
transcriptomes their own table? I could probably try to do this...
There is a data manager available that unfortunately has a bug. We fixed
that and it now populates the reference genome data table.
I would probably modify this as well use the new table. Could this be
useful? I'm not sure how to proceed...would I give you the modified
Salmon wrapper for inclusion in the package?
Best regards,
Christopher
--
*Dr. Christopher Previti*
Genomics and Proteomics Core Facility
High Throughput Sequencing (W190)
Bioinformatician
German Cancer Research Center (DKFZ)
Foundation under Public Law
Im Neuenheimer Feld 580
69120 Heidelberg
Germany
Room: B2.102 (INF580/TP3)
Phone: +49 6221 42-4661
christopher.previti(a)dkfz.de <http://www.dkfz.de/>
www.dkfz.de <http://www.dkfz.de/>
Management Board: Prof. Dr. Michael Baumann, Prof. Dr. Josef Puchta
VAT-ID No.: DE143293537
Vertraulichkeitshinweis: Diese Nachricht ist ausschließlich für die
Personen bestimmt, an die sie adressiert ist.
Sie kann vertrauliche und/oder nur für den/die Empfänger bestimmte
Informationen enthalten. Sollten Sie nicht
der bestimmungsgemäße Empfänger sein, kontaktieren Sie bitte den
Absender und löschen Sie die Mitteilung.
Jegliche unbefugte Verwendung der Informationen in dieser Nachricht ist
untersagt.
2 years, 11 months
Local galaxy of release_18.05 is too slow
by 沈维燕(Steven Shen)
Hi list,
I use Anaconda2-5.2.0-Linux-x86_64.sh and galaxy release_18.05 for my local galaxy, but I found it was too slow to open the web pages. The release_17.09 on the same server is much faster.
I can see some limit when release_18.05 start:
your processes number limit is 4096 your memory page size is 4096 bytes detected max file descriptor number: 65535 !!! no /etc/mime.types file found !!! lock engine: pthread robust mutexes
Is this related to my problem? or which options could I set ? Thank you.
Steven
2 years, 11 months
Re: [galaxy-dev] galaxy config from 17.05 to 18.05
by Fernandez Edgar
Hey gents,
I would really appreciate some help with this.
If you can find some kind of error in my configuration.
I believe I have found all the configuration to run galaxy on port 7112 and have an apache proxy on 8081.
Also I would like two handlers one for local and one for drama.
I however still getting the following error in the apache logs:
[Tue Sep 04 12:22:57 2018] [debug] mod_proxy_http.c(56): proxy: HTTP: canonicalising URL //localhost:7112/
[Tue Sep 04 12:22:57 2018] [debug] proxy_util.c(1544): [client 10.33.158.247] proxy: *: found reverse proxy worker for http://localhost:7112/
[Tue Sep 04 12:22:57 2018] [debug] mod_proxy.c(1026): Running scheme http handler (attempt 0)
[Tue Sep 04 12:22:57 2018] [debug] mod_proxy_http.c(2018): proxy: HTTP: serving URL http://localhost:7112/
[Tue Sep 04 12:22:57 2018] [debug] proxy_util.c(2102): proxy: HTTP: has acquired connection for (*)
[Tue Sep 04 12:22:57 2018] [debug] proxy_util.c(2158): proxy: connecting http://localhost:7112/ to localhost:7112
[Tue Sep 04 12:22:57 2018] [debug] proxy_util.c(2289): proxy: connected / to localhost:7112
[Tue Sep 04 12:22:57 2018] [debug] proxy_util.c(2540): proxy: HTTP: fam 2 socket created to connect to *
[Tue Sep 04 12:22:57 2018] [debug] proxy_util.c(2672): proxy: HTTP: connection complete to 127.0.0.1:7112 (localhost)
[Tue Sep 04 12:23:01 2018] [error] [client 10.33.158.247] (20014)Internal error: proxy: error reading status line from remote server localhost
[Tue Sep 04 12:23:01 2018] [debug] mod_proxy_http.c(1499): [client 10.33.158.247] proxy: NOT Closing connection to client although reading from backend server localhost failed.
[Tue Sep 04 12:23:01 2018] [error] [client 10.33.158.247] proxy: Error reading from remote server returned by /galaxy-prod/
[Tue Sep 04 12:23:01 2018] [debug] proxy_util.c(2120): proxy: HTTP: has released connection for (*)
[Tue Sep 04 12:23:01 2018] [debug] mod_deflate.c(687): [client 10.33.158.247] Zlib: Compressed 497 to 333 : URL /galaxy-prod/
Hers is some of my config:
diff galaxy.yml.sample galaxy.yml
19a20
> socket: 127.0.0.1:7112
24c25
< http: 127.0.0.1:8080
---
> #http: 127.0.0.1:8080
30c31
< buffer-size: 16384
---
> buffer-size: 32768
35c36
< processes: 1
---
> processes: 2
53c54
< master: false
---
> master: true
68c69
< thunder-lock: false
---
> thunder-lock: true
87a89,92
> mule: lib/galaxy/main.py
> mule: lib/galaxy/main.py
> farm: job-handlers:1,2
>
88a94
> server_name: main
93c99
< #filter-with: proxy-prefix
---
> filter-with: proxy-prefix
100c106
< #cookie_path: null
---
> cookie_path: /galaxy-prod
107a114
> database_connection: mysql://galaxy:qaz1wsx2@localhost/galaxy_db_prod?unix_socket=/var/lib/mysql/mysql.sock
122a130
> database_engine_option_pool_recycle: 7200
168a177
> file_path: /home/galaxy/galaxy-prod/database/files
171a181
> new_file_path: /home/galaxy/galaxy-prod/database/tmp
177a188
> tool_config_file: '/home/galaxy/galaxy-prod/config/tool_conf.xml,/home/galaxy/galaxy-prod/config/shed_tool_conf.xml'
192a204
> migrated_tools_config: /home/galaxy/galaxy-prod/config/migrated_tools_conf.xml
199a212
> integrated_tool_panel_config: /home/galaxy/galaxy-prod/config/integrated_tool_panel.xml
214a228
> tool_dependency_dir: /home/galaxy/galaxy-prod/database/dependencies
281a296
> tool_sheds_config_file: /home/galaxy/galaxy-prod/config/tool_sheds_conf.xml
342a358
> tool_data_table_config_path: /home/galaxy/galaxy-prod/config/tool_data_table_conf.xml
351a368
> shed_tool_data_table_config: /home/galaxy/galaxy-prod/config/shed_tool_data_table_conf.xml
441a459
> job_working_directory: /home/galaxy/galaxy-prod/database/jobs_directory
445a464
> cluster_files_directory: /home/galaxy/galaxy-prod/database/pbs
508a528
> smtp_server: smtp.umontreal.ca:25
522a543
> smtp_ssl: false
535a557
> error_email_to: edgar.fernandez(a)umontreal.ca
542a565
> email_from: Galaxy ESI Project <esi-galaxy-no-reply(a)umontreal.ca>
564a588
> user_activation_on: true
775a800
> apache_xsendfile: true
786a812
> upstream_gzip: false
989a1016
> use_interactive: false
1204a1232
> id_secret: 3s1G@l@xyPr0j3ct
1256a1285
> admin_users: rootbac(a)esi.umontreal.ca
1259a1289
> require_login: true
1263a1294
> show_welcome_with_login: true
1267a1299
> allow_user_creation: false
1270a1303
> allow_user_deletion: true
1273a1307
> allow_user_impersonation: true
1282a1317
> allow_user_dataset_purge: true
1288a1324
> new_user_dataset_access_role_default_private: true
1493a1530
> enable_quotas: true
1499a1537
> expose_dataset_path: true
1532a1571
> job_config_file: /home/galaxy/galaxy-prod/config/job_conf.xml
1622a1662
> cleanup_job: onsuccess
Also,
cat job_conf.xml
<?xml version="1.0"?>
<job_conf>
<plugins>
<plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner"/>
<plugin id="drmaa" type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner"/>
</plugins>
<handlers default="main.job-handlers.1">
<handler id="main.web.1" tags="handlers"><plugin id="local"/></handler>
<handler id="main.job-handlers.1" tags="handlers"><plugin id="drmaa"/></handler>
</handlers>
<destinations default="torque">
<destination id="local" runner="local"></destination>
<destination id="torque" runner="drmaa"><param id="native_specification">-l nodes=1:ppn=8</param></destination>
</destinations>
<tools>
<tool id="upload1" handler="main" destination="local"/>
</tools>
</job_conf>
Finally,
cat /etc/httpd/vhosts.d/galaxy_prod.conf
User galaxy
Group users
Listen 8081
NameVirtualHost *:8081
<VirtualHost *:8081>
# don't decode encoded slashes in path info
AllowEncodedSlashes NoDecode
# enable compression on all relevant types
AddOutputFilterByType DEFLATE text/html text/plain text/xml
AddOutputFilterByType DEFLATE text/css
AddOutputFilterByType DEFLATE application/x-javascript application/javascript application/ecmascript
AddOutputFilterByType DEFLATE application/rss+xml
AddOutputFilterByType DEFLATE application/xml
AddOutputFilterByType DEFLATE application/json
# allow access to static content
<Directory "/home/galaxy/galaxy-prod/static">
AllowOverride None
Require all granted
</Directory>
# allow up to 3 minutes for Galaxy to respond to slow requests before timing out
ProxyTimeout 180
# uWSGI on a TCP socket
ProxyPass /galaxy-prod/ uwsgi://127.0.0.1:7112/
# serve framework static content
RewriteEngine On
RewriteRule ^/galaxy-prod$ /home/galaxy/galaxy-prod/ [R]
RewriteRule ^/galaxy-prod/static/style/(.*) /home/galaxy/galaxy-prod/static/style/blue/$1 [L]
RewriteRule ^/galaxy-prod/static/(.*) /home/galaxy/galaxy-prod/static/$1 [L]
RewriteRule ^/galaxy-prod/favicon.ico /home/galaxy/galaxy-prod/static/favicon.ico [L]
RewriteRule ^/galaxy-prod/robots.txt /home/galaxy/galaxy-prod/static/robots.txt [L]
RewriteRule ^/galaxy-prod(.*) http://localhost:7112$1 [P]
# enable caching on static content
<Location "/static">
ExpiresActive On
ExpiresDefault "access plus 24 hours"
</Location>
# serve visualization and interactive environment plugin static content
<Directory "/home/galaxy/galaxy-prod/config/plugins/(.+)/(.+)/static">
AllowOverride None
Require all granted
</Directory>
RewriteRule ^/galaxy-prod/plugins/(.+)/(.+)/static/(.*)$ /home/galaxy/galaxy-prod/config/plugins/$1/$2/static/$3 [L]
<Location "/">
XSendFile on
XSendFilePath /
</Location>
# Log levels
LogLevel debug
ErrorLog /var/log/httpd/galaxy-prod-error_log
CustomLog /var/log/httpd/galaxy-prod-access_log common
</VirtualHost>
Please let me know what you guys think...
Thank you,
Edgar Fernandez
Administrateur Système (Linux)
Technologies de l'Information
Université de Montréal
PAVILLON ROGER-GAUDRY, bureau X-210
* Bur. : 1-514-343-6111 poste 16568
From: Fernandez Edgar
Sent: August 31, 2018 10:52 AM
To: 'galaxy-dev(a)bx.psu.edu' <galaxy-dev(a)bx.psu.edu>
Subject: galaxy config from 17.05 to 18.05
Hello gents,
I would greatly appreciate some help configuring my new installation of galaxy 18.05.
I found most of the configuration I needed but I'm missing the following configuration in green.
Here is my galaxy 17.05:
[galaxy@esilbac3a ~]$ diff galaxy-17.05/config/galaxy.ini galaxy-17.05/config/galaxy.ini.sample
32c32
< port = 7112
---
> #port = 8080
37c37
< host = 0.0.0.0
---
> #host = 127.0.0.1
50,59d49
< # ---- HTTP PBS DRMAA HANDLER -----------------------------------------------
<
< # Configuration of the pbs drmaa handler
<
< [server:handler0]
< use = egg:Paste#http
< port = 8088
< use_threadpool = True
< threadpool_workers = 10
<
74c64
< prefix = /galaxy-prod
---
> prefix = /galaxy
95c85
< filter-with = proxy-prefix
---
> #filter-with = proxy-prefix
109c99
< database_connection = mysql://galaxy:qaz1wsx2@localhost/galaxy_db_prod2?unix_socket=/var/lib/mysql/mysql.sock
---
> #database_connection = sqlite:///./database/universe.sqlite?isolation_level=IMMEDIATE
119c109
< database_engine_option_pool_recycle = 7200
---
> #database_engine_option_pool_recycle = -1
152c142
< file_path = /home/galaxy/galaxy-prod/database/files
---
> #file_path = database/files
155c145
< new_file_path = /home/galaxy/galaxy-prod/database/tmp
---
> #new_file_path = database/tmp
161c151
< tool_config_file = /home/galaxy/galaxy-prod/config/tool_conf.xml,/home/galaxy/galaxy-prod/config/shed_tool_conf.xml
---
> #tool_config_file = config/tool_conf.xml,config/shed_tool_conf.xml
197c187
< tool_dependency_dir = /home/galaxy/galaxy-prod/tool-data/toolshed.dependency.dir
---
> #tool_dependency_dir = database/dependencies
401c391
< job_working_directory = /home/galaxy/galaxy-prod/database/jobs_directory
---
> #job_working_directory = database/jobs_directory
466c456
< smtp_server = smtp.umontreal.ca:25
---
> #smtp_server = None
475c465
< smtp_ssl = False
---
> #smtp_ssl = False
486c476
< error_email_to = edgar.fernandez(a)umontreal.ca<mailto:edgar.fernandez@umontreal.ca>
---
> #error_email_to = None
493c483
< email_from = Galaxy ESI Project <rootbac(a)esi.umontreal.ca<mailto:rootbac@esi.umontreal.ca>>
---
> #email_from = None
679c669
< apache_xsendfile = True
---
> #apache_xsendfile = False
693c683
< upstream_gzip = False
---
> #upstream_gzip = False
848c838
< debug = False
---
> #debug = False
861c851
< use_interactive = False
---
> use_interactive = True
989c979
< id_secret = 3s1G@l@xyPr0j3ct
---
> #id_secret = USING THE DEFAULT IS NOT SECURE!
1038c1028
< admin_users = rootbac(a)esi.umontreal.ca<mailto:rootbac@esi.umontreal.ca>
---
> #admin_users = None
1041c1031
< require_login = True
---
> #require_login = False
1049c1039
< allow_user_creation = False
---
> #allow_user_creation = True
1052c1042
< allow_user_deletion = True
---
> #allow_user_deletion = False
1055c1045
< allow_user_impersonation = True
---
> #allow_user_impersonation = False
1060c1050
< allow_user_dataset_purge = True
---
> #allow_user_dataset_purge = True
1066c1056
< new_user_dataset_access_role_default_private = True
---
> #new_user_dataset_access_role_default_private = False
1200c1190
< enable_quotas = True
---
> #enable_quotas = False
1234c1224
< job_config_file = /home/galaxy/galaxy-prod/config/job_conf.xml
---
> #job_config_file = config/job_conf.xml
1292c1282
< retry_job_output_collection = 20
---
> #retry_job_output_collection = 0
1310c1300
< cleanup_job = onsuccess
---
> #cleanup_job = always
Please help me find what I am missing.
Best regards,
Edgar Fernandez
Administrateur Système (Linux)
Technologies de l'Information
Université de Montréal
PAVILLON ROGER-GAUDRY, bureau X-210
* Bur. : 1-514-343-6111 poste 16568
2 years, 11 months
