- galaxy-dev - lists.galaxyproject.org

From Paste to uWSGI
by Jochen Bick 10 Jul '18

10 Jul '18

Hi all, we would like to update to 18.05 and thereby also update our Paste Server. https://docs.galaxyproject.org/en/latest/admin/scaling.html Is there any tutorial on how to update from Paste to uWSGI? Cheers Jochen -- ETH Zurich *Jochen Bick* Animal Physiology Institute of Agricultural Sciences Postal address: Universitätstrasse 2 / LFW B 58.1 8092 Zurich, Switzerland Office: Eschikon 27 8315 Lindau, Switzerland Phone +41 52 354 92 06 jochen.bick(a)usys.ethz.ch <mailto:jochen.bick@usys.ethz.ch> www.ap.ethz.ch

3 3

Re: [galaxy-dev] Conda problems: Failed to activate conda environment
by Matthias Bernt 06 Jul '18

06 Jul '18

Dear Gerhard, I don't know about the python error.. could you tell us the output of `locale` when logged in as the user running the Galaxy server? For the second problem, I remember that I also have a similar problem. For some reasons some symlinks in the conda environment are not created. you can check if in the respective environment's bin directory symlinks to the conda, activate and deactivate binaries/scripts exist. If not you can fix the missing links (I run the following after each tool installation) ``` for env in "{{ galaxy_server_dir }}"/database/dependencies/_conda/envs/*; do for link in conda activate deactivate; do \[ -h $env/bin/$link \] || ln -s "{{ galaxy_server_dir }}"/database/dependencies/_conda/bin/$link $env/bin/$link; done done ``` Hope this helps. Cheers, Matthias On 05.07.2018 18:00, galaxy-dev-request(a)lists.galaxyproject.org wrote: > Message: 1 > Date: Thu, 5 Jul 2018 08:14:35 +0000 > From: "Van Wageningen, GERHARD [gerhardv(a)sun.ac.za]" > <gerhardv(a)sun.ac.za> > To:"galaxy-dev(a)lists.galaxyproject.org" > <galaxy-dev(a)lists.galaxyproject.org> > Subject: [galaxy-dev] Conda problems: Failed to activate conda > environment > Message-ID: > <AM0PR07MB4354EEAD1AFB5809409F9F3692400(a)AM0PR07MB4354.eurprd07.prod.outlook.com> > > Content-Type: text/plain; charset="iso-8859-1" > > Hi All > > I have recently started learning the Galaxy setup starting with an upgrade. > Made some headway but I was wondering if someone could comment on a problem that has emerged. > The system is a Galaxy 18.05 installation (upgraded from 17.01) on an Altair PBS scheduler with drmaa setup (working) > > Conda problem: Failed to activate conda environment. > > Conda works during tool installation or reinstallation. > > Galaxy tools have been updated/reinstalled so they all look "green" in manage tools. > > In Manage Dependencies there are a number of "Dependency resolved but version not found" warnings, otherwise the deps look clean. > > When running tools in Galaxy errors occur, after the jobs have correctly deployed on the cluster. > > > The galaxy application raised errors like the two below. > > "Failed to activate conda environment! Error was: > Fatal Python error: Py_Initialize: Unable to get the locale encoding > File "/home/.galaxy/tooldeps/python/2.7/iuc/package_python_2_7/8b09fe018cac/lib/python2.7/encodings/__init__.py", line 123 > raise C" > > > " > > Failed to activate conda environment! Error was: CondaEnvironmentError: Environment error: Cannot activate environment bash. User does not have write access for conda symlinks. > > > " > > > Galaxy started and stopped using supervisord with config below: -- ------------------------------------------- Matthias Bernt Bioinformatics Service Molekulare Systembiologie (MOLSYB) Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/ Helmholtz Centre for Environmental Research GmbH - UFZ Permoserstraße 15, 04318 Leipzig, Germany Phone +49 341 235 482296, m.bernt(a)ufz.de, www.ufz.de Sitz der Gesellschaft/Registered Office: Leipzig Registergericht/Registration Office: Amtsgericht Leipzig Handelsregister Nr./Trade Register Nr.: B 4703 Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig Wilfried Kraus Wissenschaftlicher Geschäftsführer/Scientific Managing Director: Prof. Dr. Dr. h.c. Georg Teutsch Administrative Geschäftsführerin/ Administrative Managing Director: Prof. Dr. Heike Graßmann -------------------------------------------

2 3

Strange error in galaxy
by Evan Clark 27 Jun '18

27 Jun '18

Has anyone seen this error in galaxy before? I'm not sure exactly what it indicates. Traceback (most recent call last): File "/mnt/ceph/home/galaxy/galaxy/tools/data_source/upload.py", line 425, in<module> __main__() File "/mnt/ceph/home/galaxy/galaxy/tools/data_source/upload.py", line 413, in __main__ add_file( dataset, registry, json_file, output_path ) File "/mnt/ceph/home/galaxy/galaxy/tools/data_source/upload.py", line 325, in add_file shutil.move( dataset.path, output_path ) File "/usr/lib64/python2.7/shutil.py", line 301, in move copy2(src, real_dst) File "/usr/lib64/python2.7/shutil.py", line 130, in copy2 copyfile(src, dst) File "/usr/lib64/python2.7/shutil.py", line 84, in copyfile copyfileobj(fsrc, fdst) File "/usr/lib64/python2.7/shutil.py", line 52, in copyfileobj fdst.write(buf) IOError: [Errno 4] Interrupted system call

1 0

REST API api/tools
by Dmitry Repchevsky 27 Jun '18

27 Jun '18

Hello, It looks like there where changes in the data returned by the API. I used to get "citations" and "requirements" properties from GET /api/tools JSON... How can I get bioconda packeges for the Galaxy tool? Thank you, Dmitry

1 0

jobs not running with galaxy.yml
by Matthias Bernt 27 Jun '18

27 Jun '18

Dear list, I'm trying to switch from galaxy.ini to yml on my dev instance (using a drmaa runner starting jobs as real user). Somehow my jobs are not starting anymore (it seems that they are not given to the job handler). The only output I get is: galaxy.tools.actions INFO 2018-06-27 13:11:37,445 [p:19863,w:1,m:0] [uWSGIWorker1Core2] Handled output named output for tool idba_ud (25965.752 ms) galaxy.tools.actions INFO 2018-06-27 13:11:37,459 [p:19863,w:1,m:0] [uWSGIWorker1Core2] Added output datasets to history (13.657 ms) galaxy.tools.actions INFO 2018-06-27 13:11:37,484 [p:19863,w:1,m:0] [uWSGIWorker1Core2] Verified access to datasets for Job[unflushed,tool_id=idba_ud] (5.423 ms) galaxy.tools.actions INFO 2018-06-27 13:11:37,484 [p:19863,w:1,m:0] [uWSGIWorker1Core2] Setup for job Job[unflushed,tool_id=idba_ud] complete, ready to flush (25.058 ms) galaxy.tools.actions INFO 2018-06-27 13:11:37,522 [p:19863,w:1,m:0] [uWSGIWorker1Core2] Flushed transaction for job Job[id=4366,tool_id=idba_ud] (37.777 ms) galaxy.tools.execute DEBUG 2018-06-27 13:11:37,522 [p:19863,w:1,m:0] [uWSGIWorker1Core2] Tool [idba_ud] created job [4366] (26050.612 ms) galaxy.tools.execute DEBUG 2018-06-27 13:11:37,538 [p:19863,w:1,m:0] [uWSGIWorker1Core2] Executed 1 job(s) for tool idba_ud request: (26075.593 ms) In addition to manual conversion I tried the auto conversion using `make config-convert` with the same result. I also tried to validate the yml file: ``` make config-validate ``` but get only: ``` pykwalify.errors.RuleError: <RuleError: error code 4: Types do not match: 'None' --> '<type 'str'>': Path: '/mapping/galaxy/mapping/error_email_to'> ``` Any ideas where I could start digging? Cheers, Matthias -- ------------------------------------------- Matthias Bernt Bioinformatics Service Molekulare Systembiologie (MOLSYB) Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/ Helmholtz Centre for Environmental Research GmbH - UFZ Permoserstraße 15, 04318 Leipzig, Germany Phone +49 341 235 482296, m.bernt(a)ufz.de, www.ufz.de Sitz der Gesellschaft/Registered Office: Leipzig Registergericht/Registration Office: Amtsgericht Leipzig Handelsregister Nr./Trade Register Nr.: B 4703 Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig Wilfried Kraus Wissenschaftlicher Geschäftsführer/Scientific Managing Director: Prof. Dr. Dr. h.c. Georg Teutsch Administrative Geschäftsführerin/ Administrative Managing Director: Prof. Dr. Heike Graßmann -------------------------------------------

1 0

MySQL errors and warnings with a fresh install of Galaxy 18-05
by SAPET, Frederic 25 Jun '18

25 Jun '18

Hi, I try to install galaxy 18-05. My database backend is MysqL (5.7.11-4 Percona Server) I start from scratch (drop database, new directory with galaxy files fetched with git) So the database is first created when run.sh is used : : Creating database for URI [mysql://.... I see warnings about duplicated indexes and OperationalError from SQLAlchemy. Some tables are then missing in my database (workflow_invocation_output_dataset_collection_association and workflow_invocation_output_dataset_collection_association). And fields are still present (ie in request table : library_id and folder_id) Could you please help me ? Thank you. Fred >grep "^OperationalError:" FirstRun.log | cut -f 2- -d "," "Cannot drop index 'ix_request_library_id': needed in a foreign key constraint") [SQL: u'\nALTER TABLE request DROP COLUMN library_id'] (Background on this error at: http://sqlalche.me/e/e3q8) "Cannot drop index 'ix_request_folder_id': needed in a foreign key constraint") [SQL: u'\nALTER TABLE request DROP COLUMN folder_id'] (Background on this error at: http://sqlalche.me/e/e3q8) "BLOB/TEXT column 'annotation' used in key specification without a key length") [SQL: u'CREATE INDEX ix_page_annotation_association_annotation ON page_annotation_association (annotation)'] (Background on this error at: http://sqlalche.me/e/e3q8) "Cannot drop index 'sequencer_id': needed in a foreign key constraint") [SQL: u'\nALTER TABLE request_type DROP COLUMN sequencer_id'] (Background on this error at: http://sqlalche.me/e/e3q8) "BLOB/TEXT column 'guid' used in key specification without a key length") [SQL: u'CREATE UNIQUE INDEX ix_tool_id_guid_map_guid ON tool_id_guid_map (guid)'] (Background on this error at: http://sqlalche.me/e/e3q8) "BLOB/TEXT column 'layout' used in key specification without a key length") [SQL: u'CREATE INDEX ix_form_definition_layout ON form_definition (layout)'] (Background on this error at: http://sqlalche.me/e/e3q8) "BLOB/TEXT column 'notification' used in key specification without a key length") [SQL: u'CREATE INDEX ix_request_notification ON request (notification)'] (Background on this error at: http://sqlalche.me/e/e3q8) "BLOB/TEXT column 'data_manager_id' used in key specification without a key length") [SQL: u'CREATE INDEX ix_data_manager_job_association_data_manager_id ON data_manager_job_association (data_manager_id)'] (Background on this error at: http://sqlalche.me/e/e3q8) "Identifier name 'workflow_invocation_output_dataset_collection_association_ibfk_1' is too long") [SQL: u'\nCREATE TABLE workflow_invocation_output_dataset_collection_association (\n\tid INTEGER NOT NULL AUTO_INCREMENT, \n\tworkflow_invocation_id INTEGER, \n\tworkflow_step_id INTEGER, \n\tdataset_collection_id INTEGER, \n\tworkflow_output_id INTEGER, \n\tPRIMARY KEY (id), \n\tFOREIGN KEY(workflow_invocation_id) REFERENCES workflow_invocation (id), \n\tFOREIGN KEY(workflow_step_id) REFERENCES workflow_step (id), \n\tFOREIGN KEY(dataset_collection_id) REFERENCES history_dataset_collection_association (id), \n\tFOREIGN KEY(workflow_output_id) REFERENCES workflow_output (id)\n)\n\n'] (Background on this error at: http://sqlalche.me/e/e3q8) "Identifier name 'workflow_invocation_step_output_dataset_collection_association_ibfk_1' is too long") [SQL: u'\nCREATE TABLE workflow_invocation_step_output_dataset_collection_association (\n\tid INTEGER NOT NULL AUTO_INCREMENT, \n\tworkflow_invocation_step_id INTEGER, \n\tworkflow_step_id INTEGER, \n\tdataset_collection_id INTEGER, \n\toutput_name VARCHAR(255), \n\tPRIMARY KEY (id), \n\tFOREIGN KEY(workflow_invocation_step_id) REFERENCES workflow_invocation_step (id), \n\tFOREIGN KEY(workflow_step_id) REFERENCES workflow_step (id), \n\tFOREIGN KEY(dataset_collection_id) REFERENCES history_dataset_collection_association (id)\n)\n\n'] (Background on this error at: http://sqlalche.me/e/e3q8) >grep "Warning" FirstRun.log | cut -f 3,4 -d ":" Warning: Duplicate index 'ix_galaxy_user_deleted' defined on the table 'galaxydbBGAdev.galaxy_user'. This is deprecated and will be disallowed in a future release. Warning: Duplicate index 'ix_galaxy_user_purged' defined on the table 'galaxydbBGAdev.galaxy_user'. This is deprecated and will be disallowed in a future release. Warning: Duplicate index 'ix_hdadaa_history_dataset_association_id' defined on the table 'galaxydbBGAdev.history_dataset_association_display_at_authorization'. This is deprecated and will be disallowed in a future release. Warning: Duplicate index 'ix_lddaia_ldda_id' defined on the table 'galaxydbBGAdev.library_dataset_dataset_info_association'. This is deprecated and will be disallowed in a future release. Warning: Duplicate index 'ix_hda_ta_history_dataset_association_id' defined on the table 'galaxydbBGAdev.history_dataset_association_tag_association'. This is deprecated and will be disallowed in a future release. Warning: Duplicate index 'ix_galaxy_user_username' defined on the table 'galaxydbBGAdev.galaxy_user'. This is deprecated and will be disallowed in a future release. Warning: Duplicate index 'ix_galaxy_user_form_values_id' defined on the table 'galaxydbBGAdev.galaxy_user'. This is deprecated and will be disallowed in a future release. Warning: Duplicate index 'ix_lia_deleted' defined on the table 'galaxydbBGAdev.library_info_association'. This is deprecated and will be disallowed in a future release. Warning: Duplicate index 'ix_lfia_deleted' defined on the table 'galaxydbBGAdev.library_folder_info_association'. This is deprecated and will be disallowed in a future release. Warning: Duplicate index 'ix_lddia_deleted' defined on the table 'galaxydbBGAdev.library_dataset_dataset_info_association'. This is deprecated and will be disallowed in a future release. Warning: Duplicate index 'ix_sample_folder_id' defined on the table 'galaxydbBGAdev.sample'. This is deprecated and will be disallowed in a future release. Warning: Duplicate index 'ix_lia_inheritable' defined on the table 'galaxydbBGAdev.library_info_association'. This is deprecated and will be disallowed in a future release. Warning: Duplicate index 'ix_lfia_inheritable' defined on the table 'galaxydbBGAdev.library_folder_info_association'. This is deprecated and will be disallowed in a future release. Warning: Duplicate index 'ix_sd_external_service_id' defined on the table 'galaxydbBGAdev.sample_dataset'. This is deprecated and will be disallowed in a future release. Warning: Duplicate index 'ix_icda_ldda_parent_id' defined on the table 'galaxydbBGAdev.implicitly_converted_dataset_association'. This is deprecated and will be disallowed in a future release. Warning: Duplicate index 'ix_icda_ldda_id' defined on the table 'galaxydbBGAdev.implicitly_converted_dataset_association'. This is deprecated and will be disallowed in a future release. Warning: Duplicate index 'ix_tsr_includes_datatypes' defined on the table 'galaxydbBGAdev.tool_shed_repository'. This is deprecated and will be disallowed in a future release. >grep ERROR FirstRun.log | grep failed 0037_samples_library ERROR 2018-06-13 12:01:45,817 [p:17852,w:0,m:0] [MainThread] Deleting column 'library_id' to request table failed. 0037_samples_library ERROR 2018-06-13 12:01:45,832 [p:17852,w:0,m:0] [MainThread] Deleting column 'folder_id' to request table failed. galaxy.model.migrate.check INFO 2018-06-13 12:01:47,129 [p:17852,w:0,m:0] [MainThread] 0037_samples_library ERROR 2018-06-13 12:01:45,817 Deleting column 'library_id' to request table failed. galaxy.model.migrate.check INFO 2018-06-13 12:01:47,133 [p:17852,w:0,m:0] [MainThread] 0037_samples_library ERROR 2018-06-13 12:01:45,817 Deleting column 'library_id' to request table failed. galaxy.model.migrate.check INFO 2018-06-13 12:01:47,137 [p:17852,w:0,m:0] [MainThread] 0037_samples_library ERROR 2018-06-13 12:01:45,832 Deleting column 'folder_id' to request table failed. galaxy.model.migrate.check INFO 2018-06-13 12:01:47,142 [p:17852,w:0,m:0] [MainThread] 0037_samples_library ERROR 2018-06-13 12:01:45,832 Deleting column 'folder_id' to request table failed. 0040_page_annotations ERROR 2018-06-13 12:01:49,598 [p:17852,w:0,m:0] [MainThread] Creating page_annotation_association table failed. 0068_rename_sequencer_to_external_services ERROR 2018-06-13 12:02:26,658 [p:17852,w:0,m:0] [MainThread] Deleting column 'sequencer_id' from the 'request_type' table failed. 0087_tool_id_guid_map_table ERROR 2018-06-13 12:02:48,300 [p:17852,w:0,m:0] [MainThread] Creating tool_id_guid_map table failed. galaxy.model.migrate.check INFO 2018-06-13 12:02:48,309 [p:17852,w:0,m:0] [MainThread] 0087_tool_id_guid_map_table ERROR 2018-06-13 12:02:48,300 Creating tool_id_guid_map table failed. galaxy.model.migrate.check INFO 2018-06-13 12:02:48,313 [p:17852,w:0,m:0] [MainThread] 0087_tool_id_guid_map_table ERROR 2018-06-13 12:02:48,300 Creating tool_id_guid_map table failed. 0112_add_data_manager_history_association_and_data_manager_job_association_tables ERROR 2018-06-13 12:03:22,252 [p:17852,w:0,m:0] [MainThread] Creating data_manager_job_association table failed. 0136_collection_and_workflow_state ERROR 2018-06-13 12:03:57,766 [p:17852,w:0,m:0] [MainThread] Creating workflow_invocation_output_dataset_collection_association table failed. 0136_collection_and_workflow_state ERROR 2018-06-13 12:03:58,326 [p:17852,w:0,m:0] [MainThread] Creating workflow_invocation_step_output_dataset_collection_association table failed.

3 4

instance management inspiration
by Matthias Bernt 22 Jun '18

22 Jun '18

Dear list, I'm looking for some inspiration for managing my galaxy instance (with ansible). In particular I'm looking for ansible roles to - setup quotas - manage the tool panel (I need to get order in the chaos) I'm wondering how the usegalaxy instances manage this? Is there some github that I could browse? Cheers, Matthias -- ------------------------------------------- Matthias Bernt Bioinformatics Service Molekulare Systembiologie (MOLSYB) Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/ Helmholtz Centre for Environmental Research GmbH - UFZ Permoserstraße 15, 04318 Leipzig, Germany Phone +49 341 235 482296, m.bernt(a)ufz.de, www.ufz.de Sitz der Gesellschaft/Registered Office: Leipzig Registergericht/Registration Office: Amtsgericht Leipzig Handelsregister Nr./Trade Register Nr.: B 4703 Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig Wilfried Kraus Wissenschaftlicher Geschäftsführer/Scientific Managing Director: Prof. Dr. Dr. h.c. Georg Teutsch Administrative Geschäftsführerin/ Administrative Managing Director: Prof. Dr. Heike Graßmann -------------------------------------------

2 3

pbs/torque jobs
by Briand, Sheldon (NRC/CNRC) 14 Jun '18

14 Jun '18

Hi, I have upgraded to Galaxy 18.05 (from 17.05). I am running a torque job scheduler (version 6.02). When I submit a job through galaxy and I look in the admin/manage jobs section I see that the job has been submitted successfully. It shows that the job is queued and waiting to run. On the cluster I see that the job runs and goes to completion. However, the status in galaxy continue to show that the job is waiting to run and the status never gets updated. I'm using postgres as my database and I am running through a nginx proxy. I see no errors in my galaxy.log file. I switched from galaxy.ini to galaxy.yml and from paste to uwsgi. Is this a configuration problem on my end? Where should I be looking? Thanks, -Sheldon Sheldon Briand Computer Systems and Applications Analyst National Research Council/Government of Canada Sheldon.briand(a)canada.ca/<mailto:Sheldon.briand@canada.ca/> Tel: (902) 426-1677

2 10

Conflicting conda dependencies for Galaxy tool
by Peter Briggs 14 Jun '18

14 Jun '18

Dear devs I'm currently trying to update a tool developed to wrap an in-house scripted pipeline written by a local researcher, so that it gets its dependencies from conda. The pipeline uses a variety of third-party software packages (many of which are now at "legacy" versions), and I've hit a problem where two of these packages turn out to have conflicting dependencies. Specifically the issue is with qiime=1.8.0, which it turns out is not compatible with any version of R which is available in the channels used by Galaxy. So when Galaxy builds the conda environment for the tool at execution time, it fails with an error from conda: UnsatisfiableError: The following specifications were found to be in conflict: - cairo==1.14.10=0 -> freetype==2.8.1 - cairo==1.14.10=0 -> pixman=0.34 - py2cairo==1.10.0=py27_1 -> cairo==1.12.2 -> freetype=2.4 - py2cairo==1.10.0=py27_1 -> cairo==1.12.2 -> pixman==0.26.2 (The py2cairo requirement comes from an older version of matplotlib that qiime 1.8.0 pulls in.) It's a long shot, but: does Galaxy offer any options for working around conflicting conda dependencies like these? Or is the only option to try and resolve them at the conda packaging level (which might not be practical, or even possible)? Thanks in advance for any advice, Best wishes Peter -- Peter Briggs peter.briggs(a)manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482

1 0

python: error while loading shared libraries
by Floreline TOUCHARD 13 Jun '18

13 Jun '18

Dear Galaxy community, I'm currently testing cross-platform compatibility with our Galaxy instance (15.10) : - [on PROD environment] Galaxy service runs on a RedHat 6.5 server (using python2.6 installed in a virtualenv) and submits jobs on cluster nodes also in RedHat 6.5/python2.6 (default version installed on the system). - [on TEST environment] I changed the configuration of our Galaxy TEST instance (still RedHat 6.5 / python 2.6) to submit jobs on cluster nodes in CentOS7/python2.7 (default version installed on the system). - Galaxy installations (test and prod) and virtualenv folders are located in a volume mounted on Galaxy servers and all cluster nodes. On TEST environment python tools (like the internal Galaxy tool "upload1", or the NCBI BLAST toolkit) if submitted on the cluster failed with the following message : "python: error while loading shared libraries: libpython2.6.so.1.0: cannot open shared object file: No such file or directory" I guess the problem lies in the shared library libpython2.6.so.1.0 located in /usr/lib64/libpython2.6.so.1.0 on the Galaxy server and absent from the cluster nodes (it should be something like /usr/lib64/python2.7/libpython2.7.so.1.0). Is it possible to integrate this shared library in the Python virtualenv and use a specific configuration to indicate Galaxy to use it when submitting jobs on the cluster ? Thanks in advance, Floreline.

1 0