- galaxy-dev - lists.galaxyproject.org

Improved DRMAAJobRunner
by Matthias Bernt 07 Jul '17

07 Jul '17

Dear list, I have implemented a new DRMAA job runner allows to detect runtime and memory violations. This is done by using the drmaa library functions job_info + wait or as fallback the commandline tools qstat + qacct. This has the additional advantage that the runner works also in setups using the external runner scripts to start jobs as real user (the original DRMAAJobRunner can not query these jobs at all). You may have a look here: https://github.com/galaxyproject/galaxy/pull/4275 I have successfully tested the but settings: galaxy user/real user submits the jobs. Also the resubmission in case of memory and time violations seems to work. Would be great to get some comments. One problem that I encountered is that upload jobs do not work in the real user setting (for the original and the new runner): The permissions of the uploaded file in /gpfs1/data/galaxy_server/galaxy-dev/database/tmp/ are not changed. Any idea what needs to be changed to get this running? Best, Matthias -- ------------------------------------------- Matthias Bernt Bioinformatics Service Molekulare Systembiologie (MOLSYB) Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/ Helmholtz Centre for Environmental Research GmbH - UFZ Permoserstraße 15, 04318 Leipzig, Germany Phone +49 341 235 482296, m.bernt(a)ufz.de, www.ufz.de Sitz der Gesellschaft/Registered Office: Leipzig Registergericht/Registration Office: Amtsgericht Leipzig Handelsregister Nr./Trade Register Nr.: B 4703 Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig Wilfried Kraus Wissenschaftlicher Geschäftsführer/Scientific Managing Director: Prof. Dr. Dr. h.c. Georg Teutsch Administrative Geschäftsführerin/ Administrative Managing Director: Prof. Dr. Heike Graßmann -------------------------------------------

2 1

running tests angainst an existing galaxy instance
by Gudrun Amedick 07 Jul '17

07 Jul '17

Hi, I'm trying to run tests against my galaxy server. I'm using: ./run_tests.sh --external_url my.server.example --external_master_key somekey To my understanding, this should run functional tests against my server. However, my server doesn't see any traffic at all. I can log into my Galaxy Server, so at least some parts are running without errors. We're using version "origin/release_17.01" from the git repository on Debian 8. Has anybody a good advice? Am I missing any environment variables? Have a nice weekend, Gudrun Amedick -- Gudrun Amedick IT-Systemtechnik UNIVERSITÄT ZU LÜBECK IT-Service-Center Ratzeburger Allee 160 23562 Lübeck Tel +49 451 3101 2035 Fax +49 451 3101 2004 amedick(a)itsc.uni-luebeck.de www.itsc.uni-luebeck.de

1 0

pyconfig.h missing
by Fiona Mary Roche 06 Jul '17

06 Jul '17

Hi there, I am trying to install cutadapt (package_cutadapt_1_8, from iuc) on a newly installed 17.05 Galaxy on Ubuntu. Both cutadapt and cython (a dependency) fail with the following errors: cutadapt/_align.c:8:22: fatal error: pyconfig.h: No such file or directory compilation terminated. error: command 'x86_64-linux-gnu-gcc' failed with exit status 1 Cython/Plex/Scanners.c:8:22: fatal error: pyconfig.h: No such file or directory compilation terminated. error: command 'x86_64-linux-gnu-gcc' failed with exit status 1 It seems that the python-dev package is missing. Should Galaxy have installed that (through the virtual environment) or does it need to be installed separately on the command line. If so, where would it have to go? Thanks for any help. Fiona

1 0

"Best practice" for maintaining genome list in Galaxy
by Peter van Heusden 06 Jul '17

06 Jul '17

Hi there Is there current "best practice" for maintaining a collection of reference genome data in a Galaxy server? I.e. filling in the dbkeys.loc and all_fasta.loc table and associated data. I know of the following discussions on this topic: 1. The data integration page: https://galaxyproject.org/admin/data-integration/ 2. The data managers page: https://galaxyproject.org/admin/tools/data-managers/ 3. This biostars answer: https://biostar.usegalaxy.org/p/7176/ 4. This page on custom genomes that seems to be user / history oriented: https://galaxyproject.org/learn/custom-genomes/ 5. This page on rsyncing from the Galaxy reference data collection: https://galaxyproject.org/admin/use-galaxy-rsync/ - quick test gives me rsync error though 6. This guide to using the human reference data: https://biostar.usegalaxy.org/p/14777/ 7. I hear the usegalaxy.org reference collection is available via cvmfs - I think this was discussed in a Galaxy Admins meetup at some point. None of this is comprehensive. There are 2 sets of questions for me (in my role as Galaxy admin): 1. If I want to make reference genomes and common indices (like for HISAT and BWA) available for e.g. human and mouse, what is the best way to do this? 2. If I want to add a genome for a non-model organism (e.g. M. tuberculosis or L. calcarifer) available, what is the best way to do this? Which data manager should or could I use? Thanks, Peter

1 0

Moving Shared Data Library Folders
by D K 06 Jul '17

06 Jul '17

Is it possible to move the file locations of Shared data libraries that have already been added to the system? I just tried moving some of these files to a new location (e.g. /somewhere/files to /somewhere_else/libraries) and updating the galaxy.ini entry to point to the new location. However, files previously added to a library still point to the old location. I found this question on biostars: https://biostar.usegalaxy.org/p/1119/ Is the only way to do this sort of update to manually change such entries in the database? Thanks!

2 1

DeepTools installation error
by Timo Janßen 05 Jul '17

05 Jul '17

Hi all, trying to install the deeptools_suite from the toolshed I only get the following error: "500 Internal Server Error nginx/1.11.10" This happens as well for the dependencies, e.g. deeptools_bigwig_compare. I am currently using Galaxy 16.10 with nginx proxy. Also the installation of the dependency package_python_2_7_matplotlib_1_4 fails because of a missing dependency: "Error installing tool dependency python version 2.7.10: Unable to locate required tool shed repository named package_python_2_7_10 owned by iuc with revision 80931fb5f433." Has anyone seen these problems before and knows a solution? Best regards, Timo Janßen -- ------------------------------------------------------------------------ Timo Janßen Wissenschaftliche Hilfskraft Arbeitsgruppe "Anwendungs- und Informationssysteme" Tel.: +49(0)551/201-1791 E-Mail: timo.janssen(a)gwdg.de ------------------------------------------------------------------------ Gesellschaft für wissenschaftliche Datenverarbeitung mbH Göttingen (GWDG) Am Faßberg 11, 37077 Göttingen, URL: http://www.gwdg.de Tel.: +49 551 201-1510, Fax: +49 551 201-2150, E-Mail: gwdg(a)gwdg.de Service-Hotline: Tel.: +49 551 201-1523, E-Mail: support(a)gwdg.de Geschäftsführer: Prof. Dr. Ramin Yahyapour Aufsichtsratsvorsitzender: Prof. Dr. Norbert Lassau Sitz der Gesellschaft: Göttingen Registergericht: Göttingen, Handelsregister-Nr. B 598 ------------------------------------------------------------------------ Zertifiziert nach ISO 9001 ------------------------------------------------------------------------

6 9

conda problem with R libraries
by Jochen Bick 04 Jul '17

04 Jul '17

Hi all, I'm testing a tool with planemo and the conda dependencies are not running: <requirements> <requirement type="package" version="3.3.2">r-base</requirement> <requirement type="package" version="0.4.1">r-ggfortify</requirement> <requirement type="package" version="0.1.0">r-getoptlong</requirement> </requirements> galaxy.jobs.handler INFO 2017-07-04 13:11:42,935 (2) Job dispatched galaxy.util.object_wrapper WARNING 2017-07-04 13:11:43,062 Unable to create dynamic subclass for <type 'instance'>, None: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases ...The following specifications were found to be in conflict: - cairo 1.14.6 4 - r-assertthat (target=r-assertthat-0.1-r3.3.2_4.tar.bz2) -> r-base 3.3.2* ... ... somehow the conflicts can not be resolved=? Cheers Jochen -- ETH Zurich *Jochen Bick* Animal Physiology Institute of Agricultural Sciences Postal address: Universitätstrasse 2 / LFW B 58.1 Office: Tannenstrasse 1 / TAN D 6.2 8092 Zurich, Switzerland Phone +41 44 632 28 25 jochen.bick(a)usys.ethz.ch <mailto:jochen.bick@usys.ethz.ch> www.ap.ethz.ch

1 0

Python modules not found when starting with init system
by Maximilian Friedersdorff 28 Jun '17

28 Jun '17

Hi All, when my local galaxy is started using the init system (systemd on Ubuntu 16.04 in this instance). Some tools will fail, indicating that some python module do not exist. This is the result of uploading a 2KB plain text file: Traceback (most recent call last): File "/home/galaxy/galaxy/tools/data_source/upload.py", line 20, in <module> from galaxy.datatypes import sniff File "/home/galaxy/galaxy/lib/galaxy/datatypes/sniff.py", line 31, in <module> from galaxy.datatypes.binary import Binary File "/home/galaxy/galaxy/lib/galaxy/datatypes/binary.py", line 14, in <module> import pysam ImportError: No module named pysam Traceback (most recent call last): File "/home/galaxy/galaxy/database/jobs_directory/000/39/set_metadata_EI05GD.py", line 1, in <module> from galaxy_ext.metadata.set_metadata import set_metadata; set_metadata() File "/home/galaxy/galaxy/lib/galaxy_ext/metadata/set_metadata.py", line 23, in <module> from sqlalchemy.orm import clear_mappers ImportError: No module named sqlalchemy.orm When the same galaxy install is started directly using the 'run.sh' script in the install directory, everything works fine. I've verified that I can manually import pysam and sqlalchemy.orm if I execute the python interpreter located at <install_dir>/.venv/bin/python directly. Any ideas? I've attached the init script. TL;DR Galaxy cannot import some python modules when started by the init system. No idea why. Galaxy version: 17.05 OS: Ubuntu 16.04 Many Thanks Max -------------------------------------------------------------------- Un o’r 4 prifysgol uchaf yn y DU a’r orau yng Nghymru am fodlonrwydd myfyrwyr. (Arolwg Cenedlaethol y Myfyrwyr 2016) www.aber.ac.uk Top 4 UK university and best in Wales for student satisfaction (National Student Survey 2016) www.aber.ac.uk

3 4

new installation galaxy-17.05 - Uncaught error.
by Fernandez Edgar 26 Jun '17

26 Jun '17

Hey Guys, Hope everyone is doing ok. I have a small issue with a brand new installation of galaxy-1705. It starts properly and I installed some tools. But when I try to test any tools I get and "Uncaught error" (see picture). I can't find anywhere in the logs what is happening. Any ideas? Regards, Edgar Fernandez System Administrator (Linux) Technologies de l'Information Université de Montréal PAVILLON ROGER-GAUDRY, bureau X-210 * Bur. : 1-514-343-6111 poste 16568

4 9

Galaxy Dinner @ ISMB/ECCB/BOSC 2017
by Dave Clements 25 Jun '17

25 Jun '17

Hello all, If you are interested in getting together for dinner at ISMB/ECCB/BOSC 2017 #BOSC2017 then please *let us know by Friday:* http://bit.ly/ismbdinnerpoll Thanks, Dave C -- https://galaxyproject.org/ https://getgalaxy.org/ https://usegalaxy.org/ https://gcc2017.sciencesconf.org/

1 0