March 2016 - galaxy-dev - lists.galaxyproject.org

error loading files into galaxy
by Hakeem Almabrazi 04 Mar '16

04 Mar '16

Hi, I started getting the following error whenever I try to load a file into Galaxy local. Traceback (most recent call last): File "/usr/local/galaxy/galaxy-dist/tools/data_source/upload.py", line 8, in <module> from galaxy import eggs ImportError: cannot import name eggs I will appreciate if someone can tell me what could happen to cause such issue and how to resolve it. Regards,

2 1

playbook download error
by William Ge 03 Mar '16

03 Mar '16

Hi, all, I tried to download the playbook for automatic installation of Galaxy tools, but encountered the following error: [galaxy@Pegasus galaxy]$ git clone --recursive git@github.com:afgane/galaxy-tools-playbook.git Initialized empty Git repository in /home/galaxy/galaxy/galaxy-tools-playbook/.git/ Permission denied (publickey). fatal: The remote end hung up unexpectedly Is this repository still open to public download? I'm new and not quite familiar with the playbook application. Any step by step guide will be helpful. Thanks, Bill

2 2

nginx data upload request never reaches /api/tools
by Preussner, Jens 03 Mar '16

03 Mar '16

Hi all, I set up the data upload using nginx as described in the wiki (https://wiki.galaxyproject.org/Admin/Config/nginxProxy, using the subdirectory /galaxy). The uploaded file also appears inside the uploade_store folder on the galaxy server, but galaxy does not add the file. Investigating the log files, I found the corresponding lines in the nginx.error.log: # Post request is received 2016/02/29 12:55:37 [notice] 13156#0: *4 "^(GET|HEAD|POST)$" matches "POST", client: 172.16.17.17, server: , request: "POST /_upload HTTP/1.1", host: "172.16.12.122", referrer: http://172.16.12.122/galaxy/ # Upload starts 2016/02/29 12:55:37 [info] 13156#0: *4 started uploading file "boxes.txt" to "/mnt/idrive1/database/tmp/upload_store/0020132707" (field "files_0|file_data", content type "text/plain"), client: 172.16.17.17, server: , request: "POST /_upload HTTP/1.1", host: "172.16.12.122", referrer: http://172.16.12.122/galaxy/ # Upload done, the file is present in the given path 2016/02/29 12:55:37 [info] 13156#0: *4 finished uploading file "boxes.txt" to "/mnt/idrive1/database/tmp/upload_store/0020132707", client: 172.16.17.17, server: , request: "POST /_upload HTTP/1.1", host: "172.16.12.122", referrer: "http://172.16.12.122/galaxy/" # Redirection starts? 2016/02/29 12:55:37 [notice] 13156#0: *4 "^(GET|HEAD|POST)$" matches "POST", client: 172.16.17.17, server: , request: "POST /_upload HTTP/1.1", host: "172.16.12.122", referrer: "http://172.16.12.122/galaxy/" 2016/02/29 12:55:37 [notice] 13156#0: *4 "nginx_redir=([^&]+)" does not match "", client: 172.16.17.17, server: , request: "POST /_upload HTTP/1.1", host: "172.16.12.122", referrer: "http://172.16.12.122/galaxy/" 2016/02/29 12:55:37 [notice] 13156#0: *4 "" matches "/_upload_done", client: 172.16.17.17, server: , request: "POST /_upload HTTP/1.1", host: "172.16.12.122", referrer: "http://172.16.12.122/galaxy/" # Redirection should be done here 2016/02/29 12:55:37 [notice] 13156#0: *4 rewritten data: "/galaxy/api/tools", args: "", client: 172.16.17.17, server: , request: "POST /_upload HTTP/1.1", host: "172.16.12.122", referrer: "http://172.16.12.122/galaxy/" # ModSecurity seems to block the redirect? 2016/02/29 12:55:37 [info] 13156#0: *4 ModSecurity: invalid "Content-Length" header, client: 172.16.17.17, server: , request: "POST /_upload HTTP/1.1", host: "172.16.12.122", referrer: "http://172.16.12.122/galaxy/" 2016/02/29 12:55:37 [notice] 13156#0: *4 "^(GET|HEAD|POST)$" matches "GET", client: 172.16.17.17, server: , request: "POST /_upload HTTP/1.1", host: "172.16.12.122", referrer: "http://172.16.12.122/galaxy/" A ModSecurityEnabled off; within the /_upload and /_upload_done of the nginx config did not help. Any idea why the Content-Length header causes problems and is blocked by modSec? Any workaround for ModSec to accept everything that goes from internal to internal? Thanks for your help! Best, Jens

2 2

workflows "show in menu" by default
by Cameron Jack 02 Mar '16

02 Mar '16

Hi all, In the /workflow/configure_menu screen there is an tickbox option for each installed workflow to "show in menu". Is there a way I could enable this by default, or in code via bioblend (couldn't see any hint of this in the docs)? Perhaps I could extend bioblend to do this? Cheers, Cam Cameron Jack Acting Manager & Bioinformatician ANU Bioinformatics Consultancy The Australian National University The John Curtin School of Medical Research Building 131 Garran Road Acton 0200, ACT Ph (office): +61 2 612 51128 Ph (mobile): +61 4 2368 0745 Email: cameron.jack(a)anu.edu.au Group email: abc(a)anu.edu.au

2 1

Galaxy on Azure
by Alex Lenail 02 Mar '16

02 Mar '16

Hello, My name is Alex LeNail and I'm a researcher at MIT. I'm looking to set up a Galaxy server, and our lab has a great deal of Microsoft Azure credits, so we were hoping to deploy Galaxy on there. Where should I start if I wanted to set that up? I haven't really found any resources online with any description of how that might be accomplished. Thanks, --Alex

2 2

Dependencies for sudo scripts/drmaa_external_runner.py ?
by Peter Cock 02 Mar '16

02 Mar '16

Hello all, QUESTION: When submitting jobs to the cluster as the real user, how should sudo scripts/drmaa_external_runner.py be told which Python to use, and how would it activate the venv if needed for the DRMAA dependency? BACKGROUND: We're currently trying Galaxy out on a new CentOS 6 VM, with matching CentOS 6 cluster, where jobs are submitted to SGE via DRMAA and run as the Linux user rather than a generic Galaxy Linux user account. This is documented on the wiki here: https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster This all seemed to be working under Galaxy v15.10 (using eggs), but we're now targeting the recently released Galaxy v16.10 (using wheels) instead and have run into problems. https://github.com/galaxyproject/galaxy/issues/1596 Because Galaxy is deprecating support for Python 2.6 (the default bundled with CentOS 6), we're now using a local copy of Python 2.7 (compiled from source) on a shared mount. This mismatch seems to be the root cause of the problem I will now describe. During job submission to SGE, Galaxy will attempt to run a command like this: $ sudo scripts/drmaa_external_runner.py 1005 /mnt/shared/galaxy/galaxy-dist/database/sge/132.jt_json >From the terminal output from ./run.sh we'd see: RuntimeError: External_runjob failed (exit code 1) Child process reported error: Traceback (most recent call last): File "/mnt/shared/galaxy/galaxy-dist/scripts/drmaa_external_runner.py", line 15, in <module> import drmaa ImportError: No module named drmaa Although a drmaa wheel was installed within the Python 2.7 virtual environment under ~/galaxy-dist/.venv Galaxy makes no attempt to activate the venv for scripts/drmaa_external_runner.py We then installed DRMAA under our local copy of Python 2.7, and realised sudo scripts/drmaa_external_runner.py was not even using this copy of Python. Changing the hash bang line was a crude way to solve that (see below). This in turn lead to finding $DRMAA_LIBRARY_PATH and $SGE_ROOT were not set in the sudo environment. Again, you can hack around this by modifying scripts/drmaa_external_runner.py (see below). In our case, I suspect the least invasive change would be to install the DRMAA libraries under the system provided Python 2.6, and let sudo scripts/drmaa_external_runner.py execute that way. We still need to solve why sudo scripts/drmaa_external_runner.py does not see $DRMAA_LIBRARY_PATH and $SGE_ROOT but we have some clues to follow up on that: http://stackoverflow.com/questions/257616/sudo-changes-path-why Peter P.S. See also https://twitter.com/pjacock/status/704335582651162624 -- Here's our workaround diff - lots of hard coded strings, not portable at all but it worked for testing/debugging: $ git diff scripts/drmaa_external_runner.py diff --git a/scripts/drmaa_external_runner.py b/scripts/drmaa_external_runner.py index a1474fe..61d2383 100755 --- a/scripts/drmaa_external_runner.py +++ b/scripts/drmaa_external_runner.py @@ -1,5 +1,6 @@ +#!/mnt/shared/galaxy/apps/python/2.7.11/bin/python +#Was #!/usr/bin/env python - """ Submit a DRMAA job given a user id and a job template file (in JSON format) defining any or all of the following: args, remoteCommand, outputPath, @@ -12,8 +13,15 @@ import os import pwd import sys +# Hack +# print "$DRMAA_LIBRARY_PATH is %s" % os.environ.get('DRMAA_LIBRARY_PATH') +# print "$SGE_ROOT is %s" % os.environ.get('SGE_ROOT') +os.environ['DRMAA_LIBRARY_PATH'] = '/mnt/sge/lib/lx-amd64/libdrmaa.so' +os.environ['SGE_ROOT'] = '/mnt/sge' + import drmaa DRMAA_jobTemplate_attributes = [ 'args', 'remoteCommand', 'outputPath', 'errorPath', 'nativeSpecification', 'workingDirectory', 'jobName', 'email', 'project' ]

2 4

Load tools without service interruption
by Floreline TOUCHARD 02 Mar '16

02 Mar '16

Hi Galaxy team, Is there a way to add a tool without service interruption ? Something like a "sh run.sh --reload" (not "restart") command. I think I saw this information once, but I can't put my hand on it anymore. Thanks in advance ! F.T

3 2

Docker tools
by Abdulrahman Azab 02 Mar '16

02 Mar '16

Hi Galaxy, I'm working on a project for running galaxy on the top of a docker swarm cluster. The docker tag in the tool xml so far supports access only to the local docker host. To submit to the swarm manager, the command should be: $ docker -H tcp://<manager-ip>:<manager-port> run ... instead of: $ docker run ... Is it possible to add an xml tag attribute to support that option? This would off course require that all files are located in a shared file-system. Thank you, Yours sincerely, Abdulrahman Azab Head engineer ELIXIR.NO, Genomic Hyperbrowser Team Research Group for Biomedical Informatics, Department of Informatics (IFI) Research Support Services Group, University Center for Information Technology (USIT) University of Oslo, Boks 1072 Blindern, NO-0316 OSLO, Norway Email: azab(a)ifi.uio.no, azab(a)usit.uio.no Cell-phone: +47 46797339 ---- Senior Lecturer in Computer Engineering Faculty of Engineering, University of Mansoura, 35516-Mansoura, Egypt Email: abdulrahman.azab(a)mans.edu.eg<https://mail.uio.no/owa/abdulrahman.azab@mans.edu.eg>

3 2

Fwd: BOSC 2016 Call for Abstracts
by Peter Cock 02 Mar '16

02 Mar '16

Dear all, As BOSC co-chair I would like to encourage you all to think about attending BOSC 2016, and if you are working on your own open source software for bioinformatics please consider submitting an abstract. See the email below and: http://news.open-bio.org/2016/03/01/bosc-2016-call-for-abstracts/ We've had lots of Galaxy related talks at BOSC over the years, and look forward to more abstract submissions from the Galaxy community this year as well. Also, as a member of the Open Bioinformatics Foundation (OBF) Board of Directors, I am delighted to let you know about the new OBF Travel Fellowship which could be used to attend BOSC, or the Galaxy Community Conference (GCC2016) if your work fits: http://news.open-bio.org/2016/03/01/obf-travel-fellowship-program/ Thank you, Peter [Galaxy contributor & tool developer, BOSC co-chair, OBF Secretary, etc.] P.S. In case you missed the earlier announcement last year, we finally got rid of the paper forms for OBF membership, see: http://news.open-bio.org/2015/12/10/online-membership-form/ ---------- Forwarded message ---------- From: Peter Cock <p.j.a.cock(a)googlemail.com> Date: Tue, Mar 1, 2016 at 5:56 PM Subject: BOSC 2016 Call for Abstracts To: bosc-announce(a)lists.open-bio.org Cc: bosc-reviewers(a)lists.open-bio.org, "bosc(a)open-bio.org" <bosc(a)open-bio.org> Call for Abstracts for the 17th Annual Bioinformatics Open Source Conference (BOSC 2016), a Special Interest Group (SIG) of ISMB 2016. Dates: July 8-9, 2016 Location: Orlando, FL Web site: http://www.open-bio.org/wiki/BOSC_2016 Email: bosc(a)open-bio.org BOSC announcements mailing list: http://lists.open-bio.org/mailman/listinfo/bosc-announce Twitter: @OBF_BOSC and @OBF_News Important Dates: * Call for one-page abstracts opens: March 1, 2016 * Abstract submission deadline: April 1, 2016 * Travel fellowship application deadline: April 15, 2016 * Authors notified: May 6, 2016 * Codefest 2016: July 6-7, 2016, Orlando, FL (confirming venue) * BOSC 2016: July 8-9, 2016, Orlando, FL * ISMB 2016: July 8-12, 2016, Orlando, FL The Bioinformatics Open Source Conference (BOSC) is run as a two-day meeting before the annual ISMB conference. It is organized by the Open Bioinformatics Foundation (OBF), a non-profit group dedicated to promoting the practice and philosophy of open source software development and open science within the biological research community. BOSC offers a focused environment for developers and users to interact and share ideas about standards; software development practices; practical techniques for solving bioinformatics problems; and approaches that promote open science and sharing of data, results and software. We welcome one-page abstracts on any topic of relevance to open source bioinformatics and open science. Presentation formats are lightning talks, longer talks, and/or posters. We plan to offer a limited number of travel fellowships to help offset expenses for some accepted speakers who would not otherwise be able to attend BOSC – please see the OBF Travel Fellowship announcement for more information: http://news.open-bio.org/2016/03/01/obf-travel-fellowship-program/ Session topics include: * Open Science and Reproducible Research * Standards and Interoperability * Data Science * Visualization * Translational Bioinformatics * Bioinformatics Open Source Libraries and Projects If your company or organization is interested in helping to sponsor BOSC 2016, please contact us! Sponsors in 2015 included Google, GigaScience, Curoverse and Bina – we thank them for their support. Please see http://www.open-bio.org/wiki/BOSC_Abstract_Submission for abstract submission instructions, Thank you, BOSC 2016 Organizing Committee: Nomi Harris and Peter Cock (co-chairs), Brad Chapman, Christopher Fields, Karsten Hokamp, Hilmar Lapp, Mónica Muñoz-Torres, Heather Wiencko

1 0

Galaxy External Authentication and API access
by Lance Parsons 02 Mar '16

02 Mar '16

I am currently using Apache and CAS (https://en.wikipedia.org/wiki/Central_Authentication_Service) to provide external authentication to Galaxy. I'd like to start using the API, however, I've not been able to find a way to get that to work. A while back, someone suggested using the following: <Location "/api"> Satisfy Any Allow from all </Location> However, this breaks the history view at least, since the ajax call is an /api call and no HTTP_REMOTE_USER gets set. Does anyone have any suggestions? I'd love to get this going with some simple configuration... I know I could move over to LDAP, but that is not desirable since it's less secure than CAS. Another option would be to implement CAS within Galaxy, but that does seem like a good bit of work, but perhaps someone is already working on it or at least interested? -- Lance Parsons - Scientific Programmer Carl C. Icahn Laboratory - Room 141 (Temporary) Lewis-Sigler Institute for Integrative Genomics Princeton University

2 1