August 2015 - galaxy-dev - lists.galaxyproject.org

Installing Cairo into Gakaxy
by Christian Brenninkmeijer 13 Oct '15

13 Oct '15

Hi All, Has anyone got a working example tool_dependencies.xml to install the Cairo package into R. (I actually need a package that depends on R Cairo but the current issue is Cairo) I am having the cairo.h not found issue I have cloned https://toolshed.g2.bx.psu.edu/view/devteam/package_cairo_1_12_14/b39299b4b… (and its dependencies) As well as https://toolshed.g2.bx.psu.edu/view/iuc/package_r_3_1_2/9f2fddb9d6e2 I am trying to use <action type="setup_r_environment"> But can not get the package_cairo to be seen by the R I have tried <action type="set_environment_for_install"> both inside and outside of the setup_r_environment but it does not appear to be working, Full attempted tool_dependencies.xml attached, Thanks in advance Christian University of Manchester

4 24

Re: [galaxy-dev] validator type="empty_field" is not working within repeat tags
by Anmol Hemrom 25 Sep '15

25 Sep '15

Hi, I was trying to enforce not to allow empty string as an input to text field. It is working outside the repeat tags but not working within repeat tags. 1) Working fine <param name="input" type="text" label="Enter Name"> <validator type="empty_field"/> </param> 2) Not Working within repeat tags. <repeat name="test" title="Test"> <param name="input" type="text" label="Enter Name"> <validator type="empty_field"/> </param> </repeat> Thanks Anmol On Fri, May 29, 2015 at 2:00 PM, Dave Clements <clements(a)galaxyproject.org> wrote: > Hi Anmol, > > The best place to post this question is the Galaxy Dev List < > galaxy-dev(a)lists.galaxyproject.org>. You can also search to see if > anyone has asked a similar question at > https://galaxyproject.org/search/web/ > > Thanks, > > Dave C > > On Fri, May 29, 2015 at 6:56 AM, <anmol.jh(a)gmail.com> wrote: > >> Hi, >> >> I was trying to enforce not to allow empty string as an input to text >> field. It is working outside the repeat tags but not working within repeat >> tags. >> 1) Working fine >> >> <param name="input" type="text" label="Enter Name"> >> <validator type="empty_field"/> >> </param> >> >> 2) Not Working within repeat tags. >> >> <repeat name="test" title="Test"> >> <param name="input" type="text" label="Enter Name"> >> <validator type="empty_field"/> >> </param> >> </repeat> >> >> Thanks >> >> Anmol >> >> _____________________________________ >> Sent from http://dev.list.galaxyproject.org >> >> > > > -- > http://galaxyproject.org/ > http://getgalaxy.org/ > http://usegalaxy.org/ > https://wiki.galaxyproject.org/ >

2 1

Adding categories to Main
by Keith Suderman 12 Sep '15

12 Sep '15

Dear Galaxy Team, A colleague would like to upload some NLP (Natural Language Processing) tools to the Galaxy Test/Main tools sheds, but we are unclear what "categories" to use for the tools. I see that the Main/Test tools sheds have a category for "Text Manipulation", but that does not seem appropriate for NLP tools. Is it possible to have new categories added to the tool shed(s)? If so, what is the process? I am just starting to investigate setting up our own local tool shed and I am coming across mentions of repository capsules and exporting tool sheds. Would it be preferable to install the NLP tools to a local tool shed and then export a repository/capsule to be imported to the Test/Main tools sheds? What happens if our tool shed uses a disjoint set of categories than Test/Main? Cheers, Keith ------------------------------ Research Associate Department of Computer Science Vassar College Poughkeepsie, NY

4 9

Unable to run simple Docker tool
by Mikel Egaña Aranguren 29 Aug '15

29 Aug '15

Hi; I'm trying to develop a Docker based tool, as suggested by a peer-reviewer who might be reading this :P However, I'm having trouble with even the most basic setting, and I don't know what might be wrong, so any help will be much appreciated. I have developed a very simple docker image and corresponding Galaxy tool, so that I get it working before starting with the actual tool, but when I execute it through Galaxy it simply stays executing forever, instead of failing or terminating. My image simply executes a shell script that reads the content of a file and concatenates a string to it. The image: FROM busybox:ubuntu-14.04 MAINTAINER Mikel Egaña Aranguren <mikel.egana.aranguren(a)gmail.com> RUN mkdir /sadi COPY test-io.sh /sadi/ RUN chmod a+x /sadi/test-io.sh ENV PATH $PATH:/sadi The test-io.sh script within the image: #!/bin/sh cat $1 echo "AAA" Invoking the container and executing the script through a normal shell works fine: REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE mikeleganaaranguren/busybox-galaxy-test-io v1 9c2b8bdade1d 54 minutes ago 5.609 MB docker run -i -t mikeleganaaranguren/busybox-galaxy-test-io:v1 BusyBox v1.21.1 (Ubuntu 1:1.21.0-1ubuntu1) built-in shell (ash) Enter 'help' for a list of built-in commands. ~ # ~ # echo "BBB" > test ~ # test-io.sh test BBB AAA This is the tool file I'm using in Galaxy: <tool id="SADIBUSYBOX" name="SADIBUSYBOX"> <description>IO</description> <requirements> <container type="docker">mikeleganaaranguren/busybox-galaxy-test-io:v1</container> </requirements> <command> test-io.sh $input > $output </command> <inputs> <param name="input" type="data" label="Dataset"/> </inputs> <outputs> <data format="txt" name="output" /> </outputs> <help> </help> </tool> And my job_conf.xml: <?xml version="1.0"?>  <job_conf> <plugins> <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="4"/> </plugins> <handlers> <handler id="main"/> </handlers> <destinations default="docker_local"> <destination id="local" runner="local"/> <destination id="docker_local" runner="local"> <param id="docker_enabled">true</param> </destination> </destinations> </job_conf> As I said, when I execute the tool in Galaxy, it simply executes forever, it stays in a "yellow state", till I kill the Galaxy server. The log says: 127.0.0.1 - - [18/Aug/2015:19:08:00 +0200] "GET /tool_runner?tool_id=SADIBUSYBOX HTTP/1.1" 200 - "http://127.0.0.1:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0" galaxy.tools.actions INFO 2015-08-18 19:08:07,178 Handled output (77.825 ms) galaxy.tools.actions INFO 2015-08-18 19:08:07,280 Verified access to datasets (12.387 ms) galaxy.tools.execute DEBUG 2015-08-18 19:08:07,307 Tool [SADIBUSYBOX] created job [2] (217.508 ms) 127.0.0.1 - - [18/Aug/2015:19:08:07 +0200] "POST /api/tools HTTP/1.1" 200 - "http://127.0.0.1:8080/tool_runner?tool_id=SADIBUSYBOX" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0" 127.0.0.1 - - [18/Aug/2015:19:08:07 +0200] "GET /api/histories/f2db41e1fa331b3e/contents HTTP/1.1" 200 - " http://127.0.0.1:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0" galaxy.jobs DEBUG 2015-08-18 19:08:07,714 (2) Working directory for job is: /home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2 galaxy.jobs.handler DEBUG 2015-08-18 19:08:07,721 (2) Dispatching to local runner galaxy.jobs DEBUG 2015-08-18 19:08:07,839 (2) Persisting job destination (destination id: docker_local) galaxy.jobs.runners DEBUG 2015-08-18 19:08:07,849 Job [2] queued (128.176 ms) galaxy.jobs.handler INFO 2015-08-18 19:08:07,897 (2) Job dispatched galaxy.jobs.command_factory INFO 2015-08-18 19:08:08,163 Built script [/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2/tool_script.sh] for tool command[/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2/tool_script.sh] galaxy.jobs.runners DEBUG 2015-08-18 19:08:08,322 (2) command is: sudo docker inspect mikeleganaaranguren/busybox-galaxy-test-io:v1 > /dev/null 2>&1 [ $? -ne 0 ] && sudo docker pull mikeleganaaranguren/busybox-galaxy-test-io:v1 > /dev/null 2>&1 sudo docker run -e "GALAXY_SLOTS=$GALAXY_SLOTS" -v /home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy:/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy:ro -v /home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/tools/catDocker:/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/tools/catDocker:ro -v /home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2:/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2:rw -v /home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/files:/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/files:rw -w /home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2 --net none --rm -u 1001 mikeleganaaranguren/busybox-galaxy-test-io:v1 /home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2/tool_script.sh; return_code=$?; python "/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2/set_metadata_3aWBf9.py" "/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/tmp/tmpH2gy3X" "/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2/galaxy.json" "/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2/metadata_in_HistoryDatasetAssociation_2_WxbKNP,/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2/metadata_kwds_HistoryDatasetAssociation_2_p8UK91,/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2/metadata_out_HistoryDatasetAssociation_2_wPm09U,/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2/metadata_results_HistoryDatasetAssociation_2_irSo7t,/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/files/000/dataset_2.dat,/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2/metadata_override_HistoryDatasetAssociation_2_1glVfb" 5242880; sh -c "exit $return_code" galaxy.jobs.runners.local DEBUG 2015-08-18 19:08:08,323 (2) executing job script: /home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2/galaxy_2.sh galaxy.jobs DEBUG 2015-08-18 19:08:08,375 (2) Persisting job destination (destination id: docker_local) 127.0.0.1 - - [18/Aug/2015:19:08:11 +0200] "GET /api/histories/f2db41e1fa331b3e/contents HTTP/1.1" 200 - " http://127.0.0.1:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0" 127.0.0.1 - - [18/Aug/2015:19:08:21 +0200] "GET /history/current_history_json HTTP/1.1" 200 - "http://127.0.0.1:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0" 127.0.0.1 - - [18/Aug/2015:19:08:21 +0200] "GET /api/histories/f2db41e1fa331b3e/contents?dataset_details=ebfb8f50c6abde6d%2C33b43b4e7093c91f%2Ca799d38679e985db%2C5969b1f7201f12ae%2Cdf7a1f0c02a5b08e%2C0a248a1f62a0cc04%2C03501d7626bd192f%2C3f5830403180d620%2Ce85a3be143d5905b%2Cc9468fdb6dc5c5f1%2C2a56795cad3c7db3 HTTP/1.1" 200 - "http://127.0.0.1:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0" 127.0.0.1 - - [18/Aug/2015:19:08:25 +0200] "GET /api/histories/f2db41e1fa331b3e/contents HTTP/1.1" 200 - " http://127.0.0.1:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0" 127.0.0.1 - - [18/Aug/2015:19:08:29 +0200] "GET /api/histories/f2db41e1fa331b3e/contents HTTP/1.1" 200 - " http://127.0.0.1:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0" 127.0.0.1 - - [18/Aug/2015:19:08:33 +0200] "GET /api/histories/f2db41e1fa331b3e/contents HTTP/1.1" 200 - " http://127.0.0.1:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0" 127.0.0.1 - - [18/Aug/2015:19:08:38 +0200] "GET /api/histories/f2db41e1fa331b3e/contents HTTP/1.1" 200 - " http://127.0.0.1:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0" 127.0.0.1 - - [18/Aug/2015:19:08:42 +0200] "GET /api/histories/f2db41e1fa331b3e/contents HTTP/1.1" 200 - " http://127.0.0.1:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0" 127.0.0.1 - - [18/Aug/2015:19:08:46 +0200] "GET /api/histories/f2db41e1fa331b3e/contents HTTP/1.1" 200 - " http://127.0.0.1:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0" Thanks! -- Mikel Egaña Aranguren, Ph.D. http://mikeleganaaranguren.com

2 5

Get dataset/API ids for a dataset
by Asma Riyaz 28 Aug '15

28 Aug '15

Hello Galaxy-dev, I thank you so much for all the help you have given me. I have a question about data set ids in galaxy. As a background, I am running my own galaxy instance on a server. A pipeline implemented in the lab produces the following files in the history: 1) 2 BAM files 2) A JSON file My goal is to use this JSON file to pass the path/URL of bam files into a custom JS we wrote for visualization purpose. This JSON file contains among many other details the paths/URLs to the above bam files. I am using JSON filetypes to send data to the JS visualization within Galaxy. To do this, I have my own JS which loads a BAM file from URL provided into an IGV.js track. IGV.js, which is responsible for making the tracks, expects a valid URL which is updated in the JSON file in this manner: 1) Extract the API_key and history id from a loaded BAM file 2) Edit the JSON file to reflect the BAM file's dataset id to be something like this: { "CLL-HL_pilot.r1.fastq": { "DNA": "/datasets/36ddb788a0f14eb3/display?to_ext=bam", ... This works fine if I know the API Key for bam files. When a pipeline executes dataset ids are generated for each output. I want to access and include these ids in the JSON file and load the updated JSON file into the history with the bams. Is there a way to get the ids from the history in this manner? Thank you, Asma

2 12

login problems when using a proxy and serving at a subdirectory
by Bernau, Christoph 28 Aug '15

28 Aug '15

Dear all, I have a problem with configuring my galaxy instance with a proxy server when I try to serve at a subdirectory. I followed the description at https://wiki.galaxyproject.org/Admin/Config/nginxProxy The basic configuration works just fine. When I use the procedure described in the section "serving galaxy at a subdirectory", the proxying principally also works. However, when registered users try to log in to galaxy, galaxy will simply directs them to the starting page (which I think is what normally happens after logging in) without displaying an error message. Nonetheless, users do not have access to the functionalities which only logged in users can use. E.g. each time they try to use the "workflows"-button they are told to log in. Could you please provide any help on this issue? Cheers, Christoph

3 3

Rename output from a repeat
by Ulf Schaefer 27 Aug '15

27 Aug '15

Hi all We frequently use the syntax below to rename outputs of workflows that we run in batch. It is convenient to have sample names from fastqs carried over to sams, bams, vcfs, etc. #{input1 | basename}.bam This does not seem to be working for inputs that are in repeats, e.g. the VelvetOptimiser. Does anybody know if there is a syntax to make this work, maybe #{repeatname[0].input1 | basename}.bam ? Thanks a lot for your help Ulf ************************************************************************** The information contained in the EMail and any attachments is confidential and intended solely and for the attention and use of the named addressee(s). It may not be disclosed to any other person without the express authority of Public Health England, or the intended recipient, or both. If you are not the intended recipient, you must not disclose, copy, distribute or retain this message or any part of it. This footnote also confirms that this EMail has been swept for computer viruses by Symantec.Cloud, but please re-sweep any attachments before opening or saving. http://www.gov.uk/PHE **************************************************************************

3 2

samtools dependency changes ?
by Wolfgang Maier 27 Aug '15

27 Aug '15

Dear all, with older Galaxies (prior to latest_15.03 I think), you could satisfy Galaxy's samtools dependence for indexing bam files by having a samtools executable in tool-dependencies/samtools/default/bin (with the tool-dependencies directory declared as tool_dependency_dir in galaxy.ini of course). Now (checked with latest_15.03 and .05), this is not working any more! The executable will still be used during bam uploads, but not when a bam file gets created by a tool. The reason is that before the job runner (tested this with the local job runner only) used to build the dependency shell command for dependency 'samtools' before finishing a job, but now the job wrapper finish method fails because it naively expects to find samtools on $PATH. Best, Wolfgang

3 3

Galaxy and PBS/Torque Cluster
by Ryan Johnson 27 Aug '15

27 Aug '15

Hello All, We are using a Bio-Linux 8 Galaxy instance and are currently trying to integrate in into a PBS/Torque cluster. Currently we are able to run jobs locally but not via PBS. Can anyone provide us with a step by step guide to configure Galaxy and the PBS. I have included some of the errors we are receiving below: galaxy.datatypes.metadata DEBUG 2015-08-19 14:56:42,920 Failed to cleanup MetadataTempFile temp files from /usr/lib/galaxy-server/database/job_working_directory/000/96 /metadata_out_HistoryDatasetAssociation_100_Kk1ajw: No JSON object could be decoded galaxy.jobs.runners.pbs ERROR 2015-08-19 14:56:42,960 Connection to PBS server for submit failed: 0: no error We can not figure out what is stopping our jobs from entering the PBS queue. Below is our job_conf.xml. <?xml version="1.0"?> <job_conf> <plugins workers="2">   <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="2"/> <plugin id="pbs" type="runner" load="galaxy.jobs.runners.pbs:PBSJobRunner"/> </plugins> <handlers> <handler id="main"/>  </handlers> <destinations default="defaultq">  <destination id="defaultq" runner="pbs" tags="cluster"> <param id="-q">defaultq</param> <param id="-l">walltime=24:00:00</param> </destination> </destinations> </job_conf> Thank you very much for all the assistance! Regards, Ryan -- Ryan S. Johnson, PhD Applications Scientist Center for Advanced Research Computing P: (650) 430-6194 E:rjohns03@carc.unm.edu

2 1

Change genome build for multiple files
by Frederik Coppens 27 Aug '15

27 Aug '15

Hi is there a convenient way to change the genome build for a lot of files at once (e.g. a whole data library)? I want to ‘update’ to a newer genome build and the only way I found is to go through ‘Edit information’ for each file separately. I assume it can be changed directly in the db, but I’m not familiar with the scheme. thanks Frederik ================================================================== Frederik Coppens, Ph.D. Project Leader Applied Bioinformatics & Biostatistics IT manager Tel: 32 (0)9 3313812 Fax:32 (0)9 3313809 VIB DEPARTMENT OF PLANT SYSTEMS BIOLOGY, UGent Technologiepark 927, 9052 Gent, BELGIUM frcop(a)psb.ugent.be http://www.psb.ugent.be/abb ==================================================================

2 2