[galaxy-dev] Unable to run simple Docker tool

18 Aug 2015

      Hi;

I'm trying to develop a Docker based tool, as suggested by a peer-reviewer
who might be reading this :P

However, I'm having trouble with even the most basic setting, and I don't
know what might be wrong, so any help will be much appreciated. I have
developed a very simple docker image and corresponding Galaxy tool, so that
I get it working before starting with the actual tool, but when I execute
it through Galaxy it simply stays executing forever, instead of failing or
terminating.

My image simply executes a shell script that reads the content of a file
and concatenates a string to it. The image:

FROM busybox:ubuntu-14.04
MAINTAINER Mikel Egaña Aranguren <mikel.egana.aranguren@gmail.com>

RUN mkdir /sadi
COPY test-io.sh /sadi/
RUN chmod a+x /sadi/test-io.sh
ENV PATH $PATH:/sadi

The test-io.sh script within the image:

#!/bin/sh

cat $1
echo "AAA"

Invoking the container and executing the script through a normal shell
works fine:

REPOSITORY                                   TAG                 IMAGE
ID            CREATED             VIRTUAL SIZE
mikeleganaaranguren/busybox-galaxy-test-io   v1
9c2b8bdade1d        54 minutes ago      5.609 MB

docker run -i -t mikeleganaaranguren/busybox-galaxy-test-io:v1

BusyBox v1.21.1 (Ubuntu 1:1.21.0-1ubuntu1) built-in shell (ash)
Enter 'help' for a list of built-in commands.

~ #
~ # echo "BBB" > test
~ # test-io.sh test
BBB
AAA

This is the tool file I'm using in Galaxy:

<tool id="SADIBUSYBOX" name="SADIBUSYBOX">
    <description>IO</description>
    <requirements>
      <container
type="docker">mikeleganaaranguren/busybox-galaxy-test-io:v1</container>
    </requirements>
    <command>
      test-io.sh $input > $output
    </command>
    <inputs>
        <param name="input" type="data" label="Dataset"/>
    </inputs>
    <outputs>
        <data format="txt" name="output" />
    </outputs>
    <help>
    </help>
</tool>

And my job_conf.xml:

<?xml version="1.0"?>
<!-- A sample job config that explicitly configures job running the way it
is configured by default (if there is no explicit config). -->
<job_conf>
    <plugins>
        <plugin id="local" type="runner"
load="galaxy.jobs.runners.local:LocalJobRunner" workers="4"/>
    </plugins>
    <handlers>
        <handler id="main"/>
    </handlers>
    <destinations default="docker_local">
        <destination id="local" runner="local"/>
    <destination id="docker_local" runner="local">
      <param id="docker_enabled">true</param>
        </destination>
    </destinations>
</job_conf>

As I said, when I execute the tool in Galaxy, it simply executes forever,
it stays in a "yellow state", till I kill the Galaxy server. The log says:

127.0.0.1 - - [18/Aug/2015:19:08:00 +0200] "GET
/tool_runner?tool_id=SADIBUSYBOX HTTP/1.1" 200 - "http://127.0.0.1:8080/"
"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:40.0) Gecko/20100101
Firefox/40.0"
galaxy.tools.actions INFO 2015-08-18 19:08:07,178 Handled output (77.825 ms)
galaxy.tools.actions INFO 2015-08-18 19:08:07,280 Verified access to
datasets (12.387 ms)
galaxy.tools.execute DEBUG 2015-08-18 19:08:07,307 Tool [SADIBUSYBOX]
created job [2] (217.508 ms)
127.0.0.1 - - [18/Aug/2015:19:08:07 +0200] "POST /api/tools HTTP/1.1" 200 -
"http://127.0.0.1:8080/tool_runner?tool_id=SADIBUSYBOX" "Mozilla/5.0 (X11;
Ubuntu; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0"
127.0.0.1 - - [18/Aug/2015:19:08:07 +0200] "GET
/api/histories/f2db41e1fa331b3e/contents HTTP/1.1" 200 - "
http://127.0.0.1:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:40.0)
Gecko/20100101 Firefox/40.0"
galaxy.jobs DEBUG 2015-08-18 19:08:07,714 (2) Working directory for job is:
/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2
galaxy.jobs.handler DEBUG 2015-08-18 19:08:07,721 (2) Dispatching to local
runner
galaxy.jobs DEBUG 2015-08-18 19:08:07,839 (2) Persisting job destination
(destination id: docker_local)
galaxy.jobs.runners DEBUG 2015-08-18 19:08:07,849 Job [2] queued (128.176
ms)
galaxy.jobs.handler INFO 2015-08-18 19:08:07,897 (2) Job dispatched
galaxy.jobs.command_factory INFO 2015-08-18 19:08:08,163 Built script
[/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2/tool_script.sh]
for tool
command[/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2/tool_script.sh]
galaxy.jobs.runners DEBUG 2015-08-18 19:08:08,322 (2) command is: sudo
docker inspect mikeleganaaranguren/busybox-galaxy-test-io:v1 > /dev/null
2>&1
[ $? -ne 0 ] && sudo docker pull
mikeleganaaranguren/busybox-galaxy-test-io:v1 > /dev/null 2>&1

sudo docker run -e "GALAXY_SLOTS=$GALAXY_SLOTS" -v
/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy:/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy:ro
-v
/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/tools/catDocker:/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/tools/catDocker:ro
-v
/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2:/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2:rw
-v
/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/files:/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/files:rw
-w
/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2
--net none --rm -u 1001 mikeleganaaranguren/busybox-galaxy-test-io:v1
/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2/tool_script.sh;
return_code=$?; python
"/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2/set_metadata_3aWBf9.py"
"/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/tmp/tmpH2gy3X"
"/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2/galaxy.json"
"/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2/metadata_in_HistoryDatasetAssociation_2_WxbKNP,/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2/metadata_kwds_HistoryDatasetAssociation_2_p8UK91,/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2/metadata_out_HistoryDatasetAssociation_2_wPm09U,/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2/metadata_results_HistoryDatasetAssociation_2_irSo7t,/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/files/000/dataset_2.dat,/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2/metadata_override_HistoryDatasetAssociation_2_1glVfb"
5242880; sh -c "exit $return_code"
galaxy.jobs.runners.local DEBUG 2015-08-18 19:08:08,323 (2) executing job
script:
/home/mikel/UPV-EHU/SADI-Docker-Galaxy/galaxy/database/job_working_directory/000/2/galaxy_2.sh
galaxy.jobs DEBUG 2015-08-18 19:08:08,375 (2) Persisting job destination
(destination id: docker_local)
127.0.0.1 - - [18/Aug/2015:19:08:11 +0200] "GET
/api/histories/f2db41e1fa331b3e/contents HTTP/1.1" 200 - "
http://127.0.0.1:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:40.0)
Gecko/20100101 Firefox/40.0"
127.0.0.1 - - [18/Aug/2015:19:08:21 +0200] "GET
/history/current_history_json HTTP/1.1" 200 - "http://127.0.0.1:8080/"
"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:40.0) Gecko/20100101
Firefox/40.0"
127.0.0.1 - - [18/Aug/2015:19:08:21 +0200] "GET
/api/histories/f2db41e1fa331b3e/contents?dataset_details=ebfb8f50c6abde6d%2C33b43b4e7093c91f%2Ca799d38679e985db%2C5969b1f7201f12ae%2Cdf7a1f0c02a5b08e%2C0a248a1f62a0cc04%2C03501d7626bd192f%2C3f5830403180d620%2Ce85a3be143d5905b%2Cc9468fdb6dc5c5f1%2C2a56795cad3c7db3
HTTP/1.1" 200 - "http://127.0.0.1:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux
x86_64; rv:40.0) Gecko/20100101 Firefox/40.0"
127.0.0.1 - - [18/Aug/2015:19:08:25 +0200] "GET
/api/histories/f2db41e1fa331b3e/contents HTTP/1.1" 200 - "
http://127.0.0.1:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:40.0)
Gecko/20100101 Firefox/40.0"
127.0.0.1 - - [18/Aug/2015:19:08:29 +0200] "GET
/api/histories/f2db41e1fa331b3e/contents HTTP/1.1" 200 - "
http://127.0.0.1:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:40.0)
Gecko/20100101 Firefox/40.0"
127.0.0.1 - - [18/Aug/2015:19:08:33 +0200] "GET
/api/histories/f2db41e1fa331b3e/contents HTTP/1.1" 200 - "
http://127.0.0.1:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:40.0)
Gecko/20100101 Firefox/40.0"
127.0.0.1 - - [18/Aug/2015:19:08:38 +0200] "GET
/api/histories/f2db41e1fa331b3e/contents HTTP/1.1" 200 - "
http://127.0.0.1:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:40.0)
Gecko/20100101 Firefox/40.0"
127.0.0.1 - - [18/Aug/2015:19:08:42 +0200] "GET
/api/histories/f2db41e1fa331b3e/contents HTTP/1.1" 200 - "
http://127.0.0.1:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:40.0)
Gecko/20100101 Firefox/40.0"
127.0.0.1 - - [18/Aug/2015:19:08:46 +0200] "GET
/api/histories/f2db41e1fa331b3e/contents HTTP/1.1" 200 - "
http://127.0.0.1:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:40.0)
Gecko/20100101 Firefox/40.0"

Thanks!

-- 
Mikel Egaña Aranguren, Ph.D.

http://mikeleganaaranguren.com