more control over the "docker" command
Hi, Thanks for supporting the running of docker containers in Galaxy. I have two requests for more control over the docker command that is run. According to https://github.com/apetkau/galaxy-hackathon-2014 , the docker command that is run when a docker-enabled tool is run might look something like this: command is: sudo docker run -e "GALAXY_SLOTS=$GALAXY_SLOTS" -v /home/aaron/Projects/galaxy-central:/home/aaron/Proje cts/galaxy-central:ro -v /home/aaron/Projects/galaxy-central/tools/docker:/home/aaron/Projects/galaxy-central/tools/docker:ro -v /home/aaron/Projects/galaxy-central/datab ase/job_working_directory/000/6:/home/aaron/Projects/galaxy-central/database/job_working_directory/000/6:rw -v /home/aaron/Projects/galaxy-central/database/files:/home/aa ron/Projects/galaxy-central/database/files:rw -w /home/aaron/Projects/galaxy-central/database/job_working_directory/000/6 --net none busybox:ubuntu-14.04 /home/aaron/Projects/galaxy-central/database/job_working_directory/000/6/container.sh; return_code=$?; if [ -f /home/aaron/Projects/galaxy-central/database/job_working_directory/000/6/wo rking_file ] ; then cp /home/aaron/Projects/galaxy-central/database/job_working_directory/000/6/working_file /home/aaron/Projects/galaxy-central/database/files/000/dataset_10.dat ; fi; sh -c "exit $return_code" I'd like to be able to specify extra flags to be included in the command. In my case I'd like to include "--link server:server" because I want to link this container with another container that contains a long-running server process (I can describe my use case in greater detail if desired). Can there be a way to do this in my tool wrapper? Secondly, rather than giving my galaxy user passwordless sudo, I'd like to add it to the docker group, then I can run docker commands without prepending sudo. This seems a lot safer. Could this be exposed, maybe by a sudo="false" attribute in the tool wrapper? Thanks, Dan
Hi Dan, I don't necessarily agree with adding the galaxy users to the docker group as a "safer" alternative to sudo. Note that the sudoers configuration files can limit the commands that get executed and you don't have to give "all sudo privileges", which is the easiest thing to do. Sudoers supports command matching/globing to permit more specific permissions such as giving the ability to add docker images but not removing them. See http://ubuntuforums.org/showthread.php?t=1132821 for extensive information on how you would limit sudo. Iyad Kandalaft Agriculture and Agri-Food Canada -----Original Message----- From: galaxy-dev [mailto:galaxy-dev-bounces@lists.galaxyproject.org] On Behalf Of Dan Tenenbaum Sent: Monday, November 24, 2014 10:55 AM To: galaxy-dev@lists.galaxyproject.org Subject: [galaxy-dev] more control over the "docker" command Hi, Thanks for supporting the running of docker containers in Galaxy. I have two requests for more control over the docker command that is run. According to https://github.com/apetkau/galaxy-hackathon-2014 , the docker command that is run when a docker-enabled tool is run might look something like this: command is: sudo docker run -e "GALAXY_SLOTS=$GALAXY_SLOTS" -v /home/aaron/Projects/galaxy-central:/home/aaron/Proje cts/galaxy-central:ro -v /home/aaron/Projects/galaxy-central/tools/docker:/home/aaron/Projects/galaxy-central/tools/docker:ro -v /home/aaron/Projects/galaxy-central/datab ase/job_working_directory/000/6:/home/aaron/Projects/galaxy-central/database/job_working_directory/000/6:rw -v /home/aaron/Projects/galaxy-central/database/files:/home/aa ron/Projects/galaxy-central/database/files:rw -w /home/aaron/Projects/galaxy-central/database/job_working_directory/000/6 --net none busybox:ubuntu-14.04 /home/aaron/Projects/galaxy-central/database/job_working_directory/000/6/container.sh; return_code=$?; if [ -f /home/aaron/Projects/galaxy-central/database/job_working_directory/000/6/wo rking_file ] ; then cp /home/aaron/Projects/galaxy-central/database/job_working_directory/000/6/working_file /home/aaron/Projects/galaxy-central/database/files/000/dataset_10.dat ; fi; sh -c "exit $return_code" I'd like to be able to specify extra flags to be included in the command. In my case I'd like to include "--link server:server" because I want to link this container with another container that contains a long-running server process (I can describe my use case in greater detail if desired). Can there be a way to do this in my tool wrapper? Secondly, rather than giving my galaxy user passwordless sudo, I'd like to add it to the docker group, then I can run docker commands without prepending sudo. This seems a lot safer. Could this be exposed, maybe by a sudo="false" attribute in the tool wrapper? Thanks, Dan ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
On Mon, Nov 24, 2014 at 10:54 AM, Dan Tenenbaum <dtenenba@fredhutch.org> wrote:
Hi,
Thanks for supporting the running of docker containers in Galaxy.
I have two requests for more control over the docker command that is run.
According to https://github.com/apetkau/galaxy-hackathon-2014 , the docker command that is run when a docker-enabled tool is run might look something like this:
command is: sudo docker run -e "GALAXY_SLOTS=$GALAXY_SLOTS" -v /home/aaron/Projects/galaxy-central:/home/aaron/Proje cts/galaxy-central:ro -v /home/aaron/Projects/galaxy-central/tools/docker:/home/aaron/Projects/galaxy-central/tools/docker:ro -v /home/aaron/Projects/galaxy-central/datab ase/job_working_directory/000/6:/home/aaron/Projects/galaxy-central/database/job_working_directory/000/6:rw -v /home/aaron/Projects/galaxy-central/database/files:/home/aa ron/Projects/galaxy-central/database/files:rw -w /home/aaron/Projects/galaxy-central/database/job_working_directory/000/6 --net none busybox:ubuntu-14.04 /home/aaron/Projects/galaxy-central/database/job_working_directory/000/6/container.sh; return_code=$?; if [ -f /home/aaron/Projects/galaxy-central/database/job_working_directory/000/6/wo rking_file ] ; then cp /home/aaron/Projects/galaxy-central/database/job_working_directory/000/6/working_file /home/aaron/Projects/galaxy-central/database/files/000/dataset_10.dat ; fi; sh -c "exit $return_code"
I'd like to be able to specify extra flags to be included in the command. In my case I'd like to include "--link server:server" because I want to link this container with another container that contains a long-running server process (I can describe my use case in greater detail if desired). Can there be a way to do this in my tool wrapper?
I certainly don't mind adding more options - we've added a bunch as different interesting things have been attempted (e.g. docker in docker). I would like to keep the tool wrapper very general though and expose these options in the job_conf.xml instead - since for the most part they seem like they will be related to the tool's deployment and not the abstract command being produced. I have added this functionality with the following changeset - https://bitbucket.org/galaxy/galaxy-central/commits/ebceda0721f3f3215447c19f.... Simply add <param id="docker_run_extra_arguments">--link server:server</param> to your job destination. Let me know if this doesn't work out.
Secondly, rather than giving my galaxy user passwordless sudo, I'd like to add it to the docker group, then I can run docker commands without prepending sudo. This seems a lot safer. Could this be exposed, maybe by a sudo="false" attribute in the tool wrapper?
This is currently possible - in your job_conf.xml just add to the job destination parameter: <param id="docker_sudo">false</param> Checkout the admittedly huge file config/job_conf.xml.sample_advanced for a full list of the supported options. The docker related options are all near each other and apply to any built-in job runner. I suspect tot running with sudo really should be the default and I am growing frustrated it is not. I am unsure if I should "fix" it now and break the couple sites doing Docker in Galaxy or just leave it and have everyone who wants to turn sudo off disable it explicitly. At any rate - keep us informed about how things are going - I would love to hear about what you are attempting to do with the linked persistent docker container :). -John
Thanks, Dan
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
----- Original Message -----
From: "John Chilton" <jmchilton@gmail.com> To: "Dan Tenenbaum" <dtenenba@fredhutch.org> Cc: galaxy-dev@lists.galaxyproject.org Sent: Monday, November 24, 2014 9:30:50 AM Subject: Re: [galaxy-dev] more control over the "docker" command
On Mon, Nov 24, 2014 at 10:54 AM, Dan Tenenbaum <dtenenba@fredhutch.org> wrote:
Hi,
Thanks for supporting the running of docker containers in Galaxy.
I have two requests for more control over the docker command that is run.
According to https://github.com/apetkau/galaxy-hackathon-2014 , the docker command that is run when a docker-enabled tool is run might look something like this:
command is: sudo docker run -e "GALAXY_SLOTS=$GALAXY_SLOTS" -v /home/aaron/Projects/galaxy-central:/home/aaron/Proje cts/galaxy-central:ro -v /home/aaron/Projects/galaxy-central/tools/docker:/home/aaron/Projects/galaxy-central/tools/docker:ro -v /home/aaron/Projects/galaxy-central/datab ase/job_working_directory/000/6:/home/aaron/Projects/galaxy-central/database/job_working_directory/000/6:rw -v /home/aaron/Projects/galaxy-central/database/files:/home/aa ron/Projects/galaxy-central/database/files:rw -w /home/aaron/Projects/galaxy-central/database/job_working_directory/000/6 --net none busybox:ubuntu-14.04 /home/aaron/Projects/galaxy-central/database/job_working_directory/000/6/container.sh; return_code=$?; if [ -f /home/aaron/Projects/galaxy-central/database/job_working_directory/000/6/wo rking_file ] ; then cp /home/aaron/Projects/galaxy-central/database/job_working_directory/000/6/working_file /home/aaron/Projects/galaxy-central/database/files/000/dataset_10.dat ; fi; sh -c "exit $return_code"
I'd like to be able to specify extra flags to be included in the command. In my case I'd like to include "--link server:server" because I want to link this container with another container that contains a long-running server process (I can describe my use case in greater detail if desired). Can there be a way to do this in my tool wrapper?
I certainly don't mind adding more options - we've added a bunch as different interesting things have been attempted (e.g. docker in docker). I would like to keep the tool wrapper very general though and expose these options in the job_conf.xml instead - since for the most part they seem like they will be related to the tool's deployment and not the abstract command being produced. I have added this functionality with the following changeset - https://bitbucket.org/galaxy/galaxy-central/commits/ebceda0721f3f3215447c19f.... Simply add <param id="docker_run_extra_arguments">--link server:server</param> to your job destination. Let me know if this doesn't work out.
Thanks very much for the quick turnaround! I think it will work out. I am not an expert in job_conf.xml. My main concern is being able to pass extra arguments to some docker-enabled tools but not to others, is that possible?
Secondly, rather than giving my galaxy user passwordless sudo, I'd like to add it to the docker group, then I can run docker commands without prepending sudo. This seems a lot safer. Could this be exposed, maybe by a sudo="false" attribute in the tool wrapper?
This is currently possible - in your job_conf.xml just add to the job destination parameter:
<param id="docker_sudo">false</param>
Checkout the admittedly huge file config/job_conf.xml.sample_advanced for a full list of the supported options. The docker related options are all near each other and apply to any built-in job runner.
I suspect tot running with sudo really should be the default and I am growing frustrated it is not. I am unsure if I should "fix" it now and break the couple sites doing Docker in Galaxy or just leave it and have everyone who wants to turn sudo off disable it explicitly.
Great, thanks. Actually, as was pointed out here (and as I realize now), adding the galaxy user to the docker group is not necessarily safer (though it superficially looks like it because the word 'sudo' disappears). So I think perhaps docker_sudo should remain true by default.
At any rate - keep us informed about how things are going - I would
I will let you know how it goes.
love to hear about what you are attempting to do with the linked persistent docker container :).
Briefly, our tool is an R script that loads several R packages and then performs a calculation based on user input. The problem is that it can take several minutes just to load the packages required to perform the calculation, but the calculation itself only takes a couple of seconds. Fast performance is important to us, so we set up an instance of R that already has the packages loaded and we "serve" it using the Rserve package. So now the galaxy tool just has to send an expression to Rserve which can evaluate it quickly and reduce the tool running time from minutes to seconds. We are already using this setup in a non-docker context and your change now (I hope) enables us to do this with Docker. Dan
-John
Thanks, Dan
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
On Mon, Nov 24, 2014 at 12:56 PM, Dan Tenenbaum <dtenenba@fredhutch.org> wrote:
----- Original Message -----
From: "John Chilton" <jmchilton@gmail.com> To: "Dan Tenenbaum" <dtenenba@fredhutch.org> Cc: galaxy-dev@lists.galaxyproject.org Sent: Monday, November 24, 2014 9:30:50 AM Subject: Re: [galaxy-dev] more control over the "docker" command
On Mon, Nov 24, 2014 at 10:54 AM, Dan Tenenbaum <dtenenba@fredhutch.org> wrote:
Hi,
Thanks for supporting the running of docker containers in Galaxy.
I have two requests for more control over the docker command that is run.
According to https://github.com/apetkau/galaxy-hackathon-2014 , the docker command that is run when a docker-enabled tool is run might look something like this:
command is: sudo docker run -e "GALAXY_SLOTS=$GALAXY_SLOTS" -v /home/aaron/Projects/galaxy-central:/home/aaron/Proje cts/galaxy-central:ro -v /home/aaron/Projects/galaxy-central/tools/docker:/home/aaron/Projects/galaxy-central/tools/docker:ro -v /home/aaron/Projects/galaxy-central/datab ase/job_working_directory/000/6:/home/aaron/Projects/galaxy-central/database/job_working_directory/000/6:rw -v /home/aaron/Projects/galaxy-central/database/files:/home/aa ron/Projects/galaxy-central/database/files:rw -w /home/aaron/Projects/galaxy-central/database/job_working_directory/000/6 --net none busybox:ubuntu-14.04 /home/aaron/Projects/galaxy-central/database/job_working_directory/000/6/container.sh; return_code=$?; if [ -f /home/aaron/Projects/galaxy-central/database/job_working_directory/000/6/wo rking_file ] ; then cp /home/aaron/Projects/galaxy-central/database/job_working_directory/000/6/working_file /home/aaron/Projects/galaxy-central/database/files/000/dataset_10.dat ; fi; sh -c "exit $return_code"
I'd like to be able to specify extra flags to be included in the command. In my case I'd like to include "--link server:server" because I want to link this container with another container that contains a long-running server process (I can describe my use case in greater detail if desired). Can there be a way to do this in my tool wrapper?
I certainly don't mind adding more options - we've added a bunch as different interesting things have been attempted (e.g. docker in docker). I would like to keep the tool wrapper very general though and expose these options in the job_conf.xml instead - since for the most part they seem like they will be related to the tool's deployment and not the abstract command being produced. I have added this functionality with the following changeset - https://bitbucket.org/galaxy/galaxy-central/commits/ebceda0721f3f3215447c19f.... Simply add <param id="docker_run_extra_arguments">--link server:server</param> to your job destination. Let me know if this doesn't work out.
Thanks very much for the quick turnaround!
I think it will work out. I am not an expert in job_conf.xml. My main concern is being able to pass extra arguments to some docker-enabled tools but not to others, is that possible?
Many job destinations can map to the same underlying compute resource (cluster, server, etc...) - so I would just duplicate your job destination definition and customize each one for the different tool and then your tool section - make sure a given tool maps to the correct destination. There will be some unfortunate duplication - but it shouldn't be too bad unless you are doing a bunch of very cool things with dozens of different tools. (https://trello.com/c/wt1LflNh).
Secondly, rather than giving my galaxy user passwordless sudo, I'd like to add it to the docker group, then I can run docker commands without prepending sudo. This seems a lot safer. Could this be exposed, maybe by a sudo="false" attribute in the tool wrapper?
This is currently possible - in your job_conf.xml just add to the job destination parameter:
<param id="docker_sudo">false</param>
Checkout the admittedly huge file config/job_conf.xml.sample_advanced for a full list of the supported options. The docker related options are all near each other and apply to any built-in job runner.
I suspect tot running with sudo really should be the default and I am growing frustrated it is not. I am unsure if I should "fix" it now and break the couple sites doing Docker in Galaxy or just leave it and have everyone who wants to turn sudo off disable it explicitly.
Great, thanks. Actually, as was pointed out here (and as I realize now), adding the galaxy user to the docker group is not necessarily safer (though it superficially looks like it because the word 'sudo' disappears). So I think perhaps docker_sudo should remain true by default.
At any rate - keep us informed about how things are going - I would
I will let you know how it goes.
love to hear about what you are attempting to do with the linked persistent docker container :).
Briefly, our tool is an R script that loads several R packages and then performs a calculation based on user input. The problem is that it can take several minutes just to load the packages required to perform the calculation, but the calculation itself only takes a couple of seconds. Fast performance is important to us, so we set up an instance of R that already has the packages loaded and we "serve" it using the Rserve package. So now the galaxy tool just has to send an expression to Rserve which can evaluate it quickly and reduce the tool running time from minutes to seconds. We are already using this setup in a non-docker context and your change now (I hope) enables us to do this with Docker.
Very cool, good luck! -John
Dan
-John
Thanks, Dan
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
participants (3)
-
Dan Tenenbaum
-
John Chilton
-
Kandalaft, Iyad