Galaxy, Docker Containers, and Installing tools not in Toolshed
Hello Galaxy Team, We're from the Bioinformatics Shared Resources at Cold Spring Harbor Lab and we're trying to create a new flavour of Galaxy for local use in the lab using docker containers. We want to add tools that are specific for our lab, such as importing files from our system, onto this flavour of Galaxy using Docker containers. Is there a recommended way to do this? I've taken a look at the README from: https://github.com/bgruening/docker-galaxy-stable I have been able to successfully create an instance of Galaxy where the homepage is different and tools are imported from the Tool Shed. However, I cannot figure out how to install tools onto Galaxy that are not from the Tool Shed. I have tried creating a container, putting the tools into the tools folder, changing the tool conf and the integrated tool panel xml file. Then committing the container and creating a new image. From the new image, I tried creating another container and running it. This does not create a Galaxy instance with the new tool. (This also does not seem like the best way to go about something like this.) Thank you for your help. Best, Miuki Yip
Hi Miuki! Am 19.07.2016 um 22:36 schrieb Yip, Miu ki:
Hello Galaxy Team,
We're from the Bioinformatics Shared Resources at Cold Spring Harbor Lab and we're trying to create a new flavour of Galaxy for local use in the lab using docker containers.
We want to add tools that are specific for our lab, such as importing files from our system, onto this flavour of Galaxy using Docker containers. Is there a recommended way to do this?
Can you try the following, assuming that all your tools are in my_tools_tolder. ADD ./my_tools_folder /inhouse_tools/ Now we need a tool_conf.xml file. You can create an additional one, no need to modify the existing one. I would propose to include this config file also in ./my_tools_folder so it will end up in /inhouse_tools/tool_conf.xml With the following environment variable you tell Galaxy about the additional tool_conf file. ENV GALAXY_CONFIG_TOOL_CONFIG_FILE = config/tool_conf.xml,config/shed_tool_conf.xml,/inhouse_tools/tool_conf.xml Please make sure that your new tools in /inhouse_tools/tool_conf.xml have as path "/inhouse_tools/XXXX.xml". Let me know if this works, we should add this to the readme. Thanks, Bjoern
I've taken a look at the README from: https://github.com/bgruening/docker-galaxy-stable
I have been able to successfully create an instance of Galaxy where the homepage is different and tools are imported from the Tool Shed. However, I cannot figure out how to install tools onto Galaxy that are not from the Tool Shed. I have tried creating a container, putting the tools into the tools folder, changing the tool conf and the integrated tool panel xml file. Then committing the container and creating a new image. From the new image, I tried creating another container and running it. This does not create a Galaxy instance with the new tool. (This also does not seem like the best way to go about something like this.)
Thank you for your help.
Best, Miuki Yip ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hello Bjoern, Thanks for the quick response, I appreciate that. Concerning the instructions, I am unclear of a couple of things. I have a folder with my tools, there is no preinstalled folder that these tools need to go in called my_tools_folder, right? Additionally, would I place this tool in the same directory level as Galaxy_Central or as the config directory? (My current set up is galaxy_storage -> galaxy-central -> config, I have my tools folder in the same level as galaxy-central.) Lastly, the environment variable, is there a specific file that all the environment variables are stored for docker Galaxy? I have tried looking for an env file, but could not find it. Thanks so much! Miuki ________________________________________ From: Björn Grüning [bjoern.gruening@gmail.com] Sent: Wednesday, July 20, 2016 2:23 AM To: Yip, Miu ki; galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Galaxy, Docker Containers, and Installing tools not in Toolshed Hi Miuki! Am 19.07.2016 um 22:36 schrieb Yip, Miu ki:
Hello Galaxy Team,
We're from the Bioinformatics Shared Resources at Cold Spring Harbor Lab and we're trying to create a new flavour of Galaxy for local use in the lab using docker containers.
We want to add tools that are specific for our lab, such as importing files from our system, onto this flavour of Galaxy using Docker containers. Is there a recommended way to do this?
Can you try the following, assuming that all your tools are in my_tools_tolder. ADD ./my_tools_folder /inhouse_tools/ Now we need a tool_conf.xml file. You can create an additional one, no need to modify the existing one. I would propose to include this config file also in ./my_tools_folder so it will end up in /inhouse_tools/tool_conf.xml With the following environment variable you tell Galaxy about the additional tool_conf file. ENV GALAXY_CONFIG_TOOL_CONFIG_FILE = config/tool_conf.xml,config/shed_tool_conf.xml,/inhouse_tools/tool_conf.xml Please make sure that your new tools in /inhouse_tools/tool_conf.xml have as path "/inhouse_tools/XXXX.xml". Let me know if this works, we should add this to the readme. Thanks, Bjoern
I've taken a look at the README from: https://github.com/bgruening/docker-galaxy-stable
I have been able to successfully create an instance of Galaxy where the homepage is different and tools are imported from the Tool Shed. However, I cannot figure out how to install tools onto Galaxy that are not from the Tool Shed. I have tried creating a container, putting the tools into the tools folder, changing the tool conf and the integrated tool panel xml file. Then committing the container and creating a new image. From the new image, I tried creating another container and running it. This does not create a Galaxy instance with the new tool. (This also does not seem like the best way to go about something like this.)
Thank you for your help.
Best, Miuki Yip ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hi,
Hello Bjoern,
Thanks for the quick response, I appreciate that.
Concerning the instructions, I am unclear of a couple of things.
I have a folder with my tools, there is no preinstalled folder that these tools need to go in called my_tools_folder, right?
No, you can create such a folder wherever you want and name it like you want :)
Additionally, would I place this tool in the same directory level as Galaxy_Central or as the config directory? (My current set up is galaxy_storage -> galaxy-central -> config, I have my tools folder in the same level as galaxy-central.)
This does not matter, what matters is that the folder path is correct in the env variable.
Lastly, the environment variable, is there a specific file that all the environment variables are stored for docker Galaxy? I have tried looking for an env file, but could not find it.
There is no file. These are real environment variables that are created by Docker during container start. You can change them during start with (e.g): docker run -e MY_ENV_VAR=foo bgruening/galaxy-stable If such a variable, beginning with GALAXY_CONFIG_XXX exists this will be picked up by Galaxy. Have a look at the container there are a lot GALAXY_CONFIG_ vars set. Hope this helps, Bjoern
Thanks so much! Miuki ________________________________________ From: Björn Grüning [bjoern.gruening@gmail.com] Sent: Wednesday, July 20, 2016 2:23 AM To: Yip, Miu ki; galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Galaxy, Docker Containers, and Installing tools not in Toolshed
Hi Miuki!
Am 19.07.2016 um 22:36 schrieb Yip, Miu ki:
Hello Galaxy Team,
We're from the Bioinformatics Shared Resources at Cold Spring Harbor Lab and we're trying to create a new flavour of Galaxy for local use in the lab using docker containers.
We want to add tools that are specific for our lab, such as importing files from our system, onto this flavour of Galaxy using Docker containers. Is there a recommended way to do this?
Can you try the following, assuming that all your tools are in my_tools_tolder.
ADD ./my_tools_folder /inhouse_tools/
Now we need a tool_conf.xml file. You can create an additional one, no need to modify the existing one. I would propose to include this config file also in ./my_tools_folder so it will end up in /inhouse_tools/tool_conf.xml With the following environment variable you tell Galaxy about the additional tool_conf file.
ENV GALAXY_CONFIG_TOOL_CONFIG_FILE = config/tool_conf.xml,config/shed_tool_conf.xml,/inhouse_tools/tool_conf.xml
Please make sure that your new tools in /inhouse_tools/tool_conf.xml have as path "/inhouse_tools/XXXX.xml".
Let me know if this works, we should add this to the readme.
Thanks, Bjoern
I've taken a look at the README from: https://github.com/bgruening/docker-galaxy-stable
I have been able to successfully create an instance of Galaxy where the homepage is different and tools are imported from the Tool Shed. However, I cannot figure out how to install tools onto Galaxy that are not from the Tool Shed. I have tried creating a container, putting the tools into the tools folder, changing the tool conf and the integrated tool panel xml file. Then committing the container and creating a new image. From the new image, I tried creating another container and running it. This does not create a Galaxy instance with the new tool. (This also does not seem like the best way to go about something like this.)
Thank you for your help.
Best, Miuki Yip ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hi, I apologize for all the questions, but would all of this work be done within the container after it is created by the Docker run command or are you referring to the local out of container area? I am getting an error like: DEBUG:galaxy.app:python path is: lib/, ., , /galaxy-central/eggs/PasteDeploy-1.5.0-py2.7.egg, /galaxy-central/.venv/lib/python2.7, /galaxy-central/.venv/lib/python2.7/plat-x86_64-linux-gnu, /galaxy-central/.venv/lib/python2.7/lib-tk, /galaxy-central/.venv/lib/python2.7/lib-old, /galaxy-central/.venv/lib/python2.7/lib-dynload, /usr/lib/python2.7, /usr/lib/python2.7/plat-x86_64-linux-gnu, /usr/lib/python2.7/lib-tk, /galaxy-central/.venv/local/lib/python2.7/site-packages Traceback (most recent call last): File "lib/galaxy/webapps/galaxy/buildapp.py", line 63, in paste_app_factory app = galaxy.app.UniverseApplication( global_conf=global_conf, **kwargs ) File "lib/galaxy/app.py", line 44, in __init__ self.config.check() File "lib/galaxy/config.py", line 636, in check raise ConfigurationError("Tool config file not found: %s" % path ) ConfigurationError: Tool config file not found: /root/../home/user/galaxy_storage/galaxy-central/config/tool_conf.xml.main if I try to run when I am adding outside of the container, I've cat-ed the path and standard out displays the contents of the file. I am getting another error about files not being found also when I add to the container itself when I attach. Thanks so much, Miuki ________________________________________ From: Björn Grüning [bjoern.gruening@gmail.com] Sent: Wednesday, July 20, 2016 2:05 PM To: Yip, Miu ki; Björn Grüning; galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Galaxy, Docker Containers, and Installing tools not in Toolshed Hi,
Hello Bjoern,
Thanks for the quick response, I appreciate that.
Concerning the instructions, I am unclear of a couple of things.
I have a folder with my tools, there is no preinstalled folder that these tools need to go in called my_tools_folder, right?
No, you can create such a folder wherever you want and name it like you want :)
Additionally, would I place this tool in the same directory level as Galaxy_Central or as the config directory? (My current set up is galaxy_storage -> galaxy-central -> config, I have my tools folder in the same level as galaxy-central.)
This does not matter, what matters is that the folder path is correct in the env variable.
Lastly, the environment variable, is there a specific file that all the environment variables are stored for docker Galaxy? I have tried looking for an env file, but could not find it.
There is no file. These are real environment variables that are created by Docker during container start. You can change them during start with (e.g): docker run -e MY_ENV_VAR=foo bgruening/galaxy-stable If such a variable, beginning with GALAXY_CONFIG_XXX exists this will be picked up by Galaxy. Have a look at the container there are a lot GALAXY_CONFIG_ vars set. Hope this helps, Bjoern
Thanks so much! Miuki ________________________________________ From: Björn Grüning [bjoern.gruening@gmail.com] Sent: Wednesday, July 20, 2016 2:23 AM To: Yip, Miu ki; galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Galaxy, Docker Containers, and Installing tools not in Toolshed
Hi Miuki!
Am 19.07.2016 um 22:36 schrieb Yip, Miu ki:
Hello Galaxy Team,
We're from the Bioinformatics Shared Resources at Cold Spring Harbor Lab and we're trying to create a new flavour of Galaxy for local use in the lab using docker containers.
We want to add tools that are specific for our lab, such as importing files from our system, onto this flavour of Galaxy using Docker containers. Is there a recommended way to do this?
Can you try the following, assuming that all your tools are in my_tools_tolder.
ADD ./my_tools_folder /inhouse_tools/
Now we need a tool_conf.xml file. You can create an additional one, no need to modify the existing one. I would propose to include this config file also in ./my_tools_folder so it will end up in /inhouse_tools/tool_conf.xml With the following environment variable you tell Galaxy about the additional tool_conf file.
ENV GALAXY_CONFIG_TOOL_CONFIG_FILE = config/tool_conf.xml,config/shed_tool_conf.xml,/inhouse_tools/tool_conf.xml
Please make sure that your new tools in /inhouse_tools/tool_conf.xml have as path "/inhouse_tools/XXXX.xml".
Let me know if this works, we should add this to the readme.
Thanks, Bjoern
I've taken a look at the README from: https://github.com/bgruening/docker-galaxy-stable
I have been able to successfully create an instance of Galaxy where the homepage is different and tools are imported from the Tool Shed. However, I cannot figure out how to install tools onto Galaxy that are not from the Tool Shed. I have tried creating a container, putting the tools into the tools folder, changing the tool conf and the integrated tool panel xml file. Then committing the container and creating a new image. From the new image, I tried creating another container and running it. This does not create a Galaxy instance with the new tool. (This also does not seem like the best way to go about something like this.)
Thank you for your help.
Best, Miuki Yip ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hi, all this can be done in your Dockerfile if you create a Galaxy flavour or during startup. For example you can start your container like this docker run -v /home/miuki/tools/:/tools/ -e GALAXY_CONFIG_TOOL_CONFIG_FILE = config/tool_conf.xml,config/shed_tool_conf.xml,/tools/tool_conf.xml bgruening/galaxy-stable This should work if inside of /home/miuki/tools/tool_conf.xml all tools have the path /tools/my_tool.xml Ciao, Bjoern
Hi,
I apologize for all the questions, but would all of this work be done within the container after it is created by the Docker run command or are you referring to the local out of container area?
I am getting an error like: DEBUG:galaxy.app:python path is: lib/, ., , /galaxy-central/eggs/PasteDeploy-1.5.0-py2.7.egg, /galaxy-central/.venv/lib/python2.7, /galaxy-central/.venv/lib/python2.7/plat-x86_64-linux-gnu, /galaxy-central/.venv/lib/python2.7/lib-tk, /galaxy-central/.venv/lib/python2.7/lib-old, /galaxy-central/.venv/lib/python2.7/lib-dynload, /usr/lib/python2.7, /usr/lib/python2.7/plat-x86_64-linux-gnu, /usr/lib/python2.7/lib-tk, /galaxy-central/.venv/local/lib/python2.7/site-packages Traceback (most recent call last): File "lib/galaxy/webapps/galaxy/buildapp.py", line 63, in paste_app_factory app = galaxy.app.UniverseApplication( global_conf=global_conf, **kwargs ) File "lib/galaxy/app.py", line 44, in __init__ self.config.check() File "lib/galaxy/config.py", line 636, in check raise ConfigurationError("Tool config file not found: %s" % path ) ConfigurationError: Tool config file not found: /root/../home/user/galaxy_storage/galaxy-central/config/tool_conf.xml.main
if I try to run when I am adding outside of the container, I've cat-ed the path and standard out displays the contents of the file. I am getting another error about files not being found also when I add to the container itself when I attach.
Thanks so much, Miuki ________________________________________ From: Björn Grüning [bjoern.gruening@gmail.com] Sent: Wednesday, July 20, 2016 2:05 PM To: Yip, Miu ki; Björn Grüning; galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Galaxy, Docker Containers, and Installing tools not in Toolshed
Hi,
Hello Bjoern,
Thanks for the quick response, I appreciate that.
Concerning the instructions, I am unclear of a couple of things.
I have a folder with my tools, there is no preinstalled folder that these tools need to go in called my_tools_folder, right? No, you can create such a folder wherever you want and name it like you want :)
Additionally, would I place this tool in the same directory level as Galaxy_Central or as the config directory? (My current set up is galaxy_storage -> galaxy-central -> config, I have my tools folder in the same level as galaxy-central.) This does not matter, what matters is that the folder path is correct in the env variable.
Lastly, the environment variable, is there a specific file that all the environment variables are stored for docker Galaxy? I have tried looking for an env file, but could not find it. There is no file. These are real environment variables that are created by Docker during container start. You can change them during start with (e.g): docker run -e MY_ENV_VAR=foo bgruening/galaxy-stable
If such a variable, beginning with GALAXY_CONFIG_XXX exists this will be picked up by Galaxy. Have a look at the container there are a lot GALAXY_CONFIG_ vars set.
Hope this helps, Bjoern
Thanks so much! Miuki ________________________________________ From: Björn Grüning [bjoern.gruening@gmail.com] Sent: Wednesday, July 20, 2016 2:23 AM To: Yip, Miu ki; galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Galaxy, Docker Containers, and Installing tools not in Toolshed
Hi Miuki!
Am 19.07.2016 um 22:36 schrieb Yip, Miu ki:
Hello Galaxy Team,
We're from the Bioinformatics Shared Resources at Cold Spring Harbor Lab and we're trying to create a new flavour of Galaxy for local use in the lab using docker containers.
We want to add tools that are specific for our lab, such as importing files from our system, onto this flavour of Galaxy using Docker containers. Is there a recommended way to do this? Can you try the following, assuming that all your tools are in my_tools_tolder.
ADD ./my_tools_folder /inhouse_tools/
Now we need a tool_conf.xml file. You can create an additional one, no need to modify the existing one. I would propose to include this config file also in ./my_tools_folder so it will end up in /inhouse_tools/tool_conf.xml With the following environment variable you tell Galaxy about the additional tool_conf file.
ENV GALAXY_CONFIG_TOOL_CONFIG_FILE = config/tool_conf.xml,config/shed_tool_conf.xml,/inhouse_tools/tool_conf.xml
Please make sure that your new tools in /inhouse_tools/tool_conf.xml have as path "/inhouse_tools/XXXX.xml".
Let me know if this works, we should add this to the readme.
Thanks, Bjoern
I've taken a look at the README from: https://github.com/bgruening/docker-galaxy-stable
I have been able to successfully create an instance of Galaxy where the homepage is different and tools are imported from the Tool Shed. However, I cannot figure out how to install tools onto Galaxy that are not from the Tool Shed. I have tried creating a container, putting the tools into the tools folder, changing the tool conf and the integrated tool panel xml file. Then committing the container and creating a new image. From the new image, I tried creating another container and running it. This does not create a Galaxy instance with the new tool. (This also does not seem like the best way to go about something like this.)
Thank you for your help.
Best, Miuki Yip ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
participants (3)
-
Bjoern Gruening -
Björn Grüning
-
Yip, Miu ki