Planemo inside of Conda
Hi Galaxy-Devs I would like to have a single python / planemo environment that is portable across computers where I can use planemo test (which instantiates an instance of Galaxy). Currently, we're using conda on our current network. That way we have shared environments where the version of python and associated modules is consistent across network computers/users. This way, any user (including the galaxy system) can use exactly the same python version and modules which greatly simplifies management. I guess it's possible to manage dependencies through toolsheds, but we didn't want to do that. Although probably not optimal, what we do is to start a conda environment between the <command> tags in a tool's xml for python scripts, e.g.: <command> source activate pythonEnv; python Script.py $input1 > $output1 </command> The problem is that planemo is also installed in this environment, and since one cannot start a galaxy instance within a conda environment, planemo test now fails (since the environment doesn't get loaded correctly). I can get this to work in planemo 0.14, but not in the newer versions. Is there a workaround for this, a better setup, or must I stay with planemo 0.14? Thanks!
Hi, I'm a little bit confused are you using a recent version of Galaxy with conda build-in support and this is not working? Or do you run a Galaxy without conda support and want to set it up? Galaxy can handle conda dependencies for you and tools automatically. Essentially, Galaxy can be configured in galaxy.ini to install it's own miniconda and resolve dependencies on it's own. Hope this helps, Bjoern Am 03.05.2016 um 23:05 schrieb D K:
Hi Galaxy-Devs
I would like to have a single python / planemo environment that is portable across computers where I can use planemo test (which instantiates an instance of Galaxy).
Currently, we're using conda on our current network. That way we have shared environments where the version of python and associated modules is consistent across network computers/users. This way, any user (including the galaxy system) can use exactly the same python version and modules which greatly simplifies management. I guess it's possible to manage dependencies through toolsheds, but we didn't want to do that.
Although probably not optimal, what we do is to start a conda environment between the <command> tags in a tool's xml for python scripts, e.g.:
<command> source activate pythonEnv; python Script.py $input1 > $output1 </command>
The problem is that planemo is also installed in this environment, and since one cannot start a galaxy instance within a conda environment, planemo test now fails (since the environment doesn't get loaded correctly). I can get this to work in planemo 0.14, but not in the newer versions. Is there a workaround for this, a better setup, or must I stay with planemo 0.14?
Thanks!
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Galaxy and Planemo cannot co-exist in the same conda environment I don't think - because planemo requires galaxy-lib (https://github.com/galaxyproject/galaxy-lib) which cannot be installed in Galaxy's environment. I don't understand the intricacies of what you are trying to do or why it worked in the past but not now - there were changes to the way planemo creates virtual environments for Galaxy that may interfere with this setup but I don't know how to disentangle that. If you can figure out exactly what the new planemo is doing wrong (creating a virtualenv for Galaxy maybe?) - we can add an option to exclude that behavior? All though -- I'm with Björn - Galaxy can now use installed conda environments to resolve requirements for tools. It should be able to target and leverage existing conda installations created manually as long as the names line up. This was even if you don't use the tool shed for installation of tools - you can use the tools in there without modification. It may require modifying the conda resolver to add the ability to do mapping for your specific setup but this is probably something it should do anyway. -John On Tue, May 3, 2016 at 5:14 PM, Björn Grüning <bjoern.gruening@gmail.com> wrote:
Hi,
I'm a little bit confused are you using a recent version of Galaxy with conda build-in support and this is not working? Or do you run a Galaxy without conda support and want to set it up?
Galaxy can handle conda dependencies for you and tools automatically. Essentially, Galaxy can be configured in galaxy.ini to install it's own miniconda and resolve dependencies on it's own.
Hope this helps, Bjoern
Am 03.05.2016 um 23:05 schrieb D K:
Hi Galaxy-Devs
I would like to have a single python / planemo environment that is portable across computers where I can use planemo test (which instantiates an instance of Galaxy).
Currently, we're using conda on our current network. That way we have shared environments where the version of python and associated modules is consistent across network computers/users. This way, any user (including the galaxy system) can use exactly the same python version and modules which greatly simplifies management. I guess it's possible to manage dependencies through toolsheds, but we didn't want to do that.
Although probably not optimal, what we do is to start a conda environment between the <command> tags in a tool's xml for python scripts, e.g.:
<command> source activate pythonEnv; python Script.py $input1 > $output1 </command>
The problem is that planemo is also installed in this environment, and since one cannot start a galaxy instance within a conda environment, planemo test now fails (since the environment doesn't get loaded correctly). I can get this to work in planemo 0.14, but not in the newer versions. Is there a workaround for this, a better setup, or must I stay with planemo 0.14?
Thanks!
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hi Bjorn and John, Thanks for the suggestion regarding the conda dependency resolver. I found this page: https://docs.galaxyproject.org/en/master/admin/dependency_resolvers.html Is there anywhere I can find some examples? I'm unsure how to implement this feature based on this document. Thanks! On Thu, May 5, 2016 at 7:12 AM, John Chilton <jmchilton@gmail.com> wrote:
Galaxy and Planemo cannot co-exist in the same conda environment I don't think - because planemo requires galaxy-lib (https://github.com/galaxyproject/galaxy-lib) which cannot be installed in Galaxy's environment.
I don't understand the intricacies of what you are trying to do or why it worked in the past but not now - there were changes to the way planemo creates virtual environments for Galaxy that may interfere with this setup but I don't know how to disentangle that. If you can figure out exactly what the new planemo is doing wrong (creating a virtualenv for Galaxy maybe?) - we can add an option to exclude that behavior?
All though -- I'm with Björn - Galaxy can now use installed conda environments to resolve requirements for tools. It should be able to target and leverage existing conda installations created manually as long as the names line up. This was even if you don't use the tool shed for installation of tools - you can use the tools in there without modification. It may require modifying the conda resolver to add the ability to do mapping for your specific setup but this is probably something it should do anyway.
-John
On Tue, May 3, 2016 at 5:14 PM, Björn Grüning <bjoern.gruening@gmail.com> wrote:
Hi,
I'm a little bit confused are you using a recent version of Galaxy with conda build-in support and this is not working? Or do you run a Galaxy without conda support and want to set it up?
Galaxy can handle conda dependencies for you and tools automatically. Essentially, Galaxy can be configured in galaxy.ini to install it's own miniconda and resolve dependencies on it's own.
Hope this helps, Bjoern
Am 03.05.2016 um 23:05 schrieb D K:
Hi Galaxy-Devs
I would like to have a single python / planemo environment that is portable across computers where I can use planemo test (which instantiates an instance of Galaxy).
Currently, we're using conda on our current network. That way we have shared environments where the version of python and associated modules is consistent across network computers/users. This way, any user (including the galaxy system) can use exactly the same python version and modules which greatly simplifies management. I guess it's possible to manage dependencies through toolsheds, but we didn't want to do that.
Although probably not optimal, what we do is to start a conda environment between the <command> tags in a tool's xml for python scripts, e.g.:
<command> source activate pythonEnv; python Script.py $input1 > $output1 </command>
The problem is that planemo is also installed in this environment, and since one cannot start a galaxy instance within a conda environment, planemo test now fails (since the environment doesn't get loaded correctly). I can get this to work in planemo 0.14, but not in the newer versions. Is there a workaround for this, a better setup, or must I stay with planemo 0.14?
Thanks!
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
participants (3)
-
Björn Grüning
-
D K
-
John Chilton