Tools declaring Python dependencies
Hi all, Galaxy has long had two very simple (but largely unused) dependency mechanisms in a tool's XML (pre-dating the tool shed), for binaries and Python modules, e.g. <requirements> <requirement type="python-module">numpy</requirement> <requirement type="binary">taxBuilder</requirement> </requirements> http://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax#A.3Crequirement.3... This has since been supplemented by the Tool Shed centric package requirement mechanism. How can a tool declare a Python dependency, and have it installed automatically via the Tool Shed? Presumably it would have to end up as an egg in the Galaxy eggs folder, and not be installed under the default Python's PYTHONPATH? There is nothing about Python modules mentioned here: http://wiki.galaxyproject.org/ToolShedToolFeatures What I would suggest is that the the existing <requirement type="python-module"> is compared to the curated modules on http://eggs.g2.bx.psu.edu/ and if available the relevant egg be download from there. In which was, I have a request for additional eggs to be hosted there ;) Regards, Peter
On Tue, Apr 16, 2013 at 5:23 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
Hi all,
Galaxy has long had two very simple (but largely unused) dependency mechanisms in a tool's XML (pre-dating the tool shed), for binaries and Python modules, e.g.
<requirements> <requirement type="python-module">numpy</requirement> <requirement type="binary">taxBuilder</requirement> </requirements>
http://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax#A.3Crequirement.3...
This has since been supplemented by the Tool Shed centric package requirement mechanism.
How can a tool declare a Python dependency, and have it installed automatically via the Tool Shed? Presumably it would have to end up as an egg in the Galaxy eggs folder, and not be installed under the default Python's PYTHONPATH?
Actually thinking about this on the way home, I had that back to front - we do want to install the dependency so that is it available to the system's default Python (and that will not look at the Galaxy eggs folder). The Galaxy Tool Shed dependency installer doesn't have root does it? That would help for many install issues for libraries etc at system level, not just for Python but similarly for Perl, R, etc. Peter
No it does not have root, and it is very important that we not install anything at the system level since we need to maintain version isolation. We should always be able to inject things into the environment so that Python/Perl/R look in the right place. -- James Taylor, Assistant Professor, Biology/CS, Emory University On Tue, Apr 16, 2013 at 2:05 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
On Tue, Apr 16, 2013 at 5:23 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
Hi all,
Galaxy has long had two very simple (but largely unused) dependency mechanisms in a tool's XML (pre-dating the tool shed), for binaries and Python modules, e.g.
<requirements> <requirement type="python-module">numpy</requirement> <requirement type="binary">taxBuilder</requirement> </requirements>
http://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax#A.3Crequirement.3...
This has since been supplemented by the Tool Shed centric package requirement mechanism.
How can a tool declare a Python dependency, and have it installed automatically via the Tool Shed? Presumably it would have to end up as an egg in the Galaxy eggs folder, and not be installed under the default Python's PYTHONPATH?
Actually thinking about this on the way home, I had that back to front - we do want to install the dependency so that is it available to the system's default Python (and that will not look at the Galaxy eggs folder).
The Galaxy Tool Shed dependency installer doesn't have root does it? That would help for many install issues for libraries etc at system level, not just for Python but similarly for Perl, R, etc.
Peter ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
On Wed, Apr 17, 2013 at 6:05 PM, James Taylor <james@jamestaylor.org> wrote:
No it does not have root, and it is very important that we not install anything at the system level since we need to maintain version isolation. We should always be able to inject things into the environment so that Python/Perl/R look in the right place.
So is there any guidance on how that is best done? Python has various options including virtual env, but I doubt we can make any assumptions about the host system's python - even which version exactly. Does that mean installing Python modules locally and then manipulating the PYTHONPATH is the best plan? Thanks, Peter
On Wed, Apr 17, 2013 at 2:43 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
Does that mean installing Python modules locally and then manipulating the PYTHONPATH is the best plan?
That's what I had in mind, there may be better ways though. -- James Taylor, Assistant Professor, Biology/CS, Emory University
Hi Peter,
On Wed, Apr 17, 2013 at 6:05 PM, James Taylor <james@jamestaylor.org> wrote:
No it does not have root, and it is very important that we not install anything at the system level since we need to maintain version isolation. We should always be able to inject things into the environment so that Python/Perl/R look in the right place.
So is there any guidance on how that is best done?
I do not now it it is best practise but have a look at package_numpy_1_7 and package_matplotlib_1_2 I will try to setup a few common python deps, so we can include them easily in tool_dependecies.xml
Python has various options including virtual env, but I doubt we can make any assumptions about the host system's python - even which version exactly. Does that mean installing Python modules locally and then manipulating the PYTHONPATH is the best plan?
Thanks,
Peter ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
On Wed, Apr 17, 2013 at 9:24 PM, Björn Grüning wrote:
Hi Peter,
On Wed, Apr 17, 2013 at 6:05 PM, James Taylor <james@jamestaylor.org> wrote:
No it does not have root, and it is very important that we not install anything at the system level since we need to maintain version isolation. We should always be able to inject things into the environment so that Python/Perl/R look in the right place.
So is there any guidance on how that is best done?
I do not now it it is best practise but have a look at
package_numpy_1_7 and package_matplotlib_1_2
I will try to setup a few common python deps, so we can include them easily in tool_dependecies.xml
Thanks - those will be useful. Meanwhile it looks like this thread is now merging with this more general one about dependencies: http://lists.bx.psu.edu/pipermail/galaxy-dev/2013-April/014198.html Regards, Peter
I modified the PYTHONPATH in htseq_count (http://toolshed.g2.bx.psu.edu/view/lparsons/htseq_count) and cutadapt (http://toolshed.g2.bx.psu.edu/view/lparsons/cutadapt). It would be great to have some common dependencies setup in the toolshed such as numpy, etc. Lance Björn Grüning wrote:
Hi Peter,
On Wed, Apr 17, 2013 at 6:05 PM, James Taylor<james@jamestaylor.org> wrote:
No it does not have root, and it is very important that we not install anything at the system level since we need to maintain version isolation. We should always be able to inject things into the environment so that Python/Perl/R look in the right place. So is there any guidance on how that is best done?
I do not now it it is best practise but have a look at
package_numpy_1_7 and package_matplotlib_1_2
I will try to setup a few common python deps, so we can include them easily in tool_dependecies.xml
Python has various options including virtual env, but I doubt we can make any assumptions about the host system's python - even which version exactly. Does that mean installing Python modules locally and then manipulating the PYTHONPATH is the best plan?
Thanks,
Peter ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
-- Lance Parsons - Scientific Programmer 134 Carl C. Icahn Laboratory Lewis-Sigler Institute for Integrative Genomics Princeton University
participants (4)
-
Björn Grüning -
James Taylor -
Lance Parsons -
Peter Cock