Tool Shed / Fabric upcoming integration
Dear Greg, according to the Tool Shed wiki page, you will be enabling installation of Tool Shed tools with 3rd party dependencies very soon. Together with Enis Afgan I created several fabric scripts a while ago that install some of our tools into an EC2 instance of Galaxy. We are now in the process of updating these scripts and wanted to add new ones. Is there already a (preliminary) framework that those fabric scripts should follow if they are to be packaged in the tool shed repository? Thanks for your help! -- Sebastian
Hello Sebastian, Sorry I haven't had a chance to update the tool shed wiki with this information. I'm not quite done with the implementation (although I'm very close), so I've held off on the documentation for now. However, here are the highlights: My goal is for most of the tool shed repositories ( 90% or more ) to require inclusion of only an xml file that provides information for installing and building the tool dependencies. In these cases, the Galaxy framework will handle the installation, whether it's via fabric or something else. This xml file must be named tool_dependencies.xml. Here is what this file looks like for the emboss_5 tool shed repository (I have not yet uploaded this file to the repository in the main Galaxy tool shed, so you won't find it there at least for a short wile longer). ======================= <?xml version="1.0"?> <tool_dependency> <package name="emboss" version="5.0.0"> <fabfile> <method name="install_and_build_package"> <param name="download_url">ftp://emboss.open-bio.org/pub/EMBOSS/old/5.0.0/EMBOSS-5.0.0.tar.gz</param> <param name="build_commands"> <command>./configure --without-x --prefix=$INSTALL_DIR</command> <command>make</command> <command>make install</command> </param> </method> <method name="install_and_build_package"> <param name="download_url">ftp://emboss.open-bio.org/pub/EMBOSS/old/5.0.0/PHYLIP-3.6b.tar.gz</param> <param name ="build_commands"> <command>./configure --without-x --prefix=$INSTALL_DIR</command> <command>make</command> <command>make install</command> </param> </method> </fabfile> <readme> These links provide information for building the Emboss package in most environments . System requirements http://emboss.sourceforge.net/download/#Requirements Platform-dependent notes http://emboss.sourceforge.net/download/#Platforms </readme> </package> </tool_dependency> ======================= $INSTALL_DIR is a reserved variable that Galaxy replaces with the installation directory that is determined at the time the repository and tool dependencies are being installed. Notice that the above xml file defines the name of the package to be installed as well as the version of that package. This information must be found in the <requirements> tag set of at least 1 of the tool config files within that downloadable changeset revision of the tool shed repository. So for the emboss_5 tools configs (e.g., emboss_antigenic.xml, etc), the requirements tag set looks like this. <requirements><requirement type="package" version="5.0.0">emboss</requirement></requirements> If the dependency name / version information does not match, the tool dependencies cannot be automatically installed when the Galaxy admin installs the tool shed repository into their local galaxy instance. I have several other tool_dependencies.xml examples I can provide if you'd like. But it might be easer if you send me the tool dependencies for which you are writing your script and I can provide you the xml definition that will handle the installation. The details will be different for those tool shed repositories in which the owner chooses to include a proprietary fabric script. This these cases, the <fabfile> tag becomes <proprietary_fabfile>. I have a working prototype for handling this approach, but things may change slightly before I call it finished. I look forward to your response on this. Thanks very much for your interest! Greg Von Kuster On Jun 1, 2012, at 12:06 PM, Sebastian J Schultheiss wrote:
Dear Greg,
according to the Tool Shed wiki page, you will be enabling installation of Tool Shed tools with 3rd party dependencies very soon.
Together with Enis Afgan I created several fabric scripts a while ago that install some of our tools into an EC2 instance of Galaxy. We are now in the process of updating these scripts and wanted to add new ones. Is there already a (preliminary) framework that those fabric scripts should follow if they are to be packaged in the tool shed repository?
Thanks for your help!
-- Sebastian ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Dear Greg, thanks a lot for your detailed answer. You've done some amazing work for the tool shed! I really like the requirements tags, they make a lot of sense for many of our tools. I still have some questions about the way our tools could benefit from this new method. Some of our tools require specific packages installed on ubuntu (or whatever OS galaxy is running on), and the current fabric scripts I co-wrote with Enis contain apt-get calls with sudo rights. Will there be a way to require OS packages in the new tool_dependencies.xml as well? Maybe it makes sense to create a separate fabric script for these kinds of dependencies and reference it in the read me tags you show. Where will the contents of this read me be displayed? Will users be able to see them before they install the tool? Thanks again for your time! -- Sebastian On 01.06.2012, at 22:02, Greg Von Kuster wrote:
Hello Sebastian,
Sorry I haven't had a chance to update the tool shed wiki with this information. I'm not quite done with the implementation (although I'm very close), so I've held off on the documentation for now. However, here are the highlights:
My goal is for most of the tool shed repositories ( 90% or more ) to require inclusion of only an xml file that provides information for installing and building the tool dependencies. In these cases, the Galaxy framework will handle the installation, whether it's via fabric or something else. This xml file must be named tool_dependencies.xml. Here is what this file looks like for the emboss_5 tool shed repository (I have not yet uploaded this file to the repository in the main Galaxy tool shed, so you won't find it there at least for a short wile longer).
======================= <?xml version="1.0"?> <tool_dependency> <package name="emboss" version="5.0.0"> <fabfile> <method name="install_and_build_package"> <param name="download_url">ftp://emboss.open-bio.org/pub/EMBOSS/old/5.0.0/EMBOSS-5.0.0.tar.gz</param> <param name="build_commands"> <command>./configure --without-x --prefix=$INSTALL_DIR</command> <command>make</command> <command>make install</command> </param> </method> <method name="install_and_build_package"> <param name="download_url">ftp://emboss.open-bio.org/pub/EMBOSS/old/5.0.0/PHYLIP-3.6b.tar.gz</param> <param name ="build_commands"> <command>./configure --without-x --prefix=$INSTALL_DIR</command> <command>make</command> <command>make install</command> </param> </method> </fabfile> <readme> These links provide information for building the Emboss package in most environments .
System requirements http://emboss.sourceforge.net/download/#Requirements
Platform-dependent notes http://emboss.sourceforge.net/download/#Platforms </readme> </package> </tool_dependency> =======================
$INSTALL_DIR is a reserved variable that Galaxy replaces with the installation directory that is determined at the time the repository and tool dependencies are being installed.
Notice that the above xml file defines the name of the package to be installed as well as the version of that package. This information must be found in the <requirements> tag set of at least 1 of the tool config files within that downloadable changeset revision of the tool shed repository. So for the emboss_5 tools configs (e.g., emboss_antigenic.xml, etc), the requirements tag set looks like this.
<requirements><requirement type="package" version="5.0.0">emboss</requirement></requirements>
If the dependency name / version information does not match, the tool dependencies cannot be automatically installed when the Galaxy admin installs the tool shed repository into their local galaxy instance.
I have several other tool_dependencies.xml examples I can provide if you'd like. But it might be easer if you send me the tool dependencies for which you are writing your script and I can provide you the xml definition that will handle the installation.
The details will be different for those tool shed repositories in which the owner chooses to include a proprietary fabric script. This these cases, the <fabfile> tag becomes <proprietary_fabfile>. I have a working prototype for handling this approach, but things may change slightly before I call it finished.
I look forward to your response on this. Thanks very much for your interest!
Greg Von Kuster
On Jun 1, 2012, at 12:06 PM, Sebastian J Schultheiss wrote:
Dear Greg,
according to the Tool Shed wiki page, you will be enabling installation of Tool Shed tools with 3rd party dependencies very soon.
Together with Enis Afgan I created several fabric scripts a while ago that install some of our tools into an EC2 instance of Galaxy. We are now in the process of updating these scripts and wanted to add new ones. Is there already a (preliminary) framework that those fabric scripts should follow if they are to be packaged in the tool shed repository?
Thanks for your help!
-- Sebastian ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
participants (2)
-
Greg Von Kuster
-
Sebastian J Schultheiss