James, et. al. I think it is interesting that there was push back on providing infrastructure (tool actions) for obtaining CBL from github and performing installs based on it because it was not in the tool shed and therefore less reproducible, but the team believes infrastructure should be put in place to support pypi. I understand there are any number of distinctions that could be made here - perhaps you have made the calculation pypi is more stable than github (either in terms or immutability or funding), perhaps the setuptools mechanism is more general and could potentially support grabbing these tar balls from the tool shed (or a tool shed adjacent object store). I think we all value reproduciblity here, but we make different calculations on what is reproducible. I think in terms of implementing the ideas James has laid out or similar things I have proposed, it might be beneficial to have some final answers on what external resources are allowed - both for obtaining a Galaxy IUC gold star and for the tool shed providing infrastructure to support their usage. I don't know if this takes for the form of the IUC voting or James and/or Greg issuing a proclamation, but it would be good to get firm answers on these two questions for the following sites rubygems, pypi, github, bitbucket, cpan, cran, sourceforge, and google code. It would also be great to have a process in place for deciding these questions for future repositories. Thanks, -John On Mon, Aug 26, 2013 at 9:05 AM, James Taylor <james@jamestaylor.org> wrote:
All,
I've been seeing some examples of tool_depedencies.xml come across of the list, and I'm wondering if there are ways that it can be simplified. When we were first defining these features, we talked about having high level recipes for certain types of installs. This could greatly simplify things. For example, can this:
<tool_dependency> <package name="requests" version="1.2.3"> <install version="1.0"> <actions> <action type="download_by_url">http://pypi.python.org/packages/source/r/requests/re quests-1.2.3.tar.gz</action> <action type="make_directory">$INSTALL_DIR/lib/python</action> <action type="shell_command">export PYTHONPATH=$PYTHONPATH:$INSTALL_DIR/lib/python && python setup.py install --home $INSTALL_DIR --install-scripts $INSTALL_DIR/bin</action> <action type="set_environment"> <environment_variable name="PYTHONPATH" action="append_to">$INSTALL_DIR/lib/python</environment_variable> <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable> </action>
</actions> </install> <readme> </readme> </package> </tool_dependency>
Be simplified to:
<tool_dependency> <package name="requests" version="1.2.3"> <install recipe="python_package_setuptools"
url="http://pypi.python.org/packages/source/r/requests/requests-1.2.3.tar.gz" /> </tool_dependency>
The assumptions are: when version is not provided, it is 1.0 (we've always maintained compatibility in the past for config files so hopefully this never changes), when installing a python package the install directories and environment variables that need to be set are always the same.
Similar recipes could be:
autoconf: default to configure; make; make install, allow providing configuration options make_install: just make; make install; allow providing make options python_virtualenv ruby_rbenv r_package ...
Basically, most of the times the steps to install a particular package are boilerplate, this would remove a ton of duplication in the recipe files. Also, a likely less popular proposal would be to go one step further, tool_dependencies.yaml:
recipe: python_package_setuptools name: requests version: 1.2.3 url: http://pypi.python.org/packages/source/r/requests/requests-${version}.tar.gz
-- jt ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/