On Sun, Sep 29, 2013 at 10:43 PM, Guest, Simon <Simon.Guest@agresearch.co.nz> wrote:
At Fri, 27 Sep 2013 00:23:37 -0500, John Chilton wrote:
Simon,
What is the advantage of putting that XML definition in the tool shed? It is not 100% true because of prior_install_required dependencies, but for the most part sourcing/load the environment for tools is a Galaxy problem, not so much a tool shed one. What if we did this instead?
Add an option to Galaxy's universe_wsgi.ini with the following default:
tool_dependency_resolution_order = gx_package_manual, gx_package_toolshed
Which essentially implements my idea above, with James' additional configuration. But which can be overridden as:
tool_dependency_resolution_order = plugin_module, gx_package_manual, gx_package_toolshed
If set this way then placing <requirement package="0.5.9">bwa</requirement> in a tool will result in the module bwa/0.5.9 being loaded if it is 'avail'able, else it will check for a manually installed env.sh (which is where MSI is currently putting its module loads), and else it will fallback to source the tool shed installed dependency.
I feel like this will give you everything you want without any extra XML or configuration. Let me know if I am wrong.
Hi John,
I think you're right. Your scheme is neater than what I was proposing. The extra flexibility I was aiming at via some toolshed XML stuff appears not to be necessary upon further reflection. (I wanted to ensure a Galaxy admin could just install some RPMs, install a toolshed tool, and have everything resolve nicely. You seem to have achieved that with your scheme.)
I haven't had a chance to try your code yet, but as soon as I can I will do so, and get back to you.
If I leave out gx_package_toolshed altogether from tool_dependency_resolution_order, will the tool installation in Galaxy simply fail with a nice error message if the environment module and/or env.sh files are not found? (This is what I would like, as it would serve as a prompt to the Galaxy admin to install some extra RPMs or whatever.)
First tool_dependency_resolution_order is not how I landed up implementing it. Each "resolver" may need parameters so I decided to do an XML configuration kind of like Nate's job_conf.xml stuff. So to leave the tool shed stuff off you can just add dependency_resolvers_conf.xml to your Galaxy root. <dependency_resolvers> <galaxy_packages /> <modules /> <modules versionless="true" /> </dependency> You can also just leave galaxy_packages off if you are not using any manually installed env.sh files. The first modules resolver will try to match each tag like this: <requirement type="package" version="3.0.1">R</requirement> with a module load like: R/3.0.1 if that exact module is available. The second versionless form of the tag will result in Galaxy just falling back to auto-load whatever the default R module is if the exact version specified (3.0.1) is not available. This code doesn't touch the tool shed at all, so if you check to install dependencies, they will still be installed and if you don't they will not be installed. This configuration just tells the Galaxy how to use the dependencies that happen to be installed. You will need to review installed tools and make sure you have matching modules. Additionally, some tools may use this set_environment requirement type, I am not sure how to implement this or how prevalent its use is. -John
Will this also work for those toolshed packages which bundle their package definitions (to download, make and install the tool dependency) along with their wrappers?
Can you opt not to install packages for such repositories? Either way, the answer is I guess the same as above, the tool shed is unchanged, its just how Galaxy utilizes the installed dependencies that is being modified here.
Thanks for working on this.
cheers, Simon
======================================================================= Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. =======================================================================