Hi Greg,
I've retitled the thread, previously about a ToolShed nightly test failure.
A brief recap, we're talking about the Galaxy ToolShed XML installation recipes for the NCBI BLAST+ packages and my MIRA4 wrapper in their tool_dependencies.xml files:
http://toolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_2_29 http://testtoolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_2_29 http://testtoolshed.g2.bx.psu.edu/view/peterjc/mira4_assembler
These use the pattern of having os/arch specific <action> tags (which download and install the tool author's precompiled binaries) and a fall back default <action> which is to report an error with the os/arch combination and that there are no ready made binaries available.
Greg is instead advocating the fall back action be to download the source code, and do a local compile.
My reply is below...
On Thu, Mar 6, 2014 at 5:24 PM, Peter Cock p.j.a.cock@googlemail.com wrote:
On Thu, Mar 6, 2014 at 4:53 PM, Greg Von Kuster greg@bx.psu.edu wrote:
As we briefly discussed earlier, your mira4 recipe is not currently following best practices. Although you uncovered a problem in the framework which has now been corrected, your recipe's fall back <actions> tag set should be the recipe for installing mira4 from source ( http://sourceforge.net/projects/mira-assembler/ ) since there is no licensing issues for doing so. This would be a more ideal approach than echoing the error messages.
Thanks very much for helping us discover this problem though!
Greg Von Kuster
Hi Greg,
No problem - I'm good at discovering problems ;)
If the download approach failed, it it most likely due to a transient error (e.g. network issues with download). Here I would much prefer Galaxy aborted and reported this as an error (and does not attempt the default action). Is that what you just fixed?
As to best practice for the fall back action, I think that needs a new thread.
Regards,
Peter
As to best practice, I do not agree that in cases like this (MIRA4, NCBI BLAST+) where there are provided binaries for the major platforms that the fall back should be compiling from source.
The NCBI BLAST+ provide binaries for 32 bit and 64 bit Linux and Mac OS X (which I believe covers all the mainstream platforms Galaxy runs on).
Similarly, MIRA4 provides binaries for 64 bit Linux and Mac OS X. Note that 32 bit binaries are not provided, but would be very restricted in terms of the datasets they could be used on anyway - and I doubt many of the systems Galaxy runs on these days are 32 bits.
If the os/arch combination is exotic enough that precompiled binaries are not available, then it is likely compilation will be tricky anyway - or not supported for that tool, or Galaxy itself.
Essentially I am arguing that where the precompiled tool binaries cover any mainstream system Galaxy might be used on, a local compile fall back is not needed.
Also, these are both complex tools which are relatively slow to compile, and have quite a large compile time dependency set (e.g. MIRA4 requires at least a quite recent GCC, BOOST, flex, expat, and strongly recommends TCmalloc). Here at least some of the dependencies have been packaged for the ToolShed (probably by Bjoern?) but in the case of MIRA4 and BLAST+ this is still a lot of effort for no practical gain.
I also feel there is an argument that the Galaxy goal of reproducibility should favour using precompiled binaries if available: A locally compiled binary will generally mean a different compiler version, perhaps with different optimisation flags, and different library versions. It will not necessarily give the same results as the tool author's provided precompiled binary.
(Wow, this ended up being a long email!)
Regards,
Peter