On Mon, Aug 26, 2013 at 11:48 AM, John Chilton chilton@msi.umn.edu wrote:
I think it is interesting that there was push back on providing infrastructure (tool actions) for obtaining CBL from github and performing installs based on it because it was not in the tool shed and therefore less reproducible, but the team believes infrastructure should be put in place to support pypi.
Well, first, I'm not sure what "the team" believes, I'm stating what I believe and engaging in a discussion with "the community". At some point this should evolve into what we are actually going to do and be codified in a spec as a Trello card, which is even then not set in stone.
Second, I'm not suggesting we depend on PyPI. The nice thing about the second format I proposed on galaxy-dev is that we can easily parse out the URL and archive that file. Then someday we could provide a fallback repository where if the PyPI URL no longer works we still have it stored.
I think we all value reproduciblity here, but we make different calculations on what is reproducible. I think in terms of implementing the ideas James has laid out or similar things I have proposed, it might be beneficial to have some final answers on what external resources are allowed - both for obtaining a Galaxy IUC gold star and for the tool shed providing infrastructure to support their usage.
My focus is ensuring that we can archive things that pass through the toolshed. Tarballs from *anywhere* are easy enough to deal with. External version control repositories are a bit more challenging, especially when you are pulling just a particular file out, so that's where things got a little hinky for me.
Since we don't have the archival mechanism in place yet anyway, this is more a philosophical discussion and setting the right precedent.
And yes, keeping an archive of all the software in the world is a scary prospect, though compared to the amount of data we currently keep for people it is a blip. And I'm not sure how else we can really achieve the level of reproducibility we desire.