AFAIK it's not yet possible, but a tool shed API to expose subversion hooks for repository creation and update is a logical development which is already on the radar AFAIK.
For the bigger picture of complexity for the user, I think some key clarifications required include: which 'user' is experiencing complexity; how big a barrier it is to generating tools; how readily can we make it go away; how much will it help?
Tool builders (even 'users' of the TF) might reasonably be expected to have a capacity for substantial complexity that the end user of the generated tools should not need to possess - after all, once a generated tool is installed in Galaxy, the Galaxy user can't tell the difference and sees no more complexity from the generated tool than a hand written one.
On absent local tool sheds: it's trivial to run a toolshed on a dev laptop and not hard to run in production - but tool makers can always use the public test and main TS if they want to distribute new tools so I'm not sure how big a barrier a missing local toolshed really is.
On Thu, Jul 18, 2013 at 12:08 PM, Michael E. Cotterell < mepcotterell@gmail.com> wrote:
I didn't mean to imply it was unclear. It definitely made sense.
I completely agree with you. However, is there for a way (maybe via the toolshed api) for my tool to push these tools to a TS repo? Also, doesn't that add some more complexity for the user? Not everyone runs a local tool shed.
Sincerely, Michael E. Cotterell
Ph.D. Student in Computer Science, University of Georgia Instructor of Record, Graduate RA & TA, University of Georgia Faculty Liaison, CS Graduate Student Association, University of Georgia mepcotterell@gmail.com (mailto:mepcotterell@gmail.com) mepcott@uga.edu (mailto:mepcott@uga.edu) mec@cs.uga.edu (mailto:mec@cs.uga.edu) http://michaelcotterell.com/
On Wednesday, July 17, 2013 at 10:05 PM, Ross wrote:
Sorry if my post was rambling and unclear.
Here's the executive summary for what it's worth: IMHO, a Galaxy tool that generates tools should emit them as tool shed
compatible artefacts, ready for uploading to a TS repository for automated installation to any target instance.
On Thu, Jul 18, 2013 at 11:25 AM, Michael E. Cotterell <
mepcotterell@gmail.com (mailto:mepcotterell@gmail.com)> wrote:
Ross,
Thanks for the comments.
I've already made my main tool "tool_shed" compatible. What I'm
wondering about is what do I do with the tools that my tool creates?
Sincerely, Michael E. Cotterell
Ph.D. Student in Computer Science, University of Georgia Instructor of Record, Graduate RA & TA, University of Georgia Faculty Liaison, CS Graduate Student Association, University of Georgia mepcotterell@gmail.com (mailto:mepcotterell@gmail.com) (mailto:
mepcotterell@gmail.com)
mepcott@uga.edu (mailto:mepcott@uga.edu) (mailto:mepcott@uga.edu) mec@cs.uga.edu (mailto:mec@cs.uga.edu) (mailto:mec@cs.uga.edu) http://michaelcotterell.com/
On Wednesday, July 17, 2013 at 7:44 PM, Ross wrote:
Hi, Michael,
As others have said, in the long term, tool wrappers and their
dependencies will be distributed and installed through tool sheds rather than being distributed with Galaxy source, so you might want to plan to generate and upload tool shed compatible archives from the get-go. The test and main tool sheds contain plenty of examples and Björn's tools exercise pretty much all of the functionality - you can browse the repo structure which is identical to the gz archive structure you need to upload to a new repo, or download the repo as a gz, unpack it and get exactly the kind of directory structure and contents you need to emulate for your tools. Once you have a working tool packaging the archive up is straightforward.
The format for tool shed repo uploads and syntax for the tags used
to define dependencies is very well documented in the tool shed section of the wiki. As Björn points out, the tool factory python wrapper might be a useful source of ideas and perhaps code. Your tool generator will need to do something similar to write the content and generate complete tool shed archives. When generating a new tool, the TF uses an ugly XML wrapper generator (contributed improvements would be very welcome!) and (probably more usefully) the few lines of code you need to package up the functional test data and the XML and wrapper if you need one in toolshed archive format.
On Thu, Jul 18, 2013 at 5:30 AM, Björn Grüning <
bjoern.gruening@pharmazie.uni-freiburg.de (mailto: bjoern.gruening@pharmazie.uni-freiburg.de) (mailto: bjoern.gruening@pharmazie.uni-freiburg.de)> wrote:
Hi Michael,
I think you will enter new ground with your tool. The closest tool
that
will do something similar is Ross toolfactory, I think:
http://toolshed.g2.bx.psu.edu/view/fubar/toolfactory
For me one question is, do you really want to offer that service to every user and is it save?
TL;DR I have a tool that generates other tools. Where should
those tools go, and how should I add them (without modifying Galaxy code)?
I apologize, in advance, for the wall of text. I would like some
advice on what the best way to go about setting up a tool that generates other tools...
As some of you (especially those who saw my boss's talk at GCC)
already know, I'm working on a tool that lets galaxy users add web service operations as tools to Galaxy.
We have a general purpose client for invoking web services, and
we have a tool that let's a user enter a WSDL/WADL URL, select the operations they want to add, and generates tool config XML files for Galaxy.
Right now, I'm in the process of making our tool "tool shed"
compatible, and that's going along great
(I especially like the tool dependency installation stuff), but
I'm trying to determine the best course of action on the following:
- Where is the best place to put the tool config (XML) files
that are generated by my tool? My predecessors just placed them in a subdirectory under our tool's directory.
I'm ok with this, but does that mean that all the generated
files will get removed if they upgrade our tool via the tool shed (haven't taken the time to see if
Galaxy just does an "hg pull" or if it's more complicated than
that)?
It is mainly a hg pull. But if you deinstall that repository and reinstall it, all your files are lost.
Would it be better to have my tool create a local tool shed
repository
(just for these generated tools), and if so, can the tool shed
API let my tool install this repository? Having not looked at install_tool_shed_repositories.py yet,
it's unclear from the wiki documentation whether or not POST
/api/tool_shed_repositories/install_repository_revision requires that the tool shed already registered in tool_sheds_conf.xml)?
I think that is a better way to do it. The API can be used to
install
your tools. But it need to be configured beforehand from the user, administrator.
- What's the best way to add the generated tools to Galaxy? My
predecessors modified the tool_config.xml directly and required users to restart Galaxy.
Now, If these tools are in a tool shed repo, I know that I can
use the add_to_tool_panel() method in lib/tool_shed/util/tool_util.py if these tools are in a tool shed repo, however,
if that's not the recommended course of action, then I'd like
some advice on what to do?
There is a feature request, that you can specify in your toolshed-repository the preferred location in the tool panel. For
now I
would ignore it and just install the tools somewhere and hope that feature request is quiet high in Gregs ToDo list :)
Cheers, Bjoern