Hi,
Ok, thanks for all the quick feedback folks, and your tool links John. So much work rolling off everyone's presses!
John, on <command> and nixing interpreter="" : Ok, I like that, to change it so that people can see <command> tag is a place for shell interaction.
Re. Update / Revision distinction: So looking at this in detail (Peter and Martin ;) ) - it turns out the "update" and "revision" terms are actually from "Install latest revision" vs "Get updates" options that occur in the Galaxy interface (you can see from snapshots I did of galaxy admin view in tool versioning tutorial) when downloading tool updates. So my definitions were an attempt to explain the difference there. Perhaps then we should have a ticket to have Galaxy force any toolshed update to have a version id that increments, then no opportunity for confusion, and no need for "update" phrase in galaxy UI?
In tutorial I do want to show people the implications of not incrementing. ... I'll let trainees know that incrementing a tool version number leads to a separately selectable tool version to run. To keep an up-to-date Galaxy install is to engage the admin to be bringing in these tool versions/choices more frequently even though they functionally don't do anything different. Advice is to encourage the dev to batch their changes into just a few revisions per year say for main Galaxy toolshed. I've downgraded this whole "update" thing to just say its possible to upload a change without version increment but not advisable for main toolshed.
Cheers,
Damion
P.s. for the ffp_phylogeny tool I wanted to do piping in the <command> tag e.g. "<command>progx -a | progy -b -c | progz</command>". Easy enough unless one wants to catch errors when they happen within the piped processes. I found that verbatim one would unfortunately never see any error info from earlier stages of the pipe on return to Galaxy, right?. I took a stab at writing a "check_output(command)" call in ffp_phylogeny tool that sets up error reporting for each stage of pipe, which seems to do ok (though only tested manually), but wondering if anyone else has tackled the piping error reporting?
Hsiao lab, BC Public Health Microbiology & Reference Laboratory, BC Centre for Disease Control 655 West 12th Avenue, Vancouver, British Columbia, V5Z 4R4 Canada ________________________________________ From: John Chilton [jmchilton@gmail.com] Sent: Friday, March 13, 2015 7:56 AM To: Peter Cock Cc: Martin Čech; galaxy-dev@lists.bx.psu.edu; Hsiao, William; Dooley, Damion Subject: Re: [galaxy-dev] Comment on Training Documentation
Awesome work Damion - thanks for publishing and sharing these! I hope the workshop goes well.
I recently created a wiki page just with tool development resource links
https://wiki.galaxyproject.org/Develop/ResourcesTools
that now gets embedded in other more common landing pages including
https://wiki.galaxyproject.org/Admin/Tools/AddToolTutorial https://wiki.galaxyproject.org/Develop
I have added links to your content from to this resource list.
One quick thought on the content. The interpreter="python" idiom for wrappers has been around for years and is used widely but it has some serious drawbacks - in particular nothing can come before the wrapper in the tool XML - no cheetah directives like #import or #set and no linking in files (for instance the following setup idiom:
ln -s "${input_bam}" temp_input.bam && ln -s "${input_bam.metadata.bam_index}" temp_input.bam.bai && actual_command --args
In the planemo documentation I am not going to mention interpreter for this reason - I think it is frustrating for people when it does not work the way intuitively it should. The forthcoming 15.03 injects a new variable called $__tool_directory__ that I think should be the new best practice.
ln -s "${input_bam}" temp_input.bam && ln -s "${input_bam.metadata.bam_index}" temp_input.bam.bai && python $__tool_directory__/wrapper.py --args
I understand that it might be best to not base your tutorial on features not even released yet :) - but I did want to take the opportunity to mention this idiom.
As you mentioned I have also been working on updated tool development documentation lately at (http://planemo.readthedocs.org/en/latest/). I am trying to build a set of smaller resources that can be composed into tailored resources (e.g. build tools without planemo, with planemo, with planemo virtual appliance), (normal tutorial, verbose tutorial, slide-based tutorial), (with and without Docker). I don't know if rst is up to this task, but we will see.
Another effort along these lines worth checking out is Kyle's tool tutorial - https://github.com/kellrott/galaxy-tooldev-docs/tree/master/docs. This tutorial is focused on using the planemo appliance and Dockerizing tools.
-John
On Fri, Mar 13, 2015 at 4:35 AM, Peter Cock p.j.a.cock@googlemail.com wrote:
On Fri, Mar 13, 2015 at 12:23 AM, Martin Čech marten@bx.psu.edu wrote:
Your tutorial looks great Damion! Thank you for sharing.
I am not sure about the vocabulary you are trying to estabilish though (update vs revision). I understand what you mean but I would just stick with 'If the tool changed behavior you have to bump the version.' No need to specifically name the steps as it can cause confusion (revision has a different meaning in version control world).
I was also confused about your (Damion's) update vs revision terminology, and agree with Martin that any behaviour change (especially bug fixes, even minor ones) should come with a version change.
I've not had time to look at your slides, but thanks for sharing them.
Peter