Thanks for including me Damion.

For symlinking, you're right John and I never thought about any of the issues with deleting datasets in Galaxy afterwards.

The ability to define a connection for passing/failing a subsequent tool you mentioned sounds exactly what we were trying to accomplish.  Is there any link to documentation on how to do this?  Passing a dummy input "passing_qc_text_file" could work too, but we were designing our QC tool to handle input from many types of tools (along with a "rules" file for how to evaluate the quality), so we would end up modifying a lot of existing tools just to add a "passing_qc_text_file" input.  I'm not sure how much time we'd want to spend updating a bunch of tools if something new is on the way.

Aaron

-----"Dooley, Damion" <Damion.Dooley@bccdc.ca> wrote: -----
To: John Chilton <jmchilton@gmail.com>
From: "Dooley, Damion" <Damion.Dooley@bccdc.ca>
Date: 11/17/2015 01:31PM
Cc: "galaxy-dev@lists.bx.psu.edu" <galaxy-dev@lists.bx.psu.edu>, "aaron.petkau@phac-aspc.gc.ca" <aaron.petkau@phac-aspc.gc.ca>
Subject: Re: [galaxy-dev] Tool code for symlinking a data collection from input to output?

Ah, I can see how symlinking could lead to file management issues.  Well,
we were trying to avoid the situation where use of our qc tool would
require customizing any subsequent tools in a workflow, and as well,
reduce disk overhead of hundred megabyte files being passed along in a
workflow.

So wow on the second paragraph - enabling dependencies outside of tool
file I/o.  I agree with Eric, this will be great.

Now in our current canned workflows we actually don't need this to be
edited via the interface - so are there details on how to edit a workflow
file directly to get this dependency of tool B on tool A in place?

Thanks,

Damion

On 2015-11-17, 11:18 AM, "John Chilton" <jmchilton@gmail.com> wrote:

>Slowly trying to catch up on e-mail after a lot of travel in November
>and I answered a variant of this to Damion directly, the most relevant
>snippet was:
>
>"
>I would not symbolic link the
>files though. I would just take the original collection and pipe it
>into the next tool and add a dummy input to the next tool
>("passing_qc_text_file") that would cause the workflow to fail if the
>qc fails. This is a bit hacky, but symbolic linking will break
>Galaxy's deletion, purging, etc.... You can delete the original
>dataset collection and the result would affect the files on disk for
>the output collection without Galaxy having anyway to know.
>
>The workflow subsystem has the ability to define a connection like
>this (just wait for one tool to pass before calling the next without a
>input/output relationship) but it hasn't been exposed in the workflow
>editor yet."
>
>-John