Re: [galaxy-dev] New post action with workflows : DeleteIntermediatesActionOutput

23 Sep 2014

      Hi Dannon,

Thanks for your answer. Sorry but I think I don't understand all of
your explanations.

First : if an output is marked with a highlighted snowflake, it will
be not deleted : it's OK
Second : if a dataset is used in several steps of my workflow, it will
be deleted when all these steps will be terminated : it's OK

But : "Perhaps the confusion is that they're also marked as 'hidden'
since they are not output steps?"
I don't understand :
so in my mind, when the dataset is marked with a grey snowflake, it is
automatically hidden and it's translated in the file of the workflow
with this code :
"post_job_actions": {
        "HideDatasetActionout_file1": {
                    "action_arguments": {},
                    "action_type": "HideDatasetAction",
                    "output_name": "out_file1"
                }
            },

But when I add another postJobAction to such dataset , with this
translation in the workflow file :
"post_job_actions": {
        "HideDatasetActionout_file1": {
                    "action_arguments": {},
                    "action_type": "HideDatasetAction",
                    "output_name": "out_file1"
                },
        "DeleteIntermediatesActionout_file1": {
                    "action_arguments": {},
                    "action_type": "DeleteIntermediatesAction",
                    "output_name": "out_file1"
                }
            },

The result in my history is an hidden dataset (visible when I chek the
hisotry option "include hidden dataset") but not a deleted dataset
(visible when I chek the history option "include deleted datasets")
And it is the case for datasets used in several steps of workflow AND
for dataset which is just an output of a job not used in an other job.

Is an hidden dataset not deletable ?
I've changed the order of postJobActions in the file, but no effect.
And If I remove the "HideDatasetAction", the dataset is not removed
because it become marked with highligthed snowflake.

So is your final purpose the alone solution to delete intermediates
datasets ? And if I correctly understand, I must create a tool that
take all my deletable datasets as input and which will remove all of
these.  In this case, the postJobAction "DeleteIntermediates" is not
usefull, is it ?

Thanks.

Julie

2014-09-23 14:15 GMT+02:00 Dannon Baker <dannon.baker@gmail.com>:
...
Hi Julie!
The action resolves at the completion time of the job it is attached to,
deleting datasets of any other already-completed jobs in that workflow which
are not marked as an output step (with a highlighted snowflake).  This will
*not* delete datasets of any job marked as an output.
Perhaps the confusion is that they're also marked as 'hidden' since they are
not output steps?
The general use-case for this is to put it on one of the 'final' steps of a
workflow (or multiple if you have a workflow with two long parallel tracks)
to clean up and delete datasets from intermediate jobs (which have already
completed) that you don't want to save as an output.
-Dannon
On Tue, Sep 23, 2014 at 4:03 AM, julie dubois <dubjulie@gmail.com> wrote:
...
Hi all,
I'm not sure how do this post-action act . I've been make some test
and it's very confused for me.
First, can this post-action be used on output which are used on other
steps of the workflow ? if yes, I suppose that galaxy wait for end of
use of this output before deleting.
Second, must I higlight the snowfalke of this output to permit its
deletion or must I keep the snowflake in grey ?
I post this question in galaxy-dev because I've  added this
post-action on several outputs in my workflow and it has had no effect
on my outputs. And when I open the .ga file which describe my
workflow, I've seen just one of this post-action in one output.
Thanks.
Julie
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/