Purging datasets as part of workflow?
Hi, Is there an automatic way to permanently delete and purge datasets that are the result of intermittent tools in a workflow? That is, how can one automatically keep only the first and the last steps of an workflow without having to manually delete all the intermittent datasets and running the purge scripts? Thanks, Sveinung Gundersen -- Sveinung Gundersen, PhD. Head engineer, ELIXIR.NO / The Genomic HyperBrowser team Department of Informatics, University of Oslo, Boks 1072 Blindern, NO-0316 OSLO, Norway Email: sveinung.gundersen@medisin.uio.no. Phone: +47 93 00 94 54
On Thu, Mar 13, 2014 at 6:30 PM, Sveinung Gundersen <sveinung.gundersen@medisin.uio.no> wrote:
Hi,
Is there an automatic way to permanently delete and purge datasets that are the result of intermittent tools in a workflow? That is, how can one automatically keep only the first and the last steps of an workflow without having to manually delete all the intermittent datasets and running the purge scripts?
Thanks, Sveinung Gundersen
+1 In the workflows editor we can star particular outputs to be kept, and the rest are "deleted". I'd much prefer they be more aggressively "permanently deleted". Alternately, when editing a workflow there (used to be) a delete dataset action - how about a "permanently delete" action which would be great for some big-data workflows? Peter
I don't even think you can delete datasets in workflow presently - let alone permanently delete them - only "hide" them. https://trello.com/c/YfLGkJKe The core team just meet and a number of high priorities for the next 9 months were identified and reworking workflow scheduling was very high on this list. I would expect an number of key workflow modifications to be made by early fall that would make this much easier. https://trello.com/c/K2qLZCrg My own opinion on the particulars of this topic are that whether datasets are deleted or permanently deleted should be up to the workflow runner not the workflow author. In particular, the workflow "model" describes what datasets are to be deleted and the workflow runner could opt to permanently delete them if their Galaxy instance allows this. It is exactly because this is not a universally allowed option that I think it should not be part of the workflow description that can be shared between instances and on the tool shed. -John On Thu, Mar 13, 2014 at 1:38 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
On Thu, Mar 13, 2014 at 6:30 PM, Sveinung Gundersen <sveinung.gundersen@medisin.uio.no> wrote:
Hi,
Is there an automatic way to permanently delete and purge datasets that are the result of intermittent tools in a workflow? That is, how can one automatically keep only the first and the last steps of an workflow without having to manually delete all the intermittent datasets and running the purge scripts?
Thanks, Sveinung Gundersen
+1
In the workflows editor we can star particular outputs to be kept, and the rest are "deleted". I'd much prefer they be more aggressively "permanently deleted".
Alternately, when editing a workflow there (used to be) a delete dataset action - how about a "permanently delete" action which would be great for some big-data workflows?
Peter ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
participants (3)
-
John Chilton -
Peter Cock -
Sveinung Gundersen