Hi everybody,
I have a hopefully basic questions regarding temporary output files.
A user was generating >4TB output files via the program bedtools intersectbed. The jobs were cancelled, but I still see the (temporary) output files that were generated.
Does Galaxy clean these up? Does it do this regularly?
Would a quota prevent something like this from happening again or would the jobs have to finish before being "counted"?
Thanks and best regards,
Christopher
Hi Christopher
It depends on the setting of "cleanup_job" ('always', 'onsuccess', 'never'), whether the temporary files are cleaned up or not. Though, to be honest, I don't know, if this setting as any influence, when you 'delete' a job.
For the quotas, only dataset in the histories are counted
The temporary files are very handy for debugging, so I have 'onsuccess' on our production server and 'never' on our development server. I just regularly check the total size of the working directories and do the clean up manually
Regards, Hans-Rudolf
On 04/27/2018 01:51 PM, Previti wrote:
Hi everybody,
I have a hopefully basic questions regarding temporary output files.
A user was generating >4TB output files via the program bedtools intersectbed. The jobs were cancelled, but I still see the (temporary) output files that were generated.
Does Galaxy clean these up? Does it do this regularly?
Would a quota prevent something like this from happening again or would the jobs have to finish before being "counted"?
Thanks and best regards,
Christopher
-- *Dr. Christopher Previti* Genomics and Proteomics Core Facility High Throughput Sequencing (W190) Bioinformatician
German Cancer Research Center (DKFZ) Foundation under Public Law Im Neuenheimer Feld 580 69120 Heidelberg Germany Room: B2.102 (INF580/TP3) Phone: +49 6221 42-4661
christopher.previti@dkfz.de http://www.dkfz.de/ www.dkfz.de http://www.dkfz.de/
Management Board: Prof. Dr. Michael Baumann, Prof. Dr. Josef Puchta VAT-ID No.: DE143293537
Vertraulichkeitshinweis: Diese Nachricht ist ausschließlich für die Personen bestimmt, an die sie adressiert ist. Sie kann vertrauliche und/oder nur für den/die Empfänger bestimmte Informationen enthalten. Sollten Sie nicht der bestimmungsgemäße Empfänger sein, kontaktieren Sie bitte den Absender und löschen Sie die Mitteilung. Jegliche unbefugte Verwendung der Informationen in dieser Nachricht ist untersagt.
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
See also:
https://galaxyproject.github.io/training-material/topics/admin/tutorials/mon...
Cheers, Nicola
On 27/04/18 15:05, Hans-Rudolf Hotz wrote:
Hi Christopher
It depends on the setting of "cleanup_job" ('always', 'onsuccess', 'never'), whether the temporary files are cleaned up or not. Though, to be honest, I don't know, if this setting as any influence, when you 'delete' a job.
For the quotas, only dataset in the histories are counted
The temporary files are very handy for debugging, so I have 'onsuccess' on our production server and 'never' on our development server. I just regularly check the total size of the working directories and do the clean up manually
Regards, Hans-Rudolf
On 04/27/2018 01:51 PM, Previti wrote:
Hi everybody,
I have a hopefully basic questions regarding temporary output files.
A user was generating >4TB output files via the program bedtools intersectbed. The jobs were cancelled, but I still see the (temporary) output files that were generated.
Does Galaxy clean these up? Does it do this regularly?
Would a quota prevent something like this from happening again or would the jobs have to finish before being "counted"?
Thanks and best regards,
Christopher
-- *Dr. Christopher Previti* Genomics and Proteomics Core Facility High Throughput Sequencing (W190) Bioinformatician
German Cancer Research Center (DKFZ) Foundation under Public Law Im Neuenheimer Feld 580 69120 Heidelberg Germany Room: B2.102 (INF580/TP3) Phone: +49 6221 42-4661
christopher.previti@dkfz.de http://www.dkfz.de/ www.dkfz.de http://www.dkfz.de/
Management Board: Prof. Dr. Michael Baumann, Prof. Dr. Josef Puchta VAT-ID No.: DE143293537
Vertraulichkeitshinweis: Diese Nachricht ist ausschließlich für die Personen bestimmt, an die sie adressiert ist. Sie kann vertrauliche und/oder nur für den/die Empfänger bestimmte Informationen enthalten. Sollten Sie nicht der bestimmungsgemäße Empfänger sein, kontaktieren Sie bitte den Absender und löschen Sie die Mitteilung. Jegliche unbefugte Verwendung der Informationen in dieser Nachricht ist untersagt.
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
Thanks! Super useful!
Cheers,
Christopher
On 04/27/2018 04:12 PM, Nicola Soranzo wrote:
See also:
https://galaxyproject.github.io/training-material/topics/admin/tutorials/mon...
Cheers, Nicola
On 27/04/18 15:05, Hans-Rudolf Hotz wrote:
Hi Christopher
It depends on the setting of "cleanup_job" ('always', 'onsuccess', 'never'), whether the temporary files are cleaned up or not. Though, to be honest, I don't know, if this setting as any influence, when you 'delete' a job.
For the quotas, only dataset in the histories are counted
The temporary files are very handy for debugging, so I have 'onsuccess' on our production server and 'never' on our development server. I just regularly check the total size of the working directories and do the clean up manually
Regards, Hans-Rudolf
On 04/27/2018 01:51 PM, Previti wrote:
Hi everybody,
I have a hopefully basic questions regarding temporary output files.
A user was generating >4TB output files via the program bedtools intersectbed. The jobs were cancelled, but I still see the (temporary) output files that were generated.
Does Galaxy clean these up? Does it do this regularly?
Would a quota prevent something like this from happening again or would the jobs have to finish before being "counted"?
Thanks and best regards,
Christopher
-- *Dr. Christopher Previti* Genomics and Proteomics Core Facility High Throughput Sequencing (W190) Bioinformatician
German Cancer Research Center (DKFZ) Foundation under Public Law Im Neuenheimer Feld 580 69120 Heidelberg Germany Room: B2.102 (INF580/TP3) Phone: +49 6221 42-4661
christopher.previti@dkfz.de http://www.dkfz.de/ www.dkfz.de http://www.dkfz.de/
Management Board: Prof. Dr. Michael Baumann, Prof. Dr. Josef Puchta VAT-ID No.: DE143293537
Vertraulichkeitshinweis: Diese Nachricht ist ausschließlich für die Personen bestimmt, an die sie adressiert ist. Sie kann vertrauliche und/oder nur für den/die Empfänger bestimmte Informationen enthalten. Sollten Sie nicht der bestimmungsgemäße Empfänger sein, kontaktieren Sie bitte den Absender und löschen Sie die Mitteilung. Jegliche unbefugte Verwendung der Informationen in dieser Nachricht ist untersagt.
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
galaxy-dev@lists.galaxyproject.org