Problem with cleanup.py
Hello, We are incurring a problem with cleanup.py: We have already run two cycles of cleanup.py (1-6) with a cut off of 30 days, we see in the logs that many datasets are deleted, but when we do df -kh to check the file system, we see creeping up utilization of space - no relinquishing of any space. We are running in a docker environment and our cleanup job looks as follows: ================================================================ #!/bin/bash export GALAXY_VIRTUAL_ENV=/galaxy_venv source $GALAXY_VIRTUAL_ENV/bin/activate export script="/galaxy-central/scripts/cleanup_datasets/cleanup_datasets.py" export config="/etc/galaxy/galaxy.ini" export logdir="/export/galaxy-central/logs" export days=30 export num=$1 nohup $script $config -d $days -$num -r -f >> $logdir/cleanup$num.log & ================================================================ and we run it with ./run_cleanup.sh 1 or 2, 3 etc. Any ideas ? Thanks! George Weingart PhD Huttenhower Lab Harvard School of Public Health
Hi George, on our instance I have noticed that the tmp (new_file_path) was taking up quite some space and files were only deletable after restarting the web / uwsgi process. FYI a while ago I wrote this script which should ease the cronjob setup a bit (does not cleanup tmp though): https://github.com/galaxyproject/galaxy/blob/dev/scripts/maintenance.sh Cheers, Jelle On Tue, Oct 30, 2018 at 5:04 AM George Weingart <george.weingart@gmail.com> wrote:
Hello,
We are incurring a problem with cleanup.py:
We have already run two cycles of cleanup.py (1-6) with a cut off of 30 days, we see in the logs that many datasets are deleted, but when we do df -kh to check the file system, we see creeping up utilization of space - no relinquishing of any space.
We are running in a docker environment and our cleanup job looks as follows: ================================================================
#!/bin/bash
export GALAXY_VIRTUAL_ENV=/galaxy_venv
source $GALAXY_VIRTUAL_ENV/bin/activate
export script="/galaxy-central/scripts/cleanup_datasets/cleanup_datasets.py"
export config="/etc/galaxy/galaxy.ini"
export logdir="/export/galaxy-central/logs"
export days=30
export num=$1
nohup $script $config -d $days -$num -r -f >> $logdir/cleanup$num.log &
================================================================
and we run it with ./run_cleanup.sh 1
or 2, 3 etc.
Any ideas ?
Thanks!
George Weingart PhD
Huttenhower Lab
Harvard School of Public Health ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
participants (2)
-
George Weingart
-
Jelle Scholtalbers