Hello,
I have check the new galaxy code. There is no new update for the cleanup file.
I have also check my purging script line by line. I have no error (!), even if I launch all the code (purging_galaxy.sh) from bash or shell interpreter !?
If no one has any idea on this list I will redirect output error to my purging log file...
-Regards
Now, I have some others directories with this error :
Le 30 mars 2012 10:36, Rémy Dernat remy.d1@gmail.com a écrit :
Hello list,
We have some problems with the purging function.
Here are our errors :
find: `/share/work-galaxy/cur/004/4455': No such file or directory find: `/share/work-galaxy/cur/004/4459': No such file or directory find: `/share/work-galaxy/cur/004/4462': No such file or directory find: `/share/work-galaxy/cur/004/4458': No such file or directory find: `/share/work-galaxy/cur/004/4460': No such file or directory find: `/share/work-galaxy/cur/004/4467': No such file or directory find: `/share/work-galaxy/cur/004/4469': No such file or directory find: `/share/work-galaxy/cur/004/4463': No such file or directory find: `/share/work-galaxy/cur/004/4468': No such file or directory find: `/share/work-galaxy/cur/004/4466': No such file or directory find: `/share/work-galaxy/cur/004/4464': No such file or directory find: `/share/work-galaxy/cur/004/4470': No such file or directory find: `/share/work-galaxy/cur/004/4461': No such file or directory
We have that kind of errors by mail every days with postfix. But each times it is for the folder /share/work-galaxy/cur/004/.
These problems appears 1 month ago. We do not know why. The only specific configuration is that the database was a restoration from a previous galaxy server. It occurs for differents subfolders in /share/work-galaxy/cur/004/. In database, I do not find anything suspect, but on my linux system, these folders missing like the error message said.
The cron job is "sh /home/igalaxy/cron/purging_galaxy.sh 1 > /home/igalaxy/log/purge.log"
And purging.sh :
cat /home/igalaxy/cron/purging_galaxy.sh #!/bin/bash # purge galaxy datasets, histories, folders, libraires # not updated since $day day=$1 cd /home/igalaxy/galaxy-dist python scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini -d $day -1 python scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini -d $day -2 -r python scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini -d $day -3 -r python scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini -d $day -5 -r python scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini -d $day -4 -r python scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini -d $day -6 -r find /share/work-galaxy/tmp/ -mtime +30 -exec rm -rf "{}" ; find /share/work-galaxy/cur/ -mtime +30 -exec rm -rf "{}" ; find /share/work-galaxy/out/ -mtime +30 -exec rm -rf "{}" ;
The "cur" directory is the job_working_directory :
grep "cur" /home/igalaxy/galaxy-dist/universe_wsgi.ini |grep -v "#"
job_working_directory = /share/work-galaxy/cur
The purge log content is the following :
cat /home/igalaxy/log/purge.log
MySQL_python egg successfully loaded for mysql dialect
##########################################
# 2012-03-30 01:00:02 - Handling stuff older than 1 days
Datasets will NOT be removed from disk.
Deleted 0 histories
Elapsed time: 0.310899019241
##########################################
MySQL_python egg successfully loaded for mysql dialect
##########################################
# 2012-03-30 01:00:04 - Handling stuff older than 1 days
Datasets will be removed from disk.
Purged 0 histories.
Elapsed time: 0.31987786293
##########################################
MySQL_python egg successfully loaded for mysql dialect
##########################################
# 2012-03-30 01:00:05 - Handling stuff older than 1 days
Datasets will be removed from disk.
Removing disk, file /share/work-galaxy/files/009/dataset_9110.dat
Purging dataset id 9110
Removing disk, file /share/work-galaxy/files/009/dataset_9107.dat
Purging dataset id 9107
Removing disk, file /share/work-galaxy/files/009/dataset_9108.dat
Purging dataset id 9108
Removing disk, file /share/work-galaxy/files/009/dataset_9137.dat
Purging dataset id 9137
Removing disk, file /share/work-galaxy/files/009/dataset_9136.dat
Purging dataset id 9136
Removing disk, file /share/work-galaxy/files/009/dataset_9135.dat
Purging dataset id 9135
Purged 6 datasets
Freed disk space: 44272
Elapsed time: 0.6712641716
##########################################
MySQL_python egg successfully loaded for mysql dialect
##########################################
# 2012-03-30 01:00:07 - Handling stuff older than 1 days
Datasets will be removed from disk.
# Purged 0 folders.
Elapsed time: 0.310595035553
##########################################
MySQL_python egg successfully loaded for mysql dialect
##########################################
# 2012-03-30 01:00:09 - Handling stuff older than 1 days
Datasets will be removed from disk.
# Purged 0 libraries .
Elapsed time: 0.308594942093
##########################################
MySQL_python egg successfully loaded for mysql dialect
##########################################
# 2012-03-30 01:00:10 - Handling stuff older than 1 days
Datasets will be removed from disk.
######### Processing dataset id: 1590
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 1591
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 1592
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 1593
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 1594
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 1595
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 1596
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 1597
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 1598
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 1599
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 1600
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 1601
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 1602
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 1603
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 1604
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 1605
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 1606
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 1607
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 1608
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 1609
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 1610
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 1611
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 1612
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 1613
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 1614
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 1615
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 2997
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 3008
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 3201
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 3247
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 3447
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 3448
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 3586
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 3691
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 4522
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 4524
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 4525
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 4526
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 4581
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 4582
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 4583
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 4584
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 4585
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 4586
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 4587
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 4588
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 4589
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 4590
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 4591
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 4592
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 6123
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 6124
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 6266
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 6434
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 6470
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 6530
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 6531
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 6549
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 7337
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 7346
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 7349
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 7350
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 9145
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 9146
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 9147
Dataset is not deletable (shared between multiple histories/libraries, at
least one is not deleted)
######### Processing dataset id: 9308
Marking as deleted: HistoryDatasetAssociation id 9291 (for dataset id
Deleting dataset id 9308
######### Processing dataset id: 9307
Marking as deleted: HistoryDatasetAssociation id 9290 (for dataset id
Deleting dataset id 9307
######### Processing dataset id: 9317
Marking as deleted: HistoryDatasetAssociation id 9300 (for dataset id
Deleting dataset id 9317
######### Processing dataset id: 9318
Marking as deleted: HistoryDatasetAssociation id 9301 (for dataset id
Deleting dataset id 9318
######### Processing dataset id: 9341
Marking as deleted: HistoryDatasetAssociation id 9324 (for dataset id
Deleting dataset id 9341
######### Processing dataset id: 9342
Marking as deleted: HistoryDatasetAssociation id 9325 (for dataset id
Deleting dataset id 9342
######### Processing dataset id: 9348
Marking as deleted: HistoryDatasetAssociation id 9331 (for dataset id
Deleting dataset id 9348
######### Processing dataset id: 9349
Marking as deleted: HistoryDatasetAssociation id 9332 (for dataset id
Deleting dataset id 9349
######### Processing dataset id: 9420
Marking as deleted: HistoryDatasetAssociation id 9406 (for dataset id
Deleting dataset id 9420
######### Processing dataset id: 9419
Marking as deleted: HistoryDatasetAssociation id 9405 (for dataset id
Deleting dataset id 9419
######### Processing dataset id: 9418
Marking as deleted: HistoryDatasetAssociation id 9404 (for dataset id
Deleting dataset id 9418
######### Processing dataset id: 9421
Marking as deleted: HistoryDatasetAssociation id 9407 (for dataset id
Deleting dataset id 9421
Examined 77 datasets, marked 12 datasets and 12 dataset instances (HDA)
as deleted
Total elapsed time: 2.08013796806
##########################################
Is this error normal ? If so, should I redirect output error of my cron to /dev/null ?
Or how can I repair that in database ?
-Regards
Remy.