Hello list,
We have some problems with the purging function.
Here are our errors :
find: `/share/work-galaxy/cur/004/4455': No such file or directory
find: `/share/work-galaxy/cur/004/4459': No such file or directory
find: `/share/work-galaxy/cur/004/4462': No such file or directory
find: `/share/work-galaxy/cur/004/4458': No such file or directory
find: `/share/work-galaxy/cur/004/4460': No such file or directory
find: `/share/work-galaxy/cur/004/4467': No such file or directory
find: `/share/work-galaxy/cur/004/4469': No such file or directory
find: `/share/work-galaxy/cur/004/4463': No such file or directory
find: `/share/work-galaxy/cur/004/4468': No such file or directory
find: `/share/work-galaxy/cur/004/4466': No such file or directory
find: `/share/work-galaxy/cur/004/4464': No such file or directory
find: `/share/work-galaxy/cur/004/4470': No such file or directory
find: `/share/work-galaxy/cur/004/4461': No such file or directory
We have that kind of errors by mail every days with postfix. But each times
it is for the folder /share/work-galaxy/cur/004/.
These problems appears 1 month ago. We do not know why. The only specific
configuration is that the database was a restoration from a previous galaxy
server. It occurs for differents subfolders in /share/work-galaxy/cur/004/.
In database, I do not find anything suspect, but on my linux system, these
folders missing like the error message said.
The cron job is
"sh /home/igalaxy/cron/purging_galaxy.sh 1 > /home/igalaxy/log/purge.log"
And purging.sh :
> cat /home/igalaxy/cron/purging_galaxy.sh
> #!/bin/bash
> # purge galaxy datasets, histories, folders, libraires
> # not updated since $day
> day=$1
> cd /home/igalaxy/galaxy-dist
> python scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini -d
> $day -1
> python scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini -d
> $day -2 -r
> python scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini -d
> $day -3 -r
> python scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini -d
> $day -5 -r
> python scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini -d
> $day -4 -r
> python scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini -d
> $day -6 -r
> find /share/work-galaxy/tmp/ -mtime +30 -exec rm -rf "{}" \;
> find /share/work-galaxy/cur/ -mtime +30 -exec rm -rf "{}" \;
> find /share/work-galaxy/out/ -mtime +30 -exec rm -rf "{}" \;
The "cur" directory is the job_working_directory :
> grep "cur" /home/igalaxy/galaxy-dist/universe_wsgi.ini |grep -v "#"
>
> job_working_directory = /share/work-galaxy/cur
>
>
The purge log content is the following :
cat /home/igalaxy/log/purge.log
>
> MySQL_python egg successfully loaded for mysql dialect
>
> ##########################################
>
>
>> # 2012-03-30 01:00:02 - Handling stuff older than 1 days
>
> Datasets will NOT be removed from disk.
>
>
>> Deleted 0 histories
>
> Elapsed time: 0.310899019241
>
> ##########################################
>
> MySQL_python egg successfully loaded for mysql dialect
>
> ##########################################
>
>
>> # 2012-03-30 01:00:04 - Handling stuff older than 1 days
>
> Datasets will be removed from disk.
>
>
>> Purged 0 histories.
>
> Elapsed time: 0.31987786293
>
> ##########################################
>
> MySQL_python egg successfully loaded for mysql dialect
>
> ##########################################
>
>
>> # 2012-03-30 01:00:05 - Handling stuff older than 1 days
>
> Datasets will be removed from disk.
>
>
>> Removing disk, file /share/work-galaxy/files/009/dataset_9110.dat
>
> Purging dataset id 9110
>
> Removing disk, file /share/work-galaxy/files/009/dataset_9107.dat
>
> Purging dataset id 9107
>
> Removing disk, file /share/work-galaxy/files/009/dataset_9108.dat
>
> Purging dataset id 9108
>
> Removing disk, file /share/work-galaxy/files/009/dataset_9137.dat
>
> Purging dataset id 9137
>
> Removing disk, file /share/work-galaxy/files/009/dataset_9136.dat
>
> Purging dataset id 9136
>
> Removing disk, file /share/work-galaxy/files/009/dataset_9135.dat
>
> Purging dataset id 9135
>
> Purged 6 datasets
>
> Freed disk space: 44272
>
> Elapsed time: 0.6712641716
>
> ##########################################
>
> MySQL_python egg successfully loaded for mysql dialect
>
> ##########################################
>
>
>> # 2012-03-30 01:00:07 - Handling stuff older than 1 days
>
> Datasets will be removed from disk.
>
>
>> # Purged 0 folders.
>
> Elapsed time: 0.310595035553
>
> ##########################################
>
> MySQL_python egg successfully loaded for mysql dialect
>
> ##########################################
>
>
>> # 2012-03-30 01:00:09 - Handling stuff older than 1 days
>
> Datasets will be removed from disk.
>
>
>> # Purged 0 libraries .
>
> Elapsed time: 0.308594942093
>
> ##########################################
>
> MySQL_python egg successfully loaded for mysql dialect
>
> ##########################################
>
>
>> # 2012-03-30 01:00:10 - Handling stuff older than 1 days
>
> Datasets will be removed from disk.
>
>
>> ######### Processing dataset id: 1590
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 1591
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 1592
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 1593
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 1594
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 1595
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 1596
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 1597
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 1598
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 1599
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 1600
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 1601
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 1602
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 1603
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 1604
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 1605
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 1606
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 1607
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 1608
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 1609
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 1610
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 1611
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 1612
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 1613
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 1614
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 1615
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 2997
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 3008
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 3201
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 3247
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 3447
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 3448
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 3586
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 3691
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 4522
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 4524
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 4525
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 4526
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 4581
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 4582
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 4583
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 4584
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 4585
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 4586
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 4587
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 4588
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 4589
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 4590
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 4591
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 4592
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 6123
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 6124
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 6266
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 6434
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 6470
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 6530
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 6531
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 6549
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 7337
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 7346
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 7349
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 7350
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 9145
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 9146
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 9147
>
> Dataset is not deletable (shared between multiple histories/libraries, at
>> least one is not deleted)
>
> ######### Processing dataset id: 9308
>
> Marking as deleted: HistoryDatasetAssociation id 9291 (for dataset id 9308)
>
> Deleting dataset id 9308
>
> ######### Processing dataset id: 9307
>
> Marking as deleted: HistoryDatasetAssociation id 9290 (for dataset id 9307)
>
> Deleting dataset id 9307
>
> ######### Processing dataset id: 9317
>
> Marking as deleted: HistoryDatasetAssociation id 9300 (for dataset id 9317)
>
> Deleting dataset id 9317
>
> ######### Processing dataset id: 9318
>
> Marking as deleted: HistoryDatasetAssociation id 9301 (for dataset id 9318)
>
> Deleting dataset id 9318
>
> ######### Processing dataset id: 9341
>
> Marking as deleted: HistoryDatasetAssociation id 9324 (for dataset id 9341)
>
> Deleting dataset id 9341
>
> ######### Processing dataset id: 9342
>
> Marking as deleted: HistoryDatasetAssociation id 9325 (for dataset id 9342)
>
> Deleting dataset id 9342
>
> ######### Processing dataset id: 9348
>
> Marking as deleted: HistoryDatasetAssociation id 9331 (for dataset id 9348)
>
> Deleting dataset id 9348
>
> ######### Processing dataset id: 9349
>
> Marking as deleted: HistoryDatasetAssociation id 9332 (for dataset id 9349)
>
> Deleting dataset id 9349
>
> ######### Processing dataset id: 9420
>
> Marking as deleted: HistoryDatasetAssociation id 9406 (for dataset id 9420)
>
> Deleting dataset id 9420
>
> ######### Processing dataset id: 9419
>
> Marking as deleted: HistoryDatasetAssociation id 9405 (for dataset id 9419)
>
> Deleting dataset id 9419
>
> ######### Processing dataset id: 9418
>
> Marking as deleted: HistoryDatasetAssociation id 9404 (for dataset id 9418)
>
> Deleting dataset id 9418
>
> ######### Processing dataset id: 9421
>
> Marking as deleted: HistoryDatasetAssociation id 9407 (for dataset id 9421)
>
> Deleting dataset id 9421
>
> Examined 77 datasets, marked 12 datasets and 12 dataset instances (HDA) as
>> deleted
>
> Total elapsed time: 2.08013796806
>
> ##########################################
>
>
>
Is this error normal ? If so, should I redirect output error of my cron to
/dev/null ?
Or how can I repair that in database ?
-Regards
Remy.
