I believe I may have found the bug...
Line 258 of cleanup_datasets.py is the query for library_dataset_ids when "force_retry" is set. While it correctly eliminates the the app.model.LibraryDataset.table.c.purged == False from the where clause, it incorrectly eliminates the app.model.LibraryDataset.table.c.deleted == True.
My suggested fix would be to correct that query by adding app.model.LibraryDataset.table.c.deleted == True to the where clause.
Cleaning up users databases might be a bit trickier, but I think you could find the affected library datasets by the following query:
select * from library_dataset where deleted=f and purged=t;
With those, I would think setting purged to f and then setting deleted = f for the associated library_dataset_dataset_association records.
Lance
Nate Coraor wrote:
Lance Parsons wrote:
Good catch, that does indeed seem to be the issue. And it only seems to happen with my older data libraries.
I didn't see a bug report for this, but I'd be happy to file one and/or take a stab at a fix if someone can point me in the right direction in the code.
Any community offers to debug/fix are always happily welcomed. =)
My reply to Alex:
See line 1925 in lib/galaxy/web/controllers/library_common.py. Try adding a bit of debugging to find out if hda.deleted == True when the copied HistoryDatasetAssociation is created, and whether the source LibraryDatasetDatasetAssociation is itself deleted (which should be the only way the resulting hda could be deleted).
I'll add that a key to figuring this out is probably is determining what's different about these older library items and the newer ones that don't exhibit the problem (if this is also true in your environment as it is in his).
If no answers turn up in a few days, then we can open an issue for it in Bitbucket.
--nate
Lance
Nate Coraor wrote:
Lance Parsons wrote:
I'm running a local instance of Galaxy that has been working quite well. However, I've recently run into a problem when importing datasets from a data library into a history. The Size on Disk of the history grows by the size of the imported dataset, however, the new dataset is not visible (and the number of datasets does not change in the saved histories view).
Any thoughts on what might be going on here? Thanks.
Hey Lance,
No real ideas, but Alex Bossers just posted up saying that old library datasets are importing into his histories as deleted. Could you check and see if this is the case in your instance as well?
Thanks, --nate
-- Lance Parsons - Scientific Programmer 134 Carl C. Icahn Laboratory Lewis-Sigler Institute for Integrative Genomics Princeton University
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
-- Lance Parsons - Scientific Programmer 134 Carl C. Icahn Laboratory Lewis-Sigler Institute for Integrative Genomics Princeton University