That did it! Thanks Nate!
From:
Nate Coraor <nate@bx.psu.edu>
To:
Liisa Koski <liisa.koski@basf.com>
Cc:
galaxy-dev@lists.bx.psu.edu
Date:
12/11/2012 12:13 PM
Subject:
Re: [galaxy-dev]
Error when running cleanup_datasets.py
On Nov 8, 2012, at 8:55 PM, Liisa Koski wrote:
> Hi Nate,
> I'm back to trying to figure this out again as I am running out of
disk space. I added the bit of code you suggested below, but I don't think
it helped, I'm not so familiar with python.
>
> I'm now running Galaxy Reports and it tells me that I have 7479 datasets
that were deleted but have not yet been purged.
>
> I get the error below when I run cleanup_datasets.py with both the
-5 and -4 flag
>
> Marking as deleted: LibraryDatasetDatasetAssociation id 6907 (for
dataset id 51991)
> Deleting dataset id 51991
> Deleting library dataset id 7225
> Traceback (most recent call last):
> File "scripts/cleanup_datasets/cleanup_datasets.py",
line 526, in <module>
> if __name__ == "__main__": main()
> File "scripts/cleanup_datasets/cleanup_datasets.py",
line 124, in main
> purge_folders( app, cutoff_time, options.remove_from_disk,
info_only = options.info_only, force_retry = options.force_retry )
> File "scripts/cleanup_datasets/cleanup_datasets.py",
line 247, in purge_folders
> _purge_folder( folder, app, remove_from_disk, info_only
= info_only )
> File "scripts/cleanup_datasets/cleanup_datasets.py",
line 499, in _purge_folder
> _purge_folder( sub_folder, app, remove_from_disk, info_only
= info_only )
> File "scripts/cleanup_datasets/cleanup_datasets.py",
line 499, in _purge_folder
> _purge_folder( sub_folder, app, remove_from_disk, info_only
= info_only )
> File "scripts/cleanup_datasets/cleanup_datasets.py",
line 497, in _purge_folder
> _purge_dataset_instance( ldda, app, remove_from_disk,
info_only = info_only ) #mark a DatasetInstance as deleted, clear associated
files, and mark the Dataset as deleted if it is deletable
> File "scripts/cleanup_datasets/cleanup_datasets.py",
line 373, in _purge_dataset_instance
> log.debug( '%s %s has None dataset' % ( type( dataset_instance
), dataset_instance.id ) )
> AttributeError: 'NoneType' object has no attribute 'id'
Ah, I was looking at the wrong level. You have a library_dataset
without an associated library_dataset_dataset_association. The following
SQL should return it and any others:
select id from library_dataset where library_dataset_dataset_association_id
is null
Although the output indicates that the offending library_dataset id should
be 7225. The easiest way to solve this problem is probably to orphan
the broken library dataset, e.g.:
update library_dataset set folder_id = null where id=7225;
--nate
>
> Thanks in advance for your help,
> Liisa
>
>
>
>
>
> From: Nate Coraor <nate@bx.psu.edu>
> To: Liisa Koski <liisa.koski@basf.com>
> Cc: galaxy-dev@lists.bx.psu.edu
> Date: 02/10/2012 10:50 AM
> Subject: Re: [galaxy-dev] Error when running
cleanup_datasets.py
>
>
>
> On Oct 2, 2012, at 10:44 AM, Liisa Koski wrote:
>
> > Hi Nate,
> > That select statement does not return anything :(
>
> Could you add a bit of debugging to the script to see what the id
is of the dataset_instance that has a None dataset?
>
> if dataset_instance is None:
> log.debug( '%s %s has None dataset' % ( type( dataset_instance
), dataset_instance.id ) )
>
> Thanks,
> --nate
>
> >
> > Thanks,
> > Liisa
> >
> >
> >
> >
> >
> > From: Nate Coraor <nate@bx.psu.edu>
> > To: Liisa Koski <liisa.koski@basf.com>
> > Cc: galaxy-dev@lists.bx.psu.edu
> > Date: 01/10/2012 01:01 PM
> > Subject: Re: [galaxy-dev] Error when
running cleanup_datasets.py
> >
> >
> >
> > On Sep 24, 2012, at 10:41 AM, Liisa Koski wrote:
> >
> > > Hello,
> > > I am trying to run the cleanup scripts on my local installation
but get stuck when trying to run the following:
> > >
> > > ./scripts/cleanup_datasets/cleanup_datasets.py universe_wsgi.ini
-d 10 -5 -r
> > >
> > > Deleting library dataset id 7225
> > > Traceback (most recent call last):
> > > File "./scripts/cleanup_datasets/cleanup_datasets.py",
line 524, in <module>
> > > if __name__ == "__main__": main()
> > > File "./scripts/cleanup_datasets/cleanup_datasets.py",
line 124, in main
> > > purge_folders( app, cutoff_time, options.remove_from_disk,
info_only = options.info_only, force_retry = options.force_retry )
> > > File "./scripts/cleanup_datasets/cleanup_datasets.py",
line 247, in purge_folders
> > > _purge_folder( folder, app, remove_from_disk,
info_only = info_only )
> > > File "./scripts/cleanup_datasets/cleanup_datasets.py",
line 497, in _purge_folder
> > > _purge_folder( sub_folder, app, remove_from_disk,
info_only = info_only )
> > > File "./scripts/cleanup_datasets/cleanup_datasets.py",
line 497, in _purge_folder
> > > _purge_folder( sub_folder, app, remove_from_disk,
info_only = info_only )
> > > File "./scripts/cleanup_datasets/cleanup_datasets.py",
line 495, in _purge_folder
> > > _purge_dataset_instance( ldda, app, remove_from_disk,
info_only = info_only ) #mark a DatasetInstance as deleted, clear associated
files, and mark the Dataset as deleted if it is deletable
> > > File "./scripts/cleanup_datasets/cleanup_datasets.py",
line 376, in _purge_dataset_instance
> > > ( dataset_instance.__class__.__name__, dataset_instance.id,
dataset_instance.dataset.id )
> > > AttributeError: 'NoneType' object has no attribute 'id'
> >
> > Hi Liisa,
> >
> > It'd appear that you have a library_dataset_dataset_association
in your dataset that lacks an associated dataset. Does 'select id
from library_dataset_dataset_association where dataset_id is null' in your
database return anything?
> >
> > --nate
> >
> > >
> > >
> > > Any help would be much appreciated.
> > >
> > > Thanks,
> > > Liisa
> > > ___________________________________________________________
> > > Please keep all replies on the list by using "reply
all"
> > > in your mail client. To manage your subscriptions
to this
> > > and other Galaxy lists, please use the interface at:
> > >
> > > http://lists.bx.psu.edu/
> >
> >
>
>