14 Jun
2012
14 Jun
'12
3:43 p.m.
On Jun 14, 2012, at 11:37 AM, Peter Cock wrote:
My hunch is that all the child-jobs are created and added to the queue and some of this is happening in parallel leading to contention over the SQLite database. Does this sound likely?
Yep, this is exactly what's happening. It isn't just the job splitting code either, this happens in other circumstances as well.
Is this something that could be more robust in Galaxy, or should I simply switch to a real database (SQLite or PostreSQL) for these tests?
I've never taken a serious look at what it'd take to prevent this happening, but I use (and recommend) Postgres for any of hte parallelism work. -Dannon