On Thu, Feb 14, 2013 at 10:40 AM, Nate Coraor <nate@bx.psu.edu> wrote:
What I am facing, is that I have 100 FASTQ pairs or so, for a single flowcell, I can start the analysis of that set from the UI, but it will just crank through them and takes about 2-4 minutues for each pair to be processed, so with 100 pairs or so, you are looking 3-4 hours of an hourglass before control is given back to the user...
Is this in the context of starting a workflow with hundreds of inputs?
I believe this is entirely in the web process, creating a workflow requires creating tons of datasets, which requires tons of database flushes. I have a patch to fix this from a long time ago but it would need substantial testing. I think we should also background the process, but this depends on having a way to attach the workflow invocation to the history, hence dataset groups. All the pieces are coming together on this. -- James Taylor, Assistant Professor, Biology/CS, Emory University