The web process is definitely the bottleneck right now, creating more should improve things. I'd guess it wouldn't speed up any individual submission, but it'd allow you to submit several at once. Lastly, this is a known issue that we're hoping to resolve with a rewrite of the workflow dispatching mechanism. -Dannon On Fri, Jun 14, 2013 at 2:17 PM, Thon Deboer <thondeboer@me.com> wrote:
Hi,****
** **
I am routinely running hundreds of pair-end sequencing runs and find that STARTING a workflow on all of these can easily take hours since it seems each workflow preparation takes about 3 minutes, so for doing 128 fastq pairs like this it will take over 3 hours just to get all workflows running….****
** **
Is there any way for me to speed up this process, maybe by adding more handlers (although I am suspecting the WEB server is the bottleneck here)?
I already run 8 handlers but only one web server…****
I could loadbalance the webservers through my proxy server…Will that speed up the submissions or are we stuck with this linear process?****
Thanks,****
** **
Thon****
** **
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/