bioblend run_workflow (no_wait?)
I'm using bioblend to launch a workflow, but i find that the gi.workflows.run_workflow() command does not provide an option for no_wait. Can anyone provide justification for why this needs to be synchronous? -Evan Bollig Research Associate | Application Developer | User Support Consultant Minnesota Supercomputing Institute 599 Walter Library 612 624 1447 evan@msi.umn.edu boll0107@umn.edu
There is no reason workflow invocation *has* to be synchronous, but that's how galaxy works right now as, mostly, a historical artifact. Workflow scheduling happens inside the web request, and other than bioblend issuing the request in another thread there won't be an implementation w/ no_wait. This is definitely suboptimal, and backgrounded (among other enhancements to scheduling) workflow invocation is something we're working on right now, and it's a high priority. On Thu, Aug 21, 2014 at 4:12 PM, Evan Bollig <boll0107@umn.edu> wrote:
I'm using bioblend to launch a workflow, but i find that the gi.workflows.run_workflow() command does not provide an option for no_wait.
Can anyone provide justification for why this needs to be synchronous?
-Evan Bollig Research Associate | Application Developer | User Support Consultant Minnesota Supercomputing Institute 599 Walter Library 612 624 1447 evan@msi.umn.edu boll0107@umn.edu ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Thanks Dannon. Looking forward to the resolution. We've got some long workflows (100+ steps) that take days to compute. It takes about 2 minutes to finish queuing tasks, and then I found that they continue to execute even if I interrupt the bioblend thread. I'll add a signal.alarm interrupt to kill the bioblend thread and continue manual monitoring. -E -Evan Bollig Research Associate | Application Developer | User Support Consultant Minnesota Supercomputing Institute 599 Walter Library 612 624 1447 evan@msi.umn.edu boll0107@umn.edu On Thu, Aug 21, 2014 at 4:20 PM, Dannon Baker <dannon.baker@gmail.com> wrote:
There is no reason workflow invocation *has* to be synchronous, but that's how galaxy works right now as, mostly, a historical artifact. Workflow scheduling happens inside the web request, and other than bioblend issuing the request in another thread there won't be an implementation w/ no_wait. This is definitely suboptimal, and backgrounded (among other enhancements to scheduling) workflow invocation is something we're working on right now, and it's a high priority.
On Thu, Aug 21, 2014 at 4:12 PM, Evan Bollig <boll0107@umn.edu> wrote:
I'm using bioblend to launch a workflow, but i find that the gi.workflows.run_workflow() command does not provide an option for no_wait.
Can anyone provide justification for why this needs to be synchronous?
-Evan Bollig Research Associate | Application Developer | User Support Consultant Minnesota Supercomputing Institute 599 Walter Library 612 624 1447 evan@msi.umn.edu boll0107@umn.edu ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
So, see the commentary on the bioblend issue you added as well, but in short -- while you should currently expect to wait while the entire workflow was queued, bioblend *should not* be blocking waiting for all jobs to complete and that sounds like a bioblend bug if it is. That's what the 'wait' param is for after all. On Thu, Aug 21, 2014 at 5:27 PM, Evan Bollig <boll0107@umn.edu> wrote:
Thanks Dannon. Looking forward to the resolution. We've got some long workflows (100+ steps) that take days to compute. It takes about 2 minutes to finish queuing tasks, and then I found that they continue to execute even if I interrupt the bioblend thread. I'll add a signal.alarm interrupt to kill the bioblend thread and continue manual monitoring.
-E -Evan Bollig Research Associate | Application Developer | User Support Consultant Minnesota Supercomputing Institute 599 Walter Library 612 624 1447 evan@msi.umn.edu boll0107@umn.edu
There is no reason workflow invocation *has* to be synchronous, but
On Thu, Aug 21, 2014 at 4:20 PM, Dannon Baker <dannon.baker@gmail.com> wrote: that's
how galaxy works right now as, mostly, a historical artifact. Workflow scheduling happens inside the web request, and other than bioblend issuing the request in another thread there won't be an implementation w/ no_wait. This is definitely suboptimal, and backgrounded (among other enhancements to scheduling) workflow invocation is something we're working on right now, and it's a high priority.
On Thu, Aug 21, 2014 at 4:12 PM, Evan Bollig <boll0107@umn.edu> wrote:
I'm using bioblend to launch a workflow, but i find that the gi.workflows.run_workflow() command does not provide an option for no_wait.
Can anyone provide justification for why this needs to be synchronous?
-Evan Bollig Research Associate | Application Developer | User Support Consultant Minnesota Supercomputing Institute 599 Walter Library 612 624 1447 evan@msi.umn.edu boll0107@umn.edu ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
participants (2)
-
Dannon Baker -
Evan Bollig