On Aug 7, 2013, at 7:46 AM, Jasper Jan Koehorst wrote:
Hello,
I have a very simple jobrunner script that allows only 1 job to be run as this galaxy is used for one thing in particular which consumes almost all resources.
To do this I set the following:
<plugins> <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="1"/> </plugins>This works, but is it possible to create a second worker that is only allowed to be used for uploading files? So people can upload their data already and only enter the queue for the big analysis? Otherwise if they turn their computer off in the evening the upload is gone...
Hi Jasper,
Even if an upload job can't run because it's waiting for a free worker, that should not cause the upload to fail as long as the browser finishes transferring the file. Once the file is transferred, the upload job will simply wait in the queue until the worker slot is available.
Regardless, you should be able to load the local plugin twice, and utilize tool-to-destination mapping and concurrency limits to do what you want, e.g.:
<plugins> <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="1"/> <plugin id="local_upload" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="4"/> </plugins> <destinations default="non_upload"> <destination id="non_upload" runner="local" /> <destination id="upload" runner="local_upload" /> </destinations> <tools> <tool id="upload1" destination="upload" /> </tools> <limits> <limit type="concurrent_jobs" id="non_upload">1</limit> <limit type="concurrent_jobs" id="upload">1</limit> </limits>
The extra workers on the upload plugin would allow multiple users' uploads to be processed at once.
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/