Limit number of Jobs to Trinity
Hi guys, I have a running local galaxy installation on one of our servers. I've just installed Trinity for de novo assembly of transcripts and it will use up to 32 processors. However i would like to limit the number of concurrent jobs only for this tool. Do you know if it would be possible to limit the number of concurrent jobs per tool basis? Bests, Carlos -- Carlos Canchaya
Carlos, Are you using any sort of scheduler? SGE/Torque, etc? How are jobs being dispatched to your node(s)? More info here: http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebrenne@uci.edu On Sat, Sep 21, 2013 at 4:47 AM, Carlos Canchaya <ccanchaya@gmail.com>wrote:
Hi guys,
I have a running local galaxy installation on one of our servers. I've just installed Trinity for de novo assembly of transcripts and it will use up to 32 processors. However i would like to limit the number of concurrent jobs only for this tool. Do you know if it would be possible to limit the number of concurrent jobs per tool basis?
Bests,
Carlos
-- Carlos Canchaya
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hi adam, So far we have just a standalone server since we are still setting up our tools (we will scale up to our sge cluster, though). Any clue how to limit jobs per tool basis? Best, Carlos On Saturday, September 21, 2013, Adam Brenner wrote:
Carlos,
Are you using any sort of scheduler? SGE/Torque, etc? How are jobs being dispatched to your node(s)? More info here: http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster
-- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences
Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/
University of California, Irvine www.ics.uci.edu/~aebrenne/ aebrenne@uci.edu <javascript:_e({}, 'cvml', 'aebrenne@uci.edu');>
On Sat, Sep 21, 2013 at 4:47 AM, Carlos Canchaya <ccanchaya@gmail.com<javascript:_e({}, 'cvml', 'ccanchaya@gmail.com');>
wrote:
Hi guys,
I have a running local galaxy installation on one of our servers. I've just installed Trinity for de novo assembly of transcripts and it will use up to 32 processors. However i would like to limit the number of concurrent jobs only for this tool. Do you know if it would be possible to limit the number of concurrent jobs per tool basis?
Bests,
Carlos
-- Carlos Canchaya
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
-- Carlos A. Canchaya Universidade de Vigo
As your Galaxy instance becomes more production ready I would definitely recommend installing a job queue such as grid engine, Torque, or SLURM. For this simple use case though, you should be able t define multiple local job runner instances to ensure only one Trinity job runs a time. Your job_conf.xml should look something like this: <?xml version="1.0"?> <job_conf> <plugins> <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="4"/> <plugin id="local2" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="1"/> </plugins> <handlers> <handler id="main"/> </handlers> <destinations default="local"> <destination id="local" runner="local"/> <destination id="local2" runner="local2" /> </destinations> <tools> <tool id="trinity" destination="local2"/> </tools> </job_conf> Hope this helps. -John On Sat, Sep 21, 2013 at 10:40 AM, Carlos Canchaya <ccanchaya@gmail.com> wrote:
Hi adam,
So far we have just a standalone server since we are still setting up our tools (we will scale up to our sge cluster, though). Any clue how to limit jobs per tool basis?
Best,
Carlos
On Saturday, September 21, 2013, Adam Brenner wrote:
Carlos,
Are you using any sort of scheduler? SGE/Torque, etc? How are jobs being dispatched to your node(s)? More info here: http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster
-- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences
Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/
University of California, Irvine www.ics.uci.edu/~aebrenne/ aebrenne@uci.edu
On Sat, Sep 21, 2013 at 4:47 AM, Carlos Canchaya <ccanchaya@gmail.com> wrote:
Hi guys,
I have a running local galaxy installation on one of our servers. I've just installed Trinity for de novo assembly of transcripts and it will use up to 32 processors. However i would like to limit the number of concurrent jobs only for this tool. Do you know if it would be possible to limit the number of concurrent jobs per tool basis?
Bests,
Carlos
-- Carlos Canchaya
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
-- Carlos A. Canchaya Universidade de Vigo
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hi Carlos and John Please allow me to pick up this thread. I have been experimenting with this, but I cant get it to work. The default destination is always used. Does it work for you, Carlos? Regards, Hans-Rudolf On 09/23/2013 03:32 AM, John Chilton wrote:
As your Galaxy instance becomes more production ready I would definitely recommend installing a job queue such as grid engine, Torque, or SLURM. For this simple use case though, you should be able t define multiple local job runner instances to ensure only one Trinity job runs a time.
Your job_conf.xml should look something like this:
<?xml version="1.0"?> <job_conf> <plugins> <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="4"/> <plugin id="local2" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="1"/> </plugins> <handlers> <handler id="main"/> </handlers> <destinations default="local"> <destination id="local" runner="local"/> <destination id="local2" runner="local2" /> </destinations> <tools> <tool id="trinity" destination="local2"/> </tools> </job_conf>
Hope this helps.
-John
On Sat, Sep 21, 2013 at 10:40 AM, Carlos Canchaya <ccanchaya@gmail.com> wrote:
Hi adam,
So far we have just a standalone server since we are still setting up our tools (we will scale up to our sge cluster, though). Any clue how to limit jobs per tool basis?
Best,
Carlos
On Saturday, September 21, 2013, Adam Brenner wrote:
Carlos,
Are you using any sort of scheduler? SGE/Torque, etc? How are jobs being dispatched to your node(s)? More info here: http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster
-- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences
Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/
University of California, Irvine www.ics.uci.edu/~aebrenne/ aebrenne@uci.edu
On Sat, Sep 21, 2013 at 4:47 AM, Carlos Canchaya <ccanchaya@gmail.com> wrote:
Hi guys,
I have a running local galaxy installation on one of our servers. I've just installed Trinity for de novo assembly of transcripts and it will use up to 32 processors. However i would like to limit the number of concurrent jobs only for this tool. Do you know if it would be possible to limit the number of concurrent jobs per tool basis?
Bests,
Carlos
-- Carlos Canchaya
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
-- Carlos A. Canchaya Universidade de Vigo
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
well, I was just reading today's commits and I spotted: Nate's commit: "Bugfix for tool-to-destination mapping, tool ids are lowercased but the mapping id was not lowercased." - Changeset: 0b955e54451c So I changed: <tool id="addValue" destination="local2" /> to <tool id="addvalue" destination="local2" /> and it seems to work. Regards, Hans-Rudolf On 09/26/2013 05:43 PM, Hans-Rudolf Hotz wrote:
Hi Carlos and John
Please allow me to pick up this thread.
I have been experimenting with this, but I cant get it to work. The default destination is always used. Does it work for you, Carlos?
Regards, Hans-Rudolf
On 09/23/2013 03:32 AM, John Chilton wrote:
As your Galaxy instance becomes more production ready I would definitely recommend installing a job queue such as grid engine, Torque, or SLURM. For this simple use case though, you should be able t define multiple local job runner instances to ensure only one Trinity job runs a time.
Your job_conf.xml should look something like this:
<?xml version="1.0"?> <job_conf> <plugins> <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="4"/> <plugin id="local2" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="1"/> </plugins> <handlers> <handler id="main"/> </handlers> <destinations default="local"> <destination id="local" runner="local"/> <destination id="local2" runner="local2" /> </destinations> <tools> <tool id="trinity" destination="local2"/> </tools> </job_conf>
Hope this helps.
-John
On Sat, Sep 21, 2013 at 10:40 AM, Carlos Canchaya <ccanchaya@gmail.com> wrote:
Hi adam,
So far we have just a standalone server since we are still setting up our tools (we will scale up to our sge cluster, though). Any clue how to limit jobs per tool basis?
Best,
Carlos
On Saturday, September 21, 2013, Adam Brenner wrote:
Carlos,
Are you using any sort of scheduler? SGE/Torque, etc? How are jobs being dispatched to your node(s)? More info here: http://wiki.galaxyproject.org/Admin/Config/Performance/Cluster
-- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences
Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/
University of California, Irvine www.ics.uci.edu/~aebrenne/ aebrenne@uci.edu
On Sat, Sep 21, 2013 at 4:47 AM, Carlos Canchaya <ccanchaya@gmail.com> wrote:
Hi guys,
I have a running local galaxy installation on one of our servers. I've just installed Trinity for de novo assembly of transcripts and it will use up to 32 processors. However i would like to limit the number of concurrent jobs only for this tool. Do you know if it would be possible to limit the number of concurrent jobs per tool basis?
Bests,
Carlos
-- Carlos Canchaya
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
-- Carlos A. Canchaya Universidade de Vigo
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
participants (4)
-
Adam Brenner -
Carlos Canchaya -
Hans-Rudolf Hotz -
John Chilton