Galaxy on HPC and Bright Cluster Manager?
Good day everyone, Has anyone of you been able to implement Galaxy on a HPC using Bright Cluster Manager as the main DRM? I noticed that only a few have been known to work with Galaxy, but the list does not include Bright. Any advice/ideas will be greatly appreciated. TORQUE Resource Manager PBS Professional Open Grid Engine Univa Grid Engine (previously known as Sun Grid Engine and Oracle Grid Engine) Platform LSF HTCondor Slurm Galaxy Pulsar (formerly LWR) Thanks. Carlo
Hello Carlos, I have never heard of anyone running Galaxy with Bright Cluster Manager (though hopefully someone will chime in if they have). If you are interested in adding support it should be possible. One complication is that Bright Cluster Manager doesn't appear to have a DRMAA interface (http://www.drmaa.org/) which is the most direct way to utilize new DRMs. Without that my approach would be to build a new CLI runner: There are a few examples here that one can use as template: https://github.com/galaxyproject/galaxy/tree/dev/lib/galaxy/jobs/runners/uti... I guess you would have to write a new one targeting cmsub I guess - you also need to be able to parse a job status somehow - I haven't figured out how to do that from the documentation - but I assume there is a way. I looks like Bright supports running SGE, SLURM, and Torque on the cluster - doing this and interfacing with one of those more common options directly might be a better approach for Galaxy (and other users if your cluster has them). -John On Wed, Apr 22, 2015 at 8:56 AM, Carlos Lijeron <clijeron@hunter.cuny.edu> wrote:
Good day everyone,
Has anyone of you been able to implement Galaxy on a HPC using Bright Cluster Manager as the main DRM? I noticed that only a few have been known to work with Galaxy, but the list does not include Bright. Any advice/ideas will be greatly appreciated.
TORQUE Resource Manager
PBS Professional
Open Grid Engine
Univa Grid Engine (previously known as Sun Grid Engine and Oracle Grid Engine)
Platform LSF
HTCondor
Slurm
Galaxy Pulsar (formerly LWR)
Thanks.
Carlo
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Carlo, We have Bright Cluster Manager in use on our cluster for node provisioning etc. but the actual job scheduler in use in our case is SLURM, which we use directly. Are you using one of the integrated workload managers such as SLURM / SGE / TORQUE directly, or indirectly via cmsub? I guess the easiest way to come up with some kind of advice is if you can provide an example of generic job script you are using on your system. If you're using cmsub is it specifying a --wlmanager etc. DT -----Original Message----- From: galaxy-dev [mailto:galaxy-dev-bounces@lists.galaxyproject.org] On Behalf Of John Chilton Sent: Wednesday, April 22, 2015 8:26 AM To: Carlos Lijeron Cc: galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Galaxy on HPC and Bright Cluster Manager? Hello Carlos, I have never heard of anyone running Galaxy with Bright Cluster Manager (though hopefully someone will chime in if they have). If you are interested in adding support it should be possible. One complication is that Bright Cluster Manager doesn't appear to have a DRMAA interface (http://www.drmaa.org/) which is the most direct way to utilize new DRMs. Without that my approach would be to build a new CLI runner: There are a few examples here that one can use as template: https://github.com/galaxyproject/galaxy/tree/dev/lib/galaxy/jobs/runners/uti... I guess you would have to write a new one targeting cmsub I guess - you also need to be able to parse a job status somehow - I haven't figured out how to do that from the documentation - but I assume there is a way. I looks like Bright supports running SGE, SLURM, and Torque on the cluster - doing this and interfacing with one of those more common options directly might be a better approach for Galaxy (and other users if your cluster has them). -John On Wed, Apr 22, 2015 at 8:56 AM, Carlos Lijeron <clijeron@hunter.cuny.edu> wrote:
Good day everyone,
Has anyone of you been able to implement Galaxy on a HPC using Bright Cluster Manager as the main DRM? I noticed that only a few have been known to work with Galaxy, but the list does not include Bright. Any advice/ideas will be greatly appreciated.
TORQUE Resource Manager
PBS Professional
Open Grid Engine
Univa Grid Engine (previously known as Sun Grid Engine and Oracle Grid Engine)
Platform LSF
HTCondor
Slurm
Galaxy Pulsar (formerly LWR)
Thanks.
Carlo
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ________________________________ UT Southwestern Medical Center The future of medicine, today.
participants (3)
-
Carlos Lijeron
-
David Trudgian
-
John Chilton