learning pulsar for galaxy
Hello, We are working on a project in providing a Cloud based research platform for Health Informatics and Bioinformatics. Long story short, one of the components to be provided by the project is a Galaxy Server for our genomics researchers; and on the other hand, GWU also has a HPC system for high performance and high throughput computing. Our Galaxy Server for this project is hosing in the Cloud (Azure) and the HPC cluster is onsite at GW. To allow a galaxy server to utilize external compute power of a HPC, galaxy has provided job runner plugins of DRMAA for job schedulers as well as Pulsar. For our situation, the HPC is a remote system w.r.t. the Galaxy server in the cloud and thus there isn't a shared file system between them. It seems to me that Pulsar would be the appropriate solution for us. Am I on the right track? Any suggestions are much appreciated! Best, Adam --- Adam K. L. Wong, PhD. High Performance Computing Specialist for Genomics Division of Information Technology The George Washington University Email: aklwong@gwu.edu
Hi,
Hello,
We are working on a project in providing a Cloud based research platform for Health Informatics and Bioinformatics. Long story short, one of the components to be provided by the project is a Galaxy Server for our genomics researchers; and on the other hand, GWU also has a HPC system for high performance and high throughput computing.
Our Galaxy Server for this project is hosing in the Cloud (Azure) and the HPC cluster is onsite at GW. To allow a galaxy server to utilize external compute power of a HPC, galaxy has provided job runner plugins of DRMAA for job schedulers as well as Pulsar. For our situation, the HPC is a remote system w.r.t. the Galaxy server in the cloud and thus there isn't a shared file system between them. It seems to me that Pulsar would be the appropriate solution for us. Am I on the right track?
You are on the right track, indeed! Pulsar is what you are searching for :) Cheers, Bjoern
Any suggestions are much appreciated!
Best, Adam --- Adam K. L. Wong, PhD. High Performance Computing Specialist for Genomics Division of Information Technology The George Washington University Email: aklwong@gwu.edu <mailto:aklwong@gwu.edu>
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
I'm also interested in a similar setup. One detail that I'd be interested in hearing about is how to minimize the cost involved in shuttling data in & out of the cloud with this sort of setup. Dan On Wed, Oct 3, 2018 at 9:38 AM Björn Grüning <bjoern.gruening@gmail.com> wrote:
Hi,
Hello,
We are working on a project in providing a Cloud based research platform for Health Informatics and Bioinformatics. Long story short, one of the components to be provided by the project is a Galaxy Server for our genomics researchers; and on the other hand, GWU also has a HPC system for high performance and high throughput computing.
Our Galaxy Server for this project is hosing in the Cloud (Azure) and the HPC cluster is onsite at GW. To allow a galaxy server to utilize external compute power of a HPC, galaxy has provided job runner plugins of DRMAA for job schedulers as well as Pulsar. For our situation, the HPC is a remote system w.r.t. the Galaxy server in the cloud and thus there isn't a shared file system between them. It seems to me that Pulsar would be the appropriate solution for us. Am I on the right track?
You are on the right track, indeed! Pulsar is what you are searching for :)
Cheers, Bjoern
Any suggestions are much appreciated!
Best, Adam --- Adam K. L. Wong, PhD. High Performance Computing Specialist for Genomics Division of Information Technology The George Washington University Email: aklwong@gwu.edu <mailto:aklwong@gwu.edu>
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
participants (3)
-
Björn Grüning -
Dan Fornika -
Wong, Kai Leung