Hello everyone, We are soon acquiring our cluster which has CentOS on the master node along with Bright Cluster Manager and workload manager. We are wondering if there is a guideline on how to install and support Galaxy on CentOS. Does anyone have any info on this? Thanks ! Carlos.
Hi Carlos, We're running Galaxy on CentOS 6.6, so that in itself shouldn't be a problem. Most of the effort was sorting out shared storage with the cluster, which in our case is managed with SGE (fairly commonly used with Galaxy). In reply to your thread last month David Trudgian said he was using Bright Cluster Manager: http://dev.list.galaxyproject.org/Galaxy-on-HPC-and-Bright-Cluster-Manager-t... Regards, Peter On Tue, May 5, 2015 at 2:51 AM, Carlos Lijeron <clijeron@hunter.cuny.edu> wrote:
Hello everyone,
We are soon acquiring our cluster which has CentOS on the master node along with Bright Cluster Manager and workload manager. We are wondering if there is a guideline on how to install and support Galaxy on CentOS. Does anyone have any info on this?
Thanks !
Carlos.
Hi Carlos, Our Galaxy is running on RedHat 6.6 on our cluster - which is basically identical to CentOS 6.6. Per Peter's email we haven't had any big issues with Galaxy on CentOS 6.x. You are likely to want to use the postgresql packages from www.postrgresql.org, not the out-of-date versions that come with the distribution. I also installed an up-to-date python 2.7.x for Galaxy using pyenv but this shouldn't be necessary - Galaxy should work fine on python 2.6. In my case I did it to have a standard setup with other apps that do need python 2.7. Which workload manager will your cluster manager be configured to use? DT -----Original Message----- From: galaxy-dev [mailto:galaxy-dev-bounces@lists.galaxyproject.org] On Behalf Of Peter Cock Sent: Tuesday, May 05, 2015 3:13 AM To: Carlos Lijeron Cc: galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Galaxy on CentOS? Hi Carlos, We're running Galaxy on CentOS 6.6, so that in itself shouldn't be a problem. Most of the effort was sorting out shared storage with the cluster, which in our case is managed with SGE (fairly commonly used with Galaxy). In reply to your thread last month David Trudgian said he was using Bright Cluster Manager: http://dev.list.galaxyproject.org/Galaxy-on-HPC-and-Bright-Cluster-Manager-t... Regards, Peter On Tue, May 5, 2015 at 2:51 AM, Carlos Lijeron <clijeron@hunter.cuny.edu> wrote:
Hello everyone,
We are soon acquiring our cluster which has CentOS on the master node along with Bright Cluster Manager and workload manager. We are wondering if there is a guideline on how to install and support Galaxy on CentOS. Does anyone have any info on this?
Thanks !
Carlos.
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ________________________________ UT Southwestern Medical Center The future of medicine, today.
David, We are in our final configuration stage and tomorrow we have a conversation with Silicon Mechanics business integration team to discuss the workload manager, and other details about the data center backbone speed. As per your suggestion I inquired about using SLURM instead of SGI, but I haven¹t been able to make a good case as to why this is. My only support is your email / suggestion and your existing experience with an HPC, Bright and SLURM. My inexperience with this type of setups and lack of personnel at our institution makes me rely 100% of everyone's feedback in this group Thank you David and anyone else in the group for your contributions and suggestions. Attached is the diagram of our future implementation. Another concern that I have is that our sequencer will be located 0.5 miles from the HPC, but there is a 10 Gpbs point to point connection between the 2 buildings. I wanted to find a way to calculate / predict the average amount of traffic that will go from the sequencer to the HPC so we can allocate specific bandwidth for our use and leave the rest to other users. Carlos. On 5/5/15, 10:24 AM, "David Trudgian" <David.Trudgian@UTSouthwestern.edu> wrote:
Our Galaxy is running on RedHat 6.6 on our cluster - which is basically identical to CentOS 6.6. Per Peter's email we haven't had any big issues with Galaxy on CentOS 6.x.
You are likely to want to use the postgresql packages from www.postrgresql.org, not the out-of-date versions that come with the distribution. I also installed an up-to-date python 2.7.x for Galaxy using pyenv but this shouldn't be necessary - Galaxy should work fine on python 2.6. In my case I did it to have a standard setup with other apps that do need python 2.7.
Which workload manager will your cluster manager be configured to use?
DT
-----Original Message----- From: galaxy-dev [mailto:galaxy-dev-bounces@lists.galaxyproject.org] On Behalf Of Peter Cock Sent: Tuesday, May 05, 2015 3:13 AM To: Carlos Lijeron Cc: galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Galaxy on CentOS?
Hi Carlos,
We're running Galaxy on CentOS 6.6, so that in itself shouldn't be a problem. Most of the effort was sorting out shared storage with the cluster, which in our case is managed with SGE (fairly commonly used with Galaxy).
In reply to your thread last month David Trudgian said he was using Bright Cluster Manager:
http://dev.list.galaxyproject.org/Galaxy-on-HPC-and-Bright-Cluster-Manager -tc4667015.html
Regards,
Peter
Hi Carlos, I previously used Galaxy with a GridEngine cluster. Having GridEngine is no more complex than having SLURM. R.E. you other email, recording cluster usage for labs' galaxy jobs, that requires the 'running jobs as real user' setup... https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#Submitting_J... There are caveats with that setup r.e. file permissions / privacy of data between groups. In our situation everything has to be kept private, so to ensure jobs run as a real cluster user we have a complex setup using Galaxy pulsar as well as SLURM. Some notes on that in a GitHub issue here: https://github.com/galaxyproject/pulsar/issues/65 Cheers, DT -----Original Message----- From: Carlos Lijeron [mailto:clijeron@hunter.cuny.edu] Sent: Tuesday, May 05, 2015 11:18 AM To: David Trudgian; Peter Cock Cc: galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Galaxy on CentOS? David, We are in our final configuration stage and tomorrow we have a conversation with Silicon Mechanics business integration team to discuss the workload manager, and other details about the data center backbone speed. As per your suggestion I inquired about using SLURM instead of SGI, but I haven¹t been able to make a good case as to why this is. My only support is your email / suggestion and your existing experience with an HPC, Bright and SLURM. My inexperience with this type of setups and lack of personnel at our institution makes me rely 100% of everyone's feedback in this group Thank you David and anyone else in the group for your contributions and suggestions. Attached is the diagram of our future implementation. Another concern that I have is that our sequencer will be located 0.5 miles from the HPC, but there is a 10 Gpbs point to point connection between the 2 buildings. I wanted to find a way to calculate / predict the average amount of traffic that will go from the sequencer to the HPC so we can allocate specific bandwidth for our use and leave the rest to other users. Carlos. On 5/5/15, 10:24 AM, "David Trudgian" <David.Trudgian@UTSouthwestern.edu> wrote:
Our Galaxy is running on RedHat 6.6 on our cluster - which is basically identical to CentOS 6.6. Per Peter's email we haven't had any big issues with Galaxy on CentOS 6.x.
You are likely to want to use the postgresql packages from www.postrgresql.org, not the out-of-date versions that come with the distribution. I also installed an up-to-date python 2.7.x for Galaxy using pyenv but this shouldn't be necessary - Galaxy should work fine on python 2.6. In my case I did it to have a standard setup with other apps that do need python 2.7.
Which workload manager will your cluster manager be configured to use?
DT
-----Original Message----- From: galaxy-dev [mailto:galaxy-dev-bounces@lists.galaxyproject.org] On Behalf Of Peter Cock Sent: Tuesday, May 05, 2015 3:13 AM To: Carlos Lijeron Cc: galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Galaxy on CentOS?
Hi Carlos,
We're running Galaxy on CentOS 6.6, so that in itself shouldn't be a problem. Most of the effort was sorting out shared storage with the cluster, which in our case is managed with SGE (fairly commonly used with Galaxy).
In reply to your thread last month David Trudgian said he was using Bright Cluster Manager:
http://dev.list.galaxyproject.org/Galaxy-on-HPC-and-Bright-Cluster-Mana ger -tc4667015.html
Regards,
Peter
________________________________ UT Southwestern Medical Center The future of medicine, today.
On Tue, May 5, 2015 at 10:24 AM, David Trudgian <David.Trudgian@utsouthwestern.edu> wrote:
I also installed an up-to-date python 2.7.x for Galaxy using pyenv but this shouldn't be necessary - Galaxy should work fine on python 2.6.
I would strongly suggest using the latest Python 2.7 for any new Galaxy deployment. Though there is no plan to right now, inevitably we will have to deprecate 2.7 support. In particular, if we want to do a Python 2 and 3 compatible Galaxy we would need to use 2.7 features. -- James Taylor Ralph S. O'Connor Associate Professor of Biology Associate Professor of Computer Science Johns Hopkins University
Thank you Peter, I am now getting a clear idea on how our deployment will go. Yes, I followed up with David and he provided additional insights into his deployment. What a great virtual team !!! Thank you. Carlos On 5/5/15, 4:12 AM, "Peter Cock" <p.j.a.cock@googlemail.com> wrote:
Hi Carlos,
We're running Galaxy on CentOS 6.6, so that in itself shouldn't be a problem. Most of the effort was sorting out shared storage with the cluster, which in our case is managed with SGE (fairly commonly used with Galaxy).
In reply to your thread last month David Trudgian said he was using Bright Cluster Manager:
http://dev.list.galaxyproject.org/Galaxy-on-HPC-and-Bright-Cluster-Manager -tc4667015.html
Regards,
Peter
On Tue, May 5, 2015 at 2:51 AM, Carlos Lijeron <clijeron@hunter.cuny.edu> wrote:
Hello everyone,
We are soon acquiring our cluster which has CentOS on the master node along with Bright Cluster Manager and workload manager. We are wondering if there is a guideline on how to install and support Galaxy on CentOS. Does anyone have any info on this?
Thanks !
Carlos.
Dnia 2015-05-05, wto o godzinie 01:51 +0000, Carlos Lijeron pisze:
Hello everyone,
We are soon acquiring our cluster which has CentOS on the master node along with Bright Cluster Manager and workload manager. We are wondering if there is a guideline on how to install and support Galaxy on CentOS. Does anyone have any info on this?
I am running Galaxy on Centos 6.6. But we are considering to move to Fedora Server. Of course it is possible to run Galaxy on Centos, however I had to compile python and several other packets to make all tools to work. -- Pozdrawiam, -- Zuzanna K. Filutowska, HPC Systems Administrator Poznan Supercomputing and Networking Center Institute of Bioorganic Chemistry Polish Academy of Sciences Seize the day boys! Make your lifes extraordinary! --John Keating
Thank you for the feedback Zuzanna, Would you have documentation on scripts your updated or any lessons learned during your deployment? I¹m trying to get as much feedback as possible to hit the ground running when the cluster is delivered in our facility. One more question, do you use SGI or SLURM for the workload manager? Thanks again ! Carlos. On 5/5/15, 9:41 AM, "Zuzanna K. Filutowska" <platyna@man.poznan.pl> wrote:
wondering if there is a guideline on how to install and support Galaxy on CentOS. Does anyone have any info on this?
I am running Galaxy on Centos 6.6. But we are considering to move to Fedora Server. Of course it is possible to run Galaxy on Centos, however I had to compile python and several other packets to make all tools to work.
participants (5)
-
Carlos Lijeron -
David Trudgian -
James Taylor -
Peter Cock -
Zuzanna K. Filutowska