Resizing galaxy cloudman /mnt/galaxyData partition
Hi Team, I urgently need to resize the data partition on my galaxy cloudman instance without loosing anything. I notice that there a gui on the cloudman admin page that will allow me to increase it to 1000GB. If I use that will it restart all of the processes? I need to maintain this as a functional instance since it needs to be available for use tomorrow at 8:00 AM and I do not have the time to rebuild it. The gui says it will create a snapshot. The following is what it says: "Expand Disk Space Through this form you may increase the disk space available to Galaxy. All of the cluster services (but not the cluster) WILL BE STOPPED until the new disk is ready, at which point they will all be restarted. This may result in Galaxy jobs that are currently running to fail. Note that the new disk size must be larger than the current disk size. During this process, a snapshot of your data volume will be created, which can optionally be left in your account. If you decide to leave the snapshot for reference, you may also provide a brief note that will later be visible in the snapshot's description. New Disk Size (minimum 100GB, maximum 1000GB): OK Note (optional): or delete the created snapshot after filesystem resizing? If checked, the created snapshot will not be kept" What is the likelihood that this will be successful? Has anyone utilized it? Is there another way to resize the /mnt/galaxyData/ and /mnt/galaxyIndices partitions that will allow me to do what I need to? Thanks, Iry Witham The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.
Hi Iry, This should work, I use it regularly. Just make sure you're not ssh'ed in and 'in' /mnt/galaxyData or anything like that. There's not really another way to do this that I'd recommend trying if you need the instance to continue working for something tomorrow. Let me know if you have any issues and I can help. -Dannon On Mon, May 6, 2013 at 1:54 PM, Iry Witham <Iry.Witham@jax.org> wrote:
Hi Team,
I urgently need to resize the data partition on my galaxy cloudman instance without loosing anything. I notice that there a gui on the cloudman admin page that will allow me to increase it to 1000GB. If I use that will it restart all of the processes? I need to maintain this as a functional instance since it needs to be available for use tomorrow at 8:00 AM and I do not have the time to rebuild it. The gui says it will create a snapshot. The following is what it says:
"Expand Disk Space Through this form you may increase the disk space available to Galaxy. All of the cluster services (but not the cluster) WILL BE STOPPED until the new disk is ready, at which point they will all be restarted. This may result in Galaxy jobs that are currently running to fail. Note that the new disk size must be larger than the current disk size.
During this process, a snapshot of your data volume will be created, which can optionally be left in your account. If you decide to leave the snapshot for reference, you may also provide a brief note that will later be visible in the snapshot's description. New Disk Size (minimum 100GB, maximum 1000GB):
OK Note (optional):
or delete the created snapshot after filesystem resizing? If checked, the created snapshot will not be kept"
What is the likelihood that this will be successful? Has anyone utilized it? Is there another way to resize the /mnt/galaxyData/ and /mnt/galaxyIndices partitions that will allow me to do what I need to?
Thanks,
Iry Witham
The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hi Dannon, I will give it a shot. Are you familiar with the load balancer and how to make it work? Or the auto scaling feature? Thanks, Iry From: Dannon Baker <dannon.baker@gmail.com<mailto:dannon.baker@gmail.com>> Date: Monday, May 6, 2013 2:06 PM To: Iry Witham <iry.witham@jax.org<mailto:iry.witham@jax.org>> Cc: "galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>" <galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition Hi Iry, This should work, I use it regularly. Just make sure you're not ssh'ed in and 'in' /mnt/galaxyData or anything like that. There's not really another way to do this that I'd recommend trying if you need the instance to continue working for something tomorrow. Let me know if you have any issues and I can help. -Dannon On Mon, May 6, 2013 at 1:54 PM, Iry Witham <Iry.Witham@jax.org<mailto:Iry.Witham@jax.org>> wrote: Hi Team, I urgently need to resize the data partition on my galaxy cloudman instance without loosing anything. I notice that there a gui on the cloudman admin page that will allow me to increase it to 1000GB. If I use that will it restart all of the processes? I need to maintain this as a functional instance since it needs to be available for use tomorrow at 8:00 AM and I do not have the time to rebuild it. The gui says it will create a snapshot. The following is what it says: "Expand Disk Space Through this form you may increase the disk space available to Galaxy. All of the cluster services (but not the cluster) WILL BE STOPPED until the new disk is ready, at which point they will all be restarted. This may result in Galaxy jobs that are currently running to fail. Note that the new disk size must be larger than the current disk size. During this process, a snapshot of your data volume will be created, which can optionally be left in your account. If you decide to leave the snapshot for reference, you may also provide a brief note that will later be visible in the snapshot's description. New Disk Size (minimum 100GB, maximum 1000GB): OK Note (optional): or delete the created snapshot after filesystem resizing? If checked, the created snapshot will not be kept" What is the likelihood that this will be successful? Has anyone utilized it? Is there another way to resize the /mnt/galaxyData/ and /mnt/galaxyIndices partitions that will allow me to do what I need to? Thanks, Iry Witham The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible. ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.
Hi Dannon, I have attempted the resizing. I clicked on the disk and set it for 1000GB. The instance restarted, but the volume remains at 100GB. Is this due to the lack of disk space? I don't see a snapshot when I look at the AWS console after the restart. Any ideas? Thanks, IRy From: Dannon Baker <dannon.baker@gmail.com<mailto:dannon.baker@gmail.com>> Date: Monday, May 6, 2013 2:06 PM To: Iry Witham <iry.witham@jax.org<mailto:iry.witham@jax.org>> Cc: "galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>" <galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition Hi Iry, This should work, I use it regularly. Just make sure you're not ssh'ed in and 'in' /mnt/galaxyData or anything like that. There's not really another way to do this that I'd recommend trying if you need the instance to continue working for something tomorrow. Let me know if you have any issues and I can help. -Dannon On Mon, May 6, 2013 at 1:54 PM, Iry Witham <Iry.Witham@jax.org<mailto:Iry.Witham@jax.org>> wrote: Hi Team, I urgently need to resize the data partition on my galaxy cloudman instance without loosing anything. I notice that there a gui on the cloudman admin page that will allow me to increase it to 1000GB. If I use that will it restart all of the processes? I need to maintain this as a functional instance since it needs to be available for use tomorrow at 8:00 AM and I do not have the time to rebuild it. The gui says it will create a snapshot. The following is what it says: "Expand Disk Space Through this form you may increase the disk space available to Galaxy. All of the cluster services (but not the cluster) WILL BE STOPPED until the new disk is ready, at which point they will all be restarted. This may result in Galaxy jobs that are currently running to fail. Note that the new disk size must be larger than the current disk size. During this process, a snapshot of your data volume will be created, which can optionally be left in your account. If you decide to leave the snapshot for reference, you may also provide a brief note that will later be visible in the snapshot's description. New Disk Size (minimum 100GB, maximum 1000GB): OK Note (optional): or delete the created snapshot after filesystem resizing? If checked, the created snapshot will not be kept" What is the likelihood that this will be successful? Has anyone utilized it? Is there another way to resize the /mnt/galaxyData/ and /mnt/galaxyIndices partitions that will allow me to do what I need to? Thanks, Iry Witham The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible. ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.
The first thing that comes to mind is a bug that existed in a particular version of cloudman regarding the max size allowable -- can you try 999GB? On Mon, May 6, 2013 at 2:57 PM, Iry Witham <Iry.Witham@jax.org> wrote:
Hi Dannon,
I have attempted the resizing. I clicked on the disk and set it for 1000GB. The instance restarted, but the volume remains at 100GB. Is this due to the lack of disk space? I don't see a snapshot when I look at the AWS console after the restart. Any ideas?
Thanks,
IRy
From: Dannon Baker <dannon.baker@gmail.com> Date: Monday, May 6, 2013 2:06 PM To: Iry Witham <iry.witham@jax.org> Cc: "galaxy-dev@lists.bx.psu.edu" <galaxy-dev@lists.bx.psu.edu> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition
Hi Iry,
This should work, I use it regularly. Just make sure you're not ssh'ed in and 'in' /mnt/galaxyData or anything like that. There's not really another way to do this that I'd recommend trying if you need the instance to continue working for something tomorrow.
Let me know if you have any issues and I can help.
-Dannon
On Mon, May 6, 2013 at 1:54 PM, Iry Witham <Iry.Witham@jax.org> wrote:
Hi Team,
I urgently need to resize the data partition on my galaxy cloudman instance without loosing anything. I notice that there a gui on the cloudman admin page that will allow me to increase it to 1000GB. If I use that will it restart all of the processes? I need to maintain this as a functional instance since it needs to be available for use tomorrow at 8:00 AM and I do not have the time to rebuild it. The gui says it will create a snapshot. The following is what it says:
"Expand Disk Space Through this form you may increase the disk space available to Galaxy. All of the cluster services (but not the cluster) WILL BE STOPPED until the new disk is ready, at which point they will all be restarted. This may result in Galaxy jobs that are currently running to fail. Note that the new disk size must be larger than the current disk size.
During this process, a snapshot of your data volume will be created, which can optionally be left in your account. If you decide to leave the snapshot for reference, you may also provide a brief note that will later be visible in the snapshot's description. New Disk Size (minimum 100GB, maximum 1000GB):
OK Note (optional):
or delete the created snapshot after filesystem resizing? If checked, the created snapshot will not be kept"
What is the likelihood that this will be successful? Has anyone utilized it? Is there another way to resize the /mnt/galaxyData/ and /mnt/galaxyIndices partitions that will allow me to do what I need to?
Thanks,
Iry Witham
The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.
That seems to be working. From: Dannon Baker <dannon.baker@gmail.com<mailto:dannon.baker@gmail.com>> Date: Monday, May 6, 2013 3:01 PM To: Iry Witham <iry.witham@jax.org<mailto:iry.witham@jax.org>> Cc: "galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>" <galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition The first thing that comes to mind is a bug that existed in a particular version of cloudman regarding the max size allowable -- can you try 999GB? On Mon, May 6, 2013 at 2:57 PM, Iry Witham <Iry.Witham@jax.org<mailto:Iry.Witham@jax.org>> wrote: Hi Dannon, I have attempted the resizing. I clicked on the disk and set it for 1000GB. The instance restarted, but the volume remains at 100GB. Is this due to the lack of disk space? I don't see a snapshot when I look at the AWS console after the restart. Any ideas? Thanks, IRy From: Dannon Baker <dannon.baker@gmail.com<mailto:dannon.baker@gmail.com>> Date: Monday, May 6, 2013 2:06 PM To: Iry Witham <iry.witham@jax.org<mailto:iry.witham@jax.org>> Cc: "galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>" <galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition Hi Iry, This should work, I use it regularly. Just make sure you're not ssh'ed in and 'in' /mnt/galaxyData or anything like that. There's not really another way to do this that I'd recommend trying if you need the instance to continue working for something tomorrow. Let me know if you have any issues and I can help. -Dannon On Mon, May 6, 2013 at 1:54 PM, Iry Witham <Iry.Witham@jax.org<mailto:Iry.Witham@jax.org>> wrote: Hi Team, I urgently need to resize the data partition on my galaxy cloudman instance without loosing anything. I notice that there a gui on the cloudman admin page that will allow me to increase it to 1000GB. If I use that will it restart all of the processes? I need to maintain this as a functional instance since it needs to be available for use tomorrow at 8:00 AM and I do not have the time to rebuild it. The gui says it will create a snapshot. The following is what it says: "Expand Disk Space Through this form you may increase the disk space available to Galaxy. All of the cluster services (but not the cluster) WILL BE STOPPED until the new disk is ready, at which point they will all be restarted. This may result in Galaxy jobs that are currently running to fail. Note that the new disk size must be larger than the current disk size. During this process, a snapshot of your data volume will be created, which can optionally be left in your account. If you decide to leave the snapshot for reference, you may also provide a brief note that will later be visible in the snapshot's description. New Disk Size (minimum 100GB, maximum 1000GB): OK Note (optional): or delete the created snapshot after filesystem resizing? If checked, the created snapshot will not be kept" What is the likelihood that this will be successful? Has anyone utilized it? Is there another way to resize the /mnt/galaxyData/ and /mnt/galaxyIndices partitions that will allow me to do what I need to? Thanks, Iry Witham The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible. ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible. The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.
Great, I'm glad it's working, and I'm sorry you ran into this bug. As far as autoscaling, etc., what are you interested in? I should be able to help out with just about anything related to Cloudman. On Mon, May 6, 2013 at 3:03 PM, Iry Witham <Iry.Witham@jax.org> wrote:
That seems to be working.
From: Dannon Baker <dannon.baker@gmail.com> Date: Monday, May 6, 2013 3:01 PM
To: Iry Witham <iry.witham@jax.org> Cc: "galaxy-dev@lists.bx.psu.edu" <galaxy-dev@lists.bx.psu.edu> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition
The first thing that comes to mind is a bug that existed in a particular version of cloudman regarding the max size allowable -- can you try 999GB?
On Mon, May 6, 2013 at 2:57 PM, Iry Witham <Iry.Witham@jax.org> wrote:
Hi Dannon,
I have attempted the resizing. I clicked on the disk and set it for 1000GB. The instance restarted, but the volume remains at 100GB. Is this due to the lack of disk space? I don't see a snapshot when I look at the AWS console after the restart. Any ideas?
Thanks,
IRy
From: Dannon Baker <dannon.baker@gmail.com> Date: Monday, May 6, 2013 2:06 PM To: Iry Witham <iry.witham@jax.org> Cc: "galaxy-dev@lists.bx.psu.edu" <galaxy-dev@lists.bx.psu.edu> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition
Hi Iry,
This should work, I use it regularly. Just make sure you're not ssh'ed in and 'in' /mnt/galaxyData or anything like that. There's not really another way to do this that I'd recommend trying if you need the instance to continue working for something tomorrow.
Let me know if you have any issues and I can help.
-Dannon
On Mon, May 6, 2013 at 1:54 PM, Iry Witham <Iry.Witham@jax.org> wrote:
Hi Team,
I urgently need to resize the data partition on my galaxy cloudman instance without loosing anything. I notice that there a gui on the cloudman admin page that will allow me to increase it to 1000GB. If I use that will it restart all of the processes? I need to maintain this as a functional instance since it needs to be available for use tomorrow at 8:00 AM and I do not have the time to rebuild it. The gui says it will create a snapshot. The following is what it says:
"Expand Disk Space Through this form you may increase the disk space available to Galaxy. All of the cluster services (but not the cluster) WILL BE STOPPED until the new disk is ready, at which point they will all be restarted. This may result in Galaxy jobs that are currently running to fail. Note that the new disk size must be larger than the current disk size.
During this process, a snapshot of your data volume will be created, which can optionally be left in your account. If you decide to leave the snapshot for reference, you may also provide a brief note that will later be visible in the snapshot's description. New Disk Size (minimum 100GB, maximum 1000GB):
OK Note (optional):
or delete the created snapshot after filesystem resizing? If checked, the created snapshot will not be kept"
What is the likelihood that this will be successful? Has anyone utilized it? Is there another way to resize the /mnt/galaxyData/ and /mnt/galaxyIndices partitions that will allow me to do what I need to?
Thanks,
Iry Witham
The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.
The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.
I have setup a load balancer, but it shows that only 1 of 9 instances are 'In Service". I have set the health check to: Ping: http:80/ Timeout: 40 seconds Interval: 60 seconds Unhealthy threshold: 2 Healty threshold: 10 As far as autoscaling, I am not really interested since I see no real need at this point. Thanks, Iry From: Dannon Baker <dannon.baker@gmail.com<mailto:dannon.baker@gmail.com>> Date: Monday, May 6, 2013 3:41 PM To: Iry Witham <iry.witham@jax.org<mailto:iry.witham@jax.org>> Cc: "galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>" <galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition Great, I'm glad it's working, and I'm sorry you ran into this bug. As far as autoscaling, etc., what are you interested in? I should be able to help out with just about anything related to Cloudman. On Mon, May 6, 2013 at 3:03 PM, Iry Witham <Iry.Witham@jax.org<mailto:Iry.Witham@jax.org>> wrote: That seems to be working. From: Dannon Baker <dannon.baker@gmail.com<mailto:dannon.baker@gmail.com>> Date: Monday, May 6, 2013 3:01 PM To: Iry Witham <iry.witham@jax.org<mailto:iry.witham@jax.org>> Cc: "galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>" <galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition The first thing that comes to mind is a bug that existed in a particular version of cloudman regarding the max size allowable -- can you try 999GB? On Mon, May 6, 2013 at 2:57 PM, Iry Witham <Iry.Witham@jax.org<mailto:Iry.Witham@jax.org>> wrote: Hi Dannon, I have attempted the resizing. I clicked on the disk and set it for 1000GB. The instance restarted, but the volume remains at 100GB. Is this due to the lack of disk space? I don't see a snapshot when I look at the AWS console after the restart. Any ideas? Thanks, IRy From: Dannon Baker <dannon.baker@gmail.com<mailto:dannon.baker@gmail.com>> Date: Monday, May 6, 2013 2:06 PM To: Iry Witham <iry.witham@jax.org<mailto:iry.witham@jax.org>> Cc: "galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>" <galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition Hi Iry, This should work, I use it regularly. Just make sure you're not ssh'ed in and 'in' /mnt/galaxyData or anything like that. There's not really another way to do this that I'd recommend trying if you need the instance to continue working for something tomorrow. Let me know if you have any issues and I can help. -Dannon On Mon, May 6, 2013 at 1:54 PM, Iry Witham <Iry.Witham@jax.org<mailto:Iry.Witham@jax.org>> wrote: Hi Team, I urgently need to resize the data partition on my galaxy cloudman instance without loosing anything. I notice that there a gui on the cloudman admin page that will allow me to increase it to 1000GB. If I use that will it restart all of the processes? I need to maintain this as a functional instance since it needs to be available for use tomorrow at 8:00 AM and I do not have the time to rebuild it. The gui says it will create a snapshot. The following is what it says: "Expand Disk Space Through this form you may increase the disk space available to Galaxy. All of the cluster services (but not the cluster) WILL BE STOPPED until the new disk is ready, at which point they will all be restarted. This may result in Galaxy jobs that are currently running to fail. Note that the new disk size must be larger than the current disk size. During this process, a snapshot of your data volume will be created, which can optionally be left in your account. If you decide to leave the snapshot for reference, you may also provide a brief note that will later be visible in the snapshot's description. New Disk Size (minimum 100GB, maximum 1000GB): OK Note (optional): or delete the created snapshot after filesystem resizing? If checked, the created snapshot will not be kept" What is the likelihood that this will be successful? Has anyone utilized it? Is there another way to resize the /mnt/galaxyData/ and /mnt/galaxyIndices partitions that will allow me to do what I need to? Thanks, Iry Witham The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible. ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible. The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible. The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.
So you have a cluster with 9 instances and you're using ec2 load balancer with all of them to divert traffic? On Mon, May 6, 2013 at 3:56 PM, Iry Witham <Iry.Witham@jax.org> wrote:
I have setup a load balancer, but it shows that only 1 of 9 instances are 'In Service". I have set the health check to:
Ping: http:80/ Timeout: 40 seconds Interval: 60 seconds Unhealthy threshold: 2 Healty threshold: 10
As far as autoscaling, I am not really interested since I see no real need at this point.
Thanks,
Iry
From: Dannon Baker <dannon.baker@gmail.com> Date: Monday, May 6, 2013 3:41 PM
To: Iry Witham <iry.witham@jax.org> Cc: "galaxy-dev@lists.bx.psu.edu" <galaxy-dev@lists.bx.psu.edu> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition
Great, I'm glad it's working, and I'm sorry you ran into this bug.
As far as autoscaling, etc., what are you interested in? I should be able to help out with just about anything related to Cloudman.
On Mon, May 6, 2013 at 3:03 PM, Iry Witham <Iry.Witham@jax.org> wrote:
That seems to be working.
From: Dannon Baker <dannon.baker@gmail.com> Date: Monday, May 6, 2013 3:01 PM
To: Iry Witham <iry.witham@jax.org> Cc: "galaxy-dev@lists.bx.psu.edu" <galaxy-dev@lists.bx.psu.edu> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition
The first thing that comes to mind is a bug that existed in a particular version of cloudman regarding the max size allowable -- can you try 999GB?
On Mon, May 6, 2013 at 2:57 PM, Iry Witham <Iry.Witham@jax.org> wrote:
Hi Dannon,
I have attempted the resizing. I clicked on the disk and set it for 1000GB. The instance restarted, but the volume remains at 100GB. Is this due to the lack of disk space? I don't see a snapshot when I look at the AWS console after the restart. Any ideas?
Thanks,
IRy
From: Dannon Baker <dannon.baker@gmail.com> Date: Monday, May 6, 2013 2:06 PM To: Iry Witham <iry.witham@jax.org> Cc: "galaxy-dev@lists.bx.psu.edu" <galaxy-dev@lists.bx.psu.edu> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition
Hi Iry,
This should work, I use it regularly. Just make sure you're not ssh'ed in and 'in' /mnt/galaxyData or anything like that. There's not really another way to do this that I'd recommend trying if you need the instance to continue working for something tomorrow.
Let me know if you have any issues and I can help.
-Dannon
On Mon, May 6, 2013 at 1:54 PM, Iry Witham <Iry.Witham@jax.org> wrote:
Hi Team,
I urgently need to resize the data partition on my galaxy cloudman instance without loosing anything. I notice that there a gui on the cloudman admin page that will allow me to increase it to 1000GB. If I use that will it restart all of the processes? I need to maintain this as a functional instance since it needs to be available for use tomorrow at 8:00 AM and I do not have the time to rebuild it. The gui says it will create a snapshot. The following is what it says:
"Expand Disk Space Through this form you may increase the disk space available to Galaxy. All of the cluster services (but not the cluster) WILL BE STOPPED until the new disk is ready, at which point they will all be restarted. This may result in Galaxy jobs that are currently running to fail. Note that the new disk size must be larger than the current disk size.
During this process, a snapshot of your data volume will be created, which can optionally be left in your account. If you decide to leave the snapshot for reference, you may also provide a brief note that will later be visible in the snapshot's description. New Disk Size (minimum 100GB, maximum 1000GB):
OK Note (optional):
or delete the created snapshot after filesystem resizing? If checked, the created snapshot will not be kept"
What is the likelihood that this will be successful? Has anyone utilized it? Is there another way to resize the /mnt/galaxyData/ and /mnt/galaxyIndices partitions that will allow me to do what I need to?
Thanks,
Iry Witham
The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.
The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.
The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.
I was not clear how the load balancer worked in this instance, but my goal was to improve performance. I am not sure this was the correct use, but it seemed to work. From: Dannon Baker <dannon.baker@gmail.com<mailto:dannon.baker@gmail.com>> Date: Monday, May 6, 2013 4:12 PM To: Iry Witham <iry.witham@jax.org<mailto:iry.witham@jax.org>> Cc: "galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>" <galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition So you have a cluster with 9 instances and you're using ec2 load balancer with all of them to divert traffic? On Mon, May 6, 2013 at 3:56 PM, Iry Witham <Iry.Witham@jax.org<mailto:Iry.Witham@jax.org>> wrote: I have setup a load balancer, but it shows that only 1 of 9 instances are 'In Service". I have set the health check to: Ping: http:80/ Timeout: 40 seconds Interval: 60 seconds Unhealthy threshold: 2 Healty threshold: 10 As far as autoscaling, I am not really interested since I see no real need at this point. Thanks, Iry From: Dannon Baker <dannon.baker@gmail.com<mailto:dannon.baker@gmail.com>> Date: Monday, May 6, 2013 3:41 PM To: Iry Witham <iry.witham@jax.org<mailto:iry.witham@jax.org>> Cc: "galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>" <galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition Great, I'm glad it's working, and I'm sorry you ran into this bug. As far as autoscaling, etc., what are you interested in? I should be able to help out with just about anything related to Cloudman. On Mon, May 6, 2013 at 3:03 PM, Iry Witham <Iry.Witham@jax.org<mailto:Iry.Witham@jax.org>> wrote: That seems to be working. From: Dannon Baker <dannon.baker@gmail.com<mailto:dannon.baker@gmail.com>> Date: Monday, May 6, 2013 3:01 PM To: Iry Witham <iry.witham@jax.org<mailto:iry.witham@jax.org>> Cc: "galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>" <galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition The first thing that comes to mind is a bug that existed in a particular version of cloudman regarding the max size allowable -- can you try 999GB? On Mon, May 6, 2013 at 2:57 PM, Iry Witham <Iry.Witham@jax.org<mailto:Iry.Witham@jax.org>> wrote: Hi Dannon, I have attempted the resizing. I clicked on the disk and set it for 1000GB. The instance restarted, but the volume remains at 100GB. Is this due to the lack of disk space? I don't see a snapshot when I look at the AWS console after the restart. Any ideas? Thanks, IRy From: Dannon Baker <dannon.baker@gmail.com<mailto:dannon.baker@gmail.com>> Date: Monday, May 6, 2013 2:06 PM To: Iry Witham <iry.witham@jax.org<mailto:iry.witham@jax.org>> Cc: "galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>" <galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition Hi Iry, This should work, I use it regularly. Just make sure you're not ssh'ed in and 'in' /mnt/galaxyData or anything like that. There's not really another way to do this that I'd recommend trying if you need the instance to continue working for something tomorrow. Let me know if you have any issues and I can help. -Dannon On Mon, May 6, 2013 at 1:54 PM, Iry Witham <Iry.Witham@jax.org<mailto:Iry.Witham@jax.org>> wrote: Hi Team, I urgently need to resize the data partition on my galaxy cloudman instance without loosing anything. I notice that there a gui on the cloudman admin page that will allow me to increase it to 1000GB. If I use that will it restart all of the processes? I need to maintain this as a functional instance since it needs to be available for use tomorrow at 8:00 AM and I do not have the time to rebuild it. The gui says it will create a snapshot. The following is what it says: "Expand Disk Space Through this form you may increase the disk space available to Galaxy. All of the cluster services (but not the cluster) WILL BE STOPPED until the new disk is ready, at which point they will all be restarted. This may result in Galaxy jobs that are currently running to fail. Note that the new disk size must be larger than the current disk size. During this process, a snapshot of your data volume will be created, which can optionally be left in your account. If you decide to leave the snapshot for reference, you may also provide a brief note that will later be visible in the snapshot's description. New Disk Size (minimum 100GB, maximum 1000GB): OK Note (optional): or delete the created snapshot after filesystem resizing? If checked, the created snapshot will not be kept" What is the likelihood that this will be successful? Has anyone utilized it? Is there another way to resize the /mnt/galaxyData/ and /mnt/galaxyIndices partitions that will allow me to do what I need to? Thanks, Iry Witham The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible. ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible. The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible. The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible. The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.
Ahh, ok. So, unfortunately Cloudman isn't designed to be used with a load balancer (at least at this point). In any given cluster you have exactly one master node and all of the rest are workers (which do not run web processes, hence the 1/9 health). What was slow that you were trying to improve the performance of? One thing I'd definitely recommend for head node performance, if you do have extra worker nodes, would be to go to the admin panel (<your_ec2_instance>/cloud/admin) and disable job running on the head node using the "Set master to not run jobs" option. On Mon, May 6, 2013 at 4:15 PM, Iry Witham <Iry.Witham@jax.org> wrote:
I was not clear how the load balancer worked in this instance, but my goal was to improve performance. I am not sure this was the correct use, but it seemed to work.
From: Dannon Baker <dannon.baker@gmail.com> Date: Monday, May 6, 2013 4:12 PM
To: Iry Witham <iry.witham@jax.org> Cc: "galaxy-dev@lists.bx.psu.edu" <galaxy-dev@lists.bx.psu.edu> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition
So you have a cluster with 9 instances and you're using ec2 load balancer with all of them to divert traffic?
On Mon, May 6, 2013 at 3:56 PM, Iry Witham <Iry.Witham@jax.org> wrote:
I have setup a load balancer, but it shows that only 1 of 9 instances are 'In Service". I have set the health check to:
Ping: http:80/ Timeout: 40 seconds Interval: 60 seconds Unhealthy threshold: 2 Healty threshold: 10
As far as autoscaling, I am not really interested since I see no real need at this point.
Thanks,
Iry
From: Dannon Baker <dannon.baker@gmail.com> Date: Monday, May 6, 2013 3:41 PM
To: Iry Witham <iry.witham@jax.org> Cc: "galaxy-dev@lists.bx.psu.edu" <galaxy-dev@lists.bx.psu.edu> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition
Great, I'm glad it's working, and I'm sorry you ran into this bug.
As far as autoscaling, etc., what are you interested in? I should be able to help out with just about anything related to Cloudman.
On Mon, May 6, 2013 at 3:03 PM, Iry Witham <Iry.Witham@jax.org> wrote:
That seems to be working.
From: Dannon Baker <dannon.baker@gmail.com> Date: Monday, May 6, 2013 3:01 PM
To: Iry Witham <iry.witham@jax.org> Cc: "galaxy-dev@lists.bx.psu.edu" <galaxy-dev@lists.bx.psu.edu> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition
The first thing that comes to mind is a bug that existed in a particular version of cloudman regarding the max size allowable -- can you try 999GB?
On Mon, May 6, 2013 at 2:57 PM, Iry Witham <Iry.Witham@jax.org> wrote:
Hi Dannon,
I have attempted the resizing. I clicked on the disk and set it for 1000GB. The instance restarted, but the volume remains at 100GB. Is this due to the lack of disk space? I don't see a snapshot when I look at the AWS console after the restart. Any ideas?
Thanks,
IRy
From: Dannon Baker <dannon.baker@gmail.com> Date: Monday, May 6, 2013 2:06 PM To: Iry Witham <iry.witham@jax.org> Cc: "galaxy-dev@lists.bx.psu.edu" <galaxy-dev@lists.bx.psu.edu> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition
Hi Iry,
This should work, I use it regularly. Just make sure you're not ssh'ed in and 'in' /mnt/galaxyData or anything like that. There's not really another way to do this that I'd recommend trying if you need the instance to continue working for something tomorrow.
Let me know if you have any issues and I can help.
-Dannon
On Mon, May 6, 2013 at 1:54 PM, Iry Witham <Iry.Witham@jax.org> wrote:
Hi Team,
I urgently need to resize the data partition on my galaxy cloudman instance without loosing anything. I notice that there a gui on the cloudman admin page that will allow me to increase it to 1000GB. If I use that will it restart all of the processes? I need to maintain this as a functional instance since it needs to be available for use tomorrow at 8:00 AM and I do not have the time to rebuild it. The gui says it will create a snapshot. The following is what it says:
"Expand Disk Space Through this form you may increase the disk space available to Galaxy. All of the cluster services (but not the cluster) WILL BE STOPPED until the new disk is ready, at which point they will all be restarted. This may result in Galaxy jobs that are currently running to fail. Note that the new disk size must be larger than the current disk size.
During this process, a snapshot of your data volume will be created, which can optionally be left in your account. If you decide to leave the snapshot for reference, you may also provide a brief note that will later be visible in the snapshot's description. New Disk Size (minimum 100GB, maximum 1000GB):
OK Note (optional):
or delete the created snapshot after filesystem resizing? If checked, the created snapshot will not be kept"
What is the likelihood that this will be successful? Has anyone utilized it? Is there another way to resize the /mnt/galaxyData/ and /mnt/galaxyIndices partitions that will allow me to do what I need to?
Thanks,
Iry Witham
The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.
The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.
The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.
The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.
I did add nodes about the same time I setup load balancing so that was likely the fix. I will definitely set the head node not to run jobs. Thanks, Iry From: Dannon Baker <dannon.baker@gmail.com<mailto:dannon.baker@gmail.com>> Date: Monday, May 6, 2013 4:19 PM To: Iry Witham <iry.witham@jax.org<mailto:iry.witham@jax.org>> Cc: "galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>" <galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition Ahh, ok. So, unfortunately Cloudman isn't designed to be used with a load balancer (at least at this point). In any given cluster you have exactly one master node and all of the rest are workers (which do not run web processes, hence the 1/9 health). What was slow that you were trying to improve the performance of? One thing I'd definitely recommend for head node performance, if you do have extra worker nodes, would be to go to the admin panel (<your_ec2_instance>/cloud/admin) and disable job running on the head node using the "Set master to not run jobs" option. On Mon, May 6, 2013 at 4:15 PM, Iry Witham <Iry.Witham@jax.org<mailto:Iry.Witham@jax.org>> wrote: I was not clear how the load balancer worked in this instance, but my goal was to improve performance. I am not sure this was the correct use, but it seemed to work. From: Dannon Baker <dannon.baker@gmail.com<mailto:dannon.baker@gmail.com>> Date: Monday, May 6, 2013 4:12 PM To: Iry Witham <iry.witham@jax.org<mailto:iry.witham@jax.org>> Cc: "galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>" <galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition So you have a cluster with 9 instances and you're using ec2 load balancer with all of them to divert traffic? On Mon, May 6, 2013 at 3:56 PM, Iry Witham <Iry.Witham@jax.org<mailto:Iry.Witham@jax.org>> wrote: I have setup a load balancer, but it shows that only 1 of 9 instances are 'In Service". I have set the health check to: Ping: http:80/ Timeout: 40 seconds Interval: 60 seconds Unhealthy threshold: 2 Healty threshold: 10 As far as autoscaling, I am not really interested since I see no real need at this point. Thanks, Iry From: Dannon Baker <dannon.baker@gmail.com<mailto:dannon.baker@gmail.com>> Date: Monday, May 6, 2013 3:41 PM To: Iry Witham <iry.witham@jax.org<mailto:iry.witham@jax.org>> Cc: "galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>" <galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition Great, I'm glad it's working, and I'm sorry you ran into this bug. As far as autoscaling, etc., what are you interested in? I should be able to help out with just about anything related to Cloudman. On Mon, May 6, 2013 at 3:03 PM, Iry Witham <Iry.Witham@jax.org<mailto:Iry.Witham@jax.org>> wrote: That seems to be working. From: Dannon Baker <dannon.baker@gmail.com<mailto:dannon.baker@gmail.com>> Date: Monday, May 6, 2013 3:01 PM To: Iry Witham <iry.witham@jax.org<mailto:iry.witham@jax.org>> Cc: "galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>" <galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition The first thing that comes to mind is a bug that existed in a particular version of cloudman regarding the max size allowable -- can you try 999GB? On Mon, May 6, 2013 at 2:57 PM, Iry Witham <Iry.Witham@jax.org<mailto:Iry.Witham@jax.org>> wrote: Hi Dannon, I have attempted the resizing. I clicked on the disk and set it for 1000GB. The instance restarted, but the volume remains at 100GB. Is this due to the lack of disk space? I don't see a snapshot when I look at the AWS console after the restart. Any ideas? Thanks, IRy From: Dannon Baker <dannon.baker@gmail.com<mailto:dannon.baker@gmail.com>> Date: Monday, May 6, 2013 2:06 PM To: Iry Witham <iry.witham@jax.org<mailto:iry.witham@jax.org>> Cc: "galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>" <galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition Hi Iry, This should work, I use it regularly. Just make sure you're not ssh'ed in and 'in' /mnt/galaxyData or anything like that. There's not really another way to do this that I'd recommend trying if you need the instance to continue working for something tomorrow. Let me know if you have any issues and I can help. -Dannon On Mon, May 6, 2013 at 1:54 PM, Iry Witham <Iry.Witham@jax.org<mailto:Iry.Witham@jax.org>> wrote: Hi Team, I urgently need to resize the data partition on my galaxy cloudman instance without loosing anything. I notice that there a gui on the cloudman admin page that will allow me to increase it to 1000GB. If I use that will it restart all of the processes? I need to maintain this as a functional instance since it needs to be available for use tomorrow at 8:00 AM and I do not have the time to rebuild it. The gui says it will create a snapshot. The following is what it says: "Expand Disk Space Through this form you may increase the disk space available to Galaxy. All of the cluster services (but not the cluster) WILL BE STOPPED until the new disk is ready, at which point they will all be restarted. This may result in Galaxy jobs that are currently running to fail. Note that the new disk size must be larger than the current disk size. During this process, a snapshot of your data volume will be created, which can optionally be left in your account. If you decide to leave the snapshot for reference, you may also provide a brief note that will later be visible in the snapshot's description. New Disk Size (minimum 100GB, maximum 1000GB): OK Note (optional): or delete the created snapshot after filesystem resizing? If checked, the created snapshot will not be kept" What is the likelihood that this will be successful? Has anyone utilized it? Is there another way to resize the /mnt/galaxyData/ and /mnt/galaxyIndices partitions that will allow me to do what I need to? Thanks, Iry Witham The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible. ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible. The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible. The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible. The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible. The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.
Ok, cool. It's worth noting that worker nodes are completely disposable and that if there are any not running jobs at the top of the hour (when billing happens) it's a waste. What you may want to do, at least until the event tomorrow, is scale back down to 1 master and 1 worker node and turn on autoscaling. Cloudman's autoscaling will automatically kill idle instances at the end of the hour (prior to being billed for the next hour), and will start more if the job queue starts to get backed up (within the limits set in the admin interface, of course). On Mon, May 6, 2013 at 4:31 PM, Iry Witham <Iry.Witham@jax.org> wrote:
I did add nodes about the same time I setup load balancing so that was likely the fix. I will definitely set the head node not to run jobs.
Thanks, Iry
From: Dannon Baker <dannon.baker@gmail.com> Date: Monday, May 6, 2013 4:19 PM
To: Iry Witham <iry.witham@jax.org> Cc: "galaxy-dev@lists.bx.psu.edu" <galaxy-dev@lists.bx.psu.edu> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition
Ahh, ok. So, unfortunately Cloudman isn't designed to be used with a load balancer (at least at this point). In any given cluster you have exactly one master node and all of the rest are workers (which do not run web processes, hence the 1/9 health).
What was slow that you were trying to improve the performance of? One thing I'd definitely recommend for head node performance, if you do have extra worker nodes, would be to go to the admin panel (<your_ec2_instance>/cloud/admin) and disable job running on the head node using the "Set master to not run jobs" option.
On Mon, May 6, 2013 at 4:15 PM, Iry Witham <Iry.Witham@jax.org> wrote:
I was not clear how the load balancer worked in this instance, but my goal was to improve performance. I am not sure this was the correct use, but it seemed to work.
From: Dannon Baker <dannon.baker@gmail.com> Date: Monday, May 6, 2013 4:12 PM
To: Iry Witham <iry.witham@jax.org> Cc: "galaxy-dev@lists.bx.psu.edu" <galaxy-dev@lists.bx.psu.edu> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition
So you have a cluster with 9 instances and you're using ec2 load balancer with all of them to divert traffic?
On Mon, May 6, 2013 at 3:56 PM, Iry Witham <Iry.Witham@jax.org> wrote:
I have setup a load balancer, but it shows that only 1 of 9 instances are 'In Service". I have set the health check to:
Ping: http:80/ Timeout: 40 seconds Interval: 60 seconds Unhealthy threshold: 2 Healty threshold: 10
As far as autoscaling, I am not really interested since I see no real need at this point.
Thanks,
Iry
From: Dannon Baker <dannon.baker@gmail.com> Date: Monday, May 6, 2013 3:41 PM
To: Iry Witham <iry.witham@jax.org> Cc: "galaxy-dev@lists.bx.psu.edu" <galaxy-dev@lists.bx.psu.edu> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition
Great, I'm glad it's working, and I'm sorry you ran into this bug.
As far as autoscaling, etc., what are you interested in? I should be able to help out with just about anything related to Cloudman.
On Mon, May 6, 2013 at 3:03 PM, Iry Witham <Iry.Witham@jax.org> wrote:
That seems to be working.
From: Dannon Baker <dannon.baker@gmail.com> Date: Monday, May 6, 2013 3:01 PM
To: Iry Witham <iry.witham@jax.org> Cc: "galaxy-dev@lists.bx.psu.edu" <galaxy-dev@lists.bx.psu.edu> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition
The first thing that comes to mind is a bug that existed in a particular version of cloudman regarding the max size allowable -- can you try 999GB?
On Mon, May 6, 2013 at 2:57 PM, Iry Witham <Iry.Witham@jax.org> wrote:
Hi Dannon,
I have attempted the resizing. I clicked on the disk and set it for 1000GB. The instance restarted, but the volume remains at 100GB. Is this due to the lack of disk space? I don't see a snapshot when I look at the AWS console after the restart. Any ideas?
Thanks,
IRy
From: Dannon Baker <dannon.baker@gmail.com> Date: Monday, May 6, 2013 2:06 PM To: Iry Witham <iry.witham@jax.org> Cc: "galaxy-dev@lists.bx.psu.edu" <galaxy-dev@lists.bx.psu.edu> Subject: Re: [galaxy-dev] Resizing galaxy cloudman /mnt/galaxyData partition
Hi Iry,
This should work, I use it regularly. Just make sure you're not ssh'ed in and 'in' /mnt/galaxyData or anything like that. There's not really another way to do this that I'd recommend trying if you need the instance to continue working for something tomorrow.
Let me know if you have any issues and I can help.
-Dannon
On Mon, May 6, 2013 at 1:54 PM, Iry Witham <Iry.Witham@jax.org> wrote:
Hi Team,
I urgently need to resize the data partition on my galaxy cloudman instance without loosing anything. I notice that there a gui on the cloudman admin page that will allow me to increase it to 1000GB. If I use that will it restart all of the processes? I need to maintain this as a functional instance since it needs to be available for use tomorrow at 8:00 AM and I do not have the time to rebuild it. The gui says it will create a snapshot. The following is what it says:
"Expand Disk Space Through this form you may increase the disk space available to Galaxy. All of the cluster services (but not the cluster) WILL BE STOPPED until the new disk is ready, at which point they will all be restarted. This may result in Galaxy jobs that are currently running to fail. Note that the new disk size must be larger than the current disk size.
During this process, a snapshot of your data volume will be created, which can optionally be left in your account. If you decide to leave the snapshot for reference, you may also provide a brief note that will later be visible in the snapshot's description. New Disk Size (minimum 100GB, maximum 1000GB):
OK Note (optional):
or delete the created snapshot after filesystem resizing? If checked, the created snapshot will not be kept"
What is the likelihood that this will be successful? Has anyone utilized it? Is there another way to resize the /mnt/galaxyData/ and /mnt/galaxyIndices partitions that will allow me to do what I need to?
Thanks,
Iry Witham
The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.
The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.
The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.
The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.
The information in this email, including attachments, may be confidential and is intended solely for the addressee(s). If you believe you received this email by mistake, please notify the sender by return email as soon as possible.
Dannon, I've set up galaxy at the local linux. After data configuration, I run fastqc to check how it runs. Here I got error message as below -- error An error occurred running this job: Traceback (most recent call last): File "../galaxy-dist/tools/rgenetics/rgFastQC.py", line 158, in assert os.path.isfile(opts.executable),'##rgFastQC.py error - cannot find executable %s' % opts.executable AssertionError: ##r Here is the detail of the error -- Tool: FastQC:Read QC Name: FastQC_FASTQ Groomer on data 6.html Created: May 02, 2013 Filesize: 0 bytes Dbkey: mm9 Format: html Tool Version: Tool Standard Output: stdout Tool Standard Error: stderr Tool Exit Code: 1 Input Parameter Value Short read data from your current history 8: FASTQ Groomer on data 6 Title for the output file - to remind you what the job was for FastQC Contaminant list No dataset What might be the problem? Thank you! Kathryn
Hi Kathryn, It looks like you haven't installed FastQC (or potentially installed it and just haven't made it accessible to Galaxy). You'll need to install FastQC from http://www.bioinformatics.babraham.ac.uk/projects/fastqc/ if you haven't already, and the main fastqc perl script must be executable from tool-data/shared/jars/FastQC/fastqc. For dependencies like this, you can usually look at http://wiki.galaxyproject.org/Admin/Tools/Tool%20Dependencies for a few pointers. Good luck, Dannon On Thu, May 9, 2013 at 9:43 AM, Kathryn Sun <kathryn.sun@ymail.com> wrote:
Dannon,
I've set up galaxy at the local linux. After data configuration, I run fastqc to check how it runs. Here I got error message as below --
error An error occurred running this job: Traceback (most recent call last): File "../galaxy-dist/tools/rgenetics/rgFastQC.py", line 158, in assert os.path.isfile(opts.executable),'##rgFastQC.py error - cannot find executable %s' % opts.executable AssertionError: ##r
Here is the detail of the error -- Tool: FastQC:Read QC Name: FastQC_FASTQ Groomer on data 6.html Created: May 02, 2013 Filesize: 0 bytes Dbkey: mm9 Format: html Tool Version: Tool Standard Output: stdout Tool Standard Error: stderr Tool Exit Code: 1
Input Parameter Value Short read data from your current history 8: FASTQ Groomer on data 6 Title for the output file - to remind you what the job was for FastQC Contaminant list No dataset
What might be the problem? Thank you! Kathryn
Dannon, Thank you very much for the help. I will install those dependancies based on the information you sent. Is there any restrictions for path/directory to be installed, for example, should those tools only be installed under ../galaxy-dist/tools? Thank you! Kathryn ________________________________ From: Dannon Baker <dannon.baker@gmail.com> To: Kathryn Sun <kathryn.sun@ymail.com> Cc: "galaxy-dev@lists.bx.psu.edu" <galaxy-dev@lists.bx.psu.edu> Sent: Friday, May 10, 2013 10:07 AM Subject: Re: Error with fastqc Hi Kathryn, It looks like you haven't installed FastQC (or potentially installed it and just haven't made it accessible to Galaxy). You'll need to install FastQC from http://www.bioinformatics.babraham.ac.uk/projects/fastqc/ if you haven't already, and the main fastqc perl script must be executable from tool-data/shared/jars/FastQC/fastqc. For dependencies like this, you can usually look at http://wiki.galaxyproject.org/Admin/Tools/Tool%20Dependencies for a few pointers. Good luck, Dannon On Thu, May 9, 2013 at 9:43 AM, Kathryn Sun <kathryn.sun@ymail.com> wrote: Dannon,
I've set up galaxy at the local linux. After data configuration, I run
fastqc to check how it runs. Here I got error message as below --
error An
error occurred running this job: Traceback (most recent call last): File "../galaxy-dist/tools/rgenetics/rgFastQC.py", line 158, in assert os.path.isfile(opts.executable),'##rgFastQC.py error - cannot find executable %s' % opts.executable AssertionError: ##r
Here is the detail of the error -- Tool: FastQC:Read QC Name: FastQC_FASTQ Groomer on data 6.html Created: May 02, 2013 Filesize: 0 bytes Dbkey: mm9 Format: html Tool Version: Tool Standard Output: stdout Tool Standard Error: stderr Tool Exit Code: 1
Input Parameter Value Short read data from your current history 8: FASTQ Groomer on data 6 Title for the output file - to remind you what the job was for FastQC Contaminant list No
dataset
What might be the problem? Thank you! Kathryn
Dannon, Thanks and it worked! Kathryn ________________________________ From: Dannon Baker <dannon.baker@gmail.com> To: Kathryn Sun <kathryn.sun@ymail.com> Cc: "galaxy-dev@lists.bx.psu.edu" <galaxy-dev@lists.bx.psu.edu> Sent: Friday, May 10, 2013 10:07 AM Subject: Re: Error with fastqc Hi Kathryn, It looks like you haven't installed FastQC (or potentially installed it and just haven't made it accessible to Galaxy). You'll need to install FastQC from http://www.bioinformatics.babraham.ac.uk/projects/fastqc/ if you haven't already, and the main fastqc perl script must be executable from tool-data/shared/jars/FastQC/fastqc. For dependencies like this, you can usually look at http://wiki.galaxyproject.org/Admin/Tools/Tool%20Dependencies for a few pointers. Good luck, Dannon On Thu, May 9, 2013 at 9:43 AM, Kathryn Sun <kathryn.sun@ymail.com> wrote: Dannon,
I've set up galaxy at the local linux. After data configuration, I run
fastqc to check how it runs. Here I got error message as below --
error An
error occurred running this job: Traceback (most recent call last): File "../galaxy-dist/tools/rgenetics/rgFastQC.py", line 158, in assert os.path.isfile(opts.executable),'##rgFastQC.py error - cannot find executable %s' % opts.executable AssertionError: ##r
Here is the detail of the error -- Tool: FastQC:Read QC Name: FastQC_FASTQ Groomer on data 6.html Created: May 02, 2013 Filesize: 0 bytes Dbkey: mm9 Format: html Tool Version: Tool Standard Output: stdout Tool Standard Error: stderr Tool Exit Code: 1
Input Parameter Value Short read data from your current history 8: FASTQ Groomer on data 6 Title for the output file - to remind you what the job was for FastQC Contaminant list No
dataset
What might be the problem? Thank you! Kathryn
participants (3)
-
Dannon Baker -
Iry Witham -
Kathryn Sun