Re: [galaxy-dev] [galaxy-user] mi-tools- tools_fabfile.py permissions requirements (local install)
Note: Local install questions are normally directed to the galaxy-dev list (CC'd) rather than galaxy-user
On Mon, Aug 1, 2011 at 3:21 PM, Joseph Hargitai joseph.hargitai@einstein.yu.edu wrote:
Enis,
a few things we came across:
a, local install
R - not the correct download url,
Which URL are you talking about? The wiki page on dependencies http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Dependencies correctly links to the R project as http://www.r-project.org/
rpy - will this be fixed?
What needs to be fixed? Moving Galaxy to rpy2? https://bitbucket.org/galaxy/galaxy-central/issue/103/upgrade-rpy-to-latest-...
permissions:
the script seems to use sudo and galaxy interchangeably, which creates an issue with permissions writing to folders.
Is there a way to install as galaxy user only?
Do you mean installing Galaxy without admin rights (i.e. without using sudo)?
If we do need the switching between sudo and galaxy - is there a set of directory permissions you suggest for the original galaxy-dist install and the galaxyToools folder?
Good question - having manually reset permissions on a test server while debugging driving mappings, I'd like to know what they should be.
Are you planing on using the Linux user group functionality as part of your setup? e.g. Both your personal account and the Galaxy user account could be part of a Galaxy admin group.
b, as a second category of questions: are there any tools that do not work on the standard cloud install? (skipped over entirely or installed but do not work?)
By that do you mean of the standard tool set in the main repository? (Because there will be lots of Galaxy tools in the Tool Shed which are not in the standard cloud install).
Peter
R - not the correct download url,
Which URL are you talking about? The wiki page on dependencies http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Dependencies correctly links to the R project as http://www.r-project.org/
you are correct, however the tools_fabfile.py has
version = "2.11.1" url = "http://mira.sunsite.utk.edu/CRAN/src/base/R-2/R-%s.tar.gz" % version
rpy - will this be fixed?
What needs to be fixed? Moving Galaxy to rpy2? https://bitbucket.org/galaxy/galaxy-central/issue/103/upgrade-rpy-to-latest-...
again script has a comment:
def _install_rpy(): # *Does not work in reality*
Do you mean installing Galaxy without admin rights (i.e. without using sudo)?
either way - but should work. The fab script for instance adds some jar files to the galaxy-dist structure - then more added via the same script as sudo, and when it wants to change permissions for the entire folder for the next application the script fails. Hence just curious what a smoother solution would be.
In general - is the cloud install using the mi- script to install the tools? If so, where is it located? I'd like to compare it to the one i use for the local install. If it uses another script where/what is it?
thanks, joe
Hi Jeseph,
I see now you are talking about Enis' code for automated building of virtual machine Galaxy images, https://bitbucket.org/afgane/mi-deployment/src - mentioned briefly here: http://wiki.g2.bx.psu.edu/Admin/NGS%20Local%20Setup
I know very little about this area of Galaxy - we install any tools needed for Galaxy by hand (and try to document how we did it for future reference).
On Mon, Aug 1, 2011 at 3:50 PM, Joseph Hargitai joseph.hargitai@einstein.yu.edu wrote:
R - not the correct download url,
Which URL are you talking about? The wiki page on dependencies http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Dependencies correctly links to the R project as http://www.r-project.org/
you are correct, however the tools_fabfile.py has
version = "2.11.1" url = "http://mira.sunsite.utk.edu/CRAN/src/base/R-2/R-%s.tar.gz" % version
That was a CRAN mirror, which seems to have gone now. Not listed here: http://cran.r-project.org/mirrors.html
rpy - will this be fixed?
What needs to be fixed? Moving Galaxy to rpy2? https://bitbucket.org/galaxy/galaxy-central/issue/103/upgrade-rpy-to-latest-...
again script has a comment:
def _install_rpy(): # *Does not work in reality*
Do you mean installing Galaxy without admin rights (i.e. without using sudo)?
either way - but should work. The fab script for instance adds some jar files to the galaxy-dist structure - then more added via the same script as sudo, and when it wants to change permissions for the entire folder for the next application the script fails. Hence just curious what a smoother solution would be.
In general - is the cloud install using the mi- script to install the tools? If so, where is it located? I'd like to compare it to the one i use for the local install. If it uses another script where/what is it?
thanks, joe
Regards,
Peter
Hi Joe, mi-deployment scripts are currently being used to create cloud images so this is the correct code your looking at. However, when it comes to R and rpy, those tools are (as you discovered) not being installed via those scripts; they are installed from packages but, in the case of Galaxy cloud images, they are inherited from the CloudBioLinux images that the Galaxy cloud images build on top of so not even included in the mi-deployment scripts. If you would like to follow suit and install those tools from packages, these are the ones (for debian): r-base, r-base-core, r-base-core-ra, r-base-dev, r-base-html, python-rpy
As far as the permissions issues goes, I'll look (over the next couple of days) more closely at those and focus on consistency.
Enis
On Mon, Aug 1, 2011 at 11:04 AM, Peter Cock p.j.a.cock@googlemail.comwrote:
Hi Jeseph,
I see now you are talking about Enis' code for automated building of virtual machine Galaxy images, https://bitbucket.org/afgane/mi-deployment/src - mentioned briefly here: http://wiki.g2.bx.psu.edu/Admin/NGS%20Local%20Setup
I know very little about this area of Galaxy - we install any tools needed for Galaxy by hand (and try to document how we did it for future reference).
On Mon, Aug 1, 2011 at 3:50 PM, Joseph Hargitai joseph.hargitai@einstein.yu.edu wrote:
R - not the correct download url,
Which URL are you talking about? The wiki page on dependencies http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Dependencies correctly links to the R project as http://www.r-project.org/
you are correct, however the tools_fabfile.py has
version = "2.11.1" url = "http://mira.sunsite.utk.edu/CRAN/src/base/R-2/R-%s.tar.gz" %
version
That was a CRAN mirror, which seems to have gone now. Not listed here: http://cran.r-project.org/mirrors.html
rpy - will this be fixed?
What needs to be fixed? Moving Galaxy to rpy2?
https://bitbucket.org/galaxy/galaxy-central/issue/103/upgrade-rpy-to-latest-...
again script has a comment:
def _install_rpy(): # *Does not work in reality*
Do you mean installing Galaxy without admin rights (i.e. without using
sudo)?
either way - but should work. The fab script for instance adds some jar
files
to the galaxy-dist structure - then more added via the same script as
sudo,
and when it wants to change permissions for the entire folder for the
next
application the script fails. Hence just curious what a smoother solution
would be.
In general - is the cloud install using the mi- script to install the
tools? If so,
where is it located? I'd like to compare it to the one i use for the
local install.
If it uses another script where/what is it?
thanks, joe
Regards,
Peter
Hi,
so for local install i can uncomment the R and rpy from your script as far as at the _install_R.. part?
j
________________________________ From: Enis Afgan [eafgan@emory.edu] Sent: Monday, August 01, 2011 12:37 PM To: Joseph Hargitai Cc: Galaxy Dev Subject: Re: [galaxy-user] mi-tools- tools_fabfile.py permissions requirements (local install)
Hi Joe, mi-deployment scripts are currently being used to create cloud images so this is the correct code your looking at. However, when it comes to R and rpy, those tools are (as you discovered) not being installed via those scripts; they are installed from packages but, in the case of Galaxy cloud images, they are inherited from the CloudBioLinux images that the Galaxy cloud images build on top of so not even included in the mi-deployment scripts. If you would like to follow suit and install those tools from packages, these are the ones (for debian): r-base, r-base-core, r-base-core-ra, r-base-dev, r-base-html, python-rpy
As far as the permissions issues goes, I'll look (over the next couple of days) more closely at those and focus on consistency.
Enis
On Mon, Aug 1, 2011 at 11:04 AM, Peter Cock <p.j.a.cock@googlemail.commailto:p.j.a.cock@googlemail.com> wrote: Hi Jeseph,
I see now you are talking about Enis' code for automated building of virtual machine Galaxy images, https://bitbucket.org/afgane/mi-deployment/src - mentioned briefly here: http://wiki.g2.bx.psu.edu/Admin/NGS%20Local%20Setup
I know very little about this area of Galaxy - we install any tools needed for Galaxy by hand (and try to document how we did it for future reference).
On Mon, Aug 1, 2011 at 3:50 PM, Joseph Hargitai <joseph.hargitai@einstein.yu.edumailto:joseph.hargitai@einstein.yu.edu> wrote:
R - not the correct download url,
Which URL are you talking about? The wiki page on dependencies http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Dependencies correctly links to the R project as http://www.r-project.org/
you are correct, however the tools_fabfile.py has
version = "2.11.1" url = "http://mira.sunsite.utk.edu/CRAN/src/base/R-2/R-%s.tar.gz" % version
That was a CRAN mirror, which seems to have gone now. Not listed here: http://cran.r-project.org/mirrors.html
rpy - will this be fixed?
What needs to be fixed? Moving Galaxy to rpy2? https://bitbucket.org/galaxy/galaxy-central/issue/103/upgrade-rpy-to-latest-...
again script has a comment:
def _install_rpy(): # *Does not work in reality*
Do you mean installing Galaxy without admin rights (i.e. without using sudo)?
either way - but should work. The fab script for instance adds some jar files to the galaxy-dist structure - then more added via the same script as sudo, and when it wants to change permissions for the entire folder for the next application the script fails. Hence just curious what a smoother solution would be.
In general - is the cloud install using the mi- script to install the tools? If so, where is it located? I'd like to compare it to the one i use for the local install. If it uses another script where/what is it?
thanks, joe
Regards,
Peter
Well, you can give it a shot like that and see if it still works. I quit using (and thus updating) the R-installation methods a while ago now so not sure if the process was changed. There is high likelihood that R will work (you should probably update the version number in the method itself). rpy however, will probably not work - rpy was very hard to install to begin with and that is why we moved away from this approach. Alternatively, you can add packages I listed in my last email to the list of packages under _required_packages method.
Enis
On Mon, Aug 1, 2011 at 12:49 PM, Joseph Hargitai < joseph.hargitai@einstein.yu.edu> wrote:
Hi,
so for local install i can uncomment the R and rpy from your script as far as at the _install_R.. part?
j
*From:* Enis Afgan [eafgan@emory.edu] *Sent:* Monday, August 01, 2011 12:37 PM *To:* Joseph Hargitai *Cc:* Galaxy Dev *Subject:* Re: [galaxy-user] mi-tools- tools_fabfile.py permissions requirements (local install)
Hi Joe, mi-deployment scripts are currently being used to create cloud images so this is the correct code your looking at. However, when it comes to R and rpy, those tools are (as you discovered) not being installed via those scripts; they are installed from packages but, in the case of Galaxy cloud images, they are inherited from the CloudBioLinux images that the Galaxy cloud images build on top of so not even included in the mi-deployment scripts. If you would like to follow suit and install those tools from packages, these are the ones (for debian): r-base, r-base-core, r-base-core-ra, r-base-dev, r-base-html, python-rpy
As far as the permissions issues goes, I'll look (over the next couple of days) more closely at those and focus on consistency.
Enis
On Mon, Aug 1, 2011 at 11:04 AM, Peter Cock p.j.a.cock@googlemail.comwrote:
Hi Jeseph,
I see now you are talking about Enis' code for automated building of virtual machine Galaxy images, https://bitbucket.org/afgane/mi-deployment/src - mentioned briefly here: http://wiki.g2.bx.psu.edu/Admin/NGS%20Local%20Setup
I know very little about this area of Galaxy - we install any tools needed for Galaxy by hand (and try to document how we did it for future reference).
On Mon, Aug 1, 2011 at 3:50 PM, Joseph Hargitai joseph.hargitai@einstein.yu.edu wrote:
R - not the correct download url,
Which URL are you talking about? The wiki page on dependencies http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Dependencies correctly links to the R project as http://www.r-project.org/
you are correct, however the tools_fabfile.py has
version = "2.11.1" url = "http://mira.sunsite.utk.edu/CRAN/src/base/R-2/R-%s.tar.gz" %
version
That was a CRAN mirror, which seems to have gone now. Not listed here: http://cran.r-project.org/mirrors.html
rpy - will this be fixed?
What needs to be fixed? Moving Galaxy to rpy2?
https://bitbucket.org/galaxy/galaxy-central/issue/103/upgrade-rpy-to-latest-...
again script has a comment:
def _install_rpy(): # *Does not work in reality*
Do you mean installing Galaxy without admin rights (i.e. without using
sudo)?
either way - but should work. The fab script for instance adds some jar
files
to the galaxy-dist structure - then more added via the same script as
sudo,
and when it wants to change permissions for the entire folder for the
next
application the script fails. Hence just curious what a smoother
solution would be.
In general - is the cloud install using the mi- script to install the
tools? If so,
where is it located? I'd like to compare it to the one i use for the
local install.
If it uses another script where/what is it?
thanks, joe
Regards,
Peter
Cool,
one question remains:
since the cloud instance works and installs - there needs to be a script that is fully functional -
a - installs dependencies b - installs tools c - creates indexes d - points tools to indexes
j
________________________________ From: Enis Afgan [eafgan@emory.edu] Sent: Monday, August 01, 2011 12:54 PM To: Joseph Hargitai Cc: Galaxy Dev Subject: Re: [galaxy-user] mi-tools- tools_fabfile.py permissions requirements (local install)
Well, you can give it a shot like that and see if it still works. I quit using (and thus updating) the R-installation methods a while ago now so not sure if the process was changed. There is high likelihood that R will work (you should probably update the version number in the method itself). rpy however, will probably not work - rpy was very hard to install to begin with and that is why we moved away from this approach. Alternatively, you can add packages I listed in my last email to the list of packages under _required_packages method.
Enis
On Mon, Aug 1, 2011 at 12:49 PM, Joseph Hargitai <joseph.hargitai@einstein.yu.edumailto:joseph.hargitai@einstein.yu.edu> wrote: Hi,
so for local install i can uncomment the R and rpy from your script as far as at the _install_R.. part?
j
________________________________ From: Enis Afgan [eafgan@emory.edumailto:eafgan@emory.edu] Sent: Monday, August 01, 2011 12:37 PM To: Joseph Hargitai Cc: Galaxy Dev Subject: Re: [galaxy-user] mi-tools- tools_fabfile.py permissions requirements (local install)
Hi Joe, mi-deployment scripts are currently being used to create cloud images so this is the correct code your looking at. However, when it comes to R and rpy, those tools are (as you discovered) not being installed via those scripts; they are installed from packages but, in the case of Galaxy cloud images, they are inherited from the CloudBioLinux images that the Galaxy cloud images build on top of so not even included in the mi-deployment scripts. If you would like to follow suit and install those tools from packages, these are the ones (for debian): r-base, r-base-core, r-base-core-ra, r-base-dev, r-base-html, python-rpy
As far as the permissions issues goes, I'll look (over the next couple of days) more closely at those and focus on consistency.
Enis
On Mon, Aug 1, 2011 at 11:04 AM, Peter Cock <p.j.a.cock@googlemail.commailto:p.j.a.cock@googlemail.com> wrote: Hi Jeseph,
I see now you are talking about Enis' code for automated building of virtual machine Galaxy images, https://bitbucket.org/afgane/mi-deployment/src - mentioned briefly here: http://wiki.g2.bx.psu.edu/Admin/NGS%20Local%20Setup
I know very little about this area of Galaxy - we install any tools needed for Galaxy by hand (and try to document how we did it for future reference).
On Mon, Aug 1, 2011 at 3:50 PM, Joseph Hargitai <joseph.hargitai@einstein.yu.edumailto:joseph.hargitai@einstein.yu.edu> wrote:
R - not the correct download url,
Which URL are you talking about? The wiki page on dependencies http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Dependencies correctly links to the R project as http://www.r-project.org/
you are correct, however the tools_fabfile.py has
version = "2.11.1" url = "http://mira.sunsite.utk.edu/CRAN/src/base/R-2/R-%s.tar.gz" % version
That was a CRAN mirror, which seems to have gone now. Not listed here: http://cran.r-project.org/mirrors.html
rpy - will this be fixed?
What needs to be fixed? Moving Galaxy to rpy2? https://bitbucket.org/galaxy/galaxy-central/issue/103/upgrade-rpy-to-latest-...
again script has a comment:
def _install_rpy(): # *Does not work in reality*
Do you mean installing Galaxy without admin rights (i.e. without using sudo)?
either way - but should work. The fab script for instance adds some jar files to the galaxy-dist structure - then more added via the same script as sudo, and when it wants to change permissions for the entire folder for the next application the script fails. Hence just curious what a smoother solution would be.
In general - is the cloud install using the mi- script to install the tools? If so, where is it located? I'd like to compare it to the one i use for the local install. If it uses another script where/what is it?
thanks, joe
Regards,
Peter
Hi Joe, I just updated the code for the Galaxy installation process as part of mi-deployment to use the user defined via env.galaxy_user. There is still a (slightly guarded) chown called at the end of the script run due to the way other tools are being installed. We keep talking about rewriting this code entirely so it hasn't been generalized terribly much, which means that it is likely that some customizations for the current deployment will be desired. Overall, if you have suggestions and/or would like to fork the project and issue pull requests with code changes, please do, I'd appreciate all of that a lot.
Let me know if this change helps.
Enis
On Mon, Aug 1, 2011 at 2:44 PM, Joseph Hargitai < joseph.hargitai@einstein.yu.edu> wrote:
Cool,
one question remains:
since the cloud instance works and installs - there needs to be a script that is fully functional -
a - installs dependencies b - installs tools c - creates indexes d - points tools to indexes
j
*From:* Enis Afgan [eafgan@emory.edu] *Sent:* Monday, August 01, 2011 12:54 PM
*To:* Joseph Hargitai *Cc:* Galaxy Dev *Subject:* Re: [galaxy-user] mi-tools- tools_fabfile.py permissions requirements (local install)
Well, you can give it a shot like that and see if it still works. I quit using (and thus updating) the R-installation methods a while ago now so not sure if the process was changed. There is high likelihood that R will work (you should probably update the version number in the method itself). rpy however, will probably not work - rpy was very hard to install to begin with and that is why we moved away from this approach. Alternatively, you can add packages I listed in my last email to the list of packages under _required_packages method.
Enis
On Mon, Aug 1, 2011 at 12:49 PM, Joseph Hargitai < joseph.hargitai@einstein.yu.edu> wrote:
Hi,
so for local install i can uncomment the R and rpy from your script as far as at the _install_R.. part?
j
*From:* Enis Afgan [eafgan@emory.edu] *Sent:* Monday, August 01, 2011 12:37 PM *To:* Joseph Hargitai *Cc:* Galaxy Dev *Subject:* Re: [galaxy-user] mi-tools- tools_fabfile.py permissions requirements (local install)
Hi Joe, mi-deployment scripts are currently being used to create cloud images so this is the correct code your looking at. However, when it comes to R and rpy, those tools are (as you discovered) not being installed via those scripts; they are installed from packages but, in the case of Galaxy cloud images, they are inherited from the CloudBioLinux images that the Galaxy cloud images build on top of so not even included in the mi-deployment scripts. If you would like to follow suit and install those tools from packages, these are the ones (for debian): r-base, r-base-core, r-base-core-ra, r-base-dev, r-base-html, python-rpy
As far as the permissions issues goes, I'll look (over the next couple of days) more closely at those and focus on consistency.
Enis
On Mon, Aug 1, 2011 at 11:04 AM, Peter Cock p.j.a.cock@googlemail.comwrote:
Hi Jeseph,
I see now you are talking about Enis' code for automated building of virtual machine Galaxy images, https://bitbucket.org/afgane/mi-deployment/src - mentioned briefly here: http://wiki.g2.bx.psu.edu/Admin/NGS%20Local%20Setup
I know very little about this area of Galaxy - we install any tools needed for Galaxy by hand (and try to document how we did it for future reference).
On Mon, Aug 1, 2011 at 3:50 PM, Joseph Hargitai joseph.hargitai@einstein.yu.edu wrote:
R - not the correct download url,
Which URL are you talking about? The wiki page on dependencies http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Dependencies correctly links to the R project as http://www.r-project.org/
you are correct, however the tools_fabfile.py has
version = "2.11.1" url = "http://mira.sunsite.utk.edu/CRAN/src/base/R-2/R-%s.tar.gz" %
version
That was a CRAN mirror, which seems to have gone now. Not listed here: http://cran.r-project.org/mirrors.html
rpy - will this be fixed?
What needs to be fixed? Moving Galaxy to rpy2?
https://bitbucket.org/galaxy/galaxy-central/issue/103/upgrade-rpy-to-latest-...
again script has a comment:
def _install_rpy(): # *Does not work in reality*
Do you mean installing Galaxy without admin rights (i.e. without using
sudo)?
either way - but should work. The fab script for instance adds some jar
files
to the galaxy-dist structure - then more added via the same script as
sudo,
and when it wants to change permissions for the entire folder for the
next
application the script fails. Hence just curious what a smoother
solution would be.
In general - is the cloud install using the mi- script to install the
tools? If so,
where is it located? I'd like to compare it to the one i use for the
local install.
If it uses another script where/what is it?
thanks, joe
Regards,
Peter
galaxy-dev@lists.galaxyproject.org
-
Enis Afgan -
Joseph Hargitai -
Peter Cock