Can't get automatic toolshed installation to work....
Hi all, We just upgraded our local galaxy to the latest version and now I am trying to get automatic tool installation to work properly. I changed the universe_wsgi.ini file to enable the toolshed xml file, I created a "shed_tools" directory one above the galaxy install, and restarted galaxy. But when I try to install a repository, I get this error as a popup: "Initializing repository installation failed". I tried this with various repositories, but I get the same error. Then when I go to "Manage installed tool shed repositories", all of the repositories I tried to install are stuck in the "Cloning" state. Finally, if I try to uninstall any of them, I get this error, with a traceback: AttributeError: 'NoneType' object has no attribute 'startswith' Any ideas? - Nik. -- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w)
Hello Nikhil, On Aug 2, 2012, at 9:32 PM, Nikhil Joshi wrote:
Hi all,
We just upgraded our local galaxy to the latest version and now I am trying to get automatic tool installation to work properly. I changed the universe_wsgi.ini file to enable the toolshed xml file,
Regarding this, do you mean that you have something like the following setting? tool_config_files = tool_conf.xml,shed_tool_conf.xml
I created a "shed_tools" directory one above the galaxy install, and restarted galaxy. But when I try to install a repository, I get this error as a popup: "Initializing repository installation failed".
From what tool shed are you trying to install Galaxy? When this pop-up is displayed, the last line of your Galaxy paster log should be a url. If you paste that url into a browser it will display a page that displays the entire stack trace for the exception. We'll need to see this information in order to determine the cause of the problem.
I tried this with various repositories, but I get the same error. Then when I go to "Manage installed tool shed repositories", all of the repositories I tried to install are stuck in the "Cloning" state. Finally, if I try to uninstall any of them, I get this error, with a traceback:
AttributeError: 'NoneType' object has no attribute 'startswith'
Any ideas?
- Nik.
-- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w) ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Hi Greg, So I looked at the the URL from the log file and it was trying to write to a tmp directory owned by root, so that was a problem. I traced the problem to a python function (make_tmp_directory) where it was using the environment variable "TMPDIR", which was not defined. So in our launch script we defined the variable and that seemed to work. However, the repositories that I installed before fixing this are in a limbo state. When I try to uninstall them I get that same AttributeError below. Any ideas on how I can reset my installed Galaxy toolshed repositories so that they are not in a limbo state? I've included the entire traceback below: Error Traceback: View as: Interactive | Text | XML (full) ⇝ AttributeError: 'NoneType' object has no attribute 'startswith' URL: http://optidavis.genomecenter.ucdavis.edu:8080/admin_toolshed/browse_repositories?operation=deactivate+or+uninstall&id=33b43b4e7093c91f Module weberror.evalexception.middleware:364 in respond view
app_iter = self.application(environ, detect_start_response) Module paste.debug.prints:98 in __call__ view environ, self.app) Module paste.wsgilib:539 in intercept_output view app_iter = application(environ, replacement_start_response) Module paste.recursive:80 in __call__ view return self.application(environ, start_response) Module paste.httpexceptions:632 in __call__ view return self.application(environ, start_response) Module galaxy.web.framework.base:160 in __call__ view body = method( trans, **kwargs ) Module galaxy.web.framework:184 in decorator view return func( self, trans, *args, **kwargs ) Module galaxy.web.controllers.admin_toolshed:356 in browse_repositories view return self.deactivate_or_uninstall_repository( trans, **kwd ) Module galaxy.web.framework:184 in decorator view return func( self, trans, *args, **kwargs ) Module galaxy.web.controllers.admin_toolshed:408 in deactivate_or_uninstall_repository view repository_install_dir = os.path.abspath ( relative_install_dir ) Module posixpath:343 in abspath view if not isabs(path): Module posixpath:53 in isabs view return s.startswith('/') AttributeError: 'NoneType' object has no attribute 'startswith'
Any help is highly appreciated! BTW, it was really nice to get to talk to you at the conference. :) - Nik. On Fri, Aug 3, 2012 at 3:49 AM, Greg Von Kuster <greg@bx.psu.edu> wrote:
Hello Nikhil,
On Aug 2, 2012, at 9:32 PM, Nikhil Joshi wrote:
Hi all,
We just upgraded our local galaxy to the latest version and now I am trying to get automatic tool installation to work properly. I changed the universe_wsgi.ini file to enable the toolshed xml file,
Regarding this, do you mean that you have something like the following setting?
tool_config_files = tool_conf.xml,shed_tool_conf.xml
I created a "shed_tools" directory one above the galaxy install, and restarted galaxy. But when I try to install a repository, I get this error as a popup: "Initializing repository installation failed".
From what tool shed are you trying to install Galaxy? When this pop-up is displayed, the last line of your Galaxy paster log should be a url. If you paste that url into a browser it will display a page that displays the entire stack trace for the exception. We'll need to see this information in order to determine the cause of the problem.
I tried this with various repositories, but I get the same error. Then when I go to "Manage installed tool shed repositories", all of the repositories I tried to install are stuck in the "Cloning" state. Finally, if I try to uninstall any of them, I get this error, with a traceback:
AttributeError: 'NoneType' object has no attribute 'startswith'
Any ideas?
- Nik.
-- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w) ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
-- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w)
To get the repositories out of the limbo state, you'll have to get into your database and run the following sql command. Doing this will allow you to install those repositories that were in limbo. After they are installed, you can uninstall them if you want to. The case on the Strings is necessary. update tool_shed_repository set status = 'New' where status = 'Cloning'; It was great to talk with you at the conference as well! Greg On Aug 3, 2012, at 8:36 PM, Nikhil Joshi wrote:
Hi Greg,
So I looked at the the URL from the log file and it was trying to write to a tmp directory owned by root, so that was a problem. I traced the problem to a python function (make_tmp_directory) where it was using the environment variable "TMPDIR", which was not defined. So in our launch script we defined the variable and that seemed to work. However, the repositories that I installed before fixing this are in a limbo state. When I try to uninstall them I get that same AttributeError below. Any ideas on how I can reset my installed Galaxy toolshed repositories so that they are not in a limbo state? I've included the entire traceback below:
Error Traceback: View as: Interactive | Text | XML (full) ⇝ AttributeError: 'NoneType' object has no attribute 'startswith' URL: http://optidavis.genomecenter.ucdavis.edu:8080/admin_toolshed/browse_repositories?operation=deactivate+or+uninstall&id=33b43b4e7093c91f Module weberror.evalexception.middleware:364 in respond view
app_iter = self.application(environ, detect_start_response) Module paste.debug.prints:98 in __call__ view environ, self.app) Module paste.wsgilib:539 in intercept_output view app_iter = application(environ, replacement_start_response) Module paste.recursive:80 in __call__ view return self.application(environ, start_response) Module paste.httpexceptions:632 in __call__ view return self.application(environ, start_response) Module galaxy.web.framework.base:160 in __call__ view body = method( trans, **kwargs ) Module galaxy.web.framework:184 in decorator view return func( self, trans, *args, **kwargs ) Module galaxy.web.controllers.admin_toolshed:356 in browse_repositories view return self.deactivate_or_uninstall_repository( trans, **kwd ) Module galaxy.web.framework:184 in decorator view return func( self, trans, *args, **kwargs ) Module galaxy.web.controllers.admin_toolshed:408 in deactivate_or_uninstall_repository view repository_install_dir = os.path.abspath ( relative_install_dir ) Module posixpath:343 in abspath view if not isabs(path): Module posixpath:53 in isabs view return s.startswith('/') AttributeError: 'NoneType' object has no attribute 'startswith'
Any help is highly appreciated! BTW, it was really nice to get to talk to you at the conference. :)
- Nik.
On Fri, Aug 3, 2012 at 3:49 AM, Greg Von Kuster <greg@bx.psu.edu> wrote:
Hello Nikhil,
On Aug 2, 2012, at 9:32 PM, Nikhil Joshi wrote:
Hi all,
We just upgraded our local galaxy to the latest version and now I am trying to get automatic tool installation to work properly. I changed the universe_wsgi.ini file to enable the toolshed xml file,
Regarding this, do you mean that you have something like the following setting?
tool_config_files = tool_conf.xml,shed_tool_conf.xml
I created a "shed_tools" directory one above the galaxy install, and restarted galaxy. But when I try to install a repository, I get this error as a popup: "Initializing repository installation failed".
From what tool shed are you trying to install Galaxy? When this pop-up is displayed, the last line of your Galaxy paster log should be a url. If you paste that url into a browser it will display a page that displays the entire stack trace for the exception. We'll need to see this information in order to determine the cause of the problem.
I tried this with various repositories, but I get the same error. Then when I go to "Manage installed tool shed repositories", all of the repositories I tried to install are stuck in the "Cloning" state. Finally, if I try to uninstall any of them, I get this error, with a traceback:
AttributeError: 'NoneType' object has no attribute 'startswith'
Any ideas?
- Nik.
-- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w) ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
-- Nikhil Joshi Bioinformatics Analyst/Programmer UC Davis Bioinformatics Core http://bioinformatics.ucdavis.edu/ najoshi -at- ucdavis -dot- edu 530.752.2698 (w)
Dear Greg and Nikhil, We just installed a local instance of galaxy and encountered the same problem as Nikhil for the tools we tried to install from toolshed. Unfortunately I am not skilled as you are and didn't manage to solve the problem from the information in your posts. In my case I have in the univers_wgi.ini file: tool_config_file = tool_conf.xml,shed_tool_conf.xml ... with, in my main galaxy installation directory, a shed_tools link to the directory /var/lib/galaxy-server/shed_tools The problem is that when I try to install a tool from toolshed, it shows me the popup with "Initializing repository installation failed" and after that the tool stays in the cloning state for ever. When I try to uninstall it shows me the following message with traceback: AttributeError: 'NoneType' object has no attribute 'startswith' The entire stack trace for the exception I get just after the popup is as follow: Error Traceback: View as: Interactive | Text | XML (full) ⇝ IOError: [Errno 2] No such file or directory: u'shed_tools/shed_tools/toolshed.g2.bx.psu.edu/repos/malex/beast/677344fb75c1 /beast/beast/beast.xml' URL: http://192.167.142.244:8080/admin_toolshed/manage_repositories File '/usr/lib/galaxy-server/eggs/WebError-0.8a-py2.7.egg/weberror/evalexception /middleware.py', line 364 in respond app_iter = self.application(environ, detect_start_response) File '/usr/lib/galaxy-server/eggs/Paste-1.6-py2.7.egg/paste/debug/prints.py', line 98 in __call__ environ, self.app) File '/usr/lib/galaxy-server/eggs/Paste-1.6-py2.7.egg/paste/wsgilib.py', line 539 in intercept_output app_iter = application(environ, replacement_start_response) File '/usr/lib/galaxy-server/eggs/Paste-1.6-py2.7.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/usr/lib/galaxy-server/eggs/Paste-1.6-py2.7.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/usr/lib/galaxy-server/lib/galaxy/web/framework/base.py', line 160 in __call__ body = method( trans, **kwargs ) File '/usr/lib/galaxy-server/lib/galaxy/web/framework/__init__.py', line 184 in decorator return func( self, trans, *args, **kwargs ) File '/usr/lib/galaxy-server/lib/galaxy/web/controllers/admin_toolshed.py', line 846 in manage_repositories self.install_tool_shed_repositories( trans, repositories_for_installation, reinstalling=reinstalling, **decoded_kwd ) File '/usr/lib/galaxy-server/lib/galaxy/web/framework/__init__.py', line 184 in decorator return func( self, trans, *args, **kwargs ) File '/usr/lib/galaxy-server/lib/galaxy/web/controllers/admin_toolshed.py', line 634 in install_tool_shed_repositories reinstalling=reinstalling ) File '/usr/lib/galaxy-server/lib/galaxy/web/controllers/admin_toolshed.py', line 707 in handle_repository_contents repository_tools_tups, sample_files_copied = handle_missing_index_file( trans.app, tool_path, sample_files, repository_tools_tups ) File '/usr/lib/galaxy-server/lib/galaxy/util/shed_util.py', line 1180 in handle_missing_index_file repository_tool = app.toolbox.load_tool( os.path.join( tool_path, tup_path ), guid=guid ) File '/usr/lib/galaxy-server/lib/galaxy/tools/__init__.py', line 437 in load_tool tree = util.parse_xml( config_file ) File '/usr/lib/galaxy-server/lib/galaxy/util/__init__.py', line 106 in parse_xml tree = ElementTree.parse(fname) File '/usr/lib/galaxy-server/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree /ElementTree.py', line 859 in parse File '/usr/lib/galaxy-server/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree /ElementTree.py', line 576 in parse IOError: [Errno 2] No such file or directory: u'shed_tools/shed_tools/toolshed.g2.bx.psu.edu/repos/malex/beast/677344fb75c1 /beast/beast/beast.xml' I didn't manage to understand what is the original error to see if the problem is the exact same as for Nikhil. Therefore I have three questions: - how do I get in the database to run the following command: update tool_shed_repository set status = 'New' where status = 'Cloning' - how do I check for the problem that Nikhil underpinned (the python function with environment variable TMPDIR undefined)? - how do I implement the solution Nikhil gave in the case I see it is the same problem? Thank you very much in advance, best regards, fab.
Hello Fab, What version of Galaxy are you running? Did you install it from the galaxy-dist repo on bitbucket? Am I understaning correctly that you are using a symbolic link to point your installation directory to /var/lib/galaxy-server/shed_tools? If so, why not just set the config setting to that path? From your paster log, it looks like that directory is not being used:
IOError: [Errno 2] No such file or directory: u'shed_tools/shed_tools/toolshed.g2.bx.psu.edu/repos/malex/beast/677344fb75c1 /beast/beast/beast.xml'
See my inline comments for your 3 questions. On Nov 27, 2012, at 4:10 AM, fab wrote:
Dear Greg and Nikhil,
We just installed a local instance of galaxy and encountered the same problem as Nikhil for the tools we tried to install from toolshed. Unfortunately I am not skilled as you are and didn't manage to solve the problem from the information in your posts.
In my case I have in the univers_wgi.ini file: tool_config_file = tool_conf.xml,shed_tool_conf.xml ... with, in my main galaxy installation directory, a shed_tools link to the directory /var/lib/galaxy-server/shed_tools
The problem is that when I try to install a tool from toolshed, it shows me the popup with "Initializing repository installation failed" and after that the tool stays in the cloning state for ever. When I try to uninstall it shows me the following message with traceback: AttributeError: 'NoneType' object has no attribute 'startswith' The entire stack trace for the exception I get just after the popup is as follow:
Error Traceback: View as: Interactive | Text | XML (full) ⇝ IOError: [Errno 2] No such file or directory: u'shed_tools/shed_tools/toolshed.g2.bx.psu.edu/repos/malex/beast/677344fb75c1 /beast/beast/beast.xml' URL: http://192.167.142.244:8080/admin_toolshed/manage_repositories File '/usr/lib/galaxy-server/eggs/WebError-0.8a-py2.7.egg/weberror/evalexception /middleware.py', line 364 in respond app_iter = self.application(environ, detect_start_response) File '/usr/lib/galaxy-server/eggs/Paste-1.6-py2.7.egg/paste/debug/prints.py', line 98 in __call__ environ, self.app) File '/usr/lib/galaxy-server/eggs/Paste-1.6-py2.7.egg/paste/wsgilib.py', line 539 in intercept_output app_iter = application(environ, replacement_start_response) File '/usr/lib/galaxy-server/eggs/Paste-1.6-py2.7.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/usr/lib/galaxy-server/eggs/Paste-1.6-py2.7.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/usr/lib/galaxy-server/lib/galaxy/web/framework/base.py', line 160 in __call__ body = method( trans, **kwargs ) File '/usr/lib/galaxy-server/lib/galaxy/web/framework/__init__.py', line 184 in decorator return func( self, trans, *args, **kwargs ) File '/usr/lib/galaxy-server/lib/galaxy/web/controllers/admin_toolshed.py', line 846 in manage_repositories self.install_tool_shed_repositories( trans, repositories_for_installation, reinstalling=reinstalling, **decoded_kwd ) File '/usr/lib/galaxy-server/lib/galaxy/web/framework/__init__.py', line 184 in decorator return func( self, trans, *args, **kwargs ) File '/usr/lib/galaxy-server/lib/galaxy/web/controllers/admin_toolshed.py', line 634 in install_tool_shed_repositories reinstalling=reinstalling ) File '/usr/lib/galaxy-server/lib/galaxy/web/controllers/admin_toolshed.py', line 707 in handle_repository_contents repository_tools_tups, sample_files_copied = handle_missing_index_file( trans.app, tool_path, sample_files, repository_tools_tups ) File '/usr/lib/galaxy-server/lib/galaxy/util/shed_util.py', line 1180 in handle_missing_index_file repository_tool = app.toolbox.load_tool( os.path.join( tool_path, tup_path ), guid=guid ) File '/usr/lib/galaxy-server/lib/galaxy/tools/__init__.py', line 437 in load_tool tree = util.parse_xml( config_file ) File '/usr/lib/galaxy-server/lib/galaxy/util/__init__.py', line 106 in parse_xml tree = ElementTree.parse(fname) File '/usr/lib/galaxy-server/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree /ElementTree.py', line 859 in parse File '/usr/lib/galaxy-server/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree /ElementTree.py', line 576 in parse IOError: [Errno 2] No such file or directory: u'shed_tools/shed_tools/toolshed.g2.bx.psu.edu/repos/malex/beast/677344fb75c1 /beast/beast/beast.xml'
I didn't manage to understand what is the original error to see if the problem is the exact same as for Nikhil. Therefore I have three questions: - how do I get in the database to run the following command: update tool_shed_repository set status = 'New' where status = 'Cloning'
It depends on the database you have configured in your community_wsgi.ini file. What database do you have configured there?
- how do I check for the problem that Nikhil underpinned (the python function with environment variable TMPDIR undefined)?
I'm not sure to what you are referring here, so I cannot answer this.
- how do I implement the solution Nikhil gave in the case I see it is the same problem?
Again, I'm not sure of the solution Nikhil gave, and I don't have time to investigate it. I'm fairly certain, however, that the solution is outdated, as the features for handling installation errors have been vastly enhanced since my email exchange with him.
Thank you very much in advance, best regards,
fab.
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
participants (3)
-
fab -
Greg Von Kuster -
Nikhil Joshi