How to remove a broken toolshed install
Hi all, I managed to break a toolshed-installed tool by fiddling with the files under shed_tools. This led to a situation in which the Galaxy admin interface claims the tool is still installed, but can't find any files for it. I manually put the repository files where I think they should go, but this didn't fix the situation, so what I really want to do is just get rid of it altogether and reinstall cleanly. I'm not certain that the tool was working properly before I fiddled with it, either. Galaxy won't let me uninstall, deactivate or update it (because it can't find it properly) and it won't let me install it (because it thinks it's installed). It also seems (judging by the last of the errors below) to be unable to find some config information that it expects, but I don't really understand what's going on there. So my question is: given a messy, screwed up install, how can I completely remove it and start from scratch? What are the different components and config files I need to remove it from and are they all manually accessible? Thanks in advance for any help! If it's relevant to my question, here are some of the behaviours I see currently: The tool appears as "Installed" under Admin -> Manage installed tool shed repositories, but doesn't show up in the tools panel. If I try Repository Actions -> Get repository updates , I get the error: The directory containing the installed repository named 'bcftools_view' cannot be found. But if I try Repository Actions -> Reset repository metadata , it apparently works, I get Metadata has been reset on repository bcftools_view. And, if I try to 'Deactivate or uninstall' the apparently-installed repository, I get: URL: http://galaxy-tut.genome.edu.au/admin_toolshed/deactivate_or_uninstall_repos... Module paste.exceptions.errormiddleware:144 in __call__
app_iter = self.application(environ, sr_checker) Module paste.debug.prints:106 in __call__ environ, self.app) Module paste.wsgilib:543 in intercept_output app_iter = application(environ, replacement_start_response) Module paste.recursive:84 in __call__ return self.application(environ, start_response) Module paste.httpexceptions:633 in __call__ return self.application(environ, start_response) Module galaxy.web.framework.base:160 in __call__ body = method( trans, **kwargs ) Module galaxy.web.framework:205 in decorator return func( self, trans, *args, **kwargs ) Module galaxy.webapps.galaxy.controllers.admin_toolshed:452 in deactivate_or_uninstall_repository remove_from_tool_panel( trans, tool_shed_repository, shed_tool_conf, uninstall=remove_from_disk_checked ) Module galaxy.util.shed_util:1781 in remove_from_tool_panel tool_panel_dict = generate_tool_panel_dict_from_shed_tool_conf_entries( trans, repository ) Module galaxy.util.shed_util:942 in generate_tool_panel_dict_from_shed_tool_conf_entries tree = util.parse_xml( shed_tool_conf ) Module galaxy.util:135 in parse_xml tree = ElementTree.parse(fname) Module elementtree.ElementTree:859 in parse Module elementtree.ElementTree:576 in parse TypeError: coercing to Unicode: need string or buffer, NoneType found
Thanks, Clare -- Clare Sloggett Research Fellow / Bioinformatician Life Sciences Computation Centre Victorian Life Sciences Computation Initiative University of Melbourne, Parkville Campus 187 Grattan Street, Carlton, Melbourne Victoria 3010, Australia Ph: 03 903 53357 M: 0414 854 759
Hello Clare, On Oct 16, 2012, at 1:02 AM, Clare Sloggett wrote:
Hi all,
I managed to break a toolshed-installed tool by fiddling with the files under shed_tools.
As you've discovered, this is not a good thing to try. Always use the Galaxy interface features to perform tasks like this.
This led to a situation in which the Galaxy admin interface claims the tool is still installed, but can't find any files for it. I manually put the repository files where I think they should go, but this didn't fix the situation, so what I really want to do is just get rid of it altogether and reinstall cleanly. I'm not certain that the tool was working properly before I fiddled with it, either.
Depending on the changes you've made, you should be able to do the following: 1. Manually remove the installed repository subdirectory hierarchy from disk. 2. If the repository included any tools, manually remove entries for each of them from the shed_tool-conf.xml file ( or the equivalent file you have configured for handling installed repositories ) 3. Manually update the database using the following command (assuming your installed repository is named 'bcftools_view' and it is the only repository you have installed with that name) - letter capitalization is required: The following assumes you're using postgres: update tool_shed_repository set deleted=True, uninstalled=True, status='Uninstalled', error_message=Null where name = 'bcftools_view';
Galaxy won't let me uninstall, deactivate or update it (because it can't find it properly) and it won't let me install it (because it thinks it's installed). It also seems (judging by the last of the errors below) to be unable to find some config information that it expects, but I don't really understand what's going on there.
So my question is: given a messy, screwed up install, how can I completely remove it and start from scratch? What are the different components and config files I need to remove it from and are they all manually accessible?
Thanks in advance for any help!
If it's relevant to my question, here are some of the behaviours I see currently:
The tool appears as "Installed" under Admin -> Manage installed tool shed repositories, but doesn't show up in the tools panel.
If I try Repository Actions -> Get repository updates , I get the error: The directory containing the installed repository named 'bcftools_view' cannot be found.
But if I try Repository Actions -> Reset repository metadata , it apparently works, I get Metadata has been reset on repository bcftools_view.
And, if I try to 'Deactivate or uninstall' the apparently-installed repository, I get:
URL: http://galaxy-tut.genome.edu.au/admin_toolshed/deactivate_or_uninstall_repos... Module paste.exceptions.errormiddleware:144 in __call__
app_iter = self.application(environ, sr_checker) Module paste.debug.prints:106 in __call__ environ, self.app) Module paste.wsgilib:543 in intercept_output app_iter = application(environ, replacement_start_response) Module paste.recursive:84 in __call__ return self.application(environ, start_response) Module paste.httpexceptions:633 in __call__ return self.application(environ, start_response) Module galaxy.web.framework.base:160 in __call__ body = method( trans, **kwargs ) Module galaxy.web.framework:205 in decorator return func( self, trans, *args, **kwargs ) Module galaxy.webapps.galaxy.controllers.admin_toolshed:452 in deactivate_or_uninstall_repository remove_from_tool_panel( trans, tool_shed_repository, shed_tool_conf, uninstall=remove_from_disk_checked ) Module galaxy.util.shed_util:1781 in remove_from_tool_panel tool_panel_dict = generate_tool_panel_dict_from_shed_tool_conf_entries( trans, repository ) Module galaxy.util.shed_util:942 in generate_tool_panel_dict_from_shed_tool_conf_entries tree = util.parse_xml( shed_tool_conf ) Module galaxy.util:135 in parse_xml tree = ElementTree.parse(fname) Module elementtree.ElementTree:859 in parse Module elementtree.ElementTree:576 in parse TypeError: coercing to Unicode: need string or buffer, NoneType found
Thanks, Clare
--
Clare Sloggett Research Fellow / Bioinformatician Life Sciences Computation Centre Victorian Life Sciences Computation Initiative University of Melbourne, Parkville Campus 187 Grattan Street, Carlton, Melbourne Victoria 3010, Australia Ph: 03 903 53357 M: 0414 854 759 ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Hi Greg, Thanks for this! On 17 October 2012 01:17, Greg Von Kuster <greg@bx.psu.edu> wrote:
I managed to break a toolshed-installed tool by fiddling with the files under shed_tools.
As you've discovered, this is not a good thing to try. Always use the Galaxy interface features to perform tasks like this.
Actually, the reason I did this was because I didn't know how to solve a different problem, so maybe I should ask you about that one as well. We had some occasions where we'd try to install a tool from the toolshed, and it would hang - it appeared that the hg pull was timing out. In these cases the config files wouldn't get set up, but a partial repository was pulled / directories were created, and the repository files would then get in the way of trying to install the tool (it seemed to think it was already there). The only way to fix it seemed to be to manually delete the partially-pulled repository under shed_tools. This worked fine for fixing failed installs. But, this time, I thought (wrongly) that this had happened again and I deleted a repository - then realised that it was actually installed and registered in the database, etc. So, if the hg pull times out, is there a right way to clean up the resulting files? I got in the habit of doing it manually, which of course is dangerous, because I didn't know any way to do it via the admin interface.
Depending on the changes you've made, you should be able to do the following:
1. Manually remove the installed repository subdirectory hierarchy from disk. 2. If the repository included any tools, manually remove entries for each of them from the shed_tool-conf.xml file ( or the equivalent file you have configured for handling installed repositories ) 3. Manually update the database using the following command (assuming your installed repository is named 'bcftools_view' and it is the only repository you have installed with that name) - letter capitalization is required:
The following assumes you're using postgres:
update tool_shed_repository set deleted=True, uninstalled=True, status='Uninstalled', error_message=Null where name = 'bcftools_view';
Thanks very much! Yes it's postgres. I'll let you know if I succeed. Clare -- Clare Sloggett Research Fellow / Bioinformatician Life Sciences Computation Centre Victorian Life Sciences Computation Initiative University of Melbourne, Parkville Campus 187 Grattan Street, Carlton, Melbourne Victoria 3010, Australia Ph: 03 903 53357 M: 0414 854 759
Hello Clare, On Oct 17, 2012, at 2:22 AM, Clare Sloggett wrote:
Actually, the reason I did this was because I didn't know how to solve a different problem, so maybe I should ask you about that one as well.
We had some occasions where we'd try to install a tool from the toolshed, and it would hang - it appeared that the hg pull was timing out.
Was the timeout a regular occurrence? if so, do you know that cause, and were you able to get it resolved?
In these cases the config files wouldn't get set up, but a partial repository was pulled / directories were created, and the repository files would then get in the way of trying to install the tool (it seemed to think it was already there). The only way to fix it seemed to be to manually delete the partially-pulled repository under shed_tools.
The October 5, 2012 Galaxy distribution news brief includes the following link to information about the process for handling repository installation errors, specifically when the errors occur during cloning. http://wiki.g2.bx.psu.edu/InstallingRepositoriesToGalaxy#Handling_repository... If you're running an older version of Galaxy, you'll need to update to the October 5 release in order to have these features. The news brief release information is: upgrade: $ hg pull -u -r b5bda7a5c345 Let me know if this is not what you're looking for. Thanks, Greg Von Kuster
This worked fine for fixing failed installs. But, this time, I thought (wrongly) that this had happened again and I deleted a repository - then realised that it was actually installed and registered in the database, etc.
So, if the hg pull times out, is there a right way to clean up the resulting files? I got in the habit of doing it manually, which of course is dangerous, because I didn't know any way to do it via the admin interface.
Clare Sloggett Research Fellow / Bioinformatician Life Sciences Computation Centre Victorian Life Sciences Computation Initiative University of Melbourne, Parkville Campus 187 Grattan Street, Carlton, Melbourne Victoria 3010, Australia Ph: 03 903 53357 M: 0414 854 759 ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Hi Greg,
We had some occasions where we'd try to install a tool from the toolshed, and it would hang - it appeared that the hg pull was timing out.
Was the timeout a regular occurrence? if so, do you know that cause, and were you able to get it resolved?
It was repeated, but after a few tries the install would succeed without me really 'resolving' the issue. We haven't run into the issue in a while and I honestly have no idea if this is due to a galaxy upgrade or if we had a temporary run of bad luck with those particular tools.
The October 5, 2012 Galaxy distribution news brief includes the following link to information about the process for handling repository installation errors, specifically when the errors occur during cloning.
http://wiki.g2.bx.psu.edu/InstallingRepositoriesToGalaxy#Handling_repository...
If you're running an older version of Galaxy, you'll need to update to the October 5 release in order to have these features.
The news brief release information is:
upgrade: $ hg pull -u -r b5bda7a5c345
Let me know if this is not what you're looking for.
Thanks again, this is great. I think we have upgraded past that point now. Clare -- Clare Sloggett Research Fellow / Bioinformatician Life Sciences Computation Centre Victorian Life Sciences Computation Initiative University of Melbourne, Parkville Campus 187 Grattan Street, Carlton, Melbourne Victoria 3010, Australia Ph: 03 903 53357 M: 0414 854 759
participants (2)
-
Clare Sloggett -
Greg Von Kuster