Re: [galaxy-dev] shed_tool_conf.xml not updated

Hi Anne, see my inline comments. On Jan 27, 2012, at 10:26 AM, Anne Pajon wrote: > Hi Greg, > > Thanks a lot. I have the impression from what I read on the wiki that Galaxy will be using the tool shed to provide more flexibility in choosing tools instead of having them bundle in the current distribution. We are very much interested by these new features because we only use a subset of the available tools plus some that we have developed in house. Yes, this is correct. At some point, a subset of the tools currently in the Galaxy distribution will be moved to the main Galaxy tool shed from which they can be easily installed using Galaxy's new installation manager. You'll be able to very easily install any portion of the tools that will be eliminated from the distribution. All of this is now functional in the latest version of Galaxy as well as the main Galaxy tool shed, but the timeline for beginning to eliminate tools from the distribution is yet to be determined. The tip of the central repository, which will be added to the Galaxy distribution repository today, includes the ability to deactivate or uninstall repositories that you've installed from the tool shed, enabling much more flexibility in managing the tools that are included in your Galaxy tool panel. The last section of the tool shed wiki describes this new feature set: http://wiki.g2.bx.psu.edu/Tool%20Shed#Deactivating_and_uninstalling_tool_... > >> From what you are saying, I have the impression that I will be better of running my Galaxy instance against the central repository instead of the distribution one if I do not want to wait for bug fixes. Am I right of thinking that? If so, I will migrate to the central repository and let you know how it goes. Although this is not ideal, it is unfortunately necessary at least for the current time. At some point, the communication layers between Galaxy and the tool shed will stabilize to the point that the distribution repository can be used, but in the meantime, the central repository is required. My advice for now would be to have your Galaxy instance track central, but not necessarily update it all of the time. I believe the update to the distribution going out today will be change set 6621:26920e20157f, which includes all new tool shed features. So although this is not the latest on the central repo, you can update to that revision if you want. To do so, change your hgrc to pull from Galaxy central and use the following. hg pull -u 26920e20157f If you are comfortable in updating to the latest from central, just use hg pull -u > > Before migrating and updating I will manually remove the installed tool shed repository from disk as you advised. > > Kind regards, > Anne. > > > On 27 Jan 2012, at 14:57, Greg Von Kuster wrote: > >> Hi Anne, >> >> A new update to the Galaxy distribution repository is planned for today, I believe, so unless that changes you should be able to update your Galaxy instance either today or Monday using the distribution repository. However, this problem will continue to exist between Galaxy instances that track the distribution repository and that also want to take advantage of the Galaxy tool shed features that enable direct interaction between Galaxy and the tool shed. The tool shed features are implement much more rapidly than the Galaxy distribution is updated, resulting in this ongoing problem. >> >> Regarding the state of your local instance and the tool shed repository you attempted to install - if I understand correctly, the repository was successfully installed on disk, but the XML tag set entry for the contained tool was not added to your shed_tool_conf.xml file. If this is the case, then you'll need to manually remove the installed tool shed repository from disk before you try installing it again after updating your Galaxy instance to the new upcoming distribution. >> >> Sorry for the inconvenience. >> >> Greg >> >> On Jan 27, 2012, at 9:35 AM, Anne Pajon wrote: >> >>> Hello Greg, >>> >>> Thanks a lot for your answer and sorry for having sent two emails. >>> >>> Regarding the version of our local instance, I understand perfectly that the Galaxy distribution repository is behind the Galaxy central repository but we would like to use Galaxy on a production server and we are looking at stable releases not development ones. What would you advice us to do? >>> >>> Here is the value for tool_path in my shed_tool_conf.xml: >>> <?xml version="1.0"?> >>> <toolbox tool_path="../shed_tools"> >>> </toolbox> >>> >>> I'm waiting your advice to migrate my repository from Galaxy distribution to Galaxy central. Thanks in advance. >>> >>> Kind regards, >>> Anne. >>> >>> >>> On 27 Jan 2012, at 13:03, Greg Von Kuster wrote: >>> >>>> Hello Anne, >>>> >>>> I apologize for the delay in responding to you on this. The first issue I see is that the revision of your local Galaxy instance implies that your repo is not being updated from the Galaxy central repository on bitbucket. Unfortunately in order to use the tool shed, you local Galaxy instance must be up to date with the central repository since the Galaxy distribution repository generally lags 6 to 8 weeks behind the central repository, and so the communication between Galaxy and the tool shed is often not functionally correct. The central repository on bitbucket is current at revision 6626:d8af2f7b1f35. >>>> >>>> For clarification what value have you set for the tool_path config setting in your shed_tool_conf.xml file? >>>> >>>> After you update your Galaxy instance, try installing the tool again and let me know if you encounter issues. >>>> >>>> Thanks very much! >>>> >>>> Greg Von Kuster >>>> >>>> >>>> On Jan 27, 2012, at 5:55 AM, Anne Pajon wrote: >>>> >>>>> Hello, >>>>> >>>>> When installing a tool from the tool shed repository into our local instance, I've got a page with "Server error". When restarting Galaxy, the imported tool has disappeared from the blue left list of tools. After installing the tool, the file shed_tool_conf.xml has not been updated. >>>>> >>>>> Bellow is my previous email "server error when installing tool from tool shed repository into local instance" sent two days ago detailing what I have done and the log related to the error. >>>>> >>>>> Sorry to ask again but I did not solve the problem and I am still looking for help. I just wanted to re-formulate the issue in a shorter way. >>>>> >>>>> Let me know if you have any suggestions. >>>>> Kind regards, >>>>> Anne. >>>>> >>>>> >>>>> On 25 Jan 2012, at 16:43, Anne Pajon wrote: >>>>> >>>>>> Hello, >>>>>> >>>>>> I have the version 6528:63bc46cc73b7 of galaxy running locally on our server installed from https://bitbucket.org/galaxy/galaxy-dist/. >>>>>> >>>>>> I've tried to bring tabular2html tool from Galaxy main tool shed repository following the wiki information (http://wiki.g2.bx.psu.edu/Tool%20Shed#Automatic_installation_of_Galaxy_to...) without success. >>>>>> >>>>>> Here is what I've done. >>>>>> >>>>>> (1) I've edited my universe_wsgi.ini >>>>>> # -- Files and directories >>>>>> >>>>>> # Tool config files, defines what tools are available in Galaxy. >>>>>> # Tools can be locally developed or installed from Galaxy tool sheds. >>>>>> tool_config_file = tool_conf.xml,shed_tool_conf.xml >>>>>> >>>>>> # Default path to the directory containing the tools defined in tool_conf.xml. >>>>>> # Other tool config files must include the tool_path as an attribute in the <toolbox> tag. >>>>>> tool_path = tools >>>>>> >>>>>> (2) I've created the directory shed_tools/ above the galaxy-dist/ as recommended >>>>>> >>>>>> (3) I've restarted the server >>>>>> >>>>>> (4) I've logged in as "admin" user. I've clicked the "Admin" link in the top Galaxy tool panel to display the Galaxy Administration interface and then "Search and browse tool sheds" under "Tool sheds" from the left blue section to display the Accessible Galaxy tool sheds page, click "Galaxy main tool shed" to display the valid repositories and choose tabular2html and click on Install to local Galaxy, select the panel section and click Install. >>>>>> >>>>>> I get a page with "Server Error". >>>>>> >>>>>> Strangely, the tool appears in the list of tools and works in my Galaxy instance. The tool files have been installed on disk at the directory created in (2) but the shed_tool_conf.xml has not been updated. After restarting my Galaxy server, the tool is not visible anymore. >>>>>> >>>>>> Here is what appears in the log related to the installation of the tool: >>>>>> galaxy.util.shed_util DEBUG 2012-01-25 14:20:54,451 Installing repository 'tabular2html' >>>>>> galaxy.util.shed_util DEBUG 2012-01-25 14:20:54,451 Cloning http://toolshed.g2.bx.psu.edu/repos/ondovb/tabular2html >>>>>> galaxy.util.shed_util DEBUG 2012-01-25 14:20:55,486 Updating cloned repository to revision "a5814dd5a11a" >>>>>> galaxy.tools DEBUG 2012-01-25 14:20:55,652 Reloading section: My Tools >>>>>> galaxy.tools DEBUG 2012-01-25 14:20:55,673 Loaded tool id: toolshed.g2.bx.psu.edu/repos/ondovb/tabular2html/tabular2HTML/1.0.0, version: 1.0 >>>>>> .0. >>>>>> 143.65.172.178 - - [25/Jan/2012:14:20:54 +0100] "POST /galaxy/admin_toolshed/install_repository?tool_shed_url=http%3A%2F%2Ftoolshed.g2.bx.ps >>>>>> u.edu&repo_info_dict=3abd29129982504350fb737085131cc07e47cc03%3A7b22746162756c61723268746d6c223a205b22437265617465732048544d4c207461626c6573 >>>>>> 2066726f6d20746162756c617220646174612e222c2022687474703a2f2f746f6f6c736865642e67322e62782e7073752e6564752f7265706f732f6f6e646f76622f74616275 >>>>>> 6c61723268746d6c222c2022613538313464643561313161225d7d&includes_tools=True HTTP/1.1" 500 - "http://uk-cri-lbio08/galaxy/admin_toolshed/insta >>>>>> ll_repository?tool_shed_url=http://toolshed.g2.bx.psu.edu&repo_info_dict=3abd29129982504350fb737085131cc07e47cc03:7b22746162756c61723268746d >>>>>> 6c223a205b22437265617465732048544d4c207461626c65732066726f6d20746162756c617220646174612e222c2022687474703a2f2f746f6f6c736865642e67322e62782e >>>>>> 7073752e6564752f7265706f732f6f6e646f76622f746162756c61723268746d6c222c2022613538313464643561313161225d7d&includes_tools=True" "Mozilla/5.0 ( >>>>>> Macintosh; U; Intel Mac OS X 10.6; en-GB; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13" >>>>>> Error - <type 'exceptions.TypeError'>: cannot serialize None (type NoneType) >>>>>> URL: http://uk-cri-lbio08/galaxy/admin_toolshed/install_repository?tool_shed_u... >>>>>> 9982504350fb737085131cc07e47cc03%3A7b22746162756c61723268746d6c223a205b22437265617465732048544d4c207461626c65732066726f6d20746162756c6172206 >>>>>> 46174612e222c2022687474703a2f2f746f6f6c736865642e67322e62782e7073752e6564752f7265706f732f6f6e646f76622f746162756c61723268746d6c222c202261353 >>>>>> 8313464643561313161225d7d&includes_tools=True >>>>>> File '/opt/local/home/webapp/galaxy/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/exceptions/errormiddleware.py', line 143 in __call__ >>>>>> app_iter = self.application(environ, start_response) >>>>>> File '/opt/local/home/webapp/galaxy/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/recursive.py', line 80 in __call__ >>>>>> return self.application(environ, start_response) >>>>>> File '/opt/local/home/webapp/galaxy/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpexceptions.py', line 632 in __call__ >>>>>> return self.application(environ, start_response) >>>>>> File '/opt/local/home/webapp/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 160 in __call__ >>>>>> body = method( trans, **kwargs ) >>>>>> File '/opt/local/home/webapp/galaxy/galaxy-dist/lib/galaxy/web/framework/__init__.py', line 173 in decorator >>>>>> return func( self, trans, *args, **kwargs ) >>>>>> File '/opt/local/home/webapp/galaxy/galaxy-dist/lib/galaxy/web/controllers/admin_toolshed.py', line 275 in install_repository >>>>>> new_install=True ) >>>>>> File '/opt/local/home/webapp/galaxy/galaxy-dist/lib/galaxy/util/shed_util.py', line 644 in load_repository_contents >>>>>> add_to_config=True ) >>>>>> File '/opt/local/home/webapp/galaxy/galaxy-dist/lib/galaxy/util/shed_util.py', line 699 in load_altered_part_of_tool_panel >>>>>> add_shed_tool_conf_entry( app, shed_tool_conf, elem_entry ) >>>>>> File '/opt/local/home/webapp/galaxy/galaxy-dist/lib/galaxy/util/shed_util.py', line 41 in add_shed_tool_conf_entry >>>>>> new_shed_tool_conf.write( util.xml_to_string( tool_panel_entry, pretty=True ) ) >>>>>> File '/opt/local/home/webapp/galaxy/galaxy-dist/lib/galaxy/util/__init__.py', line 113 in xml_to_string >>>>>> return ElementTree.tostring( pretty_print_xml( elem ) ) >>>>>> File '/opt/local/home/webapp/galaxy/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py', line 1006 in tostring >>>>>> File '/opt/local/home/webapp/galaxy/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py', line 660 in write >>>>>> File '/opt/local/home/webapp/galaxy/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py', line 695 in _write >>>>>> File '/opt/local/home/webapp/galaxy/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py', line 827 in _escape_a >>>>>> ttrib >>>>>> File '/opt/local/home/webapp/galaxy/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py', line 774 in _raise_se >>>>>> rialization_error >>>>>> TypeError: cannot serialize None (type NoneType) >>>>>> >>>>>> Any help would be very welcome. Thanks a lot in advance. >>>>>> Kind regards, >>>>>> Anne. >>>>>> >>>>>> -- >>>>>> Anne Pajon, Ph.D. >>>>>> Cancer Research UK - Cambridge Research Institute >>>>>> Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE >>>>>> anne.pajon(a)cancer.org.uk | +44 (0)7958 511 353 >>>>>> >>>>>> >>>>> >>>>> -- >>>>> Anne Pajon, Ph.D. >>>>> Cancer Research UK - Cambridge Research Institute >>>>> Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE >>>>> anne.pajon(a)cancer.org.uk | +44 (0)7958 511 353 >>>>> >>>>> >>>>> >>>>> NOTICE AND DISCLAIMER >>>>> This e-mail (including any attachments) is intended for the above-named person(s). If you are not the intended recipient, notify the sender immediately, delete this email from your system and do not disclose or use for any purpose. >>>>> >>>>> We may monitor all incoming and outgoing emails in line with current legislation. We have taken steps to ensure that this email and attachments are free from any virus, but it remains your responsibility to ensure that viruses do not adversely affect you. >>>>> Cancer Research UK >>>>> Registered in England and Wales >>>>> Company Registered Number: 4325234. >>>>> Registered Charity Number: 1089464 and Scotland SC041666 >>>>> Registered Office Address: Angel Building, 407 St John Street, London EC1V 4AD. >>>>> >>>>> ___________________________________________________________ >>>>> Please keep all replies on the list by using "reply all" >>>>> in your mail client. To manage your subscriptions to this >>>>> and other Galaxy lists, please use the interface at: >>>>> >>>>> http://lists.bx.psu.edu/ >>>> >>>> Greg Von Kuster >>>> Galaxy Development Team >>>> greg(a)bx.psu.edu >>>> >>>> >>>> >>> >>> -- >>> Anne Pajon, Ph.D. >>> Cancer Research UK - Cambridge Research Institute >>> Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE >>> anne.pajon(a)cancer.org.uk | +44 (0)7958 511 353 >>> >>> >>> >>> NOTICE AND DISCLAIMER >>> This e-mail (including any attachments) is intended for the above-named person(s). If you are not the intended recipient, notify the sender immediately, delete this email from your system and do not disclose or use for any purpose. >>> >>> We may monitor all incoming and outgoing emails in line with current legislation. We have taken steps to ensure that this email and attachments are free from any virus, but it remains your responsibility to ensure that viruses do not adversely affect you. >>> Cancer Research UK >>> Registered in England and Wales >>> Company Registered Number: 4325234. >>> Registered Charity Number: 1089464 and Scotland SC041666 >>> Registered Office Address: Angel Building, 407 St John Street, London EC1V 4AD. >>> >> >> Greg Von Kuster >> Galaxy Development Team >> greg(a)bx.psu.edu >> >> >> > > -- > Anne Pajon, Ph.D. > Cancer Research UK - Cambridge Research Institute > Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE > anne.pajon(a)cancer.org.uk | +44 (0)7958 511 353 > > > > NOTICE AND DISCLAIMER > This e-mail (including any attachments) is intended for the above-named person(s). If you are not the intended recipient, notify the sender immediately, delete this email from your system and do not disclose or use for any purpose. > > We may monitor all incoming and outgoing emails in line with current legislation. We have taken steps to ensure that this email and attachments are free from any virus, but it remains your responsibility to ensure that viruses do not adversely affect you. > Cancer Research UK > Registered in England and Wales > Company Registered Number: 4325234. > Registered Charity Number: 1089464 and Scotland SC041666 > Registered Office Address: Angel Building, 407 St John Street, London EC1V 4AD. > > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ Greg Von Kuster Galaxy Development Team greg(a)bx.psu.edu