tool shed updates broken on my local server
Hi there, I am currently trying to migrate our Galaxy production server onto new hardware and I am facing many challenges. Recently I've been unable to update tools from the tool shed. During the migration I've moved across the files, the data, the tools and their dependencies as well as the database. But I am getting this error while trying to update any tools that are marked 'Updates are available in the Tool Shed for this revision' from the GUI: Admin > Manage installed tool shed repositories > fasta_formatter menu Get updates: Internal Server Error Galaxy was unable to successfully complete your request An error occurred. This may be an intermittent problem due to load or other unpredictable factors, reloading the page may address the problem. The error has been logged to our team. Here is the log from paster.log: 10.20.13.16 - - [29/Jan/2015:16:43:45 +0100] "GET /admin_toolshed/update_to_changeset_revision?tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=fasta_formatter&owner=devteam&changeset_revision=8f0ae92440b8&latest_changeset_revision=1dbb5181c327&latest_ctx_rev=1 HTTP/1.1" 500 - "http://bioinf-gal001:8080/admin_toolshed/browse_repositories?async=false&sort=name&page=1&show_item_checkboxes=false&advanced_search=false&f-deleted=False&f-free-text-search=fasta_formatter" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:35.0) Gecko/20100101 Firefox/35.0" Error - <type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'get' URL: http://bioinf-gal001:8080/admin_toolshed/update_to_changeset_revision?tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=fasta_formatter&owner=devteam&changeset_revision=8f0ae92440b8&latest_changeset_revision=1dbb5181c327&latest_ctx_rev=1 File '/mnt/nfs/data/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/mnt/nfs/data/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/mnt/nfs/data/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 190 in handle_request body = method( trans, **kwargs ) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/galaxy/web/framework/decorators.py', line 87 in decorator return func( self, trans, *args, **kwargs ) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py', line 1953 in update_to_changeset_revision persist=True ) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/metadata/installed_repository_metadata_manager.py', line 31 in __init__ metadata_dict=metadata_dict, user=None ) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/tool_shed/metadata/metadata_generator.py', line 60 in __init__ self.metadata_dict = { 'shed_config_filename' : self.shed_config_dict.get( 'config_filename', None ) } AttributeError: 'NoneType' object has no attribute 'get' CGI Variables ------------- CONTENT_LENGTH: '0' HTTP_ACCEPT: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' HTTP_ACCEPT_ENCODING: 'gzip, deflate' HTTP_ACCEPT_LANGUAGE: 'en-US,en;q=0.5' HTTP_CONNECTION: 'keep-alive' HTTP_COOKIE: 'galaxysession=c6ca0ddb55be603a44ec9cad39b01202307c62c9a25600e232d5d3d597344cb74ba4cd8a7123f6e9' HTTP_HOST: 'bioinf-gal001:8080' HTTP_REFERER: 'http://bioinf-gal001:8080/admin_toolshed/browse_repositories?async=false&sort=name&page=1&show_item_checkboxes=false&advanced_search=false&f-deleted=False&f-free-text-search=fasta_formatter' HTTP_USER_AGENT: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:35.0) Gecko/20100101 Firefox/35.0' PATH_INFO: '/admin_toolshed/update_to_changeset_revision' QUERY_STRING: 'tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=fasta_formatter&owner=devteam&changeset_revision=8f0ae92440b8&latest_changeset_revision=1dbb5181c327&latest_ctx_rev=1' REMOTE_ADDR: '10.20.13.16' REQUEST_METHOD: 'GET' SERVER_NAME: '0.0.0.0' SERVER_PORT: '8080' SERVER_PROTOCOL: 'HTTP/1.1' WSGI Variables -------------- application: <paste.recursive.RecursiveMiddleware object at 0x7f34f7a50590> is_api_request: False paste.cookies: (<SimpleCookie: galaxysession='c6ca0ddb55be603a44ec9cad39b01202307c62c9a25600e232d5d3d597344cb74ba4cd8a7123f6e9'>, 'galaxysession=c6ca0ddb55be603a44ec9cad39b01202307c62c9a25600e232d5d3d597344cb74ba4cd8a7123f6e9') paste.expected_exceptions: [<class 'paste.httpexceptions.HTTPException'>] paste.httpexceptions: <paste.httpexceptions.HTTPExceptionHandler object at 0x7f34f7a50550> paste.httpserver.thread_pool: <paste.httpserver.ThreadPool object at 0x7f34f76ef1d0> paste.parsed_querystring: ([('tool_shed_url', 'https://toolshed.g2.bx.psu.edu/'), ('name', 'fasta_formatter'), ('owner', 'devteam'), ('changeset_revision', '8f0ae92440b8'), ('latest_changeset_revision', '1dbb5181c327'), ('latest_ctx_rev', '1')], 'tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=fasta_formatter&owner=devteam&changeset_revision=8f0ae92440b8&latest_changeset_revision=1dbb5181c327&latest_ctx_rev=1') paste.recursive.forward: <paste.recursive.Forwarder from /> paste.recursive.include: <paste.recursive.Includer from /> paste.recursive.include_app_iter: <paste.recursive.IncluderAppIter from /> paste.recursive.script_name: '' paste.throw_errors: True request_id: 'fdaf9004a7d511e49d63005056a77528' webob._parsed_query_vars: (MultiDict([('tool_shed_url', 'https://toolshed.g2.bx.psu.edu/'), ('name', 'fasta_formatter'), ('owner', 'devteam'), ('changeset_revision', '8f0ae92440b8'), ('latest_changeset_revision', '1dbb5181c327'), ('latest_ctx_rev', '1')]), 'tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=fasta_formatter&owner=devteam&changeset_revision=8f0ae92440b8&latest_changeset_revision=1dbb5181c327&latest_ctx_rev=1') wsgi process: 'Multithreaded' ------------------------------------------------------------ Anything obvious that I've missed? Many thanks in advance for your help. Kind regards, Anne. -- Dr Anne Pajon - Bioinformatics Core Cancer Research UK - Cambridge Institute Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE anne.pajon@cruk.cam.ac.uk | +44 (0)1223 769 631
Hi Anne, moving the whole Galaxy installation is a tough job and we do not really have a good process for it. Your problem is most probably caused by the inconsistency in the DB. The path to the shed_tool_conf.xml is stored there and the changes you made to the location of the installation did not propagate there. Please have a look at the following script that Dave B. made: https://gist.github.com/InitHello/d208cd056baf088aca1f This script will soon be part of the distribution (in /scripts) and should help you solving your issues. Thank you for using Galaxy. Martin, Galaxy Team On Thu Jan 29 2015 at 11:47:19 AM Anne Pajon <Anne.Pajon@cruk.cam.ac.uk> wrote:
Hi there,
I am currently trying to migrate our Galaxy production server onto new hardware and I am facing many challenges. Recently I've been unable to update tools from the tool shed. During the migration I've moved across the files, the data, the tools and their dependencies as well as the database. But I am getting this error while trying to update any tools that are marked 'Updates are available in the Tool Shed for this revision' from the GUI: Admin > Manage installed tool shed repositories > fasta_formatter menu Get updates:
Internal Server Error
Galaxy was unable to successfully complete your request
An error occurred.
This may be an intermittent problem due to load or other unpredictable factors, reloading the page may address the problem. The error has been logged to our team.
Here is the log from paster.log:
10.20.13.16 - - [29/Jan/2015:16:43:45 +0100] "GET /admin_toolshed/update_to_changeset_revision?tool_shed_url= https://toolshed.g2.bx.psu.edu/&name=fasta_formatter& owner=devteam&changeset_revision=8f0ae92440b8&latest_changeset_revision= 1dbb5181c327&latest_ctx_rev=1 HTTP/1.1" 500 - "http://bioinf-gal001:8080/ admin_toolshed/browse_repositories?async=false&sort=name&page=1&show_item_ checkboxes=false&advanced_search=false&f-deleted=False& f-free-text-search=fasta_formatter" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:35.0) Gecko/20100101 Firefox/35.0" Error - <type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'get' URL: http://bioinf-gal001:8080/admin_toolshed/update_to_ changeset_revision?tool_shed_url=https://toolshed.g2.bx. psu.edu/&name=fasta_formatter&owner=devteam&changeset_ revision=8f0ae92440b8&latest_changeset_revision= 1dbb5181c327&latest_ctx_rev=1 File '/mnt/nfs/data/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/mnt/nfs/data/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/mnt/nfs/data/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 190 in handle_request body = method( trans, **kwargs ) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/galaxy/web/framework/decorators.py', line 87 in decorator return func( self, trans, *args, **kwargs ) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/galaxy/webapps/ galaxy/controllers/admin_toolshed.py', line 1953 in update_to_changeset_revision persist=True ) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/tool_shed/galaxy_ install/metadata/installed_repository_metadata_manager.py', line 31 in __init__ metadata_dict=metadata_dict, user=None ) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/tool_shed/metadata/metadata_generator.py', line 60 in __init__ self.metadata_dict = { 'shed_config_filename' : self.shed_config_dict.get( 'config_filename', None ) } AttributeError: 'NoneType' object has no attribute 'get'
CGI Variables ------------- CONTENT_LENGTH: '0' HTTP_ACCEPT: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*; q=0.8' HTTP_ACCEPT_ENCODING: 'gzip, deflate' HTTP_ACCEPT_LANGUAGE: 'en-US,en;q=0.5' HTTP_CONNECTION: 'keep-alive' HTTP_COOKIE: 'galaxysession=c6ca0ddb55be603a44ec9cad39b012 02307c62c9a25600e232d5d3d597344cb74ba4cd8a7123f6e9' HTTP_HOST: 'bioinf-gal001:8080' HTTP_REFERER: 'http://bioinf-gal001:8080/admin_toolshed/browse_ repositories?async=false&sort=name&page=1&show_item_ checkboxes=false&advanced_search=false&f-deleted=False& f-free-text-search=fasta_formatter' HTTP_USER_AGENT: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:35.0) Gecko/20100101 Firefox/35.0' PATH_INFO: '/admin_toolshed/update_to_changeset_revision' QUERY_STRING: 'tool_shed_url=https://toolshed.g2.bx.psu.edu/&name= fasta_formatter&owner=devteam&changeset_revision= 8f0ae92440b8&latest_changeset_revision=1dbb5181c327&latest_ctx_rev=1' REMOTE_ADDR: '10.20.13.16' REQUEST_METHOD: 'GET' SERVER_NAME: '0.0.0.0' SERVER_PORT: '8080' SERVER_PROTOCOL: 'HTTP/1.1'
WSGI Variables -------------- application: <paste.recursive.RecursiveMiddleware object at 0x7f34f7a50590> is_api_request: False paste.cookies: (<SimpleCookie: galaxysession=' c6ca0ddb55be603a44ec9cad39b01202307c62c9a25600e232d5d3d597344cb74ba4cd8a7123f6e9'>, 'galaxysession=c6ca0ddb55be603a44ec9cad39b012 02307c62c9a25600e232d5d3d597344cb74ba4cd8a7123f6e9') paste.expected_exceptions: [<class 'paste.httpexceptions. HTTPException'>] paste.httpexceptions: <paste.httpexceptions.HTTPExceptionHandler object at 0x7f34f7a50550> paste.httpserver.thread_pool: <paste.httpserver.ThreadPool object at 0x7f34f76ef1d0> paste.parsed_querystring: ([('tool_shed_url', ' https://toolshed.g2.bx.psu.edu/'), ('name', 'fasta_formatter'), ('owner', 'devteam'), ('changeset_revision', '8f0ae92440b8'), ('latest_changeset_revision', '1dbb5181c327'), ('latest_ctx_rev', '1')], 'tool_shed_url=https://toolshed.g2.bx.psu.edu/&name= fasta_formatter&owner=devteam&changeset_revision= 8f0ae92440b8&latest_changeset_revision=1dbb5181c327&latest_ctx_rev=1') paste.recursive.forward: <paste.recursive.Forwarder from /> paste.recursive.include: <paste.recursive.Includer from /> paste.recursive.include_app_iter: <paste.recursive.IncluderAppIter from /> paste.recursive.script_name: '' paste.throw_errors: True request_id: 'fdaf9004a7d511e49d63005056a77528' webob._parsed_query_vars: (MultiDict([('tool_shed_url', ' https://toolshed.g2.bx.psu.edu/'), ('name', 'fasta_formatter'), ('owner', 'devteam'), ('changeset_revision', '8f0ae92440b8'), ('latest_changeset_revision', '1dbb5181c327'), ('latest_ctx_rev', '1')]), 'tool_shed_url=https://toolshed.g2.bx.psu.edu/&name= fasta_formatter&owner=devteam&changeset_revision= 8f0ae92440b8&latest_changeset_revision=1dbb5181c327&latest_ctx_rev=1') wsgi process: 'Multithreaded' ------------------------------------------------------------
Anything obvious that I've missed? Many thanks in advance for your help. Kind regards, Anne. -- Dr Anne Pajon - Bioinformatics Core Cancer Research UK - Cambridge Institute Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE anne.pajon@cruk.cam.ac.uk | +44 (0)1223 769 631
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hi Martin, Many thanks for your answer, and sorry for this late reply I've been off sick. In order to ease my migration, I am looking into reducing the number of tools installed that are not in used. We do run a cut down version of galaxy with much fewer tools that currently in Galaxy Main, but the tool migration scripts have installed lots of tools within my system that are not needed as I see them in 'Installed tool shed repositories' . I wish I could find a way to automatically remove installed tools that are not in use. Any ideas if such a script already exists? Or should I write it myself? Any guidance on where to start would be great. Many thanks in advance for your help. Cheers, Anne. On 30 Jan 2015, at 15:46, Martin Čech <marten@bx.psu.edu> wrote:
Hi Anne,
moving the whole Galaxy installation is a tough job and we do not really have a good process for it. Your problem is most probably caused by the inconsistency in the DB. The path to the shed_tool_conf.xml is stored there and the changes you made to the location of the installation did not propagate there.
Please have a look at the following script that Dave B. made: https://gist.github.com/InitHello/d208cd056baf088aca1f
This script will soon be part of the distribution (in /scripts) and should help you solving your issues.
Thank you for using Galaxy.
Martin, Galaxy Team
On Thu Jan 29 2015 at 11:47:19 AM Anne Pajon <Anne.Pajon@cruk.cam.ac.uk> wrote: Hi there,
I am currently trying to migrate our Galaxy production server onto new hardware and I am facing many challenges. Recently I've been unable to update tools from the tool shed. During the migration I've moved across the files, the data, the tools and their dependencies as well as the database. But I am getting this error while trying to update any tools that are marked 'Updates are available in the Tool Shed for this revision' from the GUI: Admin > Manage installed tool shed repositories > fasta_formatter menu Get updates:
Internal Server Error
Galaxy was unable to successfully complete your request
An error occurred.
This may be an intermittent problem due to load or other unpredictable factors, reloading the page may address the problem. The error has been logged to our team.
Here is the log from paster.log:
10.20.13.16 - - [29/Jan/2015:16:43:45 +0100] "GET /admin_toolshed/update_to_changeset_revision?tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=fasta_formatter&owner=devteam&changeset_revision=8f0ae92440b8&latest_changeset_revision=1dbb5181c327&latest_ctx_rev=1 HTTP/1.1" 500 - "http://bioinf-gal001:8080/admin_toolshed/browse_repositories?async=false&sort=name&page=1&show_item_checkboxes=false&advanced_search=false&f-deleted=False&f-free-text-search=fasta_formatter" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:35.0) Gecko/20100101 Firefox/35.0" Error - <type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'get' URL: http://bioinf-gal001:8080/admin_toolshed/update_to_changeset_revision?tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=fasta_formatter&owner=devteam&changeset_revision=8f0ae92440b8&latest_changeset_revision=1dbb5181c327&latest_ctx_rev=1 File '/mnt/nfs/data/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/mnt/nfs/data/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/mnt/nfs/data/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 190 in handle_request body = method( trans, **kwargs ) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/galaxy/web/framework/decorators.py', line 87 in decorator return func( self, trans, *args, **kwargs ) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py', line 1953 in update_to_changeset_revision persist=True ) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/metadata/installed_repository_metadata_manager.py', line 31 in __init__ metadata_dict=metadata_dict, user=None ) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/tool_shed/metadata/metadata_generator.py', line 60 in __init__ self.metadata_dict = { 'shed_config_filename' : self.shed_config_dict.get( 'config_filename', None ) } AttributeError: 'NoneType' object has no attribute 'get'
CGI Variables ------------- CONTENT_LENGTH: '0' HTTP_ACCEPT: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' HTTP_ACCEPT_ENCODING: 'gzip, deflate' HTTP_ACCEPT_LANGUAGE: 'en-US,en;q=0.5' HTTP_CONNECTION: 'keep-alive' HTTP_COOKIE: 'galaxysession=c6ca0ddb55be603a44ec9cad39b01202307c62c9a25600e232d5d3d597344cb74ba4cd8a7123f6e9' HTTP_HOST: 'bioinf-gal001:8080' HTTP_REFERER: 'http://bioinf-gal001:8080/admin_toolshed/browse_repositories?async=false&sort=name&page=1&show_item_checkboxes=false&advanced_search=false&f-deleted=False&f-free-text-search=fasta_formatter' HTTP_USER_AGENT: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:35.0) Gecko/20100101 Firefox/35.0' PATH_INFO: '/admin_toolshed/update_to_changeset_revision' QUERY_STRING: 'tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=fasta_formatter&owner=devteam&changeset_revision=8f0ae92440b8&latest_changeset_revision=1dbb5181c327&latest_ctx_rev=1' REMOTE_ADDR: '10.20.13.16' REQUEST_METHOD: 'GET' SERVER_NAME: '0.0.0.0' SERVER_PORT: '8080' SERVER_PROTOCOL: 'HTTP/1.1'
WSGI Variables -------------- application: <paste.recursive.RecursiveMiddleware object at 0x7f34f7a50590> is_api_request: False paste.cookies: (<SimpleCookie: galaxysession='c6ca0ddb55be603a44ec9cad39b01202307c62c9a25600e232d5d3d597344cb74ba4cd8a7123f6e9'>, 'galaxysession=c6ca0ddb55be603a44ec9cad39b01202307c62c9a25600e232d5d3d597344cb74ba4cd8a7123f6e9') paste.expected_exceptions: [<class 'paste.httpexceptions.HTTPException'>] paste.httpexceptions: <paste.httpexceptions.HTTPExceptionHandler object at 0x7f34f7a50550> paste.httpserver.thread_pool: <paste.httpserver.ThreadPool object at 0x7f34f76ef1d0> paste.parsed_querystring: ([('tool_shed_url', 'https://toolshed.g2.bx.psu.edu/'), ('name', 'fasta_formatter'), ('owner', 'devteam'), ('changeset_revision', '8f0ae92440b8'), ('latest_changeset_revision', '1dbb5181c327'), ('latest_ctx_rev', '1')], 'tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=fasta_formatter&owner=devteam&changeset_revision=8f0ae92440b8&latest_changeset_revision=1dbb5181c327&latest_ctx_rev=1') paste.recursive.forward: <paste.recursive.Forwarder from /> paste.recursive.include: <paste.recursive.Includer from /> paste.recursive.include_app_iter: <paste.recursive.IncluderAppIter from /> paste.recursive.script_name: '' paste.throw_errors: True request_id: 'fdaf9004a7d511e49d63005056a77528' webob._parsed_query_vars: (MultiDict([('tool_shed_url', 'https://toolshed.g2.bx.psu.edu/'), ('name', 'fasta_formatter'), ('owner', 'devteam'), ('changeset_revision', '8f0ae92440b8'), ('latest_changeset_revision', '1dbb5181c327'), ('latest_ctx_rev', '1')]), 'tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=fasta_formatter&owner=devteam&changeset_revision=8f0ae92440b8&latest_changeset_revision=1dbb5181c327&latest_ctx_rev=1') wsgi process: 'Multithreaded' ------------------------------------------------------------
Anything obvious that I've missed? Many thanks in advance for your help. Kind regards, Anne. -- Dr Anne Pajon - Bioinformatics Core Cancer Research UK - Cambridge Institute Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE anne.pajon@cruk.cam.ac.uk | +44 (0)1223 769 631
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
-- Dr Anne Pajon - Bioinformatics Core Cancer Research UK - Cambridge Institute Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE anne.pajon@cruk.cam.ac.uk | +44 (0)1223 769 631
Hi Anne, as far as I know Galaxy does not have an API endpoint to uninstall repositories. I would use an admin interface for this - the process of uninstalling is pretty fast and unless you have some crazy number of installed tools it should be done quick. M. On Thu Feb 05 2015 at 12:24:03 PM Anne Pajon <Anne.Pajon@cruk.cam.ac.uk> wrote:
Hi Martin,
Many thanks for your answer, and sorry for this late reply I've been off sick.
In order to ease my migration, I am looking into reducing the number of tools installed that are not in used. We do run a cut down version of galaxy with much fewer tools that currently in Galaxy Main, but the tool migration scripts have installed lots of tools within my system that are not needed as I see them in 'Installed tool shed repositories' . I wish I could find a way to automatically remove installed tools that are not in use. Any ideas if such a script already exists? Or should I write it myself? Any guidance on where to start would be great. Many thanks in advance for your help.
Cheers, Anne.
On 30 Jan 2015, at 15:46, Martin Čech <marten@bx.psu.edu> wrote:
Hi Anne,
moving the whole Galaxy installation is a tough job and we do not really have a good process for it. Your problem is most probably caused by the inconsistency in the DB. The path to the shed_tool_conf.xml is stored there and the changes you made to the location of the installation did not propagate there.
Please have a look at the following script that Dave B. made: https://gist.github.com/InitHello/d208cd056baf088aca1f
This script will soon be part of the distribution (in /scripts) and should help you solving your issues.
Thank you for using Galaxy.
Martin, Galaxy Team
On Thu Jan 29 2015 at 11:47:19 AM Anne Pajon <Anne.Pajon@cruk.cam.ac.uk> wrote: Hi there,
I am currently trying to migrate our Galaxy production server onto new hardware and I am facing many challenges. Recently I've been unable to update tools from the tool shed. During the migration I've moved across the files, the data, the tools and their dependencies as well as the database. But I am getting this error while trying to update any tools that are marked 'Updates are available in the Tool Shed for this revision' from the GUI: Admin > Manage installed tool shed repositories > fasta_formatter menu Get updates:
Internal Server Error
Galaxy was unable to successfully complete your request
An error occurred.
This may be an intermittent problem due to load or other unpredictable factors, reloading the page may address the problem. The error has been logged to our team.
Here is the log from paster.log:
10.20.13.16 - - [29/Jan/2015:16:43:45 +0100] "GET /admin_toolshed/update_to_changeset_revision?tool_shed_url= https://toolshed.g2.bx.psu.edu/&name=fasta_formatter& owner=devteam&changeset_revision=8f0ae92440b8&latest_changeset_revision= 1dbb5181c327&latest_ctx_rev=1 HTTP/1.1" 500 - "http://bioinf-gal001:8080/ admin_toolshed/browse_repositories?async=false&sort=name&page=1&show_item_ checkboxes=false&advanced_search=false&f-deleted=False& f-free-text-search=fasta_formatter" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:35.0) Gecko/20100101 Firefox/35.0" Error - <type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'get' URL: http://bioinf-gal001:8080/admin_toolshed/update_to_ changeset_revision?tool_shed_url=https://toolshed.g2.bx. psu.edu/&name=fasta_formatter&owner=devteam&changeset_ revision=8f0ae92440b8&latest_changeset_revision= 1dbb5181c327&latest_ctx_rev=1 File '/mnt/nfs/data/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/mnt/nfs/data/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/mnt/nfs/data/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 190 in handle_request body = method( trans, **kwargs ) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/galaxy/web/framework/decorators.py', line 87 in decorator return func( self, trans, *args, **kwargs ) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/galaxy/webapps/ galaxy/controllers/admin_toolshed.py', line 1953 in update_to_changeset_revision persist=True ) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/tool_shed/galaxy_ install/metadata/installed_repository_metadata_manager.py', line 31 in __init__ metadata_dict=metadata_dict, user=None ) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/tool_shed/metadata/metadata_generator.py', line 60 in __init__ self.metadata_dict = { 'shed_config_filename' : self.shed_config_dict.get( 'config_filename', None ) } AttributeError: 'NoneType' object has no attribute 'get'
CGI Variables ------------- CONTENT_LENGTH: '0' HTTP_ACCEPT: 'text/html,application/xhtml+ xml,application/xml;q=0.9,*/*;q=0.8' HTTP_ACCEPT_ENCODING: 'gzip, deflate' HTTP_ACCEPT_LANGUAGE: 'en-US,en;q=0.5' HTTP_CONNECTION: 'keep-alive' HTTP_COOKIE: 'galaxysession=c6ca0ddb55be603a44ec9cad39b012 02307c62c9a25600e232d5d3d597344cb74ba4cd8a7123f6e9' HTTP_HOST: 'bioinf-gal001:8080' HTTP_REFERER: 'http://bioinf-gal001:8080/admin_toolshed/browse_ repositories?async=false&sort=name&page=1&show_item_ checkboxes=false&advanced_search=false&f-deleted=False& f-free-text-search=fasta_formatter' HTTP_USER_AGENT: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:35.0) Gecko/20100101 Firefox/35.0' PATH_INFO: '/admin_toolshed/update_to_changeset_revision' QUERY_STRING: 'tool_shed_url=https://toolshed.g2.bx.psu.edu/&name= fasta_formatter&owner=devteam&changeset_revision= 8f0ae92440b8&latest_changeset_revision=1dbb5181c327&latest_ctx_rev=1' REMOTE_ADDR: '10.20.13.16' REQUEST_METHOD: 'GET' SERVER_NAME: '0.0.0.0' SERVER_PORT: '8080' SERVER_PROTOCOL: 'HTTP/1.1'
WSGI Variables -------------- application: <paste.recursive.RecursiveMiddleware object at 0x7f34f7a50590> is_api_request: False paste.cookies: (<SimpleCookie: galaxysession=' c6ca0ddb55be603a44ec9cad39b01202307c62c9a25600e232d5d3d597344cb74ba4cd8a7123f6e9'>, 'galaxysession=c6ca0ddb55be603a44ec9cad39b012 02307c62c9a25600e232d5d3d597344cb74ba4cd8a7123f6e9') paste.expected_exceptions: [<class 'paste.httpexceptions. HTTPException'>] paste.httpexceptions: <paste.httpexceptions.HTTPExceptionHandler object at 0x7f34f7a50550> paste.httpserver.thread_pool: <paste.httpserver.ThreadPool object at 0x7f34f76ef1d0> paste.parsed_querystring: ([('tool_shed_url', ' https://toolshed.g2.bx.psu.edu/'), ('name', 'fasta_formatter'), ('owner', 'devteam'), ('changeset_revision', '8f0ae92440b8'), ('latest_changeset_revision', '1dbb5181c327'), ('latest_ctx_rev', '1')], 'tool_shed_url=https://toolshed.g2.bx.psu.edu/&name= fasta_formatter&owner=devteam&changeset_revision= 8f0ae92440b8&latest_changeset_revision=1dbb5181c327&latest_ctx_rev=1') paste.recursive.forward: <paste.recursive.Forwarder from /> paste.recursive.include: <paste.recursive.Includer from /> paste.recursive.include_app_iter: <paste.recursive.IncluderAppIter from /> paste.recursive.script_name: '' paste.throw_errors: True request_id: 'fdaf9004a7d511e49d63005056a77528' webob._parsed_query_vars: (MultiDict([('tool_shed_url', ' https://toolshed.g2.bx.psu.edu/'), ('name', 'fasta_formatter'), ('owner', 'devteam'), ('changeset_revision', '8f0ae92440b8'), ('latest_changeset_revision', '1dbb5181c327'), ('latest_ctx_rev', '1')]), 'tool_shed_url=https://toolshed.g2.bx.psu.edu/&name= fasta_formatter&owner=devteam&changeset_revision= 8f0ae92440b8&latest_changeset_revision=1dbb5181c327&latest_ctx_rev=1') wsgi process: 'Multithreaded' ------------------------------------------------------------
Anything obvious that I've missed? Many thanks in advance for your help. Kind regards, Anne. -- Dr Anne Pajon - Bioinformatics Core Cancer Research UK - Cambridge Institute Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE anne.pajon@cruk.cam.ac.uk | +44 (0)1223 769 631
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
-- Dr Anne Pajon - Bioinformatics Core Cancer Research UK - Cambridge Institute Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE anne.pajon@cruk.cam.ac.uk | +44 (0)1223 769 631
Hi Martin, Many thanks for your email. I've ran this query to find the number of installed tools that are effectively not in use: select distinct (tool.name) as unused_tool from tool_shed_repository as tool left outer join job on job.tool_id = tool.name where job.tool_id is Null In our case we just have one tool from the Installed tool shed repositories that is in use, all the others could be uninstalled to simplify the migration process. So I do have at the end 176 tools to uninstall and I would love to write a script to do so instead of doing it via the admin interface. By the way, these tools were not installed by ourself but they have been added by the automatic migration script which moves tools installed within Galaxy to the tool shed. Is there absolutely no entry point to delete them in Galaxy automatically as there seems to be one to install them? Any guidance would be great. Many thanks in advance for your precious help. Cheers, Anne. On 5 Feb 2015, at 17:34, Martin Čech <marten@bx.psu.edu> wrote:
Hi Anne,
as far as I know Galaxy does not have an API endpoint to uninstall repositories. I would use an admin interface for this - the process of uninstalling is pretty fast and unless you have some crazy number of installed tools it should be done quick.
M.
On Thu Feb 05 2015 at 12:24:03 PM Anne Pajon <Anne.Pajon@cruk.cam.ac.uk> wrote: Hi Martin,
Many thanks for your answer, and sorry for this late reply I've been off sick.
In order to ease my migration, I am looking into reducing the number of tools installed that are not in used. We do run a cut down version of galaxy with much fewer tools that currently in Galaxy Main, but the tool migration scripts have installed lots of tools within my system that are not needed as I see them in 'Installed tool shed repositories' . I wish I could find a way to automatically remove installed tools that are not in use. Any ideas if such a script already exists? Or should I write it myself? Any guidance on where to start would be great. Many thanks in advance for your help.
Cheers, Anne.
On 30 Jan 2015, at 15:46, Martin Čech <marten@bx.psu.edu> wrote:
Hi Anne,
moving the whole Galaxy installation is a tough job and we do not really have a good process for it. Your problem is most probably caused by the inconsistency in the DB. The path to the shed_tool_conf.xml is stored there and the changes you made to the location of the installation did not propagate there.
Please have a look at the following script that Dave B. made: https://gist.github.com/InitHello/d208cd056baf088aca1f
This script will soon be part of the distribution (in /scripts) and should help you solving your issues.
Thank you for using Galaxy.
Martin, Galaxy Team
On Thu Jan 29 2015 at 11:47:19 AM Anne Pajon <Anne.Pajon@cruk.cam.ac.uk> wrote: Hi there,
I am currently trying to migrate our Galaxy production server onto new hardware and I am facing many challenges. Recently I've been unable to update tools from the tool shed. During the migration I've moved across the files, the data, the tools and their dependencies as well as the database. But I am getting this error while trying to update any tools that are marked 'Updates are available in the Tool Shed for this revision' from the GUI: Admin > Manage installed tool shed repositories > fasta_formatter menu Get updates:
Internal Server Error
Galaxy was unable to successfully complete your request
An error occurred.
This may be an intermittent problem due to load or other unpredictable factors, reloading the page may address the problem. The error has been logged to our team.
Here is the log from paster.log:
10.20.13.16 - - [29/Jan/2015:16:43:45 +0100] "GET /admin_toolshed/update_to_changeset_revision?tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=fasta_formatter&owner=devteam&changeset_revision=8f0ae92440b8&latest_changeset_revision=1dbb5181c327&latest_ctx_rev=1 HTTP/1.1" 500 - "http://bioinf-gal001:8080/admin_toolshed/browse_repositories?async=false&sort=name&page=1&show_item_checkboxes=false&advanced_search=false&f-deleted=False&f-free-text-search=fasta_formatter" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:35.0) Gecko/20100101 Firefox/35.0" Error - <type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'get' URL: http://bioinf-gal001:8080/admin_toolshed/update_to_changeset_revision?tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=fasta_formatter&owner=devteam&changeset_revision=8f0ae92440b8&latest_changeset_revision=1dbb5181c327&latest_ctx_rev=1 File '/mnt/nfs/data/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/mnt/nfs/data/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/mnt/nfs/data/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 190 in handle_request body = method( trans, **kwargs ) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/galaxy/web/framework/decorators.py', line 87 in decorator return func( self, trans, *args, **kwargs ) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py', line 1953 in update_to_changeset_revision persist=True ) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/tool_shed/galaxy_install/metadata/installed_repository_metadata_manager.py', line 31 in __init__ metadata_dict=metadata_dict, user=None ) File '/mnt/nfs/data/galaxy/galaxy-dist/lib/tool_shed/metadata/metadata_generator.py', line 60 in __init__ self.metadata_dict = { 'shed_config_filename' : self.shed_config_dict.get( 'config_filename', None ) } AttributeError: 'NoneType' object has no attribute 'get'
CGI Variables ------------- CONTENT_LENGTH: '0' HTTP_ACCEPT: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' HTTP_ACCEPT_ENCODING: 'gzip, deflate' HTTP_ACCEPT_LANGUAGE: 'en-US,en;q=0.5' HTTP_CONNECTION: 'keep-alive' HTTP_COOKIE: 'galaxysession=c6ca0ddb55be603a44ec9cad39b01202307c62c9a25600e232d5d3d597344cb74ba4cd8a7123f6e9' HTTP_HOST: 'bioinf-gal001:8080' HTTP_REFERER: 'http://bioinf-gal001:8080/admin_toolshed/browse_repositories?async=false&sort=name&page=1&show_item_checkboxes=false&advanced_search=false&f-deleted=False&f-free-text-search=fasta_formatter' HTTP_USER_AGENT: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:35.0) Gecko/20100101 Firefox/35.0' PATH_INFO: '/admin_toolshed/update_to_changeset_revision' QUERY_STRING: 'tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=fasta_formatter&owner=devteam&changeset_revision=8f0ae92440b8&latest_changeset_revision=1dbb5181c327&latest_ctx_rev=1' REMOTE_ADDR: '10.20.13.16' REQUEST_METHOD: 'GET' SERVER_NAME: '0.0.0.0' SERVER_PORT: '8080' SERVER_PROTOCOL: 'HTTP/1.1'
WSGI Variables -------------- application: <paste.recursive.RecursiveMiddleware object at 0x7f34f7a50590> is_api_request: False paste.cookies: (<SimpleCookie: galaxysession='c6ca0ddb55be603a44ec9cad39b01202307c62c9a25600e232d5d3d597344cb74ba4cd8a7123f6e9'>, 'galaxysession=c6ca0ddb55be603a44ec9cad39b01202307c62c9a25600e232d5d3d597344cb74ba4cd8a7123f6e9') paste.expected_exceptions: [<class 'paste.httpexceptions.HTTPException'>] paste.httpexceptions: <paste.httpexceptions.HTTPExceptionHandler object at 0x7f34f7a50550> paste.httpserver.thread_pool: <paste.httpserver.ThreadPool object at 0x7f34f76ef1d0> paste.parsed_querystring: ([('tool_shed_url', 'https://toolshed.g2.bx.psu.edu/'), ('name', 'fasta_formatter'), ('owner', 'devteam'), ('changeset_revision', '8f0ae92440b8'), ('latest_changeset_revision', '1dbb5181c327'), ('latest_ctx_rev', '1')], 'tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=fasta_formatter&owner=devteam&changeset_revision=8f0ae92440b8&latest_changeset_revision=1dbb5181c327&latest_ctx_rev=1') paste.recursive.forward: <paste.recursive.Forwarder from /> paste.recursive.include: <paste.recursive.Includer from /> paste.recursive.include_app_iter: <paste.recursive.IncluderAppIter from /> paste.recursive.script_name: '' paste.throw_errors: True request_id: 'fdaf9004a7d511e49d63005056a77528' webob._parsed_query_vars: (MultiDict([('tool_shed_url', 'https://toolshed.g2.bx.psu.edu/'), ('name', 'fasta_formatter'), ('owner', 'devteam'), ('changeset_revision', '8f0ae92440b8'), ('latest_changeset_revision', '1dbb5181c327'), ('latest_ctx_rev', '1')]), 'tool_shed_url=https://toolshed.g2.bx.psu.edu/&name=fasta_formatter&owner=devteam&changeset_revision=8f0ae92440b8&latest_changeset_revision=1dbb5181c327&latest_ctx_rev=1') wsgi process: 'Multithreaded' ------------------------------------------------------------
Anything obvious that I've missed? Many thanks in advance for your help. Kind regards, Anne. -- Dr Anne Pajon - Bioinformatics Core Cancer Research UK - Cambridge Institute Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE anne.pajon@cruk.cam.ac.uk | +44 (0)1223 769 631
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
-- Dr Anne Pajon - Bioinformatics Core Cancer Research UK - Cambridge Institute Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE anne.pajon@cruk.cam.ac.uk | +44 (0)1223 769 631
-- Dr Anne Pajon - Bioinformatics Core Cancer Research UK - Cambridge Institute Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE anne.pajon@cruk.cam.ac.uk | +44 (0)1223 769 631
participants (2)
-
Anne Pajon -
Martin Čech