Looks like I forgot to reset that value. So making progress... I can install freebayes successfully (the front end timeout but it worked)

However, samtools fails with the following message in web worker log:

[localhost] local: sed -i .bak -e 's/-lcurses/-lncurses/g' Makefile
Warning: local() encountered an error (return code 2) while executing 'sed -i .bak -e 's/-lcurses/-lncurses/g' Makefile'

Front end error: "sed: can't read .bak: No such file or directory"

So for some reason the sed did not work...


Philip Mabon
Senior Bioinformatician
National Microbiology Laboratory
Public Health Agency of Canada


On Wed, Aug 29, 2012 at 8:48 AM, Greg Von Kuster <greg@bx.psu.edu> wrote:
Hi Philip,

What do you have for the following configuration setting in the [app:main] section of your universe_wsgi.ini file?  The config setting for tool_dependency_dir should be a valid subdirectory that is writable by the user that starts your Galaxy server.

# Location for installing tool dependencies
tool_dependency_dir = ../tool_dependencies

Greg Von Kuster


On Aug 29, 2012, at 9:34 AM, Philip Mabon wrote:

I upgrade Galaxy to changeset 2531e085f262 and attempt to install freebayes from the admin panel.

It installed the freebaye xml tool and appears in the tool panel but both dependencies did not install (freebayes itself and samtools)
When I attempt to install either of them from the  "Tool shed repository 'freebayes' tool dependencies" page , both produces the following error in the web log:


Error - <type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'endswith'
File '/opt/galaxy/galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py', line 143 in __call__
  app_iter = self.application(environ, start_response)
File '/opt/galaxy/galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/recursive.py', line 80 in __call__
  return self.application(environ, start_response)
File '/opt/galaxy/galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py', line 632 in __call__
  return self.application(environ, start_response)
File '/opt/galaxy/galaxy_dist/lib/galaxy/web/framework/base.py', line 160 in __call__
  body = method( trans, **kwargs )
File '/opt/galaxy/galaxy_dist/lib/galaxy/web/framework/__init__.py', line 184 in decorator
  return func( self, trans, *args, **kwargs )
File '/opt/galaxy/galaxy_dist/lib/galaxy/web/controllers/admin_toolshed.py', line 902 in manage_tool_dependencies
  self.initiate_tool_dependency_installation( trans, tool_dependencies_for_installation )
File '/opt/galaxy/galaxy_dist/lib/galaxy/web/framework/__init__.py', line 184 in decorator
  return func( self, trans, *args, **kwargs )
File '/opt/galaxy/galaxy_dist/lib/galaxy/web/controllers/admin_toolshed.py', line 533 in initiate_tool_dependency_installation
  tool_dependencies=tool_dependencies )
File '/opt/galaxy/galaxy_dist/lib/galaxy/util/shed_util.py', line 1352 in handle_tool_dependencies
  tool_dependency = install_package( app, elem, tool_shed_repository, tool_dependencies=tool_dependencies )
File '/opt/galaxy/galaxy_dist/lib/galaxy/tool_shed/tool_dependencies/install_util.py', line 52 in install_package
  install_dir = get_tool_dependency_install_dir( app, tool_shed_repository, package_name, package_version )
File '/opt/galaxy/galaxy_dist/lib/galaxy/tool_shed/tool_dependencies/install_util.py', line 42 in get_tool_dependency_install_dir
  repository.installed_changeset_revision ) )
File '/usr/lib64/python2.6/posixpath.py', line 67 in join
  elif path == '' or path.endswith('/'):
AttributeError: 'NoneType' object has no attribute 'endswith'


CGI Variables
-------------
  CONTENT_LENGTH: '0'
  HTTP_ACCEPT: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'
  HTTP_ACCEPT_CHARSET: 'ISO-8859-1,utf-8;q=0.7,*;q=0.3'
  HTTP_ACCEPT_ENCODING: 'gzip,deflate,sdch'
  HTTP_ACCEPT_LANGUAGE: 'en-GB,en-US;q=0.8,en;q=0.6'
  HTTP_CONNECTION: 'Keep-Alive'
  HTTP_COOKIE: 'galaxysession=c6ca0ddb55be603a1f81117bcdba38660416eeac1cee9b82eec853e080ab1083adff6bfb3578df08'
  HTTP_HOST: 'galaxy.badd.ca'
  HTTP_USER_AGENT: 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.81 Safari/537.1'
  HTTP_X_FORWARDED_FOR: '10.66.8.100'
  HTTP_X_FORWARDED_HOST: 'galaxy.badd.ca'
  HTTP_X_FORWARDED_SERVER: 'galaxy.badd.ca'
  ORGINAL_HTTP_HOST: 'localhost:8081'
  ORGINAL_REMOTE_ADDR: '127.0.0.1'
  PATH_INFO: '/admin_toolshed/manage_tool_dependencies'
  QUERY_STRING: 'sort=-create_time&show_item_checkboxes=false&async=false&page=1&webapp=galaxy&operation=install&id=33b43b4e7093c91f&'
  REMOTE_ADDR: '10.66.8.100'
  REQUEST_METHOD: 'GET'
  SERVER_NAME: '127.0.0.1'
  SERVER_PORT: '8081'
  SERVER_PROTOCOL: 'HTTP/1.1'


Configuration
-------------
  __file__: '/opt/galaxy/galaxy_dist/universe_wsgi.ini'
  allow_library_path_paste: 'True'
  allow_user_creation: 'True'
  allow_user_dataset_purge: 'True'
  allow_user_impersonation: 'True'
  api_allow_run_as: 'philip.mabon@phac-aspc.gc.ca'
  database_connection: 'mysql://galaxy:fahN9Le8@snowwhite.badd.ca/galaxy'
  database_engine_option_pool_recycle: '7200'
  database_engine_option_strategy: 'threadlocal'
  debug: 'False'
  default_cluster_job_runner: 'drmaa://-u forensics-galaxy'
  enable_pages: 'True'
  enable_quotas: 'True'
  enable_tracks: 'True'
  environment_setup_file: '/home/forensics-galaxy/env.sh'
  error_email_to: 'philip.mabon@phac-aspc.gc.ca'
  file_path: '/opt/galaxy/data/database/files'
  here: '/opt/galaxy/galaxy_dist'
  job_handlers: 'handler0,handler1,handler2,handler3'
  job_manager: 'manager'
  len_file_path: 'tool-data/shared/ucsc/chrom'
  library_import_dir: 'None'
  new_file_path: '/opt/galaxy/data/database/tmp'
  new_user_dataset_access_role_default_private: 'True'
  nglims_config_file: 'tool-data/nglims.yaml'
  require_login: 'False'
  retry_metadata_internally: 'False'
  set_metadata_externally: 'True'
  start_job_runners: 'drmaa'
  static_cache_time: '360'
  static_dir: '/opt/galaxy/galaxy_dist/static/'
  static_enabled: 'True'
  static_favicon_dir: '/opt/galaxy/galaxy_dist/static/favicon.ico'
  static_images_dir: '/opt/galaxy/galaxy_dist/static/images'
  static_robots_txt: '/opt/galaxy/galaxy_dist/static/robots.txt'
  static_scripts_dir: '/opt/galaxy/galaxy_dist/static/scripts/'
  static_style_dir: '/opt/galaxy/galaxy_dist/static/june_2007_style/blue'
  tool_config_file: 'tool_conf.xml,shed_tool_conf.xml'
  tool_data_path: 'tool-data'
  tool_path: 'tools'
  use_interactive: 'false'
  use_nglims: 'False'


WSGI Variables
--------------
  application: <paste.recursive.RecursiveMiddleware object at 0x80e7910>
  paste.cookies: (<SimpleCookie: galaxysession='c6ca0ddb55be603a1f81117bcdba38660416eeac1cee9b82eec853e080ab1083adff6bfb3578df08'>, 'galaxysession=c6ca0ddb55be603a1f81117bcdba38660416eeac1cee9b82eec853e080ab1083adff6bfb3578df08')
  paste.expected_exceptions: [<class 'paste.httpexceptions.HTTPException'>]
  paste.httpexceptions: <paste.httpexceptions.HTTPExceptionHandler object at 0x80c9c90>
  paste.httpserver.thread_pool: <paste.httpserver.ThreadPool object at 0x1ac37d0>
  paste.parsed_querystring: ([('sort', '-create_time'), ('show_item_checkboxes', 'false'), ('async', 'false'), ('page', '1'), ('webapp', 'galaxy'), ('operation', 'install'), ('id', '33b43b4e7093c91f')], 'sort=-create_time&show_item_checkboxes=false&async=false&page=1&webapp=galaxy&operation=install&id=33b43b4e7093c91f&')
  paste.recursive.forward: <paste.recursive.Forwarder from />
  paste.recursive.include: <paste.recursive.Includer from />
  paste.recursive.include_app_iter: <paste.recursive.IncluderAppIter from />
  paste.recursive.script_name: ''
  paste.throw_errors: True
  webob._parsed_query_vars: (MultiDict([('sort', '-create_time'), ('show_item_checkboxes', 'false'), ('async', 'false'), ('page', '1'), ('webapp', 'galaxy'), ('operation', 'install'), ('id', '33b43b4e7093c91f')]), 'sort=-create_time&show_item_checkboxes=false&async=false&page=1&webapp=galaxy&operation=install&id=33b43b4e7093c91f&')
  wsgi process: 'Multithreaded'
------------------------------------------------------------


Philip Mabon
Senior Bioinformatician
National Microbiology Laboratory
Public Health Agency of Canada


On Tue, Aug 28, 2012 at 8:11 AM, Greg Von Kuster <greg@bx.psu.edu> wrote:
Philip,

Is it possible for you to upgrade your Galaxy instance code base to at least changeset 2531e085f262 (the human-readable changeset revision is 7503)?  I'm not sure what is causing the behavior in your environment, but I had to upgrade the tool shed to this rev in order to deal with some issues related to setting metadata on repositories in the tool shed, so this may have resulted in problems installing the somewhat special-case freebayes repository into local Galaxy instances running the older dist release.  You can also choose to wait until the next Galaxy release to upgrade if you do not need the freebayes tool in your local instance immediately.

Sorry for the inconvenience I'm sure this is causing.

Greg Von Kuster


On Aug 27, 2012, at 4:18 PM, Philip Mabon wrote:

I already had that entry to the tool_data_table_conf.xml but still getting the same error.

After some debugging and print statement, I found that it initially finds the correct path to the 'sam_fa_indexes' when reading tool_data_table_conf.xml in Galaxy installation directory but re-reads another file in the shed_tools dir.
It's the sample tool_data_table_conf.xml from the freebayes directory.

Maybe the  tool_data_table_conf.xml.sample is invalid?

Philip Mabon
Senior Bioinformatician
National Microbiology Laboratory
Public Health Agency of Canada


On Mon, Aug 27, 2012 at 9:41 AM, Greg Von Kuster <greg@bx.psu.edu> wrote:
Hi Phillip,

I'm not able to reproduce this behavior, so it's difficult to determine what may be the cause.  I have not seen others in the community encounter this problem, but that may simply be due to the fact that no one is yet installing the freebayes tool from the tool shed.

A possible work-aound is to add the following entry into your local tool_data_table_conf.xml file located in your Galaxy installation directory.

    <table name="sam_fa_indexes" comment_char="#">
        <columns>line_type, value, path</columns>
        <file path="tool-data/sam_fa_indices.loc" />
    </table>

Let us know if this still does not correct the problem.

Greg Von Kuster

On Aug 27, 2012, at 10:07 AM, Philip Mabon wrote:

Any ideas anyone?

Philip Mabon
Senior Bioinformatician
National Microbiology Laboratory
Public Health Agency of Canada


On Tue, Aug 21, 2012 at 10:34 AM, Philip Mabon <philipmabon@gmail.com> wrote:
I just upgrade our galaxy to the latest release :  7487:be81990d148a and ran the migration tool script for freebayes. 

I received the following error:

No handlers could be found for logger "galaxy.tools"
/opt/galaxy/galaxy_dist/eggs/pysam-0.4.2_kanwei_b10f6e722e9a-py2.6-linux-x86_64-ucs4.egg/pysam/__init__.py:1: RuntimeWarning: __builtin__.file size changed, may indicate binary incompatibility
  from csamtools import *
Repositories will be installed into configured tool_path location  ../shed_tools
Adding new row (or updating an existing row) for repository 'freebayes' in the tool_shed_repository table.
Traceback (most recent call last):
  File "./scripts/migrate_tools/migrate_tools.py", line 21, in <module>
    app = MigrateToolsApplication( sys.argv[ 1 ] )
  File "/opt/galaxy/galaxy_dist/lib/galaxy/tool_shed/migrate/common.py", line 150, in __init__
    install_dependencies=install_dependencies )
  File "/opt/galaxy/galaxy_dist/lib/galaxy/tool_shed/install_manager.py", line 37, in __init__
    self.install_repository( repository_elem, install_dependencies )
  File "/opt/galaxy/galaxy_dist/lib/galaxy/tool_shed/install_manager.py", line 262, in install_repository
    install_dependencies=install_dependencies )
  File "/opt/galaxy/galaxy_dist/lib/galaxy/tool_shed/install_manager.py", line 150, in handle_repository_contents
    repository_tools_tups = get_repository_tools_tups( self.app, metadata_dict )
  File "/opt/galaxy/galaxy_dist/lib/galaxy/util/shed_util.py", line 1077, in get_repository_tools_tups
    tool = app.toolbox.load_tool( os.path.abspath( relative_path ), guid=guid )
  File "/opt/galaxy/galaxy_dist/lib/galaxy/tools/__init__.py", line 452, in load_tool
    return ToolClass( config_file, root, self.app, guid=guid )
  File "/opt/galaxy/galaxy_dist/lib/galaxy/tools/__init__.py", line 790, in __init__
    self.parse( root, guid=guid )
  File "/opt/galaxy/galaxy_dist/lib/galaxy/tools/__init__.py", line 961, in parse
    self.parse_inputs( root )
  File "/opt/galaxy/galaxy_dist/lib/galaxy/tools/__init__.py", line 1045, in parse_inputs
    display, inputs = self.parse_input_page( page, enctypes )
  File "/opt/galaxy/galaxy_dist/lib/galaxy/tools/__init__.py", line 1447, in parse_input_page
    inputs = self.parse_input_elem( input_elem, enctypes )
  File "/opt/galaxy/galaxy_dist/lib/galaxy/tools/__init__.py", line 1514, in parse_input_elem
    case.inputs = self.parse_input_elem( case_elem, enctypes, context )
  File "/opt/galaxy/galaxy_dist/lib/galaxy/tools/__init__.py", line 1470, in parse_input_elem
    group.inputs = self.parse_input_elem( elem, enctypes, context )
  File "/opt/galaxy/galaxy_dist/lib/galaxy/tools/__init__.py", line 1540, in parse_input_elem
    param = self.parse_param_elem( elem, enctypes, context )
  File "/opt/galaxy/galaxy_dist/lib/galaxy/tools/__init__.py", line 1552, in parse_param_elem
    param = ToolParameter.build( self, input_elem )
  File "/opt/galaxy/galaxy_dist/lib/galaxy/tools/parameters/basic.py", line 176, in build
    return parameter_types[param_type]( tool, param )
  File "/opt/galaxy/galaxy_dist/lib/galaxy/tools/parameters/basic.py", line 1361, in __init__
    ToolParameter.__init__( self, tool, elem )
  File "/opt/galaxy/galaxy_dist/lib/galaxy/tools/parameters/basic.py", line 43, in __init__
    self.validators.append( validation.Validator.from_element( self, elem ) )
  File "/opt/galaxy/galaxy_dist/lib/galaxy/tools/parameters/validation.py", line 23, in from_element
    return validator_types[type].from_element( param, elem )
  File "/opt/galaxy/galaxy_dist/lib/galaxy/tools/parameters/validation.py", line 279, in from_element
    tool_data_table = param.tool.app.tool_data_tables[ table_name ]
  File "/opt/galaxy/galaxy_dist/lib/galaxy/tools/data/__init__.py", line 25, in __getitem__
    return self.data_tables.__getitem__( key )
KeyError: 'sam_fa_indexes'


What happen is that the freebayes itself installs correctly into the shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/* but the dependency on samtools 1.1.8 fails.

The value 'sam_fa_indexes' is present in the tool_data_table_conf.xml  and I know it works since all my samtools are functional.

The status column for the freebayes entry in the database is 'Cloning'. 

I get the same error if I try to install freebayes thru the web front end. ( Had to delete the row in the db and remove the freebayes install directory)

Any ideas Greg?

Thanks!

Philip Mabon (Takadonet)
Senior Bioinformatician
National Microbiology Laboratory
Public Health Agency of Canada

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/