Thanks for information Greg.  Actually, it turned out to be two things causing my issues:

1) My copy of tool-data/sam_fa_indices.loc.sample was not owned by the galaxy user (though it was writable by that user). This caused the exception which caused the installation to get "stuck" at "cloning":

File '/data/local/galaxy/galaxy-prod/lib/galaxy/web/controllers/admin_toolshed.py', line 709 in handle_repository_contents
  copy_sample_files( trans.app, sample_files, sample_files_copied=sample_files_copied )
File '/data/local/galaxy/galaxy-prod/lib/galaxy/util/shed_util.py', line 311 in copy_sample_files
  copy_sample_file( app, filename, dest_path=dest_path )
File '/data/local/galaxy/galaxy-prod/lib/galaxy/util/shed_util.py', line 299 in copy_sample_file
  shutil.copy( full_source_path, full_destination_path )
File '/usr/local/python/lib/python2.6/shutil.py', line 85 in copy
  copymode(src, dst)
File '/usr/local/python/lib/python2.6/shutil.py', line 59 in copymode
  os.chmod(dst, mode)
OSError: [Errno 1] Operation not permitted: '/data/local/galaxy/galaxy-prod/tool-data/sam_fa_indices.loc.sample'

2) Once that happened, the directory to clone the repository into still existed (and wasn't empty), which caused the error below, also causing the process to get stuck in "cloning" status.

File '/data/local/galaxy/galaxy-prod/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py', line 143 in __call__
  app_iter = self.application(environ, start_response)
File '/data/local/galaxy/galaxy-prod/eggs/Paste-1.6-py2.6.egg/paste/recursive.py', line 80 in __call__
  return self.application(environ, start_response)
File '/data/local/galaxy/galaxy-prod/lib/galaxy/web/framework/middleware/remoteuser.py', line 91 in __call__
  return self.app( environ, start_response )
File '/data/local/galaxy/galaxy-prod/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py', line 632 in __call__
  return self.application(environ, start_response)
File '/data/local/galaxy/galaxy-prod/lib/galaxy/web/framework/base.py', line 160 in __call__
  body = method( trans, **kwargs )
File '/data/local/galaxy/galaxy-prod/lib/galaxy/web/framework/__init__.py', line 184 in decorator
  return func( self, trans, *args, **kwargs )
File '/data/local/galaxy/galaxy-prod/lib/galaxy/web/controllers/admin_toolshed.py', line 846 in manage_repositories
  self.install_tool_shed_repositories( trans, repositories_for_installation, reinstalling=reinstalling, **decoded_kwd )
File '/data/local/galaxy/galaxy-prod/lib/galaxy/web/framework/__init__.py', line 184 in decorator
  return func( self, trans, *args, **kwargs )
File '/data/local/galaxy/galaxy-prod/lib/galaxy/web/controllers/admin_toolshed.py', line 618 in install_tool_shed_repositories
  clone_repository( repository_clone_url, os.path.abspath( relative_install_dir ), ctx_rev )
File '/data/local/galaxy/galaxy-prod/lib/galaxy/util/shed_util.py', line 287 in clone_repository
  rev=util.listify( str( ctx_rev ) ) )
File '/data/local/galaxy/galaxy-prod/eggs/mercurial-2.1.2-py2.6-linux-x86_64-ucs2.egg/mercurial/commands.py', line 1157 in clone
  branch=opts.get('branch'))
File '/data/local/galaxy/galaxy-prod/eggs/mercurial-2.1.2-py2.6-linux-x86_64-ucs2.egg/mercurial/hg.py', line 270 in clone
  raise util.Abort(_("destination '%s' is not empty") % dest)
Abort: destination '/data/local/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/freebayes/d3bf1e86b243/freebayes' is not empty

Once I manually cleaned those issues up things progressed normally.  Hope this helps to make the process more robust.

Lance

Greg Von Kuster wrote:
Hi Lance,

You'll need to execute the following sql manually in your Galaxy database.

update tool_shed_repository set status = 'New' where status = 'Cloning';

Next, you'll have to update your production Galaxy instance code base to at least changeset 2531e085f262 (the human-readable changeset revision is 7503).  This revision is required to install the freebayes tool from the main Galaxy tool shed.  You can also choose to wait until the next Galaxy release to upgrade if you do not need the freebayes tool in your local instance immediately.

Sorry for the inconvenience on this.

Greg Von Kuster


On Aug 31, 2012, at 3:59 PM, Lance Parsons wrote:

Thanks for the explanation.  That makes sense. I went ahead and installed the new version along with dependencies in my development environments and all went well.  Then in production, the installation seems to have failed and I'm now stuck with freebayes in the "cloning" status.  I tried to deactivate and uninstall it so I could try again to monitor what went wrong and I just get an error:

Error - <type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'startswith'
URL: http://galaxy.princeton.edu/admin_toolshed/browse_repositories?operation=deactivate+or+uninstall&id=ac6e1b48798577db
File '/data/local/galaxy/galaxy-prod/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py', line 143 in __call__
  app_iter = self.application(environ, start_response)
File '/data/local/galaxy/galaxy-prod/eggs/Paste-1.6-py2.6.egg/paste/recursive.py', line 80 in __call__
  return self.application(environ, start_response)
File '/data/local/galaxy/galaxy-prod/lib/galaxy/web/framework/middleware/remoteuser.py', line 91 in __call__
  return self.app( environ, start_response )
File '/data/local/galaxy/galaxy-prod/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py', line 632 in __call__
  return self.application(environ, start_response)
File '/data/local/galaxy/galaxy-prod/lib/galaxy/web/framework/base.py', line 160 in __call__
  body = method( trans, **kwargs )
File '/data/local/galaxy/galaxy-prod/lib/galaxy/web/framework/__init__.py', line 184 in decorator
  return func( self, trans, *args, **kwargs )
File '/data/local/galaxy/galaxy-prod/lib/galaxy/web/controllers/admin_toolshed.py', line 356 in browse_repositories
  return self.deactivate_or_uninstall_repository( trans, **kwd )
File '/data/local/galaxy/galaxy-prod/lib/galaxy/web/framework/__init__.py', line 184 in decorator
  return func( self, trans, *args, **kwargs )
File '/data/local/galaxy/galaxy-prod/lib/galaxy/web/controllers/admin_toolshed.py', line 408 in deactivate_or_uninstall_repository
  repository_install_dir = os.path.abspath ( relative_install_dir )
File '/usr/local/python/lib/python2.6/posixpath.py', line 342 in abspath
  if not isabs(path):
File '/usr/local/python/lib/python2.6/posixpath.py', line 52 in isabs
  return s.startswith('/')
AttributeError: 'NoneType' object has no attribute 'startswith'

Any thoughts on how I can get rid of the partially installed freebayes tool?  Or how to troubleshoot the failing installation?

Lance

Daniel Blankenberg wrote:
Hi Lance,

This has to do with the way that the updating currently works. Since the migrated version is 0.0.2, the automatic update will only update that particular Tool version (newest version of 0.0.2). To get 0.0.3, you need to install it as a new tool. This way you are able to have both versions installed and available e.g. for having a reproducible rerun functionality. In the latest Galaxy-central / upcoming Galaxy-dist release, only the newest version is shown in the lefthand tool panel. 

Future enhancements will include allowing the selection of different tool versions from within the tool interface (e.g. allowing the rerun of an older version to use a newer version) and to show and allow updating (installing) new tool versions from the same interface (e.g. 2 to 3).


Thanks for using Galaxy,

Dan


On Aug 31, 2012, at 11:47 AM, Lance Parsons wrote:

Thanks for the quick turnaround Dan.  Much appreciated.  When I tried to test updating Freebayes from the toolshed, I get a message that no updates are available. Perhaps it takes some time before they are recognized?  Or am I missing something?

Lance


Daniel Blankenberg wrote:
Hi Lance,

Thank you for the pull request. I think this is an excellent way to handle patches until the toolshed itself provides a better mechanism, particularly for the devteam owned repositories; other repository owners may prefer different methods of providing patches, but they can be contacted directly by using the contact form on the toolshed. Future enhancements to the toolshed will integrate better methods of providing this functionality, but that likely some ways off. Any suggestions and community contributions on this would be appreciated. 


FreeBayes has been updated in the Toolshed to 0.9.6_9608597d12e127c847ae03aa03440ab63992fedf, with mostly minor differences from your pull request. Please let us know if you experience any issues installing or using the updated version (0.0.3).


Thanks,

Dan


On Aug 30, 2012, at 11:35 AM, Lance Parsons wrote:

I have been using the FreeBayes tool in our local Galaxy instance for some time.  As FreeBayes is a rapidly evolving tool, it has been very helpful to have Daniel Blankenberg work on keeping an updated wrapper.

Recently, I've run into a rather troublesome bug in FreeBayes (https://github.com/ekg/freebayes/issues/22) that is causing it to crash on nearly every real dataset.  Luckily, the bug has been fixed in the latest version of FreeBayes (0.9.6_9608597d12e127c847ae03aa03440ab63992fedf).

Unfortunately, the latest version has a few parameter changes that require an update to the wrapper code. I've taken the liberty to make some changes to the wrapper and they seem to work. However, now that I have the tool installed from the Tool Shed, I'm not sure of the best way to 1) incorporate this update into my local galaxy instance and 2) contribute this change back to the Tool Shed.

The best solution is to get the Tool Shed updated and then pull that update into my production Galaxy instance, however, I'm wondering if there is a way I could push "hot-fixes" to my production instance without causing me headaches down the line when the Tool Shed is inevitably updated.

As far as coordinating updates of Tool Shed tools, I suppose the process might be different for different tool authors. I was wondering, however, if anyone had thoughts on how this might work best. I've created a clone of the galaxy tool shed repository in Bitbucket (https://bitbucket.org/lance_parsons/freebayes/) and given Daniel and Greg access to it in hopes that would make things easier. Let me know if there is a better way to coordinate, etc.

Thanks again.


--
Lance Parsons - Scientific Programmer
134 Carl C. Icahn Laboratory
Lewis-Sigler Institute for Integrative Genomics
Princeton University

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/


--
Lance Parsons - Scientific Programmer
134 Carl C. Icahn Laboratory
Lewis-Sigler Institute for Integrative Genomics
Princeton University



--
Lance Parsons - Scientific Programmer
134 Carl C. Icahn Laboratory
Lewis-Sigler Institute for Integrative Genomics
Princeton University

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/


--
Lance Parsons - Scientific Programmer
134 Carl C. Icahn Laboratory
Lewis-Sigler Institute for Integrative Genomics
Princeton University