Thanks for information Greg. Actually, it turned out to be two things
causing my issues:
1) My copy of tool-data/sam_fa_indices.loc.sample was not owned by the
galaxy user (though it was writable by that user). This caused the
exception which caused the installation to get "stuck" at "cloning":
File
'/data/local/galaxy/galaxy-prod/lib/galaxy/web/controllers/admin_toolshed.py',
line 709 in handle_repository_contents
copy_sample_files( trans.app, sample_files,
sample_files_copied=sample_files_copied )
File '/data/local/galaxy/galaxy-prod/lib/galaxy/util/shed_util.py', line
311 in copy_sample_files
copy_sample_file( app, filename, dest_path=dest_path )
File '/data/local/galaxy/galaxy-prod/lib/galaxy/util/shed_util.py', line
299 in copy_sample_file
shutil.copy( full_source_path, full_destination_path )
File '/usr/local/python/lib/python2.6/shutil.py', line 85 in copy
copymode(src, dst)
File '/usr/local/python/lib/python2.6/shutil.py', line 59 in copymode
os.chmod(dst, mode)
OSError: [Errno 1] Operation not permitted:
'/data/local/galaxy/galaxy-prod/tool-data/sam_fa_indices.loc.sample'
2) Once that happened, the directory to clone the repository into still
existed (and wasn't empty), which caused the error below, also causing
the process to get stuck in "cloning" status.
File
'/data/local/galaxy/galaxy-prod/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py',
line 143 in __call__
app_iter = self.application(environ, start_response)
File
'/data/local/galaxy/galaxy-prod/eggs/Paste-1.6-py2.6.egg/paste/recursive.py',
line 80 in __call__
return self.application(environ, start_response)
File
'/data/local/galaxy/galaxy-prod/lib/galaxy/web/framework/middleware/remoteuser.py',
line 91 in __call__
return self.app( environ, start_response )
File
'/data/local/galaxy/galaxy-prod/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py',
line 632 in __call__
return self.application(environ, start_response)
File '/data/local/galaxy/galaxy-prod/lib/galaxy/web/framework/base.py',
line 160 in __call__
body = method( trans, **kwargs )
File
'/data/local/galaxy/galaxy-prod/lib/galaxy/web/framework/__init__.py',
line 184 in decorator
return func( self, trans, *args, **kwargs )
File
'/data/local/galaxy/galaxy-prod/lib/galaxy/web/controllers/admin_toolshed.py',
line 846 in manage_repositories
self.install_tool_shed_repositories( trans,
repositories_for_installation, reinstalling=reinstalling, **decoded_kwd )
File
'/data/local/galaxy/galaxy-prod/lib/galaxy/web/framework/__init__.py',
line 184 in decorator
return func( self, trans, *args, **kwargs )
File
'/data/local/galaxy/galaxy-prod/lib/galaxy/web/controllers/admin_toolshed.py',
line 618 in install_tool_shed_repositories
clone_repository( repository_clone_url, os.path.abspath(
relative_install_dir ), ctx_rev )
File '/data/local/galaxy/galaxy-prod/lib/galaxy/util/shed_util.py', line
287 in clone_repository
rev=util.listify( str( ctx_rev ) ) )
File
'/data/local/galaxy/galaxy-prod/eggs/mercurial-2.1.2-py2.6-linux-x86_64-ucs2.egg/mercurial/commands.py',
line 1157 in clone
branch=opts.get('branch'))
File
'/data/local/galaxy/galaxy-prod/eggs/mercurial-2.1.2-py2.6-linux-x86_64-ucs2.egg/mercurial/hg.py',
line 270 in clone
raise util.Abort(_("destination '%s' is not empty") % dest)
Abort: destination
'/data/local/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/freebayes/d3bf1e86b243/freebayes'
is not empty
Once I manually cleaned those issues up things progressed normally.
Hope this helps to make the process more robust.
Lance
Greg Von Kuster wrote:
Hi Lance,
You'll need to execute the following sql manually in your Galaxy database.
update tool_shed_repository set status = 'New' where status = 'Cloning';
Next, you'll have to update your production Galaxy instance code base
to at least changeset 2531e085f262 (the human-readable changeset
revision is 7503). This revision is required to install the freebayes
tool from the main Galaxy tool shed. You can also choose to wait
until the next Galaxy release to upgrade if you do not need the
freebayes tool in your local instance immediately.
Sorry for the inconvenience on this.
Greg Von Kuster
On Aug 31, 2012, at 3:59 PM, Lance Parsons wrote:
> Thanks for the explanation. That makes sense. I went ahead and
> installed the new version along with dependencies in my development
> environments and all went well. Then in production, the installation
> seems to have failed and I'm now stuck with freebayes in the
> "cloning" status. I tried to deactivate and uninstall it so I could
> try again to monitor what went wrong and I just get an error:
>
> Error - <type 'exceptions.AttributeError'>: 'NoneType' object
has no
> attribute 'startswith'
> URL:
>
http://galaxy.princeton.edu/admin_toolshed/browse_repositories?operation=...
> File
>
'/data/local/galaxy/galaxy-prod/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py',
> line 143 in __call__
> app_iter = self.application(environ, start_response)
> File
> '/data/local/galaxy/galaxy-prod/eggs/Paste-1.6-py2.6.egg/paste/recursive.py',
> line 80 in __call__
> return self.application(environ, start_response)
> File
>
'/data/local/galaxy/galaxy-prod/lib/galaxy/web/framework/middleware/remoteuser.py',
> line 91 in __call__
> return self.app( environ, start_response )
> File
>
'/data/local/galaxy/galaxy-prod/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py',
> line 632 in __call__
> return self.application(environ, start_response)
> File
> '/data/local/galaxy/galaxy-prod/lib/galaxy/web/framework/base.py',
> line 160 in __call__
> body = method( trans, **kwargs )
> File
> '/data/local/galaxy/galaxy-prod/lib/galaxy/web/framework/__init__.py', line
> 184 in decorator
> return func( self, trans, *args, **kwargs )
> File
>
'/data/local/galaxy/galaxy-prod/lib/galaxy/web/controllers/admin_toolshed.py',
> line 356 in browse_repositories
> return self.deactivate_or_uninstall_repository( trans, **kwd )
> File
> '/data/local/galaxy/galaxy-prod/lib/galaxy/web/framework/__init__.py', line
> 184 in decorator
> return func( self, trans, *args, **kwargs )
> File
>
'/data/local/galaxy/galaxy-prod/lib/galaxy/web/controllers/admin_toolshed.py',
> line 408 in deactivate_or_uninstall_repository
> repository_install_dir = os.path.abspath ( relative_install_dir )
> File '/usr/local/python/lib/python2.6/posixpath.py', line 342 in abspath
> if not isabs(path):
> File '/usr/local/python/lib/python2.6/posixpath.py', line 52 in isabs
> return s.startswith('/')
> AttributeError: 'NoneType' object has no attribute 'startswith'
>
> Any thoughts on how I can get rid of the partially installed
> freebayes tool? Or how to troubleshoot the failing installation?
>
> Lance
>
> Daniel Blankenberg wrote:
>> Hi Lance,
>>
>> This has to do with the way that the updating currently works. Since
>> the migrated version is 0.0.2, the automatic update will only update
>> that particular Tool version (newest version of 0.0.2). To get
>> 0.0.3, you need to install it as a new tool. This way you are able
>> to have both versions installed and available e.g. for having a
>> reproducible rerun functionality. In the latest Galaxy-central /
>> upcoming Galaxy-dist release, only the newest version is shown in
>> the lefthand tool panel.
>>
>> Future enhancements will include allowing the selection of different
>> tool versions from within the tool interface (e.g. allowing the
>> rerun of an older version to use a newer version) and to show and
>> allow updating (installing) new tool versions from the same
>> interface (e.g. 2 to 3).
>>
>>
>> Thanks for using Galaxy,
>>
>> Dan
>>
>>
>> On Aug 31, 2012, at 11:47 AM, Lance Parsons wrote:
>>
>>> Thanks for the quick turnaround Dan. Much appreciated. When I
>>> tried to test updating Freebayes from the toolshed, I get a message
>>> that no updates are available. Perhaps it takes some time before
>>> they are recognized? Or am I missing something?
>>>
>>> Lance
>>>
>>> Daniel Blankenberg wrote:
>>>> Hi Lance,
>>>>
>>>> Thank you for the pull request. I think this is an excellent way
>>>> to handle patches until the toolshed itself provides a better
>>>> mechanism, particularly for the devteam owned repositories; other
>>>> repository owners may prefer different methods of providing
>>>> patches, but they can be contacted directly by using the contact
>>>> form on the toolshed. Future enhancements to the toolshed will
>>>> integrate better methods of providing this functionality, but that
>>>> likely some ways off. Any suggestions and community contributions
>>>> on this would be appreciated.
>>>>
>>>>
>>>> FreeBayes has been updated in the Toolshed to
>>>> 0.9.6_9608597d12e127c847ae03aa03440ab63992fedf, with mostly minor
>>>> differences from your pull request. Please let us know if you
>>>> experience any issues installing or using the updated version (0.0.3).
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Dan
>>>>
>>>>
>>>> On Aug 30, 2012, at 11:35 AM, Lance Parsons wrote:
>>>>
>>>>> I have been using the FreeBayes tool in our local Galaxy instance
>>>>> for some time. As FreeBayes is a rapidly evolving tool, it has
>>>>> been very helpful to have Daniel Blankenberg work on keeping an
>>>>> updated wrapper.
>>>>>
>>>>> Recently, I've run into a rather troublesome bug in FreeBayes
>>>>> (
https://github.com/ekg/freebayes/issues/22) that is causing it
>>>>> to crash on nearly every real dataset. Luckily, the bug has been
>>>>> fixed in the latest version of FreeBayes
>>>>> (0.9.6_9608597d12e127c847ae03aa03440ab63992fedf).
>>>>>
>>>>> Unfortunately, the latest version has a few parameter changes
>>>>> that require an update to the wrapper code. I've taken the
>>>>> liberty to make some changes to the wrapper and they seem to
>>>>> work. However, now that I have the tool installed from the Tool
>>>>> Shed, I'm not sure of the best way to 1) incorporate this update
>>>>> into my local galaxy instance and 2) contribute this change back
>>>>> to the Tool Shed.
>>>>>
>>>>> The best solution is to get the Tool Shed updated and then pull
>>>>> that update into my production Galaxy instance, however, I'm
>>>>> wondering if there is a way I could push "hot-fixes" to my
>>>>> production instance without causing me headaches down the line
>>>>> when the Tool Shed is inevitably updated.
>>>>>
>>>>> As far as coordinating updates of Tool Shed tools, I suppose the
>>>>> process might be different for different tool authors. I was
>>>>> wondering, however, if anyone had thoughts on how this might work
>>>>> best. I've created a clone of the galaxy tool shed repository in
>>>>> Bitbucket (
https://bitbucket.org/lance_parsons/freebayes/) and
>>>>> given Daniel and Greg access to it in hopes that would make
>>>>> things easier.Let me know if there is a better way to coordinate,
>>>>> etc.
>>>>>
>>>>> Thanks again.
>>>>>
>>>>>
>>>>> --
>>>>> Lance Parsons - Scientific Programmer
>>>>> 134 Carl C. Icahn Laboratory
>>>>> Lewis-Sigler Institute for Integrative Genomics
>>>>> Princeton University
>>>>>
>>>>> ___________________________________________________________
>>>>> Please keep all replies on the list by using "reply all"
>>>>> in your mail client. To manage your subscriptions to this
>>>>> and other Galaxy lists, please use the interface at:
>>>>>
>>>>>
http://lists.bx.psu.edu/
>>>>
>>>
>>> --
>>> Lance Parsons - Scientific Programmer
>>> 134 Carl C. Icahn Laboratory
>>> Lewis-Sigler Institute for Integrative Genomics
>>> Princeton University
>>>
>>
>
> --
> Lance Parsons - Scientific Programmer
> 134 Carl C. Icahn Laboratory
> Lewis-Sigler Institute for Integrative Genomics
> Princeton University
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client. To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>
>
http://lists.bx.psu.edu/
--
Lance Parsons - Scientific Programmer
134 Carl C. Icahn Laboratory
Lewis-Sigler Institute for Integrative Genomics
Princeton University