Thanks John - Would it be more productive for us to port gsummary.py as a shed tool? best - david On Oct 2, 2014, at 6:46 AM, John Chilton <jmchilton@gmail.com> wrote:
Thanks for the additional details - I am responding on list for people who may encounter similar problems in the future.
The job running code will provide both core tools and tool shed tools with the same PYTHONPATH - but some tools might setup a Python virtualenv for instance which would replace the PYTHONPATH completely I believe. Since gsummary is not in the tool shed - it will not be able to find any of the tool shed dependencies. These rpy issues are tough ones.
It should be possible to hack something together if you change the requirements in gsummary to be package requirements for R and Rpy and then find working env.sh files in tool_dependency_dir and copy them to tool_dependency_dir/R/2.11/env.sh and tool_dependency_dir/Rpy/1.0.3/env.sh respectively.
The tool shed is suppose to solve all of these pesky dependency problems - but not all the tools have been migrated and fixed yet. Thanks for your patience and for using Galaxy!
-John
On Wed, Oct 1, 2014 at 1:36 PM, David Bernick <dbernick@soe.ucsc.edu> wrote:
Hi John -
Yes, we are using MySQL. This event happened as a result of attempting to add/delete tools from our instance (Rpy_1_0_3 and R_2_11). We ended up reloading the database from our nightly backup, and that got us going again.
The issue I was trying to solve when I "broke" the database was Summary_statistics not being able to find its Rpy module. I now have R and Rpy installed, such that tool-shed tools are happy with it, but gsummary.py is still unable to find Rpy. We are attempting to solve this, but are as yet without a fix. In the meantime, I have installed datamash, and that is working, It seems that tool-shed tools have a different PYTHONPATH than galaxy-dist tools?
As it happens, we also had a few cases with multiple instances of tool-shed tools being installed. Since this server is brand new, it was fine to delete them. I suppose that maybe we are now fitting in MySQL BLOBs now??
thanks again - David
On Oct 1, 2014, at 9:44 AM, John Chilton <jmchilton@gmail.com> wrote:
Hello David,
We don't have a workaround yet - but we (Dan did most of the investigating) understand the problem I think. You are using a MySQL database I assume? (Ignore the rest of this e-mail if not). I think MySQL has lower limits on the size of BLOBs that can be stored in the database than postgres - and we think the tool shed is tried to storing a really large blob of metadata in the database and it got truncated and now Galaxy is failing because it cannot load that entry as JSON.
The work around would be something like trying to find the truncated column in the tool_repositories table - deleting it and references - changing that column to a LONGBLOB and reinstalling that tool.
I have created a Trello card for this bug report.
It is said a lot but we strongly encourage not using MySQL.
It might be worth trying this patch (https://bitbucket.org/galaxy/galaxy-central/pull-request/511/allow-model-obj...) from Dan as well. I am not sure what the tool shed would do if it cannot find the metadata though.
-John
On Mon, Sep 29, 2014 at 11:41 AM, David Bernick <dbernick@soe.ucsc.edu> wrote:
Good morning -
I have acquired an Internal server error on our local galaxy install. It seems to be an unterminated string. How do I resolve this? thanks in advance - David Bernick UCSC
Here is the relevant portion of the log (I hope).
Error - <type 'exceptions.ValueError'>: Unterminated string starting at: line 1 column 65528 (char 65528) URL: http://campusrocks.cse.ucsc.edu:8080/admin/index File '/projects/galaxy/bin/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/projects/galaxy/bin/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/projects/galaxy/bin/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/projects/galaxy/bin/galaxy-dist/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '/projects/galaxy/bin/galaxy-dist/lib/galaxy/web/framework/base.py', line 190 in handle_request body = method( trans, **kwargs ) File '/projects/galaxy/bin/galaxy-dist/lib/galaxy/web/framework/__init__.py', line 383 in decorator return func( self, trans, *args, **kwargs ) File '/projects/galaxy/bin/galaxy-dist/lib/galaxy/web/base/controllers/admin.py', line 34 in index installing_repository_ids = suc.get_ids_of_tool_shed_repositories_being_installed( trans.app, as_string=True ) File '/projects/galaxy/bin/galaxy-dist/lib/tool_shed/util/shed_util_common.py', line 277 in get_ids_of_tool_shed_repositories_being_installed app.install_model.ToolShedRepository.status == loading_datatypes_status ) ): File '/projects/galaxy/bin/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/query.py', line 2348 in instances File '/projects/galaxy/bin/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/mapper.py', line 2120 in _instance File '/projects/galaxy/bin/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/mapper.py', line 1974 in populate_state File '/projects/galaxy/bin/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/orm/strategies.py', line 150 in fetch_col File '/projects/galaxy/bin/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/types.py', line 658 in process File '/projects/galaxy/bin/galaxy-dist/lib/galaxy/model/custom_types.py', line 52 in process_result_value return json_decoder.decode( str( _sniffnfix_pg9_hex(value) ) ) File '/usr/lib64/python2.6/json/decoder.py', line 319 in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File '/usr/lib64/python2.6/json/decoder.py', line 336 in raw_decode obj, end = self._scanner.iterscan(s, **kw).next() File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan rval, next_pos = action(m, context) File '/usr/lib64/python2.6/json/decoder.py', line 183 in JSONObject value, end = iterscan(s, idx=end, context=context).next() File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan rval, next_pos = action(m, context) File '/usr/lib64/python2.6/json/decoder.py', line 217 in JSONArray value, end = iterscan(s, idx=end, context=context).next() File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan rval, next_pos = action(m, context) File '/usr/lib64/python2.6/json/decoder.py', line 183 in JSONObject value, end = iterscan(s, idx=end, context=context).next() File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan rval, next_pos = action(m, context) File '/usr/lib64/python2.6/json/decoder.py', line 217 in JSONArray value, end = iterscan(s, idx=end, context=context).next() File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan rval, next_pos = action(m, context) File '/usr/lib64/python2.6/json/decoder.py', line 183 in JSONObject value, end = iterscan(s, idx=end, context=context).next() File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan rval, next_pos = action(m, context) File '/usr/lib64/python2.6/json/decoder.py', line 217 in JSONArray value, end = iterscan(s, idx=end, context=context).next() File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan rval, next_pos = action(m, context) File '/usr/lib64/python2.6/json/decoder.py', line 217 in JSONArray value, end = iterscan(s, idx=end, context=context).next() File '/usr/lib64/python2.6/json/scanner.py', line 55 in iterscan rval, next_pos = action(m, context) File '/usr/lib64/python2.6/json/decoder.py', line 155 in JSONString return scanstring(match.string, match.end(), encoding, strict) ValueError: Unterminated string starting at: line 1 column 65528 (char 65528)
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/