This also took out my job handlers (exception below). So the introduction of non-ascii characters into the table (via the job stdout/stderr capture) can make a galaxy instance pretty useless.

I was able to find the offending records using "SELECT count(*) FROM job WHERE stderr similar to '%\x8b%';"
Turns out it was the byproduct of doing a path paste of some gzipped fastq files (so they where never decompressed, just passed along), and then fastq_groomer.py complains about the file having an invalid header (line 22?) reports back an invalid header and prints it out, thus the non-ascii characters in the stderr. 

I 'cleaned' the database with 'update job set stderr = regexp_replace(stderr, '\x8b', '\x5f');'. But there should probably be some safe guards put in place to stop this from happening.

Kyle



galaxy.jobs.handler INFO 2013-07-06 12:35:32,033 job handler stop queue started
Traceback (most recent call last):
  File "/inside/depot4/galaxy/lib/galaxy/webapps/galaxy/buildapp.py", line 35, in app_factory
    app = UniverseApplication( global_conf = global_conf, **kwargs )
  File "/inside/depot4/galaxy/lib/galaxy/app.py", line 164, in __init__
    self.job_manager = manager.JobManager( self )
  File "/inside/depot4/galaxy/lib/galaxy/jobs/manager.py", line 36, in __init__
    self.job_handler.start()
  File "/inside/depot4/galaxy/lib/galaxy/jobs/handler.py", line 34, in start
    self.job_queue.start()
  File "/inside/depot4/galaxy/lib/galaxy/jobs/handler.py", line 77, in start
    self.__check_jobs_at_startup()
  File "/inside/depot4/galaxy/lib/galaxy/jobs/handler.py", line 92, in __check_jobs_at_startup
    & ( model.Job.handler == self.app.config.server_name ) ):
  File "/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/query.py", line 2341, in instances
    fetch = cursor.fetchall()
  File "/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py", line 3204, in fetchall
    l = self.process_rows(self._fetchall_impl())
  File "/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py", line 3171, in _fetchall_impl
    return self.cursor.fetchall()
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 404: ordinal not in range(128)



On Fri, Jul 5, 2013 at 11:43 PM, Kyle Ellrott <kellrott@soe.ucsc.edu> wrote:
I'm getting an exception when trying to look at the admin job management screen. It looks like SQLAlchemy doesn't like non-ascii characters. Any ideas about what to do?


Error - <type 'exceptions.UnicodeDecodeError'>: 'ascii' codec can't decode byte 0x8b in position 404: ordinal not in range(128)
File '/inside/depot4/galaxy/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__
  app_iter = self.application(environ, sr_checker)
File '/inside/depot4/galaxy/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__
  return self.application(environ, start_response)
File '/inside/depot4/galaxy/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__
  return self.application(environ, start_response)
File '/inside/depot4/galaxy/lib/galaxy/web/framework/base.py', line 132 in __call__
  return self.handle_request( environ, start_response )
File '/inside/depot4/galaxy/lib/galaxy/web/framework/base.py', line 190 in handle_request
  body = method( trans, **kwargs )
File '/inside/depot4/galaxy/lib/galaxy/web/framework/__init__.py', line 221 in decorator
  return func( self, trans, *args, **kwargs )
File '/inside/depot4/galaxy/lib/galaxy/web/base/controllers/admin.py', line 1053 in jobs
  for job in jobs:
File '/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/query.py', line 2341 in instances
File '/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py', line 3204 in fetchall
File '/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py', line 3171 in _fetchall_impl
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 404: ordinal not in range(128)