This also took out my job handlers (exception below). So the introduction of non-ascii characters into the table (via the job stdout/stderr capture) can make a galaxy instance pretty useless.

I was able to find the offending records using "SELECT count(*) FROM job WHERE stderr similar to '%\x8b%';"

Turns out it was the byproduct of doing a path paste of some gzipped fastq files (so they where never decompressed, just passed along), and then fastq_groomer.py complains about the file having an invalid header (line 22?) reports back an invalid header and prints it out, thus the non-ascii characters in the stderr. 

I 'cleaned' the database with 'update job set stderr = regexp_replace(stderr, '\x8b', '\x5f');'. But there should probably be some safe guards put in place to stop this from happening.

Kyle

galaxy.jobs.handler INFO 2013-07-06 12:35:32,033 job handler stop queue started

Traceback (most recent call last):

  File "/inside/depot4/galaxy/lib/galaxy/webapps/galaxy/buildapp.py", line 35, in app_factory

    app = UniverseApplication( global_conf = global_conf, **kwargs )

  File "/inside/depot4/galaxy/lib/galaxy/app.py", line 164, in __init__

    self.job_manager = manager.JobManager( self )

  File "/inside/depot4/galaxy/lib/galaxy/jobs/manager.py", line 36, in __init__

    self.job_handler.start()

  File "/inside/depot4/galaxy/lib/galaxy/jobs/handler.py", line 34, in start

    self.job_queue.start()

  File "/inside/depot4/galaxy/lib/galaxy/jobs/handler.py", line 77, in start

    self.__check_jobs_at_startup()

  File "/inside/depot4/galaxy/lib/galaxy/jobs/handler.py", line 92, in __check_jobs_at_startup

    & ( model.Job.handler == self.app.config.server_name ) ):

  File "/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/query.py", line 2341, in instances

    fetch = cursor.fetchall()

  File "/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py", line 3204, in fetchall

    l = self.process_rows(self._fetchall_impl())

  File "/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py", line 3171, in _fetchall_impl

    return self.cursor.fetchall()

UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 404: ordinal not in range(128)

On Fri, Jul 5, 2013 at 11:43 PM, Kyle Ellrott <kellrott@soe.ucsc.edu> wrote:

I'm getting an exception when trying to look at the admin job management screen. It looks like SQLAlchemy doesn't like non-ascii characters. Any ideas about what to do?

Error - <type 'exceptions.UnicodeDecodeError'>: 'ascii' codec can't decode byte 0x8b in position 404: ordinal not in range(128)

URL: http://pk.kilokluster.ucsc.edu:8079/admin/jobs

File '/inside/depot4/galaxy/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__

  app_iter = self.application(environ, sr_checker)

File '/inside/depot4/galaxy/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__

  return self.application(environ, start_response)

File '/inside/depot4/galaxy/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__

  return self.application(environ, start_response)

File '/inside/depot4/galaxy/lib/galaxy/web/framework/base.py', line 132 in __call__

  return self.handle_request( environ, start_response )

File '/inside/depot4/galaxy/lib/galaxy/web/framework/base.py', line 190 in handle_request

  body = method( trans, **kwargs )

File '/inside/depot4/galaxy/lib/galaxy/web/framework/__init__.py', line 221 in decorator

  return func( self, trans, *args, **kwargs )

File '/inside/depot4/galaxy/lib/galaxy/web/base/controllers/admin.py', line 1053 in jobs

  for job in jobs:

File '/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/query.py', line 2341 in instances

File '/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py', line 3204 in fetchall

File '/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py', line 3171 in _fetchall_impl

UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 404: ordinal not in range(128)

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/