This also took out my job handlers (exception below). So the introduction of non-ascii characters into the table (via the job stdout/stderr capture) can make a galaxy instance pretty useless.
I was able to find the offending records using "SELECT count(*) FROM job WHERE stderr similar to '%\x8b%';" Turns out it was the byproduct of doing a path paste of some gzipped fastq files (so they where never decompressed, just passed along), and then fastq_groomer.py complains about the file having an invalid header (line 22?) reports back an invalid header and prints it out, thus the non-ascii characters in the stderr.
I 'cleaned' the database with 'update job set stderr = regexp_replace(stderr, '\x8b', '\x5f');'. But there should probably be some safe guards put in place to stop this from happening.
Kyle
galaxy.jobs.handler INFO 2013-07-06 12:35:32,033 job handler stop queue started Traceback (most recent call last): File "/inside/depot4/galaxy/lib/galaxy/webapps/galaxy/buildapp.py", line 35, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/inside/depot4/galaxy/lib/galaxy/app.py", line 164, in __init__ self.job_manager = manager.JobManager( self ) File "/inside/depot4/galaxy/lib/galaxy/jobs/manager.py", line 36, in __init__ self.job_handler.start() File "/inside/depot4/galaxy/lib/galaxy/jobs/handler.py", line 34, in start self.job_queue.start() File "/inside/depot4/galaxy/lib/galaxy/jobs/handler.py", line 77, in start self.__check_jobs_at_startup() File "/inside/depot4/galaxy/lib/galaxy/jobs/handler.py", line 92, in __check_jobs_at_startup & ( model.Job.handler == self.app.config.server_name ) ): File "/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/query.py", line 2341, in instances fetch = cursor.fetchall() File "/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py", line 3204, in fetchall l = self.process_rows(self._fetchall_impl()) File "/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py", line 3171, in _fetchall_impl return self.cursor.fetchall() UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 404: ordinal not in range(128)
On Fri, Jul 5, 2013 at 11:43 PM, Kyle Ellrott kellrott@soe.ucsc.edu wrote:
I'm getting an exception when trying to look at the admin job management screen. It looks like SQLAlchemy doesn't like non-ascii characters. Any ideas about what to do?
Error - <type 'exceptions.UnicodeDecodeError'>: 'ascii' codec can't decode byte 0x8b in position 404: ordinal not in range(128) URL: http://pk.kilokluster.ucsc.edu:8079/admin/jobs File '/inside/depot4/galaxy/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/inside/depot4/galaxy/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/inside/depot4/galaxy/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/inside/depot4/galaxy/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '/inside/depot4/galaxy/lib/galaxy/web/framework/base.py', line 190 in handle_request body = method( trans, **kwargs ) File '/inside/depot4/galaxy/lib/galaxy/web/framework/__init__.py', line 221 in decorator return func( self, trans, *args, **kwargs ) File '/inside/depot4/galaxy/lib/galaxy/web/base/controllers/admin.py', line 1053 in jobs for job in jobs: File '/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/query.py', line 2341 in instances File '/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py', line 3204 in fetchall File '/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py', line 3171 in _fetchall_impl UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 404: ordinal not in range(128)