That's a great point. I've made a bug card for it here:
Thanks for reporting this.
Carl
On Sat, Jul 6, 2013 at 11:33 PM, Kyle Ellrott <kellrott(a)soe.ucsc.edu> wrote:
This also took out my job handlers (exception below). So the
introduction
of non-ascii characters into the table (via the job stdout/stderr capture)
can make a galaxy instance pretty useless.
I was able to find the offending records using "SELECT count(*) FROM job
WHERE stderr similar to '%\x8b%';"
Turns out it was the byproduct of doing a path paste of some gzipped fastq
files (so they where never decompressed, just passed along), and then
fastq_groomer.py complains about the file having an invalid header (line
22?) reports back an invalid header and prints it out, thus the non-ascii
characters in the stderr.
I 'cleaned' the database with 'update job set stderr =
regexp_replace(stderr, '\x8b', '\x5f');'. But there should probably
be some
safe guards put in place to stop this from happening.
Kyle
galaxy.jobs.handler INFO 2013-07-06 12:35:32,033 job handler stop queue
started
Traceback (most recent call last):
File "/inside/depot4/galaxy/lib/galaxy/webapps/galaxy/buildapp.py", line
35, in app_factory
app = UniverseApplication( global_conf = global_conf, **kwargs )
File "/inside/depot4/galaxy/lib/galaxy/app.py", line 164, in __init__
self.job_manager = manager.JobManager( self )
File "/inside/depot4/galaxy/lib/galaxy/jobs/manager.py", line 36, in
__init__
self.job_handler.start()
File "/inside/depot4/galaxy/lib/galaxy/jobs/handler.py", line 34, in
start
self.job_queue.start()
File "/inside/depot4/galaxy/lib/galaxy/jobs/handler.py", line 77, in
start
self.__check_jobs_at_startup()
File "/inside/depot4/galaxy/lib/galaxy/jobs/handler.py", line 92, in
__check_jobs_at_startup
& ( model.Job.handler == self.app.config.server_name ) ):
File
"/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/query.py",
line 2341, in instances
fetch = cursor.fetchall()
File
"/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py",
line 3204, in fetchall
l = self.process_rows(self._fetchall_impl())
File
"/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py",
line 3171, in _fetchall_impl
return self.cursor.fetchall()
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 404:
ordinal not in range(128)
On Fri, Jul 5, 2013 at 11:43 PM, Kyle Ellrott <kellrott(a)soe.ucsc.edu>wrote:
> I'm getting an exception when trying to look at the admin job management
> screen. It looks like SQLAlchemy doesn't like non-ascii characters. Any
> ideas about what to do?
>
>
> Error - <type 'exceptions.UnicodeDecodeError'>: 'ascii' codec
can't
> decode byte 0x8b in position 404: ordinal not in range(128)
> URL:
http://pk.kilokluster.ucsc.edu:8079/admin/jobs
> File
> '/inside/depot4/galaxy/lib/galaxy/web/framework/middleware/error.py', line
> 149 in __call__
> app_iter = self.application(environ, sr_checker)
> File
> '/inside/depot4/galaxy/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py',
> line 84 in __call__
> return self.application(environ, start_response)
> File
>
'/inside/depot4/galaxy/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py',
> line 633 in __call__
> return self.application(environ, start_response)
> File '/inside/depot4/galaxy/lib/galaxy/web/framework/base.py', line 132
> in __call__
> return self.handle_request( environ, start_response )
> File '/inside/depot4/galaxy/lib/galaxy/web/framework/base.py', line 190
> in handle_request
> body = method( trans, **kwargs )
> File '/inside/depot4/galaxy/lib/galaxy/web/framework/__init__.py', line
> 221 in decorator
> return func( self, trans, *args, **kwargs )
> File '/inside/depot4/galaxy/lib/galaxy/web/base/controllers/admin.py',
> line 1053 in jobs
> for job in jobs:
> File
>
'/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/query.py',
> line 2341 in instances
> File
>
'/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py',
> line 3204 in fetchall
> File
>
'/inside/depot4/galaxy/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py',
> line 3171 in _fetchall_impl
> UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position
404:
> ordinal not in range(128)
>
>
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/