At Thu, 19 Sep 2013 11:44:35 +0200, Bjoern Gruening wrote:
Hi Simon,
it's just a guess but can you try the attached patch if you have a reproducible case. It is not tested, but at least it should point you in the right direction. If its working, I will create a pull request.
Hi Bjoern,
Thanks for your reply and your patch. (Sorry for the delay in responding, I have been busy with other stuff.)
While your patch didn't solve my problem, it certainly pointed me in the right direction.
I made a new patch based on your idea, which I attach. The idea is, stdout and stderr of each tool run is converted from possible UTF-8 to ASCII, using a new function asciify which I added to Galaxy util.
I made a repository on the Galaxy test toolshed called unicode_tester, to test this. This has 2 tools which generates some Unicode output, either on stdout or stderr accordingly. Log output shows that unicode characters are being removed.
What else should I do to contribute this patch into Galaxy core?
cheers, Simon
My optimism was short lived. I think I've hit a galaxy bug in Unicode processing. Having managed to get the deseq tool not to crash galaxy in the sqlalchemy layer, I now get a crash when clicking on the info->stderr for the tool output dataset, probably because it contains Unicode. Here's the traceback from paster.log:
Exception happened during processing of request from ('127.0.0.1', 47124) Traceback (most recent call last): File "/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py", line 1068, in process_request_in_thread self.finish_request(request, client_address) File "/usr/lib64/python2.6/SocketServer.py", line 322, in finish_request self.RequestHandlerClass(request, client_address, self) File "/usr/lib64/python2.6/SocketServer.py", line 617, in __init__ self.handle() File "/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py", line 442, in handle BaseHTTPRequestHandler.handle(self) File "/usr/lib64/python2.6/BaseHTTPServer.py", line 329, in handle self.handle_one_request() File "/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py", line 437, in handle_one_request self.wsgi_execute() File "/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py", line 290, in wsgi_execute self.wsgi_write_chunk(chunk) File "/home/galaxy-dev/galaxy/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py", line 150, in wsgi_write_chunk self.wfile.write(chunk) File "/usr/lib64/python2.6/socket.py", line 316, in write data = str(data) # XXX Should really reject non-string non-buffers UnicodeEncodeError: 'ascii' codec can't encode character u'\u2018' in position 169: ordinal not in range(128)
I don't know where to look to find the actual stderr output. Any pointers?
Is this actually a galaxy bug now?
cheers, Simon
======================================================================= Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. =======================================================================