strange error having to do with stderr redirection?
Hi all, I have been writing an R script to run as a tool in galaxy and it seems to work just fine. I needed to add some fine-grained error detection in the script and I wanted it to produce a "red X" if the error occurred. In order to do this I wrote to the stderr handle. Before I had put in the error code, I had redirected stderr to /dev/null in the xml, but I took that out so that galaxy could see the errors I wrote to the stderr handle. For some reason if I do not redirect stderr to /dev/null, I get this error: Traceback (most recent call last): File "/opt/Bio/galaxy-dist/lib/galaxy/jobs/runners/local.py", line 125, in run_job job_wrapper.finish( stdout, stderr ) File "/opt/Bio/galaxy-dist/lib/galaxy/jobs/__init__.py", line 643, in finish self.sa_session.flush() File "/opt/Bio/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/scoping.py", line 127, in do return getattr(self.registry(), name)(*args, **kwargs) File "/opt/Bio/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/session.py", line 1356, in flush self._flush(objects) File "/opt/Bio/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/session.py", line 1434, in _flush flush_context.execute() File "/opt/Bio/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 261, in execute UOWExecutor().execute(self, tasks) File "/opt/Bio/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 753, in execute self.execute_save_steps(trans, task) File "/opt/Bio/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 768, in execute_save_steps self.save_objects(trans, task) File "/opt/Bio/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 759, in save_objects task.mapper._save_obj(task.polymorphic_tosave_objects, trans) File "/opt/Bio/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/mapper.py", line 1413, in _save_obj c = connection.execute(statement.values(value_params), params) File "/opt/Bio/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 824, in execute return Connection.executors[c](self, object, multiparams, params) File "/opt/Bio/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 874, in _execute_clauseelement return self.__execute_context(context) File "/opt/Bio/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 896, in __execute_context self._cursor_execute(context.cursor, context.statement, context.parameters[0], context=context) File "/opt/Bio/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 950, in _cursor_execute self._handle_dbapi_exception(e, statement, parameters, cursor, context) File "/opt/Bio/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 931, in _handle_dbapi_exception raise exc.DBAPIError.instance(statement, parameters, e, connection_invalidated=is_disconnect) ProgrammingError: (ProgrammingError) You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings. u'UPDATE job SET update_time=?, stdout=?, stderr=? WHERE job.id = ?' ['2012-02-22 02:50:49.979136', 'locfit 1.5-6 \t 2010-01-20 \nnull device \n 1 \nnull device \n 1 \nnull device \n 1 \nnull device \n 1 \n\n Inf -Inf NaN \n 924 447 1710 0 \nnull device \n 1 \nnull device \n 1 \nnull device \n 1 \n', 'Loading required package: Biobase\n\nWelcome to Bioconductor\n\n Vignettes contain introductory material. To view, type\n \'browseVignettes()\'. To cite Bioconductor, see\n \'citation("Biobase")\' and for packages \'citation("pkgname")\'.\n\nLoading required package: locfit\nLoading required package: akima\nLoading required package: lattice\nWarning messages:\n1: In xy.coords(x, y, xlabel, ylabel, log) :\n 1710 x values <= 0 omitted from logarithmic plot\n2: In xy.coords(x, y, xlabel, ylabel, log) :\n 2880 y values <= 0 omitted from logarithmic plot\nWarning message:\nIn xy.coords(x, y, xlabel, ylabel, log) :\n 1710 x values <= 0 omitted from logarithmic plot\nLoading required package: R.methodsS3\nR.methodsS3 v1.2.1 (2010-09-18) successfully loaded. See ?R.methodsS3 for help.\nLoading required package: R.oo\nR.oo v1.8.3 (2011-11-01) successfully loaded. See ?R.oo for help.\n\nAttaching package: \xe2\x80\x98R.oo\xe2\x80\x99\n\nThe following object(s) are masked from \xe2\x80\x98package:R.methodsS3\xe2\x80\x99:\n\n throw.default\n\nThe following object(s) are masked from \xe2\x80\x98package:methods\xe2\x80\x99:\n\n getClass, getClasses, getMethods\n\nThe following object(s) are masked from \xe2\x80\x98package:base\xe2\x80\x99:\n\n attach, detach, environment, gc, load, save\n\naroma.light v1.22.0 (2011-10-31) successfully loaded. See ?aroma.light for help.\n', 1345] It seems that R produces some output on stdout that I can't get rid of yet, but it doesn't seem to affect the outcome... it doesn't explain why this error happens... Any ideas? Is R using some sort of weird encoding that the python can't get? - Nik. -- Nikhil Joshi Bioinformatics Programmer UC Davis Genome Center Davis, CA
Hi, Nikhil, In my experience, R chatters to stderr in an unhelpful way such as to let you know a package got loaded or a function got masked... That's the main reason I've given up trying to execute R as a command directly - I find myself writing python wrappers (usually to call Rscript using subprocess) - so I can deal with this in a controlled way. It sure looks like there are lots of what look like unicode thingys in the error - eg "The following object(s) are masked from \xe2\x80\x98package:methods\xe2\x80\x99:\n\n getClass, getClasses, getMethods\n\nThe following object(s) are masked from \xe2\x80\x98package:base\xe2\x80\x99:\n\n attach, detach, environment, gc, load, save\n\naroma.light v1.22.0 (2011-10-31) successfully loaded. See ?aroma.light for help.\n', 1345]" <untested> If you don't want to write a wrapper to control R's bad habits, you could maybe try sink() in your R script to divert all messages to a file which you can then leave in the history for posterity - or not...depending on whether it matters. </untested> On Wed, Feb 22, 2012 at 2:24 PM, Nikhil Joshi <najoshi@ucdavis.edu> wrote:
Hi all,
I have been writing an R script to run as a tool in galaxy and it seems to work just fine. I needed to add some fine-grained error detection in the script and I wanted it to produce a "red X" if the error occurred. In order to do this I wrote to the stderr handle. Before I had put in the error code, I had redirected stderr to /dev/null in the xml, but I took that out so that galaxy could see the errors I wrote to the stderr handle. For some reason if I do not redirect stderr to /dev/null, I get this error:
Traceback (most recent call last): File "/opt/Bio/galaxy-dist/lib/galaxy/jobs/runners/local.py", line 125, in run_job job_wrapper.finish( stdout, stderr ) File "/opt/Bio/galaxy-dist/lib/galaxy/jobs/__init__.py", line 643, in finish self.sa_session.flush() File "/opt/Bio/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/scoping.py", line 127, in do return getattr(self.registry(), name)(*args, **kwargs) File "/opt/Bio/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/session.py", line 1356, in flush self._flush(objects) File "/opt/Bio/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/session.py", line 1434, in _flush flush_context.execute() File "/opt/Bio/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 261, in execute UOWExecutor().execute(self, tasks) File "/opt/Bio/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 753, in execute self.execute_save_steps(trans, task) File "/opt/Bio/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 768, in execute_save_steps self.save_objects(trans, task) File "/opt/Bio/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/unitofwork.py", line 759, in save_objects task.mapper._save_obj(task.polymorphic_tosave_objects, trans)
File "/opt/Bio/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/orm/mapper.py", line 1413, in _save_obj c = connection.execute(statement.values(value_params), params) File "/opt/Bio/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 824, in execute return Connection.executors[c](self, object, multiparams, params) File "/opt/Bio/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 874, in _execute_clauseelement return self.__execute_context(context) File "/opt/Bio/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 896, in __execute_context self._cursor_execute(context.cursor, context.statement, context.parameters[0], context=context) File "/opt/Bio/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 950, in _cursor_execute self._handle_dbapi_exception(e, statement, parameters, cursor, context) File "/opt/Bio/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 931, in _handle_dbapi_exception raise exc.DBAPIError.instance(statement, parameters, e, connection_invalidated=is_disconnect) ProgrammingError: (ProgrammingError) You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings. u'UPDATE job SET update_time=?, stdout=?, stderr=? WHERE job.id = ?' ['2012-02-22 02:50:49.979136', 'locfit 1.5-6 \t 2010-01-20 \nnull device \n 1 \nnull device \n 1 \nnull device \n 1 \nnull device \n 1 \n\n Inf -Inf NaN \n 924 447 1710 0 \nnull device \n 1 \nnull device \n 1 \nnull device \n 1 \n', 'Loading required package: Biobase\n\nWelcome to Bioconductor\n\n Vignettes contain introductory material. To view, type\n \'browseVignettes()\'. To cite Bioconductor, see\n \'citation("Biobase")\' and for packages \'citation("pkgname")\'.\n\nLoading required package: locfit\nLoading required package: akima\nLoading required package: lattice\nWarning messages:\n1: In xy.coords(x, y, xlabel, ylabel, log) :\n 1710 x values <= 0 omitted from logarithmic plot\n2: In xy.coords(x, y, xlabel, ylabel, log) :\n 2880 y values <= 0 omitted from logarithmic plot\nWarning message:\nIn xy.coords(x, y, xlabel, ylabel, log) :\n 1710 x values <= 0 omitted from logarithmic plot\nLoading required package: R.methodsS3\nR.methodsS3 v1.2.1 (2010-09-18) successfully loaded. See ?R.methodsS3 for help.\nLoading required package: R.oo\nR.oo v1.8.3 (2011-11-01) successfully loaded. See ?R.oo for help.\n\nAttaching package: \xe2\x80\x98R.oo\xe2\x80\x99\n\nThe following object(s) are masked from \xe2\x80\x98package:R.methodsS3\xe2\x80\x99:\n\n throw.default\n\nThe following object(s) are masked from \xe2\x80\x98package:methods\xe2\x80\x99:\n\n getClass, getClasses, getMethods\n\nThe following object(s) are masked from \xe2\x80\x98package:base\xe2\x80\x99:\n\n attach, detach, environment, gc, load, save\n\naroma.light v1.22.0 (2011-10-31) successfully loaded. See ?aroma.light for help.\n', 1345]
It seems that R produces some output on stdout that I can't get rid of yet, but it doesn't seem to affect the outcome... it doesn't explain why this error happens... Any ideas? Is R using some sort of weird encoding that the python can't get?
- Nik.
-- Nikhil Joshi Bioinformatics Programmer UC Davis Genome Center Davis, CA ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
-- Ross Lazarus MBBS MPH; Associate Professor, Harvard Medical School; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444;
participants (2)
-
Nikhil Joshi -
Ross