May 2013 - galaxy-dev - lists.galaxyproject.org

Problem with Tools Relying on R
by Panzer, Adam 13 May '13

13 May '13

Hello All, I have recently deployed a local instance of Galaxy at work and while I've been able to troubleshoot problems with all other tool types, I have a persistent issue with using any tool that relies on R. For deseq: Traceback (most recent call last): File "/home/adam/galaxy-dist/lib/galaxy/jobs/runners/local.py", line 109, in queue_job job_wrapper.finish( stdout, stderr, exit_code ) File "/home/adam/galaxy-dist/lib/galaxy/jobs/__init__.py", line 999, in finish self.sa_session.flush() File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/scoping.py", line 127, in do return getattr(self.registry(), name)(*args, **kwargs) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/session.py", line 1356, in flush self._flush(objects) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/session.py", line 1434, in _flush flush_context.execute() File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/unitofwork.py", line 261, in execute UOWExecutor().execute(self, tasks) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/unitofwork.py", line 753, in execute self.execute_save_steps(trans, task) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/unitofwork.py", line 768, in execute_save_steps self.save_objects(trans, task) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/unitofwork.py", line 759, in save_objects task.mapper._save_obj(task.polymorphic_tosave_objects, trans) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/mapper.py", line 1413, in _save_obj c = connection.execute(statement.values(value_params), params) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py", line 824, in execute return Connection.executors[c](self, object, multiparams, params) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py", line 874, in _execute_clauseelement return self.__execute_context(context) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py", line 896, in __execute_context self._cursor_execute(context.cursor, context.statement, context.parameters[0], context=context) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py", line 950, in _cursor_execute self._handle_dbapi_exception(e, statement, parameters, cursor, context) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py", line 931, in _handle_dbapi_exception raise exc.DBAPIError.instance(statement, parameters, e, connection_invalidated=is_disconnect) ProgrammingError: (ProgrammingError) You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings. u'UPDATE job SET update_time=?, stdout=?, stderr=? WHERE job.id = ?' ['2013-05-10 06:12:01.564757', '', 'Error: Loading required package: Biobase\nLoading required package: BiocGenerics\nLoading required package: methods\nLoading required package: parallel\n\nAttaching package: \xe2\x80\x98BiocGenerics\xe2\x80\x99\n\nThe following objects are masked from \xe2\x80\x98package:parallel\xe2\x80\x99:\n\n clusterApply, clusterApplyLB, clusterCall, clusterEvalQ,\n clusterExport, clusterMap, parApply, parCapply, parLapply,\n parLapplyLB, parRapply, parSapply, parSapplyLB\n\nThe following object is masked from \xe2\x80\x98package:stats\xe2\x80\x99:\n\n xtabs\n\nThe following objects are masked from \xe2\x80\x98package:base\xe2\x8 0\x99:\n\n anyDuplicated, as.data.frame, cbind, colnames, duplicated, eval,\n Filter, Find, get, intersect, lapply, Map, mapply, match, mget,\n order, paste, pmax, pmax.int, pmin, pmin.int, Position, rank,\n rbind, Reduce, rep.int, rownames, sapply, setdiff, sort, table,\n tapply, union, unique, unlist\n\nWelcome to Bioconductor\n\n Vignettes contain introductory material; view with\n \'browseVignettes()\'. To cite Bioconductor, see\n \'citation("Biobase")\', and for packages \'citation("pkgname")\'.\n\nLoading required package: locfit\nlocfit 1.5-9.1 \t 2013-03-22\nLoading required package: lattice\nWarning message:\nIn xy.coords(x, y, xlabel, ylabel, log) :\n 9 y values <= 0 omitted from logarithmic plot\nError in hclustfun(distfun(x)) : \n NA/NaN/Inf in foreign function call (arg 11)\nCalls: heatmap -> hclustfun\nExecution halted\n\n', 116] For cummerbund: Traceback (most recent call last): File "/home/adam/galaxy-dist/lib/galaxy/jobs/runners/local.py", line 109, in queue_job job_wrapper.finish( stdout, stderr, exit_code ) File "/home/adam/galaxy-dist/lib/galaxy/jobs/__init__.py", line 999, in finish self.sa_session.flush() File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/scoping.py", line 127, in do return getattr(self.registry(), name)(*args, **kwargs) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/session.py", line 1356, in flush self._flush(objects) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/session.py", line 1434, in _flush flush_context.execute() File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/unitofwork.py", line 261, in execute UOWExecutor().execute(self, tasks) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/unitofwork.py", line 753, in execute self.execute_save_steps(trans, task) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/unitofwork.py", line 768, in execute_save_steps self.save_objects(trans, task) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/unitofwork.py", line 759, in save_objects task.mapper._save_obj(task.polymorphic_tosave_objects, trans) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/mapper.py", line 1413, in _save_obj c = connection.execute(statement.values(value_params), params) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py", line 824, in execute return Connection.executors[c](self, object, multiparams, params) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py", line 874, in _execute_clauseelement return self.__execute_context(context) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py", line 896, in __execute_context self._cursor_execute(context.cursor, context.statement, context.parameters[0], context=context) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py", line 950, in _cursor_execute self._handle_dbapi_exception(e, statement, parameters, cursor, context) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py", line 931, in _handle_dbapi_exception raise exc.DBAPIError.instance(statement, parameters, e, connection_invalidated=is_disconnect) ProgrammingError: (ProgrammingError) You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings. u'UPDATE job SET update_time=?, stdout=?, stderr=? WHERE job.id = ?' ['2013-05-10 04:38:55.946925', '', 'Fatal error: Exit code 1 (CummerBund Error)\nError running R script. Error in library("cummeRbund") : there is no package called \xe2\x80\x98cummeRbund\xe2\x80\x99\nExecution halted\n\n', 112] It appears that they all fail in a similar fashion. I noticed a similar issue brought up about a year ago but deploying the fixes mentioned then (using a python wrapper to keep R messages out of stderr and telling R to run without curly quotes) has not done anything to stop it. Any ideas would be most appreciated as several R tools are a part of my target RNA-Seq pipeline. Thanks! Adam The materials in this email are private and may contain Protected Health Information. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying, distribution or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return email.

2 1

Re: [galaxy-dev] Display with IGV local
by menorca chaturvedi 13 May '13

13 May '13

Hi I am also facing a similar issue. Did you solve the problem? would be great if I could know how to! Many thanks Menorca

1 0

Column argument validation breaks on empty tabular files
by Peter Cock 10 May '13

10 May '13

Hello all, I'm testing a workflow which uses my filter tool where one of the parameters is a tabular file, and a related parameter is a multi-column selector: Tool: http://toolshed.g2.bx.psu.edu/view/peterjc/seq_filter_by_id Snippet from tool XML, <param name="input_tabular" type="data" format="tabular" label="Tabular file containing sequence identifiers"/> <param name="columns" type="data_column" data_ref="input_tabular" multiple="True" numerical="False" label="Column(s) containing sequence identifiers" help="Multi-select list - hold the appropriate key while clicking to select multiple columns"> <validator type="no_options" message="Pick at least one column"/> </param> In most cases, the preceding tool will generate a tabular file with two columns - and the workflow has column 1 as the saved value. However, it is quite valid for the tabular file to actually be empty - in which case the number of columns is unknown and so the arguments can't be checked. Despite this, Galaxy attempts to check the column argument from the workflow and fails: Traceback (most recent call last): File "/mnt/galaxy/galaxy-central/lib/galaxy/jobs/runners/__init__.py", line 119, in prepare_job job_wrapper.prepare() File "/mnt/galaxy/galaxy-central/lib/galaxy/jobs/__init__.py", line 656, in prepare self.tool.handle_unvalidated_param_values( incoming, self.app ) File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/__init__.py", line 2330, in handle_unvalidated_param_values self.handle_unvalidated_param_values_helper( self.inputs, input_values, app ) File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/__init__.py", line 2369, in handle_unvalidated_param_values_helper raise LateValidationError( message ) LateValidationError Tool execution generated the following error message: Failed runtime validation of Column(s) containing sequence identifiers (An invalid option was selected, please verify) It seems to me this is a bug, and the late validation is being too strict where as in this case the number of columns is undefined. Peter

1 0

Protocol for changing the section of an installed tool shed repo
by Joachim Jacob | VIB | 10 May '13

10 May '13

Hi all, Stupidly enough, I installed a tool shed repo under 'root' of the toolbox. Yes, it was EMBOSS5.0 (btw, installation went flawless!). 1. From the admin menu, I choose to unactivate the repo (manage tool shed repo's) 2. From the admin menu, I reinstalled the tool shed from Main Tool Shed, and chose to put it under a section 'EMBOSS'. This to no avail: all tools still under root. 3. I manually edited the shed_tool_conf.xml and added the section tags 4. The section is now displayed, containing the EMBOSS tools. BUT the tools under the root of the toolbox are still there. Any assistance here? Cheers, Joachim -- Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib

3 3

Something wrong with connecting Galaxy with mysql/postgresql server when uploading datasets
by 王聃Don 10 May '13

10 May '13

Hello, Galaxy developers: I'm a Galaxy user and to customize and deploy Galaxy server locally. I need to build a connection between Galaxy and database server now. MySQL and PostgreSQL are both configured to test the connection. ################################################ #for mysql server and db name is galaxy database_connection=mysql:///galaxy?unix_socket=/var/run/mysqld/mysqld.sock&user=<user_name>&passwd=<password_string> #for postgresql and db name is galaxy database_connection=postgresql:///galaxy?host=/var/run/postgresql&user=postgres&password=<password_string> ################################################ Now starting Galaxy server has no errors and I can create a user account on local Galaxy server successfully. But when uploading datasets into Galaxy(mostly less than 1 MB size) it remains "uploading" status(Dataset is uploading) and cannot complete datasets uploading job. I tested both PostgreSQL and MySQL and it really did not work. Shockingly using PostgreSQL the dataset uploading job can be accomplished very very slowly 3 or 4 times. wangdan

1 0

Missing revisions on (Test) Tool Shed
by Peter Cock 09 May '13

09 May '13

Hi all, There is probably a good reason for this, but why is Galaxy hiding revisions from me on the main 'manage repository' page? e.g. http://testtoolshed.g2.bx.psu.edu/view/peterjc/clinod Here the drop down box only lets me view: 4:07f2d99bc2de (current tip) 0:f886a0b8b117 Why is it hiding revisions 1, 2 and 3? 3:a66a914c39b5 2:3213023deae1 2:3213023deae1 If I try to be sneaky and use a revision specific URL like this, http://testtoolshed.g2.bx.psu.edu/view/peterjc/clinod/a66a914c39b5 then rather than going to the requested revision it jumps to (what is currently) the repository tip. I want to look at these intermediate revisions to see what effect the changes had on the unit tests. Thanks, Peter

2 2

UnicodeDecodeError: 'utf8' codec can't decode byte 0xb9 in position 15: invalid start byte
by Ramon Tiburski 09 May '13

09 May '13

Hi, I uploaded some data and when I click in the eye to see the data I get the following error. I don't know what can I do to fix this problem, I trying install some packges from python but didn't solve my problem. I'm starting think about my operational system, I have ubuntu 12.04 32bit, maybe I need get a 64bit. Someone can help me?Server Error URL: http://localhost:8080/datasets/8c49be448cfe29bc/display/?preview=True Module paste.exceptions.errormiddleware:144 in __call__ >> app_iter = self.application(environ, sr_checker) Module paste.debug.prints:106 in __call__ >> environ, self.app) Module paste.wsgilib:543 in intercept_output >> app_iter = application(environ, replacement_start_response) Module paste.recursive:84 in __call__ >> return self.application(environ, start_response) Module paste.httpexceptions:633 in __call__ >> return self.application(environ, start_response) Module galaxy.web.framework.base:132 in __call__ >> return self.handle_request( environ, start_response ) Module galaxy.web.framework.base:194 in handle_request >> return body( environ, start_response ) Module galaxy.web.framework:876 in render >> template.render_context( context ) Module mako.template:319 in render_context >> **kwargs) Module mako.runtime:692 in _render_context >> _exec_template(inherit, lclcontext, args=args, kwargs=kwargs) Module mako.runtime:718 in _exec_template >> callable_(context, *args, **kwargs) Module _base_mako:42 in render_body >> __M_writer(unicode(next.body())) Module _dataset_large_file_mako:45 in render_body >> __M_writer(filters.html_escape(unicode( unicode( truncated_data, 'utf-8' ) ))) UnicodeDecodeError: 'utf8' codec can't decode byte 0xb9 in position 15: invalid start byte

2 4

Multiple values in <when> tags for <conditiona> parameters
by Peter Cock 09 May '13

09 May '13

Hello all, I'm exploring a <conditional> example where I would like to group some of the select values into a single <when> clause. Currently it seems each value requires its own <when> which leads to duplication and worse. For example, I'd like to do something like this: <conditional name="id_opts"> <param name="set_operator" type="select" label="Source of sequence identifier list"> <option value="SINGLE">Single file - want sequences listed in ONE tabular file</option> <option value="UNION">Union - want sequences listed in ANY of given tabular files</option> <option value="INTERSECTION">Intersection - want sequences listed in ALL of given tabular files</option> </param> <when value="SINGLE">  <param name="input_tabular" type="data" format="tabular" label="Tabular file containing sequence identifiers"/> </when>  <when values="UNION,INTERSECTION"> <repeat name="identifiers" title="Tabular file(s) with sequence identifiers" min="2"> <param name="input_tabular" type="data" format="tabular" label="Tabular file containing sequence identifiers"/> </repeat> </when> </conditional> But instead I am forced to use: <conditional name="id_opts"> <param name="set_operator" type="select" label="Source of sequence identifier list"> <option value="SINGLE">Single file - want sequences listed in ONE tabular file</option> <option value="UNION">Union - want sequences listed in ANY of given tabular files</option> <option value="INTERSECTION">Intersection - want sequences listed in ALL of given tabular files</option> </param> <when value="SINGLE">  <param name="input_tabular" type="data" format="tabular" label="Tabular file containing sequence identifiers"/> </when>  <when value="UNION"> <repeat name="identifiers" title="Tabular file(s) with sequence identifiers" min="2"> <param name="input_tabular" type="data" format="tabular" label="Tabular file containing sequence identifiers"/> </repeat> </when> <when value="INTERSECTION"> <repeat name="identifiers" title="Tabular file(s) with sequence identifiers" min="2"> <param name="input_tabular" type="data" format="tabular" label="Tabular file containing sequence identifiers"/> </repeat> </when> </conditional> This has a major usability downside in that when the user switches from UNION to INTERSECTION, any previously entered values for the <when> parameters are lost. Is this something that has already been enhanced, or is the wiki page accurate regarding the current inflexible <when> behaviour? http://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax I can see how to avoid the issue where by splitting this 3-way select into two separate parameters, but that feels more awkward. Thanks, Peter

1 1

migration error
by Robert Baertsch 09 May '13

09 May '13

I upgraded to the latest galaxy-central and got an error when running migration script 115 which lengthens the password field from 40->255. It failed saying that the table migration_tmp already exists. I ran this without any existing database so I don't think it is anything on my end. Any pointers? .schema migration_tmp CREATE TABLE "migration_tmp" ( id INTEGER NOT NULL, create_time TIMESTAMP, update_time TIMESTAMP, tool_shed_repository_id INTEGER NOT NULL, name VARCHAR(255), version VARCHAR(40), type VARCHAR(40), uninstalled BOOLEAN, error_message TEXT, PRIMARY KEY (id), FOREIGN KEY(tool_shed_repository_id) REFERENCES tool_shed_repository (id) );

3 5

Re: [galaxy-dev] Multiple values in <when> tags for <conditiona> parameters
by John Chilton 09 May '13

09 May '13

Yup, off list by mistake. -John On Thu, May 9, 2013 at 8:40 AM, Peter Cock <p.j.a.cock(a)googlemail.com> wrote: > Off list by mistake? > > On Thu, May 9, 2013 at 1:25 PM, John Chilton <chil0060(a)umn.edu> wrote: >> This is one of the use cases I was trying to address with tool macros. This >> has the added benefit of reducing the level of XML nesting in such tools as >> well. >> >> https://bitbucket.org/galaxyp/galaxyp-toolshed-maxquant/src/fc2de41f51a9168… >> >> -John > > Yep - macros could be a solution here - and I could start using them in > the NCBI BLAST+ wrappers to reduce the duplication too. > > Peter

1 0