- galaxy-dev - lists.galaxyproject.org

Script to help maintain toolshed repos across toolsheds
by Ira Cooke 13 May '13

13 May '13

Hi all, I've written a script to help deal with the problem of maintaining toolshed tools across multiple toolsheds (eg test and release) The problem I encountered was that switching between test and production versions of a suite of tools can be quite painful because every repository definition like this <repository toolshed="http://toolshed.g2.bx.psu.edu" name="proteomics_datatypes" owner="iracooke" changeset_revision="463328a6967f"/> needs to be updated to a different toolshed url and (by extension) a different changeset revision. The idea with this script is that you should be able to point it at a directory containing a toolshed repository and it will create a copy of that repository in which the toolshed urls (and changeset revisions) have been updated to correct values for a different toolshed. I'm not sure how others are dealing with this issue (perhaps there is another easier way) .. but I've found this helped me alot so I thought I'd share https://bitbucket.org/iracooke/galaxy_repo_bundler/ Cheers Ira

4 16

Function for deleting specific datasets from the history
by Aristos Aristodimou 13 May '13

13 May '13

Dear dev, I have two questions 1) Is there a function that I can use to delete a dataset from the history? I have a tool that uses as input a hidden dataset and I want to delete the hidden dataset once the tool is executed. I have the filename of the dataset and I was wondering if there is a function that I can use for deleting it. 2) When a user deletes a dataset is there a way to get the filename of the deleted dataset? Thanks, Aris

2 1

How is the paster.py running to start Galaxy Framework ?
by 王聃Don 13 May '13

13 May '13

Hello, Galaxy developers: I'm a galaxy user and have customized a Galaxy instance locally. I'm very interested in Galaxy framework and the way it's running. I've search over the doc from readthedocs and look through some source code files. But I'm confused at first: How to run paster.py to start all process to build a Galaxy server? I've no idea why the last line of code "command.run()" can start the whole Galaxy server and thanks for someone to point it out. Thanks for your help! wangdan

2 1

Problem with Tools Relying on R
by Panzer, Adam 13 May '13

13 May '13

Hello All, I have recently deployed a local instance of Galaxy at work and while I've been able to troubleshoot problems with all other tool types, I have a persistent issue with using any tool that relies on R. For deseq: Traceback (most recent call last): File "/home/adam/galaxy-dist/lib/galaxy/jobs/runners/local.py", line 109, in queue_job job_wrapper.finish( stdout, stderr, exit_code ) File "/home/adam/galaxy-dist/lib/galaxy/jobs/__init__.py", line 999, in finish self.sa_session.flush() File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/scoping.py", line 127, in do return getattr(self.registry(), name)(*args, **kwargs) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/session.py", line 1356, in flush self._flush(objects) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/session.py", line 1434, in _flush flush_context.execute() File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/unitofwork.py", line 261, in execute UOWExecutor().execute(self, tasks) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/unitofwork.py", line 753, in execute self.execute_save_steps(trans, task) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/unitofwork.py", line 768, in execute_save_steps self.save_objects(trans, task) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/unitofwork.py", line 759, in save_objects task.mapper._save_obj(task.polymorphic_tosave_objects, trans) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/mapper.py", line 1413, in _save_obj c = connection.execute(statement.values(value_params), params) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py", line 824, in execute return Connection.executors[c](self, object, multiparams, params) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py", line 874, in _execute_clauseelement return self.__execute_context(context) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py", line 896, in __execute_context self._cursor_execute(context.cursor, context.statement, context.parameters[0], context=context) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py", line 950, in _cursor_execute self._handle_dbapi_exception(e, statement, parameters, cursor, context) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py", line 931, in _handle_dbapi_exception raise exc.DBAPIError.instance(statement, parameters, e, connection_invalidated=is_disconnect) ProgrammingError: (ProgrammingError) You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings. u'UPDATE job SET update_time=?, stdout=?, stderr=? WHERE job.id = ?' ['2013-05-10 06:12:01.564757', '', 'Error: Loading required package: Biobase\nLoading required package: BiocGenerics\nLoading required package: methods\nLoading required package: parallel\n\nAttaching package: \xe2\x80\x98BiocGenerics\xe2\x80\x99\n\nThe following objects are masked from \xe2\x80\x98package:parallel\xe2\x80\x99:\n\n clusterApply, clusterApplyLB, clusterCall, clusterEvalQ,\n clusterExport, clusterMap, parApply, parCapply, parLapply,\n parLapplyLB, parRapply, parSapply, parSapplyLB\n\nThe following object is masked from \xe2\x80\x98package:stats\xe2\x80\x99:\n\n xtabs\n\nThe following objects are masked from \xe2\x80\x98package:base\xe2\x8 0\x99:\n\n anyDuplicated, as.data.frame, cbind, colnames, duplicated, eval,\n Filter, Find, get, intersect, lapply, Map, mapply, match, mget,\n order, paste, pmax, pmax.int, pmin, pmin.int, Position, rank,\n rbind, Reduce, rep.int, rownames, sapply, setdiff, sort, table,\n tapply, union, unique, unlist\n\nWelcome to Bioconductor\n\n Vignettes contain introductory material; view with\n \'browseVignettes()\'. To cite Bioconductor, see\n \'citation("Biobase")\', and for packages \'citation("pkgname")\'.\n\nLoading required package: locfit\nlocfit 1.5-9.1 \t 2013-03-22\nLoading required package: lattice\nWarning message:\nIn xy.coords(x, y, xlabel, ylabel, log) :\n 9 y values <= 0 omitted from logarithmic plot\nError in hclustfun(distfun(x)) : \n NA/NaN/Inf in foreign function call (arg 11)\nCalls: heatmap -> hclustfun\nExecution halted\n\n', 116] For cummerbund: Traceback (most recent call last): File "/home/adam/galaxy-dist/lib/galaxy/jobs/runners/local.py", line 109, in queue_job job_wrapper.finish( stdout, stderr, exit_code ) File "/home/adam/galaxy-dist/lib/galaxy/jobs/__init__.py", line 999, in finish self.sa_session.flush() File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/scoping.py", line 127, in do return getattr(self.registry(), name)(*args, **kwargs) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/session.py", line 1356, in flush self._flush(objects) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/session.py", line 1434, in _flush flush_context.execute() File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/unitofwork.py", line 261, in execute UOWExecutor().execute(self, tasks) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/unitofwork.py", line 753, in execute self.execute_save_steps(trans, task) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/unitofwork.py", line 768, in execute_save_steps self.save_objects(trans, task) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/unitofwork.py", line 759, in save_objects task.mapper._save_obj(task.polymorphic_tosave_objects, trans) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/orm/mapper.py", line 1413, in _save_obj c = connection.execute(statement.values(value_params), params) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py", line 824, in execute return Connection.executors[c](self, object, multiparams, params) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py", line 874, in _execute_clauseelement return self.__execute_context(context) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py", line 896, in __execute_context self._cursor_execute(context.cursor, context.statement, context.parameters[0], context=context) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py", line 950, in _cursor_execute self._handle_dbapi_exception(e, statement, parameters, cursor, context) File "/home/adam/galaxy-dist/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/sqlalchemy/engine/base.py", line 931, in _handle_dbapi_exception raise exc.DBAPIError.instance(statement, parameters, e, connection_invalidated=is_disconnect) ProgrammingError: (ProgrammingError) You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings. u'UPDATE job SET update_time=?, stdout=?, stderr=? WHERE job.id = ?' ['2013-05-10 04:38:55.946925', '', 'Fatal error: Exit code 1 (CummerBund Error)\nError running R script. Error in library("cummeRbund") : there is no package called \xe2\x80\x98cummeRbund\xe2\x80\x99\nExecution halted\n\n', 112] It appears that they all fail in a similar fashion. I noticed a similar issue brought up about a year ago but deploying the fixes mentioned then (using a python wrapper to keep R messages out of stderr and telling R to run without curly quotes) has not done anything to stop it. Any ideas would be most appreciated as several R tools are a part of my target RNA-Seq pipeline. Thanks! Adam The materials in this email are private and may contain Protected Health Information. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying, distribution or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return email.

2 1

Re: [galaxy-dev] Display with IGV local
by menorca chaturvedi 13 May '13

13 May '13

Hi I am also facing a similar issue. Did you solve the problem? would be great if I could know how to! Many thanks Menorca

1 0

Column argument validation breaks on empty tabular files
by Peter Cock 10 May '13

10 May '13

Hello all, I'm testing a workflow which uses my filter tool where one of the parameters is a tabular file, and a related parameter is a multi-column selector: Tool: http://toolshed.g2.bx.psu.edu/view/peterjc/seq_filter_by_id Snippet from tool XML, <param name="input_tabular" type="data" format="tabular" label="Tabular file containing sequence identifiers"/> <param name="columns" type="data_column" data_ref="input_tabular" multiple="True" numerical="False" label="Column(s) containing sequence identifiers" help="Multi-select list - hold the appropriate key while clicking to select multiple columns"> <validator type="no_options" message="Pick at least one column"/> </param> In most cases, the preceding tool will generate a tabular file with two columns - and the workflow has column 1 as the saved value. However, it is quite valid for the tabular file to actually be empty - in which case the number of columns is unknown and so the arguments can't be checked. Despite this, Galaxy attempts to check the column argument from the workflow and fails: Traceback (most recent call last): File "/mnt/galaxy/galaxy-central/lib/galaxy/jobs/runners/__init__.py", line 119, in prepare_job job_wrapper.prepare() File "/mnt/galaxy/galaxy-central/lib/galaxy/jobs/__init__.py", line 656, in prepare self.tool.handle_unvalidated_param_values( incoming, self.app ) File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/__init__.py", line 2330, in handle_unvalidated_param_values self.handle_unvalidated_param_values_helper( self.inputs, input_values, app ) File "/mnt/galaxy/galaxy-central/lib/galaxy/tools/__init__.py", line 2369, in handle_unvalidated_param_values_helper raise LateValidationError( message ) LateValidationError Tool execution generated the following error message: Failed runtime validation of Column(s) containing sequence identifiers (An invalid option was selected, please verify) It seems to me this is a bug, and the late validation is being too strict where as in this case the number of columns is undefined. Peter

1 0

Protocol for changing the section of an installed tool shed repo
by Joachim Jacob | VIB | 10 May '13

10 May '13

Hi all, Stupidly enough, I installed a tool shed repo under 'root' of the toolbox. Yes, it was EMBOSS5.0 (btw, installation went flawless!). 1. From the admin menu, I choose to unactivate the repo (manage tool shed repo's) 2. From the admin menu, I reinstalled the tool shed from Main Tool Shed, and chose to put it under a section 'EMBOSS'. This to no avail: all tools still under root. 3. I manually edited the shed_tool_conf.xml and added the section tags 4. The section is now displayed, containing the EMBOSS tools. BUT the tools under the root of the toolbox are still there. Any assistance here? Cheers, Joachim -- Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib

3 3

Something wrong with connecting Galaxy with mysql/postgresql server when uploading datasets
by 王聃Don 10 May '13

10 May '13

Hello, Galaxy developers: I'm a Galaxy user and to customize and deploy Galaxy server locally. I need to build a connection between Galaxy and database server now. MySQL and PostgreSQL are both configured to test the connection. ################################################ #for mysql server and db name is galaxy database_connection=mysql:///galaxy?unix_socket=/var/run/mysqld/mysqld.sock&user=<user_name>&passwd=<password_string> #for postgresql and db name is galaxy database_connection=postgresql:///galaxy?host=/var/run/postgresql&user=postgres&password=<password_string> ################################################ Now starting Galaxy server has no errors and I can create a user account on local Galaxy server successfully. But when uploading datasets into Galaxy(mostly less than 1 MB size) it remains "uploading" status(Dataset is uploading) and cannot complete datasets uploading job. I tested both PostgreSQL and MySQL and it really did not work. Shockingly using PostgreSQL the dataset uploading job can be accomplished very very slowly 3 or 4 times. wangdan

1 0

Missing revisions on (Test) Tool Shed
by Peter Cock 09 May '13

09 May '13

Hi all, There is probably a good reason for this, but why is Galaxy hiding revisions from me on the main 'manage repository' page? e.g. http://testtoolshed.g2.bx.psu.edu/view/peterjc/clinod Here the drop down box only lets me view: 4:07f2d99bc2de (current tip) 0:f886a0b8b117 Why is it hiding revisions 1, 2 and 3? 3:a66a914c39b5 2:3213023deae1 2:3213023deae1 If I try to be sneaky and use a revision specific URL like this, http://testtoolshed.g2.bx.psu.edu/view/peterjc/clinod/a66a914c39b5 then rather than going to the requested revision it jumps to (what is currently) the repository tip. I want to look at these intermediate revisions to see what effect the changes had on the unit tests. Thanks, Peter

2 2

UnicodeDecodeError: 'utf8' codec can't decode byte 0xb9 in position 15: invalid start byte
by Ramon Tiburski 09 May '13

09 May '13

Hi, I uploaded some data and when I click in the eye to see the data I get the following error. I don't know what can I do to fix this problem, I trying install some packges from python but didn't solve my problem. I'm starting think about my operational system, I have ubuntu 12.04 32bit, maybe I need get a 64bit. Someone can help me?Server Error URL: http://localhost:8080/datasets/8c49be448cfe29bc/display/?preview=True Module paste.exceptions.errormiddleware:144 in __call__ >> app_iter = self.application(environ, sr_checker) Module paste.debug.prints:106 in __call__ >> environ, self.app) Module paste.wsgilib:543 in intercept_output >> app_iter = application(environ, replacement_start_response) Module paste.recursive:84 in __call__ >> return self.application(environ, start_response) Module paste.httpexceptions:633 in __call__ >> return self.application(environ, start_response) Module galaxy.web.framework.base:132 in __call__ >> return self.handle_request( environ, start_response ) Module galaxy.web.framework.base:194 in handle_request >> return body( environ, start_response ) Module galaxy.web.framework:876 in render >> template.render_context( context ) Module mako.template:319 in render_context >> **kwargs) Module mako.runtime:692 in _render_context >> _exec_template(inherit, lclcontext, args=args, kwargs=kwargs) Module mako.runtime:718 in _exec_template >> callable_(context, *args, **kwargs) Module _base_mako:42 in render_body >> __M_writer(unicode(next.body())) Module _dataset_large_file_mako:45 in render_body >> __M_writer(filters.html_escape(unicode( unicode( truncated_data, 'utf-8' ) ))) UnicodeDecodeError: 'utf8' codec can't decode byte 0xb9 in position 15: invalid start byte

2 4