January 2012 - galaxy-dev - lists.galaxyproject.org

How to manage stderr output from R?
by Carlos Borroto 30 Jan '12

30 Jan '12

Hi all, I'm working on integrating cummeRbund[1] into Galaxy. This is a very helpful tool for analysis and visualization of cuffdiff output. My problem is any text output from running R code, puts the galaxy output in error state. Even if everything is working correctly and the outputs are generated as intended. I tried this code from 'tools/plotting/xy_plot.xml': ## Setup R error handling to go to stderr options( show.error.messages=F, error = function () { cat( geterrmessage(), file=stderr() ); q( "no", 1, F ) } ) But it didn't help. Even for loading the library I had to go with: suppressPackageStartupMessages(library("cummeRbund")) Nothing else would work. Now the tool is outputting some messages while running R functions, which is putting the output in galaxy in error state. I'm thinking about asking the upstream developer to provide a way to run the functions in the library in 'quite' mode. Is there any other option? Right now I'm using the path of creating a R script in the XML file and using r_wrapper.sh to run it. I tried first with rpy, as it would give me more freedom, but cummeRbund use ggplot2 which seems to be supported only on rpy2. [1]http://compbio.mit.edu/cummeRbund/ Thanks in advance, Carlos

2 3

Re: [galaxy-dev] shed_tool_conf.xml not updated
by Greg Von Kuster 30 Jan '12

30 Jan '12

Anne, Thanks for all of your patience and help with this. You uncovered a problem that is very nice to have corrected. The fix is included in change set 6634:ebf72cda61e7 on our central repository. This change set will allow you to have section tags in your tool_conf.xml file that do not include version attributes. Selecting these sections to contain tools installed with a tool shed repository will now be functionally correct. Thanks again! Greg On Jan 30, 2012, at 12:08 PM, Anne Pajon wrote: > Brilliant! It is working - no more server error and I now have a shed_tool_conf.xml updated: > > Let me know when you commit your next change set, so I will be able to test. > Thanks a lot for your help. > Anne. > Greg Von Kuster Galaxy Development Team greg(a)bx.psu.edu

2 1

Problem running purge_datasets.sh cleanup scripts
by Leandro Hermida 30 Jan '12

30 Jan '12

Hello, In the latest galaxy-dist version when running the system cleanup commands in the documented order I get an error when running the purge_datasets.sh script: sh delete_userless_histories.sh sh purge_histories.sh sh purge_libraries.sh sh purge_folders.sh sh delete_datasets.sh sh purge_datasets.sh Traceback (most recent call last): File "./scripts/cleanup_datasets/cleanup_datasets.py", line 518, in <module> if __name__ == "__main__": main() File "./scripts/cleanup_datasets/cleanup_datasets.py", line 116, in main purge_datasets( app, cutoff_time, options.remove_from_disk, info_only = options.info_only, force_retry = options.force_retry ) File "./scripts/cleanup_datasets/cleanup_datasets.py", line 353, in purge_datasets _purge_dataset( app, dataset, remove_from_disk, info_only = info_only ) File "./scripts/cleanup_datasets/cleanup_datasets.py", line 478, in _purge_dataset print "Error attempting to purge data file: ", dataset.file_name, " error: ", str( exc ) File "/gpfshpc/software/pmrdcdb/galaxy/galaxy_dist_dev/lib/galaxy/model/__init__.py", line 651, in get_file_name assert self.object_store is not None, "Object Store has not been initialized for dataset %s" % self.id AssertionError: Object Store has not been initialized for dataset 58 What could be going wrong? regards, Leandro

2 1

samtools in local instance of galaxy
by Praveen Baskaran 30 Jan '12

30 Jan '12

i have installed local instance of galaxy and have a small problem with samtools. when i try to convert sam file to bam, the result pane shows that the job is running (for several hours). but when check the galaxy log file, it shows that the job is finished normally. i can also find the result (bam) file in galaxy/database/files/000/ folder. can anyone help to resolve this issue. Thanks Praveen

2 1

Some eggs are out of date...Fetch failed (Mac OS X 10.6)
by Lucilla Luzi 30 Jan '12

30 Jan '12

Hi, I'm pretty new to Galaxy, at least to this side. I'm sorry to bother you with my issue, that, I read, was quite extensively discussed previously, but even if I think to have followed correctly the instruction and further suggestions, I'm still not able to overcome the problem. I'm trying to install a local instance of Galaxy on my MAC (OS X10.6.8) My Python version is 2.6.1 and I just installed Mercurial 2.0.2 for MacOS X 10.6. FYI $ python --version Python 2.6.1 $ echo $PATH /Users/myName/System/Library/Frameworks/Python.framework/Versions/2.6/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin When I try to run the run.sh I get the error attached below. Can you envisage a possible reason why I'm getting this error? Sorry for the naive admission, but from the error message I'm not even capable to understand which are the "missing eggs"... If I'd have ignored a previous post with the solution, I apologize in advance. Thank you a lot for your attention. Lu ------------------ $ sh run.sh Some eggs are out of date, attempting to fetch... Traceback (most recent call last): File "./scripts/fetch_eggs.py", line 30, in <module> c.resolve() # Only fetch eggs required by the config File "/Users/myName/My_programs/Galaxy/galaxy-dist/lib/galaxy/eggs/__init__.py", line 345, in resolve egg.resolve() File "/Users/myName/My_programs/Galaxy/galaxy-dist/lib/galaxy/eggs/__init__.py", line 195, in resolve return self.version_conflict( e.args[0], e.args[1] ) File "/Users/myName/My_programs/Galaxy/galaxy-dist/lib/galaxy/eggs/__init__.py", line 226, in version_conflict r = pkg_resources.working_set.resolve( ( dist.as_requirement(), ), env, egg.fetch ) File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/pkg_resources.py", line 518, in resolve raise DistributionNotFound(req) # XXX put more info here pkg_resources.DistributionNotFound: numpy==1.6.0 Fetch failed.

2 3

some basic cluster dev questions
by Jaime Frey 30 Jan '12

30 Jan '12

Hello, everyone. I'm working on a Condor job-runner module for Galaxy that can provide better integration than the existing drmaa module. I'm sure I'll have many questions as I work on this module, but I have just a few to start with: * What versions of Python does Galaxy support (i.e. which newer features of Python do I need to avoid)? * What is the status of the 'staged' cluster deployment? The wiki says it's not in active use or development. But our Condor cluster doesn't have a shared filesystem, so we need to stage data files to/from the execute nodes. * The wiki also says that even with the staged cluster deployment, a shared directory (galaxy_dist/database/tmp by default) is required for some tools. These tools write temporary files on the Galaxy server and read them on the execute node. Does anyone know which tools those are? Are these files ever written after the job is handed to the batch system? Do these tools write files into galaxy_dist/database/tmp on the execute ndoes and read the files on the Galaxy server? Thanks and regards, Jaime Frey UW-Madison Condor Team

2 1

postgres user change problem
by Holger Klein 30 Jan '12

30 Jan '12

Dear all, due to some work on the user management of our servers we had to rename the user galaxy runs as. Up to now we used ident as postgres authentication method, meaning that here postgres expects unix username "galaxy" to have permissions of "galaxy" postgres user. The entry in universe_wsgi.ini is simply: database_connection = postgres:///galaxy?host=/var/run/postgresql After the renaming, the new user "galaxynew" didn't get access at all at first. Now I tried two things: - adding a user "galaxynew" to postgres with permissions for database galaxy and - dumping the contents of database "galaxy" into a file and re-reading this into database "galaxynew", which is owned by the user "galaxynew" and of course changing the config file line to database_connection = postgres:///galaxynew?host=/var/run/postgresql In both cases I end up with an error during startup of galaxy: [...] WARNING 2012-01-20 14:46:55,556 Error closing cursor: current transaction is aborted, commands ignored until end of transaction block [...] ProgrammingError: (ProgrammingError) permission denied for relation migrate_version 'SELECT migrate_version.repository_id, migrate_version.repository_path, migrate_version.version \nFROM migrate_version \nWHERE migrate_version.repository_id = %(repository_id_1)s' {'repository_id_1': 'Galaxy'} (complete traceback below). Does anyone have a hint on how I could fix this? Cheers, Holger sqlalchemy.pool.QueuePool.0x...8d10 WARNING 2012-01-20 14:46:55,556 Error closing cursor: current transaction is aborted, commands ignored until end of transaction block Traceback (most recent call last): File "/local/data/home/galaxy/galaxy-dist-2011-11-23/lib/galaxy/web/buildapp.py", line 82, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/local/data/home/galaxy/galaxy-dist-2011-11-23/lib/galaxy/app.py", line 39, in __init__ create_or_verify_database( db_url, kwargs.get( 'global_conf', {} ).get( '__file__', None ), self.config.database_engine_options ) File "/local/data/home/galaxy/galaxy-dist-2011-11-23/lib/galaxy/model/migrate/check.py", line 99, in create_or_verify_database db_schema = schema.ControlledSchema( engine, migrate_repository ) File "/local/data/home/galaxy/galaxy-dist-2011-11-23/eggs/sqlalchemy_migrate-0.5.4-py2.6.egg/migrate/versioning/schema.py", line 24, in __init__ self._load() File "/local/data/home/galaxy/galaxy-dist-2011-11-23/eggs/sqlalchemy_migrate-0.5.4-py2.6.egg/migrate/versioning/schema.py", line 41, in _load self.table.c.repository_id == str(self.repository.id))) File "/local/data/home/galaxy/galaxy-dist-2011-11-23/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 1202, in execute return connection.execute(statement, *multiparams, **params) File "/local/data/home/galaxy/galaxy-dist-2011-11-23/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 824, in execute return Connection.executors[c](self, object, multiparams, params) File "/local/data/home/galaxy/galaxy-dist-2011-11-23/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 874, in _execute_clauseelement return self.__execute_context(context) File "/local/data/home/galaxy/galaxy-dist-2011-11-23/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 896, in __execute_context self._cursor_execute(context.cursor, context.statement, context.parameters[0], context=context) File "/local/data/home/galaxy/galaxy-dist-2011-11-23/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 950, in _cursor_execute self._handle_dbapi_exception(e, statement, parameters, cursor, context) File "/local/data/home/galaxy/galaxy-dist-2011-11-23/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 931, in _handle_dbapi_exception raise exc.DBAPIError.instance(statement, parameters, e, connection_invalidated=is_disconnect) ProgrammingError: (ProgrammingError) permission denied for relation migrate_version 'SELECT migrate_version.repository_id, migrate_version.repository_path, migrate_version.version \nFROM migrate_version \nWHERE migrate_version.repository_id = %(repository_id_1)s' {'repository_id_1': 'Galaxy'} -- Dr. Holger Klein Core Facility Bioinformatics Institute of Molecular Biology gGmbH (IMB) http://www.imb-mainz.de/ Tel: +49(6131) 39 21511

2 1

server error when installing tool from tool shed repository into local instance
by Anne Pajon 30 Jan '12

30 Jan '12

Hello, I have the version 6528:63bc46cc73b7 of galaxy running locally on our server installed from https://bitbucket.org/galaxy/galaxy-dist/. I've tried to bring tabular2html tool from Galaxy main tool shed repository following the wiki information (http://wiki.g2.bx.psu.edu/Tool%20Shed#Automatic_installation_of_Galaxy_tool…) without success. Here is what I've done. (1) I've edited my universe_wsgi.ini # -- Files and directories # Tool config files, defines what tools are available in Galaxy. # Tools can be locally developed or installed from Galaxy tool sheds. tool_config_file = tool_conf.xml,shed_tool_conf.xml # Default path to the directory containing the tools defined in tool_conf.xml. # Other tool config files must include the tool_path as an attribute in the <toolbox> tag. tool_path = tools (2) I've created the directory shed_tools/ above the galaxy-dist/ as recommended (3) I've restarted the server (4) I've logged in as "admin" user. I've clicked the "Admin" link in the top Galaxy tool panel to display the Galaxy Administration interface and then "Search and browse tool sheds" under "Tool sheds" from the left blue section to display the Accessible Galaxy tool sheds page, click "Galaxy main tool shed" to display the valid repositories and choose tabular2html and click on Install to local Galaxy, select the panel section and click Install. I get a page with "Server Error". Strangely, the tool appears in the list of tools and works in my Galaxy instance. The tool files have been installed on disk at the directory created in (2) but the shed_tool_conf.xml has not been updated. After restarting my Galaxy server, the tool is not visible anymore. Here is what appears in the log related to the installation of the tool: galaxy.util.shed_util DEBUG 2012-01-25 14:20:54,451 Installing repository 'tabular2html' galaxy.util.shed_util DEBUG 2012-01-25 14:20:54,451 Cloning http://toolshed.g2.bx.psu.edu/repos/ondovb/tabular2html galaxy.util.shed_util DEBUG 2012-01-25 14:20:55,486 Updating cloned repository to revision "a5814dd5a11a" galaxy.tools DEBUG 2012-01-25 14:20:55,652 Reloading section: My Tools galaxy.tools DEBUG 2012-01-25 14:20:55,673 Loaded tool id: toolshed.g2.bx.psu.edu/repos/ondovb/tabular2html/tabular2HTML/1.0.0, version: 1.0 .0. 143.65.172.178 - - [25/Jan/2012:14:20:54 +0100] "POST /galaxy/admin_toolshed/install_repository?tool_shed_url=http%3A%2F%2Ftoolshed.g2.bx.ps u.edu&repo_info_dict=3abd29129982504350fb737085131cc07e47cc03%3A7b22746162756c61723268746d6c223a205b22437265617465732048544d4c207461626c6573 2066726f6d20746162756c617220646174612e222c2022687474703a2f2f746f6f6c736865642e67322e62782e7073752e6564752f7265706f732f6f6e646f76622f74616275 6c61723268746d6c222c2022613538313464643561313161225d7d&includes_tools=True HTTP/1.1" 500 - "http://uk-cri-lbio08/galaxy/admin_toolshed/insta ll_repository?tool_shed_url=http://toolshed.g2.bx.psu.edu&repo_info_dict=3abd29129982504350fb737085131cc07e47cc03:7b22746162756c61723268746d 6c223a205b22437265617465732048544d4c207461626c65732066726f6d20746162756c617220646174612e222c2022687474703a2f2f746f6f6c736865642e67322e62782e 7073752e6564752f7265706f732f6f6e646f76622f746162756c61723268746d6c222c2022613538313464643561313161225d7d&includes_tools=True" "Mozilla/5.0 ( Macintosh; U; Intel Mac OS X 10.6; en-GB; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13" Error - <type 'exceptions.TypeError'>: cannot serialize None (type NoneType) URL: http://uk-cri-lbio08/galaxy/admin_toolshed/install_repository?tool_shed_url… 9982504350fb737085131cc07e47cc03%3A7b22746162756c61723268746d6c223a205b22437265617465732048544d4c207461626c65732066726f6d20746162756c6172206 46174612e222c2022687474703a2f2f746f6f6c736865642e67322e62782e7073752e6564752f7265706f732f6f6e646f76622f746162756c61723268746d6c222c202261353 8313464643561313161225d7d&includes_tools=True File '/opt/local/home/webapp/galaxy/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/exceptions/errormiddleware.py', line 143 in __call__ app_iter = self.application(environ, start_response) File '/opt/local/home/webapp/galaxy/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/opt/local/home/webapp/galaxy/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/opt/local/home/webapp/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 160 in __call__ body = method( trans, **kwargs ) File '/opt/local/home/webapp/galaxy/galaxy-dist/lib/galaxy/web/framework/__init__.py', line 173 in decorator return func( self, trans, *args, **kwargs ) File '/opt/local/home/webapp/galaxy/galaxy-dist/lib/galaxy/web/controllers/admin_toolshed.py', line 275 in install_repository new_install=True ) File '/opt/local/home/webapp/galaxy/galaxy-dist/lib/galaxy/util/shed_util.py', line 644 in load_repository_contents add_to_config=True ) File '/opt/local/home/webapp/galaxy/galaxy-dist/lib/galaxy/util/shed_util.py', line 699 in load_altered_part_of_tool_panel add_shed_tool_conf_entry( app, shed_tool_conf, elem_entry ) File '/opt/local/home/webapp/galaxy/galaxy-dist/lib/galaxy/util/shed_util.py', line 41 in add_shed_tool_conf_entry new_shed_tool_conf.write( util.xml_to_string( tool_panel_entry, pretty=True ) ) File '/opt/local/home/webapp/galaxy/galaxy-dist/lib/galaxy/util/__init__.py', line 113 in xml_to_string return ElementTree.tostring( pretty_print_xml( elem ) ) File '/opt/local/home/webapp/galaxy/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py', line 1006 in tostring File '/opt/local/home/webapp/galaxy/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py', line 660 in write File '/opt/local/home/webapp/galaxy/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py', line 695 in _write File '/opt/local/home/webapp/galaxy/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py', line 827 in _escape_a ttrib File '/opt/local/home/webapp/galaxy/galaxy-dist/eggs/elementtree-1.2.6_20050316-py2.7.egg/elementtree/ElementTree.py', line 774 in _raise_se rialization_error TypeError: cannot serialize None (type NoneType) Any help would be very welcome. Thanks a lot in advance. Kind regards, Anne. -- Anne Pajon, Ph.D. Cancer Research UK - Cambridge Research Institute Li Ka Shing Centre, Robinson Way, Cambridge CB2 0RE anne.pajon(a)cancer.org.uk | +44 (0)7958 511 353 NOTICE AND DISCLAIMER This e-mail (including any attachments) is intended for the above-named person(s). If you are not the intended recipient, notify the sender immediately, delete this email from your system and do not disclose or use for any purpose. We may monitor all incoming and outgoing emails in line with current legislation. We have taken steps to ensure that this email and attachments are free from any virus, but it remains your responsibility to ensure that viruses do not adversely affect you. Cancer Research UK Registered in England and Wales Company Registered Number: 4325234. Registered Charity Number: 1089464 and Scotland SC041666 Registered Office Address: Angel Building, 407 St John Street, London EC1V 4AD.

2 13

Error downloading output file from Galaxy
by Cory Spencer 30 Jan '12

30 Jan '12

Hello all - I'm working on a tool for Galaxy that generates an output file. I am able to view the file without issue in Galaxy itself, but when I click on the "Download" icon, the History sidebar produces the following error message: "Unable to remove temporary library download archive and directory" Checking the log files, I see the following entry: galaxy.web.controllers.dataset ERROR 2012-01-17 23:14:05,599 Unable to remove temporary library download archive and directory Traceback (most recent call last): File "/n/galaxy/www/galaxy/lib/galaxy/web/controllers/dataset.py", line 284, in archive_composite_dataset os.rmdir( tmpd ) OSError: [Errno 39] Directory not empty: './database/tmp/tmprofQCp' Listing the contents on the referenced directory shows it to be empty: $ ls -la database/tmp/tmprofQCp total 16 drwx------ 2 galaxy galaxy_lab 1024 2012-01-17 23:14 . drwxrwx--- 8 galaxy galaxy_lab 1024 2012-01-17 23:14 .. Does anyone have any ideas as to what might be happening? Thank you! Cory

2 1

How to use scripts/api/update.py?
by Carlos Borroto 30 Jan '12

30 Jan '12

Hi, I'm trying to use the update.py script and is not clear for me what to use for 'url' in the usage line: usage: create.py key url [key=value ...]. Which I'm guessing is just a carry over from create.py. I tried passing the direct link to the dataset: $ ./update.py <my_key> http://localhost:8080/api/histories/0397e7c5778be5ee/contents/45d854c903d27… name=foobar HTTP Error 404: Not Found 404 Not Found The resource could not be found. No action for /api/histories/0397e7c5778be5ee/contents/45d854c903d27dcf Just to show you this dataset does exists: $ ./display.py <my_key> http://localhost:8080/api/histories/0397e7c5778be5ee/contents/45d854c903d27… Member Information ------------------ misc_blurb: 4.3 Gb name: BMD01.GAIIx.1.fastq data_type: fastqillumina deleted: False state: ok download_url: /datasets/45d854c903d27dcf/display?to_ext=fastqillumina visible: True genome_build: ? model_class: HistoryDatasetAssociation metadata_sequences: None file_size: 4644313368 metadata_data_lines: None id: 45d854c903d27dcf misc_info: uploaded fastq file metadata_dbkey: ? Thanks, Carlos

2 2