August 2009 - galaxy-dev - lists.galaxyproject.org

[hg] galaxy 2635: Updating version of Beaker egg (needed by Mako...
by Greg Von Kuster 29 Aug '09

29 Aug '09

details: http://www.bx.psu.edu/hg/galaxy/rev/3df00582ca44 changeset: 2635:3df00582ca44 user: James Taylor <james(a)jamestaylor.org> date: Fri Aug 28 15:06:41 2009 -0400 description: Updating version of Beaker egg (needed by Mako, no longer used by Galaxy directly) 1 file(s) affected in this change: eggs.ini diffs (21 lines): diff -r e87013fd0716 -r 3df00582ca44 eggs.ini --- a/eggs.ini Fri Aug 28 14:53:51 2009 -0400 +++ b/eggs.ini Fri Aug 28 15:06:41 2009 -0400 @@ -26,7 +26,7 @@ numpy = 1.2.1 [eggs:noplatform] -Beaker = 0.5 +Beaker = 1.4 docutils = 0.4 elementtree = 1.2.6_20050316 flup = 0.5 @@ -76,7 +76,7 @@ python_lzo = http://www.oberhumer.com/opensource/lzo/download/LZO-v1/python-lzo-1.08.tar… http://www.oberhumer.com/opensource/lzo/download/LZO-v1/lzo-1.08.tar.gz threadframe = http://www.majid.info/python/threadframe/threadframe-0.2.tar.gz guppy = http://pypi.python.org/packages/source/g/guppy/guppy-0.1.8.tar.gz -Beaker = http://cheeseshop.python.org/packages/source/B/Beaker/Beaker-0.5.tar.gz +Beaker = http://cheeseshop.python.org/packages/source/B/Beaker/Beaker-1.4.tar.gz docutils = http://downloads.sourceforge.net/docutils/docutils-0.4.tar.gz elementtree = http://effbot.org/downloads/elementtree-1.2.6-20050316.tar.gz flup = http://www.saddi.com/software/flup/dist/archive/flup-r2311.tar.gz

1 0

[hg] galaxy 2636: Merge
by Greg Von Kuster 29 Aug '09

29 Aug '09

details: http://www.bx.psu.edu/hg/galaxy/rev/b7a3a82b71ce changeset: 2636:b7a3a82b71ce user: James Taylor <james(a)jamestaylor.org> date: Fri Aug 28 15:07:48 2009 -0400 description: Merge 0 file(s) affected in this change: diffs (44 lines): diff -r 3df00582ca44 -r b7a3a82b71ce lib/galaxy/datatypes/images.py --- a/lib/galaxy/datatypes/images.py Fri Aug 28 15:06:41 2009 -0400 +++ b/lib/galaxy/datatypes/images.py Fri Aug 28 15:07:48 2009 -0400 @@ -9,6 +9,7 @@ from galaxy.datatypes.sniff import * from urllib import urlencode, quote_plus import zipfile +import os, subprocess, tempfile log = logging.getLogger(__name__) @@ -240,6 +241,26 @@ """Class describing a BAM binary file""" file_ext = "bam" MetadataElement( name="bam_index", desc="BAM Index File", param=metadata.FileParameter, readonly=True, no_value=None, visible=False, optional=True ) + def init_meta( self, dataset, copy_from=None ): + data.Binary.init_meta( self, dataset, copy_from=copy_from ) + def set_meta( self, dataset, overwrite = True, **kwd ): + """ + Sets index for BAM file. + """ + index_file = dataset.metadata.bam_index + if not index_file: + index_file = dataset.metadata.spec['bam_index'].param.new_file( dataset = dataset ) + tmp_dir = tempfile.gettempdir() + tmpf1 = tempfile.NamedTemporaryFile(dir=tmp_dir) + try: + subprocess.check_call(['cd', tmp_dir], shell=True) + subprocess.check_call('cp %s %s' % (dataset.file_name, tmpf1.name), shell=True) + subprocess.check_call('samtools index %s' % tmpf1.name, shell=True) + subprocess.check_call('cp %s.bai %s' % (tmpf1.name, index_file.file_name), shell=True) + except subprocess.CalledProcessError: + sys.stderr.write('There was a problem creating the index for the BAM file\n') + tmpf1.close() + dataset.metadata.bam_index = index_file def set_peek( self, dataset ): if not dataset.dataset.purged: export_url = "/history_add_to?" + urlencode({'history_id':dataset.history_id,'ext':'bam','name':'bam alignments','info':'Alignments file','dbkey':dataset.dbkey}) @@ -256,4 +277,3 @@ def get_mime(self): """Returns the mime type of the datatype""" return 'application/octet-stream' - \ No newline at end of file

1 0

[hg] galaxy 2638: One more flup reference
by Greg Von Kuster 29 Aug '09

29 Aug '09

details: http://www.bx.psu.edu/hg/galaxy/rev/d261f41a2a03 changeset: 2638:d261f41a2a03 user: James Taylor <james(a)jamestaylor.org> date: Fri Aug 28 15:29:53 2009 -0400 description: One more flup reference 1 file(s) affected in this change: lib/galaxy/web/framework/base.py diffs (11 lines): diff -r 0fd76a02c2b2 -r d261f41a2a03 lib/galaxy/web/framework/base.py --- a/lib/galaxy/web/framework/base.py Fri Aug 28 15:12:55 2009 -0400 +++ b/lib/galaxy/web/framework/base.py Fri Aug 28 15:29:53 2009 -0400 @@ -13,7 +13,6 @@ import pkg_resources; pkg_resources.require( "Paste" ) pkg_resources.require( "Routes" ) -pkg_resources.require( "flup" ) pkg_resources.require( "WebOb" ) import routes

1 0

[hg] galaxy 2637: Purge references to Beaker and flup from buildapp
by Greg Von Kuster 29 Aug '09

29 Aug '09

details: http://www.bx.psu.edu/hg/galaxy/rev/0fd76a02c2b2 changeset: 2637:0fd76a02c2b2 user: James Taylor <james(a)jamestaylor.org> date: Fri Aug 28 15:12:55 2009 -0400 description: Purge references to Beaker and flup from buildapp 2 file(s) affected in this change: eggs.ini lib/galaxy/web/buildapp.py diffs (48 lines): diff -r b7a3a82b71ce -r 0fd76a02c2b2 eggs.ini --- a/eggs.ini Fri Aug 28 15:07:48 2009 -0400 +++ b/eggs.ini Fri Aug 28 15:12:55 2009 -0400 @@ -29,7 +29,6 @@ Beaker = 1.4 docutils = 0.4 elementtree = 1.2.6_20050316 -flup = 0.5 lrucache = 0.2 ;lsprof - james Mako = 0.2.4 @@ -79,7 +78,6 @@ Beaker = http://cheeseshop.python.org/packages/source/B/Beaker/Beaker-1.4.tar.gz docutils = http://downloads.sourceforge.net/docutils/docutils-0.4.tar.gz elementtree = http://effbot.org/downloads/elementtree-1.2.6-20050316.tar.gz -flup = http://www.saddi.com/software/flup/dist/archive/flup-r2311.tar.gz lrucache = http://evan.prodromou.name/lrucache/lrucache-0.2.tar.gz Mako = http://www.makotemplates.org/downloads/Mako-0.2.4.tar.gz MyghtyUtils = http://cheeseshop.python.org/packages/source/M/MyghtyUtils/MyghtyUtils-0.52… diff -r b7a3a82b71ce -r 0fd76a02c2b2 lib/galaxy/web/buildapp.py --- a/lib/galaxy/web/buildapp.py Fri Aug 28 15:07:48 2009 -0400 +++ b/lib/galaxy/web/buildapp.py Fri Aug 28 15:12:55 2009 -0400 @@ -11,7 +11,6 @@ from paste.util import import_string from paste import httpexceptions from paste.deploy.converters import asbool -import flup.middleware.session as flup_session import pkg_resources log = logging.getLogger( __name__ ) @@ -116,17 +115,6 @@ from paste import recursive app = recursive.RecursiveMiddleware( app, conf ) log.debug( "Enabling 'recursive' middleware" ) - ## # Session middleware puts a session factory into the environment - ## if asbool( conf.get( 'use_session', True ) ): - ## store = flup_session.MemorySessionStore() - ## app = flup_session.SessionMiddleware( store, app ) - ## log.debug( "Enabling 'flup session' middleware" ) - # Beaker session middleware - if asbool( conf.get( 'use_beaker_session', False ) ): - pkg_resources.require( "Beaker" ) - import beaker.session - app = beaker.session.SessionMiddleware( app, conf ) - log.debug( "Enabling 'beaker session' middleware" ) # Various debug middleware that can only be turned on if the debug # flag is set, either because they are insecure or greatly hurt # performance

1 0

[hg] galaxy 2629: Updated the Bam datatype so that the file is i...
by Greg Von Kuster 29 Aug '09

29 Aug '09

details: http://www.bx.psu.edu/hg/galaxy/rev/d8240234bd10 changeset: 2629:d8240234bd10 user: Kelly Vincent <kpvincent(a)bx.psu.edu> date: Thu Aug 27 10:04:01 2009 -0400 description: Updated the Bam datatype so that the file is indexed and the index is stored as metadata 1 file(s) affected in this change: lib/galaxy/datatypes/images.py diffs (44 lines): diff -r d99cee3a4efd -r d8240234bd10 lib/galaxy/datatypes/images.py --- a/lib/galaxy/datatypes/images.py Wed Aug 26 07:12:04 2009 -0400 +++ b/lib/galaxy/datatypes/images.py Thu Aug 27 10:04:01 2009 -0400 @@ -9,6 +9,7 @@ from galaxy.datatypes.sniff import * from urllib import urlencode, quote_plus import zipfile +import os, subprocess, tempfile log = logging.getLogger(__name__) @@ -240,6 +241,26 @@ """Class describing a BAM binary file""" file_ext = "bam" MetadataElement( name="bam_index", desc="BAM Index File", param=metadata.FileParameter, readonly=True, no_value=None, visible=False, optional=True ) + def init_meta( self, dataset, copy_from=None ): + data.Binary.init_meta( self, dataset, copy_from=copy_from ) + def set_meta( self, dataset, overwrite = True, **kwd ): + """ + Sets index for BAM file. + """ + index_file = dataset.metadata.bam_index + if not index_file: + index_file = dataset.metadata.spec['bam_index'].param.new_file( dataset = dataset ) + tmp_dir = tempfile.gettempdir() + tmpf1 = tempfile.NamedTemporaryFile(dir=tmp_dir) + try: + subprocess.check_call(['cd', tmp_dir], shell=True) + subprocess.check_call('cp %s %s' % (dataset.file_name, tmpf1.name), shell=True) + subprocess.check_call('samtools index %s' % tmpf1.name, shell=True) + subprocess.check_call('cp %s.bai %s' % (tmpf1.name, index_file.file_name), shell=True) + except subprocess.CalledProcessError: + sys.stderr.write('There was a problem creating the index for the BAM file\n') + tmpf1.close() + dataset.metadata.bam_index = index_file def set_peek( self, dataset ): if not dataset.dataset.purged: export_url = "/history_add_to?" + urlencode({'history_id':dataset.history_id,'ext':'bam','name':'bam alignments','info':'Alignments file','dbkey':dataset.dbkey}) @@ -256,4 +277,3 @@ def get_mime(self): """Returns the mime type of the datatype""" return 'application/octet-stream' - \ No newline at end of file

1 0

[hg] galaxy 2631: Image files for tagging UI
by Greg Von Kuster 29 Aug '09

29 Aug '09

details: http://www.bx.psu.edu/hg/galaxy/rev/4b4bac1f66bd changeset: 2631:4b4bac1f66bd user: jeremy goecks <jeremy.goecks at emory.edu> date: Thu Aug 27 16:27:10 2009 -0400 description: Image files for tagging UI 4 file(s) affected in this change: static/images/add_icon.png static/images/add_icon_dark.png static/images/delete_tag_icon_gray.png static/images/delete_tag_icon_white.png diffs (8 lines): diff -r 3c48856d80eb -r 4b4bac1f66bd static/images/add_icon.png Binary file static/images/add_icon.png has changed diff -r 3c48856d80eb -r 4b4bac1f66bd static/images/add_icon_dark.png Binary file static/images/add_icon_dark.png has changed diff -r 3c48856d80eb -r 4b4bac1f66bd static/images/delete_tag_icon_gray.png Binary file static/images/delete_tag_icon_gray.png has changed diff -r 3c48856d80eb -r 4b4bac1f66bd static/images/delete_tag_icon_white.png Binary file static/images/delete_tag_icon_white.png has changed

1 0

[hg] galaxy 2630: Add tables to support tagging of histories, da...
by Greg Von Kuster 29 Aug '09

29 Aug '09

details: http://www.bx.psu.edu/hg/galaxy/rev/3c48856d80eb changeset: 2630:3c48856d80eb user: jeremy goecks <jeremy.goecks at emory.edu> date: Thu Aug 27 16:10:51 2009 -0400 description: Add tables to support tagging of histories, datasets, ands HDAs. 1 file(s) affected in this change: lib/galaxy/model/migrate/versions/0015_tagging.py diffs (61 lines): diff -r d99cee3a4efd -r 3c48856d80eb lib/galaxy/model/migrate/versions/0015_tagging.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/lib/galaxy/model/migrate/versions/0015_tagging.py Thu Aug 27 16:10:51 2009 -0400 @@ -0,0 +1,56 @@ +from sqlalchemy import * +from migrate import * + +import datetime +now = datetime.datetime.utcnow + +# Need our custom types, but don't import anything else from model +from galaxy.model.custom_types import * + +import logging +log = logging.getLogger( __name__ ) + +metadata = MetaData( migrate_engine ) + +# New tables to support tagging of histories, datasets, and history-dataset associations. +Tag_table = Table( "tag", metadata, + Column( "id", Integer, primary_key=True ), + Column( "type", Integer ), + Column( "parent_id", Integer, ForeignKey( "tag.id" ) ), + Column( "name", TrimmedString(255) ), + UniqueConstraint( "name" ) ) + +HistoryTagAssociation_table = Table( "history_tag_association", metadata, + Column( "history_id", Integer, ForeignKey( "history.id" ), index=True ), + Column( "tag_id", Integer, ForeignKey( "tag.id" ), index=True ), + Column( "user_tname", TrimmedString(255), index=True), + Column( "value", TrimmedString(255), index=True), + Column( "user_value", TrimmedString(255), index=True) ) + +DatasetTagAssociation_table = Table( "dataset_tag_association", metadata, + Column( "dataset_id", Integer, ForeignKey( "dataset.id" ), index=True ), + Column( "tag_id", Integer, ForeignKey( "tag.id" ), index=True ), + Column( "user_tname", TrimmedString(255), index=True), + Column( "value", TrimmedString(255), index=True), + Column( "user_value", TrimmedString(255), index=True) ) + +HistoryDatasetAssociationTagAssociation_table = Table( "history_dataset_association_tag_association", metadata, + Column( "history_dataset_association_id", Integer, ForeignKey( "history_dataset_association.id" ), index=True ), + Column( "tag_id", Integer, ForeignKey( "tag.id" ), index=True ), + Column( "user_tname", TrimmedString(255), index=True), + Column( "value", TrimmedString(255), index=True), + Column( "user_value", TrimmedString(255), index=True) ) + +def upgrade(): + metadata.reflect() + Tag_table.create() + HistoryTagAssociation_table.create() + DatasetTagAssociation_table.create() + HistoryDatasetAssociationTagAssociation_table.create() + +def downgrade(): + metadata.reflect() + Tag_table.drop() + HistoryTagAssociation_table.drop() + DatasetTagAssociation_table.drop() + HistoryDatasetAssociationTagAssociation_table.drop() \ No newline at end of file

1 0

[hg] galaxy 2632: Added documentation and exception handling for...
by Greg Von Kuster 29 Aug '09

29 Aug '09

details: http://www.bx.psu.edu/hg/galaxy/rev/31150f0b216d changeset: 2632:31150f0b216d user: jeremy goecks <jeremy.goecks(a)emory.edu> date: Fri Aug 28 12:52:43 2009 -0400 description: Added documentation and exception handling for mysql index error. 1 file(s) affected in this change: lib/galaxy/model/migrate/versions/0015_tagging.py diffs (98 lines): diff -r 4b4bac1f66bd -r 31150f0b216d lib/galaxy/model/migrate/versions/0015_tagging.py --- a/lib/galaxy/model/migrate/versions/0015_tagging.py Thu Aug 27 16:27:10 2009 -0400 +++ b/lib/galaxy/model/migrate/versions/0015_tagging.py Fri Aug 28 12:52:43 2009 -0400 @@ -1,3 +1,15 @@ +""" +This migration script adds the tables necessary to support tagging of histories, +datasets, and history-dataset associations (user views of datasets). + +If using mysql, this script will display the following error, which is corrected in the next +migration script: + +history_dataset_association_tag_association table failed: (OperationalError) +(1059, "Identifier name 'ix_history_dataset_association_tag_association_history_dataset_association_id' +is too long) +""" + from sqlalchemy import * from migrate import * @@ -11,6 +23,19 @@ log = logging.getLogger( __name__ ) metadata = MetaData( migrate_engine ) + +def display_migration_details(): + print "" + print "This migration script adds the tables necessary to support tagging of histories," + print "datasets, and history-dataset associations (user views of datasets)." + print "" + print "If using mysql, this script will display the following error, which is " + print "corrected in the next migration script:" + print "history_dataset_association_tag_association table failed: " + print "(OperationalError) (1059, 'Identifier name " + print "'ix_history_dataset_association_tag_association_history_dataset_association_id'" + print "is too long)" + # New tables to support tagging of histories, datasets, and history-dataset associations. Tag_table = Table( "tag", metadata, @@ -42,15 +67,48 @@ Column( "user_value", TrimmedString(255), index=True) ) def upgrade(): + display_migration_details() metadata.reflect() - Tag_table.create() - HistoryTagAssociation_table.create() - DatasetTagAssociation_table.create() - HistoryDatasetAssociationTagAssociation_table.create() + try: + Tag_table.create() + except Exception, e: + print str(e) + log.debug( "Creating tag table failed: %s" % str( e ) ) + try: + HistoryTagAssociation_table.create() + except Exception, e: + print str(e) + log.debug( "Creating history_tag_association table failed: %s" % str( e ) ) + try: + DatasetTagAssociation_table.create() + except Exception, e: + print str(e) + log.debug( "Creating dataset_tag_association table failed: %s" % str( e ) ) + try: + HistoryDatasetAssociationTagAssociation_table.create() + except Exception, e: + print str(e) + log.debug( "Creating history_dataset_association_tag_association table failed: %s" % str( e ) ) def downgrade(): metadata.reflect() - Tag_table.drop() - HistoryTagAssociation_table.drop() - DatasetTagAssociation_table.drop() - HistoryDatasetAssociationTagAssociation_table.drop() \ No newline at end of file + try: + Tag_table.drop() + except Exception, e: + print str(e) + log.debug( "Dropping tag table failed: %s" % str( e ) ) + try: + HistoryTagAssociation_table.drop() + except Exception, e: + print str(e) + log.debug( "Dropping history_tag_association table failed: %s" % str( e ) ) + try: + DatasetTagAssociation_table.drop() + except Exception, e: + print str(e) + log.debug( "Dropping dataset_tag_association table failed: %s" % str( e ) ) + try: + HistoryDatasetAssociationTagAssociation_table.drop() + except Exception, e: + print str(e) + log.debug( "Dropping history_dataset_association_tag_association table failed: %s" % str( e ) ) \ No newline at end of file

1 0

<param> multiple formats
by Dominic Tolle 28 Aug '09

28 Aug '09

Hi, is it possible for a input parameter to accept multiple format types (e.g csfasta and fast), or do I need to write two separate tool configs? For example if I have a format conversion tool which accepts a number of input formats and converts them to a specific output format do I need to write a tool config for each conversion? One way around this would be to include a conditional in the tool config and let the user specify the input type, but is there another way (preferably not involving writing large conditionals)? Cheers Dominic

2 1

[hg] galaxy 2628: Fix for importing a shared history via a link ...
by Greg Von Kuster 26 Aug '09

26 Aug '09

details: http://www.bx.psu.edu/hg/galaxy/rev/d99cee3a4efd changeset: 2628:d99cee3a4efd user: Greg Von Kuster <greg(a)bx.psu.edu> date: Wed Aug 26 07:12:04 2009 -0400 description: Fix for importing a shared history via a link for an anonymous user with no existing Galaxy cookies. 1 file(s) affected in this change: lib/galaxy/web/controllers/history.py diffs (12 lines): diff -r 03accd13066e -r d99cee3a4efd lib/galaxy/web/controllers/history.py --- a/lib/galaxy/web/controllers/history.py Tue Aug 25 16:59:28 2009 -0400 +++ b/lib/galaxy/web/controllers/history.py Wed Aug 26 07:12:04 2009 -0400 @@ -322,7 +322,7 @@ return trans.show_ok_message( """ History "%s" has been imported. Click <a href="%s">here</a> to begin.""" % ( new_history.name, web.url_for( '/' ) ) ) - elif not user_history.datasets or confirm: + elif not user_history or not user_history.datasets or confirm: new_history = import_history.copy() new_history.name = "imported: " + new_history.name new_history.user_id = None

1 0