January 2015 - galaxy-commits - lists.galaxyproject.org

commit/galaxy-central: martenson: degrade Whoosh egg to 2.4.1, adjust util fnc for pretty intervals
by commits-noreply＠bitbucket.org 30 Jan '15

30 Jan '15

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/18dacdf8674e/ Changeset: 18dacdf8674e User: martenson Date: 2015-01-30 18:21:35+00:00 Summary: degrade Whoosh egg to 2.4.1, adjust util fnc for pretty intervals remove debugs Affected #: 3 files diff -r 75739a5f696e57362238ec5c022a17963dd31929 -r 18dacdf8674e21c5e9a2bb61036382e44f6842bc eggs.ini --- a/eggs.ini +++ b/eggs.ini @@ -73,7 +73,7 @@ wsgiref = 0.1.2 Babel = 1.3 wchartype = 0.1 -Whoosh = 2.5.7 +Whoosh = 2.4.1 ; fluent_logger = 0.3.3 raven = 3.1.8 diff -r 75739a5f696e57362238ec5c022a17963dd31929 -r 18dacdf8674e21c5e9a2bb61036382e44f6842bc lib/galaxy/util/__init__.py --- a/lib/galaxy/util/__init__.py +++ b/lib/galaxy/util/__init__.py @@ -371,7 +371,7 @@ if second_diff < 86400: return str(second_diff / 3600) + " hours ago" if day_diff == 1: - return "Yesterday" + return "yesterday" if day_diff < 7: return str( day_diff ) + " days ago" if day_diff < 31: @@ -381,16 +381,16 @@ return str( day_diff / 365 ) + " years ago" else: if day_diff == 0: - return "Today" + return "today" if day_diff == 1: - return "Yesterday" + return "yesterday" if day_diff < 7: - return "this week" + return "less than a week" if day_diff < 31: - return "this month" + return "less than a month" if day_diff < 365: - return "this year" - return str( day_diff / 365 ) + " years ago" + return "less than a year" + return "a few years ago" def pretty_print_json(json_data, is_json_string=False): diff -r 75739a5f696e57362238ec5c022a17963dd31929 -r 18dacdf8674e21c5e9a2bb61036382e44f6842bc lib/galaxy/webapps/tool_shed/search/repo_search.py --- a/lib/galaxy/webapps/tool_shed/search/repo_search.py +++ b/lib/galaxy/webapps/tool_shed/search/repo_search.py @@ -45,7 +45,7 @@ use_final = True def final( self, searcher, docnum, score ): - log.debug('score before: ' + str(score) ) + # log.debug('score before: ' + str(score) ) # Arbitrary for now reasonable_hits = 100.0 @@ -59,15 +59,15 @@ if times_downloaded == 0: times_downloaded = 1 popularity_modifier = ( times_downloaded / reasonable_hits ) - log.debug('popularity_modifier: ' + str(popularity_modifier) ) + # log.debug('popularity_modifier: ' + str(popularity_modifier) ) cert_modifier = 2 if searcher.stored_fields( docnum )[ "approved" ] == 'yes' else 1 - log.debug('cert_modifier: ' + str(cert_modifier) ) + # log.debug('cert_modifier: ' + str(cert_modifier) ) # Adjust the computed score for this document by the popularity # and by the certification level. final_score = score * popularity_modifier * cert_modifier - log.debug('score after: ' + str( final_score ) ) + # log.debug('score after: ' + str( final_score ) ) return final_score @@ -108,16 +108,14 @@ 'remote_repository_url', 'repo_owner_username' ], schema = schema ) - # user_query = parser.parse( search_term ) user_query = parser.parse( '*' + search_term + '*' ) + hits = searcher.search_page( user_query, page, pagelen = 10, terms = True ) - # hits = searcher.search( user_query, terms = True ) - hits = searcher.search_page( user_query, page, pagelen = 10, terms = True ) log.debug( 'searching for: #' + str( search_term ) ) log.debug( 'total hits: ' + str( len( hits ) ) ) log.debug( 'scored hits: ' + str( hits.scored_length() ) ) results = {} - results[ 'total_results'] = str( hits.scored_length() ) + results[ 'total_results'] = str( len( hits ) ) results[ 'hits' ] = [] for hit in hits: hit_dict = {} Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: natefoo: Update tag latest_2015.01.13 for changeset c5e7535b4d22
by commits-noreply＠bitbucket.org 30 Jan '15

30 Jan '15

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/a7ae7afffa8f/ Changeset: a7ae7afffa8f Branch: stable User: natefoo Date: 2015-01-30 18:00:22+00:00 Summary: Update tag latest_2015.01.13 for changeset c5e7535b4d22 Affected #: 1 file diff -r c5e7535b4d229dbbe52d48ea35a27ab601205b7b -r a7ae7afffa8fa7551ea760ca01e9c1808809391b .hgtags --- a/.hgtags +++ b/.hgtags @@ -22,4 +22,4 @@ 2092948937ac30ef82f71463a235c66d34987088 release_2014.10.06 9bd6f8b5b8153db752f4e61ed62f2b6c01ae4a11 latest_2014.10.06 2e8dd2949dd3eee0f56f9a3a5ebf1b2baca24aee release_2015.01.13 -a481d17c04482b069e6b2ebdda4fbca01448fa9c latest_2015.01.13 +c5e7535b4d229dbbe52d48ea35a27ab601205b7b latest_2015.01.13 Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: davebgx: Set time to epoch instead of throwing server error if repository_metadata.time_last_tested is unset.
by commits-noreply＠bitbucket.org 30 Jan '15

30 Jan '15

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/c5e7535b4d22/ Changeset: c5e7535b4d22 Branch: stable User: davebgx Date: 2015-01-29 16:23:35+00:00 Summary: Set time to epoch instead of throwing server error if repository_metadata.time_last_tested is unset. Affected #: 1 file diff -r d386cf71d4f71faeb49ec871b742e81011bee266 -r c5e7535b4d229dbbe52d48ea35a27ab601205b7b lib/galaxy/webapps/tool_shed/controllers/repository.py --- a/lib/galaxy/webapps/tool_shed/controllers/repository.py +++ b/lib/galaxy/webapps/tool_shed/controllers/repository.py @@ -1690,7 +1690,10 @@ description_lines = [] # Per the RSS 2.0 specification, all dates in RSS feeds must be formatted as specified in RFC 822 # section 5.1, e.g. Sat, 07 Sep 2002 00:00:01 UT - time_tested = repository_metadata.time_last_tested.strftime( '%a, %d %b %Y %H:%M:%S UT' ) + if repository_metadata.time_last_tested is None: + time_tested = 'Thu, 01 Jan 1970 00:00:00 UT' + else: + time_tested = repository_metadata.time_last_tested.strftime( '%a, %d %b %Y %H:%M:%S UT' ) # Generate a citable URL for this repository with owner and changeset revision. repository_citable_url = common_util.url_join( tool_shed_url, 'view', Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: davebgx: Add script to update the paths to tool config files in installed repositories' metadata.
by commits-noreply＠bitbucket.org 30 Jan '15

30 Jan '15

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/75739a5f696e/ Changeset: 75739a5f696e User: davebgx Date: 2015-01-30 17:17:01+00:00 Summary: Add script to update the paths to tool config files in installed repositories' metadata. Affected #: 1 file diff -r 3618b7a91e23096ecf7ce5842c1e16c749251a4d -r 75739a5f696e57362238ec5c022a17963dd31929 scripts/update_shed_config_path.py --- /dev/null +++ b/scripts/update_shed_config_path.py @@ -0,0 +1,80 @@ +import os +import argparse +import ConfigParser +import sys +new_path = [ os.path.join( os.getcwd(), "lib" ) ] +new_path.extend( sys.path[1:] ) +sys.path = new_path + +import logging +import galaxy.model.tool_shed_install +import galaxy.model.tool_shed_install.mapping as mapping +from galaxy.model.orm import * +from galaxy import eggs +eggs.require('sqlalchemy') +import sqlalchemy + +def main( opts, session, model ): + ''' + Find all tool shed repositories with the bad path and update with the correct path. + ''' + for row in session.query( model.ToolShedRepository ).all(): + if 'shed_config_filename' in row.metadata: + if row.metadata['shed_config_filename'] == opts.bad_filename: + row.metadata['shed_config_filename'] = opts.good_filename + session.add( row ) + session.flush() + return 0 + +def create_database( config_file ): + parser = ConfigParser.SafeConfigParser() + parser.read( config_file ) + # Determine which database connection to use. + database_connection = parser.get( 'app:main', 'install_database_connection' ) + if database_connection is None: + database_connection = parser.get( 'app:main', 'database_connection' ) + if database_connection is None: + database_connection = 'sqlite:///%s' % parser.get( 'app:main', 'database_file' ) + if database_connection is None: + print 'Unable to determine correct database connection.' + exit(1) + + '''Initialize the database file.''' + dialect_to_egg = { + "sqlite" : "pysqlite>=2", + "postgres" : "psycopg2", + "postgresql" : "psycopg2", + "mysql" : "MySQL_python" + } + dialect = ( database_connection.split( ':', 1 ) )[0] + try: + egg = dialect_to_egg[ dialect ] + try: + eggs.require( egg ) + print( "%s egg successfully loaded for %s dialect" % ( egg, dialect ) ) + except: + # If the module is in the path elsewhere (i.e. non-egg), it'll still load. + print( "%s egg not found, but an attempt will be made to use %s anyway" % ( egg, dialect ) ) + except KeyError: + # Let this go, it could possibly work with db's we don't support. + print( "database_connection contains an unknown SQLAlchemy database dialect: %s" % dialect ) + + # Initialize the database connection. + engine = create_engine( database_connection ) + meta = MetaData( bind=engine ) + install_session = Session = scoped_session( sessionmaker( bind=engine, autoflush=False, autocommit=True ) ) + model = mapping.init( database_connection ) + return install_session, model + +if __name__ == '__main__': + parser = argparse.ArgumentParser() + parser.add_argument( '--config_file', dest='config_file', required=True, help="The path to your Galaxy configuration .ini file." ) + parser.add_argument( '--from', dest='bad_filename', required=True, help="The old, invalid path to the shed_tool_conf.xml or migrated_tools_conf.xml file." ) + parser.add_argument( '--to', dest='good_filename', required=True, help="The updated path to the shed_tool_conf.xml or migrated_tools_conf.xml file." ) + parser.add_argument( '--force', dest='force', action='store_true', help="Use this flag to set the new path even if the file does not (yet) exist there." ) + opts = parser.parse_args() + if not os.path.exists( opts.good_filename ) and not opts.force: + print 'The file %s does not exist, use the --force option to proceed.' % opts.good_filename + exit(1) + session, model = create_database( opts.config_file ) + exit( main( opts, session, model ) ) Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: 2 new changesets
by commits-noreply＠bitbucket.org 30 Jan '15

30 Jan '15

2 new commits in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/0fe529fb7881/ Changeset: 0fe529fb7881 User: dannon Date: 2015-01-30 16:52:12+00:00 Summary: Add sentry logging to config. Affected #: 1 file diff -r 81c215f35665f2361ead5d60fe0a12ee24f31a0b -r 0fe529fb7881daef76b08eb113bd5a3d4480e358 lib/galaxy/webapps/tool_shed/config.py --- a/lib/galaxy/webapps/tool_shed/config.py +++ b/lib/galaxy/webapps/tool_shed/config.py @@ -8,6 +8,7 @@ import logging.config from optparse import OptionParser import ConfigParser +from galaxy import eggs from galaxy.util import string_as_bool, listify log = logging.getLogger( __name__ ) @@ -288,3 +289,11 @@ # Hook everything up handler.setFormatter( formatter ) root.addHandler( handler ) + # If sentry is configured, also log to it + if config.sentry_dsn: + eggs.require( "raven" ) + from raven.handlers.logging import SentryHandler + sentry_handler = SentryHandler( config.sentry_dsn ) + sentry_handler.setLevel( logging.WARN ) + root.addHandler( sentry_handler ) + https://bitbucket.org/galaxy/galaxy-central/commits/3618b7a91e23/ Changeset: 3618b7a91e23 User: dannon Date: 2015-01-30 16:55:51+00:00 Summary: First of probably many changes related to elementtree replacement and futurewarnings. Affected #: 1 file diff -r 0fe529fb7881daef76b08eb113bd5a3d4480e358 -r 3618b7a91e23096ecf7ce5842c1e16c749251a4d lib/galaxy/datatypes/registry.py --- a/lib/galaxy/datatypes/registry.py +++ b/lib/galaxy/datatypes/registry.py @@ -346,7 +346,7 @@ """ sniffer_elem_classes = [ e.attrib[ 'type' ] for e in self.sniffer_elems ] sniffers = root.find( 'sniffers' ) - if sniffers: + if sniffers is not None: for elem in sniffers.findall( 'sniffer' ): # Keep a status of the process steps to enable stopping the process of handling the sniffer if necessary. ok = True Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: carlfeberhard: Core, controllers: when parsing ids, stringify to prevent all numeric (but valid) ids from being pre-processed as integers
by commits-noreply＠bitbucket.org 30 Jan '15

30 Jan '15

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/81c215f35665/ Changeset: 81c215f35665 User: carlfeberhard Date: 2015-01-30 16:40:21+00:00 Summary: Core, controllers: when parsing ids, stringify to prevent all numeric (but valid) ids from being pre-processed as integers Affected #: 1 file diff -r 1e0f1acf3c54ac7ed12d174f40c2f2f62b01f043 -r 81c215f35665f2361ead5d60fe0a12ee24f31a0b lib/galaxy/web/base/controller.py --- a/lib/galaxy/web/base/controller.py +++ b/lib/galaxy/web/base/controller.py @@ -103,9 +103,11 @@ # ---- parsing query params def decode_id( self, id ): try: - return self.app.security.decode_id( id ) - except: - msg = "Malformed History id ( %s ) specified, unable to decode" % ( str( id ) ) + # note: use str - occasionally a fully numeric id will be placed in post body and parsed as int via JSON + # resulting in error for valid id + return self.app.security.decode_id( str( id ) ) + except ( ValueError, TypeError ): + msg = "Malformed id ( %s ) specified, unable to decode" % ( str( id ) ) raise exceptions.MalformedId( msg, id=str( id ) ) def encode_all_ids( self, trans, rval, recursive=False ): Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: 2 new changesets
by commits-noreply＠bitbucket.org 30 Jan '15

30 Jan '15

2 new commits in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/8ebc6c5b1cbd/ Changeset: 8ebc6c5b1cbd User: dannon Date: 2015-01-30 16:21:25+00:00 Summary: Make toolshed compatible with sentry reporting. Affected #: 1 file diff -r a917e3d871f2782ae4cf1b57bb4775c0e0224efe -r 8ebc6c5b1cbd974df125b9e9d5ec885efd916949 lib/galaxy/webapps/tool_shed/config.py --- a/lib/galaxy/webapps/tool_shed/config.py +++ b/lib/galaxy/webapps/tool_shed/config.py @@ -2,6 +2,7 @@ Universe configuration builder. """ import os +import re import sys import logging import logging.config @@ -133,6 +134,19 @@ self.citation_cache_data_dir = resolve_path( kwargs.get( "citation_cache_data_dir", "database/tool_shed_citations/data" ), self.root ) self.citation_cache_lock_dir = resolve_path( kwargs.get( "citation_cache_lock_dir", "database/tool_shed_citations/locks" ), self.root ) + @property + def sentry_dsn_public( self ): + """ + Sentry URL with private key removed for use in client side scripts, + sentry server will need to be configured to accept events + """ + # TODO refactor this to a common place between toolshed/galaxy config, along + # with other duplicated methods. + if self.sentry_dsn: + return re.sub( r"^([^:/?#]+:)?//(\w+):(\w+)", r"\1//\2", self.sentry_dsn ) + else: + return None + def __parse_config_file_options( self, kwargs ): defaults = dict( datatypes_config_file = [ 'config/datatypes_conf.xml', 'datatypes_conf.xml', 'config/datatypes_conf.xml.sample' ], https://bitbucket.org/galaxy/galaxy-central/commits/1e0f1acf3c54/ Changeset: 1e0f1acf3c54 User: dannon Date: 2015-01-30 16:23:36+00:00 Summary: Add sentry_dsn to toolshed sample. Affected #: 1 file diff -r 8ebc6c5b1cbd974df125b9e9d5ec885efd916949 -r 1e0f1acf3c54ac7ed12d174f40c2f2f62b01f043 config/tool_shed.ini.sample --- a/config/tool_shed.ini.sample +++ b/config/tool_shed.ini.sample @@ -125,3 +125,6 @@ #static_favicon_dir = static/favicon.ico #static_scripts_dir = static/scripts/ #static_style_dir = static/style/blue + +# Sentry (getsentry.com) DSN for catching bugs. +#sentry_dsn = None Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

[galaxyproject/usegalaxy-playbook] 967336: Pull/distribute mutable configs on Main.
by GitHub 30 Jan '15

30 Jan '15

Branch: refs/heads/master Home: https://github.com/galaxyproject/usegalaxy-playbook Commit: 9673364495809bffd6697cc7e04c65218a0519d8 https://github.com/galaxyproject/usegalaxy-playbook/commit/9673364495809bff… Author: Nate Coraor <nate(a)bx.psu.edu> Date: 2015-01-30 (Fri, 30 Jan 2015) Changed paths: M files/galaxy/usegalaxy.org/var/integrated_tool_panel.xml M files/galaxy/usegalaxy.org/var/migrated_tools_conf.xml M files/galaxy/usegalaxy.org/var/shed_tool_conf.xml Log Message: ----------- Pull/distribute mutable configs on Main.

1 0

commit/galaxy-central: carlfeberhard: UI, history multi-view: fix column height response in firefox
by commits-noreply＠bitbucket.org 29 Jan '15

29 Jan '15

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/a917e3d871f2/ Changeset: a917e3d871f2 User: carlfeberhard Date: 2015-01-29 22:51:57+00:00 Summary: UI, history multi-view: fix column height response in firefox Affected #: 1 file diff -r a0c2bd2de611ab95292f9ae5f759dae9fa4939aa -r a917e3d871f2782ae4cf1b57bb4775c0e0224efe templates/webapps/galaxy/history/view_multiple.mako --- a/templates/webapps/galaxy/history/view_multiple.mako +++ b/templates/webapps/galaxy/history/view_multiple.mako @@ -33,6 +33,11 @@ display: -ms-flexbox; display: flex; + /* force ff to squish beyond content: + https://developer.mozilla.org/en-US/Firefox/Releases/34/Site_Compatibility#… */ + min-width: 0px; + min-height: 0px; + -webkit-align-items: stretch; -ms-align-items: stretch; align-items: stretch; Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

[galaxyproject/usegalaxy-playbook] 80a4db: Update Main and fetch mutable configs.
by GitHub 29 Jan '15

29 Jan '15

Branch: refs/heads/master Home: https://github.com/galaxyproject/usegalaxy-playbook Commit: 80a4db0bb9a8365d1e9fa3e522522f8d361addc7 https://github.com/galaxyproject/usegalaxy-playbook/commit/80a4db0bb9a8365d… Author: Nate Coraor <nate(a)bx.psu.edu> Date: 2015-01-29 (Thu, 29 Jan 2015) Changed paths: M files/galaxy/usegalaxy.org/var/shed_tool_conf.xml M files/galaxy/usegalaxy.org/var/shed_tool_data_table_conf.xml M production/group_vars/all.yml Log Message: ----------- Update Main and fetch mutable configs.

1 0