January 2015 - galaxy-commits - lists.galaxyproject.org

[galaxyproject/usegalaxy-playbook] 5058b8: supervisor fixes for Main
by GitHub 28 Jan '15

28 Jan '15

Branch: refs/heads/master Home: https://github.com/galaxyproject/usegalaxy-playbook Commit: 5058b8c1bbceebd4bbec39d1bd8968adb6b69ed8 https://github.com/galaxyproject/usegalaxy-playbook/commit/5058b8c1bbceebd4… Author: Nate Coraor <nate(a)bx.psu.edu> Date: 2015-01-28 (Wed, 28 Jan 2015) Changed paths: M production/group_vars/galaxyservers.yml M production/host_vars/galaxy-web-01.tacc.utexas.edu.yml M production/inventory M production/secret_vars/galaxyservers.yml M templates/supervisor/galaxy_group.conf.j2 Log Message: ----------- supervisor fixes for Main

1 0

commit/galaxy-central: 2 new changesets
by commits-noreply＠bitbucket.org 28 Jan '15

28 Jan '15

2 new commits in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/e3f50d8803e6/ Changeset: e3f50d8803e6 Branch: samtools1_bam_index User: natefoo Date: 2015-01-28 21:29:48+00:00 Summary: Close samtools1_bam_index branch. Affected #: 0 files https://bitbucket.org/galaxy/galaxy-central/commits/c99e026b4b0a/ Changeset: c99e026b4b0a User: natefoo Date: 2015-01-28 21:29:58+00:00 Summary: merge. Affected #: 0 files Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: natefoo: Remove GeneTrack remnants.
by commits-noreply＠bitbucket.org 28 Jan '15

28 Jan '15

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/0d832a69da40/ Changeset: 0d832a69da40 User: natefoo Date: 2015-01-28 21:29:21+00:00 Summary: Remove GeneTrack remnants. Affected #: 4 files diff -r 1fa9a9409fc92cea7c1775e57c462a56d01c3185 -r 0d832a69da40612bbbcded0d71230b53a6a7c1e2 eggs.ini --- a/eggs.ini +++ b/eggs.ini @@ -40,7 +40,6 @@ drmaa = 0.7.6 elementtree = 1.2.6_20050316 Fabric = 1.7.0 -GeneTrack = 2.0.0_beta_1 importlib = 1.0.3 kombu = 3.0.13 lrucache = 0.2 diff -r 1fa9a9409fc92cea7c1775e57c462a56d01c3185 -r 0d832a69da40612bbbcded0d71230b53a6a7c1e2 scripts/scramble/patches/GeneTrack/setup.py --- a/scripts/scramble/patches/GeneTrack/setup.py +++ /dev/null @@ -1,15 +0,0 @@ -from os import walk -from os.path import join -from setuptools import setup, find_packages - -def walk_files( top ): - for dir, dirs, files in walk( top ): - yield( dir, [ join( dir, f ) for f in files ] ) - -setup( - name = "GeneTrack", - version = "2.0.0-beta-1", - packages = ['genetrack','genetrack.scripts'], - data_files = [ f for f in walk_files('tests') ], - zip_safe = False -) diff -r 1fa9a9409fc92cea7c1775e57c462a56d01c3185 -r 0d832a69da40612bbbcded0d71230b53a6a7c1e2 static/genetrack/genetrack.css --- a/static/genetrack/genetrack.css +++ /dev/null @@ -1,78 +0,0 @@ - -body { - font-family: "Trebuchet MS", Arial, tahoma, sans-serif; - font-size: 14px; - line-height: 1.6em; - margin: 0; - padding: 0; - border-top: 9px solid #CCD9FF; -} - -/* Error message style */ -.error{ - background: #FFFF66; -} - -/* Error message style */ -.message{ - background: #33FF66; -} - -/* Odd data row in the table */ -.selected { - background-color: #FFFFCC; -} - -.nav_button{ - background-color:#EEEEEE; - border:1px solid; - color: #000000; -} - -.nav_button:hover{ - background-color:#000000; - border:1px solid; - color: #FFFFFF; -} - -.grey { - background-color: #EFEFEF; -} - -.odd { - background-color: #ECECEC; -} - -.even { - background-color: #FFFFFF; -} - -/* Text table style */ -.data_table { - border: 1px solid #CCCCCC; - background-color: white; -} - -/* Footer is added to every page */ -#footer { - background: #EFEFEF; - text-align:center; - padding:.2em; - border-top: 1px solid #CCD9FF; - border-bottom: 1px solid #CCD9FF; - clear: both; -} - -#footer p { - font-size:.94em; line-height:2em; color:#cccccc; margin: 0; - } - -#tag { - font-size:.80em; margin: 4px; padding: 2px; - } - - -#footer img { - vertical-align: middle; margin-left: 3px; padding-bottom: 2px; -} - diff -r 1fa9a9409fc92cea7c1775e57c462a56d01c3185 -r 0d832a69da40612bbbcded0d71230b53a6a7c1e2 static/genetrack/genetrack.js --- a/static/genetrack/genetrack.js +++ /dev/null @@ -1,79 +0,0 @@ -var cookie_name = "genetrack_ui" -var now = new Date(); -now.setTime(now.getTime() + 365 * 24 * 60 * 60 * 1000); - -// this toggles between none and block -function toggle(name){ - var elem = get(name) - if (elem) { - if (elem.style.display=="none"){ - elem.style.display="block" - setCookie(cookie_name, name, now) - } else { - elem.style.display="none" - setCookie(cookie_name, '', now) - } - - } -} - -function main(){ - //executed upon main body load - var value = getCookie(cookie_name); - toggle( value ) -} - -// this toggles between visible and hidden -function show(name){ - var elem = get(name) - if (elem.style.visibility=="hidden"){ - elem.style.visibility="visible"; - } else { - elem.style.visibility="hidden"; - } -} - -// utility function to get the length of on object -function len(obj){ - return obj.length; -} - -// utility function to get an element by id -function get(name){ - return document.getElementById(name); -} - -// pops up a window -function pop_up(url) { - day = new Date(); - id = day.getTime(); - eval("page" + id + " = window.open(url, '" + id + "', 'toolbar=0,scrollbars=1,location=0,statusbar=1,menubar=0,resizable=1,width=500,height=300');"); -} - -// -// cookie management off the web -// http://www.webreference.com/js/column8/property.html -// -function setCookie(name, value, expires, path, domain, secure) { - var curCookie = name + "=" + escape(value) + - ((expires) ? "; expires=" + expires.toGMTString() : "") + - ((path) ? "; path=" + path : "") + - ((domain) ? "; domain=" + domain : "") + - ((secure) ? "; secure" : ""); - document.cookie = curCookie; -} - -function getCookie(name) { - var dc = document.cookie; - var prefix = name + "="; - var begin = dc.indexOf("; " + prefix); - if (begin == -1) { - begin = dc.indexOf(prefix); - if (begin != 0) return null; - } else - begin += 2; - var end = document.cookie.indexOf(";", begin); - if (end == -1) - end = dc.length; - return unescape(dc.substring(begin + prefix.length, end)); -} Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: guerler: Parameters: Multiselect by default not optional
by commits-noreply＠bitbucket.org 28 Jan '15

28 Jan '15

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/1fa9a9409fc9/ Changeset: 1fa9a9409fc9 User: guerler Date: 2015-01-28 21:21:23+00:00 Summary: Parameters: Multiselect by default not optional Affected #: 1 file diff -r 9c767090274eb2a213854dcdb367135d782fecec -r 1fa9a9409fc92cea7c1775e57c462a56d01c3185 lib/galaxy/tools/parameters/basic.py --- a/lib/galaxy/tools/parameters/basic.py +++ b/lib/galaxy/tools/parameters/basic.py @@ -772,8 +772,6 @@ input_source = ensure_input_source( input_source ) ToolParameter.__init__( self, tool, input_source ) self.multiple = input_source.get_bool( 'multiple', False ) - # Multiple selects are optional by default, single selection is the inverse. - self.optional = input_source.parse_optional( self.multiple ) self.display = input_source.get( 'display', None ) self.separator = input_source.get( 'separator', ',' ) self.legal_values = set() Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: natefoo: Remove memdump, which probably has not worked for many years.
by commits-noreply＠bitbucket.org 28 Jan '15

28 Jan '15

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/9c767090274e/ Changeset: 9c767090274e User: natefoo Date: 2015-01-28 21:20:30+00:00 Summary: Remove memdump, which probably has not worked for many years. Affected #: 9 files diff -r c8b49753bf99d03462bace0dbb2f38b9c0f52915 -r 9c767090274eb2a213854dcdb367135d782fecec config/galaxy.ini.sample --- a/config/galaxy.ini.sample +++ b/config/galaxy.ini.sample @@ -629,10 +629,6 @@ # lot of CPU. #use_heartbeat = False -# Enable the memory debugging interface (careful, negatively impacts server -# performance). -#use_memdump = False - # -- Data Libraries # These library upload options are described in much more detail in the wiki: diff -r c8b49753bf99d03462bace0dbb2f38b9c0f52915 -r 9c767090274eb2a213854dcdb367135d782fecec eggs.ini --- a/eggs.ini +++ b/eggs.ini @@ -26,7 +26,6 @@ pysqlite = 2.5.6 python_lzo = 1.08_2.03_static PyYAML = 3.10 -guppy = 0.1.10 SQLAlchemy = 0.7.9 ; msgpack_python = 0.2.4 diff -r c8b49753bf99d03462bace0dbb2f38b9c0f52915 -r 9c767090274eb2a213854dcdb367135d782fecec lib/galaxy/app.py --- a/lib/galaxy/app.py +++ b/lib/galaxy/app.py @@ -107,10 +107,8 @@ self.quota_agent = galaxy.quota.QuotaAgent( self.model ) else: self.quota_agent = galaxy.quota.NoQuotaAgent( self.model ) - # Heartbeat and memdump for thread / heap profiling + # Heartbeat for thread profiling self.heartbeat = None - self.memdump = None - self.memory_usage = None # Container for OpenID authentication routines if self.config.enable_openid: from galaxy.web.framework import openid_manager @@ -125,11 +123,6 @@ self.heartbeat = heartbeat.Heartbeat( fname=self.config.heartbeat_log ) self.heartbeat.daemon = True self.heartbeat.start() - # Enable the memdump signal catcher if configured and available - if self.config.use_memdump: - from galaxy.util import memdump - if memdump.Memdump: - self.memdump = memdump.Memdump() # Transfer manager client if self.config.get_bool( 'enable_beta_job_managers', False ): from galaxy.jobs import transfer_manager diff -r c8b49753bf99d03462bace0dbb2f38b9c0f52915 -r 9c767090274eb2a213854dcdb367135d782fecec lib/galaxy/config.py --- a/lib/galaxy/config.py +++ b/lib/galaxy/config.py @@ -227,7 +227,6 @@ self.external_chown_script = kwargs.get('external_chown_script', None) self.environment_setup_file = kwargs.get( 'environment_setup_file', None ) self.use_heartbeat = string_as_bool( kwargs.get( 'use_heartbeat', 'False' ) ) - self.use_memdump = string_as_bool( kwargs.get( 'use_memdump', 'False' ) ) self.log_actions = string_as_bool( kwargs.get( 'log_actions', 'False' ) ) self.log_events = string_as_bool( kwargs.get( 'log_events', 'False' ) ) self.sanitize_all_html = string_as_bool( kwargs.get( 'sanitize_all_html', True ) ) diff -r c8b49753bf99d03462bace0dbb2f38b9c0f52915 -r 9c767090274eb2a213854dcdb367135d782fecec lib/galaxy/eggs/__init__.py --- a/lib/galaxy/eggs/__init__.py +++ b/lib/galaxy/eggs/__init__.py @@ -385,8 +385,6 @@ "DRMAA_python": lambda: "sge" in self.config.get( "app:main", "start_job_runners" ).split(","), "drmaa": lambda: "drmaa" in self.config.get( "app:main", "start_job_runners" ).split(","), "pbs_python": lambda: "pbs" in self.config.get( "app:main", "start_job_runners" ).split(","), - "threadframe": lambda: self.config.get( "app:main", "use_heartbeat" ), - "guppy": lambda: self.config.get( "app:main", "use_memdump" ), "python_openid": lambda: self.config.get( "app:main", "enable_openid" ), "python_daemon": lambda: sys.version_info[:2] >= ( 2, 5 ), "pysam": lambda: check_pysam(), diff -r c8b49753bf99d03462bace0dbb2f38b9c0f52915 -r 9c767090274eb2a213854dcdb367135d782fecec lib/galaxy/util/memdump.py --- a/lib/galaxy/util/memdump.py +++ /dev/null @@ -1,50 +0,0 @@ - -# Attempt to load guppy module, and only define Memdump class -# if available - -try: - - import pkg_resources - pkg_resources.require( "guppy" ) - -except: - - import sys - print >> sys.stderr, "No guppy module, Memdump not available" - Memdump = None - -else: - - import os, sys, signal, time, guppy - - class Memdump( object ): - def __init__( self, signum=signal.SIGUSR1, fname="memdump.log" ): - self.fname = fname - signal.signal( signum, self.dump ) - self.heapy = guppy.hpy() - self.heap = None - def dump( self, signum, stack ): - file = open( self.fname, "a" ) - print >> file, "Memdump for pid %d at %s" % ( os.getpid(), time.asctime() ) - print >> file - try: - self.heap = self.heapy.heap() - print >> file, "heap():" - print >> file, self.heap - print >> file, "\nbyrcs:" - print >> file, self.heap.byrcs - print >> file, "\nbyrcs[0].byid:" - print >> file, self.heap.byrcs[0].byid - print >> file, "\nget_rp():" - print >> file, self.heap.get_rp() - self.heapy.setref() - except AssertionError: - pass - print >> file, "\nEnd dump\n" - file.close() - def setref( self ): - self.heapy.setref() - def get( self, update=False ): - if update: - self.heap = self.heapy.heap() - return self.heap diff -r c8b49753bf99d03462bace0dbb2f38b9c0f52915 -r 9c767090274eb2a213854dcdb367135d782fecec lib/galaxy/web/base/controllers/admin.py --- a/lib/galaxy/web/base/controllers/admin.py +++ b/lib/galaxy/web/base/controllers/admin.py @@ -1026,48 +1026,6 @@ @web.expose @web.require_admin - def memdump( self, trans, ids = 'None', sorts = 'None', pages = 'None', new_id = None, new_sort = None, **kwd ): - if self.app.memdump is None: - return trans.show_error_message( "Memdump is not enabled (set <code>use_memdump = True</code> in galaxy.ini)" ) - heap = self.app.memdump.get() - p = util.Params( kwd ) - msg = None - if p.dump: - heap = self.app.memdump.get( update = True ) - msg = "Heap dump complete" - elif p.setref: - self.app.memdump.setref() - msg = "Reference point set (dump to see delta from this point)" - ids = ids.split( ',' ) - sorts = sorts.split( ',' ) - if new_id is not None: - ids.append( new_id ) - sorts.append( 'None' ) - elif new_sort is not None: - sorts[-1] = new_sort - breadcrumb = "<a href='%s' class='breadcrumb'>heap</a>" % web.url_for(controller='admin', action='memdump') - # new lists so we can assemble breadcrumb links - new_ids = [] - new_sorts = [] - for id, sort in zip( ids, sorts ): - new_ids.append( id ) - if id != 'None': - breadcrumb += "<a href='%s' class='breadcrumb'>[%s]</a>" % ( web.url_for(controller='admin', action='memdump', ids=','.join( new_ids ), sorts=','.join( new_sorts ) ), id ) - heap = heap[int(id)] - new_sorts.append( sort ) - if sort != 'None': - breadcrumb += "<a href='%s' class='breadcrumb'>.by('%s')</a>" % ( web.url_for(controller='admin', action='memdump', ids=','.join( new_ids ), sorts=','.join( new_sorts ) ), sort ) - heap = heap.by( sort ) - ids = ','.join( new_ids ) - sorts = ','.join( new_sorts ) - if p.theone: - breadcrumb += ".theone" - heap = heap.theone - return trans.fill_template( '/admin/memdump.mako', heap = heap, ids = ids, sorts = sorts, breadcrumb = breadcrumb, msg = msg ) - - - @web.expose - @web.require_admin def jobs( self, trans, stop = [], stop_msg = None, cutoff = 180, job_lock = None, ajl_submit = None, **kwd ): deleted = [] msg = None diff -r c8b49753bf99d03462bace0dbb2f38b9c0f52915 -r 9c767090274eb2a213854dcdb367135d782fecec templates/admin/memdump.mako --- a/templates/admin/memdump.mako +++ /dev/null @@ -1,76 +0,0 @@ -<%inherit file="/base.mako"/> -<%namespace file="/message.mako" import="render_msg" /> - -<%def name="title()">Memory Profiling</%def> -<% - import re - from xml.sax.saxutils import escape, unescape -%> - -<style type="text/css"> - a.breadcrumb:link, - a.breadcrumb:visited, - a.breadcrumb:active { - text-decoration: none; - } - a.breadcrumb:hover { - text-decoration: underline; - } -</style> - -<h2>Memory Profiling</h2> - -<ul class="manage-table-actions"> - <li><a class="action-button" href="${h.url_for( controller='admin', action='memdump', dump=True )}">Dump memory (warning: hangs server!)</a></li> - <li><a class="action-button" href="${h.url_for( controller='admin', action='memdump', setref=True )}">Set reference point</a></li> -</ul> - -<%def name="htmlize( heap )"> -<% - s = escape( str( heap ) ) - new_s = "" - id_re = re.compile('^(\s+)([0-9]+)') - for line in s.split( '\n' ): - try: - id = id_re.search( line ).group( 2 ) - except: - id = None - new_s += re.sub( id_re, r'\1<a href="' + h.url_for( controller='admin', action='memdump', ids=ids, sorts=sorts, new_id=id ) + r'">\2</a>', line ) - if id and heap[int(id)].count == 1: - new_s += " <a href='%s'>theone</a>\n" % h.url_for( controller='admin', action='memdump', ids=ids, sorts=sorts, new_id=id, theone=True ) - else: - new_s += "\n" - return new_s -%> -</%def> - -%if message: - ${render_msg( message, status )} -%endif - -%if heap is None: - No memory dump available. Click "Dump memory" to create one. -%else: - <pre> - <br/> -You are here: ${breadcrumb}<br/> - %if breadcrumb.endswith( 'theone' ): - ${heap|h} - %else: - <nobr> -Sort: - <a href="${h.url_for( controller='admin', action='memdump', ids=ids, sorts=sorts, new_sort='Class')}">Class</a> | - <a href="${h.url_for( controller='admin', action='memdump', ids=ids, sorts=sorts, new_sort='Clodo' )}">Clodo</a> | - <a href="${h.url_for( controller='admin', action='memdump', ids=ids, sorts=sorts, new_sort='Id' )}">Id</a> | - <a href="${h.url_for( controller='admin', action='memdump', ids=ids, sorts=sorts, new_sort='Idset' )}">Idset</a> | - <a href="${h.url_for( controller='admin', action='memdump', ids=ids, sorts=sorts, new_sort='Module' )}">Module</a> | - <a href="${h.url_for( controller='admin', action='memdump', ids=ids, sorts=sorts, new_sort='Unity' )}">Unity</a> | - <a href="${h.url_for( controller='admin', action='memdump', ids=ids, sorts=sorts, new_sort='Rcs' )}">Rcs</a> | - <a href="${h.url_for( controller='admin', action='memdump', ids=ids, sorts=sorts, new_sort='Size' )}">Size</a> | - <a href="${h.url_for( controller='admin', action='memdump', ids=ids, sorts=sorts, new_sort='Type' )}">Type</a> | - <a href="${h.url_for( controller='admin', action='memdump', ids=ids, sorts=sorts, new_sort='Via' )}">Via</a> - </nobr> - ${htmlize( heap )} - %endif - </pre> -%endif diff -r c8b49753bf99d03462bace0dbb2f38b9c0f52915 -r 9c767090274eb2a213854dcdb367135d782fecec templates/webapps/galaxy/admin/index.mako --- a/templates/webapps/galaxy/admin/index.mako +++ b/templates/webapps/galaxy/admin/index.mako @@ -78,7 +78,6 @@ <div class="toolTitle"><a href="${h.url_for( controller='admin', action='tool_versions' )}" target="galaxy_main">View tool lineage</a></div><div class="toolTitle"><a href="${h.url_for( controller='admin', action='package_tool' )}" target="galaxy_main">Download tool tarball</a></div><div class="toolTitle"><a href="${h.url_for( controller='admin', action='reload_tool' )}" target="galaxy_main">Reload a tool's configuration</a></div> - <div class="toolTitle"><a href="${h.url_for( controller='admin', action='memdump' )}" target="galaxy_main">Profile memory usage</a></div><div class="toolTitle"><a href="${h.url_for( controller='admin', action='jobs' )}" target="galaxy_main">Manage jobs</a></div><div class="toolTitle"><a href="${h.url_for( controller='admin', action='review_tool_migration_stages' )}" target="galaxy_main">Review tool migration stages</a></div> %if installing_repository_ids: Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: guerler: ToolForm: Show errors for skipped fields
by commits-noreply＠bitbucket.org 28 Jan '15

28 Jan '15

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/c8b49753bf99/ Changeset: c8b49753bf99 User: guerler Date: 2015-01-28 20:43:42+00:00 Summary: ToolForm: Show errors for skipped fields Affected #: 3 files diff -r e0b649785f0a92c1030c42ea826d367ee7d5dc3c -r c8b49753bf99d03462bace0dbb2f38b9c0f52915 client/galaxy/scripts/mvc/tools/tools-input.js --- a/client/galaxy/scripts/mvc/tools/tools-input.js +++ b/client/galaxy/scripts/mvc/tools/tools-input.js @@ -56,11 +56,9 @@ /** Set error text */ error: function(text) { - if (!this.field.skip) { - this.$error_text.html(text); - this.$error.show(); - this.$el.addClass('ui-error'); - } + this.$error_text.html(text); + this.$error.show(); + this.$el.addClass('ui-error'); }, /** Reset this view diff -r e0b649785f0a92c1030c42ea826d367ee7d5dc3c -r c8b49753bf99d03462bace0dbb2f38b9c0f52915 static/scripts/mvc/tools/tools-input.js --- a/static/scripts/mvc/tools/tools-input.js +++ b/static/scripts/mvc/tools/tools-input.js @@ -56,11 +56,9 @@ /** Set error text */ error: function(text) { - if (!this.field.skip) { - this.$error_text.html(text); - this.$error.show(); - this.$el.addClass('ui-error'); - } + this.$error_text.html(text); + this.$error.show(); + this.$el.addClass('ui-error'); }, /** Reset this view diff -r e0b649785f0a92c1030c42ea826d367ee7d5dc3c -r c8b49753bf99d03462bace0dbb2f38b9c0f52915 static/scripts/packed/mvc/tools/tools-input.js --- a/static/scripts/packed/mvc/tools/tools-input.js +++ b/static/scripts/packed/mvc/tools/tools-input.js @@ -1,1 +1,1 @@ -define([],function(){return Backbone.View.extend({initialize:function(d,c){this.app=d;this.text_enable=d.options.text_enable||"Enable";this.text_disable=d.options.text_disable||"Disable";this.field=c.field;this.default_value=c.default_value;this.setElement(this._template(c));this.$field=this.$el.find(".ui-table-form-field");this.$title_optional=this.$el.find(".ui-table-form-title-optional");this.$error_text=this.$el.find(".ui-table-form-error-text");this.$error=this.$el.find(".ui-table-form-error");this.$field.prepend(this.field.$el);this.field.skip=false;var b=this.field.value&&this.field.value();this.field.skip=Boolean(c.optional&&((this.default_value===undefined)||((this.field.validate&&!this.field.validate())||!b||(b==this.default_value)||(Number(b)==Number(this.default_value))||(JSON.stringify(b)==JSON.stringify(this.default_value)))));this._refresh();var a=this;this.$title_optional.on("click",function(){a.field.skip=!a.field.skip;a._refresh()})},error:function(a){if(!this.field.skip){this.$error_text.html(a);this.$error.show();this.$el.addClass("ui-error")}},reset:function(){this.$error.hide();this.$el.removeClass("ui-error")},_refresh:function(){if(!this.field.skip){this.$field.fadeIn("fast");this.$title_optional.html(this.text_disable)}else{this.reset();this.$field.hide();this.$title_optional.html(this.text_enable);this.field.value&&this.field.value(this.default_value)}this.app.trigger("refresh")},_template:function(a){var b='<div class="ui-table-form-element"><div class="ui-table-form-error ui-error"><span class="fa fa-arrow-down"/><span class="ui-table-form-error-text"/></div><div class="ui-table-form-title-strong">';if(a.optional){b+=a.label+'<span>&nbsp[<span class="ui-table-form-title-optional"/>]</span>'}else{b+=a.label}b+='</div><div class="ui-table-form-field">';if(a.help){b+='<div class="ui-table-form-info">'+a.help+"</div>"}b+="</div></div>";return b}})}); \ No newline at end of file +define([],function(){return Backbone.View.extend({initialize:function(d,c){this.app=d;this.text_enable=d.options.text_enable||"Enable";this.text_disable=d.options.text_disable||"Disable";this.field=c.field;this.default_value=c.default_value;this.setElement(this._template(c));this.$field=this.$el.find(".ui-table-form-field");this.$title_optional=this.$el.find(".ui-table-form-title-optional");this.$error_text=this.$el.find(".ui-table-form-error-text");this.$error=this.$el.find(".ui-table-form-error");this.$field.prepend(this.field.$el);this.field.skip=false;var b=this.field.value&&this.field.value();this.field.skip=Boolean(c.optional&&((this.default_value===undefined)||((this.field.validate&&!this.field.validate())||!b||(b==this.default_value)||(Number(b)==Number(this.default_value))||(JSON.stringify(b)==JSON.stringify(this.default_value)))));this._refresh();var a=this;this.$title_optional.on("click",function(){a.field.skip=!a.field.skip;a._refresh()})},error:function(a){this.$error_text.html(a);this.$error.show();this.$el.addClass("ui-error")},reset:function(){this.$error.hide();this.$el.removeClass("ui-error")},_refresh:function(){if(!this.field.skip){this.$field.fadeIn("fast");this.$title_optional.html(this.text_disable)}else{this.reset();this.$field.hide();this.$title_optional.html(this.text_enable);this.field.value&&this.field.value(this.default_value)}this.app.trigger("refresh")},_template:function(a){var b='<div class="ui-table-form-element"><div class="ui-table-form-error ui-error"><span class="fa fa-arrow-down"/><span class="ui-table-form-error-text"/></div><div class="ui-table-form-title-strong">';if(a.optional){b+=a.label+'<span>&nbsp[<span class="ui-table-form-title-optional"/>]</span>'}else{b+=a.label}b+='</div><div class="ui-table-form-field">';if(a.help){b+='<div class="ui-table-form-info">'+a.help+"</div>"}b+="</div></div>";return b}})}); \ No newline at end of file Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: martenson: fix error in TS search schema
by commits-noreply＠bitbucket.org 28 Jan '15

28 Jan '15

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/e0b649785f0a/ Changeset: e0b649785f0a User: martenson Date: 2015-01-28 20:25:42+00:00 Summary: fix error in TS search schema Affected #: 1 file diff -r bbebacb3a4e57063ad911d980c67182b6fe44522 -r e0b649785f0a92c1030c42ea826d367ee7d5dc3c lib/galaxy/webapps/tool_shed/search/repo_search.py --- a/lib/galaxy/webapps/tool_shed/search/repo_search.py +++ b/lib/galaxy/webapps/tool_shed/search/repo_search.py @@ -21,14 +21,16 @@ schema = Schema( id = STORED, - name = TEXT( field_boost = 1.7 ), - description = TEXT( field_boost = 1.5 ), - long_description = TEXT, - homepage_url = TEXT, - remote_repository_url = TEXT, - repo_owner_username = TEXT, + name = TEXT( field_boost = 1.7, stored = True ), + description = TEXT( field_boost = 1.5, stored = True ), + long_description = TEXT( stored = True ), + homepage_url = TEXT( stored = True ), + remote_repository_url = TEXT( stored = True ), + repo_owner_username = TEXT( stored = True ), times_downloaded = STORED, - approved = STORED ) + approved = STORED, + last_updated = STORED, + full_last_updated = STORED ) except ImportError, e: search_ready = False Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: martenson: way better search; dont query DB; repair final scoring functions
by commits-noreply＠bitbucket.org 28 Jan '15

28 Jan '15

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/bbebacb3a4e5/ Changeset: bbebacb3a4e5 User: martenson Date: 2015-01-28 20:24:10+00:00 Summary: way better search; dont query DB; repair final scoring functions Affected #: 3 files diff -r bc4403d1f81b560c8e0685eb1093b83ff6b348e4 -r bbebacb3a4e57063ad911d980c67182b6fe44522 lib/galaxy/webapps/tool_shed/api/search.py --- a/lib/galaxy/webapps/tool_shed/api/search.py +++ b/lib/galaxy/webapps/tool_shed/api/search.py @@ -6,7 +6,9 @@ from galaxy.web import _future_expose_api_raw_anonymous as expose_api_raw_anonymous from galaxy.web.base.controller import BaseAPIController from galaxy.webapps.tool_shed.search.repo_search import RepoSearch +from galaxy.web import url_for import json + import logging log = logging.getLogger( __name__ ) @@ -28,7 +30,6 @@ repo_search = RepoSearch() results = repo_search.search( trans, search_term ) - - response = '%s(%s);' % ( kwd.get('callback'), json.dumps(results) ) - log.debug(response) + results[ 'hostname' ] = url_for( '/', qualified = True ) + response = '%s(%s);' % ( kwd.get( 'callback' ), json.dumps( results ) ) return response diff -r bc4403d1f81b560c8e0685eb1093b83ff6b348e4 -r bbebacb3a4e57063ad911d980c67182b6fe44522 lib/galaxy/webapps/tool_shed/search/repo_search.py --- a/lib/galaxy/webapps/tool_shed/search/repo_search.py +++ b/lib/galaxy/webapps/tool_shed/search/repo_search.py @@ -2,10 +2,7 @@ import datetime from galaxy import exceptions from galaxy import eggs -from galaxy.util import pretty_print_time_interval -from galaxy.web.base.controller import BaseAPIController from galaxy.webapps.tool_shed import model -from tool_shed.util.shed_util_common import generate_sharable_link_for_repository_in_tool_shed import logging log = logging.getLogger( __name__ ) @@ -27,11 +24,11 @@ name = TEXT( field_boost = 1.7 ), description = TEXT( field_boost = 1.5 ), long_description = TEXT, - repo_type = TEXT, homepage_url = TEXT, remote_repository_url = TEXT, repo_owner_username = TEXT, - times_downloaded = STORED ) + times_downloaded = STORED, + approved = STORED ) except ImportError, e: search_ready = False @@ -46,13 +43,25 @@ use_final = True def final( self, searcher, docnum, score ): + log.debug('score before: ' + str(score) ) + + # Arbitrary for now + reasonable_hits = 100.0 + times_downloaded = int( searcher.stored_fields( docnum )[ "times_downloaded" ] ) + # Add 1 to prevent 0 being divided + if times_downloaded == 0: + times_downloaded = 1 + popularity_modifier = ( times_downloaded / reasonable_hits ) + log.debug('popularity_modifier: ' + str(popularity_modifier) ) - maxhits = 300 - hitcount = searcher.stored_fields( docnum )[ "times_downloaded" ] - log.debug( 'hitcount: ' + str( hitcount ) ) + cert_modifier = 2 if searcher.stored_fields( docnum )[ "approved" ] == 'yes' else 1 + log.debug('cert_modifier: ' + str(cert_modifier) ) - # Multiply the computed score for this document by the popularity - return score * ( hitcount / maxhits ) + # Adjust the computed score for this document by the popularity + # and by the certification level. + final_score = score * popularity_modifier * cert_modifier + log.debug('score after: ' + str( final_score ) ) + return final_score class RepoSearch( object ): @@ -80,8 +89,7 @@ 'long_description_B' : 0.5, 'homepage_url_B' : 0.3, 'remote_repository_url_B' : 0.2, - 'repo_owner_username' : 0.3, - 'repo_type_B' : 0.1 } ) + 'repo_owner_username' : 0.3 } ) # log.debug(repo_weighting.__dict__) searcher = index.searcher( weighting = repo_weighting ) @@ -90,7 +98,6 @@ 'name', 'description', 'long_description', - 'repo_type', 'homepage_url', 'remote_repository_url', 'repo_owner_username' ], schema = schema ) @@ -99,30 +106,28 @@ user_query = parser.parse( '*' + search_term + '*' ) hits = searcher.search( user_query, terms = True ) + # hits = searcher.search( user_query ) # hits = searcher.search_page( user_query, 1, pagelen = 1, terms = True ) log.debug( 'searching for: #' + str( search_term ) ) log.debug( 'total hits: ' + str( len( hits ) ) ) log.debug( 'scored hits: ' + str( hits.scored_length() ) ) results = {} - results[ 'total_results'] = len( hits ) + results[ 'total_results'] = str( hits.scored_length() ) results[ 'hits' ] = [] for hit in hits: - repo = trans.sa_session.query( model.Repository ).filter_by( id = hit[ 'id' ] ).one() - approved = 'no' - for review in repo.reviews: - if review.approved == 'yes': - approved = 'yes' - break - hit_dict = repo.to_dict( view = 'element', value_mapper = { 'id': trans.security.encode_id, 'user_id': trans.security.encode_id } ) - hit_dict[ 'url'] = generate_sharable_link_for_repository_in_tool_shed( repo ) - - # Format the time since last update to be nicely readable. - time_ago = pretty_print_time_interval( repo.update_time ) - hit_dict[ 'last_updated' ] = time_ago - hit_dict[ 'full_last_updated' ] = repo.update_time.strftime( "%Y-%m-%d %I:%M %p" ) - hit_dict[ 'times_downloaded' ] = repo.times_downloaded - hit_dict[ 'approved' ] = approved - results[ 'hits' ].append( {'repository': hit_dict, 'matched_terms': hit.matched_terms() } ) + hit_dict = {} + hit_dict[ 'id' ] = trans.security.encode_id( hit.get( 'id' ) ) + hit_dict[ 'repo_owner_username' ] = hit.get( 'repo_owner_username' ) + hit_dict[ 'name' ] = hit.get( 'name' ) + hit_dict[ 'long_description' ] = hit.get( 'long_description' ) + hit_dict[ 'remote_repository_url' ] = hit.get( 'remote_repository_url' ) + hit_dict[ 'homepage_url' ] = hit.get( 'homepage_url' ) + hit_dict[ 'description' ] = hit.get( 'description' ) + hit_dict[ 'last_updated' ] = hit.get( 'last_updated' ) + hit_dict[ 'full_last_updated' ] = hit.get( 'full_last_updated' ) + hit_dict[ 'approved' ] = hit.get( 'approved' ) + hit_dict[ 'times_downloaded' ] = hit.get( 'times_downloaded' ) + results[ 'hits' ].append( {'repository': hit_dict, 'matched_terms': hit.matched_terms(), 'score': hit.score } ) return results finally: searcher.close() diff -r bc4403d1f81b560c8e0685eb1093b83ff6b348e4 -r bbebacb3a4e57063ad911d980c67182b6fe44522 scripts/tool_shed/build_ts_whoosh_index.py --- a/scripts/tool_shed/build_ts_whoosh_index.py +++ b/scripts/tool_shed/build_ts_whoosh_index.py @@ -1,21 +1,26 @@ # !/usr/bin/env python """ Build indeces for searching the TS """ -import sys, os, csv, urllib, urllib2, ConfigParser +import sys +import os +import csv +import urllib +import urllib2 +import ConfigParser new_path = [ os.path.join( os.getcwd(), "lib" ) ] new_path.extend( sys.path[1:] ) # remove scripts/ from the path sys.path = new_path +from galaxy.util import pretty_print_time_interval from galaxy import eggs +eggs.require( "SQLAlchemy" ) + +import galaxy.webapps.tool_shed.model.mapping # Whoosh is compatible with Python 2.5+ Try to import Whoosh and set flag to indicate whether search is enabled. try: eggs.require( "Whoosh" ) - import pkg_resources - pkg_resources.require( "SQLAlchemy >= 0.4" ) - import whoosh.index - import galaxy.webapps.tool_shed.model.mapping from whoosh.filedb.filestore import FileStorage from whoosh.fields import Schema, STORED, ID, KEYWORD, TEXT, STORED from whoosh.scoring import BM25F @@ -26,14 +31,16 @@ whoosh_ready = True schema = Schema( id = STORED, - name = TEXT( field_boost = 1.7 ), - description = TEXT( field_boost = 1.5 ), - long_description = TEXT, - repo_type = TEXT, - homepage_url = TEXT, - remote_repository_url = TEXT, - repo_owner_username = TEXT, - times_downloaded = STORED ) + name = TEXT( field_boost = 1.7, stored = True ), + description = TEXT( field_boost = 1.5, stored = True ), + long_description = TEXT( stored = True ), + homepage_url = TEXT( stored = True ), + remote_repository_url = TEXT( stored = True ), + repo_owner_username = TEXT( stored = True ), + times_downloaded = STORED, + approved = STORED, + last_updated = STORED, + full_last_updated = STORED ) except ImportError, e: print 'import error' @@ -51,16 +58,29 @@ return a_basestr repos_indexed = 0 - for id, name, description, long_description, repo_type, homepage_url, remote_repository_url, repo_owner_username, times_downloaded in get_repos( sa_session ): + for ( id, + name, + description, + long_description, + homepage_url, + remote_repository_url, + repo_owner_username, + times_downloaded, + approved, + last_updated, + full_last_updated ) in get_repos( sa_session ): + writer.add_document( id = id, name = to_unicode( name ), description = to_unicode( description ), long_description = to_unicode( long_description ), - repo_type = to_unicode( repo_type ), homepage_url = to_unicode( homepage_url ), remote_repository_url = to_unicode( remote_repository_url ), repo_owner_username = to_unicode( repo_owner_username ), - times_downloaded = times_downloaded ) + times_downloaded = times_downloaded, + approved = approved, + last_updated = last_updated, + full_last_updated = full_last_updated ) repos_indexed += 1 writer.commit() print "Number of repos indexed: ", repos_indexed @@ -71,17 +91,36 @@ name = repo.name description = repo.description long_description = repo.long_description - repo_type = repo.type homepage_url = repo.homepage_url remote_repository_url = repo.remote_repository_url times_downloaded = repo.times_downloaded - repo_owner_username = "" + repo_owner_username = '' if repo.user_id is not None: user = sa_session.query( model.User ).filter( model.User.id == repo.user_id ).one() repo_owner_username = user.username - yield id, name, description, long_description, repo_type, homepage_url, remote_repository_url, repo_owner_username, times_downloaded + approved = 'no' + for review in repo.reviews: + if review.approved == 'yes': + approved = 'yes' + break + + # Format the time since last update to be nicely readable. + last_updated = pretty_print_time_interval( repo.update_time ) + full_last_updated = repo.update_time.strftime( "%Y-%m-%d %I:%M %p" ) + + yield ( id, + name, + description, + long_description, + homepage_url, + remote_repository_url, + repo_owner_username, + times_downloaded, + approved, + last_updated, + full_last_updated ) def get_sa_session_and_needed_config_settings( ini_file ): conf_parser = ConfigParser.ConfigParser( { 'here' : os.getcwd() } ) Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: 3 new changesets
by commits-noreply＠bitbucket.org 28 Jan '15

28 Jan '15

3 new commits in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/7d7419ddad18/ Changeset: 7d7419ddad18 Branch: stable User: natefoo Date: 2015-01-28 15:38:22+00:00 Summary: Allow BAM's set_meta() to use samtools 1 to generate the index, if the samtools found on $PATH is samtools 1. Affected #: 1 file diff -r 837baf8a92add3084c7e230cf537016192a78b55 -r 7d7419ddad18212b60eaf1b5c30196bf9ad2328b lib/galaxy/datatypes/binary.py --- a/lib/galaxy/datatypes/binary.py +++ b/lib/galaxy/datatypes/binary.py @@ -240,10 +240,19 @@ ##$ samtools index ##Usage: samtools index <in.bam> [<out.index>] stderr_name = tempfile.NamedTemporaryFile( prefix = "bam_index_stderr" ).name - command = 'samtools index %s %s' % ( dataset.file_name, index_file.file_name ) - proc = subprocess.Popen( args=command, shell=True, stderr=open( stderr_name, 'wb' ) ) + command = [ 'samtools', 'index', dataset.file_name, index_file.file_name ] + proc = subprocess.Popen( args=command, stderr=open( stderr_name, 'wb' ) ) exit_code = proc.wait() #Did index succeed? + if exit_code == -6: + # SIGABRT, most likely samtools 1.0+ which does not accept the index name parameter. + command = [ 'samtools', 'index', dataset.file_name ] + proc = subprocess.Popen( args=command, stderr=open( stderr_name, 'wb' ) ) + exit_code = proc.wait() + if os.path.exists( os.path.join( dataset.file_name, '.bai' ) ): + shutil.move( os.path.join( dataset.file_name, '.bai' ), index_file.file_name ) + else: + open( stderr_name, 'ab+' ).write( 'Galaxy attempted to build the BAM index with samtools 1.0+ but failed\n') stderr = open( stderr_name ).read().strip() if stderr: if exit_code != 0: https://bitbucket.org/galaxy/galaxy-central/commits/d05da0df53b9/ Changeset: d05da0df53b9 Branch: stable User: natefoo Date: 2015-01-28 16:58:23+00:00 Summary: Use a symlink to the dataset in the same directory as the MetadataTempFile as the input to samtools index so there's no clobber risk. Thanks Dan. Affected #: 1 file diff -r 7d7419ddad18212b60eaf1b5c30196bf9ad2328b -r d05da0df53b98639d753d94e9ffaa1e99c918142 lib/galaxy/datatypes/binary.py --- a/lib/galaxy/datatypes/binary.py +++ b/lib/galaxy/datatypes/binary.py @@ -246,13 +246,18 @@ #Did index succeed? if exit_code == -6: # SIGABRT, most likely samtools 1.0+ which does not accept the index name parameter. - command = [ 'samtools', 'index', dataset.file_name ] - proc = subprocess.Popen( args=command, stderr=open( stderr_name, 'wb' ) ) - exit_code = proc.wait() - if os.path.exists( os.path.join( dataset.file_name, '.bai' ) ): - shutil.move( os.path.join( dataset.file_name, '.bai' ), index_file.file_name ) - else: - open( stderr_name, 'ab+' ).write( 'Galaxy attempted to build the BAM index with samtools 1.0+ but failed\n') + dataset_symlink = os.path.join( os.path.dirname( index_file.file_name ), + '__dataset_%d_%s' % ( dataset.id, os.path.basename( index_file.file_name ) ) ) + os.symlink( dataset.file_name, dataset_symlink ) + try: + command = [ 'samtools', 'index', dataset_symlink ] + proc = subprocess.Popen( args=command, stderr=open( stderr_name, 'wb' ) ) + exit_code = proc.wait() + shutil.move( dataset_symlink + '.bai', index_file.file_name ) + except Exception, e: + open( stderr_name, 'ab+' ).write( 'Galaxy attempted to build the BAM index with samtools 1.0+ but failed: %s\n' % e) + finally: + os.unlink( dataset_symlink ) stderr = open( stderr_name ).read().strip() if stderr: if exit_code != 0: https://bitbucket.org/galaxy/galaxy-central/commits/a481d17c0448/ Changeset: a481d17c0448 Branch: stable User: natefoo Date: 2015-01-28 18:25:11+00:00 Summary: Use subprocess.call() rather than Popen(), thanks Nicola. Affected #: 1 file diff -r d05da0df53b98639d753d94e9ffaa1e99c918142 -r a481d17c04482b069e6b2ebdda4fbca01448fa9c lib/galaxy/datatypes/binary.py --- a/lib/galaxy/datatypes/binary.py +++ b/lib/galaxy/datatypes/binary.py @@ -251,8 +251,7 @@ os.symlink( dataset.file_name, dataset_symlink ) try: command = [ 'samtools', 'index', dataset_symlink ] - proc = subprocess.Popen( args=command, stderr=open( stderr_name, 'wb' ) ) - exit_code = proc.wait() + exit_code = subprocess.call( args=command, stderr=open( stderr_name, 'wb' ) ) shutil.move( dataset_symlink + '.bai', index_file.file_name ) except Exception, e: open( stderr_name, 'ab+' ).write( 'Galaxy attempted to build the BAM index with samtools 1.0+ but failed: %s\n' % e) Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: guerler: Workflow: Disable new tool form
by commits-noreply＠bitbucket.org 28 Jan '15

28 Jan '15

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/bc4403d1f81b/ Changeset: bc4403d1f81b User: guerler Date: 2015-01-28 18:51:51+00:00 Summary: Workflow: Disable new tool form Affected #: 1 file diff -r f0a96002a5538b1b5e2ebafe8f80a64ed299b418 -r bc4403d1f81b560c8e0685eb1093b83ff6b348e4 lib/galaxy/webapps/galaxy/controllers/workflow.py --- a/lib/galaxy/webapps/galaxy/controllers/workflow.py +++ b/lib/galaxy/webapps/galaxy/controllers/workflow.py @@ -618,18 +618,12 @@ 'tool_state': tool_state } ) - # create tool model and default tool state (if missing) - tool_model = None - if not tool_state: - tool_model = module.tool.to_json(trans, **incoming) - module.state.inputs = copy.deepcopy(tool_model['state_inputs']) - # update module state module.update_state( incoming ) if type == 'tool': return { - 'tool_model': tool_model, + 'tool_model': None, 'tool_state': module.get_state(), 'data_inputs': module.get_data_inputs(), 'data_outputs': module.get_data_outputs(), @@ -664,7 +658,7 @@ 'name': module.get_name(), 'tool_id': module.get_tool_id(), 'tool_state': module.get_state(), - 'tool_model': module.tool.to_json(trans), + 'tool_model': None, 'tooltip': module.get_tooltip( static_path=url_for( '/static' ) ), 'data_inputs': module.get_data_inputs(), 'data_outputs': module.get_data_outputs(), Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0