- galaxy-commits - lists.galaxyproject.org

commit/galaxy-central: 3 new changesets
by commits-noreply＠bitbucket.org 23 May '14

23 May '14

3 new commits in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/a87cb6fcd722/ Changeset: a87cb6fcd722 Branch: next-stable User: dannon Date: 2014-05-23 21:02:24 Summary: Remove noodles.png from default splash (and distribution). Affected #: 2 files diff -r 1a43bcbff87c4580ac9d791d1b770e6e80a7c0d7 -r a87cb6fcd72268b454e50dd877c396c08c9ade55 static/images/noodles.png Binary file static/images/noodles.png has changed diff -r 1a43bcbff87c4580ac9d791d1b770e6e80a7c0d7 -r a87cb6fcd72268b454e50dd877c396c08c9ade55 static/welcome.html.sample --- a/static/welcome.html.sample +++ b/static/welcome.html.sample @@ -13,7 +13,6 @@ To customize this page edit <code>static/welcome.html</code></div><br/> - <img src="images/noodles.png" alt="WWFSMD?" style="display: block; margin-left: auto; margin-right: auto;" /><hr/><a target="_blank" class="reference" href="http://galaxyproject.org/">Galaxy</a> is an open, web-based platform for data intensive biomedical research. The <a target="_blank" class="reference" href="http://wiki.galaxyproject.org/GalaxyTeam">Galaxy team</a> is a part of <a target="_blank" class="reference" href="http://www.bx.psu.edu">BX</a> at <a target="_blank" class="reference" href="http://www.psu.edu">Penn State</a>, and the <a target="_blank" class="reference" href="http://www.biology.emory.edu/">Biology</a> and <a target="_blank" class="reference" href="http://www.mathcs.emory.edu/">Mathematics and Computer Science</a> departments at <a target="_blank" class="reference" href="http://www.emory.edu/">Emory University</a>. The <a target="_blank" class="reference" href="http://galaxyproject.org/">Galaxy Project</a> is supported in part by <a target="_blank" class="reference" href="http://www.genome.gov">NHGRI</a>, <a target="_blank" class="reference" href="http://www.nsf.gov">NSF</a>, <a target="_blank" class="reference" href="http://www.huck.psu.edu">The Huck Institutes of the Life Sciences</a>, <a target="_blank" class="reference" href="http://www.ics.psu.edu">The Institute for CyberScience at Penn State</a>, and <a target="_blank" class="reference" href="http://www.emory.edu/">Emory University</a>.</p> https://bitbucket.org/galaxy/galaxy-central/commits/4e1cf622493c/ Changeset: 4e1cf622493c User: dannon Date: 2014-05-23 21:05:28 Summary: Merge next-stable. Affected #: 2 files diff -r 1a38b9cf737f9b54a992e973b124daf73cf32acc -r 4e1cf622493c98cd0fe5082a98872d2a07b6ed85 static/images/noodles.png Binary file static/images/noodles.png has changed diff -r 1a38b9cf737f9b54a992e973b124daf73cf32acc -r 4e1cf622493c98cd0fe5082a98872d2a07b6ed85 static/welcome.html.sample --- a/static/welcome.html.sample +++ b/static/welcome.html.sample @@ -13,7 +13,6 @@ To customize this page edit <code>static/welcome.html</code></div><br/> - <img src="images/noodles.png" alt="WWFSMD?" style="display: block; margin-left: auto; margin-right: auto;" /><hr/><a target="_blank" class="reference" href="http://galaxyproject.org/">Galaxy</a> is an open, web-based platform for data intensive biomedical research. The <a target="_blank" class="reference" href="http://wiki.galaxyproject.org/GalaxyTeam">Galaxy team</a> is a part of <a target="_blank" class="reference" href="http://www.bx.psu.edu">BX</a> at <a target="_blank" class="reference" href="http://www.psu.edu">Penn State</a>, and the <a target="_blank" class="reference" href="http://www.biology.emory.edu/">Biology</a> and <a target="_blank" class="reference" href="http://www.mathcs.emory.edu/">Mathematics and Computer Science</a> departments at <a target="_blank" class="reference" href="http://www.emory.edu/">Emory University</a>. The <a target="_blank" class="reference" href="http://galaxyproject.org/">Galaxy Project</a> is supported in part by <a target="_blank" class="reference" href="http://www.genome.gov">NHGRI</a>, <a target="_blank" class="reference" href="http://www.nsf.gov">NSF</a>, <a target="_blank" class="reference" href="http://www.huck.psu.edu">The Huck Institutes of the Life Sciences</a>, <a target="_blank" class="reference" href="http://www.ics.psu.edu">The Institute for CyberScience at Penn State</a>, and <a target="_blank" class="reference" href="http://www.emory.edu/">Emory University</a>.</p> https://bitbucket.org/galaxy/galaxy-central/commits/3d33ae64235e/ Changeset: 3d33ae64235e User: dannon Date: 2014-05-23 21:06:06 Summary: Merge. Affected #: 9 files diff -r 4e1cf622493c98cd0fe5082a98872d2a07b6ed85 -r 3d33ae64235e910ad882b8893dfdc9100d8a8210 lib/galaxy/security/validate_user_input.py --- a/lib/galaxy/security/validate_user_input.py +++ b/lib/galaxy/security/validate_user_input.py @@ -1,5 +1,8 @@ +import logging import re +log = logging.getLogger( __name__ ) + VALID_PUBLICNAME_RE = re.compile( "^[a-z0-9\-]+$" ) VALID_PUBLICNAME_SUB = re.compile( "[^a-z0-9\-]" ) # Basic regular expression to check email validity. diff -r 4e1cf622493c98cd0fe5082a98872d2a07b6ed85 -r 3d33ae64235e910ad882b8893dfdc9100d8a8210 lib/galaxy/webapps/tool_shed/api/repositories.py --- a/lib/galaxy/webapps/tool_shed/api/repositories.py +++ b/lib/galaxy/webapps/tool_shed/api/repositories.py @@ -8,6 +8,7 @@ from galaxy import web from galaxy.util import json from galaxy.web.base.controller import BaseAPIController +from galaxy.web.base.controller import HTTPBadRequest from galaxy.web.framework.helpers import time_ago import tool_shed.repository_types.util as rt_util import tool_shed.util.shed_util_common as suc @@ -25,6 +26,48 @@ class RepositoriesController( BaseAPIController ): """RESTful controller for interactions with repositories in the Tool Shed.""" + @web.expose_api + def add_repository_registry_entry( self, trans, payload, **kwd ): + """ + POST /api/repository_revisions/add_repository_registry_entry + Adds appropriate entries to the repository registry for the repository defined by the received name and owner. + + :param key: the user's API key + + The following parameters are included in the payload. + :param tool_shed_url (required): the base URL of the Tool Shed containing the Repository + :param name (required): the name of the Repository + :param owner (required): the owner of the Repository + """ + response_dict = {} + if not trans.user_is_admin(): + response_dict[ 'status' ] = 'error' + response_dict[ 'message' ] = "You are not authorized to add entries to this Tool Shed's repository registry." + return response_dict + tool_shed_url = payload.get( 'tool_shed_url', '' ) + if not tool_shed_url: + raise HTTPBadRequest( detail="Missing required parameter 'tool_shed_url'." ) + tool_shed_url = tool_shed_url.rstrip( '/' ) + name = payload.get( 'name', '' ) + if not name: + raise HTTPBadRequest( detail="Missing required parameter 'name'." ) + owner = payload.get( 'owner', '' ) + if not owner: + raise HTTPBadRequest( detail="Missing required parameter 'owner'." ) + repository = suc.get_repository_by_name_and_owner( trans.app, name, owner ) + if repository is None: + error_message = 'Cannot locate repository with name %s and owner %s,' % ( str( name ), str( owner ) ) + log.debug( error_message ) + response_dict[ 'status' ] = 'error' + response_dict[ 'message' ] = error_message + return response_dict + # Update the repository registry. + trans.app.repository_registry.add_entry( repository ) + response_dict[ 'status' ] = 'ok' + response_dict[ 'message' ] = 'Entries for repository %s owned by %s have been added to the Tool Shed repository registry..' \ + % ( name, owner ) + return response_dict + @web.expose_api_anonymous def get_ordered_installable_revisions( self, trans, name, owner, **kwd ): """ @@ -269,6 +312,48 @@ return repository_dicts @web.expose_api + def remove_repository_registry_entry( self, trans, payload, **kwd ): + """ + POST /api/repository_revisions/add_repository_registry_entry + Removes appropriate entries from the repository registry for the repository defined by the received name and owner. + + :param key: the user's API key + + The following parameters are included in the payload. + :param tool_shed_url (required): the base URL of the Tool Shed containing the Repository + :param name (required): the name of the Repository + :param owner (required): the owner of the Repository + """ + response_dict = {} + if not trans.user_is_admin(): + response_dict[ 'status' ] = 'error' + response_dict[ 'message' ] = "You are not authorized to remove entries from this Tool Shed's repository registry." + return response_dict + tool_shed_url = payload.get( 'tool_shed_url', '' ) + if not tool_shed_url: + raise HTTPBadRequest( detail="Missing required parameter 'tool_shed_url'." ) + tool_shed_url = tool_shed_url.rstrip( '/' ) + name = payload.get( 'name', '' ) + if not name: + raise HTTPBadRequest( detail="Missing required parameter 'name'." ) + owner = payload.get( 'owner', '' ) + if not owner: + raise HTTPBadRequest( detail="Missing required parameter 'owner'." ) + repository = suc.get_repository_by_name_and_owner( trans.app, name, owner ) + if repository is None: + error_message = 'Cannot locate repository with name %s and owner %s,' % ( str( name ), str( owner ) ) + log.debug( error_message ) + response_dict[ 'status' ] = 'error' + response_dict[ 'message' ] = error_message + return response_dict + # Update the repository registry. + trans.app.repository_registry.remove_entry( repository ) + response_dict[ 'status' ] = 'ok' + response_dict[ 'message' ] = 'Entries for repository %s owned by %s have been removed from the Tool Shed repository registry.' \ + % ( name, owner ) + return response_dict + + @web.expose_api def repository_ids_for_setting_metadata( self, trans, my_writable=False, **kwd ): """ GET /api/repository_ids_for_setting_metadata diff -r 4e1cf622493c98cd0fe5082a98872d2a07b6ed85 -r 3d33ae64235e910ad882b8893dfdc9100d8a8210 lib/galaxy/webapps/tool_shed/api/repository_revisions.py --- a/lib/galaxy/webapps/tool_shed/api/repository_revisions.py +++ b/lib/galaxy/webapps/tool_shed/api/repository_revisions.py @@ -232,6 +232,8 @@ """ PUT /api/repository_revisions/{encoded_repository_metadata_id}/{payload} Updates the value of specified columns of the repository_metadata table based on the key / value pairs in payload. + + :param id: the encoded id of the `RepositoryMetadata` object """ repository_metadata_id = kwd.get( 'id', None ) if repository_metadata_id is None: diff -r 4e1cf622493c98cd0fe5082a98872d2a07b6ed85 -r 3d33ae64235e910ad882b8893dfdc9100d8a8210 lib/galaxy/webapps/tool_shed/buildapp.py --- a/lib/galaxy/webapps/tool_shed/buildapp.py +++ b/lib/galaxy/webapps/tool_shed/buildapp.py @@ -94,8 +94,10 @@ webapp.mapper.resource( 'repository', 'repositories', controller='repositories', - collection={ 'get_repository_revision_install_info' : 'GET', + collection={ 'add_repository_registry_entry' : 'POST', + 'get_repository_revision_install_info' : 'GET', 'get_ordered_installable_revisions' : 'GET', + 'remove_repository_registry_entry' : 'POST', 'repository_ids_for_setting_metadata' : 'GET', 'reset_metadata_on_repositories' : 'POST', 'reset_metadata_on_repository' : 'POST' }, diff -r 4e1cf622493c98cd0fe5082a98872d2a07b6ed85 -r 3d33ae64235e910ad882b8893dfdc9100d8a8210 lib/tool_shed/repository_registry.py --- a/lib/tool_shed/repository_registry.py +++ b/lib/tool_shed/repository_registry.py @@ -41,28 +41,34 @@ self.load_viewable_repositories_and_suites_by_category() self.load_repository_and_suite_tuples() - def add_entry( self, repository ): - if repository: - is_valid = self.is_valid( repository ) - for rca in repository.categories: - category = rca.category - category_name = str( category.name ) - self.viewable_repositories_and_suites_by_category[ category_name ] += 1 - if is_valid: - self.viewable_valid_repositories_and_suites_by_category[ category_name ] += 1 - if repository.type == rt_util.REPOSITORY_SUITE_DEFINITION: - self.viewable_suites_by_category[ category_name ] += 1 + def add_entry( self, repository ): + try: + if repository: + is_valid = self.is_valid( repository ) + certified_level_one_tuple = self.get_certified_level_one_tuple( repository ) + latest_installable_changeset_revision, is_level_one_certified = certified_level_one_tuple + for rca in repository.categories: + category = rca.category + category_name = str( category.name ) + self.viewable_repositories_and_suites_by_category[ category_name ] += 1 if is_valid: - self.viewable_valid_suites_by_category[ category_name ] += 1 - certified_level_one_tuple = self.get_certified_level_one_tuple( repository ) - latest_installable_changeset_revision, is_level_one_certified = certified_level_one_tuple - if is_level_one_certified: - self.certified_level_one_viewable_repositories_and_suites_by_category[ category_name ] += 1 - if repository.type == rt_util.REPOSITORY_SUITE_DEFINITION: - self.certified_level_one_viewable_suites_by_category[ category_name ] += 1 - self.load_repository_and_suite_tuple( repository ) - if is_level_one_certified: - self.load_certified_level_one_repository_and_suite_tuple( repository ) + self.viewable_valid_repositories_and_suites_by_category[ category_name ] += 1 + if repository.type == rt_util.REPOSITORY_SUITE_DEFINITION: + self.viewable_suites_by_category[ category_name ] += 1 + if is_valid: + self.viewable_valid_suites_by_category[ category_name ] += 1 + if is_level_one_certified: + self.certified_level_one_viewable_repositories_and_suites_by_category[ category_name ] += 1 + if repository.type == rt_util.REPOSITORY_SUITE_DEFINITION: + self.certified_level_one_viewable_suites_by_category[ category_name ] += 1 + self.load_repository_and_suite_tuple( repository ) + if is_level_one_certified: + self.load_certified_level_one_repository_and_suite_tuple( repository ) + except Exception, e: + # The viewable repository numbers and the categorized (filtered) lists of repository tuples + # may be slightly skewed, but that is no reason to result in a potential server error. All + # will be corrected at next server start. + log.exception( "Handled error adding entry to repository registry: %s." % str( e ) ) def get_certified_level_one_clause_list( self ): certified_level_one_tuples = [] @@ -242,28 +248,34 @@ if repository.type in [ rt_util.REPOSITORY_SUITE_DEFINITION ]: self.certified_level_one_viewable_suites_by_category[ category_name ] += 1 - def remove_entry( self, repository ): - if repository: - is_valid = self.is_valid( repository ) - for rca in repository.categories: - category = rca.category - category_name = str( category.name ) - self.viewable_repositories_and_suites_by_category[ category_name ] -= 1 - if is_valid: - self.viewable_valid_repositories_and_suites_by_category[ category_name ] -= 1 - if repository.type == rt_util.REPOSITORY_SUITE_DEFINITION: - self.viewable_suites_by_category[ category_name ] -= 1 + def remove_entry( self, repository ): + try: + if repository: + is_valid = self.is_valid( repository ) + certified_level_one_tuple = self.get_certified_level_one_tuple( repository ) + latest_installable_changeset_revision, is_level_one_certified = certified_level_one_tuple + for rca in repository.categories: + category = rca.category + category_name = str( category.name ) + self.viewable_repositories_and_suites_by_category[ category_name ] -= 1 if is_valid: - self.viewable_valid_suites_by_category[ category_name ] -= 1 - certified_level_one_tuple = self.get_certified_level_one_tuple( repository ) - latest_installable_changeset_revision, is_level_one_certified = certified_level_one_tuple - if is_level_one_certified: - self.certified_level_one_viewable_repositories_and_suites_by_category[ category_name ] -= 1 - if repository.type == rt_util.REPOSITORY_SUITE_DEFINITION: - self.certified_level_one_viewable_suites_by_category[ category_name ] -= 1 - self.unload_repository_and_suite_tuple( repository ) - if is_level_one_certified: - self.unload_certified_level_one_repository_and_suite_tuple( repository ) + self.viewable_valid_repositories_and_suites_by_category[ category_name ] -= 1 + if repository.type == rt_util.REPOSITORY_SUITE_DEFINITION: + self.viewable_suites_by_category[ category_name ] -= 1 + if is_valid: + self.viewable_valid_suites_by_category[ category_name ] -= 1 + if is_level_one_certified: + self.certified_level_one_viewable_repositories_and_suites_by_category[ category_name ] -= 1 + if repository.type == rt_util.REPOSITORY_SUITE_DEFINITION: + self.certified_level_one_viewable_suites_by_category[ category_name ] -= 1 + self.unload_repository_and_suite_tuple( repository ) + if is_level_one_certified: + self.unload_certified_level_one_repository_and_suite_tuple( repository ) + except Exception, e: + # The viewable repository numbers and the categorized (filtered) lists of repository tuples + # may be slightly skewed, but that is no reason to result in a potential server error. All + # will be corrected at next server start. + log.exception( "Handled error removing entry from repository registry: %s." % str( e ) ) @property def sa_session( self ): diff -r 4e1cf622493c98cd0fe5082a98872d2a07b6ed85 -r 3d33ae64235e910ad882b8893dfdc9100d8a8210 lib/tool_shed/scripts/api/add_repository_registry_entry.py --- /dev/null +++ b/lib/tool_shed/scripts/api/add_repository_registry_entry.py @@ -0,0 +1,40 @@ +#!/usr/bin/env python +""" +Add appropriate entries to the Tool Shed's repository registry for a specified repository. + +Here is a working example of how to use this script. +python ./add_repository_registry_entry.py -a <api key> -u <tool shed url> -n <repository name> -o <repository owner> +""" + +import os +import sys +import argparse +import urllib2 +sys.path.insert( 0, os.path.dirname( __file__ ) ) +from common import submit + +def main( options ): + api_key = options.api_key + if api_key: + if options.tool_shed_url and options.name and options.owner: + base_tool_shed_url = options.tool_shed_url.rstrip( '/' ) + data = {} + data[ 'tool_shed_url' ] = base_tool_shed_url + data[ 'name' ] = options.name + data[ 'owner' ] = options.owner + url = '%s%s' % ( base_tool_shed_url, '/api/repositories/add_repository_registry_entry' ) + response_dict = submit( url, data, api_key=api_key, return_formatted=False ) + print response_dict + else: + print "Invalid tool_shed: ", base_tool_shed_url, " name: ", name, " or owner: ", owner, "." + else: + print "An API key for an admin user in the Tool Shed is required to add entries into the Tool Shed's repository registry." + +if __name__ == '__main__': + parser = argparse.ArgumentParser( description='Add entries into the Tool Shed repository registry for a specified repository.' ) + parser.add_argument( "-a", "--api_key", dest="api_key", required=True, help="API Key for user adding entries into the Tool Shed's repository registry." ) + parser.add_argument( "-u", "--url", dest="tool_shed_url", required=True, help="Tool Shed URL" ) + parser.add_argument( "-n", "--name", dest='name', required=True, help="Repository name." ) + parser.add_argument( "-o", "--owner", dest='owner', required=True, help="Repository owner." ) + options = parser.parse_args() + main( options ) diff -r 4e1cf622493c98cd0fe5082a98872d2a07b6ed85 -r 3d33ae64235e910ad882b8893dfdc9100d8a8210 lib/tool_shed/scripts/api/remove_repository_registry_entry.py --- /dev/null +++ b/lib/tool_shed/scripts/api/remove_repository_registry_entry.py @@ -0,0 +1,40 @@ +#!/usr/bin/env python +""" +Remove appropriate entries from the Tool Shed's repository registry for a specified repository. + +Here is a working example of how to use this script. +python ./remove_repository_registry_entry.py -a <api key> -u <tool shed url> -n <repository name> -o <repository owner> +""" + +import os +import sys +import argparse +import urllib2 +sys.path.insert( 0, os.path.dirname( __file__ ) ) +from common import submit + +def main( options ): + api_key = options.api_key + if api_key: + if options.tool_shed_url and options.name and options.owner: + base_tool_shed_url = options.tool_shed_url.rstrip( '/' ) + data = {} + data[ 'tool_shed_url' ] = base_tool_shed_url + data[ 'name' ] = options.name + data[ 'owner' ] = options.owner + url = '%s%s' % ( base_tool_shed_url, '/api/repositories/remove_repository_registry_entry' ) + response_dict = submit( url, data, api_key=api_key, return_formatted=False ) + print response_dict + else: + print "Invalid tool_shed: ", base_tool_shed_url, " name: ", name, " or owner: ", owner, "." + else: + print "An API key for an admin user in the Tool Shed is required to remove entries from the Tool Shed's repository registry." + +if __name__ == '__main__': + parser = argparse.ArgumentParser( description='Remove entries from the Tool Shed repository registry for a specified repository.' ) + parser.add_argument( "-a", "--api_key", dest="api_key", required=True, help="API Key for user removing entries from the Tool Shed's repository registry." ) + parser.add_argument( "-u", "--url", dest="tool_shed_url", required=True, help="Tool Shed URL" ) + parser.add_argument( "-n", "--name", dest='name', required=True, help="Repository name." ) + parser.add_argument( "-o", "--owner", dest='owner', required=True, help="Repository owner." ) + options = parser.parse_args() + main( options ) diff -r 4e1cf622493c98cd0fe5082a98872d2a07b6ed85 -r 3d33ae64235e910ad882b8893dfdc9100d8a8210 test/install_and_test_tool_shed_repositories/base/util.py --- a/test/install_and_test_tool_shed_repositories/base/util.py +++ b/test/install_and_test_tool_shed_repositories/base/util.py @@ -13,6 +13,7 @@ eggs.require( 'mercurial' ) eggs.require( "nose" ) +import json import logging import install_and_test_tool_shed_repositories.base.test_db_util as test_db_util import install_and_test_tool_shed_repositories.functional.test_install_repositories as test_install_repositories @@ -31,12 +32,12 @@ from common import get_latest_downloadable_changeset_revision_via_api from common import get_repository_dict from common import json_from_url +from common import submit from common import update from galaxy.util import asbool from galaxy.util import listify from galaxy.util import unicodify -from galaxy.util.json import to_json_string import galaxy.webapps.tool_shed.model.mapping from nose.plugins import Plugin @@ -691,8 +692,8 @@ else: xml_element = tool_shed for reason_section in xml_element: - reason_text = reason_section.find( 'text' ) - if reason_text: + reason_text = reason_section.find( 'text', None ) + if reason_text is not None: reason = str( reason_text.text ) else: reason = 'No reason provided.' @@ -985,7 +986,7 @@ """ if shed_tools_dict is None: shed_tools_dict = {} - file( galaxy_shed_tools_dict_file, 'w' ).write( to_json_string( shed_tools_dict ) ) + file( galaxy_shed_tools_dict_file, 'w' ).write( json.dumps( shed_tools_dict ) ) def print_install_and_test_results( install_stage_type, install_and_test_statistics_dict, error_message ): "Print statistics for the current test run." @@ -1081,11 +1082,34 @@ owner = repository_dict.get( 'owner', None ) changeset_revision = repository_dict.get( 'changeset_revision', None ) if name is None or owner is None or changeset_revision is None: - print 'Entries for name, owner or changeset_revision missing from repository_dict %s' % repository_dict + print 'Entries for name: ', name, ' owner: ', owner, ' or changeset_revision: ', changeset_revision, \ + ' missing from repository_dict:' % repository_dict else: name = str( name ) owner = str( owner ) changeset_revision = str( changeset_revision ) + # With regard to certification level one, the status of this repository may or may not have changed between + # this install and test run and the previous install and test run. Rather than attempting to determine if + # anything has changed here, we'll let the Tool Shed's repository registry handle the process of proper + # categorization. To enable this, we'll just remove entries from the Tool Shed's repository registry and + # then add them back. This will ensure proper categorization for this repository. + registry_params = dict( tool_shed_url=galaxy_tool_shed_url, name=name, owner=owner ) + print "Removing entries for repository ", name, " owned by ", owner, "from the Tool Shed's repository registry." + url = '%s' % ( common_util.url_join( galaxy_tool_shed_url, 'api', 'repositories', 'remove_repository_registry_entry' ) ) + response_dict = submit( url, registry_params, api_key=tool_shed_api_key, return_formatted=False ) + status = response_dict.get( 'status', 'ok' ) + if status == 'error': + default_message = 'An unknown error occurred attempting to remove entries from the repository registry.' + error_message = response_dict.get( 'message', default_message ) + print error_message + print "Adding entries for repository ", name, " owned by ", owner, "into the Tool Shed's repository registry." + url = '%s' % ( common_util.url_join( galaxy_tool_shed_url, 'api', 'repositories', 'add_repository_registry_entry' ) ) + response_dict = submit( url, registry_params, api_key=tool_shed_api_key, return_formatted=False ) + status = response_dict.get( 'status', 'ok' ) + if status == 'error': + default_message = 'An unknown error occurred attempting to add entries into the repository registry.' + error_message = response_dict.get( 'message', default_message ) + print error_message print '\n=============================================================\n' print 'Inserting the following into tool_test_results for revision %s of repository %s owned by %s:\n%s' % \ ( changeset_revision, name, owner, str( tool_test_results_dict ) ) @@ -1094,7 +1118,7 @@ params[ 'tool_test_results' ] = tool_test_results_dicts # Set the time_last_tested entry so that the repository_metadata.time_last_tested will be set in the tool shed. params[ 'time_last_tested' ] = 'This entry will result in this value being set via the Tool Shed API.' - url = '%s' % ( common_util.url_join( galaxy_tool_shed_url,'api', 'repository_revisions', str( metadata_revision_id ) ) ) + url = '%s' % ( common_util.url_join( galaxy_tool_shed_url, 'api', 'repository_revisions', str( metadata_revision_id ) ) ) print 'url: ', url print 'params: ', params try: diff -r 4e1cf622493c98cd0fe5082a98872d2a07b6ed85 -r 3d33ae64235e910ad882b8893dfdc9100d8a8210 test/install_and_test_tool_shed_repositories/functional/test_install_repositories.py --- a/test/install_and_test_tool_shed_repositories/functional/test_install_repositories.py +++ b/test/install_and_test_tool_shed_repositories/functional/test_install_repositories.py @@ -11,8 +11,10 @@ def do_install( self, repository_dict ): self.logout() - self.login( email='test(a)bx.psu.edu', username='test' ) - admin_user = test_db_util.get_user( 'test(a)bx.psu.edu' ) + admin_email = 'test(a)bx.psu.edu' + admin_username = 'test' + self.login( email=admin_email, username=admin_username ) + admin_user = test_db_util.get_user( admin_email ) assert admin_user is not None, 'Problem retrieving user with email %s from the database' % admin_email admin_user_private_role = test_db_util.get_private_role( admin_user ) # Install the repository through the web interface using twill. The install_repository() method may Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: greg: Enhance the Tool Shed API as well as the Tool Shed's install and test framework in order to integrate the latter into the Tool Shed's new repository registry.
by commits-noreply＠bitbucket.org 23 May '14

23 May '14

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/af60bdf0d4ad/ Changeset: af60bdf0d4ad User: greg Date: 2014-05-23 19:18:06 Summary: Enhance the Tool Shed API as well as the Tool Shed's install and test framework in order to integrate the latter into the Tool Shed's new repository registry. Affected #: 9 files diff -r 1a38b9cf737f9b54a992e973b124daf73cf32acc -r af60bdf0d4add74f225d163eca4438ab7ccc0e52 lib/galaxy/security/validate_user_input.py --- a/lib/galaxy/security/validate_user_input.py +++ b/lib/galaxy/security/validate_user_input.py @@ -1,5 +1,8 @@ +import logging import re +log = logging.getLogger( __name__ ) + VALID_PUBLICNAME_RE = re.compile( "^[a-z0-9\-]+$" ) VALID_PUBLICNAME_SUB = re.compile( "[^a-z0-9\-]" ) # Basic regular expression to check email validity. diff -r 1a38b9cf737f9b54a992e973b124daf73cf32acc -r af60bdf0d4add74f225d163eca4438ab7ccc0e52 lib/galaxy/webapps/tool_shed/api/repositories.py --- a/lib/galaxy/webapps/tool_shed/api/repositories.py +++ b/lib/galaxy/webapps/tool_shed/api/repositories.py @@ -8,6 +8,7 @@ from galaxy import web from galaxy.util import json from galaxy.web.base.controller import BaseAPIController +from galaxy.web.base.controller import HTTPBadRequest from galaxy.web.framework.helpers import time_ago import tool_shed.repository_types.util as rt_util import tool_shed.util.shed_util_common as suc @@ -25,6 +26,48 @@ class RepositoriesController( BaseAPIController ): """RESTful controller for interactions with repositories in the Tool Shed.""" + @web.expose_api + def add_repository_registry_entry( self, trans, payload, **kwd ): + """ + POST /api/repository_revisions/add_repository_registry_entry + Adds appropriate entries to the repository registry for the repository defined by the received name and owner. + + :param key: the user's API key + + The following parameters are included in the payload. + :param tool_shed_url (required): the base URL of the Tool Shed containing the Repository + :param name (required): the name of the Repository + :param owner (required): the owner of the Repository + """ + response_dict = {} + if not trans.user_is_admin(): + response_dict[ 'status' ] = 'error' + response_dict[ 'message' ] = "You are not authorized to add entries to this Tool Shed's repository registry." + return response_dict + tool_shed_url = payload.get( 'tool_shed_url', '' ) + if not tool_shed_url: + raise HTTPBadRequest( detail="Missing required parameter 'tool_shed_url'." ) + tool_shed_url = tool_shed_url.rstrip( '/' ) + name = payload.get( 'name', '' ) + if not name: + raise HTTPBadRequest( detail="Missing required parameter 'name'." ) + owner = payload.get( 'owner', '' ) + if not owner: + raise HTTPBadRequest( detail="Missing required parameter 'owner'." ) + repository = suc.get_repository_by_name_and_owner( trans.app, name, owner ) + if repository is None: + error_message = 'Cannot locate repository with name %s and owner %s,' % ( str( name ), str( owner ) ) + log.debug( error_message ) + response_dict[ 'status' ] = 'error' + response_dict[ 'message' ] = error_message + return response_dict + # Update the repository registry. + trans.app.repository_registry.add_entry( repository ) + response_dict[ 'status' ] = 'ok' + response_dict[ 'message' ] = 'Entries for repository %s owned by %s have been added to the Tool Shed repository registry..' \ + % ( name, owner ) + return response_dict + @web.expose_api_anonymous def get_ordered_installable_revisions( self, trans, name, owner, **kwd ): """ @@ -269,6 +312,48 @@ return repository_dicts @web.expose_api + def remove_repository_registry_entry( self, trans, payload, **kwd ): + """ + POST /api/repository_revisions/add_repository_registry_entry + Removes appropriate entries from the repository registry for the repository defined by the received name and owner. + + :param key: the user's API key + + The following parameters are included in the payload. + :param tool_shed_url (required): the base URL of the Tool Shed containing the Repository + :param name (required): the name of the Repository + :param owner (required): the owner of the Repository + """ + response_dict = {} + if not trans.user_is_admin(): + response_dict[ 'status' ] = 'error' + response_dict[ 'message' ] = "You are not authorized to remove entries from this Tool Shed's repository registry." + return response_dict + tool_shed_url = payload.get( 'tool_shed_url', '' ) + if not tool_shed_url: + raise HTTPBadRequest( detail="Missing required parameter 'tool_shed_url'." ) + tool_shed_url = tool_shed_url.rstrip( '/' ) + name = payload.get( 'name', '' ) + if not name: + raise HTTPBadRequest( detail="Missing required parameter 'name'." ) + owner = payload.get( 'owner', '' ) + if not owner: + raise HTTPBadRequest( detail="Missing required parameter 'owner'." ) + repository = suc.get_repository_by_name_and_owner( trans.app, name, owner ) + if repository is None: + error_message = 'Cannot locate repository with name %s and owner %s,' % ( str( name ), str( owner ) ) + log.debug( error_message ) + response_dict[ 'status' ] = 'error' + response_dict[ 'message' ] = error_message + return response_dict + # Update the repository registry. + trans.app.repository_registry.remove_entry( repository ) + response_dict[ 'status' ] = 'ok' + response_dict[ 'message' ] = 'Entries for repository %s owned by %s have been removed from the Tool Shed repository registry.' \ + % ( name, owner ) + return response_dict + + @web.expose_api def repository_ids_for_setting_metadata( self, trans, my_writable=False, **kwd ): """ GET /api/repository_ids_for_setting_metadata diff -r 1a38b9cf737f9b54a992e973b124daf73cf32acc -r af60bdf0d4add74f225d163eca4438ab7ccc0e52 lib/galaxy/webapps/tool_shed/api/repository_revisions.py --- a/lib/galaxy/webapps/tool_shed/api/repository_revisions.py +++ b/lib/galaxy/webapps/tool_shed/api/repository_revisions.py @@ -232,6 +232,8 @@ """ PUT /api/repository_revisions/{encoded_repository_metadata_id}/{payload} Updates the value of specified columns of the repository_metadata table based on the key / value pairs in payload. + + :param id: the encoded id of the `RepositoryMetadata` object """ repository_metadata_id = kwd.get( 'id', None ) if repository_metadata_id is None: diff -r 1a38b9cf737f9b54a992e973b124daf73cf32acc -r af60bdf0d4add74f225d163eca4438ab7ccc0e52 lib/galaxy/webapps/tool_shed/buildapp.py --- a/lib/galaxy/webapps/tool_shed/buildapp.py +++ b/lib/galaxy/webapps/tool_shed/buildapp.py @@ -94,8 +94,10 @@ webapp.mapper.resource( 'repository', 'repositories', controller='repositories', - collection={ 'get_repository_revision_install_info' : 'GET', + collection={ 'add_repository_registry_entry' : 'POST', + 'get_repository_revision_install_info' : 'GET', 'get_ordered_installable_revisions' : 'GET', + 'remove_repository_registry_entry' : 'POST', 'repository_ids_for_setting_metadata' : 'GET', 'reset_metadata_on_repositories' : 'POST', 'reset_metadata_on_repository' : 'POST' }, diff -r 1a38b9cf737f9b54a992e973b124daf73cf32acc -r af60bdf0d4add74f225d163eca4438ab7ccc0e52 lib/tool_shed/repository_registry.py --- a/lib/tool_shed/repository_registry.py +++ b/lib/tool_shed/repository_registry.py @@ -41,28 +41,34 @@ self.load_viewable_repositories_and_suites_by_category() self.load_repository_and_suite_tuples() - def add_entry( self, repository ): - if repository: - is_valid = self.is_valid( repository ) - for rca in repository.categories: - category = rca.category - category_name = str( category.name ) - self.viewable_repositories_and_suites_by_category[ category_name ] += 1 - if is_valid: - self.viewable_valid_repositories_and_suites_by_category[ category_name ] += 1 - if repository.type == rt_util.REPOSITORY_SUITE_DEFINITION: - self.viewable_suites_by_category[ category_name ] += 1 + def add_entry( self, repository ): + try: + if repository: + is_valid = self.is_valid( repository ) + certified_level_one_tuple = self.get_certified_level_one_tuple( repository ) + latest_installable_changeset_revision, is_level_one_certified = certified_level_one_tuple + for rca in repository.categories: + category = rca.category + category_name = str( category.name ) + self.viewable_repositories_and_suites_by_category[ category_name ] += 1 if is_valid: - self.viewable_valid_suites_by_category[ category_name ] += 1 - certified_level_one_tuple = self.get_certified_level_one_tuple( repository ) - latest_installable_changeset_revision, is_level_one_certified = certified_level_one_tuple - if is_level_one_certified: - self.certified_level_one_viewable_repositories_and_suites_by_category[ category_name ] += 1 - if repository.type == rt_util.REPOSITORY_SUITE_DEFINITION: - self.certified_level_one_viewable_suites_by_category[ category_name ] += 1 - self.load_repository_and_suite_tuple( repository ) - if is_level_one_certified: - self.load_certified_level_one_repository_and_suite_tuple( repository ) + self.viewable_valid_repositories_and_suites_by_category[ category_name ] += 1 + if repository.type == rt_util.REPOSITORY_SUITE_DEFINITION: + self.viewable_suites_by_category[ category_name ] += 1 + if is_valid: + self.viewable_valid_suites_by_category[ category_name ] += 1 + if is_level_one_certified: + self.certified_level_one_viewable_repositories_and_suites_by_category[ category_name ] += 1 + if repository.type == rt_util.REPOSITORY_SUITE_DEFINITION: + self.certified_level_one_viewable_suites_by_category[ category_name ] += 1 + self.load_repository_and_suite_tuple( repository ) + if is_level_one_certified: + self.load_certified_level_one_repository_and_suite_tuple( repository ) + except Exception, e: + # The viewable repository numbers and the categorized (filtered) lists of repository tuples + # may be slightly skewed, but that is no reason to result in a potential server error. All + # will be corrected at next server start. + log.exception( "Handled error adding entry to repository registry: %s." % str( e ) ) def get_certified_level_one_clause_list( self ): certified_level_one_tuples = [] @@ -242,28 +248,34 @@ if repository.type in [ rt_util.REPOSITORY_SUITE_DEFINITION ]: self.certified_level_one_viewable_suites_by_category[ category_name ] += 1 - def remove_entry( self, repository ): - if repository: - is_valid = self.is_valid( repository ) - for rca in repository.categories: - category = rca.category - category_name = str( category.name ) - self.viewable_repositories_and_suites_by_category[ category_name ] -= 1 - if is_valid: - self.viewable_valid_repositories_and_suites_by_category[ category_name ] -= 1 - if repository.type == rt_util.REPOSITORY_SUITE_DEFINITION: - self.viewable_suites_by_category[ category_name ] -= 1 + def remove_entry( self, repository ): + try: + if repository: + is_valid = self.is_valid( repository ) + certified_level_one_tuple = self.get_certified_level_one_tuple( repository ) + latest_installable_changeset_revision, is_level_one_certified = certified_level_one_tuple + for rca in repository.categories: + category = rca.category + category_name = str( category.name ) + self.viewable_repositories_and_suites_by_category[ category_name ] -= 1 if is_valid: - self.viewable_valid_suites_by_category[ category_name ] -= 1 - certified_level_one_tuple = self.get_certified_level_one_tuple( repository ) - latest_installable_changeset_revision, is_level_one_certified = certified_level_one_tuple - if is_level_one_certified: - self.certified_level_one_viewable_repositories_and_suites_by_category[ category_name ] -= 1 - if repository.type == rt_util.REPOSITORY_SUITE_DEFINITION: - self.certified_level_one_viewable_suites_by_category[ category_name ] -= 1 - self.unload_repository_and_suite_tuple( repository ) - if is_level_one_certified: - self.unload_certified_level_one_repository_and_suite_tuple( repository ) + self.viewable_valid_repositories_and_suites_by_category[ category_name ] -= 1 + if repository.type == rt_util.REPOSITORY_SUITE_DEFINITION: + self.viewable_suites_by_category[ category_name ] -= 1 + if is_valid: + self.viewable_valid_suites_by_category[ category_name ] -= 1 + if is_level_one_certified: + self.certified_level_one_viewable_repositories_and_suites_by_category[ category_name ] -= 1 + if repository.type == rt_util.REPOSITORY_SUITE_DEFINITION: + self.certified_level_one_viewable_suites_by_category[ category_name ] -= 1 + self.unload_repository_and_suite_tuple( repository ) + if is_level_one_certified: + self.unload_certified_level_one_repository_and_suite_tuple( repository ) + except Exception, e: + # The viewable repository numbers and the categorized (filtered) lists of repository tuples + # may be slightly skewed, but that is no reason to result in a potential server error. All + # will be corrected at next server start. + log.exception( "Handled error removing entry from repository registry: %s." % str( e ) ) @property def sa_session( self ): diff -r 1a38b9cf737f9b54a992e973b124daf73cf32acc -r af60bdf0d4add74f225d163eca4438ab7ccc0e52 lib/tool_shed/scripts/api/add_repository_registry_entry.py --- /dev/null +++ b/lib/tool_shed/scripts/api/add_repository_registry_entry.py @@ -0,0 +1,40 @@ +#!/usr/bin/env python +""" +Add appropriate entries to the Tool Shed's repository registry for a specified repository. + +Here is a working example of how to use this script. +python ./add_repository_registry_entry.py -a <api key> -u <tool shed url> -n <repository name> -o <repository owner> +""" + +import os +import sys +import argparse +import urllib2 +sys.path.insert( 0, os.path.dirname( __file__ ) ) +from common import submit + +def main( options ): + api_key = options.api_key + if api_key: + if options.tool_shed_url and options.name and options.owner: + base_tool_shed_url = options.tool_shed_url.rstrip( '/' ) + data = {} + data[ 'tool_shed_url' ] = base_tool_shed_url + data[ 'name' ] = options.name + data[ 'owner' ] = options.owner + url = '%s%s' % ( base_tool_shed_url, '/api/repositories/add_repository_registry_entry' ) + response_dict = submit( url, data, api_key=api_key, return_formatted=False ) + print response_dict + else: + print "Invalid tool_shed: ", base_tool_shed_url, " name: ", name, " or owner: ", owner, "." + else: + print "An API key for an admin user in the Tool Shed is required to add entries into the Tool Shed's repository registry." + +if __name__ == '__main__': + parser = argparse.ArgumentParser( description='Add entries into the Tool Shed repository registry for a specified repository.' ) + parser.add_argument( "-a", "--api_key", dest="api_key", required=True, help="API Key for user adding entries into the Tool Shed's repository registry." ) + parser.add_argument( "-u", "--url", dest="tool_shed_url", required=True, help="Tool Shed URL" ) + parser.add_argument( "-n", "--name", dest='name', required=True, help="Repository name." ) + parser.add_argument( "-o", "--owner", dest='owner', required=True, help="Repository owner." ) + options = parser.parse_args() + main( options ) diff -r 1a38b9cf737f9b54a992e973b124daf73cf32acc -r af60bdf0d4add74f225d163eca4438ab7ccc0e52 lib/tool_shed/scripts/api/remove_repository_registry_entry.py --- /dev/null +++ b/lib/tool_shed/scripts/api/remove_repository_registry_entry.py @@ -0,0 +1,40 @@ +#!/usr/bin/env python +""" +Remove appropriate entries from the Tool Shed's repository registry for a specified repository. + +Here is a working example of how to use this script. +python ./remove_repository_registry_entry.py -a <api key> -u <tool shed url> -n <repository name> -o <repository owner> +""" + +import os +import sys +import argparse +import urllib2 +sys.path.insert( 0, os.path.dirname( __file__ ) ) +from common import submit + +def main( options ): + api_key = options.api_key + if api_key: + if options.tool_shed_url and options.name and options.owner: + base_tool_shed_url = options.tool_shed_url.rstrip( '/' ) + data = {} + data[ 'tool_shed_url' ] = base_tool_shed_url + data[ 'name' ] = options.name + data[ 'owner' ] = options.owner + url = '%s%s' % ( base_tool_shed_url, '/api/repositories/remove_repository_registry_entry' ) + response_dict = submit( url, data, api_key=api_key, return_formatted=False ) + print response_dict + else: + print "Invalid tool_shed: ", base_tool_shed_url, " name: ", name, " or owner: ", owner, "." + else: + print "An API key for an admin user in the Tool Shed is required to remove entries from the Tool Shed's repository registry." + +if __name__ == '__main__': + parser = argparse.ArgumentParser( description='Remove entries from the Tool Shed repository registry for a specified repository.' ) + parser.add_argument( "-a", "--api_key", dest="api_key", required=True, help="API Key for user removing entries from the Tool Shed's repository registry." ) + parser.add_argument( "-u", "--url", dest="tool_shed_url", required=True, help="Tool Shed URL" ) + parser.add_argument( "-n", "--name", dest='name', required=True, help="Repository name." ) + parser.add_argument( "-o", "--owner", dest='owner', required=True, help="Repository owner." ) + options = parser.parse_args() + main( options ) diff -r 1a38b9cf737f9b54a992e973b124daf73cf32acc -r af60bdf0d4add74f225d163eca4438ab7ccc0e52 test/install_and_test_tool_shed_repositories/base/util.py --- a/test/install_and_test_tool_shed_repositories/base/util.py +++ b/test/install_and_test_tool_shed_repositories/base/util.py @@ -13,6 +13,7 @@ eggs.require( 'mercurial' ) eggs.require( "nose" ) +import json import logging import install_and_test_tool_shed_repositories.base.test_db_util as test_db_util import install_and_test_tool_shed_repositories.functional.test_install_repositories as test_install_repositories @@ -31,12 +32,12 @@ from common import get_latest_downloadable_changeset_revision_via_api from common import get_repository_dict from common import json_from_url +from common import submit from common import update from galaxy.util import asbool from galaxy.util import listify from galaxy.util import unicodify -from galaxy.util.json import to_json_string import galaxy.webapps.tool_shed.model.mapping from nose.plugins import Plugin @@ -691,8 +692,8 @@ else: xml_element = tool_shed for reason_section in xml_element: - reason_text = reason_section.find( 'text' ) - if reason_text: + reason_text = reason_section.find( 'text', None ) + if reason_text is not None: reason = str( reason_text.text ) else: reason = 'No reason provided.' @@ -985,7 +986,7 @@ """ if shed_tools_dict is None: shed_tools_dict = {} - file( galaxy_shed_tools_dict_file, 'w' ).write( to_json_string( shed_tools_dict ) ) + file( galaxy_shed_tools_dict_file, 'w' ).write( json.dumps( shed_tools_dict ) ) def print_install_and_test_results( install_stage_type, install_and_test_statistics_dict, error_message ): "Print statistics for the current test run." @@ -1081,11 +1082,34 @@ owner = repository_dict.get( 'owner', None ) changeset_revision = repository_dict.get( 'changeset_revision', None ) if name is None or owner is None or changeset_revision is None: - print 'Entries for name, owner or changeset_revision missing from repository_dict %s' % repository_dict + print 'Entries for name: ', name, ' owner: ', owner, ' or changeset_revision: ', changeset_revision, \ + ' missing from repository_dict:' % repository_dict else: name = str( name ) owner = str( owner ) changeset_revision = str( changeset_revision ) + # With regard to certification level one, the status of this repository may or may not have changed between + # this install and test run and the previous install and test run. Rather than attempting to determine if + # anything has changed here, we'll let the Tool Shed's repository registry handle the process of proper + # categorization. To enable this, we'll just remove entries from the Tool Shed's repository registry and + # then add them back. This will ensure proper categorization for this repository. + registry_params = dict( tool_shed_url=galaxy_tool_shed_url, name=name, owner=owner ) + print "Removing entries for repository ", name, " owned by ", owner, "from the Tool Shed's repository registry." + url = '%s' % ( common_util.url_join( galaxy_tool_shed_url, 'api', 'repositories', 'remove_repository_registry_entry' ) ) + response_dict = submit( url, registry_params, api_key=tool_shed_api_key, return_formatted=False ) + status = response_dict.get( 'status', 'ok' ) + if status == 'error': + default_message = 'An unknown error occurred attempting to remove entries from the repository registry.' + error_message = response_dict.get( 'message', default_message ) + print error_message + print "Adding entries for repository ", name, " owned by ", owner, "into the Tool Shed's repository registry." + url = '%s' % ( common_util.url_join( galaxy_tool_shed_url, 'api', 'repositories', 'add_repository_registry_entry' ) ) + response_dict = submit( url, registry_params, api_key=tool_shed_api_key, return_formatted=False ) + status = response_dict.get( 'status', 'ok' ) + if status == 'error': + default_message = 'An unknown error occurred attempting to add entries into the repository registry.' + error_message = response_dict.get( 'message', default_message ) + print error_message print '\n=============================================================\n' print 'Inserting the following into tool_test_results for revision %s of repository %s owned by %s:\n%s' % \ ( changeset_revision, name, owner, str( tool_test_results_dict ) ) @@ -1094,7 +1118,7 @@ params[ 'tool_test_results' ] = tool_test_results_dicts # Set the time_last_tested entry so that the repository_metadata.time_last_tested will be set in the tool shed. params[ 'time_last_tested' ] = 'This entry will result in this value being set via the Tool Shed API.' - url = '%s' % ( common_util.url_join( galaxy_tool_shed_url,'api', 'repository_revisions', str( metadata_revision_id ) ) ) + url = '%s' % ( common_util.url_join( galaxy_tool_shed_url, 'api', 'repository_revisions', str( metadata_revision_id ) ) ) print 'url: ', url print 'params: ', params try: diff -r 1a38b9cf737f9b54a992e973b124daf73cf32acc -r af60bdf0d4add74f225d163eca4438ab7ccc0e52 test/install_and_test_tool_shed_repositories/functional/test_install_repositories.py --- a/test/install_and_test_tool_shed_repositories/functional/test_install_repositories.py +++ b/test/install_and_test_tool_shed_repositories/functional/test_install_repositories.py @@ -11,8 +11,10 @@ def do_install( self, repository_dict ): self.logout() - self.login( email='test(a)bx.psu.edu', username='test' ) - admin_user = test_db_util.get_user( 'test(a)bx.psu.edu' ) + admin_email = 'test(a)bx.psu.edu' + admin_username = 'test' + self.login( email=admin_email, username=admin_username ) + admin_user = test_db_util.get_user( admin_email ) assert admin_user is not None, 'Problem retrieving user with email %s from the database' % admin_email admin_user_private_role = test_db_util.get_private_role( admin_user ) # Install the repository through the web interface using twill. The install_repository() method may Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: dannon: Add noqa comments to prevent pep8 from asking folks to change == to 'is' here.
by commits-noreply＠bitbucket.org 23 May '14

23 May '14

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/1a38b9cf737f/ Changeset: 1a38b9cf737f User: dannon Date: 2014-05-23 18:42:29 Summary: Add noqa comments to prevent pep8 from asking folks to change == to 'is' here. Affected #: 1 file diff -r 1283375984ac83ce18e5a78259b12d49cf86d58d -r 1a38b9cf737f9b54a992e973b124daf73cf32acc lib/galaxy/webapps/galaxy/api/users.py --- a/lib/galaxy/webapps/galaxy/api/users.py +++ b/lib/galaxy/webapps/galaxy/api/users.py @@ -30,12 +30,12 @@ if f_email: query = query.filter(trans.app.model.User.email.like("%%%s%%" % f_email)) if deleted: - query = query.filter( trans.app.model.User.table.c.deleted == True ) + query = query.filter( trans.app.model.User.table.c.deleted == True ) # noqa # only admins can see deleted users if not trans.user_is_admin(): return [] else: - query = query.filter( trans.app.model.User.table.c.deleted == False ) + query = query.filter( trans.app.model.User.table.c.deleted == False ) # noqa # special case: user can see only their own user if not trans.user_is_admin(): item = trans.user.to_dict( value_mapper={ 'id': trans.security.encode_id } ) Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: 2 new changesets
by commits-noreply＠bitbucket.org 23 May '14

23 May '14

2 new commits in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/1a43bcbff87c/ Changeset: 1a43bcbff87c Branch: next-stable User: dannon Date: 2014-05-23 18:00:21 Summary: pep8 api/users controller. Affected #: 1 file diff -r 5b1e3075b05526b98529099d328501e9633a95f2 -r 1a43bcbff87c4580ac9d791d1b770e6e80a7c0d7 lib/galaxy/webapps/galaxy/api/users.py --- a/lib/galaxy/webapps/galaxy/api/users.py +++ b/lib/galaxy/webapps/galaxy/api/users.py @@ -42,7 +42,7 @@ return [item] for user in query: item = user.to_dict( value_mapper={ 'id': trans.security.encode_id } ) - #TODO: move into api_values + # TODO: move into api_values rval.append( item ) return rval @@ -78,12 +78,11 @@ else: raise HTTPBadRequest( detail='Invalid user id ( %s ) specified' % id ) item = user.to_dict( view='element', value_mapper={ 'id': trans.security.encode_id, - 'total_disk_usage': float } ) - + 'total_disk_usage': float } ) # add a list of tags used by the user (as strings) item[ 'tags_used' ] = self.get_user_tags_used( trans, user=user ) - #TODO: move into api_values (needs trans, tho - can we do that with api_keys/@property??) - #TODO: works with other users (from admin)?? + # TODO: move into api_values (needs trans, tho - can we do that with api_keys/@property??) + # TODO: works with other users (from admin)?? item['quota_percent'] = trans.app.quota_agent.get_percent( trans=trans ) item['is_admin'] = trans.user_is_admin() return item @@ -103,8 +102,8 @@ email = payload[ 'email' ] password = payload[ 'password' ] message = "\n".join( [ validate_email( trans, email ), - validate_password( trans, password, password ), - validate_publicname( trans, username ) ] ).rstrip() + validate_password( trans, password, password ), + validate_publicname( trans, username ) ] ).rstrip() if message: raise exceptions.RequestParameterInvalidException( message ) else: @@ -138,15 +137,13 @@ def undelete( self, trans, **kwd ): raise HTTPNotImplemented() - #TODO: move to more basal, common resource than this + # TODO: move to more basal, common resource than this def anon_user_api_value( self, trans ): """ Returns data for an anonymous user, truncated to only usage and quota_percent """ usage = trans.app.quota_agent.get_usage( trans ) percent = trans.app.quota_agent.get_percent( trans=trans, usage=usage ) - return { - 'total_disk_usage' : int( usage ), - 'nice_total_disk_usage' : util.nice_size( usage ), - 'quota_percent' : percent - } + return {'total_disk_usage': int( usage ), + 'nice_total_disk_usage': util.nice_size( usage ), + 'quota_percent': percent} https://bitbucket.org/galaxy/galaxy-central/commits/1283375984ac/ Changeset: 1283375984ac User: dannon Date: 2014-05-23 18:00:40 Summary: Merge next-stable. Affected #: 1 file diff -r 39217230c38250daa36e2a9554cdcd52c01542ef -r 1283375984ac83ce18e5a78259b12d49cf86d58d lib/galaxy/webapps/galaxy/api/users.py --- a/lib/galaxy/webapps/galaxy/api/users.py +++ b/lib/galaxy/webapps/galaxy/api/users.py @@ -42,7 +42,7 @@ return [item] for user in query: item = user.to_dict( value_mapper={ 'id': trans.security.encode_id } ) - #TODO: move into api_values + # TODO: move into api_values rval.append( item ) return rval @@ -78,12 +78,11 @@ else: raise HTTPBadRequest( detail='Invalid user id ( %s ) specified' % id ) item = user.to_dict( view='element', value_mapper={ 'id': trans.security.encode_id, - 'total_disk_usage': float } ) - + 'total_disk_usage': float } ) # add a list of tags used by the user (as strings) item[ 'tags_used' ] = self.get_user_tags_used( trans, user=user ) - #TODO: move into api_values (needs trans, tho - can we do that with api_keys/@property??) - #TODO: works with other users (from admin)?? + # TODO: move into api_values (needs trans, tho - can we do that with api_keys/@property??) + # TODO: works with other users (from admin)?? item['quota_percent'] = trans.app.quota_agent.get_percent( trans=trans ) item['is_admin'] = trans.user_is_admin() return item @@ -103,8 +102,8 @@ email = payload[ 'email' ] password = payload[ 'password' ] message = "\n".join( [ validate_email( trans, email ), - validate_password( trans, password, password ), - validate_publicname( trans, username ) ] ).rstrip() + validate_password( trans, password, password ), + validate_publicname( trans, username ) ] ).rstrip() if message: raise exceptions.RequestParameterInvalidException( message ) else: @@ -138,15 +137,13 @@ def undelete( self, trans, **kwd ): raise HTTPNotImplemented() - #TODO: move to more basal, common resource than this + # TODO: move to more basal, common resource than this def anon_user_api_value( self, trans ): """ Returns data for an anonymous user, truncated to only usage and quota_percent """ usage = trans.app.quota_agent.get_usage( trans ) percent = trans.app.quota_agent.get_percent( trans=trans, usage=usage ) - return { - 'total_disk_usage' : int( usage ), - 'nice_total_disk_usage' : util.nice_size( usage ), - 'quota_percent' : percent - } + return {'total_disk_usage': int( usage ), + 'nice_total_disk_usage': util.nice_size( usage ), + 'quota_percent': percent} Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: 2 new changesets
by commits-noreply＠bitbucket.org 23 May '14

23 May '14

2 new commits in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/5b1e3075b055/ Changeset: 5b1e3075b055 Branch: next-stable User: dannon Date: 2014-05-23 17:49:39 Summary: Force user api to validate creation of new users, raising RequestParameterInvalid on failure to do so. Affected #: 1 file diff -r cbd0a239259b694afadfc62bdde820eef0c30b4f -r 5b1e3075b05526b98529099d328501e9633a95f2 lib/galaxy/webapps/galaxy/api/users.py --- a/lib/galaxy/webapps/galaxy/api/users.py +++ b/lib/galaxy/webapps/galaxy/api/users.py @@ -3,10 +3,14 @@ """ import logging from paste.httpexceptions import HTTPBadRequest, HTTPNotImplemented -from galaxy import util, web + +from galaxy import util, web, exceptions +from galaxy.security.validate_user_input import validate_email +from galaxy.security.validate_user_input import validate_password +from galaxy.security.validate_user_input import validate_publicname from galaxy.web.base.controller import BaseAPIController, UsesTagsMixin +from galaxy.web.base.controller import CreatesApiKeysMixin from galaxy.web.base.controller import CreatesUsersMixin -from galaxy.web.base.controller import CreatesApiKeysMixin log = logging.getLogger( __name__ ) @@ -98,7 +102,13 @@ username = payload[ 'username' ] email = payload[ 'email' ] password = payload[ 'password' ] - user = self.create_user( trans=trans, email=email, username=username, password=password ) + message = "\n".join( [ validate_email( trans, email ), + validate_password( trans, password, password ), + validate_publicname( trans, username ) ] ).rstrip() + if message: + raise exceptions.RequestParameterInvalidException( message ) + else: + user = self.create_user( trans=trans, email=email, username=username, password=password ) else: raise HTTPNotImplemented() item = user.to_dict( view='element', value_mapper={ 'id': trans.security.encode_id, https://bitbucket.org/galaxy/galaxy-central/commits/39217230c382/ Changeset: 39217230c382 User: dannon Date: 2014-05-23 17:49:50 Summary: Merge nextstable. Affected #: 1 file diff -r 89bdb65761f631921d3a21cfd2335103715de5bb -r 39217230c38250daa36e2a9554cdcd52c01542ef lib/galaxy/webapps/galaxy/api/users.py --- a/lib/galaxy/webapps/galaxy/api/users.py +++ b/lib/galaxy/webapps/galaxy/api/users.py @@ -3,10 +3,14 @@ """ import logging from paste.httpexceptions import HTTPBadRequest, HTTPNotImplemented -from galaxy import util, web + +from galaxy import util, web, exceptions +from galaxy.security.validate_user_input import validate_email +from galaxy.security.validate_user_input import validate_password +from galaxy.security.validate_user_input import validate_publicname from galaxy.web.base.controller import BaseAPIController, UsesTagsMixin +from galaxy.web.base.controller import CreatesApiKeysMixin from galaxy.web.base.controller import CreatesUsersMixin -from galaxy.web.base.controller import CreatesApiKeysMixin log = logging.getLogger( __name__ ) @@ -98,7 +102,13 @@ username = payload[ 'username' ] email = payload[ 'email' ] password = payload[ 'password' ] - user = self.create_user( trans=trans, email=email, username=username, password=password ) + message = "\n".join( [ validate_email( trans, email ), + validate_password( trans, password, password ), + validate_publicname( trans, username ) ] ).rstrip() + if message: + raise exceptions.RequestParameterInvalidException( message ) + else: + user = self.create_user( trans=trans, email=email, username=username, password=password ) else: raise HTTPNotImplemented() item = user.to_dict( view='element', value_mapper={ 'id': trans.security.encode_id, Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: 2 new changesets
by commits-noreply＠bitbucket.org 23 May '14

23 May '14

2 new commits in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/cbd0a239259b/ Changeset: cbd0a239259b Branch: next-stable User: davebgx Date: 2014-05-23 16:00:33 Summary: Fix server error when registering a new account. Affected #: 1 file diff -r 2b0cdcea282b2ea92b649df9a5d63f40b2bd545f -r cbd0a239259b694afadfc62bdde820eef0c30b4f lib/galaxy/webapps/galaxy/controllers/user.py --- a/lib/galaxy/webapps/galaxy/controllers/user.py +++ b/lib/galaxy/webapps/galaxy/controllers/user.py @@ -1144,9 +1144,9 @@ return "A public user name is required in the tool shed." if username in [ 'repos' ]: return "The term <b>%s</b> is a reserved word in the tool shed, so it cannot be used as a public user name." % username - message = "\n".join(validate_email( trans, email ), - validate_password( trans, password, confirm ), - validate_publicname( trans, username )) + message = "\n".join( [ validate_email( trans, email ), + validate_password( trans, password, confirm ), + validate_publicname( trans, username ) ] ).rstrip() if not message: if trans.webapp.name == 'galaxy': if self.get_all_forms( trans, https://bitbucket.org/galaxy/galaxy-central/commits/89bdb65761f6/ Changeset: 89bdb65761f6 User: davebgx Date: 2014-05-23 16:01:07 Summary: Merge with next-stable. Affected #: 1 file diff -r af76f0273d3de8ccc38f78dfb0aae9974574aa7b -r 89bdb65761f631921d3a21cfd2335103715de5bb lib/galaxy/webapps/galaxy/controllers/user.py --- a/lib/galaxy/webapps/galaxy/controllers/user.py +++ b/lib/galaxy/webapps/galaxy/controllers/user.py @@ -1144,9 +1144,9 @@ return "A public user name is required in the tool shed." if username in [ 'repos' ]: return "The term <b>%s</b> is a reserved word in the tool shed, so it cannot be used as a public user name." % username - message = "\n".join(validate_email( trans, email ), - validate_password( trans, password, confirm ), - validate_publicname( trans, username )) + message = "\n".join( [ validate_email( trans, email ), + validate_password( trans, password, confirm ), + validate_publicname( trans, username ) ] ).rstrip() if not message: if trans.webapp.name == 'galaxy': if self.get_all_forms( trans, Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: guerler: Charts: Allow chart groups to have different number of rows
by commits-noreply＠bitbucket.org 22 May '14

22 May '14

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/47810c6ec62c/ Changeset: 47810c6ec62c User: guerler Date: 2014-05-22 23:16:52 Summary: Charts: Allow chart groups to have different number of rows Affected #: 1 file diff -r e959799971fc33d0490973c6b28673049d8ece61 -r 47810c6ec62c258423ea4527cf3214326a9b417b config/plugins/visualizations/charts/static/library/datasets.js --- a/config/plugins/visualizations/charts/static/library/datasets.js +++ b/config/plugins/visualizations/charts/static/library/datasets.js @@ -142,16 +142,25 @@ // log console.debug('Datasets::_fill_from_cache() - Filling request from cache.'); - // get dataset - var dataset = this.list[request_dictionary.id]; - if (!dataset) { - console.debug('FAILED - Datasets::_fill_from_cache() - Dataset not found.'); + // identify start of request + var start = request_dictionary.start; + + // identify end of request + var limit = 0; + for (var i in request_dictionary.groups) { + var group = request_dictionary.groups[i]; + for (var key in group.columns) { + var column = group.columns[key]; + var block_id = this._block_id(request_dictionary, column.index); + var column_data = this.cache[block_id]; + if (column_data) { + limit = Math.max(limit, column_data.length); + } + } } - // identify start/end of request - var start = request_dictionary.start; - var end = Math.min(request_dictionary.end, dataset.metadata_data_lines); - if (end - start <= 0) { + // check length + if (limit == 0) { console.debug('FAILED - Datasets::_fill_from_cache() - Invalid range.'); } @@ -164,7 +173,7 @@ group.values = []; // add values - for (var j = 0; j < end - start; j++) { + for (var j = 0; j < limit; j++) { // add default x values group.values[j] = { x : parseInt(j) + start @@ -184,7 +193,7 @@ // check if auto block is requested if (column.index == 'auto') { - for (var j = start; j < end; j++) { + for (var j = 0; j < limit; j++) { // get value dictionary var value = group.values[j]; @@ -197,7 +206,7 @@ var column_data = this.cache[block_id]; // go through column - for (var j = start; j < end; j++) { + for (var j = 0; j < limit; j++) { // get value dictionary var value = group.values[j]; Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: 2 new changesets
by commits-noreply＠bitbucket.org 22 May '14

22 May '14

2 new commits in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/1830e34c80c0/ Changeset: 1830e34c80c0 Branch: next-stable User: davebgx Date: 2014-05-22 16:59:18 Summary: Enable running the create_users and create_categories scripts via run_tool_shed.sh Affected #: 7 files diff -r f0a5e3666e844d907e7ef3c7fc4e23a443ba65d1 -r 1830e34c80c086696f83383871b6027f3b44ea8b lib/tool_shed/scripts/bootstrap_tool_shed/bootstrap_tool_shed.sh --- /dev/null +++ b/lib/tool_shed/scripts/bootstrap_tool_shed/bootstrap_tool_shed.sh @@ -0,0 +1,100 @@ +#!/bin/bash + +stop_err() { + echo $1 + python ./scripts/paster.py serve tool_shed_wsgi.ini --pid-file=tool_shed_bootstrap.pid --log-file=tool_shed_bootstrap.log --stop-daemon + exit 1 +} + +tool_shed=`./lib/tool_shed/scripts/bootstrap_tool_shed/parse_run_sh_args.sh $@` + +if [ $? -ne 0 ] ; then + exit 0 +fi + +log_file="lib/tool_shed/scripts/bootstrap_tool_shed/bootstrap.log" + +database_result=`python ./lib/tool_shed/scripts/bootstrap_tool_shed/bootstrap_util.py --execute check_db --config_file tool_shed_wsgi.ini` + +if [ $? -ne 0 ] ; then + stop_err "Unable to bootstrap tool shed. $database_result" +fi + +echo "Bootstrapping from tool shed at $tool_shed." +echo -n "Creating database... " +python scripts/create_db.py tool_shed + +if [ $? -eq 0 ] ; then + echo "done." +else + stop_err "failed." +fi + +if [ $? -eq 0 ] ; then + user_auth=`python ./lib/tool_shed/scripts/bootstrap_tool_shed/bootstrap_util.py --execute admin_user_info --config_file tool_shed_wsgi.ini` + local_shed_url=`python ./lib/tool_shed/scripts/bootstrap_tool_shed/bootstrap_util.py --execute get_url --config_file tool_shed_wsgi.ini` +fi + +admin_user_name=`echo $user_auth | awk 'BEGIN { FS="__SEP__" } ; { print \$1 }'` +admin_user_email=`echo $user_auth | awk 'BEGIN { FS="__SEP__" } ; { print \$2 }'` +admin_user_password=`echo $user_auth | awk 'BEGIN { FS="__SEP__" } ; { print \$3 }'` + +echo -n "Creating user '$admin_user_name' with email address '$admin_user_email'..." + +python lib/tool_shed/scripts/bootstrap_tool_shed/create_user_with_api_key.py tool_shed_wsgi.ini >> $log_file + +echo " done." + +sed -i "s/#admin_users = user1@example.org,user2@example.org/admin_users = $admin_user_email/" tool_shed_wsgi.ini +echo -n "Starting tool shed in order to populate users and categories... " + +if [ -f tool_shed_bootstrap.pid ] ; then + stop_err "A bootstrap process is already running." +fi + +python ./scripts/paster.py serve tool_shed_wsgi.ini --pid-file=tool_shed_bootstrap.pid --log-file=tool_shed_bootstrap.log --daemon > /dev/null + +shed_pid=`cat tool_shed_bootstrap.pid` + +while : ; do + tail -n 1 tool_shed_bootstrap.log | grep -q "Removing PID file tool_shed_webapp.pid" + if [ $? -eq 0 ] ; then + echo "failed." + echo "More information about this failure may be found in the following log snippet from tool_shed_bootstrap.log:" + echo "========================================" + tail -n 40 tool_shed_bootstrap.log + echo "========================================" + stop_err " " + fi + tail -n 2 tool_shed_bootstrap.log | grep -q "Starting server in PID $shed_pid" + if [ $? -eq 0 ] ; then + echo "done." + break + fi +done + +echo -n "Retrieving admin user's API key..." +api_key_json=`curl -s --user $admin_user_email:$admin_user_password $local_shed_url/api/authenticate/baseauth/` +api_key=`echo $api_key_json | grep api_key | awk 'BEGIN { FS="\"" } ; { print \$4 }' | sed 's/\\s\+//'` + +if [[ -z $api_key && ${api_key+x} ]] ; then + stop_err "Error getting API key for user $admin_user_email." +fi + +echo " done." + +if [ $? -eq 0 ] ; then + echo -n "Creating users... " + python lib/tool_shed/scripts/api/create_users.py -a $api_key -f $tool_shed -t $local_shed_url >> $log_file + echo "done." + echo -n "Creating categories... " + python lib/tool_shed/scripts/api/create_categories.py -a $api_key -f $tool_shed -t $local_shed_url >> $log_file + echo "done." +else + stop_err "Error getting API key from local tool shed." +fi + +echo "Bootstrap complete, shutting down temporary tool shed process. A log has been saved to tool_shed_bootstrap.log" +python ./scripts/paster.py serve tool_shed_wsgi.ini --pid-file=tool_shed_bootstrap.pid --log-file=tool_shed_bootstrap.log --stop-daemon + +exit 0 \ No newline at end of file diff -r f0a5e3666e844d907e7ef3c7fc4e23a443ba65d1 -r 1830e34c80c086696f83383871b6027f3b44ea8b lib/tool_shed/scripts/bootstrap_tool_shed/bootstrap_util.py --- /dev/null +++ b/lib/tool_shed/scripts/bootstrap_tool_shed/bootstrap_util.py @@ -0,0 +1,130 @@ +#!/usr/bin/python +import argparse +import ConfigParser +import os +import sys + +new_path = [ os.path.join( os.getcwd(), "lib" ) ] +new_path.extend( sys.path[1:] ) +sys.path = new_path + +from galaxy import eggs +eggs.require( "SQLAlchemy >= 0.4" ) +import galaxy.webapps.tool_shed.model.mapping as tool_shed_model +from sqlalchemy.exc import ProgrammingError +from sqlalchemy.exc import OperationalError + +from tool_shed.util import xml_util + +def check_db( config_parser ): + dburi = None + + if config_parser.has_option( 'app:main', 'database_connection' ): + dburi = config_parser.get( 'app:main', 'database_connection' ) + elif config_parser.has_option( 'app:main', 'database_file' ): + db_file = config_parser.get( 'app:main', 'database_file' ) + dburi = "sqlite:///%s?isolation_level=IMMEDIATE" % db_file + else: + print 'The database configuration setting is missing from the tool_shed_wsgi.ini file. Add this setting before attempting to bootstrap.' + exit(1) + + sa_session = None + + database_exists_message = 'The database configured for this Tool Shed is not new, so bootstrapping is not allowed. ' + database_exists_message += 'Create a new database that has not been migrated before attempting to bootstrap.' + + try: + model = tool_shed_model.init( config_parser.get( 'app:main', 'file_path' ), dburi, engine_options={}, create_tables=False ) + sa_session = model.context.current + print database_exists_message + exit(1) + except ProgrammingError, e: + pass + except OperationalError, e: + pass + + try: + if sa_session is not None: + result = sa_session.execute( 'SELECT version FROM migrate_version' ).first() + if result[0] >= 2: + print database_exists_message + exit(1) + else: + pass + except ProgrammingError, e: + pass + + if config_parser.has_option( 'app:main', 'hgweb_config_dir' ): + hgweb_config_parser = ConfigParser.ConfigParser() + hgweb_dir = config_parser.get( 'app:main', 'hgweb_config_dir' ) + hgweb_config_file = os.path.join( hgweb_dir, 'hgweb.config' ) + if not os.path.exists( hgweb_config_file ): + exit(0) + hgweb_config_parser.read( hgweb_config_file ) + configured_repos = hgweb_config_parser.items( 'paths' ) + if len( configured_repos ) >= 1: + message = "This Tool Shed's hgweb.config file contains entries, so bootstrapping is not allowed. Delete" + message += " the current hgweb.config file along with all associated repositories in the configured " + message += "location before attempting to boostrap." + print + exit(1) + else: + exit(0) + else: + exit(0) + + exit(0) + +def admin_user_info( config_parser ): + user_info_config = os.path.abspath( os.path.join( os.getcwd(), 'lib/tool_shed/scripts/bootstrap_tool_shed', 'user_info.xml' ) ) + tree, error_message = xml_util.parse_xml( user_info_config ) + if tree is None: + print "The XML file ", user_info_config, " seems to be invalid, using defaults." + email = 'admin(a)test.org' + password = 'testuser' + username = 'admin' + else: + root = tree.getroot() + for elem in root: + if elem.tag == 'email': + email = elem.text + elif elem.tag == 'password': + password = elem.text + elif elem.tag == 'username': + username = elem.text + print '%s__SEP__%s__SEP__%s' % ( username, email, password ) + return 0 + +def get_local_tool_shed_url( config_parser ): + port = '9009' + if config_parser.has_section( 'server:main' ): + if config_parser.has_option( 'server:main', 'port' ): + port = config_parser.get( 'server:main', 'port' ) + host = '127.0.0.1' + print 'http://%s:%s' % ( host, port ) + return 0 + +def main( args ): + config_parser = ConfigParser.ConfigParser() + + if os.path.exists( args.config ): + config_parser.read( args.config ) + else: + return 1 + + if args.method == 'check_db': + return check_db( config_parser ) + elif args.method == 'admin_user_info': + return admin_user_info( config_parser ) + elif args.method == 'get_url': + return get_local_tool_shed_url( config_parser ) + else: + return 1 + +parser = argparse.ArgumentParser() +parser.add_argument( '-c', '--config_file', dest='config', action='store', default='tool_shed_wsgi.ini' ) +parser.add_argument( '-e', '--execute', dest='method', action='store', default='check_db' ) +args = parser.parse_args() + +if __name__ == '__main__': + exit( main( args ) ) \ No newline at end of file diff -r f0a5e3666e844d907e7ef3c7fc4e23a443ba65d1 -r 1830e34c80c086696f83383871b6027f3b44ea8b lib/tool_shed/scripts/bootstrap_tool_shed/create_user_with_api_key.py --- a/lib/tool_shed/scripts/bootstrap_tool_shed/create_user_with_api_key.py +++ b/lib/tool_shed/scripts/bootstrap_tool_shed/create_user_with_api_key.py @@ -125,7 +125,7 @@ return '' def validate_publicname( username ): - """""Validates the public username.""" + """Validates the public username.""" if len( username ) < 3: return "Public name must be at least 3 characters in length" if len( username ) > 255: @@ -154,5 +154,7 @@ if user is not None: api_key = create_api_key( app, user ) print "Created new user with public username '", user.username, ". An API key was also created and associated with the user." + exit(0) else: print "Problem creating a new user and an associated API key." + exit(1) diff -r f0a5e3666e844d907e7ef3c7fc4e23a443ba65d1 -r 1830e34c80c086696f83383871b6027f3b44ea8b lib/tool_shed/scripts/bootstrap_tool_shed/parse_run_sh_args.sh --- /dev/null +++ b/lib/tool_shed/scripts/bootstrap_tool_shed/parse_run_sh_args.sh @@ -0,0 +1,15 @@ +#!/bin/bash + +while (( $# )) ; do + case "$1" in + -bootstrap_from_tool_shed|--bootstrap_from_tool_shed) + bootstrap="true" + tool_shed=$2 + echo $tool_shed + exit 0 + break + ;; + esac + shift 1 +done +exit 1 \ No newline at end of file diff -r f0a5e3666e844d907e7ef3c7fc4e23a443ba65d1 -r 1830e34c80c086696f83383871b6027f3b44ea8b run.sh --- a/run.sh +++ b/run.sh @@ -5,41 +5,7 @@ python ./scripts/check_python.py [ $? -ne 0 ] && exit 1 -SAMPLES=" - community_wsgi.ini.sample - datatypes_conf.xml.sample - external_service_types_conf.xml.sample - migrated_tools_conf.xml.sample - reports_wsgi.ini.sample - shed_tool_conf.xml.sample - tool_conf.xml.sample - shed_tool_data_table_conf.xml.sample - tool_data_table_conf.xml.sample - tool_sheds_conf.xml.sample - data_manager_conf.xml.sample - shed_data_manager_conf.xml.sample - openid_conf.xml.sample - job_metrics_conf.xml.sample - universe_wsgi.ini.sample - tool-data/shared/ncbi/builds.txt.sample - tool-data/shared/ensembl/builds.txt.sample - tool-data/shared/ucsc/builds.txt.sample - tool-data/shared/ucsc/publicbuilds.txt.sample - tool-data/shared/ucsc/ucsc_build_sites.txt.sample - tool-data/shared/igv/igv_build_sites.txt.sample - tool-data/shared/rviewer/rviewer_build_sites.txt.sample - tool-data/*.sample - static/welcome.html.sample -" - -# Create any missing config/location files -for sample in $SAMPLES; do - file=`echo $sample | sed -e 's/\.sample$//'` - if [ ! -f "$file" -a -f "$sample" ]; then - echo "Initializing $file from `basename $sample`" - cp $sample $file - fi -done +./scripts/copy_sample_files.sh if [ -n "$GALAXY_UNIVERSE_CONFIG_DIR" ]; then python ./scripts/build_universe_config.py "$GALAXY_UNIVERSE_CONFIG_DIR" diff -r f0a5e3666e844d907e7ef3c7fc4e23a443ba65d1 -r 1830e34c80c086696f83383871b6027f3b44ea8b run_tool_shed.sh --- a/run_tool_shed.sh +++ b/run_tool_shed.sh @@ -2,37 +2,14 @@ cd `dirname $0` -SAMPLES=" - tool_shed_wsgi.ini.sample - datatypes_conf.xml.sample - external_service_types_conf.xml.sample - migrated_tools_conf.xml.sample - reports_wsgi.ini.sample - shed_tool_conf.xml.sample - tool_conf.xml.sample - shed_tool_data_table_conf.xml.sample - tool_data_table_conf.xml.sample - tool_sheds_conf.xml.sample - openid_conf.xml.sample - universe_wsgi.ini.sample - lib/tool_shed/scripts/bootstrap_tool_shed/user_info.xml.sample - tool-data/shared/ncbi/builds.txt.sample - tool-data/shared/ensembl/builds.txt.sample - tool-data/shared/ucsc/builds.txt.sample - tool-data/shared/ucsc/publicbuilds.txt.sample - tool-data/shared/igv/igv_build_sites.txt.sample - tool-data/shared/rviewer/rviewer_build_sites.txt.sample - tool-data/*.sample - static/welcome.html.sample -" +./scripts/copy_sample_files.sh -# Create any missing config/location files -for sample in $SAMPLES; do - file=`echo $sample | sed -e 's/\.sample$//'` - if [ ! -f "$file" -a -f "$sample" ]; then - echo "Initializing $file from `basename $sample`" - cp $sample $file - fi -done +tool_shed=`./lib/tool_shed/scripts/bootstrap_tool_shed/parse_run_sh_args.sh $@` +args=$@ -python ./scripts/paster.py serve tool_shed_wsgi.ini --pid-file=tool_shed_webapp.pid --log-file=tool_shed_webapp.log $@ +if [ $? -eq 0 ] ; then + bash ./lib/tool_shed/scripts/bootstrap_tool_shed/bootstrap_tool_shed.sh $@ + args=`echo $@ | sed "s#-\?-bootstrap_from_tool_shed $tool_shed##"` +fi + +python ./scripts/paster.py serve tool_shed_wsgi.ini --pid-file=tool_shed_webapp.pid --log-file=tool_shed_webapp.log $args diff -r f0a5e3666e844d907e7ef3c7fc4e23a443ba65d1 -r 1830e34c80c086696f83383871b6027f3b44ea8b scripts/copy_sample_files.sh --- /dev/null +++ b/scripts/copy_sample_files.sh @@ -0,0 +1,39 @@ +#!/bin/bash + +SAMPLES=" + tool_shed_wsgi.ini.sample + datatypes_conf.xml.sample + external_service_types_conf.xml.sample + migrated_tools_conf.xml.sample + reports_wsgi.ini.sample + shed_tool_conf.xml.sample + tool_conf.xml.sample + shed_tool_data_table_conf.xml.sample + tool_data_table_conf.xml.sample + tool_sheds_conf.xml.sample + data_manager_conf.xml.sample + shed_data_manager_conf.xml.sample + openid_conf.xml.sample + job_metrics_conf.xml.sample + universe_wsgi.ini.sample + lib/tool_shed/scripts/bootstrap_tool_shed/user_info.xml.sample + tool-data/shared/ncbi/builds.txt.sample + tool-data/shared/ensembl/builds.txt.sample + tool-data/shared/ucsc/builds.txt.sample + tool-data/shared/ucsc/publicbuilds.txt.sample + tool-data/shared/ucsc/ucsc_build_sites.txt.sample + tool-data/shared/igv/igv_build_sites.txt.sample + tool-data/shared/rviewer/rviewer_build_sites.txt.sample + tool-data/*.sample + static/welcome.html.sample +" + +# Create any missing config/location files +for sample in $SAMPLES; do + file=${sample%.sample} + if [ ! -f "$file" -a -f "$sample" ]; then + echo "Initializing $file from `basename $sample`" + cp $sample $file + fi +done + https://bitbucket.org/galaxy/galaxy-central/commits/e959799971fc/ Changeset: e959799971fc User: davebgx Date: 2014-05-22 18:41:13 Summary: Merge next-stable. Affected #: 0 files Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: 3 new changesets
by commits-noreply＠bitbucket.org 22 May '14

22 May '14

3 new commits in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/8e0034733cf9/ Changeset: 8e0034733cf9 Branch: next-stable User: jmchilton Date: 2014-05-22 18:01:04 Summary: Fix GUI for f24bb5d. Affected #: 1 file diff -r 316db249f068ded4e541415232b480250c7cdd56 -r 8e0034733cf936643078de18f77394256e55001d static/scripts/mvc/dataset/hda-model.js --- a/static/scripts/mvc/dataset/hda-model.js +++ b/static/scripts/mvc/dataset/hda-model.js @@ -667,8 +667,8 @@ if( ids.length != 2 ){ // TODO: Do something... } - element_identifiers.push( { name: "left", src: "hda", id: ids[ 0 ] } ); - element_identifiers.push( { name: "right", src: "hda", id: ids[ 1 ] } ); + element_identifiers.push( { name: "forward", src: "hda", id: ids[ 0 ] } ); + element_identifiers.push( { name: "reverse", src: "hda", id: ids[ 1 ] } ); name = "New Dataset Pair"; } options.data = {type: "dataset_collection", https://bitbucket.org/galaxy/galaxy-central/commits/f0a5e3666e84/ Changeset: f0a5e3666e84 Branch: next-stable User: jmchilton Date: 2014-05-22 18:02:05 Summary: Pack scripts. Affected #: 1 file diff -r 8e0034733cf936643078de18f77394256e55001d -r f0a5e3666e844d907e7ef3c7fc4e23a443ba65d1 static/scripts/packed/mvc/dataset/hda-model.js --- a/static/scripts/packed/mvc/dataset/hda-model.js +++ b/static/scripts/packed/mvc/dataset/hda-model.js @@ -1,1 +1,1 @@ -define(["mvc/base-mvc","utils/localization"],function(e,c){var a=Backbone.Model.extend(e.LoggableMixin).extend({urlRoot:galaxy_config.root+"api/histories/",url:function(){return this.urlRoot+this.get("history_id")+"/contents/"+this.get("history_content_type")+"s/"+this.get("id")},hidden:function(){return !this.get("visible")},"delete":function f(m){if(this.get("deleted")){return jQuery.when()}return this.save({deleted:true},m)},undelete:function i(m){if(!this.get("deleted")||this.get("purged")){return jQuery.when()}return this.save({deleted:false},m)},hide:function d(m){if(!this.get("visible")){return jQuery.when()}return this.save({visible:false},m)},unhide:function h(m){if(this.get("visible")){return jQuery.when()}return this.save({visible:true},m)},isVisible:function(n,o){var m=true;if((!n)&&(this.get("deleted")||this.get("purged"))){m=false}if((!o)&&(!this.get("visible"))){m=false}return m},searchAttribute:function(o,m){var n=this.get(o);if(!m||(n===undefined||n===null)){return false}if(_.isArray(n)){return this._searchArrayAttribute(n,m)}return(n.toString().toLowerCase().indexOf(m.toLowerCase())!==-1)},_searchArrayAttribute:function(n,m){m=m.toLowerCase();return _.any(n,function(o){return(o.toString().toLowerCase().indexOf(m.toLowerCase())!==-1)})},search:function(m){var n=this;return _.filter(this.searchAttributes,function(o){return n.searchAttribute(o,m)})},matches:function(n){var p="=",m=n.split(p);if(m.length>=2){var o=m[0];o=this.searchAliases[o]||o;return this.searchAttribute(o,m[1])}return !!this.search(n).length},matchesAll:function(n){var m=this;n=n.match(/(".*"|\w*=".*"|\S*)/g).filter(function(o){return !!o});return _.all(n,function(o){o=o.replace(/"/g,"");return m.matches(o)})}});var l=a.extend({defaults:{history_id:null,model_class:"HistoryDatasetAssociation",history_content_type:"dataset",hid:0,id:null,name:"(unnamed dataset)",state:"new",deleted:false,visible:true,accessible:true,purged:false,data_type:"",file_size:0,file_ext:"",meta_files:[],misc_blurb:"",misc_info:"",tags:[],annotation:""},urls:function(){var n=this.get("id");if(!n){return{}}var m={purge:galaxy_config.root+"datasets/"+n+"/purge_async",display:galaxy_config.root+"datasets/"+n+"/display/?preview=True",edit:galaxy_config.root+"datasets/"+n+"/edit",download:galaxy_config.root+"datasets/"+n+"/display?to_ext="+this.get("file_ext"),report_error:galaxy_config.root+"dataset/errors?id="+n,rerun:galaxy_config.root+"tool_runner/rerun?id="+n,show_params:galaxy_config.root+"datasets/"+n+"/show_params",visualization:galaxy_config.root+"visualization",annotation:{get:galaxy_config.root+"dataset/get_annotation_async?id="+n,set:galaxy_config.root+"dataset/annotate_async?id="+n},meta_download:galaxy_config.root+"dataset/get_metadata_file?hda_id="+n+"&metadata_name="};return m},initialize:function(m){this.log(this+".initialize",this.attributes);this.log("\tparent history_id: "+this.get("history_id"));if(!this.get("accessible")){this.set("state",l.STATES.NOT_VIEWABLE)}this._setUpListeners()},_setUpListeners:function(){this.on("change:state",function(n,m){this.log(this+" has changed state:",n,m);if(this.inReadyState()){this.trigger("state:ready",n,m,this.previous("state"))}})},isDeletedOrPurged:function(){return(this.get("deleted")||this.get("purged"))},inReadyState:function(){var m=_.contains(l.READY_STATES,this.get("state"));return(this.isDeletedOrPurged()||m)},hasDetails:function(){return _.has(this.attributes,"genome_build")},hasData:function(){return(this.get("file_size")>0)},"delete":function f(m){if(this.get("deleted")){return jQuery.when()}return this.save({deleted:true},m)},undelete:function i(m){if(!this.get("deleted")||this.get("purged")){return jQuery.when()}return this.save({deleted:false},m)},hide:function d(m){if(!this.get("visible")){return jQuery.when()}return this.save({visible:false},m)},unhide:function h(m){if(this.get("visible")){return jQuery.when()}return this.save({visible:true},m)},purge:function g(m){if(this.get("purged")){return jQuery.when()}m=m||{};m.url=galaxy_config.root+"datasets/"+this.get("id")+"/purge_async";var n=this,o=jQuery.ajax(m);o.done(function(r,p,q){n.set({deleted:true,purged:true})});o.fail(function(t,p,s){var q=c("Unable to purge dataset");var r=("Removal of datasets by users is not allowed in this Galaxy instance");if(t.responseJSON&&t.responseJSON.error){q=t.responseJSON.error}else{if(t.responseText.indexOf(r)!==-1){q=r}}t.responseText=q;n.trigger("error",n,t,m,c(q),{error:q})});return o},searchAttributes:["name","file_ext","genome_build","misc_blurb","misc_info","annotation","tags"],searchAliases:{title:"name",format:"file_ext",database:"genome_build",blurb:"misc_blurb",description:"misc_blurb",info:"misc_info",tag:"tags"},toString:function(){var m=this.get("id")||"";if(this.get("name")){m=this.get("hid")+' :"'+this.get("name")+'",'+m}return"HDA("+m+")"}});l.STATES={UPLOAD:"upload",QUEUED:"queued",RUNNING:"running",SETTING_METADATA:"setting_metadata",NEW:"new",EMPTY:"empty",OK:"ok",PAUSED:"paused",FAILED_METADATA:"failed_metadata",NOT_VIEWABLE:"noPermission",DISCARDED:"discarded",ERROR:"error"};l.READY_STATES=[l.STATES.OK,l.STATES.EMPTY,l.STATES.PAUSED,l.STATES.FAILED_METADATA,l.STATES.NOT_VIEWABLE,l.STATES.DISCARDED,l.STATES.ERROR];l.NOT_READY_STATES=[l.STATES.UPLOAD,l.STATES.QUEUED,l.STATES.RUNNING,l.STATES.SETTING_METADATA,l.STATES.NEW];var b=Backbone.Collection.extend(e.LoggableMixin).extend({model:function(n,m){if(n.history_content_type=="dataset"){return new l(n,m)}else{if(n.history_content_type=="dataset_collection"){return new j(n,m)}else{}}},urlRoot:galaxy_config.root+"api/histories",url:function(){return this.urlRoot+"/"+this.historyId+"/contents"},initialize:function(n,m){m=m||{};this.historyId=m.historyId},ids:function(){return this.map(function(m){return m.id})},notReady:function(){return this.filter(function(m){return !m.inReadyState()})},running:function(){var m=[];this.each(function(n){if(!n.inReadyState()){m.push(n.get("id"))}});return m},getByHid:function(m){return _.first(this.filter(function(n){return n.get("hid")===m}))},getVisible:function(m,p,o){o=o||[];var n=new b(this.filter(function(q){return q.isVisible(m,p)}));_.each(o,function(q){if(!_.isFunction(q)){return}n=new b(n.filter(q))});return n},haveDetails:function(){return this.all(function(m){return m.hasDetails()})},fetchAllDetails:function(n){n=n||{};var m={details:"all"};n.data=(n.data)?(_.extend(n.data,m)):(m);return this.fetch(n)},ajaxQueue:function(p,o){var n=jQuery.Deferred(),m=this.length,r=[];if(!m){n.resolve([]);return n}var q=this.chain().reverse().map(function(t,s){return function(){var u=p.call(t,o);u.done(function(v){n.notify({curr:s,total:m,response:v,model:t})});u.always(function(v){r.push(v);if(q.length){q.shift()()}else{n.resolve(r)}})}}).value();q.shift()();return n},matches:function(m){return this.filter(function(n){return n.matches(m)})},set:function(o,m){var n=this;o=_.map(o,function(q){var r=n.get(q.id);if(!r){return q}var p=r.toJSON();_.extend(p,q);return p});Backbone.Collection.prototype.set.call(this,o,m)},promoteToHistoryDatasetCollection:function k(r,p,n){n=n||{};n.url=this.url();n.type="POST";var t=p;var q=[],m=null;if(p=="list"){this.chain().each(function(w){var u=w.attributes.name;var x=w.id;var v=w.attributes.history_content_type;if(v=="dataset"){if(t!="list"){console.log("Invalid collection type")}q.push({name:u,src:"hda",id:x})}else{if(t=="list"){t="list:"+w.attributes.collection_type}else{if(t!="list:"+w.attributes.collection_type){console.log("Invalid collection type")}}q.push({name:u,src:"hdca",id:x})}});m="New Dataset List"}else{if(p=="paired"){var o=this.ids();if(o.length!=2){}q.push({name:"left",src:"hda",id:o[0]});q.push({name:"right",src:"hda",id:o[1]});m="New Dataset Pair"}}n.data={type:"dataset_collection",name:m,collection_type:t,element_identifiers:JSON.stringify(q),};var s=jQuery.ajax(n);s.done(function(w,u,v){r.refresh()});s.fail(function(w,u,v){if(w.responseJSON&&w.responseJSON.error){error=w.responseJSON.error}else{error=w.responseJSON}w.responseText=error});return s},toString:function(){return(["HDACollection(",[this.historyId,this.length].join(),")"].join(""))}});var j=a.extend({defaults:{history_id:null,model_class:"HistoryDatasetCollectionAssociation",history_content_type:"dataset_collection",hid:0,id:null,name:"(unnamed dataset collection)",state:"ok",accessible:true,deleted:false,visible:true,purged:false,tags:[],annotation:""},urls:function(){},inReadyState:function(){return true},searchAttributes:["name"],searchAliases:{title:"name"},});return{HistoryDatasetAssociation:l,HDACollection:b}}); \ No newline at end of file +define(["mvc/base-mvc","utils/localization"],function(e,c){var a=Backbone.Model.extend(e.LoggableMixin).extend({urlRoot:galaxy_config.root+"api/histories/",url:function(){return this.urlRoot+this.get("history_id")+"/contents/"+this.get("history_content_type")+"s/"+this.get("id")},hidden:function(){return !this.get("visible")},"delete":function f(m){if(this.get("deleted")){return jQuery.when()}return this.save({deleted:true},m)},undelete:function i(m){if(!this.get("deleted")||this.get("purged")){return jQuery.when()}return this.save({deleted:false},m)},hide:function d(m){if(!this.get("visible")){return jQuery.when()}return this.save({visible:false},m)},unhide:function h(m){if(this.get("visible")){return jQuery.when()}return this.save({visible:true},m)},isVisible:function(n,o){var m=true;if((!n)&&(this.get("deleted")||this.get("purged"))){m=false}if((!o)&&(!this.get("visible"))){m=false}return m},searchAttribute:function(o,m){var n=this.get(o);if(!m||(n===undefined||n===null)){return false}if(_.isArray(n)){return this._searchArrayAttribute(n,m)}return(n.toString().toLowerCase().indexOf(m.toLowerCase())!==-1)},_searchArrayAttribute:function(n,m){m=m.toLowerCase();return _.any(n,function(o){return(o.toString().toLowerCase().indexOf(m.toLowerCase())!==-1)})},search:function(m){var n=this;return _.filter(this.searchAttributes,function(o){return n.searchAttribute(o,m)})},matches:function(n){var p="=",m=n.split(p);if(m.length>=2){var o=m[0];o=this.searchAliases[o]||o;return this.searchAttribute(o,m[1])}return !!this.search(n).length},matchesAll:function(n){var m=this;n=n.match(/(".*"|\w*=".*"|\S*)/g).filter(function(o){return !!o});return _.all(n,function(o){o=o.replace(/"/g,"");return m.matches(o)})}});var l=a.extend({defaults:{history_id:null,model_class:"HistoryDatasetAssociation",history_content_type:"dataset",hid:0,id:null,name:"(unnamed dataset)",state:"new",deleted:false,visible:true,accessible:true,purged:false,data_type:"",file_size:0,file_ext:"",meta_files:[],misc_blurb:"",misc_info:"",tags:[],annotation:""},urls:function(){var n=this.get("id");if(!n){return{}}var m={purge:galaxy_config.root+"datasets/"+n+"/purge_async",display:galaxy_config.root+"datasets/"+n+"/display/?preview=True",edit:galaxy_config.root+"datasets/"+n+"/edit",download:galaxy_config.root+"datasets/"+n+"/display?to_ext="+this.get("file_ext"),report_error:galaxy_config.root+"dataset/errors?id="+n,rerun:galaxy_config.root+"tool_runner/rerun?id="+n,show_params:galaxy_config.root+"datasets/"+n+"/show_params",visualization:galaxy_config.root+"visualization",annotation:{get:galaxy_config.root+"dataset/get_annotation_async?id="+n,set:galaxy_config.root+"dataset/annotate_async?id="+n},meta_download:galaxy_config.root+"dataset/get_metadata_file?hda_id="+n+"&metadata_name="};return m},initialize:function(m){this.log(this+".initialize",this.attributes);this.log("\tparent history_id: "+this.get("history_id"));if(!this.get("accessible")){this.set("state",l.STATES.NOT_VIEWABLE)}this._setUpListeners()},_setUpListeners:function(){this.on("change:state",function(n,m){this.log(this+" has changed state:",n,m);if(this.inReadyState()){this.trigger("state:ready",n,m,this.previous("state"))}})},isDeletedOrPurged:function(){return(this.get("deleted")||this.get("purged"))},inReadyState:function(){var m=_.contains(l.READY_STATES,this.get("state"));return(this.isDeletedOrPurged()||m)},hasDetails:function(){return _.has(this.attributes,"genome_build")},hasData:function(){return(this.get("file_size")>0)},"delete":function f(m){if(this.get("deleted")){return jQuery.when()}return this.save({deleted:true},m)},undelete:function i(m){if(!this.get("deleted")||this.get("purged")){return jQuery.when()}return this.save({deleted:false},m)},hide:function d(m){if(!this.get("visible")){return jQuery.when()}return this.save({visible:false},m)},unhide:function h(m){if(this.get("visible")){return jQuery.when()}return this.save({visible:true},m)},purge:function g(m){if(this.get("purged")){return jQuery.when()}m=m||{};m.url=galaxy_config.root+"datasets/"+this.get("id")+"/purge_async";var n=this,o=jQuery.ajax(m);o.done(function(r,p,q){n.set({deleted:true,purged:true})});o.fail(function(t,p,s){var q=c("Unable to purge dataset");var r=("Removal of datasets by users is not allowed in this Galaxy instance");if(t.responseJSON&&t.responseJSON.error){q=t.responseJSON.error}else{if(t.responseText.indexOf(r)!==-1){q=r}}t.responseText=q;n.trigger("error",n,t,m,c(q),{error:q})});return o},searchAttributes:["name","file_ext","genome_build","misc_blurb","misc_info","annotation","tags"],searchAliases:{title:"name",format:"file_ext",database:"genome_build",blurb:"misc_blurb",description:"misc_blurb",info:"misc_info",tag:"tags"},toString:function(){var m=this.get("id")||"";if(this.get("name")){m=this.get("hid")+' :"'+this.get("name")+'",'+m}return"HDA("+m+")"}});l.STATES={UPLOAD:"upload",QUEUED:"queued",RUNNING:"running",SETTING_METADATA:"setting_metadata",NEW:"new",EMPTY:"empty",OK:"ok",PAUSED:"paused",FAILED_METADATA:"failed_metadata",NOT_VIEWABLE:"noPermission",DISCARDED:"discarded",ERROR:"error"};l.READY_STATES=[l.STATES.OK,l.STATES.EMPTY,l.STATES.PAUSED,l.STATES.FAILED_METADATA,l.STATES.NOT_VIEWABLE,l.STATES.DISCARDED,l.STATES.ERROR];l.NOT_READY_STATES=[l.STATES.UPLOAD,l.STATES.QUEUED,l.STATES.RUNNING,l.STATES.SETTING_METADATA,l.STATES.NEW];var b=Backbone.Collection.extend(e.LoggableMixin).extend({model:function(n,m){if(n.history_content_type=="dataset"){return new l(n,m)}else{if(n.history_content_type=="dataset_collection"){return new j(n,m)}else{}}},urlRoot:galaxy_config.root+"api/histories",url:function(){return this.urlRoot+"/"+this.historyId+"/contents"},initialize:function(n,m){m=m||{};this.historyId=m.historyId},ids:function(){return this.map(function(m){return m.id})},notReady:function(){return this.filter(function(m){return !m.inReadyState()})},running:function(){var m=[];this.each(function(n){if(!n.inReadyState()){m.push(n.get("id"))}});return m},getByHid:function(m){return _.first(this.filter(function(n){return n.get("hid")===m}))},getVisible:function(m,p,o){o=o||[];var n=new b(this.filter(function(q){return q.isVisible(m,p)}));_.each(o,function(q){if(!_.isFunction(q)){return}n=new b(n.filter(q))});return n},haveDetails:function(){return this.all(function(m){return m.hasDetails()})},fetchAllDetails:function(n){n=n||{};var m={details:"all"};n.data=(n.data)?(_.extend(n.data,m)):(m);return this.fetch(n)},ajaxQueue:function(p,o){var n=jQuery.Deferred(),m=this.length,r=[];if(!m){n.resolve([]);return n}var q=this.chain().reverse().map(function(t,s){return function(){var u=p.call(t,o);u.done(function(v){n.notify({curr:s,total:m,response:v,model:t})});u.always(function(v){r.push(v);if(q.length){q.shift()()}else{n.resolve(r)}})}}).value();q.shift()();return n},matches:function(m){return this.filter(function(n){return n.matches(m)})},set:function(o,m){var n=this;o=_.map(o,function(q){var r=n.get(q.id);if(!r){return q}var p=r.toJSON();_.extend(p,q);return p});Backbone.Collection.prototype.set.call(this,o,m)},promoteToHistoryDatasetCollection:function k(r,p,n){n=n||{};n.url=this.url();n.type="POST";var t=p;var q=[],m=null;if(p=="list"){this.chain().each(function(w){var u=w.attributes.name;var x=w.id;var v=w.attributes.history_content_type;if(v=="dataset"){if(t!="list"){console.log("Invalid collection type")}q.push({name:u,src:"hda",id:x})}else{if(t=="list"){t="list:"+w.attributes.collection_type}else{if(t!="list:"+w.attributes.collection_type){console.log("Invalid collection type")}}q.push({name:u,src:"hdca",id:x})}});m="New Dataset List"}else{if(p=="paired"){var o=this.ids();if(o.length!=2){}q.push({name:"forward",src:"hda",id:o[0]});q.push({name:"reverse",src:"hda",id:o[1]});m="New Dataset Pair"}}n.data={type:"dataset_collection",name:m,collection_type:t,element_identifiers:JSON.stringify(q),};var s=jQuery.ajax(n);s.done(function(w,u,v){r.refresh()});s.fail(function(w,u,v){if(w.responseJSON&&w.responseJSON.error){error=w.responseJSON.error}else{error=w.responseJSON}w.responseText=error});return s},toString:function(){return(["HDACollection(",[this.historyId,this.length].join(),")"].join(""))}});var j=a.extend({defaults:{history_id:null,model_class:"HistoryDatasetCollectionAssociation",history_content_type:"dataset_collection",hid:0,id:null,name:"(unnamed dataset collection)",state:"ok",accessible:true,deleted:false,visible:true,purged:false,tags:[],annotation:""},urls:function(){},inReadyState:function(){return true},searchAttributes:["name"],searchAliases:{title:"name"},});return{HistoryDatasetAssociation:l,HDACollection:b}}); \ No newline at end of file https://bitbucket.org/galaxy/galaxy-central/commits/cc19c4caa043/ Changeset: cc19c4caa043 User: jmchilton Date: 2014-05-22 18:02:17 Summary: Merge next-stable. Affected #: 2 files diff -r 21a66ac74e37c80fe6d5a7c44988f83db5413d87 -r cc19c4caa0434bd871b390a64113ad735c4ba223 static/scripts/mvc/dataset/hda-model.js --- a/static/scripts/mvc/dataset/hda-model.js +++ b/static/scripts/mvc/dataset/hda-model.js @@ -667,8 +667,8 @@ if( ids.length != 2 ){ // TODO: Do something... } - element_identifiers.push( { name: "left", src: "hda", id: ids[ 0 ] } ); - element_identifiers.push( { name: "right", src: "hda", id: ids[ 1 ] } ); + element_identifiers.push( { name: "forward", src: "hda", id: ids[ 0 ] } ); + element_identifiers.push( { name: "reverse", src: "hda", id: ids[ 1 ] } ); name = "New Dataset Pair"; } options.data = {type: "dataset_collection", diff -r 21a66ac74e37c80fe6d5a7c44988f83db5413d87 -r cc19c4caa0434bd871b390a64113ad735c4ba223 static/scripts/packed/mvc/dataset/hda-model.js --- a/static/scripts/packed/mvc/dataset/hda-model.js +++ b/static/scripts/packed/mvc/dataset/hda-model.js @@ -1,1 +1,1 @@ -define(["mvc/base-mvc","utils/localization"],function(e,c){var a=Backbone.Model.extend(e.LoggableMixin).extend({urlRoot:galaxy_config.root+"api/histories/",url:function(){return this.urlRoot+this.get("history_id")+"/contents/"+this.get("history_content_type")+"s/"+this.get("id")},hidden:function(){return !this.get("visible")},"delete":function f(m){if(this.get("deleted")){return jQuery.when()}return this.save({deleted:true},m)},undelete:function i(m){if(!this.get("deleted")||this.get("purged")){return jQuery.when()}return this.save({deleted:false},m)},hide:function d(m){if(!this.get("visible")){return jQuery.when()}return this.save({visible:false},m)},unhide:function h(m){if(this.get("visible")){return jQuery.when()}return this.save({visible:true},m)},isVisible:function(n,o){var m=true;if((!n)&&(this.get("deleted")||this.get("purged"))){m=false}if((!o)&&(!this.get("visible"))){m=false}return m},searchAttribute:function(o,m){var n=this.get(o);if(!m||(n===undefined||n===null)){return false}if(_.isArray(n)){return this._searchArrayAttribute(n,m)}return(n.toString().toLowerCase().indexOf(m.toLowerCase())!==-1)},_searchArrayAttribute:function(n,m){m=m.toLowerCase();return _.any(n,function(o){return(o.toString().toLowerCase().indexOf(m.toLowerCase())!==-1)})},search:function(m){var n=this;return _.filter(this.searchAttributes,function(o){return n.searchAttribute(o,m)})},matches:function(n){var p="=",m=n.split(p);if(m.length>=2){var o=m[0];o=this.searchAliases[o]||o;return this.searchAttribute(o,m[1])}return !!this.search(n).length},matchesAll:function(n){var m=this;n=n.match(/(".*"|\w*=".*"|\S*)/g).filter(function(o){return !!o});return _.all(n,function(o){o=o.replace(/"/g,"");return m.matches(o)})}});var l=a.extend({defaults:{history_id:null,model_class:"HistoryDatasetAssociation",history_content_type:"dataset",hid:0,id:null,name:"(unnamed dataset)",state:"new",deleted:false,visible:true,accessible:true,purged:false,data_type:"",file_size:0,file_ext:"",meta_files:[],misc_blurb:"",misc_info:"",tags:[],annotation:""},urls:function(){var n=this.get("id");if(!n){return{}}var m={purge:galaxy_config.root+"datasets/"+n+"/purge_async",display:galaxy_config.root+"datasets/"+n+"/display/?preview=True",edit:galaxy_config.root+"datasets/"+n+"/edit",download:galaxy_config.root+"datasets/"+n+"/display?to_ext="+this.get("file_ext"),report_error:galaxy_config.root+"dataset/errors?id="+n,rerun:galaxy_config.root+"tool_runner/rerun?id="+n,show_params:galaxy_config.root+"datasets/"+n+"/show_params",visualization:galaxy_config.root+"visualization",annotation:{get:galaxy_config.root+"dataset/get_annotation_async?id="+n,set:galaxy_config.root+"dataset/annotate_async?id="+n},meta_download:galaxy_config.root+"dataset/get_metadata_file?hda_id="+n+"&metadata_name="};return m},initialize:function(m){this.log(this+".initialize",this.attributes);this.log("\tparent history_id: "+this.get("history_id"));if(!this.get("accessible")){this.set("state",l.STATES.NOT_VIEWABLE)}this._setUpListeners()},_setUpListeners:function(){this.on("change:state",function(n,m){this.log(this+" has changed state:",n,m);if(this.inReadyState()){this.trigger("state:ready",n,m,this.previous("state"))}})},isDeletedOrPurged:function(){return(this.get("deleted")||this.get("purged"))},inReadyState:function(){var m=_.contains(l.READY_STATES,this.get("state"));return(this.isDeletedOrPurged()||m)},hasDetails:function(){return _.has(this.attributes,"genome_build")},hasData:function(){return(this.get("file_size")>0)},"delete":function f(m){if(this.get("deleted")){return jQuery.when()}return this.save({deleted:true},m)},undelete:function i(m){if(!this.get("deleted")||this.get("purged")){return jQuery.when()}return this.save({deleted:false},m)},hide:function d(m){if(!this.get("visible")){return jQuery.when()}return this.save({visible:false},m)},unhide:function h(m){if(this.get("visible")){return jQuery.when()}return this.save({visible:true},m)},purge:function g(m){if(this.get("purged")){return jQuery.when()}m=m||{};m.url=galaxy_config.root+"datasets/"+this.get("id")+"/purge_async";var n=this,o=jQuery.ajax(m);o.done(function(r,p,q){n.set({deleted:true,purged:true})});o.fail(function(t,p,s){var q=c("Unable to purge dataset");var r=("Removal of datasets by users is not allowed in this Galaxy instance");if(t.responseJSON&&t.responseJSON.error){q=t.responseJSON.error}else{if(t.responseText.indexOf(r)!==-1){q=r}}t.responseText=q;n.trigger("error",n,t,m,c(q),{error:q})});return o},searchAttributes:["name","file_ext","genome_build","misc_blurb","misc_info","annotation","tags"],searchAliases:{title:"name",format:"file_ext",database:"genome_build",blurb:"misc_blurb",description:"misc_blurb",info:"misc_info",tag:"tags"},toString:function(){var m=this.get("id")||"";if(this.get("name")){m=this.get("hid")+' :"'+this.get("name")+'",'+m}return"HDA("+m+")"}});l.STATES={UPLOAD:"upload",QUEUED:"queued",RUNNING:"running",SETTING_METADATA:"setting_metadata",NEW:"new",EMPTY:"empty",OK:"ok",PAUSED:"paused",FAILED_METADATA:"failed_metadata",NOT_VIEWABLE:"noPermission",DISCARDED:"discarded",ERROR:"error"};l.READY_STATES=[l.STATES.OK,l.STATES.EMPTY,l.STATES.PAUSED,l.STATES.FAILED_METADATA,l.STATES.NOT_VIEWABLE,l.STATES.DISCARDED,l.STATES.ERROR];l.NOT_READY_STATES=[l.STATES.UPLOAD,l.STATES.QUEUED,l.STATES.RUNNING,l.STATES.SETTING_METADATA,l.STATES.NEW];var b=Backbone.Collection.extend(e.LoggableMixin).extend({model:function(n,m){if(n.history_content_type=="dataset"){return new l(n,m)}else{if(n.history_content_type=="dataset_collection"){return new j(n,m)}else{}}},urlRoot:galaxy_config.root+"api/histories",url:function(){return this.urlRoot+"/"+this.historyId+"/contents"},initialize:function(n,m){m=m||{};this.historyId=m.historyId},ids:function(){return this.map(function(m){return m.id})},notReady:function(){return this.filter(function(m){return !m.inReadyState()})},running:function(){var m=[];this.each(function(n){if(!n.inReadyState()){m.push(n.get("id"))}});return m},getByHid:function(m){return _.first(this.filter(function(n){return n.get("hid")===m}))},getVisible:function(m,p,o){o=o||[];var n=new b(this.filter(function(q){return q.isVisible(m,p)}));_.each(o,function(q){if(!_.isFunction(q)){return}n=new b(n.filter(q))});return n},haveDetails:function(){return this.all(function(m){return m.hasDetails()})},fetchAllDetails:function(n){n=n||{};var m={details:"all"};n.data=(n.data)?(_.extend(n.data,m)):(m);return this.fetch(n)},ajaxQueue:function(p,o){var n=jQuery.Deferred(),m=this.length,r=[];if(!m){n.resolve([]);return n}var q=this.chain().reverse().map(function(t,s){return function(){var u=p.call(t,o);u.done(function(v){n.notify({curr:s,total:m,response:v,model:t})});u.always(function(v){r.push(v);if(q.length){q.shift()()}else{n.resolve(r)}})}}).value();q.shift()();return n},matches:function(m){return this.filter(function(n){return n.matches(m)})},set:function(o,m){var n=this;o=_.map(o,function(q){var r=n.get(q.id);if(!r){return q}var p=r.toJSON();_.extend(p,q);return p});Backbone.Collection.prototype.set.call(this,o,m)},promoteToHistoryDatasetCollection:function k(r,p,n){n=n||{};n.url=this.url();n.type="POST";var t=p;var q=[],m=null;if(p=="list"){this.chain().each(function(w){var u=w.attributes.name;var x=w.id;var v=w.attributes.history_content_type;if(v=="dataset"){if(t!="list"){console.log("Invalid collection type")}q.push({name:u,src:"hda",id:x})}else{if(t=="list"){t="list:"+w.attributes.collection_type}else{if(t!="list:"+w.attributes.collection_type){console.log("Invalid collection type")}}q.push({name:u,src:"hdca",id:x})}});m="New Dataset List"}else{if(p=="paired"){var o=this.ids();if(o.length!=2){}q.push({name:"left",src:"hda",id:o[0]});q.push({name:"right",src:"hda",id:o[1]});m="New Dataset Pair"}}n.data={type:"dataset_collection",name:m,collection_type:t,element_identifiers:JSON.stringify(q),};var s=jQuery.ajax(n);s.done(function(w,u,v){r.refresh()});s.fail(function(w,u,v){if(w.responseJSON&&w.responseJSON.error){error=w.responseJSON.error}else{error=w.responseJSON}w.responseText=error});return s},toString:function(){return(["HDACollection(",[this.historyId,this.length].join(),")"].join(""))}});var j=a.extend({defaults:{history_id:null,model_class:"HistoryDatasetCollectionAssociation",history_content_type:"dataset_collection",hid:0,id:null,name:"(unnamed dataset collection)",state:"ok",accessible:true,deleted:false,visible:true,purged:false,tags:[],annotation:""},urls:function(){},inReadyState:function(){return true},searchAttributes:["name"],searchAliases:{title:"name"},});return{HistoryDatasetAssociation:l,HDACollection:b}}); \ No newline at end of file +define(["mvc/base-mvc","utils/localization"],function(e,c){var a=Backbone.Model.extend(e.LoggableMixin).extend({urlRoot:galaxy_config.root+"api/histories/",url:function(){return this.urlRoot+this.get("history_id")+"/contents/"+this.get("history_content_type")+"s/"+this.get("id")},hidden:function(){return !this.get("visible")},"delete":function f(m){if(this.get("deleted")){return jQuery.when()}return this.save({deleted:true},m)},undelete:function i(m){if(!this.get("deleted")||this.get("purged")){return jQuery.when()}return this.save({deleted:false},m)},hide:function d(m){if(!this.get("visible")){return jQuery.when()}return this.save({visible:false},m)},unhide:function h(m){if(this.get("visible")){return jQuery.when()}return this.save({visible:true},m)},isVisible:function(n,o){var m=true;if((!n)&&(this.get("deleted")||this.get("purged"))){m=false}if((!o)&&(!this.get("visible"))){m=false}return m},searchAttribute:function(o,m){var n=this.get(o);if(!m||(n===undefined||n===null)){return false}if(_.isArray(n)){return this._searchArrayAttribute(n,m)}return(n.toString().toLowerCase().indexOf(m.toLowerCase())!==-1)},_searchArrayAttribute:function(n,m){m=m.toLowerCase();return _.any(n,function(o){return(o.toString().toLowerCase().indexOf(m.toLowerCase())!==-1)})},search:function(m){var n=this;return _.filter(this.searchAttributes,function(o){return n.searchAttribute(o,m)})},matches:function(n){var p="=",m=n.split(p);if(m.length>=2){var o=m[0];o=this.searchAliases[o]||o;return this.searchAttribute(o,m[1])}return !!this.search(n).length},matchesAll:function(n){var m=this;n=n.match(/(".*"|\w*=".*"|\S*)/g).filter(function(o){return !!o});return _.all(n,function(o){o=o.replace(/"/g,"");return m.matches(o)})}});var l=a.extend({defaults:{history_id:null,model_class:"HistoryDatasetAssociation",history_content_type:"dataset",hid:0,id:null,name:"(unnamed dataset)",state:"new",deleted:false,visible:true,accessible:true,purged:false,data_type:"",file_size:0,file_ext:"",meta_files:[],misc_blurb:"",misc_info:"",tags:[],annotation:""},urls:function(){var n=this.get("id");if(!n){return{}}var m={purge:galaxy_config.root+"datasets/"+n+"/purge_async",display:galaxy_config.root+"datasets/"+n+"/display/?preview=True",edit:galaxy_config.root+"datasets/"+n+"/edit",download:galaxy_config.root+"datasets/"+n+"/display?to_ext="+this.get("file_ext"),report_error:galaxy_config.root+"dataset/errors?id="+n,rerun:galaxy_config.root+"tool_runner/rerun?id="+n,show_params:galaxy_config.root+"datasets/"+n+"/show_params",visualization:galaxy_config.root+"visualization",annotation:{get:galaxy_config.root+"dataset/get_annotation_async?id="+n,set:galaxy_config.root+"dataset/annotate_async?id="+n},meta_download:galaxy_config.root+"dataset/get_metadata_file?hda_id="+n+"&metadata_name="};return m},initialize:function(m){this.log(this+".initialize",this.attributes);this.log("\tparent history_id: "+this.get("history_id"));if(!this.get("accessible")){this.set("state",l.STATES.NOT_VIEWABLE)}this._setUpListeners()},_setUpListeners:function(){this.on("change:state",function(n,m){this.log(this+" has changed state:",n,m);if(this.inReadyState()){this.trigger("state:ready",n,m,this.previous("state"))}})},isDeletedOrPurged:function(){return(this.get("deleted")||this.get("purged"))},inReadyState:function(){var m=_.contains(l.READY_STATES,this.get("state"));return(this.isDeletedOrPurged()||m)},hasDetails:function(){return _.has(this.attributes,"genome_build")},hasData:function(){return(this.get("file_size")>0)},"delete":function f(m){if(this.get("deleted")){return jQuery.when()}return this.save({deleted:true},m)},undelete:function i(m){if(!this.get("deleted")||this.get("purged")){return jQuery.when()}return this.save({deleted:false},m)},hide:function d(m){if(!this.get("visible")){return jQuery.when()}return this.save({visible:false},m)},unhide:function h(m){if(this.get("visible")){return jQuery.when()}return this.save({visible:true},m)},purge:function g(m){if(this.get("purged")){return jQuery.when()}m=m||{};m.url=galaxy_config.root+"datasets/"+this.get("id")+"/purge_async";var n=this,o=jQuery.ajax(m);o.done(function(r,p,q){n.set({deleted:true,purged:true})});o.fail(function(t,p,s){var q=c("Unable to purge dataset");var r=("Removal of datasets by users is not allowed in this Galaxy instance");if(t.responseJSON&&t.responseJSON.error){q=t.responseJSON.error}else{if(t.responseText.indexOf(r)!==-1){q=r}}t.responseText=q;n.trigger("error",n,t,m,c(q),{error:q})});return o},searchAttributes:["name","file_ext","genome_build","misc_blurb","misc_info","annotation","tags"],searchAliases:{title:"name",format:"file_ext",database:"genome_build",blurb:"misc_blurb",description:"misc_blurb",info:"misc_info",tag:"tags"},toString:function(){var m=this.get("id")||"";if(this.get("name")){m=this.get("hid")+' :"'+this.get("name")+'",'+m}return"HDA("+m+")"}});l.STATES={UPLOAD:"upload",QUEUED:"queued",RUNNING:"running",SETTING_METADATA:"setting_metadata",NEW:"new",EMPTY:"empty",OK:"ok",PAUSED:"paused",FAILED_METADATA:"failed_metadata",NOT_VIEWABLE:"noPermission",DISCARDED:"discarded",ERROR:"error"};l.READY_STATES=[l.STATES.OK,l.STATES.EMPTY,l.STATES.PAUSED,l.STATES.FAILED_METADATA,l.STATES.NOT_VIEWABLE,l.STATES.DISCARDED,l.STATES.ERROR];l.NOT_READY_STATES=[l.STATES.UPLOAD,l.STATES.QUEUED,l.STATES.RUNNING,l.STATES.SETTING_METADATA,l.STATES.NEW];var b=Backbone.Collection.extend(e.LoggableMixin).extend({model:function(n,m){if(n.history_content_type=="dataset"){return new l(n,m)}else{if(n.history_content_type=="dataset_collection"){return new j(n,m)}else{}}},urlRoot:galaxy_config.root+"api/histories",url:function(){return this.urlRoot+"/"+this.historyId+"/contents"},initialize:function(n,m){m=m||{};this.historyId=m.historyId},ids:function(){return this.map(function(m){return m.id})},notReady:function(){return this.filter(function(m){return !m.inReadyState()})},running:function(){var m=[];this.each(function(n){if(!n.inReadyState()){m.push(n.get("id"))}});return m},getByHid:function(m){return _.first(this.filter(function(n){return n.get("hid")===m}))},getVisible:function(m,p,o){o=o||[];var n=new b(this.filter(function(q){return q.isVisible(m,p)}));_.each(o,function(q){if(!_.isFunction(q)){return}n=new b(n.filter(q))});return n},haveDetails:function(){return this.all(function(m){return m.hasDetails()})},fetchAllDetails:function(n){n=n||{};var m={details:"all"};n.data=(n.data)?(_.extend(n.data,m)):(m);return this.fetch(n)},ajaxQueue:function(p,o){var n=jQuery.Deferred(),m=this.length,r=[];if(!m){n.resolve([]);return n}var q=this.chain().reverse().map(function(t,s){return function(){var u=p.call(t,o);u.done(function(v){n.notify({curr:s,total:m,response:v,model:t})});u.always(function(v){r.push(v);if(q.length){q.shift()()}else{n.resolve(r)}})}}).value();q.shift()();return n},matches:function(m){return this.filter(function(n){return n.matches(m)})},set:function(o,m){var n=this;o=_.map(o,function(q){var r=n.get(q.id);if(!r){return q}var p=r.toJSON();_.extend(p,q);return p});Backbone.Collection.prototype.set.call(this,o,m)},promoteToHistoryDatasetCollection:function k(r,p,n){n=n||{};n.url=this.url();n.type="POST";var t=p;var q=[],m=null;if(p=="list"){this.chain().each(function(w){var u=w.attributes.name;var x=w.id;var v=w.attributes.history_content_type;if(v=="dataset"){if(t!="list"){console.log("Invalid collection type")}q.push({name:u,src:"hda",id:x})}else{if(t=="list"){t="list:"+w.attributes.collection_type}else{if(t!="list:"+w.attributes.collection_type){console.log("Invalid collection type")}}q.push({name:u,src:"hdca",id:x})}});m="New Dataset List"}else{if(p=="paired"){var o=this.ids();if(o.length!=2){}q.push({name:"forward",src:"hda",id:o[0]});q.push({name:"reverse",src:"hda",id:o[1]});m="New Dataset Pair"}}n.data={type:"dataset_collection",name:m,collection_type:t,element_identifiers:JSON.stringify(q),};var s=jQuery.ajax(n);s.done(function(w,u,v){r.refresh()});s.fail(function(w,u,v){if(w.responseJSON&&w.responseJSON.error){error=w.responseJSON.error}else{error=w.responseJSON}w.responseText=error});return s},toString:function(){return(["HDACollection(",[this.historyId,this.length].join(),")"].join(""))}});var j=a.extend({defaults:{history_id:null,model_class:"HistoryDatasetCollectionAssociation",history_content_type:"dataset_collection",hid:0,id:null,name:"(unnamed dataset collection)",state:"ok",accessible:true,deleted:false,visible:true,purged:false,tags:[],annotation:""},urls:function(){},inReadyState:function(){return true},searchAttributes:["name"],searchAliases:{title:"name"},});return{HistoryDatasetAssociation:l,HDACollection:b}}); \ No newline at end of file Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: davebgx: Enable running the create_users and create_categories scripts via run_tool_shed.sh
by commits-noreply＠bitbucket.org 22 May '14

22 May '14

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/21a66ac74e37/ Changeset: 21a66ac74e37 User: davebgx Date: 2014-05-22 16:59:18 Summary: Enable running the create_users and create_categories scripts via run_tool_shed.sh Affected #: 7 files diff -r 581e7c8e353e9348db75d269446651e7122f10ce -r 21a66ac74e37c80fe6d5a7c44988f83db5413d87 lib/tool_shed/scripts/bootstrap_tool_shed/bootstrap_tool_shed.sh --- /dev/null +++ b/lib/tool_shed/scripts/bootstrap_tool_shed/bootstrap_tool_shed.sh @@ -0,0 +1,100 @@ +#!/bin/bash + +stop_err() { + echo $1 + python ./scripts/paster.py serve tool_shed_wsgi.ini --pid-file=tool_shed_bootstrap.pid --log-file=tool_shed_bootstrap.log --stop-daemon + exit 1 +} + +tool_shed=`./lib/tool_shed/scripts/bootstrap_tool_shed/parse_run_sh_args.sh $@` + +if [ $? -ne 0 ] ; then + exit 0 +fi + +log_file="lib/tool_shed/scripts/bootstrap_tool_shed/bootstrap.log" + +database_result=`python ./lib/tool_shed/scripts/bootstrap_tool_shed/bootstrap_util.py --execute check_db --config_file tool_shed_wsgi.ini` + +if [ $? -ne 0 ] ; then + stop_err "Unable to bootstrap tool shed. $database_result" +fi + +echo "Bootstrapping from tool shed at $tool_shed." +echo -n "Creating database... " +python scripts/create_db.py tool_shed + +if [ $? -eq 0 ] ; then + echo "done." +else + stop_err "failed." +fi + +if [ $? -eq 0 ] ; then + user_auth=`python ./lib/tool_shed/scripts/bootstrap_tool_shed/bootstrap_util.py --execute admin_user_info --config_file tool_shed_wsgi.ini` + local_shed_url=`python ./lib/tool_shed/scripts/bootstrap_tool_shed/bootstrap_util.py --execute get_url --config_file tool_shed_wsgi.ini` +fi + +admin_user_name=`echo $user_auth | awk 'BEGIN { FS="__SEP__" } ; { print \$1 }'` +admin_user_email=`echo $user_auth | awk 'BEGIN { FS="__SEP__" } ; { print \$2 }'` +admin_user_password=`echo $user_auth | awk 'BEGIN { FS="__SEP__" } ; { print \$3 }'` + +echo -n "Creating user '$admin_user_name' with email address '$admin_user_email'..." + +python lib/tool_shed/scripts/bootstrap_tool_shed/create_user_with_api_key.py tool_shed_wsgi.ini >> $log_file + +echo " done." + +sed -i "s/#admin_users = user1@example.org,user2@example.org/admin_users = $admin_user_email/" tool_shed_wsgi.ini +echo -n "Starting tool shed in order to populate users and categories... " + +if [ -f tool_shed_bootstrap.pid ] ; then + stop_err "A bootstrap process is already running." +fi + +python ./scripts/paster.py serve tool_shed_wsgi.ini --pid-file=tool_shed_bootstrap.pid --log-file=tool_shed_bootstrap.log --daemon > /dev/null + +shed_pid=`cat tool_shed_bootstrap.pid` + +while : ; do + tail -n 1 tool_shed_bootstrap.log | grep -q "Removing PID file tool_shed_webapp.pid" + if [ $? -eq 0 ] ; then + echo "failed." + echo "More information about this failure may be found in the following log snippet from tool_shed_bootstrap.log:" + echo "========================================" + tail -n 40 tool_shed_bootstrap.log + echo "========================================" + stop_err " " + fi + tail -n 2 tool_shed_bootstrap.log | grep -q "Starting server in PID $shed_pid" + if [ $? -eq 0 ] ; then + echo "done." + break + fi +done + +echo -n "Retrieving admin user's API key..." +api_key_json=`curl -s --user $admin_user_email:$admin_user_password $local_shed_url/api/authenticate/baseauth/` +api_key=`echo $api_key_json | grep api_key | awk 'BEGIN { FS="\"" } ; { print \$4 }' | sed 's/\\s\+//'` + +if [[ -z $api_key && ${api_key+x} ]] ; then + stop_err "Error getting API key for user $admin_user_email." +fi + +echo " done." + +if [ $? -eq 0 ] ; then + echo -n "Creating users... " + python lib/tool_shed/scripts/api/create_users.py -a $api_key -f $tool_shed -t $local_shed_url >> $log_file + echo "done." + echo -n "Creating categories... " + python lib/tool_shed/scripts/api/create_categories.py -a $api_key -f $tool_shed -t $local_shed_url >> $log_file + echo "done." +else + stop_err "Error getting API key from local tool shed." +fi + +echo "Bootstrap complete, shutting down temporary tool shed process. A log has been saved to tool_shed_bootstrap.log" +python ./scripts/paster.py serve tool_shed_wsgi.ini --pid-file=tool_shed_bootstrap.pid --log-file=tool_shed_bootstrap.log --stop-daemon + +exit 0 \ No newline at end of file diff -r 581e7c8e353e9348db75d269446651e7122f10ce -r 21a66ac74e37c80fe6d5a7c44988f83db5413d87 lib/tool_shed/scripts/bootstrap_tool_shed/bootstrap_util.py --- /dev/null +++ b/lib/tool_shed/scripts/bootstrap_tool_shed/bootstrap_util.py @@ -0,0 +1,130 @@ +#!/usr/bin/python +import argparse +import ConfigParser +import os +import sys + +new_path = [ os.path.join( os.getcwd(), "lib" ) ] +new_path.extend( sys.path[1:] ) +sys.path = new_path + +from galaxy import eggs +eggs.require( "SQLAlchemy >= 0.4" ) +import galaxy.webapps.tool_shed.model.mapping as tool_shed_model +from sqlalchemy.exc import ProgrammingError +from sqlalchemy.exc import OperationalError + +from tool_shed.util import xml_util + +def check_db( config_parser ): + dburi = None + + if config_parser.has_option( 'app:main', 'database_connection' ): + dburi = config_parser.get( 'app:main', 'database_connection' ) + elif config_parser.has_option( 'app:main', 'database_file' ): + db_file = config_parser.get( 'app:main', 'database_file' ) + dburi = "sqlite:///%s?isolation_level=IMMEDIATE" % db_file + else: + print 'The database configuration setting is missing from the tool_shed_wsgi.ini file. Add this setting before attempting to bootstrap.' + exit(1) + + sa_session = None + + database_exists_message = 'The database configured for this Tool Shed is not new, so bootstrapping is not allowed. ' + database_exists_message += 'Create a new database that has not been migrated before attempting to bootstrap.' + + try: + model = tool_shed_model.init( config_parser.get( 'app:main', 'file_path' ), dburi, engine_options={}, create_tables=False ) + sa_session = model.context.current + print database_exists_message + exit(1) + except ProgrammingError, e: + pass + except OperationalError, e: + pass + + try: + if sa_session is not None: + result = sa_session.execute( 'SELECT version FROM migrate_version' ).first() + if result[0] >= 2: + print database_exists_message + exit(1) + else: + pass + except ProgrammingError, e: + pass + + if config_parser.has_option( 'app:main', 'hgweb_config_dir' ): + hgweb_config_parser = ConfigParser.ConfigParser() + hgweb_dir = config_parser.get( 'app:main', 'hgweb_config_dir' ) + hgweb_config_file = os.path.join( hgweb_dir, 'hgweb.config' ) + if not os.path.exists( hgweb_config_file ): + exit(0) + hgweb_config_parser.read( hgweb_config_file ) + configured_repos = hgweb_config_parser.items( 'paths' ) + if len( configured_repos ) >= 1: + message = "This Tool Shed's hgweb.config file contains entries, so bootstrapping is not allowed. Delete" + message += " the current hgweb.config file along with all associated repositories in the configured " + message += "location before attempting to boostrap." + print + exit(1) + else: + exit(0) + else: + exit(0) + + exit(0) + +def admin_user_info( config_parser ): + user_info_config = os.path.abspath( os.path.join( os.getcwd(), 'lib/tool_shed/scripts/bootstrap_tool_shed', 'user_info.xml' ) ) + tree, error_message = xml_util.parse_xml( user_info_config ) + if tree is None: + print "The XML file ", user_info_config, " seems to be invalid, using defaults." + email = 'admin(a)test.org' + password = 'testuser' + username = 'admin' + else: + root = tree.getroot() + for elem in root: + if elem.tag == 'email': + email = elem.text + elif elem.tag == 'password': + password = elem.text + elif elem.tag == 'username': + username = elem.text + print '%s__SEP__%s__SEP__%s' % ( username, email, password ) + return 0 + +def get_local_tool_shed_url( config_parser ): + port = '9009' + if config_parser.has_section( 'server:main' ): + if config_parser.has_option( 'server:main', 'port' ): + port = config_parser.get( 'server:main', 'port' ) + host = '127.0.0.1' + print 'http://%s:%s' % ( host, port ) + return 0 + +def main( args ): + config_parser = ConfigParser.ConfigParser() + + if os.path.exists( args.config ): + config_parser.read( args.config ) + else: + return 1 + + if args.method == 'check_db': + return check_db( config_parser ) + elif args.method == 'admin_user_info': + return admin_user_info( config_parser ) + elif args.method == 'get_url': + return get_local_tool_shed_url( config_parser ) + else: + return 1 + +parser = argparse.ArgumentParser() +parser.add_argument( '-c', '--config_file', dest='config', action='store', default='tool_shed_wsgi.ini' ) +parser.add_argument( '-e', '--execute', dest='method', action='store', default='check_db' ) +args = parser.parse_args() + +if __name__ == '__main__': + exit( main( args ) ) \ No newline at end of file diff -r 581e7c8e353e9348db75d269446651e7122f10ce -r 21a66ac74e37c80fe6d5a7c44988f83db5413d87 lib/tool_shed/scripts/bootstrap_tool_shed/create_user_with_api_key.py --- a/lib/tool_shed/scripts/bootstrap_tool_shed/create_user_with_api_key.py +++ b/lib/tool_shed/scripts/bootstrap_tool_shed/create_user_with_api_key.py @@ -125,7 +125,7 @@ return '' def validate_publicname( username ): - """""Validates the public username.""" + """Validates the public username.""" if len( username ) < 3: return "Public name must be at least 3 characters in length" if len( username ) > 255: @@ -154,5 +154,7 @@ if user is not None: api_key = create_api_key( app, user ) print "Created new user with public username '", user.username, ". An API key was also created and associated with the user." + exit(0) else: print "Problem creating a new user and an associated API key." + exit(1) diff -r 581e7c8e353e9348db75d269446651e7122f10ce -r 21a66ac74e37c80fe6d5a7c44988f83db5413d87 lib/tool_shed/scripts/bootstrap_tool_shed/parse_run_sh_args.sh --- /dev/null +++ b/lib/tool_shed/scripts/bootstrap_tool_shed/parse_run_sh_args.sh @@ -0,0 +1,15 @@ +#!/bin/bash + +while (( $# )) ; do + case "$1" in + -bootstrap_from_tool_shed|--bootstrap_from_tool_shed) + bootstrap="true" + tool_shed=$2 + echo $tool_shed + exit 0 + break + ;; + esac + shift 1 +done +exit 1 \ No newline at end of file diff -r 581e7c8e353e9348db75d269446651e7122f10ce -r 21a66ac74e37c80fe6d5a7c44988f83db5413d87 run.sh --- a/run.sh +++ b/run.sh @@ -5,41 +5,7 @@ python ./scripts/check_python.py [ $? -ne 0 ] && exit 1 -SAMPLES=" - community_wsgi.ini.sample - datatypes_conf.xml.sample - external_service_types_conf.xml.sample - migrated_tools_conf.xml.sample - reports_wsgi.ini.sample - shed_tool_conf.xml.sample - tool_conf.xml.sample - shed_tool_data_table_conf.xml.sample - tool_data_table_conf.xml.sample - tool_sheds_conf.xml.sample - data_manager_conf.xml.sample - shed_data_manager_conf.xml.sample - openid_conf.xml.sample - job_metrics_conf.xml.sample - universe_wsgi.ini.sample - tool-data/shared/ncbi/builds.txt.sample - tool-data/shared/ensembl/builds.txt.sample - tool-data/shared/ucsc/builds.txt.sample - tool-data/shared/ucsc/publicbuilds.txt.sample - tool-data/shared/ucsc/ucsc_build_sites.txt.sample - tool-data/shared/igv/igv_build_sites.txt.sample - tool-data/shared/rviewer/rviewer_build_sites.txt.sample - tool-data/*.sample - static/welcome.html.sample -" - -# Create any missing config/location files -for sample in $SAMPLES; do - file=`echo $sample | sed -e 's/\.sample$//'` - if [ ! -f "$file" -a -f "$sample" ]; then - echo "Initializing $file from `basename $sample`" - cp $sample $file - fi -done +./scripts/copy_sample_files.sh if [ -n "$GALAXY_UNIVERSE_CONFIG_DIR" ]; then python ./scripts/build_universe_config.py "$GALAXY_UNIVERSE_CONFIG_DIR" diff -r 581e7c8e353e9348db75d269446651e7122f10ce -r 21a66ac74e37c80fe6d5a7c44988f83db5413d87 run_tool_shed.sh --- a/run_tool_shed.sh +++ b/run_tool_shed.sh @@ -2,37 +2,14 @@ cd `dirname $0` -SAMPLES=" - tool_shed_wsgi.ini.sample - datatypes_conf.xml.sample - external_service_types_conf.xml.sample - migrated_tools_conf.xml.sample - reports_wsgi.ini.sample - shed_tool_conf.xml.sample - tool_conf.xml.sample - shed_tool_data_table_conf.xml.sample - tool_data_table_conf.xml.sample - tool_sheds_conf.xml.sample - openid_conf.xml.sample - universe_wsgi.ini.sample - lib/tool_shed/scripts/bootstrap_tool_shed/user_info.xml.sample - tool-data/shared/ncbi/builds.txt.sample - tool-data/shared/ensembl/builds.txt.sample - tool-data/shared/ucsc/builds.txt.sample - tool-data/shared/ucsc/publicbuilds.txt.sample - tool-data/shared/igv/igv_build_sites.txt.sample - tool-data/shared/rviewer/rviewer_build_sites.txt.sample - tool-data/*.sample - static/welcome.html.sample -" +./scripts/copy_sample_files.sh -# Create any missing config/location files -for sample in $SAMPLES; do - file=`echo $sample | sed -e 's/\.sample$//'` - if [ ! -f "$file" -a -f "$sample" ]; then - echo "Initializing $file from `basename $sample`" - cp $sample $file - fi -done +tool_shed=`./lib/tool_shed/scripts/bootstrap_tool_shed/parse_run_sh_args.sh $@` +args=$@ -python ./scripts/paster.py serve tool_shed_wsgi.ini --pid-file=tool_shed_webapp.pid --log-file=tool_shed_webapp.log $@ +if [ $? -eq 0 ] ; then + bash ./lib/tool_shed/scripts/bootstrap_tool_shed/bootstrap_tool_shed.sh $@ + args=`echo $@ | sed "s#-\?-bootstrap_from_tool_shed $tool_shed##"` +fi + +python ./scripts/paster.py serve tool_shed_wsgi.ini --pid-file=tool_shed_webapp.pid --log-file=tool_shed_webapp.log $args diff -r 581e7c8e353e9348db75d269446651e7122f10ce -r 21a66ac74e37c80fe6d5a7c44988f83db5413d87 scripts/copy_sample_files.sh --- /dev/null +++ b/scripts/copy_sample_files.sh @@ -0,0 +1,39 @@ +#!/bin/bash + +SAMPLES=" + tool_shed_wsgi.ini.sample + datatypes_conf.xml.sample + external_service_types_conf.xml.sample + migrated_tools_conf.xml.sample + reports_wsgi.ini.sample + shed_tool_conf.xml.sample + tool_conf.xml.sample + shed_tool_data_table_conf.xml.sample + tool_data_table_conf.xml.sample + tool_sheds_conf.xml.sample + data_manager_conf.xml.sample + shed_data_manager_conf.xml.sample + openid_conf.xml.sample + job_metrics_conf.xml.sample + universe_wsgi.ini.sample + lib/tool_shed/scripts/bootstrap_tool_shed/user_info.xml.sample + tool-data/shared/ncbi/builds.txt.sample + tool-data/shared/ensembl/builds.txt.sample + tool-data/shared/ucsc/builds.txt.sample + tool-data/shared/ucsc/publicbuilds.txt.sample + tool-data/shared/ucsc/ucsc_build_sites.txt.sample + tool-data/shared/igv/igv_build_sites.txt.sample + tool-data/shared/rviewer/rviewer_build_sites.txt.sample + tool-data/*.sample + static/welcome.html.sample +" + +# Create any missing config/location files +for sample in $SAMPLES; do + file=${sample%.sample} + if [ ! -f "$file" -a -f "$sample" ]; then + echo "Initializing $file from `basename $sample`" + cp $sample $file + fi +done + Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0