galaxy-commits February 2015

galaxy-commits@lists.galaxyproject.org

2 participants
305 discussions

[galaxyproject/usegalaxy-playbook] 7bd1a2: Turns out XML parsers can't parse invalid XML.
by GitHub 26 Feb '15

26 Feb '15

Branch: refs/heads/master Home: https://github.com/galaxyproject/usegalaxy-playbook Commit: 7bd1a20f612babbebec8c088281f83487d08c652 https://github.com/galaxyproject/usegalaxy-playbook/commit/7bd1a20f612babbe… Author: Nate Coraor <nate(a)bx.psu.edu> Date: 2015-02-26 (Thu, 26 Feb 2015) Changed paths: M templates/galaxy/test.galaxyproject.org/config/job_conf.xml.j2 Log Message: ----------- Turns out XML parsers can't parse invalid XML.

1 0

[galaxyproject/usegalaxy-playbook] 52cd35: Increase NVC memory to 16GB on Test.
by GitHub 26 Feb '15

26 Feb '15

Branch: refs/heads/master Home: https://github.com/galaxyproject/usegalaxy-playbook Commit: 52cd351e8b2324ff11d8e54937f8f8dde4fe6374 https://github.com/galaxyproject/usegalaxy-playbook/commit/52cd351e8b2324ff… Author: Nate Coraor <nate(a)bx.psu.edu> Date: 2015-02-26 (Thu, 26 Feb 2015) Changed paths: M files/galaxy/test.galaxyproject.org/var/integrated_tool_panel.xml M templates/galaxy/test.galaxyproject.org/config/job_conf.xml.j2 Log Message: ----------- Increase NVC memory to 16GB on Test.

1 0

[galaxyproject/usegalaxy-playbook] 87c37a: Update Test and Main
by GitHub 26 Feb '15

26 Feb '15

Branch: refs/heads/master Home: https://github.com/galaxyproject/usegalaxy-playbook Commit: 87c37a944d5425daee366dd712461a4b9f2e6f06 https://github.com/galaxyproject/usegalaxy-playbook/commit/87c37a944d5425da… Author: Nate Coraor <nate(a)bx.psu.edu> Date: 2015-02-25 (Wed, 25 Feb 2015) Changed paths: M files/galaxy/usegalaxy.org/var/integrated_tool_panel.xml M production/group_vars/all.yml M stage/group_vars/all.yml Log Message: ----------- Update Test and Main

1 0

commit/galaxy-central: 6 new changesets
by commits-noreply＠bitbucket.org 26 Feb '15

26 Feb '15

6 new commits in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/b0255200ef92/ Changeset: b0255200ef92 Branch: release_15.03 User: carlfeberhard Date: 2015-02-24 22:20:10+00:00 Summary: Fix to layering of dropdown buttons and menus Affected #: 3 files diff -r 4a42638e8cd45652db9dc1f1eed54be6aa2c20b1 -r b0255200ef921a6de641b303ef486ac56c0d8740 static/style/blue/base.css --- a/static/style/blue/base.css +++ b/static/style/blue/base.css @@ -2030,8 +2030,8 @@ .list-panel .controls .list-actions:after{clear:both} .list-panel .controls .list-actions:before,.list-panel .controls .list-actions:after{content:" ";display:table;} .list-panel .controls .list-actions:after{clear:both} -.list-panel .controls .list-actions .btn{padding-top:2px;padding-bottom:2px;font-size:90%;z-index:inherit} -.list-panel .controls .list-actions .list-action-menu{float:right}.list-panel .controls .list-actions .list-action-menu .dropdown-menu{z-index:inherit} +.list-panel .controls .list-actions .btn{padding-top:2px;padding-bottom:2px;font-size:90%} +.list-panel .controls .list-actions .list-action-menu{float:right} .list-panel .list-items .list-item:not(:last-child){border-bottom-width:0px} .list-panel .empty-message{display:none;margin:0px} .list-item .details .list-panel{margin-top:8px;border-radius:3px;background:white;padding:4px}.list-item .details .list-panel .list-items{border:1px solid #bfbfbf;border-radius:3px}.list-item .details .list-panel .list-items .list-item:first-child{border-top-width:0px;border-radius:3px 3px 0px 0px} @@ -2138,9 +2138,10 @@ .multi-panel-history .control-column.control-column-right{text-align:right} .multi-panel-history .outer-middle{overflow:auto} .multi-panel-history .middle{min-width:100%;margin:0px 0px 0px 0px;background-color:white;padding:8px} -.multi-panel-history .history-column{width:312px;margin:0 8px 0 0}.multi-panel-history .history-column:first-child{position:fixed;z-index:2}.multi-panel-history .history-column:first-child .history-panel{border:1px solid black;box-shadow:4px 4px 4px rgba(96,96,96,0.3)} +.multi-panel-history .history-column{width:312px;margin:0 8px 0 0}.multi-panel-history .history-column:first-child{position:fixed;z-index:10}.multi-panel-history .history-column:first-child .history-panel{border:1px solid black;box-shadow:4px 4px 4px rgba(96,96,96,0.3)} .multi-panel-history .history-column:nth-child(2){margin-left:320px} .multi-panel-history .history-column:last-child{margin-right:0px} +.multi-panel-history .history-column .dropdown-menu{z-index:inherit} .multi-panel-history .history-column .panel-controls{width:100%;height:24px;border-radius:3px;background-color:white;text-align:center;flex:0 0 auto;-webkit-align-self:auto;-ms-flex-item-align:auto;align-self:auto}.multi-panel-history .history-column .panel-controls .btn{height:20px;line-height:normal;font-size:90%;padding-top:0px;padding-bottom:0px} .multi-panel-history .history-column .panel-controls .pull-left .btn{margin-right:4px} .multi-panel-history .history-column .panel-controls .pull-right .btn{margin-left:4px} diff -r 4a42638e8cd45652db9dc1f1eed54be6aa2c20b1 -r b0255200ef921a6de641b303ef486ac56c0d8740 static/style/src/less/history.less --- a/static/style/src/less/history.less +++ b/static/style/src/less/history.less @@ -744,7 +744,7 @@ // current history &:first-child { position: fixed; - z-index : 2; + z-index : 10; // visually differentiate the current history .history-panel { @@ -759,6 +759,9 @@ &:last-child { margin-right: 0px; } + .dropdown-menu { + z-index: inherit; + } .panel-controls { width: 100%; diff -r 4a42638e8cd45652db9dc1f1eed54be6aa2c20b1 -r b0255200ef921a6de641b303ef486ac56c0d8740 static/style/src/less/list-item.less --- a/static/style/src/less/list-item.less +++ b/static/style/src/less/list-item.less @@ -167,13 +167,9 @@ padding-top: 2px; padding-bottom: 2px; font-size: 90%; - z-index: inherit; } .list-action-menu { float: right; - .dropdown-menu { - z-index: inherit; - } } } } https://bitbucket.org/galaxy/galaxy-central/commits/a83c0d80639e/ Changeset: a83c0d80639e Branch: release_15.03 User: dannon Date: 2015-02-24 23:55:50+00:00 Summary: More tweaks to mutable capabilities, adding back in a copy and stripping pickle from MutationList Affected #: 1 file diff -r b0255200ef921a6de641b303ef486ac56c0d8740 -r a83c0d80639eb92228ead6a47e3b6a26c2f1b94e lib/galaxy/model/custom_types.py --- a/lib/galaxy/model/custom_types.py +++ b/lib/galaxy/model/custom_types.py @@ -1,4 +1,5 @@ import binascii +import copy import json import logging import uuid @@ -62,6 +63,12 @@ else: return self.impl + def copy_value( self, value ): + return copy.deepcopy( value ) + + def compare_values( self, x, y ): + return ( x == y ) + class MutationObj(Mutable): """ @@ -196,12 +203,6 @@ list.remove(self, value) self.changed() - def __getstate__(self): - return list(self) - - def __setstate__(self, state): - self.update(state) - MutationObj.associate_with(JSONType) https://bitbucket.org/galaxy/galaxy-central/commits/48012dc829bb/ Changeset: 48012dc829bb Branch: release_15.03 User: jmchilton Date: 2015-02-18 16:34:23+00:00 Summary: Tool diagnostic API endpoints. It is easy to get Galaxy's database and shed tool files out of sync and the results are bad. I don't think this is new but I may have made it worse this cycle by trying to slowly walk back Galaxy's use of the database to track lineages (e.g. https://bitbucket.org/galaxy/galaxy-central/commits/08f8850853d004bf8a456a1…). New API endpoints include: /api/configuration/dynamic_tool_confs /api/configuration/tool_lineages /api/<tool_id>/diagnostics Affected #: 6 files diff -r a83c0d80639eb92228ead6a47e3b6a26c2f1b94e -r 48012dc829bb437e7bcb7c6507d91647a32a26ef lib/galaxy/tools/toolbox/lineages/interface.py --- a/lib/galaxy/tools/toolbox/lineages/interface.py +++ b/lib/galaxy/tools/toolbox/lineages/interface.py @@ -40,3 +40,9 @@ externally). """ return self.version is None + + def to_dict(self): + return dict( + id=self.id, + version=self.version, + ) diff -r a83c0d80639eb92228ead6a47e3b6a26c2f1b94e -r 48012dc829bb437e7bcb7c6507d91647a32a26ef lib/galaxy/tools/toolbox/lineages/stock.py --- a/lib/galaxy/tools/toolbox/lineages/stock.py +++ b/lib/galaxy/tools/toolbox/lineages/stock.py @@ -38,6 +38,13 @@ # method for comparing tool versions. return sorted( versions, key=_to_loose_version ) + def to_dict(self): + return dict( + tool_id=self.tool_id, + tool_versions=list(self.tool_versions), + lineage_type='stock', + ) + def _to_loose_version( tool_lineage_version ): version = str( tool_lineage_version.version ) diff -r a83c0d80639eb92228ead6a47e3b6a26c2f1b94e -r 48012dc829bb437e7bcb7c6507d91647a32a26ef lib/galaxy/tools/toolbox/lineages/tool_shed.py --- a/lib/galaxy/tools/toolbox/lineages/tool_shed.py +++ b/lib/galaxy/tools/toolbox/lineages/tool_shed.py @@ -8,9 +8,11 @@ """ Representation of tool lineage derived from tool shed repository installations. """ - def __init__(self, app, tool_version): + def __init__(self, app, tool_version, tool_shed_repository=None): self.app = app self.tool_version_id = tool_version.id + # Only used for logging + self._tool_shed_repository = tool_shed_repository @staticmethod def from_tool( app, tool, tool_shed_repository ): @@ -36,6 +38,16 @@ def get_versions( self, reverse=False ): return map( ToolLineageVersion.from_guid, self.get_version_ids( reverse=reverse ) ) + def to_dict(self): + tool_shed_repository = self._tool_shed_repository + rval = dict( + tool_version_id=self.tool_version_id, + tool_versions=map(lambda v: v.to_dict(), self.get_versions()), + tool_shed_repository=tool_shed_repository if tool_shed_repository is not None else None, + lineage_type='tool_shed', + ) + return rval + def get_install_tool_version( app, tool_id ): return app.install_model.context.query( diff -r a83c0d80639eb92228ead6a47e3b6a26c2f1b94e -r 48012dc829bb437e7bcb7c6507d91647a32a26ef lib/galaxy/webapps/galaxy/api/configuration.py --- a/lib/galaxy/webapps/galaxy/api/configuration.py +++ b/lib/galaxy/webapps/galaxy/api/configuration.py @@ -4,6 +4,8 @@ """ from galaxy.web import _future_expose_api_anonymous as expose_api_anonymous +from galaxy.web import _future_expose_api as expose_api +from galaxy.web import require_admin from galaxy.web.base.controller import BaseAPIController from galaxy.managers import base @@ -33,6 +35,13 @@ @expose_api_anonymous def version( self, trans, **kwds ): + """ + GET /api/version + Return a description of the major version of Galaxy (e.g. 15.03). + + :rtype: dict + :returns: dictionary with major version keyed on 'version_major' + """ return {"version_major": self.app.config.version_major } def get_config_dict( self, trans, return_admin=False, view=None, keys=None, default_view='all' ): @@ -52,6 +61,29 @@ view=view, keys=keys, default_view=default_view ) return serialized + @expose_api + @require_admin + def dynamic_tool_confs(self, trans): + confs = self.app.toolbox.dynamic_confs(include_migrated_tool_conf=True) + return map(_tool_conf_to_dict, confs) + + @expose_api + @require_admin + def tool_lineages(self, trans): + rval = [] + for id, tool in self.app.toolbox.tools(): + if hasattr( tool, 'lineage' ): + lineage_dict = tool.lineage.to_dict() + else: + lineage_dict = None + + entry = dict( + id=id, + lineage=lineage_dict + ) + rval.append(entry) + return rval + #TODO: for lack of a manager file for the config. May well be better in config.py? Circ imports? class ConfigSerializer( base.ModelSerializer ): @@ -122,3 +154,10 @@ 'allow_user_creation' : _defaults_to( False ), 'allow_user_deletion' : _defaults_to( False ), }) + + +def _tool_conf_to_dict(conf): + return dict( + config_filename=conf['config_filename'], + tool_path=conf['tool_path'], + ) diff -r a83c0d80639eb92228ead6a47e3b6a26c2f1b94e -r 48012dc829bb437e7bcb7c6507d91647a32a26ef lib/galaxy/webapps/galaxy/api/tools.py --- a/lib/galaxy/webapps/galaxy/api/tools.py +++ b/lib/galaxy/webapps/galaxy/api/tools.py @@ -94,6 +94,36 @@ message, status = trans.app.toolbox.reload_tool_by_id( tool_id ) return { status: message } + @_future_expose_api + @web.require_admin + def diagnostics( self, trans, id, **kwd ): + """ + GET /api/tools/{tool_id}/diagnostics + Return diagnostic information to help debug panel + and dependency related problems. + """ + to_dict = lambda x: x.to_dict() + tool = self._get_tool( id, user=trans.user ) + if hasattr( tool, 'lineage' ): + lineage_dict = tool.lineage.to_dict() + else: + lineage_dict = None + tool_shed_dependencies = tool.installed_tool_dependencies + if tool_shed_dependencies: + tool_shed_dependencies_dict = map(to_dict, tool_shed_dependencies) + else: + tool_shed_dependencies_dict = None + tool = self._get_tool( id, user=trans.user ) + return { + "tool_id": tool.id, + "tool_version": tool.version, + "dependency_shell_commands": tool.build_dependency_shell_commands(), + "lineage": lineage_dict, + "requirements": map(to_dict, tool.requirements), + "installed_tool_shed_dependencies": tool_shed_dependencies_dict, + "tool_dir": tool.tool_dir, + } + @_future_expose_api_anonymous def citations( self, trans, id, **kwds ): tool = self._get_tool( id, user=trans.user ) diff -r a83c0d80639eb92228ead6a47e3b6a26c2f1b94e -r 48012dc829bb437e7bcb7c6507d91647a32a26ef lib/galaxy/webapps/galaxy/buildapp.py --- a/lib/galaxy/webapps/galaxy/buildapp.py +++ b/lib/galaxy/webapps/galaxy/buildapp.py @@ -210,6 +210,7 @@ webapp.mapper.resource_with_deleted( 'quota', 'quotas', path_prefix='/api' ) webapp.mapper.connect( '/api/tools/{id:.+?}/build', action='build', controller="tools" ) webapp.mapper.connect( '/api/tools/{id:.+?}/reload', action='reload', controller="tools" ) + webapp.mapper.connect( '/api/tools/{id:.+?}/diagnostics', action='diagnostics', controller="tools" ) webapp.mapper.connect( '/api/tools/{id:.+?}/citations', action='citations', controller="tools" ) webapp.mapper.connect( '/api/tools/{id:.+?}/download', action='download', controller="tools" ) webapp.mapper.connect( '/api/tools/{id:.+?}', action='show', controller="tools" ) @@ -221,6 +222,18 @@ webapp.mapper.resource( 'workflow', 'workflows', path_prefix='/api' ) webapp.mapper.resource_with_deleted( 'history', 'histories', path_prefix='/api' ) webapp.mapper.connect( '/api/histories/{history_id}/citations', action='citations', controller="histories" ) + webapp.mapper.connect( + 'dynamic_tool_confs', + '/api/configuration/dynamic_tool_confs', + controller="configuration", + action="dynamic_tool_confs" + ) + webapp.mapper.connect( + 'tool_lineages', + '/api/configuration/tool_lineages', + controller="configuration", + action="tool_lineages" + ) webapp.mapper.resource( 'configuration', 'configuration', path_prefix='/api' ) webapp.mapper.connect( "configuration_version", "/api/version", controller="configuration", https://bitbucket.org/galaxy/galaxy-central/commits/e0916bcf81bb/ Changeset: e0916bcf81bb Branch: release_15.03 User: jmchilton Date: 2015-02-24 00:52:17+00:00 Summary: Improved tool diagnostics API. Affected #: 1 file diff -r 48012dc829bb437e7bcb7c6507d91647a32a26ef -r e0916bcf81bb3a27e31c919e7cc5d2ee6b987a28 lib/galaxy/webapps/galaxy/api/tools.py --- a/lib/galaxy/webapps/galaxy/api/tools.py +++ b/lib/galaxy/webapps/galaxy/api/tools.py @@ -102,6 +102,7 @@ Return diagnostic information to help debug panel and dependency related problems. """ + # TODO: Move this into tool. to_dict = lambda x: x.to_dict() tool = self._get_tool( id, user=trans.user ) if hasattr( tool, 'lineage' ): @@ -113,7 +114,6 @@ tool_shed_dependencies_dict = map(to_dict, tool_shed_dependencies) else: tool_shed_dependencies_dict = None - tool = self._get_tool( id, user=trans.user ) return { "tool_id": tool.id, "tool_version": tool.version, @@ -122,6 +122,11 @@ "requirements": map(to_dict, tool.requirements), "installed_tool_shed_dependencies": tool_shed_dependencies_dict, "tool_dir": tool.tool_dir, + "tool_shed": tool.tool_shed, + "repository_name": tool.repository_name, + "repository_owner": tool.repository_owner, + "installed_changeset_revision": None, + "guid": tool.guid, } @_future_expose_api_anonymous https://bitbucket.org/galaxy/galaxy-central/commits/4de6535bd780/ Changeset: 4de6535bd780 Branch: release_15.03 User: jmchilton Date: 2015-02-25 14:10:17+00:00 Summary: Expand out meta parameters when validating for tool.to_json(). Bring this code closer inline with what handle_input() does and should solve the problem with column validation of parameters that are being 'batched'. See https://trello.com/c/8NweTvLh and comment on https://github.com/galaxyproject/galaxy/commit/d57088409c7b23e55ead4c9f2fad…. Affected #: 1 file diff -r e0916bcf81bb3a27e31c919e7cc5d2ee6b987a28 -r 4de6535bd78019ce34511f4f4ee2784f74963396 lib/galaxy/tools/__init__.py --- a/lib/galaxy/tools/__init__.py +++ b/lib/galaxy/tools/__init__.py @@ -2299,6 +2299,8 @@ # create parameter object params = galaxy.util.Params( kwd, sanitize = False ) + expanded_incomings, _ = expand_meta_parameters( trans, self, params.__dict__ ) + params.__dict__ = expanded_incomings[ 0 ] # convert value to jsonifiable value def jsonify(v): https://bitbucket.org/galaxy/galaxy-central/commits/9dcc6d00e76e/ Changeset: 9dcc6d00e76e Branch: release_15.03 User: dan Date: 2015-02-25 17:07:39+00:00 Summary: Fix for ToolShed not having config.shed_tool_data_path Affected #: 1 file diff -r 4de6535bd78019ce34511f4f4ee2784f74963396 -r 9dcc6d00e76eae06705d79ec7d276202494ad545 lib/galaxy/webapps/tool_shed/config.py --- a/lib/galaxy/webapps/tool_shed/config.py +++ b/lib/galaxy/webapps/tool_shed/config.py @@ -137,6 +137,10 @@ self.citation_cache_lock_dir = resolve_path( kwargs.get( "citation_cache_lock_dir", "database/tool_shed_citations/locks" ), self.root ) @property + def shed_tool_data_path( self ): + return self.tool_data_path + + @property def sentry_dsn_public( self ): """ Sentry URL with private key removed for use in client side scripts, Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: 17 new changesets
by commits-noreply＠bitbucket.org 26 Feb '15

26 Feb '15

17 new commits in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/d381771a18cf/ Changeset: d381771a18cf User: dannon Date: 2015-02-25 13:22:54+00:00 Summary: Cleanup in api/library contents and folders, associated folder manager. Affected #: 3 files diff -r 658833a7988fe335a5aaaac9709f9e6dfed4a7b2 -r d381771a18cfd6e7d8ecf7563041675a26af0ccd lib/galaxy/managers/folders.py --- a/lib/galaxy/managers/folders.py +++ b/lib/galaxy/managers/folders.py @@ -2,9 +2,7 @@ Manager and Serializer for Library Folders. """ -import galaxy.web from galaxy import exceptions -from galaxy.model import orm import logging log = logging.getLogger( __name__ ) @@ -156,7 +154,7 @@ def get_current_roles( self, trans, folder ): """ - Find all roles currently connected to relevant permissions + Find all roles currently connected to relevant permissions on the folder. :param folder: the model object @@ -165,7 +163,7 @@ :returns: dict of current roles for all available permission types :rtype: dictionary """ - # Omit duplicated roles by converting to set + # Omit duplicated roles by converting to set modify_roles = set( trans.app.security_agent.get_roles_for_action( folder, trans.app.security_agent.permitted_actions.LIBRARY_MODIFY ) ) manage_roles = set( trans.app.security_agent.get_roles_for_action( folder, trans.app.security_agent.permitted_actions.LIBRARY_MANAGE ) ) add_roles = set( trans.app.security_agent.get_roles_for_action( folder, trans.app.security_agent.permitted_actions.LIBRARY_ADD ) ) @@ -173,7 +171,9 @@ modify_folder_role_list = [ ( modify_role.name, trans.security.encode_id( modify_role.id ) ) for modify_role in modify_roles ] manage_folder_role_list = [ ( manage_role.name, trans.security.encode_id( manage_role.id ) ) for manage_role in manage_roles ] add_library_item_role_list = [ ( add_role.name, trans.security.encode_id( add_role.id ) ) for add_role in add_roles ] - return dict( modify_folder_role_list=modify_folder_role_list, manage_folder_role_list=manage_folder_role_list, add_library_item_role_list=add_library_item_role_list ) + return dict( modify_folder_role_list=modify_folder_role_list, + manage_folder_role_list=manage_folder_role_list, + add_library_item_role_list=add_library_item_role_list ) def can_add_item( self, trans, folder ): """ diff -r 658833a7988fe335a5aaaac9709f9e6dfed4a7b2 -r d381771a18cfd6e7d8ecf7563041675a26af0ccd lib/galaxy/webapps/galaxy/api/folders.py --- a/lib/galaxy/webapps/galaxy/api/folders.py +++ b/lib/galaxy/webapps/galaxy/api/folders.py @@ -2,7 +2,6 @@ API operations on library folders. """ from galaxy import util -from galaxy import web from galaxy import exceptions from galaxy.managers import folders from galaxy.web import _future_expose_api as expose_api @@ -82,7 +81,7 @@ parent_folder = self.folder_manager.get( trans, decoded_parent_folder_id ) new_folder = self.folder_manager.create( trans, parent_folder.id, name, description ) return self.folder_manager.get_folder_dict( trans, new_folder ) - + @expose_api def get_permissions( self, trans, encoded_folder_id, **kwd ): """ @@ -141,18 +140,18 @@ *POST /api/folders/{encoded_folder_id}/permissions :param encoded_folder_id: the encoded id of the folder to set the permissions of - :type encoded_folder_id: an encoded id string + :type encoded_folder_id: an encoded id string :param action: (required) describes what action should be performed available actions: set_permissions - :type action: string + :type action: string :param add_ids[]: list of Role.id defining roles that should have add item permission on the folder - :type add_ids[]: string or list + :type add_ids[]: string or list :param manage_ids[]: list of Role.id defining roles that should have manage permission on the folder - :type manage_ids[]: string or list + :type manage_ids[]: string or list :param modify_ids[]: list of Role.id defining roles that should have modify permission on the folder - :type modify_ids[]: string or list + :type modify_ids[]: string or list :rtype: dictionary :returns: dict of current roles for all available permission types. @@ -189,8 +188,8 @@ else: invalid_add_roles_names.append( role_id ) if len( invalid_add_roles_names ) > 0: - log.warning( "The following roles could not be added to the add library item permission: " + str( invalid_add_roles_names ) ) - + log.warning( "The following roles could not be added to the add library item permission: " + str( invalid_add_roles_names ) ) + # MANAGE FOLDER ROLES valid_manage_roles = [] invalid_manage_roles_names = [] @@ -203,8 +202,8 @@ else: invalid_manage_roles_names.append( role_id ) if len( invalid_manage_roles_names ) > 0: - log.warning( "The following roles could not be added to the manage folder permission: " + str( invalid_manage_roles_names ) ) - + log.warning( "The following roles could not be added to the manage folder permission: " + str( invalid_manage_roles_names ) ) + # MODIFY FOLDER ROLES valid_modify_roles = [] invalid_modify_roles_names = [] @@ -217,16 +216,16 @@ else: invalid_modify_roles_names.append( role_id ) if len( invalid_modify_roles_names ) > 0: - log.warning( "The following roles could not be added to the modify folder permission: " + str( invalid_modify_roles_names ) ) + log.warning( "The following roles could not be added to the modify folder permission: " + str( invalid_modify_roles_names ) ) - permissions = { trans.app.security_agent.permitted_actions.LIBRARY_ADD : valid_add_roles } - permissions.update( { trans.app.security_agent.permitted_actions.LIBRARY_MANAGE : valid_manage_roles } ) - permissions.update( { trans.app.security_agent.permitted_actions.LIBRARY_MODIFY : valid_modify_roles } ) + permissions = { trans.app.security_agent.permitted_actions.LIBRARY_ADD: valid_add_roles } + permissions.update( { trans.app.security_agent.permitted_actions.LIBRARY_MANAGE: valid_manage_roles } ) + permissions.update( { trans.app.security_agent.permitted_actions.LIBRARY_MODIFY: valid_modify_roles } ) trans.app.security_agent.set_all_library_permissions( trans, folder, permissions ) else: - raise exceptions.RequestParameterInvalidException( 'The mandatory parameter "action" has an invalid value.' - 'Allowed values are: "set_permissions"' ) + raise exceptions.RequestParameterInvalidException( 'The mandatory parameter "action" has an invalid value.' + 'Allowed values are: "set_permissions"' ) return self.folder_manager.get_current_roles( trans, folder ) @expose_api @@ -256,7 +255,7 @@ return folder_dict @expose_api - def update( self, trans, id, library_id, payload, **kwd ): + def update( self, trans, id, library_id, payload, **kwd ): """ PUT /api/folders/{encoded_folder_id} @@ -269,7 +268,7 @@ Method loads the role from the DB based on the given role name. :param role_name: name of the role to load from the DB - :type role_name: string + :type role_name: string :rtype: Role :returns: the loaded Role object diff -r 658833a7988fe335a5aaaac9709f9e6dfed4a7b2 -r d381771a18cfd6e7d8ecf7563041675a26af0ccd lib/galaxy/webapps/galaxy/api/library_contents.py --- a/lib/galaxy/webapps/galaxy/api/library_contents.py +++ b/lib/galaxy/webapps/galaxy/api/library_contents.py @@ -212,8 +212,8 @@ if create_type == 'file' and from_hda_id: return self._copy_hda_to_library_folder( trans, from_hda_id, library_id, real_folder_id, ldda_message ) - #check for extended metadata, store it and pop it out of the param - #otherwise sanitize_param will have a fit + # check for extended metadata, store it and pop it out of the param + # otherwise sanitize_param will have a fit ex_meta_payload = payload.pop('extended_metadata', None) # Now create the desired content object, either file or folder. @@ -280,8 +280,8 @@ for path, value in self._scan_json_block(a, prefix + "[%d]" % (i)): yield path, value else: - #BUG: Everything is cast to string, which can lead to false positives - #for cross type comparisions, ie "True" == True + # BUG: Everything is cast to string, which can lead to false positives + # for cross type comparisions, ie "True" == True yield prefix, ("%s" % (meta)).encode("utf8", errors='replace') def _copy_hda_to_library_folder( self, trans, from_hda_id, library_id, folder_id, ldda_message='' ): @@ -293,15 +293,15 @@ in its payload. """ log.debug( '_copy_hda_to_library_folder: %s' % ( str(( from_hda_id, library_id, folder_id, ldda_message )) ) ) - #PRECONDITION: folder_id has already been altered to remove the folder prefix ('F') - #TODO: allow name and other, editable ldda attrs? + # PRECONDITION: folder_id has already been altered to remove the folder prefix ('F') + # TODO: allow name and other, editable ldda attrs? if ldda_message: ldda_message = util.sanitize_html.sanitize_html( ldda_message, 'utf-8' ) rval = {} try: # check permissions on (all three?) resources: hda, library, folder - #TODO: do we really need the library?? + # TODO: do we really need the library?? from_hda_id = self.decode_id( from_hda_id ) hda = self.hda_manager.get_owned( trans, from_hda_id, trans.user ) hda = self.hda_manager.error_if_uploading( trans, hda ) @@ -319,7 +319,7 @@ rval = trans.security.encode_dict_ids( ldda_dict ) except Exception, exc: - #TODO: grrr... + # TODO: grrr... if 'not accessible to the current user' in str( exc ): trans.response.status = 403 return { 'error': str( exc ) } @@ -411,7 +411,7 @@ trans.sa_session.add( ld ) trans.sa_session.flush() - #TODO: had to change this up a bit from Dataset.user_can_purge + # TODO: had to change this up a bit from Dataset.user_can_purge dataset = ld.library_dataset_dataset_association.dataset no_history_assoc = len( dataset.history_associations ) == len( dataset.purged_history_associations ) no_library_assoc = dataset.library_associations == [ ld.library_dataset_dataset_association ] https://bitbucket.org/galaxy/galaxy-central/commits/132692a47ba3/ Changeset: 132692a47ba3 User: dannon Date: 2015-02-25 13:26:11+00:00 Summary: Avoid referring to galaxy.exceptions as exceptions, overriding stdlib exceptions. Be explicit in using them. Affected #: 1 file diff -r d381771a18cfd6e7d8ecf7563041675a26af0ccd -r 132692a47ba3bd421fb59e8338fed627ea240e98 lib/galaxy/managers/folders.py --- a/lib/galaxy/managers/folders.py +++ b/lib/galaxy/managers/folders.py @@ -2,8 +2,7 @@ Manager and Serializer for Library Folders. """ -from galaxy import exceptions - +import galaxy.exceptions import logging log = logging.getLogger( __name__ ) @@ -30,12 +29,12 @@ """ try: folder = trans.sa_session.query( trans.app.model.LibraryFolder ).filter( trans.app.model.LibraryFolder.table.c.id == decoded_folder_id ).one() - except MultipleResultsFound: - raise exceptions.InconsistentDatabase( 'Multiple folders found with the same id.' ) - except NoResultFound: - raise exceptions.RequestParameterInvalidException( 'No folder found with the id provided.' ) + except galaxy.exceptions.MultipleResultsFound: + raise galaxy.exceptions.InconsistentDatabase( 'Multiple folders found with the same id.' ) + except galaxy.exceptions.NoResultFound: + raise galaxy.exceptions.RequestParameterInvalidException( 'No folder found with the id provided.' ) except Exception, e: - raise exceptions.InternalServerError( 'Error loading from the database.' + str( e ) ) + raise galaxy.exceptions.InternalServerError( 'Error loading from the database.' + str( e ) ) folder = self.secure( trans, folder, check_manageable, check_accessible ) return folder @@ -72,10 +71,10 @@ :raises: AuthenticationRequired, InsufficientPermissionsException """ if not trans.user: - raise exceptions.AuthenticationRequired( "Must be logged in to manage Galaxy items.", type='error' ) + raise galaxy.exceptions.AuthenticationRequired( "Must be logged in to manage Galaxy items.", type='error' ) current_user_roles = trans.get_current_user_roles() if not trans.app.security_agent.can_manage_library_item( current_user_roles, folder ): - raise exceptions.InsufficientPermissionsException( "You don't have permissions to manage this folder.", type='error' ) + raise galaxy.exceptions.InsufficientPermissionsException( "You don't have permissions to manage this folder.", type='error' ) else: return folder @@ -123,7 +122,7 @@ parent_folder = self.get( trans, parent_folder_id ) current_user_roles = trans.get_current_user_roles() if not ( trans.user_is_admin or trans.app.security_agent.can_add_library_item( current_user_roles, parent_folder ) ): - raise exceptions.InsufficientPermissionsException( 'You do not have proper permission to create folders under given folder.' ) + raise galaxy.exceptions.InsufficientPermissionsException( 'You do not have proper permission to create folders under given folder.' ) new_folder = trans.app.model.LibraryFolder( name=new_folder_name, description=new_folder_description ) # We are associating the last used genome build with folders, so we will always # initialize a new folder with the first dbkey in genome builds list which is currently @@ -195,7 +194,7 @@ if ( ( len( encoded_folder_id ) % 16 == 1 ) and encoded_folder_id.startswith( 'F' ) ): cut_id = encoded_folder_id[ 1: ] else: - raise exceptions.MalformedId( 'Malformed folder id ( %s ) specified, unable to decode.' % str( encoded_folder_id ) ) + raise galaxy.exceptions.MalformedId( 'Malformed folder id ( %s ) specified, unable to decode.' % str( encoded_folder_id ) ) return cut_id def decode_folder_id( self, trans, encoded_folder_id ): @@ -205,7 +204,7 @@ try: decoded_id = trans.security.decode_id( encoded_folder_id ) except ValueError: - raise exceptions.MalformedId( "Malformed folder id ( %s ) specified, unable to decode" % ( str( encoded_folder_id ) ) ) + raise galaxy.exceptions.MalformedId( "Malformed folder id ( %s ) specified, unable to decode" % ( str( encoded_folder_id ) ) ) return decoded_id def cut_and_decode( self, trans, encoded_folder_id ): https://bitbucket.org/galaxy/galaxy-central/commits/fe7ae6f22b95/ Changeset: fe7ae6f22b95 User: dannon Date: 2015-02-25 13:59:34+00:00 Summary: In web.security.encode_id, raise MalformedId exception when attempting to encode None. This will likely throw errors in other parts of the codebase, but those will need to be fixed instead of silently using what is actually a broken identifier. Affected #: 1 file diff -r 132692a47ba3bd421fb59e8338fed627ea240e98 -r fe7ae6f22b95756515da5ada8442b270bfb387e0 lib/galaxy/web/security/__init__.py --- a/lib/galaxy/web/security/__init__.py +++ b/lib/galaxy/web/security/__init__.py @@ -6,6 +6,8 @@ import pkg_resources pkg_resources.require( "pycrypto" ) +import galaxy.exceptions + from Crypto.Cipher import Blowfish from Crypto.Util.randpool import RandomPool from Crypto.Util import number @@ -45,6 +47,8 @@ self.id_ciphers_for_kind = _cipher_cache( per_kind_id_secret_base ) def encode_id( self, obj_id, kind=None ): + if obj_id is None: + raise galaxy.exceptions.MalformedId("Attempted to encode None id") id_cipher = self.__id_cipher( kind ) # Convert to string s = str( obj_id ) https://bitbucket.org/galaxy/galaxy-central/commits/0565f566e550/ Changeset: 0565f566e550 User: jmchilton Date: 2015-02-25 14:10:17+00:00 Summary: Expand out meta parameters when validating for tool.to_json(). Bring this code closer inline with what handle_input() does and should solve the problem with column validation of parameters that are being 'batched'. See https://trello.com/c/8NweTvLh and comment on https://github.com/galaxyproject/galaxy/commit/d57088409c7b23e55ead4c9f2fad…. Affected #: 1 file diff -r fe7ae6f22b95756515da5ada8442b270bfb387e0 -r 0565f566e5509d16863a7d8b76908cc7079decf4 lib/galaxy/tools/__init__.py --- a/lib/galaxy/tools/__init__.py +++ b/lib/galaxy/tools/__init__.py @@ -2299,6 +2299,8 @@ # create parameter object params = galaxy.util.Params( kwd, sanitize = False ) + expanded_incomings, _ = expand_meta_parameters( trans, self, params.__dict__ ) + params.__dict__ = expanded_incomings[ 0 ] # convert value to jsonifiable value def jsonify(v): https://bitbucket.org/galaxy/galaxy-central/commits/771789ce8753/ Changeset: 771789ce8753 User: dannon Date: 2015-02-25 14:14:59+00:00 Summary: Fix returning encoded None library folder parent ids in library_contents API; thanks to nsoranzo Affected #: 1 file diff -r 0565f566e5509d16863a7d8b76908cc7079decf4 -r 771789ce8753017b8aef602419d502ce597214a1 lib/galaxy/webapps/galaxy/api/library_contents.py --- a/lib/galaxy/webapps/galaxy/api/library_contents.py +++ b/lib/galaxy/webapps/galaxy/api/library_contents.py @@ -128,7 +128,8 @@ content = self.get_library_folder( trans, content_id, check_ownership=False, check_accessible=True ) rval = content.to_dict( view='element', value_mapper={ 'id': trans.security.encode_id } ) rval[ 'id' ] = 'F' + str( rval[ 'id' ] ) - rval[ 'parent_id' ] = 'F' + str( trans.security.encode_id( rval[ 'parent_id' ] ) ) + if rval[ 'parent_id' ] is not None: # This can happen for root folders. + rval[ 'parent_id' ] = 'F' + str( trans.security.encode_id( rval[ 'parent_id' ] ) ) rval[ 'parent_library_id' ] = trans.security.encode_id( rval[ 'parent_library_id' ] ) else: content = self.get_library_dataset( trans, content_id, check_ownership=False, check_accessible=True ) https://bitbucket.org/galaxy/galaxy-central/commits/1876ae3314d3/ Changeset: 1876ae3314d3 User: jmchilton Date: 2015-02-25 14:12:44+00:00 Summary: Handle column references on data_collection parameters... ... in column parameter's need_late_validation. There may be other places that need to be updated to get this working fully. Affected #: 1 file diff -r 771789ce8753017b8aef602419d502ce597214a1 -r 1876ae3314d3b2da4ce91ee457c5fbafc2649d03 lib/galaxy/tools/parameters/basic.py --- a/lib/galaxy/tools/parameters/basic.py +++ b/lib/galaxy/tools/parameters/basic.py @@ -1332,7 +1332,12 @@ if self.data_ref not in context: return False # Get the selected dataset if selected - datasets = util.listify( context[ self.data_ref ] ) + referent = context[ self.data_ref ] + if getattr( referent, 'history_content_type', None ) == "dataset_collection": + # TODO: also check datasets have been populated. + referent = referent.collection.dataset_instances + + datasets = util.listify( referent ) for dataset in datasets: if dataset: # Check if the dataset does not have the expected metadata for columns https://bitbucket.org/galaxy/galaxy-central/commits/9653a478480a/ Changeset: 9653a478480a User: dannon Date: 2015-02-25 14:22:56+00:00 Summary: Fix undefined variable 'flus_needed' in toolshed admin controller. Affected #: 1 file diff -r 1876ae3314d3b2da4ce91ee457c5fbafc2649d03 -r 9653a478480aebcfd3221e6249787a5c37fc814d lib/galaxy/webapps/tool_shed/controllers/admin.py --- a/lib/galaxy/webapps/tool_shed/controllers/admin.py +++ b/lib/galaxy/webapps/tool_shed/controllers/admin.py @@ -250,7 +250,7 @@ flush_needed = True if original_category_description != new_description: category.description = new_description - if not flus_needed: + if not flush_needed: flush_needed = True if flush_needed: trans.sa_session.add( category ) https://bitbucket.org/galaxy/galaxy-central/commits/51638aef252a/ Changeset: 51638aef252a User: dannon Date: 2015-02-25 14:23:50+00:00 Summary: Fix another undefined variable 'tran' in toolshed admin controller. Affected #: 1 file diff -r 9653a478480aebcfd3221e6249787a5c37fc814d -r 51638aef252a51a3296ac7a1300f2bd12123a870 lib/galaxy/webapps/tool_shed/controllers/admin.py --- a/lib/galaxy/webapps/tool_shed/controllers/admin.py +++ b/lib/galaxy/webapps/tool_shed/controllers/admin.py @@ -88,7 +88,7 @@ changeset_revision_str = 'changeset_revision_' if k.startswith( changeset_revision_str ): repository_id = trans.security.encode_id( int( k.lstrip( changeset_revision_str ) ) ) - repository = suc.get_repository_in_tool_shed( tran.apps, repository_id ) + repository = suc.get_repository_in_tool_shed( trans.app, repository_id ) if repository.tip( trans.app ) != v: return trans.response.send_redirect( web.url_for( controller='repository', action='browse_repositories', https://bitbucket.org/galaxy/galaxy-central/commits/bc6b7f12f0aa/ Changeset: bc6b7f12f0aa User: dannon Date: 2015-02-25 14:29:23+00:00 Summary: Cleanup and pep8 in galaxy admin controller prior to new dev. Affected #: 1 file diff -r 51638aef252a51a3296ac7a1300f2bd12123a870 -r bc6b7f12f0aa2afcc36e81bfb43c60a28ad79b28 lib/galaxy/webapps/galaxy/controllers/admin.py --- a/lib/galaxy/webapps/galaxy/controllers/admin.py +++ b/lib/galaxy/webapps/galaxy/controllers/admin.py @@ -25,23 +25,17 @@ class UserListGrid( grids.Grid ): - class EmailColumn( grids.TextColumn ): - def get_value( self, trans, grid, user ): return user.email - class UserNameColumn( grids.TextColumn ): - def get_value( self, trans, grid, user ): if user.username: return user.username return 'not set' - class StatusColumn( grids.GridColumn ): - def get_value( self, trans, grid, user ): if user.purged: return "purged" @@ -49,47 +43,35 @@ return "deleted" return "" - class GroupsColumn( grids.GridColumn ): - def get_value( self, trans, grid, user ): if user.groups: return len( user.groups ) return 0 - class RolesColumn( grids.GridColumn ): - def get_value( self, trans, grid, user ): if user.roles: return len( user.roles ) return 0 - class ExternalColumn( grids.GridColumn ): - def get_value( self, trans, grid, user ): if user.external: return 'yes' return 'no' - class LastLoginColumn( grids.GridColumn ): - def get_value( self, trans, grid, user ): if user.galaxy_sessions: return self.format( user.galaxy_sessions[ 0 ].update_time ) return 'never' - class TimeCreatedColumn( grids.GridColumn ): - def get_value( self, trans, grid, user ): return user.create_time.strftime('%x') - class ActivatedColumn( grids.GridColumn ): - def get_value( self, trans, grid, user ): if user.active: return 'Y' @@ -99,7 +81,7 @@ # Grid definition title = "Users" model_class = model.User - template='/admin/user/grid.mako' + template = '/admin/user/grid.mako' default_sort_key = "email" columns = [ EmailColumn( "Email", @@ -158,46 +140,33 @@ class RoleListGrid( grids.Grid ): - class NameColumn( grids.TextColumn ): - def get_value( self, trans, grid, role ): return role.name - class DescriptionColumn( grids.TextColumn ): - def get_value( self, trans, grid, role ): if role.description: return role.description return '' - class TypeColumn( grids.TextColumn ): - def get_value( self, trans, grid, role ): return role.type - class StatusColumn( grids.GridColumn ): - def get_value( self, trans, grid, role ): if role.deleted: return "deleted" return "" - class GroupsColumn( grids.GridColumn ): - def get_value( self, trans, grid, role ): if role.groups: return len( role.groups ) return 0 - class UsersColumn( grids.GridColumn ): - - def get_value( self, trans, grid, role ): if role.users: return len( role.users ) @@ -206,7 +175,7 @@ # Grid definition title = "Roles" model_class = model.Role - template='/admin/dataset_security/role/grid.mako' + template = '/admin/dataset_security/role/grid.mako' default_sort_key = "name" columns = [ NameColumn( "Name", @@ -270,31 +239,23 @@ class GroupListGrid( grids.Grid ): - class NameColumn( grids.TextColumn ): - def get_value( self, trans, grid, group ): return group.name - class StatusColumn( grids.GridColumn ): - def get_value( self, trans, grid, group ): if group.deleted: return "deleted" return "" - class RolesColumn( grids.GridColumn ): - def get_value( self, trans, grid, group ): if group.roles: return len( group.roles ) return 0 - class UsersColumn( grids.GridColumn ): - def get_value( self, trans, grid, group ): if group.members: return len( group.members ) @@ -303,7 +264,7 @@ # Grid definition title = "Groups" model_class = model.Group - template='/admin/dataset_security/group/grid.mako' + template = '/admin/dataset_security/group/grid.mako' default_sort_key = "name" columns = [ NameColumn( "Name", @@ -351,31 +312,24 @@ preserve_state = False use_paging = True + class QuotaListGrid( grids.Grid ): - class NameColumn( grids.TextColumn ): - def get_value( self, trans, grid, quota ): return quota.name - class DescriptionColumn( grids.TextColumn ): - def get_value( self, trans, grid, quota ): if quota.description: return quota.description return '' - class AmountColumn( grids.TextColumn ): - def get_value( self, trans, grid, quota ): return quota.operation + quota.display_amount - class StatusColumn( grids.GridColumn ): - def get_value( self, trans, grid, quota ): if quota.deleted: return "deleted" @@ -383,17 +337,13 @@ return "default for %s users" % quota.default[0].type return "" - class UsersColumn( grids.GridColumn ): - def get_value( self, trans, grid, quota ): if quota.users: return len( quota.users ) return 0 - class GroupsColumn( grids.GridColumn ): - def get_value( self, trans, grid, quota ): if quota.groups: return len( quota.groups ) @@ -402,7 +352,7 @@ # Grid definition title = "Quotas" model_class = model.Quota - template='/admin/quota/grid.mako' + template = '/admin/quota/grid.mako' default_sort_key = "name" columns = [ NameColumn( "Name", @@ -417,10 +367,10 @@ attach_popup=False, filterable="advanced" ), AmountColumn( "Amount", - key='amount', - model_class=model.Quota, - attach_popup=False, - filterable="advanced" ), + key='amount', + model_class=model.Quota, + attach_popup=False, + filterable="advanced" ), UsersColumn( "Users", attach_popup=False ), GroupsColumn( "Groups", attach_popup=False ), StatusColumn( "Status", attach_popup=False ), @@ -483,9 +433,7 @@ class ToolVersionListGrid( grids.Grid ): - class ToolIdColumn( grids.TextColumn ): - def get_value( self, trans, grid, tool_version ): toolbox = trans.app.toolbox if toolbox.has_tool( tool_version.tool_id, exact=True ): @@ -494,9 +442,7 @@ return '<div class="count-box state-color-ok">%s%s</a></div>' % ( link_str, tool_version.tool_id ) return tool_version.tool_id - class ToolVersionsColumn( grids.TextColumn ): - def get_value( self, trans, grid, tool_version ): tool_ids_str = '' toolbox = trans.app.toolbox @@ -508,10 +454,11 @@ else: tool_ids_str += '%s ' % tool_id return tool_ids_str + # Grid definition title = "Tool versions" model_class = install_model.ToolVersion - template='/admin/tool_version/grid.mako' + template = '/admin/tool_version/grid.mako' default_sort_key = "tool_id" columns = [ ToolIdColumn( "Tool id", @@ -531,7 +478,7 @@ num_rows_per_page = 50 preserve_state = False use_paging = True - + def build_initial_query( self, trans, **kwd ): return trans.install_model.context.query( self.model_class ) @@ -899,10 +846,11 @@ kwargs={'display_application_ids': kwd.get( 'id' )} ) reloaded, failed = trans.app.datatypes_registry.reload_display_applications( kwd.get( 'id' ) ) if not reloaded and failed: - return trans.show_error_message( 'Unable to reload any of the %i requested display applications ("%s").' % ( len( failed ), '", "'.join( failed ) ) ) + return trans.show_error_message( 'Unable to reload any of the %i requested display applications ("%s").' + % ( len( failed ), '", "'.join( failed ) ) ) if failed: return trans.show_warn_message( 'Reloaded %i display applications ("%s"), but failed to reload %i display applications ("%s").' - % ( len( reloaded ), '", "'.join( reloaded ), len( failed ), '", "'.join( failed ) ) ) + % ( len( reloaded ), '", "'.join( reloaded ), len( failed ), '", "'.join( failed ) ) ) if not reloaded: return trans.show_warn_message( 'You need to request at least one display application to reload.' ) return trans.show_ok_message( 'Reloaded %i requested display applications ("%s").' % ( len( reloaded ), '", "'.join( reloaded ) ) ) https://bitbucket.org/galaxy/galaxy-central/commits/390042b7f036/ Changeset: 390042b7f036 User: dannon Date: 2015-02-25 14:34:42+00:00 Summary: Update queries/filters in galaxy admin controller to use sqlalchemy constructs for True, False, testing for None. Affected #: 1 file diff -r bc6b7f12f0aa2afcc36e81bfb43c60a28ad79b28 -r 390042b7f0365ff1650e5a56ac4ee869952d5ad4 lib/galaxy/webapps/galaxy/controllers/admin.py --- a/lib/galaxy/webapps/galaxy/controllers/admin.py +++ b/lib/galaxy/webapps/galaxy/controllers/admin.py @@ -2,6 +2,8 @@ import logging import os +from sqlalchemy.sql import expression + import galaxy.queue_worker import galaxy.util from galaxy import model @@ -540,14 +542,14 @@ new_in_users = [] new_in_groups = [] for user in trans.sa_session.query( trans.app.model.User ) \ - .filter( trans.app.model.User.table.c.deleted==False ) \ + .filter( trans.app.model.User.table.c.deleted == expression.false() ) \ .order_by( trans.app.model.User.table.c.email ): if user.id in in_users: new_in_users.append( ( user.id, user.email ) ) else: params.out_users.append( ( user.id, user.email ) ) for group in trans.sa_session.query( trans.app.model.Group ) \ - .filter( trans.app.model.Group.table.c.deleted==False ) \ + .filter( trans.app.model.Group.table.c.deleted == expression.false() ) \ .order_by( trans.app.model.Group.table.c.name ): if group.id in in_groups: new_in_groups.append( ( group.id, group.name ) ) @@ -592,14 +594,14 @@ in_groups = [] out_groups = [] for user in trans.sa_session.query( trans.app.model.User ) \ - .filter( trans.app.model.User.table.c.deleted==False ) \ + .filter( trans.app.model.User.table.c.deleted == expression.false() ) \ .order_by( trans.app.model.User.table.c.email ): if user in [ x.user for x in quota.users ]: in_users.append( ( user.id, user.email ) ) else: out_users.append( ( user.id, user.email ) ) for group in trans.sa_session.query( trans.app.model.Group ) \ - .filter( trans.app.model.Group.table.c.deleted==False ) \ + .filter( trans.app.model.Group.table.c.deleted == expression.false()) \ .order_by( trans.app.model.Group.table.c.name ): if group in [ x.group for x in quota.groups ]: in_groups.append( ( group.id, group.name ) ) @@ -722,7 +724,7 @@ webapp=params.webapp, message=sanitize_text( str( e ) ), status='error' ) ) - if do_op == True or ( do_op != False and params.get( do_op, False ) ): + if do_op is True or ( do_op is not False and params.get( do_op, False ) ): try: message = op_method( quota, params ) return None, trans.response.send_redirect( web.url_for( controller='admin', https://bitbucket.org/galaxy/galaxy-central/commits/592c5347b33b/ Changeset: 592c5347b33b User: peterjc Date: 2015-02-25 19:37:56+00:00 Summary: Suppress filenames in SAM merge using egrep Based on fix suggested by Roberto Alonso (at CIPF) on the mailing list, see: https://lists.galaxyproject.org/pipermail/galaxy-dev/2015-February/021589.h… Affected #: 1 file diff -r 390042b7f0365ff1650e5a56ac4ee869952d5ad4 -r 592c5347b33b4fe52244ba5077a20dd6b757278c lib/galaxy/datatypes/tabular.py --- a/lib/galaxy/datatypes/tabular.py +++ b/lib/galaxy/datatypes/tabular.py @@ -480,7 +480,7 @@ if result != 0: raise Exception('Result %s from %s' % (result, cmd)) if len(split_files) > 1: - cmd = 'egrep -v "^@" %s >> %s' % ( ' '.join(split_files[1:]), output_file ) + cmd = 'egrep -v -h "^@" %s >> %s' % ( ' '.join(split_files[1:]), output_file ) result = os.system(cmd) if result != 0: raise Exception('Result %s from %s' % (result, cmd)) https://bitbucket.org/galaxy/galaxy-central/commits/86f24f45ccdf/ Changeset: 86f24f45ccdf User: martenson Date: 2015-02-25 14:56:37+00:00 Summary: fix missing sqlalchemy imports and add documentation to folder manager Affected #: 1 file diff -r 592c5347b33b4fe52244ba5077a20dd6b757278c -r 86f24f45ccdfdc8769dff0272cabb57c6ce52171 lib/galaxy/managers/folders.py --- a/lib/galaxy/managers/folders.py +++ b/lib/galaxy/managers/folders.py @@ -3,6 +3,8 @@ """ import galaxy.exceptions +from sqlalchemy.orm.exc import MultipleResultsFound +from sqlalchemy.orm.exc import NoResultFound import logging log = logging.getLogger( __name__ ) @@ -19,19 +21,21 @@ :param decoded_folder_id: decoded folder id :type decoded_folder_id: int - :param check_manageable: flag whether the check that user can manage item - :type check_manageable: bool + :param check_manageable: flag whether the check that user can manage item + :type check_manageable: bool :param check_accessible: flag whether to check that user can access item :type check_accessible: bool :returns: the requested folder :rtype: LibraryFolder + + :raises: InconsistentDatabase, RequestParameterInvalidException, InternalServerError """ try: folder = trans.sa_session.query( trans.app.model.LibraryFolder ).filter( trans.app.model.LibraryFolder.table.c.id == decoded_folder_id ).one() - except galaxy.exceptions.MultipleResultsFound: + except MultipleResultsFound: raise galaxy.exceptions.InconsistentDatabase( 'Multiple folders found with the same id.' ) - except galaxy.exceptions.NoResultFound: + except NoResultFound: raise galaxy.exceptions.RequestParameterInvalidException( 'No folder found with the id provided.' ) except Exception, e: raise galaxy.exceptions.InternalServerError( 'Error loading from the database.' + str( e ) ) @@ -139,6 +143,14 @@ """ Mark given folder deleted/undeleted based on the flag. + :param folder: the model object + :type folder: LibraryFolder + :param undelete: flag whether to delete (when False) or undelete + :type undelete: Bool + + :returns: the folder + :rtype: LibraryFolder + :raises: ItemAccessibilityException """ if not trans.user_is_admin(): @@ -190,6 +202,14 @@ def cut_the_prefix( self, encoded_folder_id ): """ Remove the prefix from the encoded folder id. + + :param encoded_folder_id: encoded id of the Folder object with 'F' prepended + :type encoded_folder_id: string + + :returns: encoded Folder id without the 'F' prefix + :rtype: string + + :raises: MalformedId """ if ( ( len( encoded_folder_id ) % 16 == 1 ) and encoded_folder_id.startswith( 'F' ) ): cut_id = encoded_folder_id[ 1: ] @@ -200,6 +220,14 @@ def decode_folder_id( self, trans, encoded_folder_id ): """ Decode the folder id given that it has already lost the prefixed 'F'. + + :param encoded_folder_id: encoded id of the Folder object + :type encoded_folder_id: string + + :returns: decoded Folder id + :rtype: int + + :raises: MalformedId """ try: decoded_id = trans.security.decode_id( encoded_folder_id ) @@ -210,5 +238,11 @@ def cut_and_decode( self, trans, encoded_folder_id ): """ Cuts the folder prefix (the prepended 'F') and returns the decoded id. + + :param encoded_folder_id: encoded id of the Folder object + :type encoded_folder_id: string + + :returns: decoded Folder id + :rtype: int """ return self.decode_folder_id( trans, self.cut_the_prefix( encoded_folder_id ) ) https://bitbucket.org/galaxy/galaxy-central/commits/e6941d81339e/ Changeset: e6941d81339e User: martenson Date: 2015-02-25 15:52:35+00:00 Summary: create library root folder with '/' as default name fix for showing actual name of root folder when calling GET /api/libraries/{library_id}/contents Affected #: 2 files diff -r 86f24f45ccdfdc8769dff0272cabb57c6ce52171 -r e6941d81339e1bf706fa8f137e14b0cbb701a77a lib/galaxy/managers/libraries.py --- a/lib/galaxy/managers/libraries.py +++ b/lib/galaxy/managers/libraries.py @@ -54,7 +54,7 @@ raise exceptions.ItemAccessibilityException( 'Only administrators can create libraries.' ) else: library = trans.app.model.Library( name=name, description=description, synopsis=synopsis ) - root_folder = trans.app.model.LibraryFolder( name=name, description='' ) + root_folder = trans.app.model.LibraryFolder( name='/', description='' ) library.root_folder = root_folder trans.sa_session.add_all( ( library, root_folder ) ) trans.sa_session.flush() diff -r 86f24f45ccdfdc8769dff0272cabb57c6ce52171 -r e6941d81339e1bf706fa8f137e14b0cbb701a77a lib/galaxy/webapps/galaxy/api/library_contents.py --- a/lib/galaxy/webapps/galaxy/api/library_contents.py +++ b/lib/galaxy/webapps/galaxy/api/library_contents.py @@ -85,12 +85,13 @@ raise exceptions.InternalServerError( 'Error loading from the database.' + str(e)) if not ( trans.user_is_admin() or trans.app.security_agent.can_access_library( current_user_roles, library ) ): raise exceptions.RequestParameterInvalidException( 'No library found with the id provided.' ) - encoded_id = 'F' + trans.security.encode_id( library.root_folder.id ) + root_encoded_id = 'F' + trans.security.encode_id( library.root_folder.id ) + root_name = library.root_folder.name # appending root folder - rval.append( dict( id=encoded_id, + rval.append( dict( id=root_encoded_id, type='folder', - name='/', - url=url_for( 'library_content', library_id=library_id, id=encoded_id ) ) ) + name=root_name, + url=url_for( 'library_content', library_id=library_id, id=root_encoded_id ) ) ) library.root_folder.api_path = '' # appending all other items in the library recursively for content in traverse( library.root_folder ): https://bitbucket.org/galaxy/galaxy-central/commits/6ff3bb74e727/ Changeset: 6ff3bb74e727 User: dannon Date: 2015-02-25 16:48:01+00:00 Summary: Initial commit of recalculating user disk usage from the admin interface (user grid). Affected #: 1 file diff -r e6941d81339e1bf706fa8f137e14b0cbb701a77a -r 6ff3bb74e727ab565f486cae42ca579159e48465 lib/galaxy/webapps/galaxy/controllers/admin.py --- a/lib/galaxy/webapps/galaxy/controllers/admin.py +++ b/lib/galaxy/webapps/galaxy/controllers/admin.py @@ -1,17 +1,16 @@ import imp import logging import os - from sqlalchemy.sql import expression import galaxy.queue_worker import galaxy.util from galaxy import model -from galaxy.model import tool_shed_install as install_model from galaxy import web from galaxy.actions.admin import AdminActions from galaxy.exceptions import MessageException -from galaxy.util import sanitize_text +from galaxy.model import tool_shed_install as install_model +from galaxy.util import nice_size, sanitize_text from galaxy.util.odict import odict from galaxy.web import url_for from galaxy.web.base.controller import BaseUIController, UsesQuotaMixin @@ -124,7 +123,11 @@ condition=( lambda item: not item.deleted ), allow_multiple=True, allow_popup=False, - url_args=dict( webapp="galaxy", action="reset_user_password" ) ) + url_args=dict( webapp="galaxy", action="reset_user_password" ) ), + grids.GridOperation( "Recalculate Disk Usage", + condition=( lambda item: not item.deleted ), + allow_multiple=False, + url_args=dict( webapp="galaxy", action="recalculate_user_disk_usage" ) ) ] standard_filters = [ grids.GridColumnFilter( "Active", args=dict( deleted=False ) ), @@ -856,3 +859,69 @@ if not reloaded: return trans.show_warn_message( 'You need to request at least one display application to reload.' ) return trans.show_ok_message( 'Reloaded %i requested display applications ("%s").' % ( len( reloaded ), '", "'.join( reloaded ) ) ) + + @web.expose + @web.require_admin + def recalculate_user_disk_usage( self, trans, **kwd ): + user_id = kwd.get( 'id', None ) + user = trans.sa_session.query( trans.model.User ).get( trans.security.decode_id( user_id ) ) + if not user: + return trans.show_error_message( "User not found for id (%s)" % sanitize_text( str( user_id ) ) ) + engine = None + if trans.app.config.database_connection: + engine = trans.app.config.database_connection.split(':')[0] + if engine not in ( 'postgres', 'postgresql' ): + done = False + while not done: + current = user.get_disk_usage() + new = user.calculate_disk_usage() + trans.sa_session.refresh( user ) + # make sure usage didn't change while calculating, set done + if user.get_disk_usage() == current: + done = True + if new not in (current, None): + user.set_disk_usage( new ) + trans.sa_session.add( user ) + trans.sa_session.flush() + else: + # We can use the lightning fast pgcalc! + current = user.get_disk_usage() + new = pgcalc( self.sa_session, user.id ) + # yes, still a small race condition between here and the flush + if new in ( current, None ): + message = 'Usage is unchanged at %s.' % nice_size( current ) + else: + message = 'Usage has changed by %s to %s.' % ( nice_size( new - current ), nice_size( current ) ) + return trans.response.send_redirect( web.url_for( controller='admin', + action='users', + message=sanitize_text( message ), + status='info' ) ) + + +def pgcalc( sa_session, id, dryrun=False ): + """ + Utility method for quickly recalculating user disk usage in postgres. + TODO: Move this elsewhere and update scripts/set_user_disk_usage.py + """ + sql = """ + UPDATE galaxy_user + SET disk_usage = (SELECT COALESCE(SUM(total_size), 0) + FROM ( SELECT DISTINCT ON (d.id) d.total_size, d.id + FROM history_dataset_association hda + JOIN history h ON h.id = hda.history_id + JOIN dataset d ON hda.dataset_id = d.id + WHERE h.user_id = :id + AND h.purged = false + AND hda.purged = false + AND d.purged = false + AND d.id NOT IN (SELECT dataset_id + FROM library_dataset_dataset_association) + ) sizes) + WHERE id = :id + RETURNING disk_usage; + """ + r = sa_session.execute(sql, {'id': id}) + new = r.fetchone()[0] + if dryrun: + sa_session.rollback() + return new https://bitbucket.org/galaxy/galaxy-central/commits/970982d0841d/ Changeset: 970982d0841d User: dan Date: 2015-02-25 17:07:39+00:00 Summary: Fix for ToolShed not having config.shed_tool_data_path Affected #: 1 file diff -r 6ff3bb74e727ab565f486cae42ca579159e48465 -r 970982d0841d3b1717ae0263f303cfed6305c46f lib/galaxy/webapps/tool_shed/config.py --- a/lib/galaxy/webapps/tool_shed/config.py +++ b/lib/galaxy/webapps/tool_shed/config.py @@ -137,6 +137,10 @@ self.citation_cache_lock_dir = resolve_path( kwargs.get( "citation_cache_lock_dir", "database/tool_shed_citations/locks" ), self.root ) @property + def shed_tool_data_path( self ): + return self.tool_data_path + + @property def sentry_dsn_public( self ): """ Sentry URL with private key removed for use in client side scripts, https://bitbucket.org/galaxy/galaxy-central/commits/7ef30f55b759/ Changeset: 7ef30f55b759 User: dannon Date: 2015-02-25 17:52:47+00:00 Summary: Refactoring of set_user_disk_usage and admin controller disk usage to use the same pgcalc Affected #: 3 files diff -r 970982d0841d3b1717ae0263f303cfed6305c46f -r 7ef30f55b759ef1bbd53cd1113887f3637fc4e96 lib/galaxy/model/util.py --- /dev/null +++ b/lib/galaxy/model/util.py @@ -0,0 +1,36 @@ +""" +Utility helpers related to the model +""" + + +def pgcalc( sa_session, id, dryrun=False ): + """ + Utility method for quickly recalculating user disk usage in postgres. + + TODO: Move this elsewhere and update scripts/set_user_disk_usage.py + + TODO: Check against the recently updated versions of sqlalchemy if this + 'special' postgresql version is even necessary. + """ + sql = """ + UPDATE galaxy_user + SET disk_usage = (SELECT COALESCE(SUM(total_size), 0) + FROM ( SELECT DISTINCT ON (d.id) d.total_size, d.id + FROM history_dataset_association hda + JOIN history h ON h.id = hda.history_id + JOIN dataset d ON hda.dataset_id = d.id + WHERE h.user_id = :id + AND h.purged = false + AND hda.purged = false + AND d.purged = false + AND d.id NOT IN (SELECT dataset_id + FROM library_dataset_dataset_association) + ) sizes) + WHERE id = :id + RETURNING disk_usage; + """ + r = sa_session.execute(sql, {'id': id}) + new = r.fetchone()[0] + if dryrun: + sa_session.rollback() + return new diff -r 970982d0841d3b1717ae0263f303cfed6305c46f -r 7ef30f55b759ef1bbd53cd1113887f3637fc4e96 lib/galaxy/webapps/galaxy/controllers/admin.py --- a/lib/galaxy/webapps/galaxy/controllers/admin.py +++ b/lib/galaxy/webapps/galaxy/controllers/admin.py @@ -10,6 +10,7 @@ from galaxy.actions.admin import AdminActions from galaxy.exceptions import MessageException from galaxy.model import tool_shed_install as install_model +from galaxy.model.util import pgcalc from galaxy.util import nice_size, sanitize_text from galaxy.util.odict import odict from galaxy.web import url_for @@ -896,32 +897,3 @@ action='users', message=sanitize_text( message ), status='info' ) ) - - -def pgcalc( sa_session, id, dryrun=False ): - """ - Utility method for quickly recalculating user disk usage in postgres. - TODO: Move this elsewhere and update scripts/set_user_disk_usage.py - """ - sql = """ - UPDATE galaxy_user - SET disk_usage = (SELECT COALESCE(SUM(total_size), 0) - FROM ( SELECT DISTINCT ON (d.id) d.total_size, d.id - FROM history_dataset_association hda - JOIN history h ON h.id = hda.history_id - JOIN dataset d ON hda.dataset_id = d.id - WHERE h.user_id = :id - AND h.purged = false - AND hda.purged = false - AND d.purged = false - AND d.id NOT IN (SELECT dataset_id - FROM library_dataset_dataset_association) - ) sizes) - WHERE id = :id - RETURNING disk_usage; - """ - r = sa_session.execute(sql, {'id': id}) - new = r.fetchone()[0] - if dryrun: - sa_session.rollback() - return new diff -r 970982d0841d3b1717ae0263f303cfed6305c46f -r 7ef30f55b759ef1bbd53cd1113887f3637fc4e96 scripts/set_user_disk_usage.py --- a/scripts/set_user_disk_usage.py +++ b/scripts/set_user_disk_usage.py @@ -1,9 +1,22 @@ #!/usr/bin/env python -import os, sys +import os +import sys + from ConfigParser import ConfigParser from optparse import OptionParser +sys.path.insert( 1, os.path.join( os.path.dirname( __file__ ), '..', 'lib' ) ) + +from galaxy import eggs +import pkg_resources + +import galaxy.config +from galaxy.model.util import pgcalc +from galaxy.util import nice_size +from galaxy.objectstore import build_object_store_from_config + + default_config = os.path.abspath( os.path.join( os.path.dirname( __file__ ), '..', 'config/galaxy.ini') ) parser = OptionParser() @@ -13,6 +26,7 @@ parser.add_option( '--dry-run', dest='dryrun', help='Dry run (show changes but do not save to database)', action='store_true', default=False ) ( options, args ) = parser.parse_args() + def init(): options.config = os.path.abspath( options.config ) @@ -21,19 +35,8 @@ if options.email == 'all': options.email = None - sys.path.insert( 1, os.path.join( os.path.dirname( __file__ ), '..', 'lib' ) ) - - from galaxy import eggs - import pkg_resources - - import galaxy.config - from galaxy.objectstore import build_object_store_from_config - - # lazy - globals()['nice_size'] = __import__( 'galaxy.util', globals(), locals(), ( 'nice_size', ) ).nice_size - - config_parser = ConfigParser( dict( here = os.getcwd(), - database_connection = 'sqlite:///database/universe.sqlite?isolation_level=IMMEDIATE' ) ) + config_parser = ConfigParser( dict( here=os.getcwd(), + database_connection='sqlite:///database/universe.sqlite?isolation_level=IMMEDIATE' ) ) config_parser.read( options.config ) config_dict = {} @@ -45,31 +48,10 @@ from galaxy.model import mapping - return mapping.init( config.file_path, config.database_connection, create_tables = False, object_store = object_store ), object_store, config.database_connection.split(':')[0] + return (mapping.init( config.file_path, config.database_connection, create_tables=False, object_store=object_store ), + object_store, + config.database_connection.split(':')[0]) -def pgcalc( sa_session, id ): - sql = """ - UPDATE galaxy_user - SET disk_usage = (SELECT COALESCE(SUM(total_size), 0) - FROM ( SELECT DISTINCT ON (d.id) d.total_size, d.id - FROM history_dataset_association hda - JOIN history h ON h.id = hda.history_id - JOIN dataset d ON hda.dataset_id = d.id - WHERE h.user_id = :id - AND h.purged = false - AND hda.purged = false - AND d.purged = false - AND d.id NOT IN (SELECT dataset_id - FROM library_dataset_dataset_association) - ) sizes) - WHERE id = :id - RETURNING disk_usage; - """ - r = sa_session.execute(sql, {'id':id}) - new = r.fetchone()[0] - if options.dryrun: - sa_session.rollback() - return new def quotacheck( sa_session, users, engine ): sa_session.refresh( user ) @@ -83,7 +65,7 @@ print 'usage changed while calculating, trying again...' return quotacheck( sa_session, user, engine ) else: - new = pgcalc( sa_session, user.id ) + new = pgcalc( sa_session, user.id, dryrun=options.dryrun ) # yes, still a small race condition between here and the flush print 'old usage:', nice_size( current ), 'change:', if new in ( current, None ): https://bitbucket.org/galaxy/galaxy-central/commits/51264e606805/ Changeset: 51264e606805 User: dannon Date: 2015-02-25 17:54:43+00:00 Summary: Remove finished TODO. Affected #: 1 file diff -r 7ef30f55b759ef1bbd53cd1113887f3637fc4e96 -r 51264e606805464d65d032b06d80021d1bf2800e lib/galaxy/model/util.py --- a/lib/galaxy/model/util.py +++ b/lib/galaxy/model/util.py @@ -7,8 +7,6 @@ """ Utility method for quickly recalculating user disk usage in postgres. - TODO: Move this elsewhere and update scripts/set_user_disk_usage.py - TODO: Check against the recently updated versions of sqlalchemy if this 'special' postgresql version is even necessary. """ Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

[galaxyproject/usegalaxy-playbook] 1d0a9a: Update test to 658833a7988fe335a5aaaac9709f9e6dfed...
by GitHub 25 Feb '15

25 Feb '15

Branch: refs/heads/master Home: https://github.com/galaxyproject/usegalaxy-playbook Commit: 1d0a9a2d58d4d1dfcfbbbbd08a9053c8e0bd8107 https://github.com/galaxyproject/usegalaxy-playbook/commit/1d0a9a2d58d4d1df… Author: Dannon Baker <dannon.baker(a)gmail.com> Date: 2015-02-25 (Wed, 25 Feb 2015) Changed paths: M files/galaxy/test.galaxyproject.org/var/integrated_tool_panel.xml M files/galaxy/test.galaxyproject.org/var/shed_tool_conf.xml M stage/group_vars/all.yml Log Message: ----------- Update test to 658833a7988fe335a5aaaac9709f9e6dfed4a7b2

1 0

commit/galaxy-central: 4 new changesets
by commits-noreply＠bitbucket.org 25 Feb '15

25 Feb '15

4 new commits in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/1177b255f17d/ Changeset: 1177b255f17d User: martenson Date: 2015-02-23 22:42:17+00:00 Summary: add API endpoint to create TS repository Affected #: 3 files diff -r a12fc23ff13785d2c2ef7d223dca100ef334ea87 -r 1177b255f17d80190849fc64a5ce9bd88bbe7fd7 lib/galaxy/webapps/tool_shed/api/repositories.py --- a/lib/galaxy/webapps/tool_shed/api/repositories.py +++ b/lib/galaxy/webapps/tool_shed/api/repositories.py @@ -10,8 +10,10 @@ from galaxy import util from galaxy import web +from galaxy import exceptions from galaxy.datatypes import checkers from galaxy.model.orm import and_ +from galaxy.web import _future_expose_api as expose_api from galaxy.web.base.controller import BaseAPIController from galaxy.web.base.controller import HTTPBadRequest from galaxy.web.framework.helpers import time_ago @@ -31,6 +33,7 @@ from tool_shed.util import shed_util_common as suc from tool_shed.util import tool_util + log = logging.getLogger( __name__ ) @@ -577,13 +580,72 @@ id=trans.security.encode_id( repository.id ) ) return repository_dict + @expose_api + def create( self, trans, payload, **kwd ): + """ + create( self, trans, payload, **kwd ) + * POST /api/repositories: + Creates a new repository. + Only ``name`` and ``synopsis`` parameters are required. + + :param payload: dictionary structure containing:: + 'name': new repo's name (required) + 'synopsis': new repo's synopsis (required) + 'description': new repo's description (optional) + 'remote_repository_url': new repo's remote repo (optional) + 'homepage_url': new repo's homepage url (optional) + 'category_ids[]': list of existing encoded TS category ids + the new repo should be associated with (optional) + 'type': new repo's type, defaults to ``unrestricted`` (optional) + + :type payload: dict + + :returns: detailed repository information + :rtype: dict + + :raises: RequestParameterMissingException + """ + params = util.Params( payload ) + name = util.restore_text( params.get( 'name', None ) ) + if not name: + raise exceptions.RequestParameterMissingException( "Missing required parameter 'name'." ) + synopsis = util.restore_text( params.get( 'synopsis', None ) ) + if not synopsis: + raise exceptions.RequestParameterMissingException( "Missing required parameter 'synopsis'." ) + + description = util.restore_text( params.get( 'description', '' ) ) + remote_repository_url = util.restore_text( params.get( 'remote_repository_url', '' ) ) + homepage_url = util.restore_text( params.get( 'homepage_url', '' ) ) + category_ids = util.listify( params.get( 'category_ids[]', '' ) ) + selected_categories = [ trans.security.decode_id( id ) for id in category_ids ] + + repo_type = kwd.get( 'type', rt_util.UNRESTRICTED ) + if repo_type not in rt_util.types: + raise exceptions.RequestParameterInvalidException( 'This repository type is not valid' ) + + invalid_message = repository_util.validate_repository_name( trans.app, name, trans.user ) + if invalid_message: + raise exceptions.RequestParameterInvalidException( invalid_message ) + + repo, message = repository_util.create_repository( app=trans.app, + name=name, + type=repo_type, + description=synopsis, + long_description=description, + user_id = trans.user.id, + category_ids=category_ids, + remote_repository_url=remote_repository_url, + homepage_url=homepage_url ) + + return repo.to_dict( view='element', value_mapper=self.__get_value_mapper( trans ) ) + @web.expose_api def create_changeset_revision( self, trans, id, payload, **kwd ): """ POST /api/repositories/{encoded_repository_id}/changeset_revision Create a new tool shed repository commit - leaving PUT on parent - resource open for updating meta-attirbutes of the repository (and + resource open for updating meta-attributes of the repository (and Galaxy doesn't allow PUT multipart data anyway https://trello.com/c/CQwmCeG6). diff -r a12fc23ff13785d2c2ef7d223dca100ef334ea87 -r 1177b255f17d80190849fc64a5ce9bd88bbe7fd7 lib/galaxy/webapps/tool_shed/buildapp.py --- a/lib/galaxy/webapps/tool_shed/buildapp.py +++ b/lib/galaxy/webapps/tool_shed/buildapp.py @@ -129,6 +129,11 @@ controller='repositories', action='create_changeset_revision', conditions=dict( method=[ "POST" ] ) ) + webapp.mapper.connect( 'create_repository', + '/api/repositories', + controller='repositories', + action='create', + conditions=dict( method=[ "POST" ] ) ) webapp.finalize_config() # Wrap the webapp in some useful middleware diff -r a12fc23ff13785d2c2ef7d223dca100ef334ea87 -r 1177b255f17d80190849fc64a5ce9bd88bbe7fd7 lib/tool_shed/util/repository_util.py --- a/lib/tool_shed/util/repository_util.py +++ b/lib/tool_shed/util/repository_util.py @@ -313,8 +313,11 @@ return associations_dict def validate_repository_name( app, name, user ): - # Repository names must be unique for each user, must be at least four characters - # in length and must contain only lower-case letters, numbers, and the '_' character. + """ + Validate whether the given name qualifies as a new TS repo name. + Repository names must be unique for each user, must be at least two characters + in length and must contain only lower-case letters, numbers, and the '_' character. + """ if name in [ 'None', None, '' ]: return 'Enter the required repository name.' if name in [ 'repos' ]: @@ -322,13 +325,13 @@ check_existing = suc.get_repository_by_name_and_owner( app, name, user.username ) if check_existing is not None: if check_existing.deleted: - return 'You have a deleted repository named %s, so choose a different name.' % name + return 'You own a deleted repository named %s, please choose a different name.' % escape( name ) else: - return "You already have a repository named %s, so choose a different name." % name + return "You already own a repository named %s, please choose a different name." % escape( name ) if len( name ) < 2: return "Repository names must be at least 2 characters in length." if len( name ) > 80: return "Repository names cannot be more than 80 characters in length." if not( VALID_REPOSITORYNAME_RE.match( name ) ): - return "Repository names must contain only lower-case letters, numbers and underscore _." + return "Repository names must contain only lower-case letters, numbers and underscore." return '' https://bitbucket.org/galaxy/galaxy-central/commits/05d1b7925348/ Changeset: 05d1b7925348 User: jmchilton Date: 2015-02-24 20:37:45+00:00 Summary: Fix tool shed-y workflowing for ce1f85a0f09689e86fce1839390a5a00a730321f. Affected #: 1 file diff -r 1177b255f17d80190849fc64a5ce9bd88bbe7fd7 -r 05d1b792534806bd064865d813edcff012dbf81f lib/tool_shed/util/workflow_util.py --- a/lib/tool_shed/util/workflow_util.py +++ b/lib/tool_shed/util/workflow_util.py @@ -6,10 +6,10 @@ import galaxy.tools import galaxy.tools.parameters -import galaxy.webapps.galaxy.controllers.workflow from galaxy.util import json from galaxy.util.sanitize_html import sanitize_html from galaxy.workflow.render import WorkflowCanvas +from galaxy.workflow.steps import attach_ordered_steps from galaxy.workflow.modules import module_types from galaxy.workflow.modules import ToolModule from galaxy.workflow.modules import WorkflowModuleFactory @@ -312,7 +312,7 @@ step.input_connections.append( conn ) del step.temp_input_connections # Order the steps if possible. - galaxy.webapps.galaxy.controllers.workflow.attach_ordered_steps( workflow, steps ) + attach_ordered_steps( workflow, steps ) # Return the in-memory Workflow object for display or later persistence to the Galaxy database. return workflow, missing_tool_tups https://bitbucket.org/galaxy/galaxy-central/commits/20bf9ebc1cd1/ Changeset: 20bf9ebc1cd1 User: carlfeberhard Date: 2015-02-24 22:20:10+00:00 Summary: Fix to layering of dropdown buttons and menus Affected #: 3 files diff -r 05d1b792534806bd064865d813edcff012dbf81f -r 20bf9ebc1cd143d29bb651caf1994f1d9c2db05a static/style/blue/base.css --- a/static/style/blue/base.css +++ b/static/style/blue/base.css @@ -2030,8 +2030,8 @@ .list-panel .controls .list-actions:after{clear:both} .list-panel .controls .list-actions:before,.list-panel .controls .list-actions:after{content:" ";display:table;} .list-panel .controls .list-actions:after{clear:both} -.list-panel .controls .list-actions .btn{padding-top:2px;padding-bottom:2px;font-size:90%;z-index:inherit} -.list-panel .controls .list-actions .list-action-menu{float:right}.list-panel .controls .list-actions .list-action-menu .dropdown-menu{z-index:inherit} +.list-panel .controls .list-actions .btn{padding-top:2px;padding-bottom:2px;font-size:90%} +.list-panel .controls .list-actions .list-action-menu{float:right} .list-panel .list-items .list-item:not(:last-child){border-bottom-width:0px} .list-panel .empty-message{display:none;margin:0px} .list-item .details .list-panel{margin-top:8px;border-radius:3px;background:white;padding:4px}.list-item .details .list-panel .list-items{border:1px solid #bfbfbf;border-radius:3px}.list-item .details .list-panel .list-items .list-item:first-child{border-top-width:0px;border-radius:3px 3px 0px 0px} @@ -2138,9 +2138,10 @@ .multi-panel-history .control-column.control-column-right{text-align:right} .multi-panel-history .outer-middle{overflow:auto} .multi-panel-history .middle{min-width:100%;margin:0px 0px 0px 0px;background-color:white;padding:8px} -.multi-panel-history .history-column{width:312px;margin:0 8px 0 0}.multi-panel-history .history-column:first-child{position:fixed;z-index:2}.multi-panel-history .history-column:first-child .history-panel{border:1px solid black;box-shadow:4px 4px 4px rgba(96,96,96,0.3)} +.multi-panel-history .history-column{width:312px;margin:0 8px 0 0}.multi-panel-history .history-column:first-child{position:fixed;z-index:10}.multi-panel-history .history-column:first-child .history-panel{border:1px solid black;box-shadow:4px 4px 4px rgba(96,96,96,0.3)} .multi-panel-history .history-column:nth-child(2){margin-left:320px} .multi-panel-history .history-column:last-child{margin-right:0px} +.multi-panel-history .history-column .dropdown-menu{z-index:inherit} .multi-panel-history .history-column .panel-controls{width:100%;height:24px;border-radius:3px;background-color:white;text-align:center;flex:0 0 auto;-webkit-align-self:auto;-ms-flex-item-align:auto;align-self:auto}.multi-panel-history .history-column .panel-controls .btn{height:20px;line-height:normal;font-size:90%;padding-top:0px;padding-bottom:0px} .multi-panel-history .history-column .panel-controls .pull-left .btn{margin-right:4px} .multi-panel-history .history-column .panel-controls .pull-right .btn{margin-left:4px} diff -r 05d1b792534806bd064865d813edcff012dbf81f -r 20bf9ebc1cd143d29bb651caf1994f1d9c2db05a static/style/src/less/history.less --- a/static/style/src/less/history.less +++ b/static/style/src/less/history.less @@ -744,7 +744,7 @@ // current history &:first-child { position: fixed; - z-index : 2; + z-index : 10; // visually differentiate the current history .history-panel { @@ -759,6 +759,9 @@ &:last-child { margin-right: 0px; } + .dropdown-menu { + z-index: inherit; + } .panel-controls { width: 100%; diff -r 05d1b792534806bd064865d813edcff012dbf81f -r 20bf9ebc1cd143d29bb651caf1994f1d9c2db05a static/style/src/less/list-item.less --- a/static/style/src/less/list-item.less +++ b/static/style/src/less/list-item.less @@ -167,13 +167,9 @@ padding-top: 2px; padding-bottom: 2px; font-size: 90%; - z-index: inherit; } .list-action-menu { float: right; - .dropdown-menu { - z-index: inherit; - } } } } https://bitbucket.org/galaxy/galaxy-central/commits/658833a7988f/ Changeset: 658833a7988f User: dannon Date: 2015-02-24 23:55:50+00:00 Summary: More tweaks to mutable capabilities, adding back in a copy and stripping pickle from MutationList Affected #: 1 file diff -r 20bf9ebc1cd143d29bb651caf1994f1d9c2db05a -r 658833a7988fe335a5aaaac9709f9e6dfed4a7b2 lib/galaxy/model/custom_types.py --- a/lib/galaxy/model/custom_types.py +++ b/lib/galaxy/model/custom_types.py @@ -1,4 +1,5 @@ import binascii +import copy import json import logging import uuid @@ -62,6 +63,12 @@ else: return self.impl + def copy_value( self, value ): + return copy.deepcopy( value ) + + def compare_values( self, x, y ): + return ( x == y ) + class MutationObj(Mutable): """ @@ -196,12 +203,6 @@ list.remove(self, value) self.changed() - def __getstate__(self): - return list(self) - - def __setstate__(self, state): - self.update(state) - MutationObj.associate_with(JSONType) Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

[galaxyproject/usegalaxy-playbook] f65108: Fetch modified integrated_tool_panel.xml from Main
by GitHub 25 Feb '15

25 Feb '15

Branch: refs/heads/master Home: https://github.com/galaxyproject/usegalaxy-playbook Commit: f65108118e2fa0f18274944b21084b1234937889 https://github.com/galaxyproject/usegalaxy-playbook/commit/f65108118e2fa0f1… Author: Nate Coraor <nate(a)bx.psu.edu> Date: 2015-02-24 (Tue, 24 Feb 2015) Changed paths: M files/galaxy/usegalaxy.org/var/integrated_tool_panel.xml Log Message: ----------- Fetch modified integrated_tool_panel.xml from Main Commit: cf4260106a86f13850876a14db8976fbf1c719d9 https://github.com/galaxyproject/usegalaxy-playbook/commit/cf4260106a86f138… Author: Nate Coraor <nate(a)bx.psu.edu> Date: 2015-02-24 (Tue, 24 Feb 2015) Changed paths: M templates/galaxy/test.galaxyproject.org/config/job_conf.xml.j2 M templates/galaxy/usegalaxy.org/config/job_conf.xml.j2 Log Message: ----------- Switch Test/Main to use a slurm-drmaa compiled without optimization, to assist with tracking down the pthread deadlock. Compare: https://github.com/galaxyproject/usegalaxy-playbook/compare/eb8faf9d51a1...…

1 0

[galaxyproject/usegalaxy-playbook] eb8faf: Unfortunately, metadata failure is still considere...
by GitHub 25 Feb '15

25 Feb '15

Branch: refs/heads/master Home: https://github.com/galaxyproject/usegalaxy-playbook Commit: eb8faf9d51a1b7cffa7f75bdf5b9bc4d5495f05b https://github.com/galaxyproject/usegalaxy-playbook/commit/eb8faf9d51a1b7cf… Author: Nate Coraor <nate(a)bx.psu.edu> Date: 2015-02-24 (Tue, 24 Feb 2015) Changed paths: M stage/group_vars/galaxyservers.yml Log Message: ----------- Unfortunately, metadata failure is still considered a job "success", and the jobdir is removed. So, set Test to never remove jobdirs while we debug metadata failures.

1 0

[galaxyproject/usegalaxy-playbook] bf3597: Update Test
by GitHub 25 Feb '15

25 Feb '15

Branch: refs/heads/master Home: https://github.com/galaxyproject/usegalaxy-playbook Commit: bf3597a0e5a22b66969f58b22cf08ce5c507bcbb https://github.com/galaxyproject/usegalaxy-playbook/commit/bf3597a0e5a22b66… Author: Nate Coraor <nate(a)bx.psu.edu> Date: 2015-02-24 (Tue, 24 Feb 2015) Changed paths: M stage/group_vars/all.yml Log Message: ----------- Update Test

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

galaxy-commits February 2015