- galaxy-commits - lists.galaxyproject.org

commit/galaxy-central: greg: Enhance and correct documentation in the new tool_shed_repositories Galaxy API controller as well as the documentation in the example API install scripts. Fix discovery of the tool_path setting in the selected shed-related tool panel config file in the tool_shed_repositories controller's install_repository_revision method.
by commits-noreply＠bitbucket.org 26 Mar '13

26 Mar '13

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/b756a49b4245/ Changeset: b756a49b4245 User: greg Date: 2013-03-26 14:29:02 Summary: Enhance and correct documentation in the new tool_shed_repositories Galaxy API controller as well as the documentation in the example API install scripts. Fix discovery of the tool_path setting in the selected shed-related tool panel config file in the tool_shed_repositories controller's install_repository_revision method. Affected #: 3 files diff -r 50e8e5efacadef15969a663ac3849883e0a49727 -r b756a49b424565f5ec4037f25a89e987a47d184b lib/galaxy/webapps/galaxy/api/tool_shed_repositories.py --- a/lib/galaxy/webapps/galaxy/api/tool_shed_repositories.py +++ b/lib/galaxy/webapps/galaxy/api/tool_shed_repositories.py @@ -58,7 +58,7 @@ GET /api/tool_shed_repositories/{encoded_tool_shed_repsository_id} Display a dictionary containing information about a specified tool_shed_repository. - :param tool_shed_repository_id: the encoded id of the `ToolShedRepository` object + :param id: the encoded id of the ToolShedRepository object """ # Example URL: http://localhost:8763/api/tool_shed_repositories/df7a1f0c02a5b08e try: @@ -80,18 +80,32 @@ POST /api/tool_shed_repositories/install_repository_revision Install a specified repository revision from a specified tool shed into Galaxy. - :param tool_shed_url: the base URL of the Tool Shed from which to install the Repository - :param name: the name of the Repository - :param owner: the owner of the Repository - :param changset_revision: the changset_revision of the RepositoryMetadata object associated with the Repository :param key: the current Galaxy admin user's API key - :param new_tool_panel_section_label: optional label of a new section to be added to the Galaxy tool panel in which to load - tools contained in the Repository. Either this parameter must be an empty string or - the tool_panel_section_id parameter must be an empty string, as both cannot be used. - :param tool_panel_section_id: optional id of the Galaxy tool panel section in which to load tools contained in the Repository. - If not set, tools will be loaded outside of any sections in the tool panel. Either this - parameter must be an empty string or the tool_panel_section_id parameter must be an empty string, - as both cannot be used. + + The following parameters are included in the payload. + :param tool_shed_url (required): the base URL of the Tool Shed from which to install the Repository + :param name (required): the name of the Repository + :param owner (required): the owner of the Repository + :param changset_revision (required): the changset_revision of the RepositoryMetadata object associated with the Repository + :param new_tool_panel_section_label (optional): label of a new section to be added to the Galaxy tool panel in which to load + tools contained in the Repository. Either this parameter must be an empty string or + the tool_panel_section_id parameter must be an empty string or both must be an empty + string (both cannot be used simultaneously). + :param tool_panel_section_id (optional): id of the Galaxy tool panel section in which to load tools contained in the Repository. + If this parameter is an empty string and the above new_tool_panel_section_label parameter is an + empty string, tools will be loaded outside of any sections in the tool panel. Either this + parameter must be an empty string or the tool_panel_section_id parameter must be an empty string + of both must be an empty string (both cannot be used simultaneously). + :param install_repository_dependencies (optional): Set to True if you want to install repository dependencies defined for the specified + repository being installed. The default setting is False. + :param install_tool_dependencies (optional): Set to True if you want to install tool dependencies defined for the specified repository being + installed. The default setting is False. + :param shed_tool_conf (optional): The shed-related tool panel configuration file configured in the "tool_config_file" setting in the Galaxy config file + (e.g., universe_wsgi.ini). At least one shed-related tool panel config file is required to be configured. Setting + this parameter to a specific file enables you to choose where the specified repository will be installed because + the tool_path attribute of the <toolbox> from the specified file is used as the installation location + (e.g., <toolbox tool_path="../shed_tools">). If this parameter is not set, a shed-related tool panel configuration + file will be selected automatically. """ # Get the information about the repository to be installed from the payload. tool_shed_url = payload.get( 'tool_shed_url', '' ) @@ -160,29 +174,30 @@ includes_tools_for_display_in_tool_panel = repository_revision_dict[ 'includes_tools_for_display_in_tool_panel' ] except: raise HTTPBadRequest( detail="Missing required parameter 'includes_tools_for_display_in_tool_panel'." ) - # Get the information about the Galaxy components (e.g., tool pane section, tool config file, etc) that will contain the repository - # information. + # Get the information about the Galaxy components (e.g., tool pane section, tool config file, etc) that will contain the repository information. install_repository_dependencies = payload.get( 'install_repository_dependencies', False ) install_tool_dependencies = payload.get( 'install_tool_dependencies', False ) new_tool_panel_section = payload.get( 'new_tool_panel_section_label', '' ) shed_tool_conf = payload.get( 'shed_tool_conf', None ) - tool_path = payload.get( 'tool_path', None ) - tool_panel_section_id = payload.get( 'tool_panel_section_id', '' ) - if tool_panel_section_id not in [ None, '' ]: - tool_panel_section = trans.app.toolbox.tool_panel[ tool_panel_section_id ] + if shed_tool_conf: + # Get the tool_path setting. + index, shed_conf_dict = suc.get_shed_tool_conf_dict( trans.app, shed_tool_conf ) + tool_path = shed_config_dict[ 'tool_path' ] else: - tool_panel_section = '' - if not shed_tool_conf or not tool_path: - # Pick a semi-random shed-related tool panel configuration file. + # Pick a semi-random shed-related tool panel configuration file and get the tool_path setting. for shed_config_dict in trans.app.toolbox.shed_tool_confs: + # Don't use migrated_tools_conf.xml. if shed_config_dict[ 'config_filename' ] != trans.app.config.migrated_tools_config: break shed_tool_conf = shed_config_dict[ 'config_filename' ] tool_path = shed_config_dict[ 'tool_path' ] if not shed_tool_conf: raise HTTPBadRequest( detail="Missing required parameter 'shed_tool_conf'." ) - if not tool_path: - raise HTTPBadRequest( detail="Missing required parameter 'tool_path'." ) + tool_panel_section_id = payload.get( 'tool_panel_section_id', '' ) + if tool_panel_section_id not in [ None, '' ]: + tool_panel_section = trans.app.toolbox.tool_panel[ tool_panel_section_id ] + else: + tool_panel_section = '' # Build the dictionary of information necessary for creating tool_shed_repository database records for each repository being installed. installation_dict = dict( install_repository_dependencies=install_repository_dependencies, new_tool_panel_section=new_tool_panel_section, diff -r 50e8e5efacadef15969a663ac3849883e0a49727 -r b756a49b424565f5ec4037f25a89e987a47d184b scripts/api/install_repository_tools_into_existing_tool_panel_section.py --- a/scripts/api/install_repository_tools_into_existing_tool_panel_section.py +++ b/scripts/api/install_repository_tools_into_existing_tool_panel_section.py @@ -9,7 +9,7 @@ <section id="from_test_tool_shed" name="From Test Tool Shed" version=""></section> -usage: ./install_repository_tools_into_existing_tool_panel_section <api_key <galaxy base url> tool_shed_url name owner changeset_revision tool_panel_section_id +usage: ./install_repository_tools_into_existing_tool_panel_section.py <api_key <galaxy base url> tool_shed_url name owner changeset_revision tool_panel_section_id Here is a working example of how to use this script to install a repository from the test tool shed. ./install_repository_tools_into_existing_tool_panel_section.py <api key><galaxy base url>/api/tool_shed_repositories/new/install_repository_revision http://testtoolshed.g2.bx.psu.edu gregs_filter greg f28d5018f9cb from_test_tool_shed diff -r 50e8e5efacadef15969a663ac3849883e0a49727 -r b756a49b424565f5ec4037f25a89e987a47d184b scripts/api/install_repository_tools_into_new_tool_panel_section.py --- a/scripts/api/install_repository_tools_into_new_tool_panel_section.py +++ b/scripts/api/install_repository_tools_into_new_tool_panel_section.py @@ -4,10 +4,10 @@ valid tools, loading them into a new section of the Galaxy tool panel. The repository has no tool dependencies or repository dependencies, so only a single repository will be installed. -usage: ./install_repository_tools_into_new_tool_panel_section <api_key <galaxy base url> tool_shed_url name owner changeset_revision new_tool_panel_section_label +usage: ./install_repository_tools_into_new_tool_panel_section.py <api_key <galaxy base url> tool_shed_url name owner changeset_revision new_tool_panel_section_label Here is a working example of how to use this script to install a repository from the test tool shed. -./install_repository_tools_into_new_tool_panel_section.py <api key><galaxy base url>/api/tool_shed_repositories/new/install_repository_revision http://testtoolshed.g2.bx.psu.edu gregs_filter greg f28d5018f9cb From%20Test%20Tool%20Shed +./install_repository_tools_into_new_tool_panel_section.py <api key><galaxy base url>/api/tool_shed_repositories/new/install_repository_revision http://testtoolshed.g2.bx.psu.edu gregs_filter greg f28d5018f9cb 'From Test Tool Shed' """ import os Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: inithello: More explicit specification of return codes, documentation of same.
by commits-noreply＠bitbucket.org 26 Mar '13

26 Mar '13

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/50e8e5efacad/ Changeset: 50e8e5efacad User: inithello Date: 2013-03-26 14:27:39 Summary: More explicit specification of return codes, documentation of same. Affected #: 1 file diff -r 5b7bc81c3fc7712894a1ff1bd75476259bf6c591 -r 50e8e5efacadef15969a663ac3849883e0a49727 test/install_and_test_tool_shed_repositories/functional_tests.py --- a/test/install_and_test_tool_shed_repositories/functional_tests.py +++ b/test/install_and_test_tool_shed_repositories/functional_tests.py @@ -787,14 +787,19 @@ print '# ----------------------------------------------------------------------------------' print "# %d repositories not installed correctly:" % len( repositories_failed_install ) show_summary_output( repositories_failed_install ) + else: + success = True + else: + success = True print "####################################################################################" - if repositories_tested > 0: - if success: - return 0 - else: - return 1 + # Normally, the value of 'success' would determine whether this test suite is marked as passed or failed + # in the automated buildbot framework. However, due to the procedure used here, we only want to report + # failure if a repository fails to install correctly. Therefore, we have overriden the value of 'success' + # here based on what actions the script has executed. + if success: + return 0 else: - return 0 + return 1 if __name__ == "__main__": now = strftime( "%Y-%m-%d %H:%M:%S" ) Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: 2 new changesets
by commits-noreply＠bitbucket.org 25 Mar '13

25 Mar '13

2 new commits in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/ea2a5e240c41/ Changeset: ea2a5e240c41 Branch: next-stable User: carlfeberhard Date: 2013-03-26 00:04:37 Summary: history panel: better bootstrapped history and hda data; UsesHistoryDatasetAssociationMixin: don't try to provide full data for deleted, purged, or non-accessible hda Affected #: 3 files diff -r 388648deecffe1d5d3c46525aabd5b4d55549248 -r ea2a5e240c41c9d58a3982d21af362781db980da lib/galaxy/web/base/controller.py --- a/lib/galaxy/web/base/controller.py +++ b/lib/galaxy/web/base/controller.py @@ -334,31 +334,40 @@ def get_hda_dict( self, trans, hda ): hda_dict = hda.get_api_value( view='element' ) history = hda.history + hda_dict[ 'api_type' ] = "file" # Add additional attributes that depend on trans can hence must be added here rather than at the model level. + can_access_hda = trans.app.security_agent.can_access_dataset( trans.get_current_user_roles(), hda.dataset ) + can_access_hda = ( trans.user_is_admin() or can_access_hda ) + hda_dict[ 'accessible' ] = can_access_hda + + # return here if deleted and purged or can't access + purged = ( hda.purged or hda.dataset.purged ) + if ( hda.deleted and purged ) or not can_access_hda: + return trans.security.encode_dict_ids( hda_dict ) + if trans.user_is_admin() or trans.app.config.expose_dataset_path: hda_dict[ 'file_name' ] = hda.file_name - if not hda_dict[ 'deleted' ]: - hda_dict[ 'download_url' ] = url_for( 'history_contents_display', history_id = trans.security.encode_id( history.id ), - history_content_id = trans.security.encode_id( hda.id ) ) + hda_dict[ 'download_url' ] = url_for( 'history_contents_display', + history_id = trans.security.encode_id( history.id ), + history_content_id = trans.security.encode_id( hda.id ) ) - can_access_hda = trans.app.security_agent.can_access_dataset( trans.get_current_user_roles(), hda.dataset ) - hda_dict[ 'accessible' ] = ( trans.user_is_admin() or can_access_hda ) - hda_dict[ 'api_type' ] = "file" - - if not( hda.purged or hda.deleted or hda.dataset.purged ): - meta_files = [] - for meta_type in hda.metadata.spec.keys(): - if isinstance( hda.metadata.spec[ meta_type ].param, FileParameter ): - meta_files.append( dict( file_type=meta_type ) ) - if meta_files: - hda_dict[ 'meta_files' ] = meta_files + meta_files = [] + for meta_type in hda.metadata.spec.keys(): + if isinstance( hda.metadata.spec[ meta_type ].param, FileParameter ): + meta_files.append( dict( file_type=meta_type ) ) + if meta_files: + hda_dict[ 'meta_files' ] = meta_files hda_dict[ 'display_apps' ] = self.get_display_apps( trans, hda ) hda_dict[ 'display_types' ] = self.get_old_display_applications( trans, hda ) hda_dict[ 'visualizations' ] = hda.get_visualizations() + # return here if deleted + if hda.deleted and not purged: + return trans.security.encode_dict_ids( hda_dict ) + if hda.creating_job and hda.creating_job.tool_id: tool_used = trans.app.toolbox.get_tool( hda.creating_job.tool_id ) if tool_used and tool_used.force_history_refresh: @@ -813,7 +822,6 @@ history = self.get_object( trans, id, 'History', check_ownership=check_ownership, check_accessible=check_accessible, deleted=deleted ) return self.security_check( trans, history, check_ownership, check_accessible ) - def get_history_datasets( self, trans, history, show_deleted=False, show_hidden=False, show_purged=False ): """ Returns history's datasets. """ query = trans.sa_session.query( trans.model.HistoryDatasetAssociation ) \ diff -r 388648deecffe1d5d3c46525aabd5b4d55549248 -r ea2a5e240c41c9d58a3982d21af362781db980da lib/galaxy/webapps/galaxy/controllers/root.py --- a/lib/galaxy/webapps/galaxy/controllers/root.py +++ b/lib/galaxy/webapps/galaxy/controllers/root.py @@ -11,7 +11,7 @@ log = logging.getLogger( __name__ ) -class RootController( BaseUIController, UsesHistoryMixin, UsesAnnotations ): +class RootController( BaseUIController, UsesHistoryMixin, UsesHistoryDatasetAssociationMixin, UsesAnnotations ): @web.expose def default(self, trans, target1=None, target2=None, **kwd): @@ -130,8 +130,39 @@ else: # get all datasets server-side, client-side will get flags and render appropriately - datasets = self.get_history_datasets( trans, history, - show_deleted=True, show_hidden=True, show_purged=True ) + hdas = self.get_history_datasets( trans, history, + show_deleted=True, show_hidden=True, show_purged=True ) + + #TODO: would be good to re-use the hdas above to get the history data... + history_dictionary = self.get_history_dict( trans, history ) + + #TODO: blech - all here for now - duplication of hist. contents, index + hda_dictionaries = [] + for hda in hdas: + try: + hda_dictionaries.append( self.get_hda_dict( trans, hda ) ) + + except Exception, exc: + # don't fail entire list if hda err's, record and move on + # (making sure http recvr knows it's err'd) + encoded_hda_id = trans.security.encode_id( hda.id ) + log.error( "Error in history API at listing contents with history %s, hda %s: (%s) %s", + history_dictionary[ 'id' ], encoded_hda_id, type( exc ), str( exc ) ) + return_val = { + 'id' : encoded_hda_id, + 'name' : hda.name, + 'hid' : hda.hid, + 'history_id': history_dictionary[ 'id' ], + 'state' : trans.model.Dataset.states.ERROR, + 'visible' : True, + 'misc_info' : str( exception ), + 'misc_blurb': 'Failed to retrieve dataset information.', + 'error' : str( exception ) + } + hda_dictionaries.append( return_val ) + + history = history_dictionary + datasets = hda_dictionaries return trans.stream_template_mako( history_panel_template, history = history, diff -r 388648deecffe1d5d3c46525aabd5b4d55549248 -r ea2a5e240c41c9d58a3982d21af362781db980da templates/webapps/galaxy/root/alternate_history.mako --- a/templates/webapps/galaxy/root/alternate_history.mako +++ b/templates/webapps/galaxy/root/alternate_history.mako @@ -90,7 +90,7 @@ <% from urllib import unquote_plus - history_class_name = history.__class__.__name__ + history_class_name = 'History' encoded_id_template = '<%= id %>' url_dict = { @@ -189,12 +189,9 @@ ## I'd rather do without these (esp. the get_hdas which hits the db twice) ## but we'd need a common map producer (something like get_api_value but more complete) ##TODO: api/web controllers should use common code, and this section should call that code -<%def name="get_history( id )"> +<%def name="get_history( history )"><% - history_json = trans.webapp.api_controllers[ 'histories' ].show( trans, trans.security.encode_id( id ) ) - #assert isinstance( history, dict ), ( - # 'Bootstrapped history was expecting a dictionary: %s' %( str( history ) ) ) - return history_json + return h.to_json_string( history ) %></%def> @@ -207,16 +204,9 @@ %></%def> -<%def name="get_hdas( history_id, hdas )"> +<%def name="get_hdas( hdas )"><% - #BUG: one inaccessible dataset will error entire list - if not hdas: - return '[]' - # rather just use the history.id (wo the datasets), but... - hda_json = trans.webapp.api_controllers[ 'history_contents' ].index( - trans, trans.security.encode_id( history_id ), - ids=( ','.join([ trans.security.encode_id( hda.id ) for hda in hdas ]) ) ) - return hda_json + return h.to_json_string( hdas ) %></%def> @@ -324,8 +314,8 @@ page_show_hidden = ${ 'true' if show_hidden == True else ( 'null' if show_hidden == None else 'false' ) }, user = ${ get_current_user() }, - history = ${ get_history( history.id ) }, - hdas = ${ get_hdas( history.id, datasets ) }; + history = ${ get_history( history ) }, + hdas = ${ get_hdas( datasets ) }; // add user data to history // i don't like this history+user relationship, but user authentication changes views/behaviour https://bitbucket.org/galaxy/galaxy-central/commits/5b7bc81c3fc7/ Changeset: 5b7bc81c3fc7 User: carlfeberhard Date: 2013-03-26 00:05:30 Summary: merge next-stable Affected #: 3 files diff -r b437eece023175a6ac8ea87f7a4dae6582ad7892 -r 5b7bc81c3fc7712894a1ff1bd75476259bf6c591 lib/galaxy/web/base/controller.py --- a/lib/galaxy/web/base/controller.py +++ b/lib/galaxy/web/base/controller.py @@ -334,31 +334,40 @@ def get_hda_dict( self, trans, hda ): hda_dict = hda.get_api_value( view='element' ) history = hda.history + hda_dict[ 'api_type' ] = "file" # Add additional attributes that depend on trans can hence must be added here rather than at the model level. + can_access_hda = trans.app.security_agent.can_access_dataset( trans.get_current_user_roles(), hda.dataset ) + can_access_hda = ( trans.user_is_admin() or can_access_hda ) + hda_dict[ 'accessible' ] = can_access_hda + + # return here if deleted and purged or can't access + purged = ( hda.purged or hda.dataset.purged ) + if ( hda.deleted and purged ) or not can_access_hda: + return trans.security.encode_dict_ids( hda_dict ) + if trans.user_is_admin() or trans.app.config.expose_dataset_path: hda_dict[ 'file_name' ] = hda.file_name - if not hda_dict[ 'deleted' ]: - hda_dict[ 'download_url' ] = url_for( 'history_contents_display', history_id = trans.security.encode_id( history.id ), - history_content_id = trans.security.encode_id( hda.id ) ) + hda_dict[ 'download_url' ] = url_for( 'history_contents_display', + history_id = trans.security.encode_id( history.id ), + history_content_id = trans.security.encode_id( hda.id ) ) - can_access_hda = trans.app.security_agent.can_access_dataset( trans.get_current_user_roles(), hda.dataset ) - hda_dict[ 'accessible' ] = ( trans.user_is_admin() or can_access_hda ) - hda_dict[ 'api_type' ] = "file" - - if not( hda.purged or hda.deleted or hda.dataset.purged ): - meta_files = [] - for meta_type in hda.metadata.spec.keys(): - if isinstance( hda.metadata.spec[ meta_type ].param, FileParameter ): - meta_files.append( dict( file_type=meta_type ) ) - if meta_files: - hda_dict[ 'meta_files' ] = meta_files + meta_files = [] + for meta_type in hda.metadata.spec.keys(): + if isinstance( hda.metadata.spec[ meta_type ].param, FileParameter ): + meta_files.append( dict( file_type=meta_type ) ) + if meta_files: + hda_dict[ 'meta_files' ] = meta_files hda_dict[ 'display_apps' ] = self.get_display_apps( trans, hda ) hda_dict[ 'display_types' ] = self.get_old_display_applications( trans, hda ) hda_dict[ 'visualizations' ] = hda.get_visualizations() + # return here if deleted + if hda.deleted and not purged: + return trans.security.encode_dict_ids( hda_dict ) + if hda.creating_job and hda.creating_job.tool_id: tool_used = trans.app.toolbox.get_tool( hda.creating_job.tool_id ) if tool_used and tool_used.force_history_refresh: @@ -813,7 +822,6 @@ history = self.get_object( trans, id, 'History', check_ownership=check_ownership, check_accessible=check_accessible, deleted=deleted ) return self.security_check( trans, history, check_ownership, check_accessible ) - def get_history_datasets( self, trans, history, show_deleted=False, show_hidden=False, show_purged=False ): """ Returns history's datasets. """ query = trans.sa_session.query( trans.model.HistoryDatasetAssociation ) \ diff -r b437eece023175a6ac8ea87f7a4dae6582ad7892 -r 5b7bc81c3fc7712894a1ff1bd75476259bf6c591 lib/galaxy/webapps/galaxy/controllers/root.py --- a/lib/galaxy/webapps/galaxy/controllers/root.py +++ b/lib/galaxy/webapps/galaxy/controllers/root.py @@ -11,7 +11,7 @@ log = logging.getLogger( __name__ ) -class RootController( BaseUIController, UsesHistoryMixin, UsesAnnotations ): +class RootController( BaseUIController, UsesHistoryMixin, UsesHistoryDatasetAssociationMixin, UsesAnnotations ): @web.expose def default(self, trans, target1=None, target2=None, **kwd): @@ -130,8 +130,39 @@ else: # get all datasets server-side, client-side will get flags and render appropriately - datasets = self.get_history_datasets( trans, history, - show_deleted=True, show_hidden=True, show_purged=True ) + hdas = self.get_history_datasets( trans, history, + show_deleted=True, show_hidden=True, show_purged=True ) + + #TODO: would be good to re-use the hdas above to get the history data... + history_dictionary = self.get_history_dict( trans, history ) + + #TODO: blech - all here for now - duplication of hist. contents, index + hda_dictionaries = [] + for hda in hdas: + try: + hda_dictionaries.append( self.get_hda_dict( trans, hda ) ) + + except Exception, exc: + # don't fail entire list if hda err's, record and move on + # (making sure http recvr knows it's err'd) + encoded_hda_id = trans.security.encode_id( hda.id ) + log.error( "Error in history API at listing contents with history %s, hda %s: (%s) %s", + history_dictionary[ 'id' ], encoded_hda_id, type( exc ), str( exc ) ) + return_val = { + 'id' : encoded_hda_id, + 'name' : hda.name, + 'hid' : hda.hid, + 'history_id': history_dictionary[ 'id' ], + 'state' : trans.model.Dataset.states.ERROR, + 'visible' : True, + 'misc_info' : str( exception ), + 'misc_blurb': 'Failed to retrieve dataset information.', + 'error' : str( exception ) + } + hda_dictionaries.append( return_val ) + + history = history_dictionary + datasets = hda_dictionaries return trans.stream_template_mako( history_panel_template, history = history, diff -r b437eece023175a6ac8ea87f7a4dae6582ad7892 -r 5b7bc81c3fc7712894a1ff1bd75476259bf6c591 templates/webapps/galaxy/root/alternate_history.mako --- a/templates/webapps/galaxy/root/alternate_history.mako +++ b/templates/webapps/galaxy/root/alternate_history.mako @@ -90,7 +90,7 @@ <% from urllib import unquote_plus - history_class_name = history.__class__.__name__ + history_class_name = 'History' encoded_id_template = '<%= id %>' url_dict = { @@ -189,12 +189,9 @@ ## I'd rather do without these (esp. the get_hdas which hits the db twice) ## but we'd need a common map producer (something like get_api_value but more complete) ##TODO: api/web controllers should use common code, and this section should call that code -<%def name="get_history( id )"> +<%def name="get_history( history )"><% - history_json = trans.webapp.api_controllers[ 'histories' ].show( trans, trans.security.encode_id( id ) ) - #assert isinstance( history, dict ), ( - # 'Bootstrapped history was expecting a dictionary: %s' %( str( history ) ) ) - return history_json + return h.to_json_string( history ) %></%def> @@ -207,16 +204,9 @@ %></%def> -<%def name="get_hdas( history_id, hdas )"> +<%def name="get_hdas( hdas )"><% - #BUG: one inaccessible dataset will error entire list - if not hdas: - return '[]' - # rather just use the history.id (wo the datasets), but... - hda_json = trans.webapp.api_controllers[ 'history_contents' ].index( - trans, trans.security.encode_id( history_id ), - ids=( ','.join([ trans.security.encode_id( hda.id ) for hda in hdas ]) ) ) - return hda_json + return h.to_json_string( hdas ) %></%def> @@ -324,8 +314,8 @@ page_show_hidden = ${ 'true' if show_hidden == True else ( 'null' if show_hidden == None else 'false' ) }, user = ${ get_current_user() }, - history = ${ get_history( history.id ) }, - hdas = ${ get_hdas( history.id, datasets ) }; + history = ${ get_history( history ) }, + hdas = ${ get_hdas( datasets ) }; // add user data to history // i don't like this history+user relationship, but user authentication changes views/behaviour Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: 2 new changesets
by commits-noreply＠bitbucket.org 25 Mar '13

25 Mar '13

2 new commits in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/388648deecff/ Changeset: 388648deecff Branch: next-stable User: natefoo Date: 2013-03-25 23:22:11 Summary: Don't misuse PBS parameters in the sample job config. Affected #: 1 file diff -r 75ba4f814129bb24ef85feb5fde99c115a707d4d -r 388648deecffe1d5d3c46525aabd5b4d55549248 job_conf.xml.sample_advanced --- a/job_conf.xml.sample_advanced +++ b/job_conf.xml.sample_advanced @@ -29,7 +29,7 @@ <destination id="pbs" runner="pbs" tags="mycluster"/><destination id="pbs_longjobs" runner="pbs" tags="mycluster,longjobs"> - <param id="Execution_Time">72:00:00</param> + <param id="Resource_List">walltime=72:00:00</param></destination><destination id="remote_cluster" runner="drmaa" tags="longjobs"/><destination id="real_user_cluster" runner="drmaa"> https://bitbucket.org/galaxy/galaxy-central/commits/b437eece0231/ Changeset: b437eece0231 User: natefoo Date: 2013-03-25 23:22:25 Summary: merge next-stable Affected #: 1 file diff -r 7df0d7eb3a0670a299d571683daba26cbc692f0b -r b437eece023175a6ac8ea87f7a4dae6582ad7892 job_conf.xml.sample_advanced --- a/job_conf.xml.sample_advanced +++ b/job_conf.xml.sample_advanced @@ -29,7 +29,7 @@ <destination id="pbs" runner="pbs" tags="mycluster"/><destination id="pbs_longjobs" runner="pbs" tags="mycluster,longjobs"> - <param id="Execution_Time">72:00:00</param> + <param id="Resource_List">walltime=72:00:00</param></destination><destination id="remote_cluster" runner="drmaa" tags="longjobs"/><destination id="real_user_cluster" runner="drmaa"> Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: greg: Fix a few typos in my last commit.
by commits-noreply＠bitbucket.org 25 Mar '13

25 Mar '13

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/7df0d7eb3a06/ Changeset: 7df0d7eb3a06 User: greg Date: 2013-03-25 22:18:17 Summary: Fix a few typos in my last commit. Affected #: 3 files diff -r efe376544e2263ec8f9788d71d25dd2b4b49a39d -r 7df0d7eb3a0670a299d571683daba26cbc692f0b lib/galaxy/webapps/galaxy/api/tool_shed_repositories.py --- a/lib/galaxy/webapps/galaxy/api/tool_shed_repositories.py +++ b/lib/galaxy/webapps/galaxy/api/tool_shed_repositories.py @@ -168,7 +168,7 @@ shed_tool_conf = payload.get( 'shed_tool_conf', None ) tool_path = payload.get( 'tool_path', None ) tool_panel_section_id = payload.get( 'tool_panel_section_id', '' ) - if tool_panel_section_id is not None: + if tool_panel_section_id not in [ None, '' ]: tool_panel_section = trans.app.toolbox.tool_panel[ tool_panel_section_id ] else: tool_panel_section = '' diff -r efe376544e2263ec8f9788d71d25dd2b4b49a39d -r 7df0d7eb3a0670a299d571683daba26cbc692f0b scripts/api/install_repository_tools_into_new_tool_panel_section.py --- a/scripts/api/install_repository_tools_into_new_tool_panel_section.py +++ b/scripts/api/install_repository_tools_into_new_tool_panel_section.py @@ -4,10 +4,10 @@ valid tools, loading them into a new section of the Galaxy tool panel. The repository has no tool dependencies or repository dependencies, so only a single repository will be installed. -usage: ./install_repository_tools_into_existing_tool_panel_section <api_key <galaxy base url> tool_shed_url name owner changeset_revision new_tool_panel_section_label +usage: ./install_repository_tools_into_new_tool_panel_section <api_key <galaxy base url> tool_shed_url name owner changeset_revision new_tool_panel_section_label Here is a working example of how to use this script to install a repository from the test tool shed. -./install_repository_tools_into_existing_tool_panel_section.py <api key><galaxy base url>/api/tool_shed_repositories/new/install_repository_revision http://testtoolshed.g2.bx.psu.edu gregs_filter greg f28d5018f9cb From%20Test%20Tool%20Shed +./install_repository_tools_into_new_tool_panel_section.py <api key><galaxy base url>/api/tool_shed_repositories/new/install_repository_revision http://testtoolshed.g2.bx.psu.edu gregs_filter greg f28d5018f9cb From%20Test%20Tool%20Shed """ import os diff -r efe376544e2263ec8f9788d71d25dd2b4b49a39d -r 7df0d7eb3a0670a299d571683daba26cbc692f0b scripts/api/install_repository_with_repository_dependencies.py --- a/scripts/api/install_repository_with_repository_dependencies.py +++ b/scripts/api/install_repository_with_repository_dependencies.py @@ -4,10 +4,10 @@ repository dependencies, so multiple repositories will ultimately be installed. Since no Galaxy tool panel section information is used, all tools contained in the installed repositories will be loaded into the Galaxy tool panel outside of any sections. -usage: ./install_repository_tools_into_existing_tool_panel_section <api_key <galaxy base url> tool_shed_url name owner changeset_revision True +usage: ./install_repository_with_repository_dependencies.py <api_key <galaxy base url> tool_shed_url name owner changeset_revision True Here is a working example of how to use this script to install a repository from the test tool shed. -./install_repository_tools_into_existing_tool_panel_section.py <api key><galaxy base url>/api/tool_shed_repositories/new/install_repository_revision http://testtoolshed.g2.bx.psu.edu emboss_5 devteam 8ddad0c9a75a True +./install_repository_with_repository_dependencies.py <api key><galaxy base url>/api/tool_shed_repositories/new/install_repository_revision http://testtoolshed.g2.bx.psu.edu emboss_5 devteam 8ddad0c9a75a True """ import os @@ -32,4 +32,4 @@ print 'usage: %s key url tool_shed_url name owner changeset_revision install_repository_dependencies' % os.path.basename( sys.argv[ 0 ] ) sys.exit( 1 ) -submit( sys.argv[ 1 ], sys.argv[ 2 ], data ) \ No newline at end of file +submit( sys.argv[ 1 ], sys.argv[ 2 ], data ) Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: greg: Add baseline cupport for installing tool shed repositories from a specified tool shed into Galaxy via a combination of the tool shed API and the now enhanced Galaxy API. There are 3 new example scripts in ~/scripts/api that demonstrate how this works. The are named like: install_repository...
by commits-noreply＠bitbucket.org 25 Mar '13

25 Mar '13

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/efe376544e22/ Changeset: efe376544e22 User: greg Date: 2013-03-25 22:05:14 Summary: Add baseline cupport for installing tool shed repositories from a specified tool shed into Galaxy via a combination of the tool shed API and the now enhanced Galaxy API. There are 3 new example scripts in ~/scripts/api that demonstrate how this works. The are named like: install_repository... In this implementation, all tools contained in repositories will be loaded into the Galaxy tool panel. The user can specify the following when sending a POST to the new Galaxy API URL: /api/tool_shed_repositories/new/install_repository_revision 1) an existing tool panel section id for loading tools or a new tool panel section label in which case a new ToolPanelSection will be created for containing the tools. 2) if they want tool dependencies installed along with the repository. 3) if they want repository dependencies installed along with the repository 4) which defined shed-related tool panel config file to use for installed tools Affected #: 10 files diff -r e741400c6578993c3bfe5faf184dd56518a2360a -r efe376544e2263ec8f9788d71d25dd2b4b49a39d lib/galaxy/webapps/galaxy/api/tool_shed_repositories.py --- /dev/null +++ b/lib/galaxy/webapps/galaxy/api/tool_shed_repositories.py @@ -0,0 +1,324 @@ +import logging +import urllib2 +from galaxy.util import json +from galaxy import util +from galaxy import web +from galaxy.web.base.controller import BaseAPIController +from tool_shed.galaxy_install import repository_util +import tool_shed.util.shed_util_common as suc + +log = logging.getLogger( __name__ ) + +def default_tool_shed_repository_value_mapper( trans, tool_shed_repository ): + value_mapper={ 'id' : trans.security.encode_id( tool_shed_repository.id ), + 'error_message' : tool_shed_repository.error_message or '' } + return value_mapper + +def get_message_for_no_shed_tool_config(): + # This Galaxy instance is not configured with a shed-related tool panel configuration file. + message = 'The tool_config_file setting in universe_wsgi.ini must include at least one shed tool configuration file name with a <toolbox> ' + message += 'tag that includes a tool_path attribute value which is a directory relative to the Galaxy installation directory in order to ' + message += 'automatically install tools from a tool shed into Galaxy (e.g., the file name shed_tool_conf.xml whose <toolbox> tag is ' + message += '<toolbox tool_path="../shed_tools">). For details, see the "Installation of Galaxy tool shed repository tools into a local ' + message += 'Galaxy instance" section of the Galaxy tool shed wiki at http://wiki.galaxyproject.org/InstallingRepositoriesToGalaxy#' + message += 'Installing_Galaxy_tool_shed_repository_tools_into_a_local_Galaxy_instance.' + return message + +class ToolShedRepositoriesController( BaseAPIController ): + """RESTful controller for interactions with tool shed repositories.""" + + @web.expose_api + def index( self, trans, **kwd ): + """ + GET /api/tool_shed_repositories + Display a list of dictionaries containing information about installed tool shed repositories. + """ + # Example URL: http://localhost:8763/api/tool_shed_repositories + tool_shed_repository_dicts = [] + try: + query = trans.sa_session.query( trans.app.model.ToolShedRepository ) \ + .order_by( trans.app.model.ToolShedRepository.table.c.name ) \ + .all() + for tool_shed_repository in query: + tool_shed_repository_dict = tool_shed_repository.get_api_value( value_mapper=default_tool_shed_repository_value_mapper( trans, tool_shed_repository ) ) + tool_shed_repository_dict[ 'url' ] = web.url_for( controller='tool_shed_repositories', + action='show', + id=trans.security.encode_id( tool_shed_repository.id ) ) + tool_shed_repository_dicts.append( tool_shed_repository_dict ) + return tool_shed_repository_dicts + except Exception, e: + message = "Error in the tool_shed_repositories API in index: %s" % str( e ) + log.error( message, exc_info=True ) + trans.response.status = 500 + return message + + @web.expose_api + def show( self, trans, id, **kwd ): + """ + GET /api/tool_shed_repositories/{encoded_tool_shed_repsository_id} + Display a dictionary containing information about a specified tool_shed_repository. + + :param tool_shed_repository_id: the encoded id of the `ToolShedRepository` object + """ + # Example URL: http://localhost:8763/api/tool_shed_repositories/df7a1f0c02a5b08e + try: + tool_shed_repository = suc.get_tool_shed_repository_by_id( trans, id ) + tool_shed_repository_dict = tool_shed_repository.as_dict( value_mapper=default_tool_shed_repository_value_mapper( trans, tool_shed_repository ) ) + tool_shed_repository_dict[ 'url' ] = web.url_for( controller='tool_shed_repositories', + action='show', + id=trans.security.encode_id( tool_shed_repository.id ) ) + return tool_shed_repository_dict + except Exception, e: + message = "Error in tool_shed_repositories API in index: " + str( e ) + log.error( message, exc_info=True ) + trans.response.status = 500 + return message + + @web.expose_api + def install_repository_revision( self, trans, payload, **kwd ): + """ + POST /api/tool_shed_repositories/install_repository_revision + Install a specified repository revision from a specified tool shed into Galaxy. + + :param tool_shed_url: the base URL of the Tool Shed from which to install the Repository + :param name: the name of the Repository + :param owner: the owner of the Repository + :param changset_revision: the changset_revision of the RepositoryMetadata object associated with the Repository + :param key: the current Galaxy admin user's API key + :param new_tool_panel_section_label: optional label of a new section to be added to the Galaxy tool panel in which to load + tools contained in the Repository. Either this parameter must be an empty string or + the tool_panel_section_id parameter must be an empty string, as both cannot be used. + :param tool_panel_section_id: optional id of the Galaxy tool panel section in which to load tools contained in the Repository. + If not set, tools will be loaded outside of any sections in the tool panel. Either this + parameter must be an empty string or the tool_panel_section_id parameter must be an empty string, + as both cannot be used. + """ + # Get the information about the repository to be installed from the payload. + tool_shed_url = payload.get( 'tool_shed_url', '' ) + if not tool_shed_url: + raise HTTPBadRequest( detail="Missing required parameter 'tool_shed_url'." ) + name = payload.get( 'name', '' ) + if not name: + raise HTTPBadRequest( detail="Missing required parameter 'name'." ) + owner = payload.get( 'owner', '' ) + if not owner: + raise HTTPBadRequest( detail="Missing required parameter 'owner'." ) + changeset_revision = payload.get( 'changeset_revision', '' ) + if not changeset_revision: + raise HTTPBadRequest( detail="Missing required parameter 'changeset_revision'." ) + # Make sure this Galaxy instance is configured with a shed-related tool panel configuration file. + if not suc.have_shed_tool_conf_for_install( trans ): + message = get_message_for_no_shed_tool_config() + log.error( message, exc_info=True ) + trans.response.status = 500 + return dict( status='error', error=message ) + # Make sure the current user's API key proves he is an admin user in this Galaxy instance. + if not trans.user_is_admin(): + raise HTTPForbidden( detail='You are not authorized to install a tool shed repository into this Galaxy instance.' ) + # Keep track of all repositories that are installed - there may be more than one if repository dependencies are installed. + installed_tool_shed_repositories = [] + # Get all of the information necessary for installing the repository from the specified tool shed. + url = suc.url_join( tool_shed_url, + 'api/repositories/get_repository_revision_install_info?name=%s&owner=%s&changeset_revision=%s' % \ + ( name, owner, changeset_revision ) ) + try: + response = urllib2.urlopen( url ) + raw_text = response.read() + response.close() + except Exception, e: + message = "Error attempting to retrieve installation information from tool shed %s for revision %s of repository %s owned by %s: %s" % \ + ( str( tool_shed_url ), str( name ), str( owner ), str( changeset_revision ), str( e ) ) + log.error( message, exc_info=True ) + trans.response.status = 500 + return dict( status='error', error=message ) + if raw_text: + items = json.from_json_string( raw_text ) + repository_dict = items[ 0 ] + repository_revision_dict = items[ 1 ] + repo_info_dict = items[ 2 ] + else: + message = "Unable to retrieve installation information from tool shed %s for revision %s of repository %s owned by %s: %s" % \ + ( str( tool_shed_url ), str( name ), str( owner ), str( changeset_revision ) ) + log.error( message, exc_info=True ) + trans.response.status = 500 + return dict( status='error', error=message ) + repo_info_dicts = [ repo_info_dict ] + # Make sure the tool shed returned everything we need for installing the repository. + try: + has_repository_dependencies = repository_revision_dict[ 'has_repository_dependencies' ] + except: + raise HTTPBadRequest( detail="Missing required parameter 'has_repository_dependencies'." ) + try: + includes_tools = repository_revision_dict[ 'includes_tools' ] + except: + raise HTTPBadRequest( detail="Missing required parameter 'includes_tools'." ) + try: + includes_tool_dependencies = repository_revision_dict[ 'includes_tool_dependencies' ] + except: + raise HTTPBadRequest( detail="Missing required parameter 'includes_tool_dependencies'." ) + try: + includes_tools_for_display_in_tool_panel = repository_revision_dict[ 'includes_tools_for_display_in_tool_panel' ] + except: + raise HTTPBadRequest( detail="Missing required parameter 'includes_tools_for_display_in_tool_panel'." ) + # Get the information about the Galaxy components (e.g., tool pane section, tool config file, etc) that will contain the repository + # information. + install_repository_dependencies = payload.get( 'install_repository_dependencies', False ) + install_tool_dependencies = payload.get( 'install_tool_dependencies', False ) + new_tool_panel_section = payload.get( 'new_tool_panel_section_label', '' ) + shed_tool_conf = payload.get( 'shed_tool_conf', None ) + tool_path = payload.get( 'tool_path', None ) + tool_panel_section_id = payload.get( 'tool_panel_section_id', '' ) + if tool_panel_section_id is not None: + tool_panel_section = trans.app.toolbox.tool_panel[ tool_panel_section_id ] + else: + tool_panel_section = '' + if not shed_tool_conf or not tool_path: + # Pick a semi-random shed-related tool panel configuration file. + for shed_config_dict in trans.app.toolbox.shed_tool_confs: + if shed_config_dict[ 'config_filename' ] != trans.app.config.migrated_tools_config: + break + shed_tool_conf = shed_config_dict[ 'config_filename' ] + tool_path = shed_config_dict[ 'tool_path' ] + if not shed_tool_conf: + raise HTTPBadRequest( detail="Missing required parameter 'shed_tool_conf'." ) + if not tool_path: + raise HTTPBadRequest( detail="Missing required parameter 'tool_path'." ) + # Build the dictionary of information necessary for creating tool_shed_repository database records for each repository being installed. + installation_dict = dict( install_repository_dependencies=install_repository_dependencies, + new_tool_panel_section=new_tool_panel_section, + no_changes_checked=False, + reinstalling=False, + repo_info_dicts=repo_info_dicts, + tool_panel_section=tool_panel_section, + tool_path=tool_path, + tool_shed_url=tool_shed_url ) + # Create the tool_shed_repository database records and gather additional information for repository installation. + created_or_updated_tool_shed_repositories, tool_panel_section_keys, repo_info_dicts, filtered_repo_info_dicts, message = \ + repository_util.handle_tool_shed_repositories( trans, installation_dict, using_api=True ) + if message and len( repo_info_dicts ) == 1: + # We're attempting to install a single repository that has already been installed into this Galaxy instance. + log.error( message, exc_info=True ) + trans.response.status = 500 + return dict( status='error', error=message ) + if created_or_updated_tool_shed_repositories: + # Build the dictionary of information necessary for installing the repositories. + installation_dict = dict( created_or_updated_tool_shed_repositories=created_or_updated_tool_shed_repositories, + filtered_repo_info_dicts=filtered_repo_info_dicts, + has_repository_dependencies=has_repository_dependencies, + includes_tool_dependencies=includes_tool_dependencies, + includes_tools=includes_tools, + includes_tools_for_display_in_tool_panel=includes_tools_for_display_in_tool_panel, + install_repository_dependencies=install_repository_dependencies, + install_tool_dependencies=install_tool_dependencies, + message='', + new_tool_panel_section=new_tool_panel_section, + shed_tool_conf=shed_tool_conf, + status='done', + tool_panel_section=tool_panel_section, + tool_panel_section_keys=tool_panel_section_keys, + tool_path=tool_path, + tool_shed_url=tool_shed_url ) + # Prepare the repositories for installation. Even though this method receives a single combination of tool_shed_url, name, owner and + # changeset_revision, there may be multiple repositories for installation at this point because repository dependencies may have added + # additional repositories for installation along with the single specified repository. + encoded_kwd, query, tool_shed_repositories, encoded_repository_ids = repository_util.initiate_repository_installation( trans, installation_dict ) + # Install the repositories, keeping track of each one for later display. + for index, tool_shed_repository in enumerate( tool_shed_repositories ): + repo_info_dict = repo_info_dicts[ index ] + tool_panel_section_key = tool_panel_section_keys[ index ] + repository_util.install_tool_shed_repository( trans, + tool_shed_repository, + repo_info_dict, + tool_panel_section_key, + shed_tool_conf, + tool_path, + install_tool_dependencies, + reinstalling=False ) + tool_shed_repository_dict = tool_shed_repository.as_dict( value_mapper=default_tool_shed_repository_value_mapper( trans, tool_shed_repository ) ) + tool_shed_repository_dict[ 'url' ] = web.url_for( controller='tool_shed_repositories', + action='show', + id=trans.security.encode_id( tool_shed_repository.id ) ) + installed_tool_shed_repositories.append( tool_shed_repository_dict ) + else: + log.error( message, exc_info=True ) + trans.response.status = 500 + return dict( status='error', error=message ) + # Display the list of installed repositories. + return installed_tool_shed_repositories + + @web.expose_api + def install_repository_revisions( self, trans, payload, **kwd ): + """ + POST /api/tool_shed_repositories/install_repository_revisions + Install one or more specified repository revisions from one or more specified tool sheds into Galaxy. The received parameters + must be ordered lists so that positional values in tool_shed_urls, names, owners and changeset_revisions are associated. + + It's questionable whether this method is needed as the above method for installing a single repository can probably cover all + desired scenarios. We'll keep this one around just in case... + + :param tool_shed_urls: the base URLs of the Tool Sheds from which to install a specified Repository + :param names: the names of the Repositories to be installed + :param owners: the owners of the Repositories to be installed + :param changset_revisions: the changset_revisions of each RepositoryMetadata object associated with each Repository to be installed + :param key: the current Galaxy admin user's API key + :param new_tool_panel_section_label: optional label of a new section to be added to the Galaxy tool panel in which to load + tools contained in the Repository. Either this parameter must be an empty string or + the tool_panel_section_id parameter must be an empty string, as both cannot be used. + :param tool_panel_section_id: optional id of the Galaxy tool panel section in which to load tools contained in the Repository. + If not set, tools will be loaded outside of any sections in the tool panel. Either this + parameter must be an empty string or the tool_panel_section_id parameter must be an empty string, + as both cannot be used. + """ + if not suc.have_shed_tool_conf_for_install( trans ): + # This Galaxy instance is not configured with a shed-related tool panel configuration file. + message = get_message_for_no_shed_tool_config() + log.error( message, exc_info=True ) + trans.response.status = 500 + return dict( status='error', error=message ) + if not trans.user_is_admin(): + raise HTTPForbidden( detail='You are not authorized to install a tool shed repository into this Galaxy instance.' ) + # Get the information about all of the repositories to be installed. + tool_shed_urls = util.listify( payload.get( 'tool_shed_urls', '' ) ) + names = util.listify( payload.get( 'names', '' ) ) + owners = util.listify( payload.get( 'owners', '' ) ) + changeset_revisions = util.listify( payload.get( 'changeset_revisions', '' ) ) + num_specified_repositories = len( tool_shed_urls ) + if len( names ) != num_specified_repositories or \ + len( owners ) != num_specified_repositories or \ + len( changeset_revisions ) != num_specified_repositories: + message = 'Error in tool_shed_repositories API in install_repository_revisions: the received parameters must be ordered ' + message += 'lists so that positional values in tool_shed_urls, names, owners and changeset_revisions are associated.' + log.error( message, exc_info=True ) + trans.response.status = 500 + return dict( status='error', error=message ) + # Get the information about the Galaxy components (e.g., tool pane section, tool config file, etc) that will contain information + # about each of the repositories being installed. + # TODO: we may want to enhance this method to allow for each of the following to be associated with each repository instead of + # forcing all repositories to use the same settings. + install_repository_dependencies = payload.get( 'install_repository_dependencies', False ) + install_tool_dependencies = payload.get( 'install_tool_dependencies', False ) + new_tool_panel_section = payload.get( 'new_tool_panel_section_label', '' ) + shed_tool_conf = payload.get( 'shed_tool_conf', None ) + tool_path = payload.get( 'tool_path', None ) + tool_panel_section_id = payload.get( 'tool_panel_section_id', '' ) + all_installed_tool_shed_repositories = [] + for index, tool_shed_url in enumerate( tool_shed_urls ): + current_payload = {} + current_payload[ 'tool_shed_url' ] = tool_shed_url + current_payload[ 'name' ] = names[ index ] + current_payload[ 'owner' ] = owners[ index ] + current_payload[ 'changeset_revision' ] = changeset_revisions[ index ] + current_payload[ 'install_repository_dependencies' ] = install_repository_dependencies + current_payload[ 'install_tool_dependencies' ] = install_tool_dependencies + current_payload[ 'new_tool_panel_section' ] = new_tool_panel_section + current_payload[ 'shed_tool_conf' ] = shed_tool_conf + current_payload[ 'tool_path' ] = tool_path + current_payload[ 'tool_panel_section_id' ] = tool_panel_section_id + installed_tool_shed_repositories = self.install_repository_revision( trans, **current_payload ) + if isinstance( installed_tool_shed_repositories, dict ): + # We encountered an error. + return installed_tool_shed_repositories + elif isinstance( installed_tool_shed_repositories, list ): + all_installed_tool_shed_repositories.extend( installed_tool_shed_repositories ) + return all_installed_tool_shed_repositories + \ No newline at end of file diff -r e741400c6578993c3bfe5faf184dd56518a2360a -r efe376544e2263ec8f9788d71d25dd2b4b49a39d lib/galaxy/webapps/galaxy/buildapp.py --- a/lib/galaxy/webapps/galaxy/buildapp.py +++ b/lib/galaxy/webapps/galaxy/buildapp.py @@ -104,12 +104,6 @@ name_prefix='group_', path_prefix='/api/groups/:group_id', parent_resources=dict( member_name='group', collection_name='groups' ) ) - webapp.api_mapper.resource( 'content', - 'contents', - controller='tool_shed_repository_contents', - name_prefix='tool_shed_repository_', - path_prefix='/api/tool_shed_repositories/:tool_shed_repository_id', - parent_resources=dict( member_name='tool_shed_repository', collection_name='tool_shed_repositories' ) ) _add_item_tags_controller( webapp, name_prefix="history_content_", path_prefix='/api/histories/:history_id/contents/:history_content_id' ) @@ -142,7 +136,6 @@ webapp.api_mapper.resource( 'tool', 'tools', path_prefix='/api' ) webapp.api_mapper.resource_with_deleted( 'user', 'users', path_prefix='/api' ) webapp.api_mapper.resource( 'genome', 'genomes', path_prefix='/api' ) - webapp.api_mapper.resource( 'tool_shed_repository', 'tool_shed_repositories', path_prefix='/api' ) webapp.api_mapper.resource( 'visualization', 'visualizations', path_prefix='/api' ) webapp.api_mapper.resource( 'workflow', 'workflows', path_prefix='/api' ) webapp.api_mapper.resource_with_deleted( 'history', 'histories', path_prefix='/api' ) @@ -155,7 +148,14 @@ webapp.api_mapper.connect("workflow_dict", '/api/workflows/{workflow_id}/download', controller='workflows', action='workflow_dict', conditions=dict(method=['GET'])) # Preserve the following download route for now for dependent applications -- deprecate at some point webapp.api_mapper.connect("workflow_dict", '/api/workflows/download/{workflow_id}', controller='workflows', action='workflow_dict', conditions=dict(method=['GET'])) - + # Galaxy API for tool shed features. + webapp.api_mapper.resource( 'tool_shed_repository', + 'tool_shed_repositories', + controller='tool_shed_repositories', + name_prefix='tool_shed_repository_', + path_prefix='/api', + new={ 'install_repository_revision' : 'POST' }, + parent_resources=dict( member_name='tool_shed_repository', collection_name='tool_shed_repositories' ) ) # Connect logger from app if app.trace_logger: webapp.trace_logger = app.trace_logger diff -r e741400c6578993c3bfe5faf184dd56518a2360a -r efe376544e2263ec8f9788d71d25dd2b4b49a39d lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py --- a/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py +++ b/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py @@ -846,7 +846,7 @@ tool_path=tool_path, tool_shed_url=tool_shed_url ) created_or_updated_tool_shed_repositories, tool_panel_section_keys, repo_info_dicts, filtered_repo_info_dicts, message = \ - repository_util.handle_tool_shed_repositories( trans, installation_dict ) + repository_util.handle_tool_shed_repositories( trans, installation_dict, using_api=False ) if message and len( repo_info_dicts ) == 1: return trans.response.send_redirect( web.url_for( controller='admin_toolshed', action='browse_repositories', diff -r e741400c6578993c3bfe5faf184dd56518a2360a -r efe376544e2263ec8f9788d71d25dd2b4b49a39d lib/galaxy/webapps/tool_shed/api/repositories.py --- a/lib/galaxy/webapps/tool_shed/api/repositories.py +++ b/lib/galaxy/webapps/tool_shed/api/repositories.py @@ -1,9 +1,10 @@ import logging from galaxy.web.framework.helpers import time_ago -import tool_shed.util.shed_util_common as suc from galaxy import web from galaxy import util from galaxy.web.base.controller import BaseAPIController +import tool_shed.util.shed_util_common as suc +from tool_shed.galaxy_install import repository_util from galaxy import eggs import pkg_resources @@ -29,7 +30,7 @@ class RepositoriesController( BaseAPIController ): """RESTful controller for interactions with repositories in the Tool Shed.""" - @web.expose_api + @web.expose_api_anonymous def index( self, trans, deleted=False, **kwd ): """ GET /api/repositories @@ -56,7 +57,7 @@ trans.response.status = 500 return message - @web.expose_api + @web.expose_api_anonymous def show( self, trans, id, **kwd ): """ GET /api/repositories/{encoded_repository_id} @@ -78,17 +79,59 @@ trans.response.status = 500 return message - @web.expose_api - def get_repository_and_revision( self, trans, name, owner, changeset_revision, **kwd ): + @web.expose_api_anonymous + def get_repository_revision_install_info( self, trans, name, owner, changeset_revision, **kwd ): """ - GET /api/repository/get_repository_and_revision - Returns information about a repository revision in the Tool Shed. + GET /api/repository/get_repository_revision_install_info - :param name: the name of the Repository object - :param owner: the owner of the Repository object - :param changset_revision: the changset_revision of the RepositoryMetadata object associated with the Repository object + :param name: the name of the Repository + :param owner: the owner of the Repository + :param changset_revision: the changset_revision of the RepositoryMetadata object associated with the Repository + + Returns a list of the following dictionaries:: + - a dictionary defining the Repository. For example: + { + "deleted": false, + "deprecated": false, + "description": "add_column hello", + "id": "f9cad7b01a472135", + "long_description": "add_column hello", + "name": "add_column", + "owner": "test", + "private": false, + "times_downloaded": 6, + "url": "/api/repositories/f9cad7b01a472135", + "user_id": "f9cad7b01a472135" + } + - a dictionary defining the Repsoitory revision (RepositoryMetadata). For example: + { + "changeset_revision": "3a08cc21466f", + "downloadable": true, + "has_repository_dependencies": false, + "id": "f9cad7b01a472135", + "includes_datatypes": false, + "includes_tool_dependencies": false, + "includes_tools": true, + "includes_tools_for_display_in_tool_panel": true, + "includes_workflows": false, + "malicious": false, + "repository_id": "f9cad7b01a472135", + "url": "/api/repository_revisions/f9cad7b01a472135" + } + - a dictionary including the additional information required to install the repository. For example: + { + "add_column": [ + "add_column hello", + "http://test@localhost:9009/repos/test/add_column", + "3a08cc21466f", + "1", + "test", + {}, + {} + ] + } """ - # Example URL: http://localhost:9009/api/repositories/get_repository_and_revision?name=add… + # Example URL: http://localhost:9009/api/repositories/get_repository_revision_install_info… try: # Get the repository information. repository = suc.get_repository_by_name_and_owner( trans.app, name, owner ) @@ -106,6 +149,7 @@ repo = hg.repository( suc.get_configured_ui(), repo_dir ) new_changeset_revision = suc.get_next_downloadable_changeset_revision( repository, repo, changeset_revision ) repository_metadata = suc.get_repository_metadata_by_changeset_revision( trans, encoded_repository_id, new_changeset_revision ) + changeset_revision = new_changeset_revision if repository_metadata: encoded_repository_metadata_id = trans.security.encode_id( repository_metadata.id ) repository_metadata_dict = repository_metadata.get_api_value( view='collection', @@ -113,14 +157,17 @@ repository_metadata_dict[ 'url' ] = web.url_for( controller='repository_revisions', action='show', id=encoded_repository_metadata_id ) - return repository_dict, repository_metadata_dict + # Get the repo_info_dict for installing the repository. + repo_info_dict, includes_tools, includes_tool_dependencies, includes_tools_for_display_in_tool_panel, has_repository_dependencies = \ + repository_util.get_repo_info_dict( trans, encoded_repository_id, changeset_revision ) + return repository_dict, repository_metadata_dict, repo_info_dict else: message = "Unable to locate repository_metadata record for repository id %d and changeset_revision %s" % ( repository.id, changeset_revision ) log.error( message, exc_info=True ) trans.response.status = 500 - return repository_dict, {} + return repository_dict, {}, {} except Exception, e: - message = "Error in the Tool Shed repositories API in get_repository_and_revision: %s" % str( e ) + message = "Error in the Tool Shed repositories API in get_repository_revision_install_info: %s" % str( e ) log.error( message, exc_info=True ) trans.response.status = 500 return message diff -r e741400c6578993c3bfe5faf184dd56518a2360a -r efe376544e2263ec8f9788d71d25dd2b4b49a39d lib/galaxy/webapps/tool_shed/buildapp.py --- a/lib/galaxy/webapps/tool_shed/buildapp.py +++ b/lib/galaxy/webapps/tool_shed/buildapp.py @@ -77,7 +77,7 @@ webapp.api_mapper.resource( 'repository', 'repositories', controller='repositories', - collection={ 'get_repository_and_revision' : 'GET' }, + collection={ 'get_repository_revision_install_info' : 'GET' }, name_prefix='repository_', path_prefix='/api', parent_resources=dict( member_name='repository', collection_name='repositories' ) ) diff -r e741400c6578993c3bfe5faf184dd56518a2360a -r efe376544e2263ec8f9788d71d25dd2b4b49a39d lib/tool_shed/galaxy_install/repository_util.py --- a/lib/tool_shed/galaxy_install/repository_util.py +++ b/lib/tool_shed/galaxy_install/repository_util.py @@ -252,7 +252,7 @@ # Load proprietary datatype display applications trans.app.datatypes_registry.load_display_applications( installed_repository_dict=repository_dict ) -def handle_tool_shed_repositories( trans, installation_dict ): +def handle_tool_shed_repositories( trans, installation_dict, using_api=False ): # The following installation_dict entries are all required. install_repository_dependencies = installation_dict[ 'install_repository_dependencies' ] new_tool_panel_section = installation_dict[ 'new_tool_panel_section' ] @@ -272,7 +272,7 @@ no_changes_checked=no_changes_checked, tool_panel_section=tool_panel_section, new_tool_panel_section=new_tool_panel_section ) - if message and len( repo_info_dicts ) == 1: + if message and len( repo_info_dicts ) == 1 and not using_api: installed_tool_shed_repository = created_or_updated_tool_shed_repositories[ 0 ] message += 'Click <a href="%s">here</a> to manage the repository. ' % \ ( web.url_for( controller='admin_toolshed', action='manage_repository', id=trans.security.encode_id( installed_tool_shed_repository.id ) ) ) diff -r e741400c6578993c3bfe5faf184dd56518a2360a -r efe376544e2263ec8f9788d71d25dd2b4b49a39d scripts/api/common.py --- a/scripts/api/common.py +++ b/scripts/api/common.py @@ -1,4 +1,7 @@ -import os, sys, urllib, urllib2 +import logging +import os +import sys +import urllib2 new_path = [ os.path.join( os.path.dirname( __file__ ), '..', '..', 'lib' ) ] new_path.extend( sys.path[1:] ) @@ -15,6 +18,8 @@ from Crypto.Util.randpool import RandomPool from Crypto.Util import number +log = logging.getLogger( __name__ ) + def make_url( api_key, url, args=None ): # Adds the API Key to the URL if it's not already there. if args is None: diff -r e741400c6578993c3bfe5faf184dd56518a2360a -r efe376544e2263ec8f9788d71d25dd2b4b49a39d scripts/api/install_repository_tools_into_existing_tool_panel_section.py --- /dev/null +++ b/scripts/api/install_repository_tools_into_existing_tool_panel_section.py @@ -0,0 +1,40 @@ +#!/usr/bin/env python +""" +Install a specified repository revision from a specified tool shed into Galaxy. This example demonstrates installation of a repository that contains +valid tools, loading them into an existing section of the Galaxy tool panel. The repository has no tool dependencies or repository dependencies, so only +a single repository will be installed. + +This example requires a tool panel config file (e.g., tool_conf.xml, shed_tool_conf.xml, etc) to contain a tool panel section like the following: + +<section id="from_test_tool_shed" name="From Test Tool Shed" version=""> +</section> + +usage: ./install_repository_tools_into_existing_tool_panel_section <api_key <galaxy base url> tool_shed_url name owner changeset_revision tool_panel_section_id + +Here is a working example of how to use this script to install a repository from the test tool shed. +./install_repository_tools_into_existing_tool_panel_section.py <api key><galaxy base url>/api/tool_shed_repositories/new/install_repository_revision http://testtoolshed.g2.bx.psu.edu gregs_filter greg f28d5018f9cb from_test_tool_shed +""" + +import os +import sys +sys.path.insert( 0, os.path.dirname( __file__ ) ) +from common import submit + +try: + assert sys.argv[ 7 ] +except IndexError: + print 'usage: %s key url tool_shed_url name owner changeset_revision tool_panel_section_id' % os.path.basename( sys.argv[ 0 ] ) + sys.exit( 1 ) + +try: + data = {} + data[ 'tool_shed_url' ] = sys.argv[ 3 ] + data[ 'name' ] = sys.argv[ 4 ] + data[ 'owner' ] = sys.argv[ 5 ] + data[ 'changeset_revision' ] = sys.argv[ 6 ] + data[ 'tool_panel_section_id' ] = sys.argv[ 7 ] +except IndexError: + print 'usage: %s key url tool_shed_url name owner changeset_revision tool_panel_section_id' % os.path.basename( sys.argv[ 0 ] ) + sys.exit( 1 ) + +submit( sys.argv[ 1 ], sys.argv[ 2 ], data ) diff -r e741400c6578993c3bfe5faf184dd56518a2360a -r efe376544e2263ec8f9788d71d25dd2b4b49a39d scripts/api/install_repository_tools_into_new_tool_panel_section.py --- /dev/null +++ b/scripts/api/install_repository_tools_into_new_tool_panel_section.py @@ -0,0 +1,35 @@ +#!/usr/bin/env python +""" +Install a specified repository revision from a specified tool shed into Galaxy. This example demonstrates installation of a repository that contains +valid tools, loading them into a new section of the Galaxy tool panel. The repository has no tool dependencies or repository dependencies, so only +a single repository will be installed. + +usage: ./install_repository_tools_into_existing_tool_panel_section <api_key <galaxy base url> tool_shed_url name owner changeset_revision new_tool_panel_section_label + +Here is a working example of how to use this script to install a repository from the test tool shed. +./install_repository_tools_into_existing_tool_panel_section.py <api key><galaxy base url>/api/tool_shed_repositories/new/install_repository_revision http://testtoolshed.g2.bx.psu.edu gregs_filter greg f28d5018f9cb From%20Test%20Tool%20Shed +""" + +import os +import sys +sys.path.insert( 0, os.path.dirname( __file__ ) ) +from common import submit + +try: + assert sys.argv[ 7 ] +except IndexError: + print 'usage: %s key url tool_shed_url name owner changeset_revision tool_panel_section_id' % os.path.basename( sys.argv[ 0 ] ) + sys.exit( 1 ) + +try: + data = {} + data[ 'tool_shed_url' ] = sys.argv[ 3 ] + data[ 'name' ] = sys.argv[ 4 ] + data[ 'owner' ] = sys.argv[ 5 ] + data[ 'changeset_revision' ] = sys.argv[ 6 ] + data[ 'new_tool_panel_section_label' ] = sys.argv[ 7 ] +except IndexError: + print 'usage: %s key url tool_shed_url name owner changeset_revision new_tool_panel_section_label' % os.path.basename( sys.argv[ 0 ] ) + sys.exit( 1 ) + +submit( sys.argv[ 1 ], sys.argv[ 2 ], data ) diff -r e741400c6578993c3bfe5faf184dd56518a2360a -r efe376544e2263ec8f9788d71d25dd2b4b49a39d scripts/api/install_repository_with_repository_dependencies.py --- /dev/null +++ b/scripts/api/install_repository_with_repository_dependencies.py @@ -0,0 +1,35 @@ +#!/usr/bin/env python +""" +Install a specified repository revision from a specified tool shed into Galaxy. This example demonstrates installation of a repository that has +repository dependencies, so multiple repositories will ultimately be installed. Since no Galaxy tool panel section information is used, all tools +contained in the installed repositories will be loaded into the Galaxy tool panel outside of any sections. + +usage: ./install_repository_tools_into_existing_tool_panel_section <api_key <galaxy base url> tool_shed_url name owner changeset_revision True + +Here is a working example of how to use this script to install a repository from the test tool shed. +./install_repository_tools_into_existing_tool_panel_section.py <api key><galaxy base url>/api/tool_shed_repositories/new/install_repository_revision http://testtoolshed.g2.bx.psu.edu emboss_5 devteam 8ddad0c9a75a True +""" + +import os +import sys +sys.path.insert( 0, os.path.dirname( __file__ ) ) +from common import submit + +try: + assert sys.argv[ 7 ] +except IndexError: + print 'usage: %s key url tool_shed_url name owner changeset_revision install_repository_dependencies' % os.path.basename( sys.argv[ 0 ] ) + sys.exit( 1 ) + +try: + data = {} + data[ 'tool_shed_url' ] = sys.argv[ 3 ] + data[ 'name' ] = sys.argv[ 4 ] + data[ 'owner' ] = sys.argv[ 5 ] + data[ 'changeset_revision' ] = sys.argv[ 6 ] + data[ 'install_repository_dependencies' ] = sys.argv[ 7 ] +except IndexError: + print 'usage: %s key url tool_shed_url name owner changeset_revision install_repository_dependencies' % os.path.basename( sys.argv[ 0 ] ) + sys.exit( 1 ) + +submit( sys.argv[ 1 ], sys.argv[ 2 ], data ) \ No newline at end of file Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: james_taylor: biostar: bugfix for encoding data
by commits-noreply＠bitbucket.org 25 Mar '13

25 Mar '13

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/e741400c6578/ Changeset: e741400c6578 User: james_taylor Date: 2013-03-25 21:30:16 Summary: biostar: bugfix for encoding data Affected #: 1 file diff -r cf74f0879bfe111b9a6763dfaff715154d4e1693 -r e741400c6578993c3bfe5faf184dd56518a2360a lib/galaxy/webapps/galaxy/controllers/biostar.py --- a/lib/galaxy/webapps/galaxy/controllers/biostar.py +++ b/lib/galaxy/webapps/galaxy/controllers/biostar.py @@ -22,7 +22,7 @@ """ Encode data to send a question to Biostar """ - text = json.dumps(data) + text = json.to_json_string(data) text = base64.urlsafe_b64encode(text) digest = hmac.new(key, text).hexdigest() return text, digest Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: 2 new changesets
by commits-noreply＠bitbucket.org 25 Mar '13

25 Mar '13

2 new commits in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/75ba4f814129/ Changeset: 75ba4f814129 Branch: next-stable User: carlfeberhard Date: 2013-03-25 18:33:30 Summary: api/histories, show: capture job states Affected #: 2 files diff -r c82a139db1c98a990760e767da00fe7aeaa03d28 -r 75ba4f814129bb24ef85feb5fde99c115a707d4d lib/galaxy/web/base/controller.py --- a/lib/galaxy/web/base/controller.py +++ b/lib/galaxy/web/base/controller.py @@ -833,7 +833,6 @@ Returns a dictionary with state counts for history's HDAs. Key is a dataset state, value is the number of states in that count. """ - # Build query to get (state, count) pairs. cols_to_select = [ trans.app.model.Dataset.table.c.state, func.count( '*' ) ] from_obj = trans.app.model.HistoryDatasetAssociation.table.join( trans.app.model.Dataset.table ) @@ -864,6 +863,135 @@ return state_count_dict + def get_hda_summary_dicts( self, trans, history ): + """Returns a list of dictionaries containing summary information + for each HDA in the given history. + """ + hda_model = trans.model.HistoryDatasetAssociation + + # outer join with job output to get job_state or None + job_subq = ( trans.sa_session.query( + trans.model.Job.id.label( 'job_id' ), + trans.model.Job.state.label( 'job_state' ), + trans.model.JobToOutputDatasetAssociation.dataset_id.label( 'hda_id' ) ) + .join( trans.model.JobToOutputDatasetAssociation ) ).subquery() + + # get state, name, etc. + columns = ( hda_model.name, hda_model.hid, hda_model.id, hda_model.deleted, + trans.model.Dataset.state, + job_subq.c.job_state, job_subq.c.job_id ) + column_keys = [ "name", "hid", "id", "deleted", "state", "job_state", "job_id" ] + + query = ( trans.sa_session.query( *columns ) + .enable_eagerloads( False ) + .filter( hda_model.history == history ) + .join( trans.model.Dataset ) + .outerjoin(( job_subq, job_subq.c.hda_id == hda_model.id )) + .order_by( hda_model.hid ) ) + + # build dictionaries, adding history id and encoding all ids + hda_dicts = [] + for hda_tuple in query.all(): + hda_dict = dict( zip( column_keys, hda_tuple ) ) + #if hda_dict[ 'job_state' ] not in [ None, 'ok' ]: + # print hda_dict[ 'hid' ], hda_dict[ 'name' ], hda_dict[ 'job_state' ] + hda_dict[ 'history_id' ] = history.id + trans.security.encode_dict_ids( hda_dict ) + hda_dicts.append( hda_dict ) + return hda_dicts + + def _get_hda_state_summaries( self, trans, hda_dict_list ): + """Returns two dictionaries (in a tuple): state_counts and state_ids. + Each is keyed according to the possible hda states: + _counts contains a sum of the datasets in each state + _ids contains a list of the encoded ids for each hda in that state + + hda_dict_list should be a list of hda data in dictionary form. + """ + #TODO: doc to rst + # init counts, ids for each state + state_counts = {} + state_ids = {} + for key, state in trans.app.model.Dataset.states.items(): + state_counts[ state ] = 0 + state_ids[ state ] = [] + + for hda_dict in hda_dict_list: + item_state = hda_dict['state'] + if not hda_dict['deleted']: + state_counts[ item_state ] = state_counts[ item_state ] + 1 + # needs to return all ids (no deleted check) + state_ids[ item_state ].append( hda_dict['id'] ) + + return ( state_counts, state_ids ) + + def _get_history_state_from_hdas( self, trans, history, hda_state_counts ): + """Returns the history state based on the states of the HDAs it contains. + """ + states = trans.app.model.Dataset.states + + num_hdas = sum( hda_state_counts.values() ) + # (default to ERROR) + state = states.ERROR + if num_hdas == 0: + state = states.NEW + + else: + if( ( hda_state_counts[ states.RUNNING ] > 0 ) + or ( hda_state_counts[ states.SETTING_METADATA ] > 0 ) + or ( hda_state_counts[ states.UPLOAD ] > 0 ) ): + state = states.RUNNING + + elif hda_state_counts[ states.QUEUED ] > 0: + state = states.QUEUED + + elif( ( hda_state_counts[ states.ERROR ] > 0 ) + or ( hda_state_counts[ states.FAILED_METADATA ] > 0 ) ): + state = states.ERROR + + elif hda_state_counts[ states.OK ] == num_hdas: + state = states.OK + + return state + + def _are_jobs_still_running( self, trans, hda_summary_list ): + """Determine whether any jobs are running from the given + list of hda summary dictionaries. + """ + job_states = trans.model.Job.states + def is_job_running( job_state ): + return ( ( job_state == job_states.NEW ) + or( job_state == job_states.UPLOAD ) + or( job_state == job_states.WAITING ) + or( job_state == job_states.QUEUED ) + or( job_state == job_states.RUNNING ) ) + + return len( filter( lambda hda: is_job_running( hda['job_state'] ), hda_summary_list ) ) + + def get_history_dict( self, trans, history ): + """Returns history data in the form of a dictionary. + """ + history_dict = history.get_api_value( view='element', value_mapper={ 'id':trans.security.encode_id }) + + history_dict[ 'nice_size' ] = history.get_disk_size( nice_size=True ) + + #TODO: separate, move to annotation api, fill on the client + history_dict[ 'annotation' ] = history.get_item_annotation_str( trans.sa_session, trans.user, history ) + if not history_dict[ 'annotation' ]: + history_dict[ 'annotation' ] = '' + + #TODO: allow passing as arg + hda_summaries = self.get_hda_summary_dicts( trans, history ) + #TODO remove the following in v2 + ( state_counts, state_ids ) = self._get_hda_state_summaries( trans, hda_summaries ) + history_dict[ 'state_details' ] = state_counts + history_dict[ 'state_ids' ] = state_ids + history_dict[ 'state' ] = self._get_history_state_from_hdas( trans, history, state_counts ) + + history_dict[ 'jobs_running' ] = self._are_jobs_still_running( trans, hda_summaries ) + + return history_dict + class UsesFormDefinitionsMixin: """Mixin for controllers that use Galaxy form objects.""" diff -r c82a139db1c98a990760e767da00fe7aeaa03d28 -r 75ba4f814129bb24ef85feb5fde99c115a707d4d lib/galaxy/webapps/galaxy/api/histories.py --- a/lib/galaxy/webapps/galaxy/api/histories.py +++ b/lib/galaxy/webapps/galaxy/api/histories.py @@ -62,26 +62,6 @@ params = util.Params( kwd ) deleted = util.string_as_bool( deleted ) - states = trans.app.model.Dataset.states - - def get_dataset_state_summaries( datasets ): - # cycles through the history's datasets, building counts and id lists for each possible ds state - state_counts = {} - state_ids = {} - - # init counts, ids for each state - for key, state in states.items(): - state_counts[state] = 0 - state_ids[state] = [] - - # cycle through datasets saving each ds' state - for dataset in datasets: - item_state = dataset.state - if not dataset.deleted: - state_counts[ item_state ] = state_counts[ item_state ] + 1 - state_ids[ item_state ].append( trans.security.encode_id( dataset.id ) ) - return ( state_counts, state_ids ) - # try to load the history, by most_recently_used or the given id try: if history_id == "most_recently_used": @@ -94,42 +74,7 @@ history = self.get_history( trans, history_id, check_ownership=False, check_accessible=True, deleted=deleted ) - history_data = history.get_api_value( view='element', value_mapper={'id':trans.security.encode_id} ) - history_data[ 'nice_size' ] = history.get_disk_size( nice_size=True ) - - #TODO: separate, move to annotation api, fill on the client - history_data[ 'annotation' ] = history.get_item_annotation_str( trans.sa_session, trans.user, history ) - if not history_data[ 'annotation' ]: - history_data[ 'annotation' ] = '' - - # get the history state using the state summaries of it's datasets (default to ERROR) - num_sets = len([ hda.id for hda in history.datasets if not hda.deleted ]) - state = states.ERROR - - ( state_counts, state_ids ) = get_dataset_state_summaries( history.datasets ) - - if num_sets == 0: - state = states.NEW - - else: - if( ( state_counts[ states.RUNNING ] > 0 ) - or ( state_counts[ states.SETTING_METADATA ] > 0 ) - or ( state_counts[ states.UPLOAD ] > 0 ) ): - state = states.RUNNING - - elif state_counts[ states.QUEUED ] > 0: - state = states.QUEUED - - elif( ( state_counts[ states.ERROR ] > 0 ) - or ( state_counts[ states.FAILED_METADATA ] > 0 ) ): - state = states.ERROR - - elif state_counts[ states.OK ] == num_sets: - state = states.OK - - history_data[ 'state' ] = state - history_data[ 'state_details' ] = state_counts - history_data[ 'state_ids' ] = state_ids + history_data = self.get_history_dict( trans, history ) history_data[ 'contents_url' ] = url_for( 'history_contents', history_id=history_id ) except Exception, e: https://bitbucket.org/galaxy/galaxy-central/commits/cf74f0879bfe/ Changeset: cf74f0879bfe User: carlfeberhard Date: 2013-03-25 18:34:27 Summary: merge next-stable Affected #: 2 files diff -r 6e09f0398ddc0f85b0776f7c98d21a6b3458b1d6 -r cf74f0879bfe111b9a6763dfaff715154d4e1693 lib/galaxy/web/base/controller.py --- a/lib/galaxy/web/base/controller.py +++ b/lib/galaxy/web/base/controller.py @@ -833,7 +833,6 @@ Returns a dictionary with state counts for history's HDAs. Key is a dataset state, value is the number of states in that count. """ - # Build query to get (state, count) pairs. cols_to_select = [ trans.app.model.Dataset.table.c.state, func.count( '*' ) ] from_obj = trans.app.model.HistoryDatasetAssociation.table.join( trans.app.model.Dataset.table ) @@ -864,6 +863,135 @@ return state_count_dict + def get_hda_summary_dicts( self, trans, history ): + """Returns a list of dictionaries containing summary information + for each HDA in the given history. + """ + hda_model = trans.model.HistoryDatasetAssociation + + # outer join with job output to get job_state or None + job_subq = ( trans.sa_session.query( + trans.model.Job.id.label( 'job_id' ), + trans.model.Job.state.label( 'job_state' ), + trans.model.JobToOutputDatasetAssociation.dataset_id.label( 'hda_id' ) ) + .join( trans.model.JobToOutputDatasetAssociation ) ).subquery() + + # get state, name, etc. + columns = ( hda_model.name, hda_model.hid, hda_model.id, hda_model.deleted, + trans.model.Dataset.state, + job_subq.c.job_state, job_subq.c.job_id ) + column_keys = [ "name", "hid", "id", "deleted", "state", "job_state", "job_id" ] + + query = ( trans.sa_session.query( *columns ) + .enable_eagerloads( False ) + .filter( hda_model.history == history ) + .join( trans.model.Dataset ) + .outerjoin(( job_subq, job_subq.c.hda_id == hda_model.id )) + .order_by( hda_model.hid ) ) + + # build dictionaries, adding history id and encoding all ids + hda_dicts = [] + for hda_tuple in query.all(): + hda_dict = dict( zip( column_keys, hda_tuple ) ) + #if hda_dict[ 'job_state' ] not in [ None, 'ok' ]: + # print hda_dict[ 'hid' ], hda_dict[ 'name' ], hda_dict[ 'job_state' ] + hda_dict[ 'history_id' ] = history.id + trans.security.encode_dict_ids( hda_dict ) + hda_dicts.append( hda_dict ) + return hda_dicts + + def _get_hda_state_summaries( self, trans, hda_dict_list ): + """Returns two dictionaries (in a tuple): state_counts and state_ids. + Each is keyed according to the possible hda states: + _counts contains a sum of the datasets in each state + _ids contains a list of the encoded ids for each hda in that state + + hda_dict_list should be a list of hda data in dictionary form. + """ + #TODO: doc to rst + # init counts, ids for each state + state_counts = {} + state_ids = {} + for key, state in trans.app.model.Dataset.states.items(): + state_counts[ state ] = 0 + state_ids[ state ] = [] + + for hda_dict in hda_dict_list: + item_state = hda_dict['state'] + if not hda_dict['deleted']: + state_counts[ item_state ] = state_counts[ item_state ] + 1 + # needs to return all ids (no deleted check) + state_ids[ item_state ].append( hda_dict['id'] ) + + return ( state_counts, state_ids ) + + def _get_history_state_from_hdas( self, trans, history, hda_state_counts ): + """Returns the history state based on the states of the HDAs it contains. + """ + states = trans.app.model.Dataset.states + + num_hdas = sum( hda_state_counts.values() ) + # (default to ERROR) + state = states.ERROR + if num_hdas == 0: + state = states.NEW + + else: + if( ( hda_state_counts[ states.RUNNING ] > 0 ) + or ( hda_state_counts[ states.SETTING_METADATA ] > 0 ) + or ( hda_state_counts[ states.UPLOAD ] > 0 ) ): + state = states.RUNNING + + elif hda_state_counts[ states.QUEUED ] > 0: + state = states.QUEUED + + elif( ( hda_state_counts[ states.ERROR ] > 0 ) + or ( hda_state_counts[ states.FAILED_METADATA ] > 0 ) ): + state = states.ERROR + + elif hda_state_counts[ states.OK ] == num_hdas: + state = states.OK + + return state + + def _are_jobs_still_running( self, trans, hda_summary_list ): + """Determine whether any jobs are running from the given + list of hda summary dictionaries. + """ + job_states = trans.model.Job.states + def is_job_running( job_state ): + return ( ( job_state == job_states.NEW ) + or( job_state == job_states.UPLOAD ) + or( job_state == job_states.WAITING ) + or( job_state == job_states.QUEUED ) + or( job_state == job_states.RUNNING ) ) + + return len( filter( lambda hda: is_job_running( hda['job_state'] ), hda_summary_list ) ) + + def get_history_dict( self, trans, history ): + """Returns history data in the form of a dictionary. + """ + history_dict = history.get_api_value( view='element', value_mapper={ 'id':trans.security.encode_id }) + + history_dict[ 'nice_size' ] = history.get_disk_size( nice_size=True ) + + #TODO: separate, move to annotation api, fill on the client + history_dict[ 'annotation' ] = history.get_item_annotation_str( trans.sa_session, trans.user, history ) + if not history_dict[ 'annotation' ]: + history_dict[ 'annotation' ] = '' + + #TODO: allow passing as arg + hda_summaries = self.get_hda_summary_dicts( trans, history ) + #TODO remove the following in v2 + ( state_counts, state_ids ) = self._get_hda_state_summaries( trans, hda_summaries ) + history_dict[ 'state_details' ] = state_counts + history_dict[ 'state_ids' ] = state_ids + history_dict[ 'state' ] = self._get_history_state_from_hdas( trans, history, state_counts ) + + history_dict[ 'jobs_running' ] = self._are_jobs_still_running( trans, hda_summaries ) + + return history_dict + class UsesFormDefinitionsMixin: """Mixin for controllers that use Galaxy form objects.""" diff -r 6e09f0398ddc0f85b0776f7c98d21a6b3458b1d6 -r cf74f0879bfe111b9a6763dfaff715154d4e1693 lib/galaxy/webapps/galaxy/api/histories.py --- a/lib/galaxy/webapps/galaxy/api/histories.py +++ b/lib/galaxy/webapps/galaxy/api/histories.py @@ -62,26 +62,6 @@ params = util.Params( kwd ) deleted = util.string_as_bool( deleted ) - states = trans.app.model.Dataset.states - - def get_dataset_state_summaries( datasets ): - # cycles through the history's datasets, building counts and id lists for each possible ds state - state_counts = {} - state_ids = {} - - # init counts, ids for each state - for key, state in states.items(): - state_counts[state] = 0 - state_ids[state] = [] - - # cycle through datasets saving each ds' state - for dataset in datasets: - item_state = dataset.state - if not dataset.deleted: - state_counts[ item_state ] = state_counts[ item_state ] + 1 - state_ids[ item_state ].append( trans.security.encode_id( dataset.id ) ) - return ( state_counts, state_ids ) - # try to load the history, by most_recently_used or the given id try: if history_id == "most_recently_used": @@ -94,42 +74,7 @@ history = self.get_history( trans, history_id, check_ownership=False, check_accessible=True, deleted=deleted ) - history_data = history.get_api_value( view='element', value_mapper={'id':trans.security.encode_id} ) - history_data[ 'nice_size' ] = history.get_disk_size( nice_size=True ) - - #TODO: separate, move to annotation api, fill on the client - history_data[ 'annotation' ] = history.get_item_annotation_str( trans.sa_session, trans.user, history ) - if not history_data[ 'annotation' ]: - history_data[ 'annotation' ] = '' - - # get the history state using the state summaries of it's datasets (default to ERROR) - num_sets = len([ hda.id for hda in history.datasets if not hda.deleted ]) - state = states.ERROR - - ( state_counts, state_ids ) = get_dataset_state_summaries( history.datasets ) - - if num_sets == 0: - state = states.NEW - - else: - if( ( state_counts[ states.RUNNING ] > 0 ) - or ( state_counts[ states.SETTING_METADATA ] > 0 ) - or ( state_counts[ states.UPLOAD ] > 0 ) ): - state = states.RUNNING - - elif state_counts[ states.QUEUED ] > 0: - state = states.QUEUED - - elif( ( state_counts[ states.ERROR ] > 0 ) - or ( state_counts[ states.FAILED_METADATA ] > 0 ) ): - state = states.ERROR - - elif state_counts[ states.OK ] == num_sets: - state = states.OK - - history_data[ 'state' ] = state - history_data[ 'state_details' ] = state_counts - history_data[ 'state_ids' ] = state_ids + history_data = self.get_history_dict( trans, history ) history_data[ 'contents_url' ] = url_for( 'history_contents', history_id=history_id ) except Exception, e: Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: inithello: Specify return codes so the automated framework will more correctly report success/failure.
by commits-noreply＠bitbucket.org 25 Mar '13

25 Mar '13

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/6e09f0398ddc/ Changeset: 6e09f0398ddc User: inithello Date: 2013-03-25 17:05:17 Summary: Specify return codes so the automated framework will more correctly report success/failure. Affected #: 1 file diff -r 48ff47731eba48aededf1c608d7dd1e03dcd7d1f -r 6e09f0398ddc0f85b0776f7c98d21a6b3458b1d6 test/install_and_test_tool_shed_repositories/functional_tests.py --- a/test/install_and_test_tool_shed_repositories/functional_tests.py +++ b/test/install_and_test_tool_shed_repositories/functional_tests.py @@ -734,7 +734,7 @@ ( changeset_revision, name ) ) # Run the cleanup method. This removes tool functional test methods from the test_toolbox module and uninstalls the # repository using Twill. - execute_uninstall_method( repository_info_dict ) + success = execute_uninstall_method( repository_info_dict ) # Set the test_toolbox.toolbox module-level variable to the new app.toolbox. test_toolbox.toolbox = app.toolbox else: @@ -782,20 +782,23 @@ print "# %d repositories failed:" % len( repositories_failed ) show_summary_output( repositories_failed ) if repositories_failed_install: + # Set success to False so that the return code will not be 0. + success = False print '# ----------------------------------------------------------------------------------' print "# %d repositories not installed correctly:" % len( repositories_failed_install ) show_summary_output( repositories_failed_install ) print "####################################################################################" - - if success: + if repositories_tested > 0: + if success: + return 0 + else: + return 1 + else: return 0 - else: - return 1 if __name__ == "__main__": now = strftime( "%Y-%m-%d %H:%M:%S" ) print "####################################################################################" print "# %s - running repository installation and testing script." % now print "####################################################################################" - return_code = main() - sys.exit( return_code ) + sys.exit( main() ) Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: 2 new changesets
by commits-noreply＠bitbucket.org 25 Mar '13

25 Mar '13

2 new commits in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/fa23454cb097/ Changeset: fa23454cb097 User: jgoecks Date: 2013-03-25 14:37:17 Summary: Trackster: more flexible parsing of location string. Affected #: 1 file diff -r f6743b36ce0a54e49acf06d09e03d01584d55514 -r fa23454cb09791cf1765d314dac07668b765795a static/scripts/viz/trackster/tracks.js --- a/static/scripts/viz/trackster/tracks.js +++ b/static/scripts/viz/trackster/tracks.js @@ -1276,7 +1276,7 @@ view.reference_track.init(); } } - if (low !== undefined && high !== undefined) { + if (low && high) { view.low = Math.max(low, 0); view.high = Math.min(high, view.max_high); } @@ -1289,34 +1289,49 @@ view.request_redraw(); } }, + + /** + * Change viewing region to that denoted by string. General format of string is: + * + * <chrom>[ {separator}<start>[-<end>] ] + * + * where separator can be whitespace or a colon. Examples: + * + * chr22 + * chr1:100-200 + * chr7 89999 + * chr8 90000 990000 + */ go_to: function(str) { - // Preprocess str to remove spaces and commas. - str = str.replace(/ |,/g, ""); - - // Go to new location. - var view = this, - new_low, - new_high, - chrom_pos = str.split(":"), + // Remove commas. + str = str.replace(/,/g, ''); + + // Replace colons and hyphens with space for easy parsing. + str = str.replace(/:|\-/g, ' '); + + // Parse new location. + var chrom_pos = str.split(/\s+/), chrom = chrom_pos[0], - pos = chrom_pos[1]; - - if (pos !== undefined) { - try { - var pos_split = pos.split("-"); - new_low = parseInt(pos_split[0], 10); - new_high = parseInt(pos_split[1], 10); - } catch (e) { - return false; - } + new_low = (chrom_pos[1] ? parseInt(chrom_pos[1], 10) : null), + new_high = (chrom_pos[2] ? parseInt(chrom_pos[2], 10) : null); + + // If no new high, new_low is the position of focus, so adjust low, high + // accordingly. + if (!new_high) { + // HACK: max resolution is 30 bases,so adjust low, high accordingly. + new_low = new_low - 15; + new_high = new_low + 15; } - view.change_chrom(chrom, new_low, new_high); + + this.change_chrom(chrom, new_low, new_high); }, + move_fraction: function(fraction) { var view = this; var span = view.high - view.low; this.move_delta(fraction * span); }, + move_delta: function(delta_chrom) { // Update low, high. var view = this; https://bitbucket.org/galaxy/galaxy-central/commits/48ff47731eba/ Changeset: 48ff47731eba User: jgoecks Date: 2013-03-25 14:37:48 Summary: Automated merge. Affected #: 1 file diff -r fa23454cb09791cf1765d314dac07668b765795a -r 48ff47731eba48aededf1c608d7dd1e03dcd7d1f test/tool_shed/base/twilltestcase.py --- a/test/tool_shed/base/twilltestcase.py +++ b/test/tool_shed/base/twilltestcase.py @@ -454,9 +454,10 @@ def fill_review_form( self, review_contents_dict, strings_displayed=[], strings_not_displayed=[] ): kwd = dict() + changed = False for label, contents in review_contents_dict.items(): - strings_displayed.append( label ) if contents: + changed = True kwd[ '%s__ESEP__comment' % label ] = contents[ 'comment' ] kwd[ '%s__ESEP__rating' % label ] = contents[ 'rating' ] if 'private' in contents: @@ -464,8 +465,10 @@ kwd[ '%s__ESEP__approved' % label ] = contents[ 'approved' ] else: kwd[ '%s__ESEP__approved' % label ] = 'not_applicable' + self.check_for_strings( strings_displayed, strings_not_displayed ) self.submit_form( 1, 'Workflows__ESEP__review_button', **kwd ) - strings_displayed.append( 'Reviews were saved' ) + if changed: + strings_displayed.append( 'Reviews were saved' ) self.check_for_strings( strings_displayed, strings_not_displayed ) def galaxy_login( self, email='test(a)bx.psu.edu', password='testuser', username='admin-user', redirect='' ): Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: inithello: Update tests to reflect new default for component review approval status.
by commits-noreply＠bitbucket.org 25 Mar '13

25 Mar '13

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/9baf9d6ce3a5/ Changeset: 9baf9d6ce3a5 User: inithello Date: 2013-03-25 14:36:02 Summary: Update tests to reflect new default for component review approval status. Affected #: 1 file diff -r f6743b36ce0a54e49acf06d09e03d01584d55514 -r 9baf9d6ce3a5b32e3f14ddcdee5eae85e1138257 test/tool_shed/base/twilltestcase.py --- a/test/tool_shed/base/twilltestcase.py +++ b/test/tool_shed/base/twilltestcase.py @@ -454,9 +454,10 @@ def fill_review_form( self, review_contents_dict, strings_displayed=[], strings_not_displayed=[] ): kwd = dict() + changed = False for label, contents in review_contents_dict.items(): - strings_displayed.append( label ) if contents: + changed = True kwd[ '%s__ESEP__comment' % label ] = contents[ 'comment' ] kwd[ '%s__ESEP__rating' % label ] = contents[ 'rating' ] if 'private' in contents: @@ -464,8 +465,10 @@ kwd[ '%s__ESEP__approved' % label ] = contents[ 'approved' ] else: kwd[ '%s__ESEP__approved' % label ] = 'not_applicable' + self.check_for_strings( strings_displayed, strings_not_displayed ) self.submit_form( 1, 'Workflows__ESEP__review_button', **kwd ) - strings_displayed.append( 'Reviews were saved' ) + if changed: + strings_displayed.append( 'Reviews were saved' ) self.check_for_strings( strings_displayed, strings_not_displayed ) def galaxy_login( self, email='test(a)bx.psu.edu', password='testuser', username='admin-user', redirect='' ): Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: greg: Refactor several methods in Galaxy's admin_toolshed controller in preparation for supporting installing tool shed repositories via the tool shed and Galaxy APIs.
by commits-noreply＠bitbucket.org 23 Mar '13

23 Mar '13

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/f6743b36ce0a/ Changeset: f6743b36ce0a User: greg Date: 2013-03-23 15:44:46 Summary: Refactor several methods in Galaxy's admin_toolshed controller in preparation for supporting installing tool shed repositories via the tool shed and Galaxy APIs. Affected #: 5 files diff -r e0da441ad10c518658d8be665a0bf6152a6acae0 -r f6743b36ce0a54e49acf06d09e03d01584d55514 lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py --- a/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py +++ b/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py @@ -1,19 +1,35 @@ -import logging, os, shutil, tempfile, urllib2 +import logging +import os +import shutil +import urllib2 from admin import AdminGalaxy -from galaxy import web, util, eggs, tools -from galaxy.web.form_builder import SelectField, CheckboxField -from galaxy.web.framework.helpers import iff, grids +from galaxy import eggs +from galaxy import web +from galaxy import util +from galaxy.web.form_builder import CheckboxField +from galaxy.web.framework.helpers import grids +from galaxy.web.framework.helpers import iff from galaxy.util import json from galaxy.model.orm import or_ import tool_shed.util.shed_util_common as suc -from tool_shed.util import common_install_util, data_manager_util, datatype_util, encoding_util, metadata_util -from tool_shed.util import readme_util, repository_dependency_util, tool_dependency_util, tool_util, workflow_util +from tool_shed.util import common_install_util +from tool_shed.util import data_manager_util +from tool_shed.util import datatype_util +from tool_shed.util import encoding_util +from tool_shed.util import metadata_util +from tool_shed.util import readme_util +from tool_shed.util import repository_dependency_util +from tool_shed.util import tool_dependency_util +from tool_shed.util import tool_util +from tool_shed.util import workflow_util from tool_shed.galaxy_install import repository_util import tool_shed.galaxy_install.grids.admin_toolshed_grids as admin_toolshed_grids import pkg_resources eggs.require( 'mercurial' ) -from mercurial import hg, ui, commands +from mercurial import commands +from mercurial import hg +from mercurial import ui pkg_resources.require( 'elementtree' ) from elementtree import ElementTree @@ -520,96 +536,21 @@ """Install specified tool shed repositories.""" shed_tool_conf = kwd.get( 'shed_tool_conf', '' ) tool_path = kwd[ 'tool_path' ] - includes_tool_dependencies = util.string_as_bool( kwd[ 'includes_tool_dependencies' ] ) install_tool_dependencies = CheckboxField.is_checked( kwd.get( 'install_tool_dependencies', '' ) ) - # There must be a one-to-one mapping between items in the 3 lists:tool_shed_repositories, tool_panel_section_keys, repo_info_dicts. + # There must be a one-to-one mapping between items in the 3 lists: tool_shed_repositories, tool_panel_section_keys, repo_info_dicts. tool_panel_section_keys = util.listify( kwd[ 'tool_panel_section_keys' ] ) repo_info_dicts = util.listify( kwd[ 'repo_info_dicts' ] ) for index, tool_shed_repository in enumerate( tool_shed_repositories ): repo_info_dict = repo_info_dicts[ index ] tool_panel_section_key = tool_panel_section_keys[ index ] - if tool_panel_section_key: - tool_section = trans.app.toolbox.tool_panel[ tool_panel_section_key ] - else: - tool_section = None - if isinstance( repo_info_dict, basestring ): - repo_info_dict = encoding_util.tool_shed_decode( repo_info_dict ) - # Clone each repository to the configured location. - suc.update_tool_shed_repository_status( trans.app, tool_shed_repository, trans.model.ToolShedRepository.installation_status.CLONING ) - repo_info_tuple = repo_info_dict[ tool_shed_repository.name ] - description, repository_clone_url, changeset_revision, ctx_rev, repository_owner, repository_dependencies, tool_dependencies = repo_info_tuple - relative_clone_dir = suc.generate_tool_shed_repository_install_dir( repository_clone_url, tool_shed_repository.installed_changeset_revision ) - clone_dir = os.path.join( tool_path, relative_clone_dir ) - relative_install_dir = os.path.join( relative_clone_dir, tool_shed_repository.name ) - install_dir = os.path.join( tool_path, relative_install_dir ) - cloned_ok, error_message = suc.clone_repository( repository_clone_url, os.path.abspath( install_dir ), ctx_rev ) - if cloned_ok: - if reinstalling: - # Since we're reinstalling the repository we need to find the latest changeset revision to which is can be updated. - changeset_revision_dict = repository_util.get_update_to_changeset_revision_and_ctx_rev( trans, tool_shed_repository ) - current_changeset_revision = changeset_revision_dict.get( 'changeset_revision', None ) - current_ctx_rev = changeset_revision_dict.get( 'ctx_rev', None ) - if current_ctx_rev != ctx_rev: - repo = hg.repository( suc.get_configured_ui(), path=os.path.abspath( install_dir ) ) - repository_util.pull_repository( repo, repository_clone_url, current_changeset_revision ) - suc.update_repository( repo, ctx_rev=current_ctx_rev ) - repository_util.handle_repository_contents( trans, - tool_shed_repository=tool_shed_repository, - tool_path=tool_path, - repository_clone_url=repository_clone_url, - relative_install_dir=relative_install_dir, - tool_shed=tool_shed_repository.tool_shed, - tool_section=tool_section, - shed_tool_conf=shed_tool_conf, - reinstalling=reinstalling ) - trans.sa_session.refresh( tool_shed_repository ) - metadata = tool_shed_repository.metadata - if 'tools' in metadata: - # Get the tool_versions from the tool shed for each tool in the installed change set. - suc.update_tool_shed_repository_status( trans.app, - tool_shed_repository, - trans.model.ToolShedRepository.installation_status.SETTING_TOOL_VERSIONS ) - tool_shed_url = suc.get_url_from_tool_shed( trans.app, tool_shed_repository.tool_shed ) - url = suc.url_join( tool_shed_url, - '/repository/get_tool_versions?name=%s&owner=%s&changeset_revision=%s' % \ - ( tool_shed_repository.name, tool_shed_repository.owner, tool_shed_repository.changeset_revision ) ) - response = urllib2.urlopen( url ) - text = response.read() - response.close() - if text: - tool_version_dicts = json.from_json_string( text ) - tool_util.handle_tool_versions( trans.app, tool_version_dicts, tool_shed_repository ) - else: - message += "Version information for the tools included in the %s repository is missing. " % name - message += "Reset all of this repository's metadata in the tool shed, then set the installed tool versions " - message += "from the installed repository's Repository Actions menu. " - status = 'error' - if install_tool_dependencies and tool_shed_repository.tool_dependencies and 'tool_dependencies' in metadata: - work_dir = tempfile.mkdtemp() - # Install tool dependencies. - suc.update_tool_shed_repository_status( trans.app, - tool_shed_repository, - trans.model.ToolShedRepository.installation_status.INSTALLING_TOOL_DEPENDENCIES ) - # Get the tool_dependencies.xml file from the repository. - tool_dependencies_config = suc.get_config_from_disk( 'tool_dependencies.xml', install_dir )#relative_install_dir ) - installed_tool_dependencies = common_install_util.handle_tool_dependencies( app=trans.app, - tool_shed_repository=tool_shed_repository, - tool_dependencies_config=tool_dependencies_config, - tool_dependencies=tool_shed_repository.tool_dependencies ) - try: - shutil.rmtree( work_dir ) - except: - pass - suc.update_tool_shed_repository_status( trans.app, tool_shed_repository, trans.model.ToolShedRepository.installation_status.INSTALLED ) - else: - # An error occurred while cloning the repository, so reset everything necessary to enable another attempt. - self.set_repository_attributes( trans, - tool_shed_repository, - status=trans.model.ToolShedRepository.installation_status.ERROR, - error_message=error_message, - deleted=False, - uninstalled=False, - remove_from_disk=True ) + repository_util.install_tool_shed_repository( trans, + tool_shed_repository, + repo_info_dict, + tool_panel_section_key, + shed_tool_conf, + tool_path, + install_tool_dependencies, + reinstalling=reinstalling ) tsr_ids_for_monitoring = [ trans.security.encode_id( tsr.id ) for tsr in tool_shed_repositories ] return trans.response.send_redirect( web.url_for( controller='admin_toolshed', action='monitor_repository_installation', @@ -844,7 +785,7 @@ @web.expose @web.require_admin def prepare_for_install( self, trans, **kwd ): - if not have_shed_tool_conf_for_install( trans ): + if not suc.have_shed_tool_conf_for_install( trans ): message = 'The tool_config_file setting in universe_wsgi.ini must include at least one shed tool configuration file name with a ' message += '<toolbox> tag that includes a tool_path attribute value which is a directory relative to the Galaxy installation ' message += 'directory in order to automatically install tools from a Galaxy tool shed (e.g., the file name shed_tool_conf.xml whose ' @@ -942,9 +883,9 @@ return trans.response.send_redirect( web.url_for( controller='admin_toolshed', action='manage_repositories', **kwd ) ) - shed_tool_conf_select_field = build_shed_tool_conf_select_field( trans ) + shed_tool_conf_select_field = tool_util.build_shed_tool_conf_select_field( trans ) tool_path = suc.get_tool_path_by_shed_tool_conf_filename( trans, shed_tool_conf ) - tool_panel_section_select_field = build_tool_panel_section_select_field( trans ) + tool_panel_section_select_field = tool_util.build_tool_panel_section_select_field( trans ) if len( repo_info_dicts ) == 1: # If we're installing a single repository, see if it contains a readme or dependencies that we can display. repo_info_dict = repo_info_dicts[ 0 ] @@ -1278,7 +1219,7 @@ original_section_name = '' else: original_section_name = '' - tool_panel_section_select_field = build_tool_panel_section_select_field( trans ) + tool_panel_section_select_field = tool_util.build_tool_panel_section_select_field( trans ) no_changes_check_box = CheckboxField( 'no_changes', checked=True ) if original_section_name: message += "The tools contained in your %s repository were last loaded into the tool panel section %s. " \ @@ -1294,7 +1235,7 @@ no_changes_check_box = None original_section_name = '' tool_panel_section_select_field = None - shed_tool_conf_select_field = build_shed_tool_conf_select_field( trans ) + shed_tool_conf_select_field = tool_util.build_shed_tool_conf_select_field( trans ) containers_dict = repository_util.populate_containers_dict_for_new_install( trans=trans, tool_shed_url=tool_shed_url, tool_path=tool_path, @@ -1398,13 +1339,13 @@ """An error occurred while cloning the repository, so reset everything necessary to enable another attempt.""" repository = suc.get_installed_tool_shed_repository( trans, kwd[ 'id' ] ) if kwd.get( 'reset_repository', False ): - self.set_repository_attributes( trans, - repository, - status=trans.model.ToolShedRepository.installation_status.NEW, - error_message=None, - deleted=False, - uninstalled=False, - remove_from_disk=True ) + suc.set_repository_attributes( trans, + repository, + status=trans.model.ToolShedRepository.installation_status.NEW, + error_message=None, + deleted=False, + uninstalled=False, + remove_from_disk=True ) new_kwd = {} new_kwd[ 'message' ] = "You can now attempt to install the repository named %s again." % repository.name new_kwd[ 'status' ] = "done" @@ -1415,20 +1356,6 @@ action='manage_repository', **kwd ) ) - def set_repository_attributes( self, trans, repository, status, error_message, deleted, uninstalled, remove_from_disk=False ): - if remove_from_disk: - relative_install_dir = repository.repo_path( trans.app ) - if relative_install_dir: - clone_dir = os.path.abspath( relative_install_dir ) - shutil.rmtree( clone_dir ) - log.debug( "Removed repository installation directory: %s" % str( clone_dir ) ) - repository.error_message = error_message - repository.status = status - repository.deleted = deleted - repository.uninstalled = uninstalled - trans.sa_session.add( repository ) - trans.sa_session.flush() - @web.expose @web.require_admin def set_tool_versions( self, trans, **kwd ): @@ -1680,46 +1607,3 @@ metadata=metadata, message=message, status=status ) - -## ---- Utility methods ------------------------------------------------------- - -def build_shed_tool_conf_select_field( trans ): - """Build a SelectField whose options are the keys in trans.app.toolbox.shed_tool_confs.""" - options = [] - for shed_tool_conf_dict in trans.app.toolbox.shed_tool_confs: - shed_tool_conf_filename = shed_tool_conf_dict[ 'config_filename' ] - if shed_tool_conf_filename != trans.app.config.migrated_tools_config: - if shed_tool_conf_filename.startswith( './' ): - option_label = shed_tool_conf_filename.replace( './', '', 1 ) - else: - option_label = shed_tool_conf_filename - options.append( ( option_label, shed_tool_conf_filename ) ) - select_field = SelectField( name='shed_tool_conf' ) - for option_tup in options: - select_field.add_option( option_tup[0], option_tup[1] ) - return select_field - -def build_tool_panel_section_select_field( trans ): - """Build a SelectField whose options are the sections of the current in-memory toolbox.""" - options = [] - for k, v in trans.app.toolbox.tool_panel.items(): - if isinstance( v, tools.ToolSection ): - options.append( ( v.name, v.id ) ) - select_field = SelectField( name='tool_panel_section', display='radio' ) - for option_tup in options: - select_field.add_option( option_tup[0], option_tup[1] ) - return select_field - -def can_select_tool_panel_section(): - pass - -def have_shed_tool_conf_for_install( trans ): - if not trans.app.toolbox.shed_tool_confs: - return False - migrated_tools_conf_path, migrated_tools_conf_name = os.path.split( trans.app.config.migrated_tools_config ) - for shed_tool_conf_dict in trans.app.toolbox.shed_tool_confs: - shed_tool_conf = shed_tool_conf_dict[ 'config_filename' ] - shed_tool_conf_path, shed_tool_conf_name = os.path.split( shed_tool_conf ) - if shed_tool_conf_name != migrated_tools_conf_name: - return True - return False diff -r e0da441ad10c518658d8be665a0bf6152a6acae0 -r f6743b36ce0a54e49acf06d09e03d01584d55514 lib/tool_shed/galaxy_install/repository_util.py --- a/lib/tool_shed/galaxy_install/repository_util.py +++ b/lib/tool_shed/galaxy_install/repository_util.py @@ -1,12 +1,16 @@ import logging import os +import shutil +import tempfile import threading import urllib2 from galaxy import tools +from galaxy.util import json from galaxy import web from galaxy.model.orm import or_ from galaxy.webapps.tool_shed.util import container_util import tool_shed.util.shed_util_common as suc +from tool_shed.util import common_install_util from tool_shed.util import data_manager_util from tool_shed.util import datatype_util from tool_shed.util import encoding_util @@ -63,7 +67,7 @@ all_repository_dependencies=None, handled_key_rd_dicts=None, circular_repository_dependencies=None ) - tool_dependencies = metadata.get( 'tool_dependencies', None ) + tool_dependencies = metadata.get( 'tool_dependencies', {} ) if tool_dependencies: new_tool_dependencies = {} for dependency_key, requirements_dict in tool_dependencies.items(): @@ -347,6 +351,91 @@ query = trans.sa_session.query( trans.model.ToolShedRepository ).filter( or_( *clause_list ) ) return encoded_kwd, query, tool_shed_repositories, encoded_repository_ids +def install_tool_shed_repository( trans, tool_shed_repository, repo_info_dict, tool_panel_section_key, shed_tool_conf, tool_path, install_tool_dependencies, + reinstalling=False ): + if tool_panel_section_key: + tool_section = trans.app.toolbox.tool_panel[ tool_panel_section_key ] + else: + tool_section = None + if isinstance( repo_info_dict, basestring ): + repo_info_dict = encoding_util.tool_shed_decode( repo_info_dict ) + # Clone each repository to the configured location. + suc.update_tool_shed_repository_status( trans.app, tool_shed_repository, trans.model.ToolShedRepository.installation_status.CLONING ) + repo_info_tuple = repo_info_dict[ tool_shed_repository.name ] + description, repository_clone_url, changeset_revision, ctx_rev, repository_owner, repository_dependencies, tool_dependencies = repo_info_tuple + relative_clone_dir = suc.generate_tool_shed_repository_install_dir( repository_clone_url, tool_shed_repository.installed_changeset_revision ) + clone_dir = os.path.join( tool_path, relative_clone_dir ) + relative_install_dir = os.path.join( relative_clone_dir, tool_shed_repository.name ) + install_dir = os.path.join( tool_path, relative_install_dir ) + cloned_ok, error_message = suc.clone_repository( repository_clone_url, os.path.abspath( install_dir ), ctx_rev ) + if cloned_ok: + if reinstalling: + # Since we're reinstalling the repository we need to find the latest changeset revision to which is can be updated. + changeset_revision_dict = get_update_to_changeset_revision_and_ctx_rev( trans, tool_shed_repository ) + current_changeset_revision = changeset_revision_dict.get( 'changeset_revision', None ) + current_ctx_rev = changeset_revision_dict.get( 'ctx_rev', None ) + if current_ctx_rev != ctx_rev: + repo = hg.repository( suc.get_configured_ui(), path=os.path.abspath( install_dir ) ) + pull_repository( repo, repository_clone_url, current_changeset_revision ) + suc.update_repository( repo, ctx_rev=current_ctx_rev ) + handle_repository_contents( trans, + tool_shed_repository=tool_shed_repository, + tool_path=tool_path, + repository_clone_url=repository_clone_url, + relative_install_dir=relative_install_dir, + tool_shed=tool_shed_repository.tool_shed, + tool_section=tool_section, + shed_tool_conf=shed_tool_conf, + reinstalling=reinstalling ) + trans.sa_session.refresh( tool_shed_repository ) + metadata = tool_shed_repository.metadata + if 'tools' in metadata: + # Get the tool_versions from the tool shed for each tool in the installed change set. + suc.update_tool_shed_repository_status( trans.app, + tool_shed_repository, + trans.model.ToolShedRepository.installation_status.SETTING_TOOL_VERSIONS ) + tool_shed_url = suc.get_url_from_tool_shed( trans.app, tool_shed_repository.tool_shed ) + url = suc.url_join( tool_shed_url, + '/repository/get_tool_versions?name=%s&owner=%s&changeset_revision=%s' % \ + ( tool_shed_repository.name, tool_shed_repository.owner, tool_shed_repository.changeset_revision ) ) + response = urllib2.urlopen( url ) + text = response.read() + response.close() + if text: + tool_version_dicts = json.from_json_string( text ) + tool_util.handle_tool_versions( trans.app, tool_version_dicts, tool_shed_repository ) + else: + message += "Version information for the tools included in the %s repository is missing. " % name + message += "Reset all of this repository's metadata in the tool shed, then set the installed tool versions " + message += "from the installed repository's Repository Actions menu. " + status = 'error' + if install_tool_dependencies and tool_shed_repository.tool_dependencies and 'tool_dependencies' in metadata: + work_dir = tempfile.mkdtemp() + # Install tool dependencies. + suc.update_tool_shed_repository_status( trans.app, + tool_shed_repository, + trans.model.ToolShedRepository.installation_status.INSTALLING_TOOL_DEPENDENCIES ) + # Get the tool_dependencies.xml file from the repository. + tool_dependencies_config = suc.get_config_from_disk( 'tool_dependencies.xml', install_dir )#relative_install_dir ) + installed_tool_dependencies = common_install_util.handle_tool_dependencies( app=trans.app, + tool_shed_repository=tool_shed_repository, + tool_dependencies_config=tool_dependencies_config, + tool_dependencies=tool_shed_repository.tool_dependencies ) + try: + shutil.rmtree( work_dir ) + except: + pass + suc.update_tool_shed_repository_status( trans.app, tool_shed_repository, trans.model.ToolShedRepository.installation_status.INSTALLED ) + else: + # An error occurred while cloning the repository, so reset everything necessary to enable another attempt. + suc.set_repository_attributes( trans, + tool_shed_repository, + status=trans.model.ToolShedRepository.installation_status.ERROR, + error_message=error_message, + deleted=False, + uninstalled=False, + remove_from_disk=True ) + def merge_containers_dicts_for_new_install( containers_dicts ): """ When installing one or more tool shed repositories for the first time, the received list of containers_dicts contains a containers_dict for diff -r e0da441ad10c518658d8be665a0bf6152a6acae0 -r f6743b36ce0a54e49acf06d09e03d01584d55514 lib/tool_shed/scripts/api/common.py --- /dev/null +++ b/lib/tool_shed/scripts/api/common.py @@ -0,0 +1,165 @@ +import os, sys, urllib, urllib2 + +new_path = [ os.path.join( os.path.dirname( __file__ ), '..', '..', '..', 'lib' ) ] +new_path.extend( sys.path[ 1: ] ) +sys.path = new_path + +from galaxy import eggs +import pkg_resources + +pkg_resources.require( "simplejson" ) +import simplejson + +pkg_resources.require( "pycrypto" ) +from Crypto.Cipher import Blowfish +from Crypto.Util.randpool import RandomPool +from Crypto.Util import number + +def encode_id( config_id_secret, obj_id ): + # Utility method to encode ID's + id_cipher = Blowfish.new( config_id_secret ) + # Convert to string + s = str( obj_id ) + # Pad to a multiple of 8 with leading "!" + s = ( "!" * ( 8 - len(s) % 8 ) ) + s + # Encrypt + return id_cipher.encrypt( s ).encode( 'hex' ) + +def delete( api_key, url, data, return_formatted=True ): + # Sends an API DELETE request and acts as a generic formatter for the JSON response - 'data' will become the JSON payload read by Galaxy. + try: + url = make_url( api_key, url ) + req = urllib2.Request( url, headers = { 'Content-Type': 'application/json' }, data = simplejson.dumps( data )) + req.get_method = lambda: 'DELETE' + r = simplejson.loads( urllib2.urlopen( req ).read() ) + except urllib2.HTTPError, e: + if return_formatted: + print e + print e.read( 1024 ) + sys.exit( 1 ) + else: + return 'Error. '+ str( e.read( 1024 ) ) + if not return_formatted: + return r + print 'Response' + print '--------' + print r + +def display( api_key, url, return_formatted=True ): + # Sends an API GET request and acts as a generic formatter for the JSON response. + try: + r = get( api_key, url ) + except urllib2.HTTPError, e: + print e + print e.read( 1024 ) # Only return the first 1K of errors. + sys.exit( 1 ) + if type( r ) == unicode: + print 'error: %s' % r + return None + if not return_formatted: + return r + elif type( r ) == list: + # Response is a collection as defined in the REST style. + print 'Collection Members' + print '------------------' + for n, i in enumerate(r): + # All collection members should have a name and url in the response. + print '#%d: %s' % (n+1, i.pop( 'url' ) ) + if 'name' in i: + print ' name: %s' % i.pop( 'name' ) + for k, v in i.items(): + print ' %s: %s' % ( k, v ) + print '' + print '%d element(s) in collection' % len( r ) + elif type( r ) == dict: + # Response is an element as defined in the REST style. + print 'Member Information' + print '------------------' + for k, v in r.items(): + print '%s: %s' % ( k, v ) + elif type( r ) == str: + print r + else: + print 'response is unknown type: %s' % type( r ) + +def get( api_key, url ): + # Do the actual GET. + url = make_url( api_key, url ) + try: + return simplejson.loads( urllib2.urlopen( url ).read() ) + except simplejson.decoder.JSONDecodeError, e: + print "URL did not return JSON data" + sys.exit(1) + +def make_url( api_key, url, args=None ): + # Adds the API Key to the URL if it's not already there. + if args is None: + args = [] + argsep = '&' + if '?' not in url: + argsep = '?' + if '?key=' not in url and '&key=' not in url: + args.insert( 0, ( 'key', api_key ) ) + return url + argsep + '&'.join( [ '='.join( t ) for t in args ] ) + +def post( api_key, url, data ): + # Do the actual POST. + url = make_url( api_key, url ) + req = urllib2.Request( url, headers = { 'Content-Type': 'application/json' }, data = simplejson.dumps( data ) ) + return simplejson.loads( urllib2.urlopen( req ).read() ) + +def put( api_key, url, data ): + # Do the actual PUT. + url = make_url( api_key, url ) + req = urllib2.Request( url, headers = { 'Content-Type': 'application/json' }, data = simplejson.dumps( data )) + req.get_method = lambda: 'PUT' + return simplejson.loads( urllib2.urlopen( req ).read() ) + +def submit( api_key, url, data, return_formatted=True ): + # Sends an API POST request and acts as a generic formatter for the JSON response - 'data' will become the JSON payload read by Galaxy. + try: + r = post( api_key, url, data ) + except urllib2.HTTPError, e: + if return_formatted: + print e + print e.read( 1024 ) + sys.exit( 1 ) + else: + return 'Error. '+ str( e.read( 1024 ) ) + if not return_formatted: + return r + print 'Response' + print '--------' + if type( r ) == list: + # Currently the only implemented responses are lists of dicts, because submission creates some number of collection elements. + for i in r: + if type( i ) == dict: + if 'url' in i: + print i.pop( 'url' ) + else: + print '----' + if 'name' in i: + print ' name: %s' % i.pop( 'name' ) + for k, v in i.items(): + print ' %s: %s' % ( k, v ) + else: + print i + else: + print r + +def update( api_key, url, data, return_formatted=True ): + # Sends an API PUT request and acts as a generic formatter for the JSON response - 'data' will become the JSON payload read by Galaxy. + try: + r = put( api_key, url, data ) + except urllib2.HTTPError, e: + if return_formatted: + print e + print e.read( 1024 ) + sys.exit( 1 ) + else: + return 'Error. '+ str( e.read( 1024 ) ) + if not return_formatted: + return r + print 'Response' + print '--------' + print r diff -r e0da441ad10c518658d8be665a0bf6152a6acae0 -r f6743b36ce0a54e49acf06d09e03d01584d55514 lib/tool_shed/util/shed_util_common.py --- a/lib/tool_shed/util/shed_util_common.py +++ b/lib/tool_shed/util/shed_util_common.py @@ -985,6 +985,17 @@ galaxy_url = trans.get_cookie( name='toolshedgalaxyurl' ) return galaxy_url +def have_shed_tool_conf_for_install( trans ): + if not trans.app.toolbox.shed_tool_confs: + return False + migrated_tools_conf_path, migrated_tools_conf_name = os.path.split( trans.app.config.migrated_tools_config ) + for shed_tool_conf_dict in trans.app.toolbox.shed_tool_confs: + shed_tool_conf = shed_tool_conf_dict[ 'config_filename' ] + shed_tool_conf_path, shed_tool_conf_name = os.path.split( shed_tool_conf ) + if shed_tool_conf_name != migrated_tools_conf_name: + return True + return False + def open_repository_files_folder( trans, folder_path ): """Return a list of dictionaries, each of which contains information for a file or directory contained within a directory in a repository file hierarchy.""" try: @@ -1072,6 +1083,20 @@ """Return a reversed list of changesets in the repository changelog up to and including the included_upper_bounds_changeset_revision.""" return reversed_lower_upper_bounded_changelog( repo, INITIAL_CHANGELOG_HASH, included_upper_bounds_changeset_revision ) +def set_repository_attributes( trans, repository, status, error_message, deleted, uninstalled, remove_from_disk=False ): + if remove_from_disk: + relative_install_dir = repository.repo_path( trans.app ) + if relative_install_dir: + clone_dir = os.path.abspath( relative_install_dir ) + shutil.rmtree( clone_dir ) + log.debug( "Removed repository installation directory: %s" % str( clone_dir ) ) + repository.error_message = error_message + repository.status = status + repository.deleted = deleted + repository.uninstalled = uninstalled + trans.sa_session.add( repository ) + trans.sa_session.flush() + def strip_path( fpath ): """Attempt to strip the path from a file name.""" if not fpath: diff -r e0da441ad10c518658d8be665a0bf6152a6acae0 -r f6743b36ce0a54e49acf06d09e03d01584d55514 lib/tool_shed/util/tool_util.py --- a/lib/tool_shed/util/tool_util.py +++ b/lib/tool_shed/util/tool_util.py @@ -1,12 +1,17 @@ -import filecmp, logging, os, shutil, tempfile -import tool_shed.util.shed_util_common as suc +import filecmp +import logging +import os +import shutil +import tempfile +import galaxy.tools from galaxy import util -import galaxy.tools -from galaxy.tools.search import ToolBoxSearch +from galaxy.datatypes import checkers from galaxy.model.orm import and_ -from galaxy.datatypes import checkers from galaxy.tools import parameters from galaxy.tools.parameters import dynamic_options +from galaxy.tools.search import ToolBoxSearch +from galaxy.web.form_builder import SelectField +import tool_shed.util.shed_util_common as suc from galaxy import eggs import pkg_resources @@ -79,6 +84,33 @@ app.toolbox.write_integrated_tool_panel_config_file() app.toolbox_search = ToolBoxSearch( app.toolbox ) +def build_shed_tool_conf_select_field( trans ): + """Build a SelectField whose options are the keys in trans.app.toolbox.shed_tool_confs.""" + options = [] + for shed_tool_conf_dict in trans.app.toolbox.shed_tool_confs: + shed_tool_conf_filename = shed_tool_conf_dict[ 'config_filename' ] + if shed_tool_conf_filename != trans.app.config.migrated_tools_config: + if shed_tool_conf_filename.startswith( './' ): + option_label = shed_tool_conf_filename.replace( './', '', 1 ) + else: + option_label = shed_tool_conf_filename + options.append( ( option_label, shed_tool_conf_filename ) ) + select_field = SelectField( name='shed_tool_conf' ) + for option_tup in options: + select_field.add_option( option_tup[ 0 ], option_tup[ 1 ] ) + return select_field + +def build_tool_panel_section_select_field( trans ): + """Build a SelectField whose options are the sections of the current in-memory toolbox.""" + options = [] + for k, v in trans.app.toolbox.tool_panel.items(): + if isinstance( v, galaxy.tools.ToolSection ): + options.append( ( v.name, v.id ) ) + select_field = SelectField( name='tool_panel_section', display='radio' ) + for option_tup in options: + select_field.add_option( option_tup[ 0 ], option_tup[ 1 ] ) + return select_field + def can_use_tool_config_disk_file( trans, repository, repo, file_path, changeset_revision ): """ Determine if repository's tool config file on disk can be used. This method is restricted to tool config files since, with the Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: 15 new changesets
by commits-noreply＠bitbucket.org 22 Mar '13

22 Mar '13

15 new commits in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/ca59c76da865/ Changeset: ca59c76da865 Branch: next-stable User: natefoo Date: 2013-03-20 17:55:52 Summary: Fix for processing Torque's 'E' state in the CLI runner. Affected #: 1 file diff -r 6a80f6558febba87a6e32b9533256543597e8d30 -r ca59c76da865a29e14dd77f9152d2ae46126f11d lib/galaxy/jobs/runners/cli_job/torque.py --- a/lib/galaxy/jobs/runners/cli_job/torque.py +++ b/lib/galaxy/jobs/runners/cli_job/torque.py @@ -128,5 +128,6 @@ return job_states.OK def __get_job_state(self, state): - return { 'R' : job_states.RUNNING, + return { 'E' : job_states.RUNNING, + 'R' : job_states.RUNNING, 'Q' : job_states.QUEUED }.get(state, state) https://bitbucket.org/galaxy/galaxy-central/commits/ea325de13398/ Changeset: ea325de13398 Branch: next-stable User: natefoo Date: 2013-03-20 17:56:13 Summary: Bugfixes for the pbs job runner and the move to job destinations. Affected #: 1 file diff -r ca59c76da865a29e14dd77f9152d2ae46126f11d -r ea325de133988fd36eeda175fd8df45081be4412 lib/galaxy/jobs/runners/pbs.py --- a/lib/galaxy/jobs/runners/pbs.py +++ b/lib/galaxy/jobs/runners/pbs.py @@ -141,6 +141,9 @@ def url_to_destination(self, url): """Convert a legacy URL to a job destination""" + if not url: + return + # Determine the the PBS server url_split = url.split("/") server = url_split[2] @@ -161,6 +164,9 @@ try: opts = url.split('/')[4].strip().lstrip('-').split(' -') assert opts != [''] + # stripping the - comes later (in parse_destination_params) + for i, opt in enumerate(opts): + opts[i] = '-' + opt except: opts = [] for opt in opts: @@ -210,6 +216,8 @@ return rval def __get_pbs_server(self, job_destination_params): + if job_destination_params is None: + return None return job_destination_params['destination'].split('@')[-1] def queue_job( self, job_wrapper ): @@ -600,9 +608,13 @@ try: pbs_server_name = self.__get_pbs_server( job.destination_params ) + if pbs_server_name is None: + log.debug("(%s) Job queued but no destination stored in job params, cannot delete" + % job_tag ) + return c = pbs.pbs_connect( pbs_server_name ) if c <= 0: - log.debug("%s Connection to PBS server for job delete failed" + log.debug("(%s) Connection to PBS server for job delete failed" % job_tag ) return pbs.pbs_deljob( c, job.get_job_runner_external_id(), '' ) https://bitbucket.org/galaxy/galaxy-central/commits/86668e4a72fc/ Changeset: 86668e4a72fc Branch: next-stable User: natefoo Date: 2013-03-20 18:11:18 Summary: Add cleanup when skipping a queued job that was deleted. Affected #: 1 file diff -r ea325de133988fd36eeda175fd8df45081be4412 -r 86668e4a72fc11d5bf0161e567b9c6c082560e63 lib/galaxy/jobs/runners/__init__.py --- a/lib/galaxy/jobs/runners/__init__.py +++ b/lib/galaxy/jobs/runners/__init__.py @@ -103,8 +103,14 @@ job_wrapper.is_ready = False # Make sure the job hasn't been deleted - if job_state != model.Job.states.QUEUED: + if job_state == model.Job.states.DELETED: + log.debug( "(%s) Job deleted by user before it entered the %s queue" % ( job_id, self.runner_name ) ) + if self.app.config.cleanup_job in ( "always", "onsuccess" ): + job_wrapper.cleanup() + return + elif job_state != model.Job.states.QUEUED: log.info( "(%d) Job is in state %s, skipping execution" % ( job_id, job_state ) ) + # cleanup may not be safe in all states return # Prepare the job https://bitbucket.org/galaxy/galaxy-central/commits/ae19f508eaa9/ Changeset: ae19f508eaa9 Branch: next-stable User: natefoo Date: 2013-03-20 18:11:45 Summary: Have the PBS runner use the superclass convenience method for pre-queueing sanity checks. Affected #: 1 file diff -r 86668e4a72fc11d5bf0161e567b9c6c082560e63 -r ae19f508eaa989a6d16ed59de1308ce8e05b0ad4 lib/galaxy/jobs/runners/pbs.py --- a/lib/galaxy/jobs/runners/pbs.py +++ b/lib/galaxy/jobs/runners/pbs.py @@ -222,29 +222,16 @@ def queue_job( self, job_wrapper ): """Create PBS script for a job and submit it to the PBS queue""" + # Superclass method has some basic sanity checks + super( LocalJobRunner, self ).queue_job( job_wrapper ) + if not job_wrapper.is_ready: + return - try: - job_wrapper.prepare() - command_line = self.build_command_line( job_wrapper, include_metadata=not( self.app.config.pbs_stage_path ) ) - except: - job_wrapper.fail( "failure preparing job", exception=True ) - log.exception("failure running job %d" % job_wrapper.job_id) - return + # command line has been added to the wrapper by the superclass queue_job() + command_line = job_wrapper.runner_command_line job_destination = job_wrapper.job_destination - # This is silly, why would we queue a job with no command line? - if not command_line: - job_wrapper.finish( '', '' ) - return - - # Check for deletion before we change state - if job_wrapper.get_state() == model.Job.states.DELETED: - log.debug( "Job %s deleted by user before it entered the PBS queue" % job_wrapper.job_id ) - if self.app.config.cleanup_job in ( "always", "onsuccess" ): - job_wrapper.cleanup() - return - # Determine the job's PBS destination (server/queue) and options from the job destination definition pbs_queue_name = None pbs_server_name = self.default_pbs_server https://bitbucket.org/galaxy/galaxy-central/commits/d0fa91fc995e/ Changeset: d0fa91fc995e Branch: next-stable User: natefoo Date: 2013-03-21 18:58:32 Summary: Standardize some job state properties common to asynchronous runners in AsynchronousJobState. Affected #: 1 file Diff not available. https://bitbucket.org/galaxy/galaxy-central/commits/c4535b8d46d5/ Changeset: c4535b8d46d5 Branch: next-stable User: natefoo Date: 2013-03-21 18:59:21 Summary: Convert the CLI runner to use job destinations. Affected #: 2 files Diff not available. https://bitbucket.org/galaxy/galaxy-central/commits/929fd34026bf/ Changeset: 929fd34026bf Branch: next-stable User: natefoo Date: 2013-03-22 18:11:57 Summary: Fix URL->destination conversion of jobs running prior to upgrade to destinations. Affected #: 1 file Diff not available. https://bitbucket.org/galaxy/galaxy-central/commits/b4da62a0e089/ Changeset: b4da62a0e089 Branch: next-stable User: natefoo Date: 2013-03-22 18:13:05 Summary: Refactor common operations in to AsynchronousJobSate/AsynchronousJobRunner. Affected #: 1 file Diff not available. https://bitbucket.org/galaxy/galaxy-central/commits/67c5a41aaaf4/ Changeset: 67c5a41aaaf4 Branch: next-stable User: natefoo Date: 2013-03-22 18:14:25 Summary: Convert drmaa runner from URLs to destinations. Affected #: 1 file Diff not available. https://bitbucket.org/galaxy/galaxy-central/commits/88c55e8ae98e/ Changeset: 88c55e8ae98e Branch: next-stable User: natefoo Date: 2013-03-22 18:14:50 Summary: Convert condor runner from URLs to destinations. Affected #: 1 file Diff not available. https://bitbucket.org/galaxy/galaxy-central/commits/eb4183e82817/ Changeset: eb4183e82817 Branch: next-stable User: natefoo Date: 2013-03-22 22:58:55 Summary: Additional CLI runner fixes for destinations. Affected #: 1 file Diff not available. https://bitbucket.org/galaxy/galaxy-central/commits/1ea94ed3afa2/ Changeset: 1ea94ed3afa2 Branch: next-stable User: natefoo Date: 2013-03-22 23:04:52 Summary: Don't finish jobs that are deleted. Affected #: 3 files Diff not available. https://bitbucket.org/galaxy/galaxy-central/commits/99b5a77e9840/ Changeset: 99b5a77e9840 Branch: next-stable User: natefoo Date: 2013-03-22 23:07:40 Summary: Additional examples for the job configuration sample. Affected #: 1 file Diff not available. https://bitbucket.org/galaxy/galaxy-central/commits/c82a139db1c9/ Changeset: c82a139db1c9 Branch: next-stable User: natefoo Date: 2013-03-22 23:08:45 Summary: Merge. Affected #: 42 files Diff not available. https://bitbucket.org/galaxy/galaxy-central/commits/e0da441ad10c/ Changeset: e0da441ad10c User: natefoo Date: 2013-03-22 23:09:33 Summary: Merge next-stable. Affected #: 8 files Diff not available. Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: greg: Add an expose_api_anonymous decorator to the Galaxy web framework to enable exposing API methods without requiring an API key or a Galaxy or Tool Shed user session.
by commits-noreply＠bitbucket.org 22 Mar '13

22 Mar '13

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/d1f25e3734f1/ Changeset: d1f25e3734f1 User: greg Date: 2013-03-22 21:26:07 Summary: Add an expose_api_anonymous decorator to the Galaxy web framework to enable exposing API methods without requiring an API key or a Galaxy or Tool Shed user session. Affected #: 2 files diff -r dab01cbc83dceddd9f02a48deb32c7ffb1817eba -r d1f25e3734f1e24cb4655758dbeaf44186f5af9a lib/galaxy/web/__init__.py --- a/lib/galaxy/web/__init__.py +++ b/lib/galaxy/web/__init__.py @@ -1,7 +1,16 @@ """ The Galaxy web application framework """ - -from framework import expose, json, json_pretty, require_login, require_admin, url_for, error, form, FormBuilder, expose_api, expose_api_raw +from framework import expose +from framework import json +from framework import json_pretty +from framework import require_login +from framework import require_admin +from framework import url_for +from framework import error +from framework import form +from framework import FormBuilder +from framework import expose_api +from framework import expose_api_anonymous +from framework import expose_api_raw from framework.base import httpexceptions - diff -r dab01cbc83dceddd9f02a48deb32c7ffb1817eba -r d1f25e3734f1e24cb4655758dbeaf44186f5af9a lib/galaxy/web/framework/__init__.py --- a/lib/galaxy/web/framework/__init__.py +++ b/lib/galaxy/web/framework/__init__.py @@ -106,7 +106,13 @@ """ return expose_api( func, to_json=False ) -def expose_api( func, to_json=True ): +def expose_api_anonymous( func, to_json=True ): + """ + Expose this function via the API but don't require an API key. + """ + return expose_api( func, to_json=to_json, key_required=False ) + +def expose_api( func, to_json=True, key_required=True ): @wraps(func) def decorator( self, trans, *args, **kwargs ): def error( environ, start_response ): @@ -114,7 +120,7 @@ return error_message error_status = '403 Forbidden' ## If there is a user, we've authenticated a session. - if not trans.user and isinstance(trans.galaxy_session, Bunch): + if key_required and not trans.user and isinstance( trans.galaxy_session, Bunch ): # If trans.user is already set, don't check for a key. # This happens when we're authenticating using session instead of an API key. # The Bunch clause is used to prevent the case where there's no user, but there is a real session. Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: greg: Extract several methods from Galaxy's admin_toolshed controller and the tool shed's repository controller and move them to ~/tool_shed/galaxy_install/repository_util in preparation for enhancing the Galaxy api to enable installing a repository from the tool shed.
by commits-noreply＠bitbucket.org 22 Mar '13

22 Mar '13

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/dab01cbc83dc/ Changeset: dab01cbc83dc User: greg Date: 2013-03-22 19:38:01 Summary: Extract several methods from Galaxy's admin_toolshed controller and the tool shed's repository controller and move them to ~/tool_shed/galaxy_install/repository_util in preparation for enhancing the Galaxy api to enable installing a repository from the tool shed. Affected #: 6 files diff -r b45e7e2cf125642efcbbcb112778a44f64de4ec2 -r dab01cbc83dceddd9f02a48deb32c7ffb1817eba lib/galaxy/model/__init__.py --- a/lib/galaxy/model/__init__.py +++ b/lib/galaxy/model/__init__.py @@ -3040,9 +3040,9 @@ pass class ToolShedRepository( object ): - api_collection_visible_keys = ( 'id', 'name', 'tool_shed', 'owner', 'installed_changeset_revision', 'changeset_revision', 'ctx_rev', 'includes_datatypes', + api_collection_visible_keys = ( 'id', 'tool_shed', 'name', 'owner', 'installed_changeset_revision', 'changeset_revision', 'ctx_rev', 'includes_datatypes', 'update_available', 'deleted', 'uninstalled', 'dist_to_shed', 'status', 'error_message' ) - api_element_visible_keys = ( 'id', 'name', 'tool_shed', 'owner', 'installed_changeset_revision', 'changeset_revision', 'ctx_rev', 'includes_datatypes', + api_element_visible_keys = ( 'id', 'tool_shed', 'name', 'owner', 'installed_changeset_revision', 'changeset_revision', 'ctx_rev', 'includes_datatypes', 'update_available', 'deleted', 'uninstalled', 'dist_to_shed', 'status', 'error_message' ) installation_status = Bunch( NEW='New', CLONING='Cloning', @@ -3079,10 +3079,8 @@ self.dist_to_shed = dist_to_shed self.status = status self.error_message = error_message - def as_dict( self, trans ): - tsr_dict = self.get_api_value( view='element' ) - tsr_dict[ 'id' ] = trans.security.encode_id( self.id ) - return tsr_dict + def as_dict( self, value_mapper=None ): + return self.get_api_value( view='element', value_mapper=value_mapper ) def repo_files_directory( self, app ): repo_path = self.repo_path( app ) if repo_path: @@ -3182,7 +3180,7 @@ try: rval[ key ] = self.__getattribute__( key ) if key in value_mapper: - rval[ key ] = value_mapper.get( key )( rval[ key ] ) + rval[ key ] = value_mapper.get( key, rval[ key ] ) except AttributeError: rval[ key ] = None return rval diff -r b45e7e2cf125642efcbbcb112778a44f64de4ec2 -r dab01cbc83dceddd9f02a48deb32c7ffb1817eba lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py --- a/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py +++ b/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py @@ -373,91 +373,6 @@ tool_version = tool_util.get_tool_version( app, guid ) return tool_version.get_version_ids( app, reverse=True ) - def handle_repository_contents( self, trans, tool_shed_repository, tool_path, repository_clone_url, relative_install_dir, tool_shed=None, - tool_section=None, shed_tool_conf=None, reinstalling=False ): - """ - Generate the metadata for the installed tool shed repository, among other things. This method is called from Galaxy (never the tool shed) - when an admin is installing a new repository or reinstalling an uninstalled repository. - """ - shed_config_dict = trans.app.toolbox.get_shed_config_dict_by_filename( shed_tool_conf ) - metadata_dict, invalid_file_tups = metadata_util.generate_metadata_for_changeset_revision( app=trans.app, - repository=tool_shed_repository, - changeset_revision=tool_shed_repository.changeset_revision, - repository_clone_url=repository_clone_url, - shed_config_dict=shed_config_dict, - relative_install_dir=relative_install_dir, - repository_files_dir=None, - resetting_all_metadata_on_repository=False, - updating_installed_repository=False, - persist=True ) - tool_shed_repository.metadata = metadata_dict - trans.sa_session.add( tool_shed_repository ) - trans.sa_session.flush() - if 'tool_dependencies' in metadata_dict and not reinstalling: - tool_dependencies = tool_dependency_util.create_tool_dependency_objects( trans.app, tool_shed_repository, relative_install_dir, set_status=True ) - if 'tools' in metadata_dict: - tool_panel_dict = tool_util.generate_tool_panel_dict_for_new_install( metadata_dict[ 'tools' ], tool_section ) - sample_files = metadata_dict.get( 'sample_files', [] ) - tool_index_sample_files = tool_util.get_tool_index_sample_files( sample_files ) - tool_util.copy_sample_files( trans.app, tool_index_sample_files, tool_path=tool_path ) - sample_files_copied = [ str( s ) for s in tool_index_sample_files ] - repository_tools_tups = suc.get_repository_tools_tups( trans.app, metadata_dict ) - if repository_tools_tups: - # Handle missing data table entries for tool parameters that are dynamically generated select lists. - repository_tools_tups = tool_util.handle_missing_data_table_entry( trans.app, relative_install_dir, tool_path, repository_tools_tups ) - # Handle missing index files for tool parameters that are dynamically generated select lists. - repository_tools_tups, sample_files_copied = tool_util.handle_missing_index_file( trans.app, - tool_path, - sample_files, - repository_tools_tups, - sample_files_copied ) - # Copy remaining sample files included in the repository to the ~/tool-data directory of the local Galaxy instance. - tool_util.copy_sample_files( trans.app, sample_files, tool_path=tool_path, sample_files_copied=sample_files_copied ) - tool_util.add_to_tool_panel( app=trans.app, - repository_name=tool_shed_repository.name, - repository_clone_url=repository_clone_url, - changeset_revision=tool_shed_repository.installed_changeset_revision, - repository_tools_tups=repository_tools_tups, - owner=tool_shed_repository.owner, - shed_tool_conf=shed_tool_conf, - tool_panel_dict=tool_panel_dict, - new_install=True ) - if 'data_manager' in metadata_dict: - new_data_managers = data_manager_util.install_data_managers( trans.app, - trans.app.config.shed_data_manager_config_file, - metadata_dict, - shed_config_dict, - relative_install_dir, - tool_shed_repository, - repository_tools_tups ) - if 'datatypes' in metadata_dict: - tool_shed_repository.status = trans.model.ToolShedRepository.installation_status.LOADING_PROPRIETARY_DATATYPES - if not tool_shed_repository.includes_datatypes: - tool_shed_repository.includes_datatypes = True - trans.sa_session.add( tool_shed_repository ) - trans.sa_session.flush() - files_dir = relative_install_dir - if shed_config_dict.get( 'tool_path' ): - files_dir = os.path.join( shed_config_dict['tool_path'], files_dir ) - datatypes_config = suc.get_config_from_disk( 'datatypes_conf.xml', files_dir ) - # Load data types required by tools. - converter_path, display_path = datatype_util.alter_config_and_load_prorietary_datatypes( trans.app, datatypes_config, files_dir, override=False ) - if converter_path or display_path: - # Create a dictionary of tool shed repository related information. - repository_dict = datatype_util.create_repository_dict_for_proprietary_datatypes( tool_shed=tool_shed, - name=tool_shed_repository.name, - owner=tool_shed_repository.owner, - installed_changeset_revision=tool_shed_repository.installed_changeset_revision, - tool_dicts=metadata_dict.get( 'tools', [] ), - converter_path=converter_path, - display_path=display_path ) - if converter_path: - # Load proprietary datatype converters - trans.app.datatypes_registry.load_datatype_converters( trans.app.toolbox, installed_repository_dict=repository_dict ) - if display_path: - # Load proprietary datatype display applications - trans.app.datatypes_registry.load_display_applications( installed_repository_dict=repository_dict ) - @web.expose @web.require_admin def import_workflow( self, trans, workflow_name, repository_id, **kwd ): @@ -638,15 +553,15 @@ repo = hg.repository( suc.get_configured_ui(), path=os.path.abspath( install_dir ) ) repository_util.pull_repository( repo, repository_clone_url, current_changeset_revision ) suc.update_repository( repo, ctx_rev=current_ctx_rev ) - self.handle_repository_contents( trans, - tool_shed_repository=tool_shed_repository, - tool_path=tool_path, - repository_clone_url=repository_clone_url, - relative_install_dir=relative_install_dir, - tool_shed=tool_shed_repository.tool_shed, - tool_section=tool_section, - shed_tool_conf=shed_tool_conf, - reinstalling=reinstalling ) + repository_util.handle_repository_contents( trans, + tool_shed_repository=tool_shed_repository, + tool_path=tool_path, + repository_clone_url=repository_clone_url, + relative_install_dir=relative_install_dir, + tool_shed=tool_shed_repository.tool_shed, + tool_section=tool_section, + shed_tool_conf=shed_tool_conf, + reinstalling=reinstalling ) trans.sa_session.refresh( tool_shed_repository ) metadata = tool_shed_repository.metadata if 'tools' in metadata: @@ -956,7 +871,7 @@ install_tool_dependencies = kwd.get( 'install_tool_dependencies', '' ) encoded_repo_info_dicts = util.listify( kwd.get( 'encoded_repo_info_dicts', None ) ) if not encoded_repo_info_dicts: - # The request originated in the tool shed. + # The request originated in the tool shed via a tool search. repository_ids = kwd.get( 'repository_ids', None ) changeset_revisions = kwd.get( 'changeset_revisions', None ) # Get the information necessary to install each repository. @@ -981,78 +896,40 @@ else: install_tool_dependencies = False tool_path = suc.get_tool_path_by_shed_tool_conf_filename( trans, shed_tool_conf ) + installation_dict = dict( install_repository_dependencies=install_repository_dependencies, + new_tool_panel_section=new_tool_panel_section, + no_changes_checked=False, + reinstalling=False, + repo_info_dicts=repo_info_dicts, + tool_panel_section=tool_panel_section, + tool_path=tool_path, + tool_shed_url=tool_shed_url ) created_or_updated_tool_shed_repositories, tool_panel_section_keys, repo_info_dicts, filtered_repo_info_dicts, message = \ - repository_dependency_util.create_repository_dependency_objects( trans, - tool_path, - tool_shed_url, - repo_info_dicts, - reinstalling=False, - install_repository_dependencies=install_repository_dependencies, - no_changes_checked=False, - tool_panel_section=tool_panel_section, - new_tool_panel_section=new_tool_panel_section ) + repository_util.handle_tool_shed_repositories( trans, installation_dict ) if message and len( repo_info_dicts ) == 1: - installed_tool_shed_repository = created_or_updated_tool_shed_repositories[ 0 ] - message+= 'Click <a href="%s">here</a> to manage the repository. ' % \ - ( web.url_for( controller='admin_toolshed', action='manage_repository', id=trans.security.encode_id( installed_tool_shed_repository.id ) ) ) return trans.response.send_redirect( web.url_for( controller='admin_toolshed', action='browse_repositories', message=message, status='error' ) ) + if created_or_updated_tool_shed_repositories: - # Handle contained tools. - if includes_tools_for_display_in_tool_panel and ( new_tool_panel_section or tool_panel_section ): - if new_tool_panel_section: - section_id = new_tool_panel_section.lower().replace( ' ', '_' ) - tool_panel_section_key = 'section_%s' % str( section_id ) - if tool_panel_section_key in trans.app.toolbox.tool_panel: - # Appending a tool to an existing section in trans.app.toolbox.tool_panel - log.debug( "Appending to tool panel section: %s" % new_tool_panel_section ) - tool_section = trans.app.toolbox.tool_panel[ tool_panel_section_key ] - else: - # Appending a new section to trans.app.toolbox.tool_panel - log.debug( "Loading new tool panel section: %s" % new_tool_panel_section ) - elem = Element( 'section' ) - elem.attrib[ 'name' ] = new_tool_panel_section - elem.attrib[ 'id' ] = section_id - elem.attrib[ 'version' ] = '' - tool_section = tools.ToolSection( elem ) - trans.app.toolbox.tool_panel[ tool_panel_section_key ] = tool_section - else: - tool_panel_section_key = 'section_%s' % tool_panel_section - tool_section = trans.app.toolbox.tool_panel[ tool_panel_section_key ] - else: - tool_panel_section_key = None - tool_section = None - encoded_repository_ids = [ trans.security.encode_id( tsr.id ) for tsr in created_or_updated_tool_shed_repositories ] - # Create a one-to-one mapping of tool shed repository id and tool panel section key. All tools contained in the repositories - # being installed will be loaded into the same section in the tool panel. - for tsr in created_or_updated_tool_shed_repositories: - tool_panel_section_keys.append( tool_panel_section_key ) - new_kwd = dict( includes_tools=includes_tools, - includes_tools_for_display_in_tool_panel=includes_tools_for_display_in_tool_panel, - has_repository_dependencies=has_repository_dependencies, - install_repository_dependencies=install_repository_dependencies, - includes_tool_dependencies=includes_tool_dependencies, - install_tool_dependencies=install_tool_dependencies, - message=message, - repo_info_dicts=filtered_repo_info_dicts, - shed_tool_conf=shed_tool_conf, - status=status, - tool_path=tool_path, - tool_panel_section_keys=tool_panel_section_keys, - tool_shed_repository_ids=encoded_repository_ids, - tool_shed_url=tool_shed_url ) - encoded_kwd = encoding_util.tool_shed_encode( new_kwd ) - tsr_ids = [ r.id for r in created_or_updated_tool_shed_repositories ] - tool_shed_repositories = [] - for tsr_id in tsr_ids: - tsr = trans.sa_session.query( trans.model.ToolShedRepository ).get( tsr_id ) - tool_shed_repositories.append( tsr ) - clause_list = [] - for tsr_id in tsr_ids: - clause_list.append( trans.model.ToolShedRepository.table.c.id == tsr_id ) - query = trans.sa_session.query( trans.model.ToolShedRepository ).filter( or_( *clause_list ) ) + installation_dict = dict( created_or_updated_tool_shed_repositories=created_or_updated_tool_shed_repositories, + filtered_repo_info_dicts=filtered_repo_info_dicts, + has_repository_dependencies=has_repository_dependencies, + includes_tool_dependencies=includes_tool_dependencies, + includes_tools=includes_tools, + includes_tools_for_display_in_tool_panel=includes_tools_for_display_in_tool_panel, + install_repository_dependencies=install_repository_dependencies, + install_tool_dependencies=install_tool_dependencies, + message=message, + new_tool_panel_section=new_tool_panel_section, + shed_tool_conf=shed_tool_conf, + status=status, + tool_panel_section=tool_panel_section, + tool_panel_section_keys=tool_panel_section_keys, + tool_path=tool_path, + tool_shed_url=tool_shed_url ) + encoded_kwd, query, tool_shed_repositories, encoded_repository_ids = repository_util.initiate_repository_installation( trans, installation_dict ) return trans.fill_template( 'admin/tool_shed_repository/initiate_repository_installation.mako', encoded_kwd=encoded_kwd, query=query, diff -r b45e7e2cf125642efcbbcb112778a44f64de4ec2 -r dab01cbc83dceddd9f02a48deb32c7ffb1817eba lib/galaxy/webapps/tool_shed/controllers/repository.py --- a/lib/galaxy/webapps/tool_shed/controllers/repository.py +++ b/lib/galaxy/webapps/tool_shed/controllers/repository.py @@ -1,26 +1,47 @@ -import os, logging, re, tempfile, ConfigParser, string -from time import gmtime, strftime +import ConfigParser +import logging +import os +import re +import string +import tempfile +from time import gmtime +from time import strftime from datetime import date, datetime -from galaxy import util, web +from galaxy import util +from galaxy import web from galaxy.util.odict import odict from galaxy.web.base.controller import BaseUIController -from galaxy.web.form_builder import CheckboxField, build_select_field +from galaxy.web.form_builder import CheckboxField +from galaxy.web.form_builder import build_select_field from galaxy.webapps.tool_shed import model from galaxy.webapps.tool_shed.model import directory_hash_id from galaxy.web.framework.helpers import grids from galaxy.util import json -from galaxy.model.orm import and_, or_ +from galaxy.model.orm import and_ +from galaxy.model.orm import or_ import tool_shed.util.shed_util_common as suc -from tool_shed.util import encoding_util, metadata_util, readme_util, repository_dependency_util, review_util, tool_dependency_util, tool_util, workflow_util +from tool_shed.util import encoding_util +from tool_shed.util import metadata_util +from tool_shed.util import readme_util +from tool_shed.util import repository_dependency_util +from tool_shed.util import review_util +from tool_shed.util import tool_dependency_util +from tool_shed.util import tool_util +from tool_shed.util import workflow_util from tool_shed.galaxy_install import repository_util -from galaxy.webapps.tool_shed.util import common_util, container_util +from galaxy.webapps.tool_shed.util import common_util +from galaxy.webapps.tool_shed.util import container_util import galaxy.tools import tool_shed.grids.repository_grids as repository_grids import tool_shed.grids.util as grids_util from galaxy import eggs eggs.require('mercurial') -from mercurial import hg, ui, patch, commands + +from mercurial import commands +from mercurial import hg +from mercurial import patch +from mercurial import ui log = logging.getLogger( __name__ ) @@ -1124,35 +1145,17 @@ includes_tool_dependencies = False repo_info_dicts = [] for tup in zip( util.listify( repository_ids ), util.listify( changeset_revisions ) ): - repository_id, changeset_revision = tup - repository = suc.get_repository_in_tool_shed( trans, repository_id ) - repository_clone_url = suc.generate_clone_url_for_repository_in_tool_shed( trans, repository ) - repository_metadata = suc.get_repository_metadata_by_changeset_revision( trans, repository_id, changeset_revision ) - metadata = repository_metadata.metadata - if not includes_tools: - if 'tools' in metadata: - includes_tools = True - if not includes_tools_for_display_in_tool_panel: - includes_tools_for_display_in_tool_panel = repository_metadata.includes_tools_for_display_in_tool_panel - if not has_repository_dependencies: - if 'repository_dependencies' in metadata: - has_repository_dependencies = True - if not includes_tool_dependencies: - if 'tool_dependencies' in metadata: - includes_tool_dependencies = True - repo_dir = repository.repo_path( trans.app ) - repo = hg.repository( suc.get_configured_ui(), repo_dir ) - ctx = suc.get_changectx_for_changeset( repo, changeset_revision ) - repo_info_dict = repository_util.create_repo_info_dict( trans=trans, - repository_clone_url=repository_clone_url, - changeset_revision=changeset_revision, - ctx_rev=str( ctx.rev() ), - repository_owner=repository.user.username, - repository_name=repository.name, - repository=repository, - repository_metadata=repository_metadata, - tool_dependencies=None, - repository_dependencies=None ) + repository_id, changeset_revision = tup + repo_info_dict, cur_includes_tools, cur_includes_tool_dependencies, cur_includes_tools_for_display_in_tool_panel, cur_has_repository_dependencies = \ + repository_util.get_repo_info_dict( trans, repository_id, changeset_revision ) + if cur_has_repository_dependencies and not has_repository_dependencies: + has_repository_dependencies = True + if cur_includes_tools and not includes_tools: + includes_tools = True + if cur_includes_tool_dependencies and not includes_tool_dependencies: + includes_tool_dependencies = True + if cur_includes_tools_for_display_in_tool_panel and not includes_tools_for_display_in_tool_panel: + includes_tools_for_display_in_tool_panel = True repo_info_dicts.append( encoding_util.tool_shed_encode( repo_info_dict ) ) return dict( includes_tools=includes_tools, includes_tools_for_display_in_tool_panel=includes_tools_for_display_in_tool_panel, diff -r b45e7e2cf125642efcbbcb112778a44f64de4ec2 -r dab01cbc83dceddd9f02a48deb32c7ffb1817eba lib/galaxy/webapps/tool_shed/model/__init__.py --- a/lib/galaxy/webapps/tool_shed/model/__init__.py +++ b/lib/galaxy/webapps/tool_shed/model/__init__.py @@ -192,10 +192,10 @@ class RepositoryMetadata( object, APIItem ): api_collection_visible_keys = ( 'id', 'repository_id', 'changeset_revision', 'malicious', 'downloadable', 'has_repository_dependencies', 'includes_datatypes', - 'includes_tools', 'includes_tool_dependencies', 'includes_workflows' ) + 'includes_tools', 'includes_tool_dependencies', 'includes_tools_for_display_in_tool_panel', 'includes_workflows' ) api_element_visible_keys = ( 'id', 'repository_id', 'changeset_revision', 'malicious', 'downloadable', 'tools_functionally_correct', 'do_not_test', 'time_last_tested', 'tool_test_errors', 'has_repository_dependencies', 'includes_datatypes', 'includes_tools', - 'includes_tool_dependencies', 'includes_workflows' ) + 'includes_tool_dependencies', 'includes_tools_for_display_in_tool_panel', 'includes_workflows' ) def __init__( self, id=None, repository_id=None, changeset_revision=None, metadata=None, tool_versions=None, malicious=False, downloadable=False, tools_functionally_correct=False, do_not_test=False, time_last_tested=None, tool_test_errors=None, has_repository_dependencies=False, includes_datatypes=False, includes_tools=False, includes_tool_dependencies=False, includes_workflows=False ): diff -r b45e7e2cf125642efcbbcb112778a44f64de4ec2 -r dab01cbc83dceddd9f02a48deb32c7ffb1817eba lib/tool_shed/galaxy_install/repository_util.py --- a/lib/tool_shed/galaxy_install/repository_util.py +++ b/lib/tool_shed/galaxy_install/repository_util.py @@ -1,14 +1,31 @@ -import os, logging, threading, urllib2 -from galaxy.web import url_for +import logging +import os +import threading +import urllib2 +from galaxy import tools +from galaxy import web +from galaxy.model.orm import or_ from galaxy.webapps.tool_shed.util import container_util import tool_shed.util.shed_util_common as suc -from tool_shed.util import encoding_util, repository_dependency_util, tool_dependency_util, tool_util +from tool_shed.util import data_manager_util +from tool_shed.util import datatype_util +from tool_shed.util import encoding_util +from tool_shed.util import repository_dependency_util +from tool_shed.util import metadata_util +from tool_shed.util import tool_dependency_util +from tool_shed.util import tool_util from galaxy import eggs import pkg_resources pkg_resources.require( 'mercurial' ) -from mercurial import hg, ui, commands +from mercurial import commands +from mercurial import hg +from mercurial import ui + +pkg_resources.require( 'elementtree' ) +from elementtree import ElementTree +from elementtree.ElementTree import Element log = logging.getLogger( __name__ ) @@ -41,7 +58,7 @@ repository_dependency_util.get_repository_dependencies_for_changeset_revision( trans=trans, repository=repository, repository_metadata=repository_metadata, - toolshed_base_url=str( url_for( '/', qualified=True ) ).rstrip( '/' ), + toolshed_base_url=str( web.url_for( '/', qualified=True ) ).rstrip( '/' ), key_rd_dicts_to_be_processed=None, all_repository_dependencies=None, handled_key_rd_dicts=None, @@ -74,6 +91,39 @@ tool_dependencies ) return repo_info_dict +def get_repo_info_dict( trans, repository_id, changeset_revision ): + repository = suc.get_repository_in_tool_shed( trans, repository_id ) + repository_clone_url = suc.generate_clone_url_for_repository_in_tool_shed( trans, repository ) + repository_metadata = suc.get_repository_metadata_by_changeset_revision( trans, repository_id, changeset_revision ) + metadata = repository_metadata.metadata + if 'tools' in metadata: + includes_tools = True + else: + includes_tools = False + includes_tools_for_display_in_tool_panel = repository_metadata.includes_tools_for_display_in_tool_panel + if 'repository_dependencies' in metadata: + has_repository_dependencies = True + else: + has_repository_dependencies = False + if 'tool_dependencies' in metadata: + includes_tool_dependencies = True + else: + includes_tool_dependencies = False + repo_dir = repository.repo_path( trans.app ) + repo = hg.repository( suc.get_configured_ui(), repo_dir ) + ctx = suc.get_changectx_for_changeset( repo, changeset_revision ) + repo_info_dict = create_repo_info_dict( trans=trans, + repository_clone_url=repository_clone_url, + changeset_revision=changeset_revision, + ctx_rev=str( ctx.rev() ), + repository_owner=repository.user.username, + repository_name=repository.name, + repository=repository, + repository_metadata=repository_metadata, + tool_dependencies=None, + repository_dependencies=None ) + return repo_info_dict, includes_tools, includes_tool_dependencies, includes_tools_for_display_in_tool_panel, has_repository_dependencies + def get_update_to_changeset_revision_and_ctx_rev( trans, repository ): """Return the changeset revision hash to which the repository can be updated.""" changeset_revision_dict = {} @@ -117,6 +167,186 @@ changeset_revision_dict[ 'ctx_rev' ] = None return changeset_revision_dict +def handle_repository_contents( trans, tool_shed_repository, tool_path, repository_clone_url, relative_install_dir, tool_shed=None, tool_section=None, shed_tool_conf=None, + reinstalling=False ): + """ + Generate the metadata for the installed tool shed repository, among other things. This method is called from Galaxy (never the tool shed) + when an administrator is installing a new repository or reinstalling an uninstalled repository. + """ + shed_config_dict = trans.app.toolbox.get_shed_config_dict_by_filename( shed_tool_conf ) + metadata_dict, invalid_file_tups = metadata_util.generate_metadata_for_changeset_revision( app=trans.app, + repository=tool_shed_repository, + changeset_revision=tool_shed_repository.changeset_revision, + repository_clone_url=repository_clone_url, + shed_config_dict=shed_config_dict, + relative_install_dir=relative_install_dir, + repository_files_dir=None, + resetting_all_metadata_on_repository=False, + updating_installed_repository=False, + persist=True ) + tool_shed_repository.metadata = metadata_dict + trans.sa_session.add( tool_shed_repository ) + trans.sa_session.flush() + if 'tool_dependencies' in metadata_dict and not reinstalling: + tool_dependencies = tool_dependency_util.create_tool_dependency_objects( trans.app, tool_shed_repository, relative_install_dir, set_status=True ) + if 'tools' in metadata_dict: + tool_panel_dict = tool_util.generate_tool_panel_dict_for_new_install( metadata_dict[ 'tools' ], tool_section ) + sample_files = metadata_dict.get( 'sample_files', [] ) + tool_index_sample_files = tool_util.get_tool_index_sample_files( sample_files ) + tool_util.copy_sample_files( trans.app, tool_index_sample_files, tool_path=tool_path ) + sample_files_copied = [ str( s ) for s in tool_index_sample_files ] + repository_tools_tups = suc.get_repository_tools_tups( trans.app, metadata_dict ) + if repository_tools_tups: + # Handle missing data table entries for tool parameters that are dynamically generated select lists. + repository_tools_tups = tool_util.handle_missing_data_table_entry( trans.app, relative_install_dir, tool_path, repository_tools_tups ) + # Handle missing index files for tool parameters that are dynamically generated select lists. + repository_tools_tups, sample_files_copied = tool_util.handle_missing_index_file( trans.app, tool_path, sample_files, repository_tools_tups, sample_files_copied ) + # Copy remaining sample files included in the repository to the ~/tool-data directory of the local Galaxy instance. + tool_util.copy_sample_files( trans.app, sample_files, tool_path=tool_path, sample_files_copied=sample_files_copied ) + tool_util.add_to_tool_panel( app=trans.app, + repository_name=tool_shed_repository.name, + repository_clone_url=repository_clone_url, + changeset_revision=tool_shed_repository.installed_changeset_revision, + repository_tools_tups=repository_tools_tups, + owner=tool_shed_repository.owner, + shed_tool_conf=shed_tool_conf, + tool_panel_dict=tool_panel_dict, + new_install=True ) + if 'data_manager' in metadata_dict: + new_data_managers = data_manager_util.install_data_managers( trans.app, + trans.app.config.shed_data_manager_config_file, + metadata_dict, + shed_config_dict, + relative_install_dir, + tool_shed_repository, + repository_tools_tups ) + if 'datatypes' in metadata_dict: + tool_shed_repository.status = trans.model.ToolShedRepository.installation_status.LOADING_PROPRIETARY_DATATYPES + if not tool_shed_repository.includes_datatypes: + tool_shed_repository.includes_datatypes = True + trans.sa_session.add( tool_shed_repository ) + trans.sa_session.flush() + files_dir = relative_install_dir + if shed_config_dict.get( 'tool_path' ): + files_dir = os.path.join( shed_config_dict[ 'tool_path' ], files_dir ) + datatypes_config = suc.get_config_from_disk( 'datatypes_conf.xml', files_dir ) + # Load data types required by tools. + converter_path, display_path = datatype_util.alter_config_and_load_prorietary_datatypes( trans.app, datatypes_config, files_dir, override=False ) + if converter_path or display_path: + # Create a dictionary of tool shed repository related information. + repository_dict = datatype_util.create_repository_dict_for_proprietary_datatypes( tool_shed=tool_shed, + name=tool_shed_repository.name, + owner=tool_shed_repository.owner, + installed_changeset_revision=tool_shed_repository.installed_changeset_revision, + tool_dicts=metadata_dict.get( 'tools', [] ), + converter_path=converter_path, + display_path=display_path ) + if converter_path: + # Load proprietary datatype converters + trans.app.datatypes_registry.load_datatype_converters( trans.app.toolbox, installed_repository_dict=repository_dict ) + if display_path: + # Load proprietary datatype display applications + trans.app.datatypes_registry.load_display_applications( installed_repository_dict=repository_dict ) + +def handle_tool_shed_repositories( trans, installation_dict ): + # The following installation_dict entries are all required. + install_repository_dependencies = installation_dict[ 'install_repository_dependencies' ] + new_tool_panel_section = installation_dict[ 'new_tool_panel_section' ] + no_changes_checked = installation_dict[ 'no_changes_checked' ] + reinstalling = installation_dict[ 'reinstalling' ] + repo_info_dicts = installation_dict[ 'repo_info_dicts' ] + tool_panel_section = installation_dict[ 'tool_panel_section' ] + tool_path = installation_dict[ 'tool_path' ] + tool_shed_url = installation_dict[ 'tool_shed_url' ] + created_or_updated_tool_shed_repositories, tool_panel_section_keys, repo_info_dicts, filtered_repo_info_dicts, message = \ + repository_dependency_util.create_repository_dependency_objects( trans=trans, + tool_path=tool_path, + tool_shed_url=tool_shed_url, + repo_info_dicts=repo_info_dicts, + reinstalling=reinstalling, + install_repository_dependencies=install_repository_dependencies, + no_changes_checked=no_changes_checked, + tool_panel_section=tool_panel_section, + new_tool_panel_section=new_tool_panel_section ) + if message and len( repo_info_dicts ) == 1: + installed_tool_shed_repository = created_or_updated_tool_shed_repositories[ 0 ] + message += 'Click <a href="%s">here</a> to manage the repository. ' % \ + ( web.url_for( controller='admin_toolshed', action='manage_repository', id=trans.security.encode_id( installed_tool_shed_repository.id ) ) ) + return created_or_updated_tool_shed_repositories, tool_panel_section_keys, repo_info_dicts, filtered_repo_info_dicts, message + +def initiate_repository_installation( trans, installation_dict ): + # The following installation_dict entries are all required. + created_or_updated_tool_shed_repositories = installation_dict[ 'created_or_updated_tool_shed_repositories' ] + filtered_repo_info_dicts = installation_dict[ 'filtered_repo_info_dicts' ] + has_repository_dependencies = installation_dict[ 'has_repository_dependencies' ] + includes_tool_dependencies = installation_dict[ 'includes_tool_dependencies' ] + includes_tools = installation_dict[ 'includes_tools' ] + includes_tools_for_display_in_tool_panel = installation_dict[ 'includes_tools_for_display_in_tool_panel' ] + install_repository_dependencies = installation_dict[ 'install_repository_dependencies' ] + install_tool_dependencies = installation_dict[ 'install_tool_dependencies' ] + message = installation_dict[ 'message' ] + new_tool_panel_section = installation_dict[ 'new_tool_panel_section' ] + shed_tool_conf = installation_dict[ 'shed_tool_conf' ] + status = installation_dict[ 'status' ] + tool_panel_section = installation_dict[ 'tool_panel_section' ] + tool_panel_section_keys = installation_dict[ 'tool_panel_section_keys' ] + tool_path = installation_dict[ 'tool_path' ] + tool_shed_url = installation_dict[ 'tool_shed_url' ] + # Handle contained tools. + if includes_tools_for_display_in_tool_panel and ( new_tool_panel_section or tool_panel_section ): + if new_tool_panel_section: + section_id = new_tool_panel_section.lower().replace( ' ', '_' ) + tool_panel_section_key = 'section_%s' % str( section_id ) + if tool_panel_section_key in trans.app.toolbox.tool_panel: + # Appending a tool to an existing section in trans.app.toolbox.tool_panel + log.debug( "Appending to tool panel section: %s" % new_tool_panel_section ) + tool_section = trans.app.toolbox.tool_panel[ tool_panel_section_key ] + else: + # Appending a new section to trans.app.toolbox.tool_panel + log.debug( "Loading new tool panel section: %s" % new_tool_panel_section ) + elem = Element( 'section' ) + elem.attrib[ 'name' ] = new_tool_panel_section + elem.attrib[ 'id' ] = section_id + elem.attrib[ 'version' ] = '' + tool_section = tools.ToolSection( elem ) + trans.app.toolbox.tool_panel[ tool_panel_section_key ] = tool_section + else: + tool_panel_section_key = 'section_%s' % tool_panel_section + tool_section = trans.app.toolbox.tool_panel[ tool_panel_section_key ] + else: + tool_panel_section_key = None + tool_section = None + encoded_repository_ids = [ trans.security.encode_id( tsr.id ) for tsr in created_or_updated_tool_shed_repositories ] + # Create a one-to-one mapping of tool shed repository id and tool panel section key. All tools contained in the repositories being installed will be loaded + # into the same section in the tool panel. + for tsr in created_or_updated_tool_shed_repositories: + tool_panel_section_keys.append( tool_panel_section_key ) + new_kwd = dict( includes_tools=includes_tools, + includes_tools_for_display_in_tool_panel=includes_tools_for_display_in_tool_panel, + has_repository_dependencies=has_repository_dependencies, + install_repository_dependencies=install_repository_dependencies, + includes_tool_dependencies=includes_tool_dependencies, + install_tool_dependencies=install_tool_dependencies, + message=message, + repo_info_dicts=filtered_repo_info_dicts, + shed_tool_conf=shed_tool_conf, + status=status, + tool_path=tool_path, + tool_panel_section_keys=tool_panel_section_keys, + tool_shed_repository_ids=encoded_repository_ids, + tool_shed_url=tool_shed_url ) + encoded_kwd = encoding_util.tool_shed_encode( new_kwd ) + tsr_ids = [ r.id for r in created_or_updated_tool_shed_repositories ] + tool_shed_repositories = [] + for tsr_id in tsr_ids: + tsr = trans.sa_session.query( trans.model.ToolShedRepository ).get( tsr_id ) + tool_shed_repositories.append( tsr ) + clause_list = [] + for tsr_id in tsr_ids: + clause_list.append( trans.model.ToolShedRepository.table.c.id == tsr_id ) + query = trans.sa_session.query( trans.model.ToolShedRepository ).filter( or_( *clause_list ) ) + return encoded_kwd, query, tool_shed_repositories, encoded_repository_ids + def merge_containers_dicts_for_new_install( containers_dicts ): """ When installing one or more tool shed repositories for the first time, the received list of containers_dicts contains a containers_dict for diff -r b45e7e2cf125642efcbbcb112778a44f64de4ec2 -r dab01cbc83dceddd9f02a48deb32c7ffb1817eba lib/tool_shed/util/shed_util_common.py --- a/lib/tool_shed/util/shed_util_common.py +++ b/lib/tool_shed/util/shed_util_common.py @@ -1,6 +1,12 @@ -import os, shutil, tempfile, logging, string, urllib2 +import logging +import os +import shutil +import string +import tempfile +import urllib2 from datetime import datetime -from time import gmtime, strftime +from time import gmtime +from time import strftime from galaxy import util from galaxy.util import json from galaxy.web import url_for @@ -16,8 +22,10 @@ from mercurial import hg, ui, commands pkg_resources.require( 'elementtree' ) -from elementtree import ElementTree, ElementInclude -from elementtree.ElementTree import Element, SubElement +from elementtree import ElementTree +from elementtree import ElementInclude +from elementtree.ElementTree import Element +from elementtree.ElementTree import SubElement eggs.require( 'markupsafe' ) import markupsafe @@ -522,7 +530,7 @@ if changeset_revision in changeset_revisions: return changeset_revision elif not found_after_changeset_revision and changeset_revision == after_changeset_revision: - # We've found the changeset in the changelog for which we need to get the next downloadable changset. + # We've found the changeset in the changelog for which we need to get the next downloadable changeset. found_after_changeset_revision = True return None Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: 2 new changesets
by commits-noreply＠bitbucket.org 22 Mar '13

22 Mar '13

2 new commits in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/cfb18b939f74/ Changeset: cfb18b939f74 Branch: next-stable User: dan Date: 2013-03-22 19:23:24 Summary: Fix for average_rating in repository_review. Affected #: 1 file diff -r fc236ecff8ecd8864dee797e9a70ec800e63948d -r cfb18b939f74aaa97c277e172aee5a8e0f24b55e lib/galaxy/webapps/tool_shed/controllers/repository_review.py --- a/lib/galaxy/webapps/tool_shed/controllers/repository_review.py +++ b/lib/galaxy/webapps/tool_shed/controllers/repository_review.py @@ -320,7 +320,8 @@ trans.model.ComponentReview.table.c.deleted == False, trans.model.ComponentReview.table.c.approved != trans.model.ComponentReview.approved_states.NA ) ) \ .scalar() - review.rating = int( average_rating ) + if average_rating is not None: + review.rating = int( average_rating ) trans.sa_session.add( review ) trans.sa_session.flush() # Update the information in components_dict. https://bitbucket.org/galaxy/galaxy-central/commits/b45e7e2cf125/ Changeset: b45e7e2cf125 User: dan Date: 2013-03-22 19:23:24 Summary: Fix for average_rating in repository_review. Affected #: 1 file diff -r c923439d8226f292160faf04b1c66a6baa2450af -r b45e7e2cf125642efcbbcb112778a44f64de4ec2 lib/galaxy/webapps/tool_shed/controllers/repository_review.py --- a/lib/galaxy/webapps/tool_shed/controllers/repository_review.py +++ b/lib/galaxy/webapps/tool_shed/controllers/repository_review.py @@ -320,7 +320,8 @@ trans.model.ComponentReview.table.c.deleted == False, trans.model.ComponentReview.table.c.approved != trans.model.ComponentReview.approved_states.NA ) ) \ .scalar() - review.rating = int( average_rating ) + if average_rating is not None: + review.rating = int( average_rating ) trans.sa_session.add( review ) trans.sa_session.flush() # Update the information in components_dict. Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: dan: Add some comments to the new-style display applications module.
by commits-noreply＠bitbucket.org 22 Mar '13

22 Mar '13

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/c923439d8226/ Changeset: c923439d8226 User: dan Date: 2013-03-22 17:33:10 Summary: Add some comments to the new-style display applications module. Affected #: 2 files diff -r 49c09e9f8c8b97ca37ddf484efe4a539c7bbf9cc -r c923439d8226f292160faf04b1c66a6baa2450af lib/galaxy/datatypes/display_applications/__init__.py --- a/lib/galaxy/datatypes/display_applications/__init__.py +++ b/lib/galaxy/datatypes/display_applications/__init__.py @@ -1,1 +1,3 @@ - +""" +Contains functionality of the newer XML defined external display applications (not hardcoded into datatype classes). +""" diff -r 49c09e9f8c8b97ca37ddf484efe4a539c7bbf9cc -r c923439d8226f292160faf04b1c66a6baa2450af lib/galaxy/datatypes/display_applications/link_generator.py --- a/lib/galaxy/datatypes/display_applications/link_generator.py +++ b/lib/galaxy/datatypes/display_applications/link_generator.py @@ -1,8 +1,11 @@ -"""Classes to generate links for display applications. +"""Classes to generate links for old-style display applications. Separating Transaction based elements of display applications from datatypes. """ +#FIXME: The code contained within this file is for old-style display applications, but +#this module namespace is intended to only handle the new-style display applications. + import urllib # for the url_for hack @@ -19,6 +22,8 @@ #TODO: these could be extended to handle file_function and parse/contain the builds.txt files +#HACK: these duplicate functionality from the individual datatype classes themselves + def get_display_app_link_generator( display_app_name ): """Returns an instance of the proper link generator class based on the display_app_name or DisplayAppLinkGenerator Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: 5 new changesets
by commits-noreply＠bitbucket.org 22 Mar '13

22 Mar '13

5 new commits in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/c78e28a5f748/ Changeset: c78e28a5f748 Branch: stable User: dan Date: 2013-03-22 17:23:05 Summary: Do not display old-style display links in server-side history when they are disabled. Affected #: 1 file diff -r c0a665a31e9160fa344255d6f3fe22a0ab2b6a6f -r c78e28a5f748e16b0a168d36ba55e5625f322548 lib/galaxy/datatypes/data.py --- a/lib/galaxy/datatypes/data.py +++ b/lib/galaxy/datatypes/data.py @@ -437,12 +437,12 @@ apply anyway. """ try: - if type in self.get_display_types(): + if app.config.enable_old_display_applications and type in self.get_display_types(): return target_frame, getattr ( self, self.supported_display_apps[type]['links_function'] ) ( dataset, type, app, base_url, **kwd ) except: log.exception( 'Function %s is referred to in datatype %s for generating links for type %s, but is not accessible' \ % ( self.supported_display_apps[type]['links_function'], self.__class__.__name__, type ) ) - return [] + return target_frame, [] def get_converter_types(self, original_dataset, datatypes_registry): """Returns available converters by type for this dataset""" return datatypes_registry.get_converters_by_datatype(original_dataset.ext) https://bitbucket.org/galaxy/galaxy-central/commits/12b05ec9d3de/ Changeset: 12b05ec9d3de Branch: next-stable User: dan Date: 2013-03-22 17:23:05 Summary: Do not display old-style display links in server-side history when they are disabled. Affected #: 1 file diff -r 66c67f015f9e8beb1bfaf4365bdaeb41d9090fe2 -r 12b05ec9d3de16202bb3005c39d61b6d643bfdcb lib/galaxy/datatypes/data.py --- a/lib/galaxy/datatypes/data.py +++ b/lib/galaxy/datatypes/data.py @@ -448,12 +448,12 @@ apply anyway. """ try: - if type in self.get_display_types(): + if app.config.enable_old_display_applications and type in self.get_display_types(): return target_frame, getattr ( self, self.supported_display_apps[type]['links_function'] ) ( dataset, type, app, base_url, **kwd ) except: log.exception( 'Function %s is referred to in datatype %s for generating links for type %s, but is not accessible' \ % ( self.supported_display_apps[type]['links_function'], self.__class__.__name__, type ) ) - return [] + return target_frame, [] def get_converter_types(self, original_dataset, datatypes_registry): """Returns available converters by type for this dataset""" return datatypes_registry.get_converters_by_datatype(original_dataset.ext) https://bitbucket.org/galaxy/galaxy-central/commits/6701d9bb937e/ Changeset: 6701d9bb937e User: dan Date: 2013-03-22 17:23:05 Summary: Do not display old-style display links in server-side history when they are disabled. Affected #: 1 file diff -r 56799709a0f35d1f5cde0aa8f38939eafea98294 -r 6701d9bb937e488ac9797d14ad81ef1ced48cd41 lib/galaxy/datatypes/data.py --- a/lib/galaxy/datatypes/data.py +++ b/lib/galaxy/datatypes/data.py @@ -448,12 +448,12 @@ apply anyway. """ try: - if type in self.get_display_types(): + if app.config.enable_old_display_applications and type in self.get_display_types(): return target_frame, getattr ( self, self.supported_display_apps[type]['links_function'] ) ( dataset, type, app, base_url, **kwd ) except: log.exception( 'Function %s is referred to in datatype %s for generating links for type %s, but is not accessible' \ % ( self.supported_display_apps[type]['links_function'], self.__class__.__name__, type ) ) - return [] + return target_frame, [] def get_converter_types(self, original_dataset, datatypes_registry): """Returns available converters by type for this dataset""" return datatypes_registry.get_converters_by_datatype(original_dataset.ext) https://bitbucket.org/galaxy/galaxy-central/commits/fc236ecff8ec/ Changeset: fc236ecff8ec Branch: next-stable User: dan Date: 2013-03-22 17:26:03 Summary: Do not display old-style display links in client-side history when they are disabled. Affected #: 1 file diff -r 12b05ec9d3de16202bb3005c39d61b6d643bfdcb -r fc236ecff8ecd8864dee797e9a70ec800e63948d lib/galaxy/web/base/controller.py --- a/lib/galaxy/web/base/controller.py +++ b/lib/galaxy/web/base/controller.py @@ -394,6 +394,8 @@ def get_old_display_applications( self, trans, hda ): display_apps = [] + if not trans.app.config.enable_old_display_applications: + return display_apps for display_app_name in hda.datatype.get_display_types(): link_generator = get_display_app_link_generator( display_app_name ) display_links = link_generator.generate_links( trans, hda ) https://bitbucket.org/galaxy/galaxy-central/commits/49c09e9f8c8b/ Changeset: 49c09e9f8c8b User: dan Date: 2013-03-22 17:26:03 Summary: Do not display old-style display links in client-side history when they are disabled. Affected #: 1 file Diff not available. Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: 3 new changesets
by commits-noreply＠bitbucket.org 22 Mar '13

22 Mar '13

3 new commits in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/c0a665a31e91/ Changeset: c0a665a31e91 Branch: stable User: dan Date: 2013-03-22 16:43:42 Summary: Allow old-style display applications to be disabled. Affected #: 3 files diff -r 779b54139013946613772d02a3fbb61e067f3267 -r c0a665a31e9160fa344255d6f3fe22a0ab2b6a6f lib/galaxy/config.py --- a/lib/galaxy/config.py +++ b/lib/galaxy/config.py @@ -141,6 +141,7 @@ self.log_actions = string_as_bool( kwargs.get( 'log_actions', 'False' ) ) self.log_events = string_as_bool( kwargs.get( 'log_events', 'False' ) ) self.sanitize_all_html = string_as_bool( kwargs.get( 'sanitize_all_html', True ) ) + self.enable_old_display_applications = string_as_bool( kwargs.get( "enable_old_display_applications", "True" ) ) self.ucsc_display_sites = kwargs.get( 'ucsc_display_sites', "main,test,archaea,ucla" ).lower().split(",") self.gbrowse_display_sites = kwargs.get( 'gbrowse_display_sites', "modencode,sgd_yeast,tair,wormbase,wormbase_ws120,wormbase_ws140,wormbase_ws170,wormbase_ws180,wormbase_ws190,wormbase_ws200,wormbase_ws204,wormbase_ws210,wormbase_ws220,wormbase_ws225" ).lower().split(",") self.genetrack_display_sites = kwargs.get( 'genetrack_display_sites', "main,test" ).lower().split(",") diff -r 779b54139013946613772d02a3fbb61e067f3267 -r c0a665a31e9160fa344255d6f3fe22a0ab2b6a6f lib/galaxy/webapps/galaxy/controllers/dataset.py --- a/lib/galaxy/webapps/galaxy/controllers/dataset.py +++ b/lib/galaxy/webapps/galaxy/controllers/dataset.py @@ -700,6 +700,8 @@ @web.expose def display_at( self, trans, dataset_id, filename=None, **kwd ): """Sets up a dataset permissions so it is viewable at an external site""" + if not trans.app.config.enable_old_display_applications: + return trans.show_error_message( "This method of accessing external display applications has been disabled by a Galaxy administrator." ) site = filename data = trans.sa_session.query( trans.app.model.HistoryDatasetAssociation ).get( dataset_id ) if not data: diff -r 779b54139013946613772d02a3fbb61e067f3267 -r c0a665a31e9160fa344255d6f3fe22a0ab2b6a6f universe_wsgi.ini.sample --- a/universe_wsgi.ini.sample +++ b/universe_wsgi.ini.sample @@ -247,6 +247,16 @@ # them). #display_servers = hgw1.cse.ucsc.edu,hgw2.cse.ucsc.edu,hgw3.cse.ucsc.edu,hgw4.cse.ucsc.edu,hgw… +# To disable the old-style display applications that are hardcoded into datatype classes, +# set enable_old_display_applications = False. +# This may be desirable due to using the new-style, XML-defined, display applications that +# have been defined for many of the datatypes that have the old-style. +# There is also a potential security concern with the old-style applications, +# where a malicious party could provide a link that appears to reference the Galaxy server, +# but contains a redirect to a third-party server, tricking a Galaxy user to access said +# site. +#enable_old_display_applications = True + # -- Next gen LIMS interface on top of existing Galaxy Sample/Request management code. use_nglims = False https://bitbucket.org/galaxy/galaxy-central/commits/66c67f015f9e/ Changeset: 66c67f015f9e Branch: next-stable User: dan Date: 2013-03-22 16:43:42 Summary: Allow old-style display applications to be disabled. Affected #: 3 files diff -r 3bbbcdb044afdbd380d6cd4c0f721ab7262c5555 -r 66c67f015f9e8beb1bfaf4365bdaeb41d9090fe2 lib/galaxy/config.py --- a/lib/galaxy/config.py +++ b/lib/galaxy/config.py @@ -151,6 +151,7 @@ self.log_actions = string_as_bool( kwargs.get( 'log_actions', 'False' ) ) self.log_events = string_as_bool( kwargs.get( 'log_events', 'False' ) ) self.sanitize_all_html = string_as_bool( kwargs.get( 'sanitize_all_html', True ) ) + self.enable_old_display_applications = string_as_bool( kwargs.get( "enable_old_display_applications", "True" ) ) self.ucsc_display_sites = kwargs.get( 'ucsc_display_sites', "main,test,archaea,ucla" ).lower().split(",") self.gbrowse_display_sites = kwargs.get( 'gbrowse_display_sites', "modencode,sgd_yeast,tair,wormbase,wormbase_ws120,wormbase_ws140,wormbase_ws170,wormbase_ws180,wormbase_ws190,wormbase_ws200,wormbase_ws204,wormbase_ws210,wormbase_ws220,wormbase_ws225" ).lower().split(",") self.brand = kwargs.get( 'brand', None ) diff -r 3bbbcdb044afdbd380d6cd4c0f721ab7262c5555 -r 66c67f015f9e8beb1bfaf4365bdaeb41d9090fe2 lib/galaxy/webapps/galaxy/controllers/dataset.py --- a/lib/galaxy/webapps/galaxy/controllers/dataset.py +++ b/lib/galaxy/webapps/galaxy/controllers/dataset.py @@ -697,6 +697,8 @@ @web.expose def display_at( self, trans, dataset_id, filename=None, **kwd ): """Sets up a dataset permissions so it is viewable at an external site""" + if not trans.app.config.enable_old_display_applications: + return trans.show_error_message( "This method of accessing external display applications has been disabled by a Galaxy administrator." ) site = filename data = trans.sa_session.query( trans.app.model.HistoryDatasetAssociation ).get( dataset_id ) if not data: diff -r 3bbbcdb044afdbd380d6cd4c0f721ab7262c5555 -r 66c67f015f9e8beb1bfaf4365bdaeb41d9090fe2 universe_wsgi.ini.sample --- a/universe_wsgi.ini.sample +++ b/universe_wsgi.ini.sample @@ -247,6 +247,16 @@ # them). #display_servers = hgw1.cse.ucsc.edu,hgw2.cse.ucsc.edu,hgw3.cse.ucsc.edu,hgw4.cse.ucsc.edu,hgw… +# To disable the old-style display applications that are hardcoded into datatype classes, +# set enable_old_display_applications = False. +# This may be desirable due to using the new-style, XML-defined, display applications that +# have been defined for many of the datatypes that have the old-style. +# There is also a potential security concern with the old-style applications, +# where a malicious party could provide a link that appears to reference the Galaxy server, +# but contains a redirect to a third-party server, tricking a Galaxy user to access said +# site. +#enable_old_display_applications = True + # -- Next gen LIMS interface on top of existing Galaxy Sample/Request management code. use_nglims = False https://bitbucket.org/galaxy/galaxy-central/commits/56799709a0f3/ Changeset: 56799709a0f3 User: dan Date: 2013-03-22 16:43:42 Summary: Allow old-style display applications to be disabled. Affected #: 3 files diff -r d629a346b859c91ba0da27f865952580d1054d80 -r 56799709a0f35d1f5cde0aa8f38939eafea98294 lib/galaxy/config.py --- a/lib/galaxy/config.py +++ b/lib/galaxy/config.py @@ -151,6 +151,7 @@ self.log_actions = string_as_bool( kwargs.get( 'log_actions', 'False' ) ) self.log_events = string_as_bool( kwargs.get( 'log_events', 'False' ) ) self.sanitize_all_html = string_as_bool( kwargs.get( 'sanitize_all_html', True ) ) + self.enable_old_display_applications = string_as_bool( kwargs.get( "enable_old_display_applications", "True" ) ) self.ucsc_display_sites = kwargs.get( 'ucsc_display_sites', "main,test,archaea,ucla" ).lower().split(",") self.gbrowse_display_sites = kwargs.get( 'gbrowse_display_sites', "modencode,sgd_yeast,tair,wormbase,wormbase_ws120,wormbase_ws140,wormbase_ws170,wormbase_ws180,wormbase_ws190,wormbase_ws200,wormbase_ws204,wormbase_ws210,wormbase_ws220,wormbase_ws225" ).lower().split(",") self.brand = kwargs.get( 'brand', None ) diff -r d629a346b859c91ba0da27f865952580d1054d80 -r 56799709a0f35d1f5cde0aa8f38939eafea98294 lib/galaxy/webapps/galaxy/controllers/dataset.py --- a/lib/galaxy/webapps/galaxy/controllers/dataset.py +++ b/lib/galaxy/webapps/galaxy/controllers/dataset.py @@ -697,6 +697,8 @@ @web.expose def display_at( self, trans, dataset_id, filename=None, **kwd ): """Sets up a dataset permissions so it is viewable at an external site""" + if not trans.app.config.enable_old_display_applications: + return trans.show_error_message( "This method of accessing external display applications has been disabled by a Galaxy administrator." ) site = filename data = trans.sa_session.query( trans.app.model.HistoryDatasetAssociation ).get( dataset_id ) if not data: diff -r d629a346b859c91ba0da27f865952580d1054d80 -r 56799709a0f35d1f5cde0aa8f38939eafea98294 universe_wsgi.ini.sample --- a/universe_wsgi.ini.sample +++ b/universe_wsgi.ini.sample @@ -247,6 +247,16 @@ # them). #display_servers = hgw1.cse.ucsc.edu,hgw2.cse.ucsc.edu,hgw3.cse.ucsc.edu,hgw4.cse.ucsc.edu,hgw… +# To disable the old-style display applications that are hardcoded into datatype classes, +# set enable_old_display_applications = False. +# This may be desirable due to using the new-style, XML-defined, display applications that +# have been defined for many of the datatypes that have the old-style. +# There is also a potential security concern with the old-style applications, +# where a malicious party could provide a link that appears to reference the Galaxy server, +# but contains a redirect to a third-party server, tricking a Galaxy user to access said +# site. +#enable_old_display_applications = True + # -- Next gen LIMS interface on top of existing Galaxy Sample/Request management code. use_nglims = False Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: greg: Eliminate unnecessary resource mappers and api classes in the tool shed api, clean up the remaining tool shed api classes, and enhance the tool shed api by adding the ability to retrieve information about a specified repository and repository revision.
by commits-noreply＠bitbucket.org 22 Mar '13

22 Mar '13

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/d629a346b859/ Changeset: d629a346b859 User: greg Date: 2013-03-22 15:43:58 Summary: Eliminate unnecessary resource mappers and api classes in the tool shed api, clean up the remaining tool shed api classes, and enhance the tool shed api by adding the ability to retrieve information about a specified repository and repository revision. Affected #: 6 files diff -r 14ab08c0fbfe3e9735e94f4e399aeabc870fe4b2 -r d629a346b859c91ba0da27f865952580d1054d80 lib/galaxy/webapps/tool_shed/api/repositories.py --- a/lib/galaxy/webapps/tool_shed/api/repositories.py +++ b/lib/galaxy/webapps/tool_shed/api/repositories.py @@ -1,10 +1,30 @@ import logging +from galaxy.web.framework.helpers import time_ago import tool_shed.util.shed_util_common as suc -from galaxy import web, util +from galaxy import web +from galaxy import util from galaxy.web.base.controller import BaseAPIController +from galaxy import eggs +import pkg_resources + +pkg_resources.require( 'mercurial' ) +from mercurial import hg, ui, commands + log = logging.getLogger( __name__ ) +def default_repository_value_mapper( trans, repository ): + value_mapper={ 'id' : trans.security.encode_id( repository.id ), + 'user_id' : trans.security.encode_id( repository.user_id ) } + return value_mapper + +def default_repository_metadata_value_mapper( trans, repository_metadata ): + value_mapper = { 'id' : trans.security.encode_id( repository_metadata.id ), + 'repository_id' : trans.security.encode_id( repository_metadata.repository_id ) } + if repository_metadata.time_last_tested: + value_mapper[ 'time_last_tested' ] = time_ago( repository_metadata.time_last_tested ) + return value_mapper + class RepositoriesController( BaseAPIController ): """RESTful controller for interactions with repositories in the Tool Shed.""" @@ -15,6 +35,7 @@ GET /api/repositories Displays a collection (list) of repositories. """ + # Example URL: http://localhost:9009/api/repositories repository_dicts = [] deleted = util.string_as_bool( deleted ) try: @@ -23,12 +44,10 @@ .order_by( trans.app.model.Repository.table.c.name ) \ .all() for repository in query: - value_mapper={ 'id' : trans.security.encode_id( repository.id ), - 'user_id' : trans.security.encode_id( repository.user_id ) } - repository_dict = repository.get_api_value( view='collection', value_mapper=value_mapper ) - repository_dict[ 'url' ] = web.url_for( controller='repository_contents', - action='index', - repository_id=trans.security.encode_id( repository.id ) ) + repository_dict = repository.get_api_value( view='collection', value_mapper=default_repository_value_mapper( trans, repository ) ) + repository_dict[ 'url' ] = web.url_for( controller='repositories', + action='show', + id=trans.security.encode_id( repository.id ) ) repository_dicts.append( repository_dict ) return repository_dicts except Exception, e: @@ -41,21 +60,67 @@ def show( self, trans, id, **kwd ): """ GET /api/repositories/{encoded_repository_id} - Displays information about a repository in the Tool Shed. + Returns information about a repository in the Tool Shed. - :param id: the encoded id of the `Repository` object + :param id: the encoded id of the Repository object """ + # Example URL: http://localhost:9009/api/repositories/f9cad7b01a472135 try: repository = suc.get_repository_in_tool_shed( trans, id ) - value_mapper={ 'id' : trans.security.encode_id( repository.id ), - 'user_id' : trans.security.encode_id( repository.user_id ) } - repository_dict = repository.get_api_value( view='element', value_mapper=value_mapper ) - repository_dict[ 'url' ] = web.url_for( controller='repository_contents', - action='index', - repository_id=trans.security.encode_id( repository.id ) ) + repository_dict = repository.get_api_value( view='element', value_mapper=default_repository_value_mapper( trans, repository ) ) + repository_dict[ 'url' ] = web.url_for( controller='repositories', + action='show', + id=trans.security.encode_id( repository.id ) ) return repository_dict except Exception, e: message = "Error in the Tool Shed repositories API in show: %s" % str( e ) log.error( message, exc_info=True ) trans.response.status = 500 return message + + @web.expose_api + def get_repository_and_revision( self, trans, name, owner, changeset_revision, **kwd ): + """ + GET /api/repository/get_repository_and_revision + Returns information about a repository revision in the Tool Shed. + + :param name: the name of the Repository object + :param owner: the owner of the Repository object + :param changset_revision: the changset_revision of the RepositoryMetadata object associated with the Repository object + """ + # Example URL: http://localhost:9009/api/repositories/get_repository_and_revision?name=add… + try: + # Get the repository information. + repository = suc.get_repository_by_name_and_owner( trans.app, name, owner ) + encoded_repository_id = trans.security.encode_id( repository.id ) + repository_dict = repository.get_api_value( view='element', value_mapper=default_repository_value_mapper( trans, repository ) ) + repository_dict[ 'url' ] = web.url_for( controller='repositories', + action='show', + id=encoded_repository_id ) + # Get the repository_metadata information. + repository_metadata = suc.get_repository_metadata_by_changeset_revision( trans, encoded_repository_id, changeset_revision ) + if not repository_metadata: + # The changeset_revision column in the repository_metadata table has been updated with a new value value, so find the + # changeset_revision to which we need to update. + repo_dir = repository.repo_path( trans.app ) + repo = hg.repository( suc.get_configured_ui(), repo_dir ) + new_changeset_revision = suc.get_next_downloadable_changeset_revision( repository, repo, changeset_revision ) + repository_metadata = suc.get_repository_metadata_by_changeset_revision( trans, encoded_repository_id, new_changeset_revision ) + if repository_metadata: + encoded_repository_metadata_id = trans.security.encode_id( repository_metadata.id ) + repository_metadata_dict = repository_metadata.get_api_value( view='collection', + value_mapper=default_repository_metadata_value_mapper( trans, repository_metadata ) ) + repository_metadata_dict[ 'url' ] = web.url_for( controller='repository_revisions', + action='show', + id=encoded_repository_metadata_id ) + return repository_dict, repository_metadata_dict + else: + message = "Unable to locate repository_metadata record for repository id %d and changeset_revision %s" % ( repository.id, changeset_revision ) + log.error( message, exc_info=True ) + trans.response.status = 500 + return repository_dict, {} + except Exception, e: + message = "Error in the Tool Shed repositories API in get_repository_and_revision: %s" % str( e ) + log.error( message, exc_info=True ) + trans.response.status = 500 + return message diff -r 14ab08c0fbfe3e9735e94f4e399aeabc870fe4b2 -r d629a346b859c91ba0da27f865952580d1054d80 lib/galaxy/webapps/tool_shed/api/repository_contents.py --- a/lib/galaxy/webapps/tool_shed/api/repository_contents.py +++ /dev/null @@ -1,33 +0,0 @@ -import logging -import tool_shed.util.shed_util_common as suc -from galaxy import web -from galaxy.web.base.controller import BaseAPIController - -log = logging.getLogger( __name__ ) - - -class RepositoryContentsController( BaseAPIController ): - - @web.expose_api - def index( self, trans, **kwd ): - """ - GET /api/repositories/{encoded_repository_id} - Displays a collection (dictionary) of repository contents. - - :param repository_id: the encoded id of the `Repository` object - """ - try: - repository_id = kwd[ 'repository_id' ] - repository = suc.get_repository_in_tool_shed( trans, repository_id ) - value_mapper={ 'id' : trans.security.encode_id( repository.id ), - 'user_id' : trans.security.encode_id( repository.user_id ) } - repository_dict = repository.as_dict( value_mapper ) - repository_dict[ 'url' ] = web.url_for( controller='repository_contents', - action='index', - repository_id=repository_id ) - return repository_dict - except Exception, e: - message = "Error in the Tool Shed repository_contents API in index: %s" % str( e ) - log.error( message, exc_info=True ) - trans.response.status = 500 - return message diff -r 14ab08c0fbfe3e9735e94f4e399aeabc870fe4b2 -r d629a346b859c91ba0da27f865952580d1054d80 lib/galaxy/webapps/tool_shed/api/repository_revision_contents.py --- a/lib/galaxy/webapps/tool_shed/api/repository_revision_contents.py +++ /dev/null @@ -1,39 +0,0 @@ -import logging -from galaxy import web -from galaxy.web.framework.helpers import time_ago -from tool_shed.util import metadata_util -from galaxy.web.base.controller import BaseAPIController - -log = logging.getLogger( __name__ ) - -def default_value_mapper( trans, repository_metadata ): - value_mapper = { 'id' : trans.security.encode_id( repository_metadata.id ), - 'repository_id' : trans.security.encode_id( repository_metadata.repository_id ) } - if repository_metadata.time_last_tested: - value_mapper[ 'time_last_tested' ] = time_ago( repository_metadata.time_last_tested ) - return value_mapper - - -class RepositoryRevisionContentsController( BaseAPIController ): - - @web.expose_api - def index( self, trans, **kwd ): - """ - GET /api/repository_revisions/{encoded_repository_metadata_id} - Displays a collection (dictionary) of repository_metadata contents. - - :param repository_metadata_id: the encoded id of the `RepositoryMetadata` object - """ - try: - repository_metadata_id = kwd.get( 'repository_metadata_id', None ) - repository_metadata = metadata_util.get_repository_metadata_by_id( trans, repository_metadata_id ) - repository_dict = repository_metadata.as_dict( value_mapper=default_value_mapper( trans, repository_metadata ) ) - repository_dict[ 'url' ] = web.url_for( controller='repository_revision_contents', - action='index', - repository_metadata_id=repository_metadata_id ) - return repository_dict - except Exception, e: - message = "Error in the Tool Shed repository_revision_contents API in index: %s" % str( e ) - log.error( message, exc_info=True ) - trans.response.status = 500 - return message diff -r 14ab08c0fbfe3e9735e94f4e399aeabc870fe4b2 -r d629a346b859c91ba0da27f865952580d1054d80 lib/galaxy/webapps/tool_shed/api/repository_revisions.py --- a/lib/galaxy/webapps/tool_shed/api/repository_revisions.py +++ b/lib/galaxy/webapps/tool_shed/api/repository_revisions.py @@ -1,7 +1,9 @@ -import datetime, logging +import datetime +import logging from galaxy.web.framework.helpers import time_ago from tool_shed.util import metadata_util -from galaxy import web, util +from galaxy import web +from galaxy import util from galaxy.model.orm import and_ from galaxy.web.base.controller import BaseAPIController @@ -24,6 +26,7 @@ GET /api/repository_revisions Displays a collection (list) of repository revisions. """ + # Example URL: http://localhost:9009/api/repository_revisions repository_metadata_dicts = [] # Build up an anded clause list of filters. clause_list = [] @@ -55,9 +58,9 @@ for repository_metadata in query: repository_metadata_dict = repository_metadata.get_api_value( view='collection', value_mapper=default_value_mapper( trans, repository_metadata ) ) - repository_metadata_dict[ 'url' ] = web.url_for( controller='repository_revision_contents', - action='index', - repository_metadata_id=trans.security.encode_id( repository_metadata.id ) ) + repository_metadata_dict[ 'url' ] = web.url_for( controller='repository_revisions', + action='show', + id=trans.security.encode_id( repository_metadata.id ) ) repository_metadata_dicts.append( repository_metadata_dict ) return repository_metadata_dicts except Exception, e: @@ -74,12 +77,13 @@ :param id: the encoded id of the `RepositoryMetadata` object """ + # Example URL: http://localhost:9009/api/repository_revisions/bb125606ff9ea620 try: repository_metadata = metadata_util.get_repository_metadata_by_id( trans, id ) repository_metadata_dict = repository_metadata.as_dict( value_mapper=default_value_mapper( trans, repository_metadata ) ) - repository_metadata_dict[ 'url' ] = web.url_for( controller='repository_revision_contents', - action='index', - repository_metadata_id=trans.security.encode_id( repository_metadata.id ) ) + repository_metadata_dict[ 'url' ] = web.url_for( controller='repository_revisions', + action='show', + id=trans.security.encode_id( repository_metadata.id ) ) return repository_metadata_dict except Exception, e: message = "Error in the Tool Shed repository_revisions API in show: %s" % str( e ) @@ -114,7 +118,7 @@ trans.response.status = 500 return message repository_metadata_dict = repository_metadata.as_dict( value_mapper=default_value_mapper( trans, repository_metadata ) ) - repository_metadata_dict[ 'url' ] = web.url_for( controller='repository_revision_contents', - action='index', - repository_metadata_id=trans.security.encode_id( repository_metadata.id ) ) + repository_metadata_dict[ 'url' ] = web.url_for( controller='repository_revisions', + action='show', + id=trans.security.encode_id( repository_metadata.id ) ) return repository_metadata_dict diff -r 14ab08c0fbfe3e9735e94f4e399aeabc870fe4b2 -r d629a346b859c91ba0da27f865952580d1054d80 lib/galaxy/webapps/tool_shed/buildapp.py --- a/lib/galaxy/webapps/tool_shed/buildapp.py +++ b/lib/galaxy/webapps/tool_shed/buildapp.py @@ -1,8 +1,11 @@ """ Provides factory methods to assemble the Galaxy web application """ - -import logging, atexit, os, os.path, sys, config +import atexit +import config +import logging +import os +import sys from inspect import isclass @@ -69,22 +72,21 @@ webapp.add_route( '/:controller/:action', action='index' ) webapp.add_route( '/:action', controller='repository', action='index' ) webapp.add_route( '/repos/*path_info', controller='hg', action='handle_request', path_info='/' ) - # Add the web API. + # Add the web API. # A good resource for RESTful services - http://routes.readthedocs.org/en/latest/restful.html webapp.add_api_controllers( 'galaxy.webapps.tool_shed.api', app ) - webapp.api_mapper.resource( 'content', - 'contents', - controller='repository_contents', + webapp.api_mapper.resource( 'repository', + 'repositories', + controller='repositories', + collection={ 'get_repository_and_revision' : 'GET' }, name_prefix='repository_', - path_prefix='/api/repositories/:repository_id', + path_prefix='/api', parent_resources=dict( member_name='repository', collection_name='repositories' ) ) - webapp.api_mapper.resource( 'content', - 'contents', - controller='repository_revision_contents', + webapp.api_mapper.resource( 'repository_revision', + 'repository_revisions', + controller='repository_revisions', name_prefix='repository_revision_', - path_prefix='/api/repository_revisions/:repository_metadata_id', + path_prefix='/api', parent_resources=dict( member_name='repository_revision', collection_name='repository_revisions' ) ) - webapp.api_mapper.resource( 'repository', 'repositories', path_prefix='/api' ) - webapp.api_mapper.resource( 'repository_revision', 'repository_revisions', path_prefix='/api' ) webapp.finalize_config() # Wrap the webapp in some useful middleware if kwargs.get( 'middleware', True ): diff -r 14ab08c0fbfe3e9735e94f4e399aeabc870fe4b2 -r d629a346b859c91ba0da27f865952580d1054d80 scripts/api/tool_shed_repository_revision_update.py --- a/scripts/api/tool_shed_repository_revision_update.py +++ /dev/null @@ -1,37 +0,0 @@ -#!/usr/bin/env python -""" -PUT/update script to update appropriate values in a repository_metadata table record in the Tool Shed. - -usage: tool_shed_repository_revision_update.py key url key1=value1 key2=value2 ... -""" - -import os, sys -sys.path.insert( 0, os.path.dirname( __file__ ) ) -from common import update - -import pkg_resources -pkg_resources.require( "simplejson" ) - -import simplejson - -to_json_string = simplejson.dumps -from_json_string = simplejson.loads - -data = {} -for key, value in [ kwarg.split( '=', 1 ) for kwarg in sys.argv[ 3: ] ]: - """ - This example script will properly handle updating the value of one or more of the following RepositoryMetadata attributes: - tools_functionally_correct, do_not_test, tool_test_errors - """ - if key in [ 'tools_functionally_correct', 'do_not_test' ]: - if str( value ).lower() in [ 'true', 'yes', 'on' ]: - new_value = True - else: - new_value = False - elif key in [ 'tool_test_errors' ]: - new_value = from_json_string( value ) - else: - new_value = str( value ) - data[ key ] = new_value - -update( sys.argv[ 1 ], sys.argv[ 2 ], data ) Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: inithello: Update install and test script to correctly load tool data tables.
by commits-noreply＠bitbucket.org 21 Mar '13

21 Mar '13

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/14ab08c0fbfe/ Changeset: 14ab08c0fbfe User: inithello Date: 2013-03-22 00:23:43 Summary: Update install and test script to correctly load tool data tables. Affected #: 1 file diff -r b79e2393c8ad8c88bda682e0767f572c0c09e26a -r 14ab08c0fbfe3e9735e94f4e399aeabc870fe4b2 test/install_and_test_tool_shed_repositories/functional_tests.py --- a/test/install_and_test_tool_shed_repositories/functional_tests.py +++ b/test/install_and_test_tool_shed_repositories/functional_tests.py @@ -107,12 +107,21 @@ else: additional_tool_data_tables = None additional_tool_data_path = None -# And set up a blank tool_data_table_conf.xml and shed_tool_data_table_conf.xml. + +# Also set up default tool data tables. +if os.path.exists( 'tool_data_table_conf.xml' ): + tool_data_table_conf = 'tool_data_table_conf.xml' +elif os.path.exists( 'tool_data_table_conf.xml.sample' ): + tool_data_table_conf = 'tool_data_table_conf.xml.sample' +else: + tool_data_table_conf = None + +# And set up a blank shed_tool_data_table_conf.xml. tool_data_table_conf_xml_template = '''<?xml version="1.0"?><tables></tables> ''' - + # The tool shed url and api key must be set for this script to work correctly. Additionally, if the tool shed url does not # point to one of the defaults, the GALAXY_INSTALL_TEST_TOOL_SHEDS_CONF needs to point to a tool sheds configuration file # that contains a definition for that tool shed. @@ -328,7 +337,7 @@ galaxy_test_proxy_port = None # Set up the configuration files for the Galaxy instance. shed_tool_data_table_conf_file = os.environ.get( 'GALAXY_INSTALL_TEST_SHED_TOOL_DATA_TABLE_CONF', os.path.join( galaxy_test_tmp_dir, 'test_shed_tool_data_table_conf.xml' ) ) - galaxy_tool_data_table_conf_file = os.environ.get( 'GALAXY_INSTALL_TEST_TOOL_DATA_TABLE_CONF', os.path.join( galaxy_test_tmp_dir, 'test_tool_data_table_conf.xml' ) ) + galaxy_tool_data_table_conf_file = os.environ.get( 'GALAXY_INSTALL_TEST_TOOL_DATA_TABLE_CONF', tool_data_table_conf ) galaxy_tool_conf_file = os.environ.get( 'GALAXY_INSTALL_TEST_TOOL_CONF', os.path.join( galaxy_test_tmp_dir, 'test_tool_conf.xml' ) ) galaxy_shed_tool_conf_file = os.environ.get( 'GALAXY_INSTALL_TEST_SHED_TOOL_CONF', os.path.join( galaxy_test_tmp_dir, 'test_shed_tool_conf.xml' ) ) galaxy_migrated_tool_conf_file = os.environ.get( 'GALAXY_INSTALL_TEST_MIGRATED_TOOL_CONF', os.path.join( galaxy_test_tmp_dir, 'test_migrated_tool_conf.xml' ) ) @@ -350,7 +359,7 @@ galaxy_file_path = os.path.join( galaxy_db_path, 'files' ) new_repos_path = tempfile.mkdtemp( dir=galaxy_test_tmp_dir ) galaxy_tempfiles = tempfile.mkdtemp( dir=galaxy_test_tmp_dir ) - galaxy_shed_tool_path = tempfile.mkdtemp( dir=galaxy_test_tmp_dir ) + galaxy_shed_tool_path = tempfile.mkdtemp( dir=galaxy_test_tmp_dir, prefix='shed_tools' ) galaxy_migrated_tool_path = tempfile.mkdtemp( dir=galaxy_test_tmp_dir ) galaxy_tool_dependency_dir = tempfile.mkdtemp( dir=galaxy_test_tmp_dir ) os.environ[ 'GALAXY_INSTALL_TEST_TOOL_DEPENDENCY_DIR' ] = galaxy_tool_dependency_dir @@ -367,9 +376,6 @@ print "Database connection: ", database_connection - # Generate the tool_data_table_conf.xml file. - file( galaxy_tool_data_table_conf_file, 'w' ).write( tool_data_table_conf_xml_template ) - os.environ[ 'GALAXY_INSTALL_TEST_TOOL_DATA_TABLE_CONF' ] = galaxy_tool_data_table_conf_file # Generate the shed_tool_data_table_conf.xml file. file( shed_tool_data_table_conf_file, 'w' ).write( tool_data_table_conf_xml_template ) os.environ[ 'GALAXY_INSTALL_TEST_SHED_TOOL_DATA_TABLE_CONF' ] = shed_tool_data_table_conf_file Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: inithello: Merge in bugfix from next-stable.
by commits-noreply＠bitbucket.org 21 Mar '13

21 Mar '13

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/3bbbcdb044af/ Changeset: 3bbbcdb044af Branch: next-stable User: inithello Date: 2013-03-21 19:54:42 Summary: Merge in bugfix from next-stable. Affected #: 27 files diff -r 24d068f7a44dab10f8574d15aa4063ccdb0edb22 -r 3bbbcdb044afdbd380d6cd4c0f721ab7262c5555 lib/galaxy/model/__init__.py --- a/lib/galaxy/model/__init__.py +++ b/lib/galaxy/model/__init__.py @@ -1516,6 +1516,11 @@ # other model classes. hda = self rval = dict( id = hda.id, + uuid = ( lambda uuid: str( uuid ) if uuid else None )( hda.dataset.uuid ), + history_id = hda.history.id, + hid = hda.hid, + file_ext = hda.ext, + peek = ( lambda hda: hda.display_peek() if hda.peek and hda.peek != 'no peek' else None )( hda ), model_class = self.__class__.__name__, name = hda.name, deleted = hda.deleted, diff -r 24d068f7a44dab10f8574d15aa4063ccdb0edb22 -r 3bbbcdb044afdbd380d6cd4c0f721ab7262c5555 lib/galaxy/web/base/controller.py --- a/lib/galaxy/web/base/controller.py +++ b/lib/galaxy/web/base/controller.py @@ -6,12 +6,16 @@ import os import re import pkg_resources +import urllib pkg_resources.require("SQLAlchemy >= 0.4") +pkg_resources.require( "Routes" ) +import routes from sqlalchemy import func, and_, select from paste.httpexceptions import HTTPBadRequest, HTTPInternalServerError, HTTPNotImplemented, HTTPRequestRangeNotSatisfiable from galaxy import util, web +from gettext import gettext from galaxy.datatypes.interval import ChromatinInteractions from galaxy.exceptions import ItemAccessibilityException, ItemDeletionException, ItemOwnershipException, MessageException from galaxy.security.validate_user_input import validate_publicname @@ -25,6 +29,11 @@ from galaxy.datatypes.data import Text +from galaxy.datatypes.display_applications import util as da_util +from galaxy.datatypes.metadata import FileParameter + +from galaxy.datatypes.display_applications.link_generator import get_display_app_link_generator + log = logging.getLogger( __name__ ) # States for passing messages @@ -322,6 +331,85 @@ return dataset.conversion_messages.PENDING return None + def get_hda_dict( self, trans, hda ): + hda_dict = hda.get_api_value( view='element' ) + history = hda.history + + # Add additional attributes that depend on trans can hence must be added here rather than at the model level. + if trans.user_is_admin() or trans.app.config.expose_dataset_path: + hda_dict[ 'file_name' ] = hda.file_name + + if not hda_dict[ 'deleted' ]: + hda_dict[ 'download_url' ] = url_for( 'history_contents_display', history_id = trans.security.encode_id( history.id ), + history_content_id = trans.security.encode_id( hda.id ) ) + + can_access_hda = trans.app.security_agent.can_access_dataset( trans.get_current_user_roles(), hda.dataset ) + hda_dict[ 'accessible' ] = ( trans.user_is_admin() or can_access_hda ) + hda_dict[ 'api_type' ] = "file" + + if not( hda.purged or hda.deleted or hda.dataset.purged ): + meta_files = [] + for meta_type in hda.metadata.spec.keys(): + if isinstance( hda.metadata.spec[ meta_type ].param, FileParameter ): + meta_files.append( dict( file_type=meta_type ) ) + if meta_files: + hda_dict[ 'meta_files' ] = meta_files + + hda_dict[ 'display_apps' ] = self.get_display_apps( trans, hda ) + hda_dict[ 'display_types' ] = self.get_old_display_applications( trans, hda ) + hda_dict[ 'visualizations' ] = hda.get_visualizations() + + if hda.creating_job and hda.creating_job.tool_id: + tool_used = trans.app.toolbox.get_tool( hda.creating_job.tool_id ) + if tool_used and tool_used.force_history_refresh: + hda_dict[ 'force_history_refresh' ] = True + + return trans.security.encode_dict_ids( hda_dict ) + + def get_display_apps( self, trans, hda ): + #TODO: make more straightforward (somehow) + display_apps = [] + + def get_display_app_url( display_app_link, hda, trans ): + web_url_for = routes.URLGenerator( trans.webapp.mapper, trans.environ ) + dataset_hash, user_hash = da_util.encode_dataset_user( trans, hda, None ) + return web_url_for( controller='dataset', + action="display_application", + dataset_id=dataset_hash, + user_id=user_hash, + app_name=urllib.quote_plus( display_app_link.display_application.id ), + link_name=urllib.quote_plus( display_app_link.id ) ) + + for display_app in hda.get_display_applications( trans ).itervalues(): + app_links = [] + for display_app_link in display_app.links.itervalues(): + app_links.append({ + 'target' : display_app_link.url.get( 'target_frame', '_blank' ), + 'href' : get_display_app_url( display_app_link, hda, trans ), + 'text' : gettext( display_app_link.name ) + }) + display_apps.append( dict( label=display_app.name, links=app_links ) ) + + return display_apps + + def get_old_display_applications( self, trans, hda ): + display_apps = [] + for display_app_name in hda.datatype.get_display_types(): + link_generator = get_display_app_link_generator( display_app_name ) + display_links = link_generator.generate_links( trans, hda ) + + app_links = [] + for display_name, display_link in display_links: + app_links.append({ + 'target' : '_blank', + 'href' : display_link, + 'text' : display_name + }) + if app_links: + display_apps.append( dict( label=hda.datatype.get_display_label( display_app_name ), links=app_links ) ) + + return display_apps + class UsesLibraryMixin: diff -r 24d068f7a44dab10f8574d15aa4063ccdb0edb22 -r 3bbbcdb044afdbd380d6cd4c0f721ab7262c5555 lib/galaxy/webapps/galaxy/api/datasets.py --- a/lib/galaxy/webapps/galaxy/api/datasets.py +++ b/lib/galaxy/webapps/galaxy/api/datasets.py @@ -50,8 +50,11 @@ elif data_type == 'genome_data': rval = self._get_genome_data( trans, dataset, kwd.get('dbkey', None) ) else: - # Default: return dataset as API value. - rval = dataset.get_api_value() + # Default: return dataset as dict. + if hda_ldda == 'hda': + rval = self.get_hda_dict( trans, dataset ) + else: + rval = dataset.get_api_value() except Exception, e: rval = "Error in dataset API at listing contents: " + str( e ) diff -r 24d068f7a44dab10f8574d15aa4063ccdb0edb22 -r 3bbbcdb044afdbd380d6cd4c0f721ab7262c5555 lib/galaxy/webapps/galaxy/api/history_contents.py --- a/lib/galaxy/webapps/galaxy/api/history_contents.py +++ b/lib/galaxy/webapps/galaxy/api/history_contents.py @@ -2,24 +2,12 @@ API operations on the contents of a history. """ import logging -import urllib -from gettext import gettext from galaxy import web from galaxy.web.base.controller import BaseAPIController, url_for from galaxy.web.base.controller import UsesHistoryDatasetAssociationMixin, UsesHistoryMixin from galaxy.web.base.controller import UsesLibraryMixin, UsesLibraryMixinItems -from galaxy.web.framework.helpers import to_unicode -from galaxy.datatypes.display_applications import util -from galaxy.datatypes.metadata import FileParameter - -from galaxy.datatypes.display_applications.link_generator import get_display_app_link_generator - -import pkg_resources -pkg_resources.require( "Routes" ) -import routes - log = logging.getLogger( __name__ ) class HistoryContentsController( BaseAPIController, UsesHistoryDatasetAssociationMixin, UsesHistoryMixin, @@ -67,7 +55,7 @@ if encoded_hda_id in ids: #TODO: share code with show try: - rval.append( get_hda_dict( trans, history, hda, for_editing=True ) ) + rval.append( self.get_hda_dict( trans, hda ) ) except Exception, exc: # don't fail entire list if hda err's, record and move on @@ -149,7 +137,7 @@ hda = self.get_history_dataset_association( trans, history, id, check_ownership=True, check_accessible=True ) - hda_dict = get_hda_dict( trans, history, hda, for_editing=True ) + hda_dict = self.get_hda_dict( trans, hda ) except Exception, e: msg = "Error in history API at listing dataset: %s" % ( str(e) ) @@ -175,107 +163,22 @@ if from_ld_id: try: ld = self.get_library_dataset( trans, from_ld_id, check_ownership=False, check_accessible=False ) - assert type( ld ) is trans.app.model.LibraryDataset, "Library content id ( %s ) is not a dataset" % from_ld_id + assert type( ld ) is trans.app.model.LibraryDataset, ( + "Library content id ( %s ) is not a dataset" % from_ld_id ) + except AssertionError, e: trans.response.status = 400 return str( e ) + except Exception, e: return str( e ) + hda = ld.library_dataset_dataset_association.to_history_dataset_association( history, add_to_history=True ) trans.sa_session.flush() return hda.get_api_value() + else: # TODO: implement other "upload" methods here. trans.response.status = 403 return "Not implemented." - -#TODO: move these into model -def get_hda_dict( trans, history, hda, for_editing ): - hda_dict = hda.get_api_value( view='element' ) - - hda_dict[ 'id' ] = trans.security.encode_id( hda.id ) - hda_dict[ 'history_id' ] = trans.security.encode_id( history.id ) - hda_dict[ 'hid' ] = hda.hid - - hda_dict[ 'file_ext' ] = hda.ext - if trans.user_is_admin() or trans.app.config.expose_dataset_path: - hda_dict[ 'file_name' ] = hda.file_name - - if not hda_dict[ 'deleted' ]: - hda_dict[ 'download_url' ] = url_for( 'history_contents_display', history_id = trans.security.encode_id( history.id ), history_content_id = trans.security.encode_id( hda.id ) ) - - can_access_hda = trans.app.security_agent.can_access_dataset( trans.get_current_user_roles(), hda.dataset ) - hda_dict[ 'accessible' ] = ( trans.user_is_admin() or can_access_hda ) - hda_dict[ 'api_type' ] = "file" - - if not( hda.purged or hda.deleted or hda.dataset.purged ): - meta_files = [] - for meta_type in hda.metadata.spec.keys(): - if isinstance( hda.metadata.spec[ meta_type ].param, FileParameter ): - meta_files.append( dict( file_type=meta_type ) ) - if meta_files: - hda_dict[ 'meta_files' ] = meta_files - - hda_dict[ 'display_apps' ] = get_display_apps( trans, hda ) - hda_dict[ 'display_types' ] = get_old_display_applications( trans, hda ) - - if hda.dataset.uuid is None: - hda_dict['uuid'] = None - else: - hda_dict['uuid'] = str(hda.dataset.uuid) - - hda_dict[ 'visualizations' ] = hda.get_visualizations() - if hda.peek and hda.peek != 'no peek': - hda_dict[ 'peek' ] = to_unicode( hda.display_peek() ) - - if hda.creating_job and hda.creating_job.tool_id: - tool_used = trans.app.toolbox.get_tool( hda.creating_job.tool_id ) - if tool_used and tool_used.force_history_refresh: - hda_dict[ 'force_history_refresh' ] = True - - return hda_dict - -def get_display_apps( trans, hda ): - #TODO: make more straightforward (somehow) - display_apps = [] - - def get_display_app_url( display_app_link, hda, trans ): - web_url_for = routes.URLGenerator( trans.webapp.mapper, trans.environ ) - dataset_hash, user_hash = util.encode_dataset_user( trans, hda, None ) - return web_url_for( controller='dataset', - action="display_application", - dataset_id=dataset_hash, - user_id=user_hash, - app_name=urllib.quote_plus( display_app_link.display_application.id ), - link_name=urllib.quote_plus( display_app_link.id ) ) - - for display_app in hda.get_display_applications( trans ).itervalues(): - app_links = [] - for display_app_link in display_app.links.itervalues(): - app_links.append({ - 'target' : display_app_link.url.get( 'target_frame', '_blank' ), - 'href' : get_display_app_url( display_app_link, hda, trans ), - 'text' : gettext( display_app_link.name ) - }) - display_apps.append( dict( label=display_app.name, links=app_links ) ) - - return display_apps - -def get_old_display_applications( trans, hda ): - display_apps = [] - for display_app_name in hda.datatype.get_display_types(): - link_generator = get_display_app_link_generator( display_app_name ) - display_links = link_generator.generate_links( trans, hda ) - - app_links = [] - for display_name, display_link in display_links: - app_links.append({ - 'target' : '_blank', - 'href' : display_link, - 'text' : display_name - }) - if app_links: - display_apps.append( dict( label=hda.datatype.get_display_label( display_app_name ), links=app_links ) ) - - return display_apps diff -r 24d068f7a44dab10f8574d15aa4063ccdb0edb22 -r 3bbbcdb044afdbd380d6cd4c0f721ab7262c5555 lib/galaxy/webapps/galaxy/controllers/tool_runner.py --- a/lib/galaxy/webapps/galaxy/controllers/tool_runner.py +++ b/lib/galaxy/webapps/galaxy/controllers/tool_runner.py @@ -312,4 +312,4 @@ Please do not use your browser\'s "stop" or "reload" buttons until the upload is complete, or it may be interrupted.You may safely continue to use Galaxy while the upload is in progress. Using "stop" and "reload" on pages other than Galaxy is also safe. """ - return trans.show_message( msg, refresh_frames='history' ) + return trans.show_message( msg ) diff -r 24d068f7a44dab10f8574d15aa4063ccdb0edb22 -r 3bbbcdb044afdbd380d6cd4c0f721ab7262c5555 lib/galaxy/webapps/tool_shed/api/repositories.py --- a/lib/galaxy/webapps/tool_shed/api/repositories.py +++ b/lib/galaxy/webapps/tool_shed/api/repositories.py @@ -1,24 +1,21 @@ +import logging import tool_shed.util.shed_util_common as suc from galaxy import web, util from galaxy.web.base.controller import BaseAPIController -from galaxy.web.framework.helpers import is_true - -import pkg_resources -pkg_resources.require( "Routes" ) -import routes -import logging log = logging.getLogger( __name__ ) + class RepositoriesController( BaseAPIController ): """RESTful controller for interactions with repositories in the Tool Shed.""" + @web.expose_api def index( self, trans, deleted=False, **kwd ): """ - GET /api/repository_revisions + GET /api/repositories Displays a collection (list) of repositories. """ - rval = [] + repository_dicts = [] deleted = util.string_as_bool( deleted ) try: query = trans.sa_session.query( trans.app.model.Repository ) \ @@ -28,30 +25,37 @@ for repository in query: value_mapper={ 'id' : trans.security.encode_id( repository.id ), 'user_id' : trans.security.encode_id( repository.user_id ) } - item = repository.get_api_value( view='collection', value_mapper=value_mapper ) - item[ 'url' ] = web.url_for( 'repository_contents', repository_id=trans.security.encode_id( repository.id ) ) - rval.append( item ) + repository_dict = repository.get_api_value( view='collection', value_mapper=value_mapper ) + repository_dict[ 'url' ] = web.url_for( controller='repository_contents', + action='index', + repository_id=trans.security.encode_id( repository.id ) ) + repository_dicts.append( repository_dict ) + return repository_dicts except Exception, e: message = "Error in the Tool Shed repositories API in index: %s" % str( e ) log.error( message, exc_info=True ) trans.response.status = 500 return message - return rval + @web.expose_api def show( self, trans, id, **kwd ): """ GET /api/repositories/{encoded_repository_id} Displays information about a repository in the Tool Shed. + + :param id: the encoded id of the `Repository` object """ try: repository = suc.get_repository_in_tool_shed( trans, id ) value_mapper={ 'id' : trans.security.encode_id( repository.id ), 'user_id' : trans.security.encode_id( repository.user_id ) } - repository_data = repository.get_api_value( view='element', value_mapper=value_mapper ) - repository_data[ 'contents_url' ] = web.url_for( 'repository_contents', repository_id=id ) + repository_dict = repository.get_api_value( view='element', value_mapper=value_mapper ) + repository_dict[ 'url' ] = web.url_for( controller='repository_contents', + action='index', + repository_id=trans.security.encode_id( repository.id ) ) + return repository_dict except Exception, e: message = "Error in the Tool Shed repositories API in show: %s" % str( e ) log.error( message, exc_info=True ) trans.response.status = 500 return message - return repository_data diff -r 24d068f7a44dab10f8574d15aa4063ccdb0edb22 -r 3bbbcdb044afdbd380d6cd4c0f721ab7262c5555 lib/galaxy/webapps/tool_shed/api/repository_contents.py --- a/lib/galaxy/webapps/tool_shed/api/repository_contents.py +++ b/lib/galaxy/webapps/tool_shed/api/repository_contents.py @@ -3,33 +3,31 @@ from galaxy import web from galaxy.web.base.controller import BaseAPIController -import pkg_resources -pkg_resources.require( "Routes" ) -import routes - log = logging.getLogger( __name__ ) + class RepositoryContentsController( BaseAPIController ): + @web.expose_api def index( self, trans, **kwd ): """ - GET /api/repositories/{encoded_repository_id}/contents + GET /api/repositories/{encoded_repository_id} Displays a collection (dictionary) of repository contents. - :param repository_id: an encoded id string of the `Repository` to inspect + :param repository_id: the encoded id of the `Repository` object """ - rval = [] - repository_id = kwd.get( 'repository_id', None ) try: + repository_id = kwd[ 'repository_id' ] repository = suc.get_repository_in_tool_shed( trans, repository_id ) - value_mapper={ 'id' : repository_id, + value_mapper={ 'id' : trans.security.encode_id( repository.id ), 'user_id' : trans.security.encode_id( repository.user_id ) } repository_dict = repository.as_dict( value_mapper ) - repository_dict[ 'url' ] = web.url_for( 'repository_contents', repository_id=repository_id ) - rval.append( repository_dict ) + repository_dict[ 'url' ] = web.url_for( controller='repository_contents', + action='index', + repository_id=repository_id ) + return repository_dict except Exception, e: message = "Error in the Tool Shed repository_contents API in index: %s" % str( e ) log.error( message, exc_info=True ) trans.response.status = 500 return message - return rval diff -r 24d068f7a44dab10f8574d15aa4063ccdb0edb22 -r 3bbbcdb044afdbd380d6cd4c0f721ab7262c5555 lib/galaxy/webapps/tool_shed/api/repository_revision_contents.py --- a/lib/galaxy/webapps/tool_shed/api/repository_revision_contents.py +++ b/lib/galaxy/webapps/tool_shed/api/repository_revision_contents.py @@ -1,14 +1,9 @@ import logging +from galaxy import web from galaxy.web.framework.helpers import time_ago -import tool_shed.util.shed_util_common as suc from tool_shed.util import metadata_util -from galaxy import web from galaxy.web.base.controller import BaseAPIController -import pkg_resources -pkg_resources.require( "Routes" ) -import routes - log = logging.getLogger( __name__ ) def default_value_mapper( trans, repository_metadata ): @@ -18,25 +13,27 @@ value_mapper[ 'time_last_tested' ] = time_ago( repository_metadata.time_last_tested ) return value_mapper + class RepositoryRevisionContentsController( BaseAPIController ): + @web.expose_api def index( self, trans, **kwd ): """ - GET /api/repository_revisions/{encoded_repository_metadata_id}/contents + GET /api/repository_revisions/{encoded_repository_metadata_id} Displays a collection (dictionary) of repository_metadata contents. - :param repository_metadata_id: an encoded id string of the `RepositoryMetadata` to inspect + :param repository_metadata_id: the encoded id of the `RepositoryMetadata` object """ - rval = [] - repository_metadata_id = kwd.get( 'repository_metadata_id', None ) try: + repository_metadata_id = kwd.get( 'repository_metadata_id', None ) repository_metadata = metadata_util.get_repository_metadata_by_id( trans, repository_metadata_id ) repository_dict = repository_metadata.as_dict( value_mapper=default_value_mapper( trans, repository_metadata ) ) - repository_dict[ 'url' ] = web.url_for( 'repository_revision_contents', repository_metadata_id=repository_metadata_id ) - rval.append( repository_dict ) + repository_dict[ 'url' ] = web.url_for( controller='repository_revision_contents', + action='index', + repository_metadata_id=repository_metadata_id ) + return repository_dict except Exception, e: - message = "Error in the Tool Shed repository_revision_contents API in index: %s" % str( e ) + message = "Error in the Tool Shed repository_revision_contents API in index: %s" % str( e ) log.error( message, exc_info=True ) trans.response.status = 500 return message - return rval diff -r 24d068f7a44dab10f8574d15aa4063ccdb0edb22 -r 3bbbcdb044afdbd380d6cd4c0f721ab7262c5555 lib/galaxy/webapps/tool_shed/api/repository_revisions.py --- a/lib/galaxy/webapps/tool_shed/api/repository_revisions.py +++ b/lib/galaxy/webapps/tool_shed/api/repository_revisions.py @@ -1,16 +1,9 @@ -import datetime +import datetime, logging from galaxy.web.framework.helpers import time_ago -import tool_shed.util.shed_util_common as suc from tool_shed.util import metadata_util from galaxy import web, util -from galaxy.model.orm import and_, or_ +from galaxy.model.orm import and_ from galaxy.web.base.controller import BaseAPIController -from galaxy.web.framework.helpers import is_true - -import pkg_resources -pkg_resources.require( "Routes" ) -import routes -import logging log = logging.getLogger( __name__ ) @@ -21,15 +14,17 @@ value_mapper[ 'time_last_tested' ] = time_ago( repository_metadata.time_last_tested ) return value_mapper + class RepositoryRevisionsController( BaseAPIController ): """RESTful controller for interactions with tool shed repository revisions.""" + @web.expose_api def index( self, trans, **kwd ): """ GET /api/repository_revisions Displays a collection (list) of repository revisions. """ - rval = [] + repository_metadata_dicts = [] # Build up an anded clause list of filters. clause_list = [] # Filter by downloadable if received. @@ -58,32 +53,40 @@ .order_by( trans.app.model.RepositoryMetadata.table.c.repository_id ) \ .all() for repository_metadata in query: - item = repository_metadata.get_api_value( view='collection', - value_mapper=default_value_mapper( trans, repository_metadata ) ) - item[ 'url' ] = web.url_for( 'repository_revision', id=trans.security.encode_id( repository_metadata.id ) ) - rval.append( item ) + repository_metadata_dict = repository_metadata.get_api_value( view='collection', + value_mapper=default_value_mapper( trans, repository_metadata ) ) + repository_metadata_dict[ 'url' ] = web.url_for( controller='repository_revision_contents', + action='index', + repository_metadata_id=trans.security.encode_id( repository_metadata.id ) ) + repository_metadata_dicts.append( repository_metadata_dict ) + return repository_metadata_dicts except Exception, e: - rval = "Error in the Tool Shed repository_revisions API in index: " + str( e ) - log.error( rval + ": %s" % str( e ) ) + message = "Error in the Tool Shed repository_revisions API in index: " + str( e ) + log.error( message, exc_info=True ) trans.response.status = 500 - return rval + return message + @web.expose_api def show( self, trans, id, **kwd ): """ GET /api/repository_revisions/{encoded_repository_metadata_id} Displays information about a repository_metadata record in the Tool Shed. + + :param id: the encoded id of the `RepositoryMetadata` object """ try: repository_metadata = metadata_util.get_repository_metadata_by_id( trans, id ) - repository_data = repository_metadata.get_api_value( view='element', - value_mapper=default_value_mapper( trans, repository_metadata ) ) - repository_data[ 'contents_url' ] = web.url_for( 'repository_revision_contents', repository_metadata_id=id ) + repository_metadata_dict = repository_metadata.as_dict( value_mapper=default_value_mapper( trans, repository_metadata ) ) + repository_metadata_dict[ 'url' ] = web.url_for( controller='repository_revision_contents', + action='index', + repository_metadata_id=trans.security.encode_id( repository_metadata.id ) ) + return repository_metadata_dict except Exception, e: message = "Error in the Tool Shed repository_revisions API in show: %s" % str( e ) log.error( message, exc_info=True ) trans.response.status = 500 return message - return repository_data + @web.expose_api def update( self, trans, payload, **kwd ): """ @@ -110,6 +113,8 @@ log.error( message, exc_info=True ) trans.response.status = 500 return message - item = repository_metadata.as_dict( value_mapper=default_value_mapper( trans, repository_metadata ) ) - item[ 'url' ] = web.url_for( 'repository_revision', id=repository_metadata_id ) - return [ item ] + repository_metadata_dict = repository_metadata.as_dict( value_mapper=default_value_mapper( trans, repository_metadata ) ) + repository_metadata_dict[ 'url' ] = web.url_for( controller='repository_revision_contents', + action='index', + repository_metadata_id=trans.security.encode_id( repository_metadata.id ) ) + return repository_metadata_dict diff -r 24d068f7a44dab10f8574d15aa4063ccdb0edb22 -r 3bbbcdb044afdbd380d6cd4c0f721ab7262c5555 lib/tool_shed/scripts/api/tool_shed_repository_revision_update.py --- /dev/null +++ b/lib/tool_shed/scripts/api/tool_shed_repository_revision_update.py @@ -0,0 +1,37 @@ +#!/usr/bin/env python +""" +PUT/update script to update appropriate values in a repository_metadata table record in the Tool Shed. + +usage: tool_shed_repository_revision_update.py key url key1=value1 key2=value2 ... +""" + +import os, sys +sys.path.insert( 0, os.path.dirname( __file__ ) ) +from common import update + +import pkg_resources +pkg_resources.require( "simplejson" ) + +import simplejson + +to_json_string = simplejson.dumps +from_json_string = simplejson.loads + +data = {} +for key, value in [ kwarg.split( '=', 1 ) for kwarg in sys.argv[ 3: ] ]: + """ + This example script will properly handle updating the value of one or more of the following RepositoryMetadata attributes: + tools_functionally_correct, do_not_test, tool_test_errors + """ + if key in [ 'tools_functionally_correct', 'do_not_test' ]: + if str( value ).lower() in [ 'true', 'yes', 'on' ]: + new_value = True + else: + new_value = False + elif key in [ 'tool_test_errors' ]: + new_value = from_json_string( value ) + else: + new_value = str( value ) + data[ key ] = new_value + +update( sys.argv[ 1 ], sys.argv[ 2 ], data ) diff -r 24d068f7a44dab10f8574d15aa4063ccdb0edb22 -r 3bbbcdb044afdbd380d6cd4c0f721ab7262c5555 test/casperjs/anon-history-tests.js --- a/test/casperjs/anon-history-tests.js +++ b/test/casperjs/anon-history-tests.js @@ -68,15 +68,10 @@ }); // ------------------------------------------------------------------- check the empty history for well formedness -// grab the history frame bounds for mouse later tests -spaceghost.then( function(){ - historyFrameInfo = this.getElementInfo( 'iframe[name="galaxy_history"]' ); - //this.debug( 'historyFrameInfo:' + this.jsonStr( historyFrameInfo ) ); -}); - spaceghost.thenOpen( spaceghost.baseUrl, function testPanelStructure(){ this.test.comment( 'history panel for anonymous user, new history' ); - this.withFrame( spaceghost.data.selectors.frames.history, function(){ + + this.withHistoryPanel( function(){ this.test.comment( "frame should have proper url and title: 'History'" ); this.test.assertMatch( this.getCurrentUrl(), /\/history/, 'Found history frame url' ); this.test.assertTitle( this.getTitle(), 'History', 'Found history frame title' ); @@ -103,15 +98,12 @@ 'Message contains "' + emptyMsgStr + '"' ); this.test.comment( 'name should have a tooltip with info on anon-user name editing' ); - // mouse over to find tooltip - this.historypanel.hoverOver( nameSelector, function testingHover(){ - this.test.assertExists( tooltipSelector, "Found tooltip after name hover" ); - this.test.assertSelectorHasText( tooltipSelector, anonNameTooltip ); - }, historyFrameInfo ); + this.historypanel.hoverOver( nameSelector ); + this.test.assertExists( tooltipSelector, "Found tooltip after name hover" ); + this.test.assertSelectorHasText( tooltipSelector, anonNameTooltip ); this.test.comment( 'name should NOT be editable when clicked by anon-user' ); - this.assertDoesntHaveClass( nameSelector, editableTextClass, - "Name field is not classed as editable text" ); + this.assertDoesntHaveClass( nameSelector, editableTextClass, "Name field is not classed as editable text" ); this.click( nameSelector ); this.test.assertDoesntExist( editableTextInput, "Clicking on name does not create an input" ); }); @@ -120,12 +112,13 @@ // ------------------------------------------------------------------- anon user can upload file spaceghost.then( function testAnonUpload(){ this.test.comment( 'anon-user should be able to upload files' ); + spaceghost.tools.uploadFile( filepathToUpload, function uploadCallback( _uploadInfo ){ - this.debug( 'uploaded HDA info: ' + this.jsonStr( _uploadInfo ) ); + this.debug( 'uploaded HDA info: ' + this.jsonStr( this.quickInfo( _uploadInfo.hdaElement ) ) ); var hasHda = _uploadInfo.hdaElement, hasClass = _uploadInfo.hdaElement.attributes[ 'class' ], hasOkClass = _uploadInfo.hdaElement.attributes[ 'class' ].indexOf( 'historyItem-ok' ) !== -1; - this.test.assert( ( hasHda && hasClass && hasOkClass ), "Uploaded file: " + _uploadInfo.name ); + this.test.assert( ( hasHda && hasClass && hasOkClass ), "Uploaded file: " + _uploadInfo.hdaElement.text ); testUploadInfo = _uploadInfo; }); }); @@ -134,19 +127,18 @@ this.test.assertNotVisible( emptyMsgSelector, 'Empty history message is not visible' ); }); - // ------------------------------------------------------------------- anon user can run tool on file // ------------------------------------------------------------------- anon user registers/logs in -> same history spaceghost.user.loginOrRegisterUser( email, password ); spaceghost.thenOpen( spaceghost.baseUrl, function(){ + this.test.comment( 'anon-user should login and be associated with previous history' ); - this.test.comment( 'anon-user should login and be associated with previous history' ); var loggedInAs = spaceghost.user.loggedInAs(); this.test.assert( loggedInAs === email, 'loggedInAs() matches email: "' + loggedInAs + '"' ); - this.withFrame( spaceghost.data.selectors.frames.history, function(){ - var hdaInfo = this.historypanel.hdaElementInfoByTitle( testUploadInfo.name, testUploadInfo.hid ); + this.historypanel.waitForHdas( function(){ + var hdaInfo = this.historypanel.hdaElementInfoByTitle( testUploadInfo.hdaElement.text ); this.test.assert( hdaInfo !== null, "After logging in - found a matching hda by name and hid" ); if( hdaInfo ){ this.test.assert( testUploadInfo.hdaElement.attributes.id === hdaInfo.attributes.id, @@ -155,17 +147,18 @@ }); }); +// ------------------------------------------------------------------- logs out -> new history spaceghost.user.logout(); spaceghost.thenOpen( spaceghost.baseUrl, function(){ this.test.comment( 'logging out should create a new, anonymous history' ); - this.withFrame( spaceghost.data.selectors.frames.history, function(){ + + this.historypanel.waitForHdas( function(){ this.test.assertSelectorHasText( nameSelector, unnamedName, 'History name is ' + unnamedName ); this.test.assertSelectorHasText( emptyMsgSelector, emptyMsgStr, 'Message contains "' + emptyMsgStr + '"' ); }); }); - // =================================================================== spaceghost.run( function(){ this.test.done(); diff -r 24d068f7a44dab10f8574d15aa4063ccdb0edb22 -r 3bbbcdb044afdbd380d6cd4c0f721ab7262c5555 test/casperjs/casperjs_runner.py --- a/test/casperjs/casperjs_runner.py +++ b/test/casperjs/casperjs_runner.py @@ -73,7 +73,7 @@ debug = False # bit of a hack - this is the beginning of the last string when capserjs --verbose=true --logLevel=debug # use this to get subprocess to stop waiting for output - casper_done_str = '# Tests complete' + casper_done_str = '# Stopping' # convert js test results to unittest.TestResults results_adapter = None #CasperJsonToUnittestResultsConverter() @@ -209,7 +209,8 @@ js_test_results = json.loads( results ) failures = js_test_results[ 'testResults' ][ 'failures' ] assert len( failures ) == 0, ( - "Some assertions failed in the headless browser tests (see the log for details)" ) + "%d assertions failed in the headless browser tests" %( len( failures ) ) + + " (see the log for details)" ) # ---------------------------------------------------------------- TestCase overrides def setUp( self ): @@ -299,10 +300,12 @@ # ==================================================================== TESTCASE EXAMPLE # these could be broken out into other py files - shouldn't be necc. ATM class Test_01_User( CasperJSTestCase ): - """TestCase that uses javascript and a headless browser to test dynamic pages. + """Tests for the Galaxy user centered functionality: + registration, login, etc. """ def test_10_registration( self ): - """User registration tests: register new user, logout, attempt bad registrations. + """User registration tests: + register new user, logout, attempt bad registrations. """ # all keywords will be compiled into a single JSON obj and passed to the server #self.run_js_script( 'registration-tests.js', @@ -331,7 +334,7 @@ class Test_03_HistoryPanel( CasperJSTestCase ): - """(Minimal) casperjs tests for tools. + """Tests for History fetching, rendering, and modeling. """ def test_00_history_panel( self ): """Test history panel basics (controls, structure, refresh, history options menu, etc.). @@ -339,13 +342,18 @@ self.run_js_script( 'history-panel-tests.js' ) def test_10_anonymous_histories( self ): + """Test history options button. + """ + self.run_js_script( 'history-options-tests.js' ) + + def test_20_anonymous_histories( self ): """Test history panel basics with an anonymous user. """ self.run_js_script( 'anon-history-tests.js' ) class Test_04_HDAs( CasperJSTestCase ): - """(Minimal) casperjs tests for tools. + """Tests for HistoryDatasetAssociation fetching, rendering, and modeling. """ def test_00_HDA_states( self ): """Test structure rendering of HDAs in all the possible HDA states diff -r 24d068f7a44dab10f8574d15aa4063ccdb0edb22 -r 3bbbcdb044afdbd380d6cd4c0f721ab7262c5555 test/casperjs/hda-state-tests.js --- a/test/casperjs/hda-state-tests.js +++ b/test/casperjs/hda-state-tests.js @@ -55,17 +55,10 @@ // start a new user spaceghost.user.loginOrRegisterUser( email, password ); -// grab the history frame bounds for later mouse tests -spaceghost.then( function(){ - historyFrameInfo = this.getElementInfo( 'iframe[name="galaxy_history"]' ); - //this.debug( 'historyFrameInfo:' + this.jsonStr( historyFrameInfo ) ); -}); - // upload a file spaceghost.then( function upload(){ spaceghost.tools.uploadFile( filepathToUpload, function uploadCallback( _uploadInfo ){ testUploadInfo = _uploadInfo; - this.info( 'testUploadInfo:' + this.jsonStr( testUploadInfo ) ); }); }); @@ -73,13 +66,12 @@ // =================================================================== TEST HELPERS //NOTE: to be called with fn.call( spaceghost, ... ) -function testTitle( hdaSelector, hid, name ){ - var titleSelector = hdaSelector + ' ' + this.historypanel.data.selectors.hda.title, - titleShouldBe = hid + ': ' + name; +function testTitle( hdaSelector, name ){ + var titleSelector = hdaSelector + ' ' + this.historypanel.data.selectors.hda.title; this.test.assertVisible( titleSelector, 'HDA title is visible' ); - this.test.assertSelectorHasText( titleSelector, titleShouldBe, - 'HDA has proper hid and title' ); + this.test.assertSelectorHasText( titleSelector, name, + 'HDA contains name (' + name + '): ' + this.fetchText( titleSelector ) ); } function testTitleButtonStructure( hdaSelector, shouldHaveTheseButtons ){ @@ -87,28 +79,9 @@ shouldHaveTheseButtons = shouldHaveTheseButtons || [ 'display', 'edit', 'delete' ]; var hdaDbId = this.getElementAttribute( hdaSelector, 'id' ).split( '-' )[1], - buttonsArea = hdaSelector + ' ' + this.historypanel.data.selectors.hda.titleButtons, - buttons = { - // this seems backwards -> TODO: move buttonsArea concat into loop below, move this data to historypanel.data - display : { - nodeName : this.historypanel.data.text.hda.ok.nodeNames.displayButton, - selector : buttonsArea + ' ' + this.historypanel.data.selectors.hda.displayButton, - tooltip : this.historypanel.data.text.hda.ok.tooltips.displayButton, - hrefTpl : this.historypanel.data.text.hda.ok.hrefs.displayButton - }, - edit : { - nodeName : this.historypanel.data.text.hda.ok.nodeNames.editAttrButton, - selector : buttonsArea + ' ' + this.historypanel.data.selectors.hda.editAttrButton, - tooltip : this.historypanel.data.text.hda.ok.tooltips.editAttrButton, - hrefTpl : this.historypanel.data.text.hda.ok.hrefs.editAttrButton - }, - 'delete' : { - nodeName : this.historypanel.data.text.hda.ok.nodeNames.deleteButton, - selector : buttonsArea + ' ' + this.historypanel.data.selectors.hda.deleteButton, - tooltip : this.historypanel.data.text.hda.ok.tooltips.deleteButton, - hrefTpl : this.historypanel.data.text.hda.ok.hrefs.deleteButton - } - }; + buttonsArea = hdaSelector + ' ' + this.historypanel.data.selectors.hda.titleButtonArea, + buttons = this.historypanel.data.hdaTitleButtons; + this.test.assertVisible( buttonsArea, 'Button area is visible' ); for( var i=0; i<shouldHaveTheseButtons.length; i++ ){ @@ -123,7 +96,7 @@ this.test.assertVisible( button.selector, buttonName + ' button is visible' ); var buttonElement = this.getElementInfo( button.selector ); - this.debug( 'buttonElement:' + this.jsonStr( buttonElement ) ); + this.debug( 'buttonElement:' + this.jsonStr( this.quickInfo( buttonElement ) ) ); // should be an anchor this.test.assert( buttonElement.nodeName === button.nodeName, @@ -135,12 +108,13 @@ this.assertTextContains( href, hrefShouldBe, buttonName + ' has proper href (' + hrefShouldBe + '): ' + href ); - this.historypanel.hoverOver( button.selector, function testingHover(){ - var tooltipText = button.tooltip; - this.test.assertVisible( tooltipSelector, buttonName + ' button tooltip is visible when hovering' ); - this.test.assertSelectorHasText( tooltipSelector, tooltipText, - buttonName + ' button has tooltip text: "' + tooltipText + '"' ); - }, historyFrameInfo ); + this.historypanel.hoverOver( button.selector ); + var tooltipText = button.tooltip; + this.test.assertVisible( tooltipSelector, buttonName + ' button tooltip is visible when hovering' ); + this.test.assertSelectorHasText( tooltipSelector, tooltipText, + buttonName + ' button has tooltip text: "' + tooltipText + '"' ); + // clear the tooltip + this.page.sendEvent( 'mouseover', 0, 0 ); } } @@ -185,6 +159,7 @@ this.test.comment( 'Primary action buttons div should exist and be visible' ); this.test.assertExists( buttonsSelector, 'Primary action buttons div exists' ); this.test.assertVisible( buttonsSelector, 'Primary action buttons div is visible' ); + //TODO: ... } function testSecondaryActionButtons( hdaSelector ){ @@ -193,6 +168,7 @@ this.test.comment( 'Secondary action buttons div should exist and be visible' ); this.test.assertExists( buttonsSelector, 'Secondary action buttons div exists' ); this.test.assertVisible( buttonsSelector, 'Secondary action buttons div is visible' ); + //TODO: ... } function testPeek( hdaSelector, expectedPeekArray ){ @@ -239,112 +215,97 @@ // =================================================================== TESTS // ------------------------------------------------------------------- ok state -spaceghost.then( function checkOkState(){ +spaceghost.withHistoryPanel( function(){ this.test.comment( 'HDAs in the "ok" state should be well formed' ); - this.withFrame( spaceghost.data.selectors.frames.history, function(){ - var uploadSelector = '#' + testUploadInfo.hdaElement.attributes.id; - this.test.assertVisible( uploadSelector, 'HDA is visible' ); + var uploadSelector = '#' + testUploadInfo.hdaElement.attributes.id; + this.test.assertVisible( uploadSelector, 'HDA is visible' ); - this.test.comment( 'should have the proper state class' ); - this.assertHasClass( uploadSelector, this.historypanel.data.selectors.hda.wrapper.stateClasses.ok, - 'HDA has ok state class' ); + this.test.comment( 'should have the proper state class' ); + this.assertHasClass( uploadSelector, this.historypanel.data.selectors.hda.wrapper.stateClasses.ok, + 'HDA has ok state class' ); - // since we're using css there's no great way to test state icon (.state-icon is empty) + // since we're using css there's no great way to test state icon (.state-icon is empty) - this.test.comment( 'should have proper title and hid' ); - testTitle.call( spaceghost, uploadSelector, testUploadInfo.hid, testUploadInfo.name ); + this.test.comment( 'should have proper title and hid' ); + testTitle.call( spaceghost, uploadSelector, testUploadInfo.filename ); - this.test.comment( 'should have all of the three, main buttons' ); - testTitleButtonStructure.call( spaceghost, uploadSelector ); + this.test.comment( 'should have all of the three, main buttons' ); + testTitleButtonStructure.call( spaceghost, uploadSelector ); - this.test.comment( 'body is not visible before clicking the hda title' ); - var body = uploadSelector + ' ' + this.historypanel.data.selectors.hda.body; - this.test.assertNotVisible( body, 'body is not visible' ); + this.test.comment( 'body is not visible before clicking the hda title' ); + var body = uploadSelector + ' ' + this.historypanel.data.selectors.hda.body; + this.test.assertNotVisible( body, 'body is not visible' ); - this.test.comment( 'clicking the hda title should expand its body' ); - var hdaTitle = uploadSelector + ' ' + this.historypanel.data.selectors.hda.title; - this.click( hdaTitle ); - this.wait( 500, function(){ + this.test.comment( 'clicking the hda title should expand its body' ); + this.historypanel.thenExpandHda( uploadSelector, function(){ + // ugh. + this.jumpToHistory( function(){ testExpandedBody.call( spaceghost, uploadSelector, summaryShouldBeArray, infoShouldBe, false ); }); }); }); +// restore to collapsed +spaceghost.then( function(){ + this.test.comment( "Collapsing hda in 'ok' state should hide body again" ); + var uploadSelector = '#' + testUploadInfo.hdaElement.attributes.id; -// restore to collapsed -spaceghost.then( function collapseOkState(){ - this.test.comment( "Collapsing hda in 'ok' state should hide body again" ); - this.withFrame( spaceghost.data.selectors.frames.history, function(){ - var uploadSelector = '#' + testUploadInfo.hdaElement.attributes.id, - hdaTitle = uploadSelector + ' ' + this.historypanel.data.selectors.hda.title; - body = uploadSelector + ' ' + this.historypanel.data.selectors.hda.body; - - this.click( hdaTitle ); - this.wait( 500, function(){ - this.test.assertNotVisible( body, 'body is not visible' ); - }); + spaceghost.historypanel.thenCollapseHda( uploadSelector, function collapseOkState(){ + this.test.assertNotVisible( uploadSelector + ' ' + this.historypanel.data.selectors.hda.body, + 'body is not visible' ); }); }); +// ------------------------------------------------------------------- new state +spaceghost.withHistoryPanel( function(){ + // set state directly through model, wait for re-render + //TODO: not ideal to test this + this.evaluate( function(){ + return Galaxy.currHistoryPanel.model.hdas.at( 0 ).set( 'state', 'new' ); + }); + this.wait( 1000, function(){ + this.test.comment( 'HDAs in the "new" state should be well formed' ); -// ------------------------------------------------------------------- new state -spaceghost.then( function checkNewState(){ - this.test.comment( 'HDAs in the "new" state should be well formed' ); + var uploadSelector = '#' + testUploadInfo.hdaElement.attributes.id; + this.test.assertVisible( uploadSelector, 'HDA is visible' ); - this.withFrame( spaceghost.data.selectors.frames.history, function(){ - // set state directly through model - //TODO: not ideal - this.evaluate( function(){ - return Galaxy.currHistoryPanel.model.hdas.at( 0 ).set( 'state', 'new' ); - }); - // wait for re-render - this.wait( 500, function(){ - var uploadSelector = '#' + testUploadInfo.hdaElement.attributes.id; - this.test.assertVisible( uploadSelector, 'HDA is visible' ); + // should have proper title and hid + testTitle.call( spaceghost, uploadSelector, testUploadInfo.filename ); - // should have proper title and hid - testTitle.call( spaceghost, uploadSelector, testUploadInfo.hid, testUploadInfo.name ); + this.test.comment( 'new HDA should have the new state class' ); + this.assertHasClass( uploadSelector, this.historypanel.data.selectors.hda.wrapper.stateClasses['new'], + 'HDA has new state class' ); - this.test.comment( 'new HDA should have the new state class' ); - this.assertHasClass( uploadSelector, this.historypanel.data.selectors.hda.wrapper.stateClasses['new'], - 'HDA has new state class' ); + this.test.comment( 'new HDA should NOT have any of the three, main buttons' ); + var buttonSelector = uploadSelector + ' ' + this.historypanel.data.selectors.hda.titleButtons + ' a'; + this.test.assertDoesntExist( buttonSelector, 'No display, edit, or delete buttons' ); - this.test.comment( 'new HDA should NOT have any of the three, main buttons' ); - var buttonSelector = uploadSelector + ' ' + this.historypanel.data.selectors.hda.titleButtons + ' a'; - this.test.assertDoesntExist( buttonSelector, 'No display, edit, or delete buttons' ); + this.test.comment( 'clicking the title of the new HDA will expand the body' ); - this.test.comment( 'clicking the title of the new HDA will expand the body' ); - var hdaTitle = uploadSelector + ' ' + this.historypanel.data.selectors.hda.title; - this.click( hdaTitle ); - this.wait( 500, function(){ - var bodySelector = uploadSelector + ' ' + this.historypanel.data.selectors.hda.body; - this.test.assertVisible( bodySelector, 'HDA body is visible (after expanding)' ); + this.historypanel.thenExpandHda( uploadSelector, function(){ + var bodySelector = uploadSelector + ' ' + this.historypanel.data.selectors.hda.body; + this.test.assertVisible( bodySelector, 'HDA body is visible (after expanding)' ); - var expectedBodyText = 'This is a new dataset'; - this.test.comment( 'the body should have the text: ' + expectedBodyText ); - this.test.assertSelectorHasText( bodySelector, expectedBodyText, - 'HDA body has text: ' + expectedBodyText ); - - // restore to collapsed - this.click( hdaTitle ); - }); + var expectedBodyText = 'This is a new dataset'; + this.test.comment( 'the body should have the text: ' + expectedBodyText ); + this.test.assertSelectorHasText( bodySelector, expectedBodyText, + 'HDA body has text: ' + expectedBodyText ); }); }); }); // restore state, collapse -spaceghost.then( function revertStateAndCollapse(){ - this.withFrame( spaceghost.data.selectors.frames.history, function(){ +spaceghost.withHistoryPanel( function revertStateAndCollapse(){ + var uploadSelector = '#' + testUploadInfo.hdaElement.attributes.id; + + this.historypanel.thenCollapseHda( uploadSelector, function(){ this.evaluate( function(){ return Galaxy.currHistoryPanel.model.hdas.at( 0 ).set( 'state', 'ok' ); }); - this.wait( 500, function(){ - var hdaTitle = '#' + testUploadInfo.hdaElement.attributes.id - + ' ' + this.historypanel.data.selectors.hda.title; - this.click( hdaTitle ); - }); }); + this.wait( 1000 ); }); - +/* +*/ // =================================================================== spaceghost.run( function(){ diff -r 24d068f7a44dab10f8574d15aa4063ccdb0edb22 -r 3bbbcdb044afdbd380d6cd4c0f721ab7262c5555 test/casperjs/history-options-tests.js --- /dev/null +++ b/test/casperjs/history-options-tests.js @@ -0,0 +1,145 @@ +// have to handle errors here - or phantom/casper won't bail but _HANG_ +try { + var utils = require( 'utils' ), + xpath = require( 'casper' ).selectXPath, + format = utils.format, + + //...if there's a better way - please let me know, universe + scriptDir = require( 'system' ).args[3] + // remove the script filename + .replace( /[\w|\.|\-|_]*$/, '' ) + // if given rel. path, prepend the curr dir + .replace( /^(?!\/)/, './' ), + spaceghost = require( scriptDir + 'spaceghost' ).create({ + // script options here (can be overridden by CLI) + //verbose: true, + //logLevel: debug, + scriptDir: scriptDir + }); + + spaceghost.start(); + +} catch( error ){ + console.debug( error ); + phantom.exit( 1 ); +} + +// =================================================================== +/* TODO: + possibly break this file up +*/ +// =================================================================== globals and helpers +var email = spaceghost.user.getRandomEmail(), + password = '123456'; +if( spaceghost.fixtureData.testUser ){ + email = spaceghost.fixtureData.testUser.email; + password = spaceghost.fixtureData.testUser.password; + spaceghost.info( 'Will use fixtureData.testUser: ' + email ); +} + +// selectors and labels +var includeDeletedOptionsLabel = spaceghost.historyoptions.data.labels.options.includeDeleted; + +// local +var filepathToUpload = '../../test-data/1.txt', + testUploadInfo = {}; + + +// =================================================================== TESTS +// ------------------------------------------------------------------- set up +// start a new user +spaceghost.user.loginOrRegisterUser( email, password ); + +spaceghost.tools.uploadFile( filepathToUpload, function uploadCallback( _uploadInfo ){ + testUploadInfo = _uploadInfo; +}); + +// ------------------------------------------------------------------- history options menu structure +//NOTE: options menu should be functionally tested elsewhere +spaceghost.historypanel.waitForHdas().then( function checkHistoryOptions(){ + this.test.comment( 'History options icon should be in place and menu should have the proper structure' ); + + // check the button and icon + this.test.assertExists( this.historyoptions.data.selectors.button, "Found history options button" ); + this.test.assertVisible( this.historyoptions.data.selectors.button, "History options button is visible" ); + this.test.assertVisible( this.historyoptions.data.selectors.buttonIcon, "History options icon is visible" ); + + // open the menu + this.click( this.historyoptions.data.selectors.button ); + this.test.assertVisible( this.historyoptions.data.selectors.menu, + "Menu is visible when options button is clicked" ); + + // check the options + var historyOptions = this.historyoptions.data.labels.options; + for( var optionKey in historyOptions ){ + if( historyOptions.hasOwnProperty( optionKey ) ){ + var optionLabel = historyOptions[ optionKey ]; + this.test.assertVisible( this.historyoptions.data.selectors.optionXpathByLabelFn( optionLabel ), + 'Option label is visible: ' + optionLabel ); + } + } + + // clear the menu + this.click( 'body' ); + this.test.assertNotVisible( this.historyoptions.data.selectors.menu, + "Clicking away from the menu closes it" ); +}); + +// ------------------------------------------------------------------- options allow showing/hiding deleted hdas +spaceghost.then( function(){ + this.test.comment( 'Deleting HDA' ); + var uploadSelector = '#' + testUploadInfo.hdaElement.attributes.id; + + this.historypanel.deleteHda( uploadSelector, function(){ + this.test.assertNotExists( uploadSelector, "Deleted HDA is NOT in the DOM" ); + }); +}); + +spaceghost.then( function(){ + this.test.comment( 'History options->' + includeDeletedOptionsLabel + ' shows deleted datasets' ); + var uploadSelector = '#' + testUploadInfo.hdaElement.attributes.id; + + this.historyoptions.includeDeleted( function(){ + this.test.assertExists( uploadSelector, + "Deleted HDA is in the DOM (using history options -> " + includeDeletedOptionsLabel + ")" ); + this.test.assertVisible( uploadSelector, + "Deleted HDA is visible again (using history options -> " + includeDeletedOptionsLabel + ")" ); + }); +}); + +spaceghost.then( function(){ + this.test.comment( 'History options->' + includeDeletedOptionsLabel + ' (again) re-hides deleted datasets' ); + + this.historyoptions.excludeDeleted( function(){ + this.test.assertDoesntExist( '#' + testUploadInfo.hdaElement.attributes.id, + "Deleted HDA is not in the DOM (using history options -> " + includeDeletedOptionsLabel + ")" ); + }); + // undelete the uploaded file + this.historypanel.undeleteHda( '#' + testUploadInfo.hdaElement.attributes.id ); +}); + +// ------------------------------------------------------------------- hidden hdas aren't shown +// ------------------------------------------------------------------- history options allows showing hidden hdas +// can't test this yet w/o a way to make hdas hidden thru the ui or api + + +// ------------------------------------------------------------------- history options collapses all expanded hdas +spaceghost.then( function(){ + this.historypanel.thenExpandHda( '#' + testUploadInfo.hdaElement.attributes.id ); +}); +spaceghost.then( function(){ + this.test.comment( 'History option collapses all expanded hdas' ); + + this.historyoptions.collapseExpanded( function(){ + var uploadedSelector = '#' + testUploadInfo.hdaElement.attributes.id; + this.withHistoryPanel( function(){ + this.test.assertNotVisible( uploadedSelector + ' ' + this.historypanel.data.selectors.hda.body, + "Body for uploaded file is not visible" ); + }); + }); +}); + +// =================================================================== +spaceghost.run( function(){ + this.test.done(); +}); diff -r 24d068f7a44dab10f8574d15aa4063ccdb0edb22 -r 3bbbcdb044afdbd380d6cd4c0f721ab7262c5555 test/casperjs/history-panel-tests.js --- a/test/casperjs/history-panel-tests.js +++ b/test/casperjs/history-panel-tests.js @@ -72,177 +72,164 @@ // ------------------------------------------------------------------- set up // start a new user spaceghost.user.loginOrRegisterUser( email, password ); -//??: why is a reload needed here? If we don't, loggedInAs === '' ... -spaceghost.thenOpen( spaceghost.baseUrl, function(){ - var loggedInAs = spaceghost.user.loggedInAs(); - this.test.assert( loggedInAs === email, 'loggedInAs() matches email: "' + loggedInAs + '"' ); + +// ------------------------------------------------------------------- check structure of empty history +spaceghost.thenOpen( spaceghost.baseUrl ).historypanel.waitForHdas( function(){ + this.test.comment( 'history panel with a new, empty history should be well formed' ); + this.test.comment( "frame should have proper url and title: 'History'" ); + this.test.assertMatch( this.getCurrentUrl(), /\/history/, 'Found history frame url' ); + this.test.assertTitle( this.getTitle(), 'History', 'Found history frame title' ); + + this.test.comment( "history name should exist, be visible, and have text " + unnamedName ); + this.test.assertExists( nameSelector, nameSelector + ' exists' ); + this.test.assertVisible( nameSelector, 'History name is visible' ); + this.test.assertSelectorHasText( nameSelector, unnamedName, 'History name is ' + unnamedName ); + + this.test.comment( "history subtitle should display size and size should be: " + initialSizeStr ); + this.test.assertExists( subtitleSelector, 'Found ' + subtitleSelector ); + this.test.assertVisible( subtitleSelector, 'History subtitle is visible' ); + this.test.assertSelectorHasText( subtitleSelector, initialSizeStr, + 'History subtitle has "' + initialSizeStr + '"' ); + + this.test.comment( "tags and annotation icons should be available" ); + this.test.assertExists( tagIconSelector, 'Tag icon button found' ); + this.test.assertExists( annoIconSelector, 'Annotation icon button found' ); + + this.test.comment( "A message about the current history being empty should be displayed" ); + this.test.assertExists( emptyMsgSelector, emptyMsgSelector + ' exists' ); + this.test.assertVisible( emptyMsgSelector, 'Empty history message is visible' ); + this.test.assertSelectorHasText( emptyMsgSelector, emptyMsgStr, + 'Message contains "' + emptyMsgStr + '"' ); }); -// grab the history frame bounds for later mouse tests -spaceghost.then( function(){ - historyFrameInfo = this.getElementInfo( 'iframe[name="galaxy_history"]' ); - //this.debug( 'historyFrameInfo:' + this.jsonStr( historyFrameInfo ) ); -}); +// ------------------------------------------------------------------- name editing +spaceghost.withHistoryPanel( function(){ + this.test.comment( 'history panel, editing the history name' ); -// ------------------------------------------------------------------- check structure of empty history -spaceghost.thenOpen( spaceghost.baseUrl, function testPanelStructure(){ - this.test.comment( 'history panel, new history' ); - this.withFrame( spaceghost.data.selectors.frames.history, function(){ - this.test.comment( "frame should have proper url and title: 'History'" ); - this.test.assertMatch( this.getCurrentUrl(), /\/history/, 'Found history frame url' ); - this.test.assertTitle( this.getTitle(), 'History', 'Found history frame title' ); + this.test.comment( 'name should have a tooltip with proper info on name editing' ); + this.historypanel.hoverOver( nameSelector ); + this.test.assertExists( tooltipSelector, "Found tooltip after name hover" ); + this.test.assertSelectorHasText( tooltipSelector, nameTooltip ); + // clear the tooltip + this.page.sendEvent( 'mousemove', -1, -1 ); - this.test.comment( "history name should exist, be visible, and have text " + unnamedName ); - this.test.assertExists( nameSelector, nameSelector + ' exists' ); - this.test.assertVisible( nameSelector, 'History name is visible' ); - this.test.assertSelectorHasText( nameSelector, unnamedName, 'History name is ' + unnamedName ); + this.test.comment( 'name should be create an input when clicked' ); + this.assertHasClass( nameSelector, editableTextClass, "Name field classed for editable text" ); + this.click( nameSelector ); + this.test.assertExists( editableTextInput, "Clicking on name creates an input" ); - this.test.comment( "history subtitle should display size and size should be: " + initialSizeStr ); - this.test.assertExists( subtitleSelector, 'Found ' + subtitleSelector ); - this.test.assertVisible( subtitleSelector, 'History subtitle is visible' ); - this.test.assertSelectorHasText( subtitleSelector, initialSizeStr, - 'History subtitle has "' + initialSizeStr + '"' ); - - this.test.comment( "tags and annotation icons should be available" ); - this.test.assertExists( tagIconSelector, 'Tag icon button found' ); - this.test.assertExists( annoIconSelector, 'Annotation icon button found' ); - - this.test.comment( "A message about the current history being empty should be displayed" ); - this.test.assertExists( emptyMsgSelector, emptyMsgSelector + ' exists' ); - this.test.assertVisible( emptyMsgSelector, 'Empty history message is visible' ); - this.test.assertSelectorHasText( emptyMsgSelector, emptyMsgStr, - 'Message contains "' + emptyMsgStr + '"' ); + this.test.comment( 'name should be editable by entering keys and pressing enter' ); + //NOTE: casperjs.sendKeys adds a click before and a selector.blur after sending - won't work here + this.page.sendEvent( 'keypress', newHistoryName ); + this.page.sendEvent( 'keypress', this.page.event.key.Enter ); + // wait for send and re-render name + this.wait( 1000, function(){ + this.test.assertSelectorHasText( nameSelector, newHistoryName, 'History name is ' + newHistoryName ); + this.test.assertDoesntExist( editableTextInput, "Input disappears after pressing enter" ); }); }); -// ------------------------------------------------------------------- name editing -spaceghost.then( function(){ - this.test.comment( 'history panel, editing the history name' ); - this.withFrame( spaceghost.data.selectors.frames.history, function(){ - this.test.comment( 'name should have a tooltip with proper info on name editing' ); - var nameInfo = this.getElementInfo( nameSelector ); - this.page.sendEvent( 'mousemove', historyFrameInfo.x + nameInfo.x + 1, historyFrameInfo.y + nameInfo.y + 1 ); - this.test.assertExists( tooltipSelector, "Found tooltip after name hover" ); - this.test.assertSelectorHasText( tooltipSelector, nameTooltip ); +spaceghost.withHistoryPanel( function(){ + this.test.comment( 'name should revert if user clicks away while editing' ); - this.test.comment( 'name should be create an input when clicked' ); - this.assertHasClass( nameSelector, editableTextClass, "Name field classed for editable text" ); - this.click( nameSelector ); - this.test.assertExists( editableTextInput, "Clicking on name creates an input" ); + this.click( nameSelector ); + this.page.sendEvent( 'keypress', "Woodchipper metagenomics, Fargo, ND" ); - this.test.comment( 'name should be editable by entering keys and pressing enter' ); - //NOTE: casperjs.sendKeys adds a click before and a selector.blur after sending - won't work here - this.page.sendEvent( 'keypress', newHistoryName ); - this.page.sendEvent( 'keypress', this.page.event.key.Enter ); - this.wait( 1000, function(){ - this.test.assertSelectorHasText( nameSelector, newHistoryName, 'History name is ' + newHistoryName ); - this.test.assertDoesntExist( editableTextInput, "Input disappears after pressing enter" ); - }); + this.page.sendEvent( 'mousedown', -1, -1 ); + this.wait( 1000, function(){ + this.test.assertSelectorHasText( nameSelector, newHistoryName, 'History name is STILL ' + newHistoryName ); + this.test.assertDoesntExist( editableTextInput, "Input disappears after clicking away" ); }); }); -spaceghost.then( function(){ - this.withFrame( spaceghost.data.selectors.frames.history, function(){ - this.test.comment( 'name should revert if user clicks away while editing' ); - this.click( nameSelector ); - this.page.sendEvent( 'keypress', "Woodchipper metagenomics, Fargo, ND" ); - // click above the name input element - var inputInfo = this.getElementInfo( editableTextInput ); - this.page.sendEvent( 'mousedown', historyFrameInfo.x + inputInfo.x + 1, historyFrameInfo.y + inputInfo.y - 5 ); +spaceghost.withHistoryPanel( function(){ + this.test.comment( 'name should revert if user hits ESC while editing' ); - this.wait( 1000, function(){ - this.test.assertSelectorHasText( nameSelector, newHistoryName, 'History name is STILL ' + newHistoryName ); - this.test.assertDoesntExist( editableTextInput, "Input disappears after clicking away" ); - }); - }); -}); -spaceghost.then( function(){ - this.withFrame( spaceghost.data.selectors.frames.history, function(){ - this.test.comment( 'name should revert if user hits ESC while editing' ); - this.click( nameSelector ); - this.page.sendEvent( 'keypress', "Arsenic Bacteria" ); + this.click( nameSelector ); + this.page.sendEvent( 'keypress', "Arsenic Bacteria" ); - this.page.sendEvent( 'keypress', this.page.event.key.Escape ); - this.wait( 1000, function(){ - this.test.assertSelectorHasText( nameSelector, newHistoryName, 'History name is STILL ' + newHistoryName ); - this.test.assertDoesntExist( editableTextInput, "Input disappears after hitting ESC" ); - }); + this.page.sendEvent( 'keypress', this.page.event.key.Escape ); + this.wait( 1000, function(){ + this.test.assertSelectorHasText( nameSelector, newHistoryName, 'History name is STILL ' + newHistoryName ); + this.test.assertDoesntExist( editableTextInput, "Input disappears after hitting ESC" ); }); }); // ------------------------------------------------------------------- check structure of NON empty history // upload file: 1.txt -spaceghost.then( function upload(){ +spaceghost.tools.uploadFile( filepathToUpload, function uploadCallback( _uploadInfo ){ this.test.comment( 'uploaded file should appear in history' ); - spaceghost.tools.uploadFile( filepathToUpload, function uploadCallback( _uploadInfo ){ - this.debug( 'uploaded HDA info: ' + this.jsonStr( _uploadInfo ) ); - var hasHda = _uploadInfo.hdaElement, - hasClass = _uploadInfo.hdaElement.attributes[ 'class' ], - hasOkClass = _uploadInfo.hdaElement.attributes[ 'class' ].indexOf( wrapperOkClassName ) !== -1; - this.test.assert( ( hasHda && hasClass && hasOkClass ), "Uploaded file: " + _uploadInfo.name ); - testUploadInfo = _uploadInfo; - }); + + this.debug( 'uploaded HDA info: ' + this.jsonStr( _uploadInfo ) ); + var hasHda = _uploadInfo.hdaElement, + hasClass = _uploadInfo.hdaElement.attributes[ 'class' ], + hasOkClass = _uploadInfo.hdaElement.attributes[ 'class' ].indexOf( wrapperOkClassName ) !== -1; + this.test.assert( ( hasHda && hasClass && hasOkClass ), "Uploaded file: " + _uploadInfo.name ); + testUploadInfo = _uploadInfo; }); -spaceghost.then( function checkPanelStructure(){ +spaceghost.withHistoryPanel( function checkPanelStructure(){ this.test.comment( 'checking structure of non-empty panel' ); - this.withFrame( spaceghost.data.selectors.frames.history, function(){ - this.test.comment( "history name should exist, be visible, and have text " + unnamedName ); - this.test.assertExists( nameSelector, nameSelector + ' exists' ); - this.test.assertVisible( nameSelector, 'History name is visible' ); - this.test.assertSelectorHasText( nameSelector, newHistoryName, 'History name is ' + newHistoryName ); + this.test.comment( "history name should exist, be visible, and have text " + unnamedName ); + this.test.assertExists( nameSelector, nameSelector + ' exists' ); + this.test.assertVisible( nameSelector, 'History name is visible' ); + this.test.assertSelectorHasText( nameSelector, newHistoryName, 'History name is ' + newHistoryName ); - this.test.comment( "history subtitle should display size and size should be " + onetxtFilesize + " bytes" ); - var onetxtFilesize = require( 'fs' ).size( this.options.scriptDir + filepathToUpload ), - expectedSubtitle = onetxtFilesize + ' bytes'; - this.test.assertExists( subtitleSelector, 'Found ' + subtitleSelector ); - this.test.assertVisible( subtitleSelector, 'History subtitle is visible' ); - this.test.assertSelectorHasText( subtitleSelector, expectedSubtitle, - 'History subtitle has "' + expectedSubtitle + '"' ); + this.test.comment( "history subtitle should display size and size should be " + onetxtFilesize + " bytes" ); + var onetxtFilesize = require( 'fs' ).size( this.options.scriptDir + filepathToUpload ), + expectedSubtitle = onetxtFilesize + ' bytes'; + this.test.assertExists( subtitleSelector, 'Found ' + subtitleSelector ); + this.test.assertVisible( subtitleSelector, 'History subtitle is visible' ); + this.test.assertSelectorHasText( subtitleSelector, expectedSubtitle, + 'History subtitle has "' + expectedSubtitle + '": ' + this.fetchText( subtitleSelector ).trim() ); - this.test.comment( "tags and annotation icons should be available" ); - this.test.assertExists( tagIconSelector, 'Tag icon button found' ); - this.test.assertExists( annoIconSelector, 'Annotation icon button found' ); + this.test.comment( "tags and annotation icons should be available" ); + this.test.assertExists( tagIconSelector, 'Tag icon button found' ); + this.test.assertExists( annoIconSelector, 'Annotation icon button found' ); - this.test.comment( "A message about the current history being empty should NOT be displayed" ); - this.test.assertExists( emptyMsgSelector, emptyMsgSelector + ' exists' ); - this.test.assertNotVisible( emptyMsgSelector, 'Empty history message is NOT visible' ); - }); + this.test.comment( "A message about the current history being empty should NOT be displayed" ); + this.test.assertExists( emptyMsgSelector, emptyMsgSelector + ' exists' ); + this.test.assertNotVisible( emptyMsgSelector, 'Empty history message is NOT visible' ); }); // ------------------------------------------------------------------- tags // keeping this light here - better for it's own test file //TODO: check tooltips -spaceghost.then( function openTags(){ +spaceghost.withHistoryPanel( function openTags(){ this.test.comment( 'tag area should open when the history panel tag icon is clicked' ); - this.withFrame( spaceghost.data.selectors.frames.history, function(){ - this.mouseEvent( 'click', tagIconSelector ); - this.wait( 1000, function(){ - this.test.assertVisible( tagAreaSelector, 'Tag area is now displayed' ); - }); + + this.click( tagIconSelector ); + this.wait( 1000, function(){ + this.test.assertVisible( tagAreaSelector, 'Tag area is now displayed' ); + }); +}); +spaceghost.withHistoryPanel( function closeAnnotation(){ + this.test.comment( 'annotation area should close when the history panel tag icon is clicked again' ); + + this.click( tagIconSelector ); + this.wait( 1000, function(){ + this.test.assertNotVisible( tagAreaSelector, 'Tag area is now hidden' ); }); }); // ------------------------------------------------------------------- annotation // keeping this light here - better for it's own test file //TODO: check tooltips -spaceghost.then( function openAnnotation(){ +spaceghost.withHistoryPanel( function openAnnotation(){ this.test.comment( 'annotation area should open when the history panel annotation icon is clicked' ); - this.withFrame( spaceghost.data.selectors.frames.history, function(){ - this.mouseEvent( 'click', annoIconSelector ); - this.wait( 1000, function(){ - this.test.assertVisible( annoAreaSelector, 'Annotation area is now displayed' ); - }); + + this.click( annoIconSelector ); + this.wait( 1000, function(){ + this.test.assertVisible( annoAreaSelector, 'Annotation area is now displayed' ); }); }); -spaceghost.then( function closeAnnotation(){ +spaceghost.withHistoryPanel( function closeAnnotation(){ this.test.comment( 'annotation area should close when the history panel tag icon is clicked again' ); - this.withFrame( spaceghost.data.selectors.frames.history, function bler(){ - this.mouseEvent( 'click', annoIconSelector ); - this.wait( 1000, function(){ - this.test.assertNotVisible( annoAreaSelector, 'Tag area is now hidden' ); - }); + + this.click( annoIconSelector ); + this.wait( 1000, function(){ + this.test.assertNotVisible( annoAreaSelector, 'Annotation area is now hidden' ); }); }); @@ -261,96 +248,16 @@ }); }); -// ------------------------------------------------------------------- history options menu structure -//NOTE: options menu should be functionally tested elsewhere -spaceghost.then( function historyOptions(){ - this.test.comment( 'History options icon should be in place and menu should have the proper structure' ); - - // check the button and icon - this.test.assertExists( this.historyoptions.data.selectors.button, "Found history options button" ); - this.test.assertVisible( this.historyoptions.data.selectors.button, "History options button is visible" ); - this.test.assertVisible( this.historyoptions.data.selectors.buttonIcon, "History options icon is visible" ); - - // open the menu - this.click( this.historyoptions.data.selectors.button ); - this.test.assertVisible( this.historyoptions.data.selectors.menu, - "Menu is visible when options button is clicked" ); - - // check the options - for( var optionKey in this.historyoptions.data.labels.options ){ - if( this.historyoptions.data.labels.options.hasOwnProperty( optionKey ) ){ - var optionLabel = this.historyoptions.data.labels.options[ optionKey ], - optionXpath = this.historyoptions.data.selectors.optionXpathByLabelFn( optionLabel ); - this.test.assertVisible( optionXpath, 'Option label is visible: ' + optionLabel ); - } - } -}); - -// ------------------------------------------------------------------- deleted hdas aren't in the dom -spaceghost.then( function(){ - this.test.comment( 'deleted hdas shouldn\'t be in the history panel DOM' ); - - this.historypanel.deleteHda( '#' + testUploadInfo.hdaElement.attributes.id, function(){ - this.test.assertDoesntExist( '#' + testUploadInfo.hdaElement.attributes.id, - "Deleted HDA is not in the DOM" ); - }); -}); - -// ------------------------------------------------------------------- options allow showing/hiding deleted hdas -spaceghost.then( function(){ - this.test.comment( 'History options->' + includeDeletedOptionsLabel + ' shows deleted datasets' ); - - this.historyoptions.includeDeleted(); - this.withFrame( spaceghost.data.selectors.frames.history, function(){ - this.waitForSelector( nameSelector, function(){ - this.test.assertExists( '#' + testUploadInfo.hdaElement.attributes.id, - "Deleted HDA is in the DOM (using history options -> " + includeDeletedOptionsLabel + ")" ); - this.test.assertVisible( '#' + testUploadInfo.hdaElement.attributes.id, - "Deleted HDA is visible again (using history options -> " + includeDeletedOptionsLabel + ")" ); - }); - }); -}); - -spaceghost.then( function(){ - this.test.comment( 'History options->' + includeDeletedOptionsLabel + ' (again) re-hides deleted datasets' ); - - this.historyoptions.includeDeleted(); - this.withFrame( spaceghost.data.selectors.frames.history, function(){ - this.waitForSelector( nameSelector, function(){ - this.test.assertDoesntExist( '#' + testUploadInfo.hdaElement.attributes.id, - "Deleted HDA is not in the DOM (using history options -> " + includeDeletedOptionsLabel + ")" ); - }); - }); -}); - -// undelete the uploaded file -spaceghost.then( function(){ - this.historyoptions.includeDeleted(); - this.withFrame( spaceghost.data.selectors.frames.history, function(){ - this.waitForSelector( nameSelector, function(){ - //TODO: to conv. fn - this.click( '#' + testUploadInfo.hdaElement.attributes.id - + ' ' + this.historypanel.data.selectors.history.undeleteLink ); - }); - }); -}); - -// ------------------------------------------------------------------- hidden hdas aren't shown -// ------------------------------------------------------------------- history options allows showing hidden hdas -// can't test this yet w/o a way to make hdas hidden thru the ui or api - // ------------------------------------------------------------------- hdas can be expanded by clicking on the hda name // broken in webkit w/ jq 1.7 -spaceghost.then( function(){ +spaceghost.historypanel.waitForHdas( function(){ this.test.comment( 'HDAs can be expanded by clicking on the name' ); var uploadedSelector = '#' + testUploadInfo.hdaElement.attributes.id; - this.withFrame( spaceghost.data.selectors.frames.history, function(){ - this.click( uploadedSelector + ' .historyItemTitle' ); - this.wait( 1000, function(){ - this.test.assertVisible( uploadedSelector + ' ' + this.historypanel.data.selectors.hda.body, - "Body for uploaded file is visible" ); - }); + this.click( uploadedSelector + ' ' + this.historypanel.data.selectors.hda.title ); + this.wait( 1000, function(){ + this.test.assertVisible( uploadedSelector + ' ' + this.historypanel.data.selectors.hda.body, + "Body for uploaded file is visible" ); }); }); @@ -360,26 +267,22 @@ var uploadedSelector = '#' + testUploadInfo.hdaElement.attributes.id; this.click( refreshButtonSelector ); - this.withFrame( spaceghost.data.selectors.frames.history, function(){ - this.waitForSelector( nameSelector, function(){ - this.test.assertVisible( uploadedSelector + ' ' + this.historypanel.data.selectors.hda.body, - "Body for uploaded file is visible" ); - }); + this.historypanel.waitForHdas( function(){ + this.test.assertVisible( uploadedSelector + ' ' + this.historypanel.data.selectors.hda.body, + "Body for uploaded file is visible" ); }); // this will break: webkit + jq 1.7 }); // ------------------------------------------------------------------- expanded hdas collapse by clicking name again -spaceghost.then( function(){ +spaceghost.withHistoryPanel( function(){ this.test.comment( 'Expanded hdas collapse by clicking name again' ); var uploadedSelector = '#' + testUploadInfo.hdaElement.attributes.id; - this.withFrame( spaceghost.data.selectors.frames.history, function(){ - this.click( uploadedSelector + ' ' + this.historypanel.data.selectors.hda.title ); - this.wait( 500, function(){ - this.test.assertNotVisible( uploadedSelector + ' ' + this.historypanel.data.selectors.hda.body, - "Body for uploaded file is not visible" ); - }); + this.click( uploadedSelector + ' ' + this.historypanel.data.selectors.hda.title ); + this.wait( 500, function(){ + this.test.assertNotVisible( uploadedSelector + ' ' + this.historypanel.data.selectors.hda.body, + "Body for uploaded file is not visible" ); }); }); @@ -389,32 +292,9 @@ var uploadedSelector = '#' + testUploadInfo.hdaElement.attributes.id; this.click( refreshButtonSelector ); - this.withFrame( spaceghost.data.selectors.frames.history, function(){ - this.waitForSelector( nameSelector, function(){ - this.test.assertNotVisible( uploadedSelector + ' ' + this.historypanel.data.selectors.hda.body, - "Body for uploaded file is not visible" ); - }); - }); -}); - -// ------------------------------------------------------------------- history options collapses all expanded hdas -spaceghost.then( function(){ - // expand again - this.withFrame( spaceghost.data.selectors.frames.history, function(){ - this.click( '#' + testUploadInfo.hdaElement.attributes.id + ' ' + this.historypanel.data.selectors.hda.title ); - this.wait( 500, function(){}); - }); -}); -spaceghost.then( function(){ - this.test.comment( 'History option collapses all expanded hdas' ); - var uploadedSelector = '#' + testUploadInfo.hdaElement.attributes.id; - - this.historyoptions.collapseExpanded(); - this.wait( 500, function(){ - this.withFrame( spaceghost.data.selectors.frames.history, function(){ - this.test.assertNotVisible( uploadedSelector + ' ' + this.historypanel.data.selectors.hda.body, - "Body for uploaded file is not visible" ); - }); + this.historypanel.waitForHdas( function(){ + this.test.assertNotVisible( uploadedSelector + ' ' + this.historypanel.data.selectors.hda.body, + "Body for uploaded file is not visible" ); }); }); diff -r 24d068f7a44dab10f8574d15aa4063ccdb0edb22 -r 3bbbcdb044afdbd380d6cd4c0f721ab7262c5555 test/casperjs/login-tests.js --- a/test/casperjs/login-tests.js +++ b/test/casperjs/login-tests.js @@ -39,6 +39,8 @@ password = spaceghost.fixtureData.testUser.password; } +var userEmailSelector = '//a[contains(text(),"Logged in as")]/span["id=#user-email"]'; + // =================================================================== TESTS // register a user (again...) spaceghost.thenOpen( spaceghost.baseUrl, function(){ @@ -52,8 +54,7 @@ spaceghost.user.logout(); }); spaceghost.then( function(){ - this.test.assertSelectorDoesntHaveText( - xpath( '//a[contains(text(),"Logged in as")]/span["id=#user-email"]' ), /\w/ ); + this.test.assertSelectorDoesntHaveText( xpath( userEmailSelector ), /\w/ ); this.test.assert( spaceghost.user.loggedInAs() === '', 'loggedInAs() is empty string' ); }); @@ -63,8 +64,7 @@ spaceghost.user._submitLogin( email, password ); //No such user }); spaceghost.thenOpen( spaceghost.baseUrl, function(){ - this.test.assertSelectorHasText( - xpath( '//a[contains(text(),"Logged in as")]/span["id=#user-email"]' ), email ); + this.test.assertSelectorHasText( xpath( userEmailSelector ), email ); this.test.assert( spaceghost.user.loggedInAs() === email, 'loggedInAs() matches email' ); }); diff -r 24d068f7a44dab10f8574d15aa4063ccdb0edb22 -r 3bbbcdb044afdbd380d6cd4c0f721ab7262c5555 test/casperjs/modules/api.js --- /dev/null +++ b/test/casperjs/modules/api.js @@ -0,0 +1,231 @@ +// =================================================================== module object, exports +/** Creates a new api module object. + * @param {SpaceGhost} spaceghost a spaceghost instance + * @exported + */ +exports.create = function createAPI( spaceghost, apikey ){ + return new API( spaceghost ); +}; + +/** User object constructor. + * @param {SpaceGhost} spaceghost a spaceghost instance + * @param {String} apikey apikey for use when not using session authentication + */ +var API = function API( spaceghost, apikey ){ + this.spaceghost = spaceghost; + this.apikey = apikey; + + this.encodedIdExpectedLength = 16; + this.jQueryLocation = '../../static/scripts/libs/jquery/jquery.js'; + + this.histories = new HistoriesAPI( this ); + this.hdas = new HDAAPI( this ); +}; +exports.API = API; + +API.prototype.toString = function toString(){ + return ( this.spaceghost + '.API:' + + (( this.apikey )?( this.apikey ):( '(session)' )) ); +}; + +// ------------------------------------------------------------------- APIError +APIError.prototype = new Error(); +APIError.prototype.constructor = Error; +/** @class Thrown when Galaxy the API returns an error from a request */ +function APIError( msg ){ + Error.apply( this, arguments ); + this.name = "APIError"; + this.message = msg; +} +exports.APIError = APIError; + +/* ------------------------------------------------------------------- TODO: + can we component-ize this to become the basis for js-based api binding/resource + +*/ +// =================================================================== INTERNAL +var utils = require( 'utils' ); + +API.prototype._ajax = function _ajax( url, options ){ + options = options || {}; + options.async = false; + + this.ensureJQuery( '../../static/scripts/libs/jquery/jquery.js' ); + var resp = this.spaceghost.evaluate( function( url, options ){ + return jQuery.ajax( url, options ); + }, url, options ); + //this.spaceghost.debug( 'resp: ' + this.spaceghost.jsonStr( resp ) ); + + if( resp.status !== 200 ){ + // grrr... this doesn't lose the \n\r\t + throw new APIError( resp.responseText.replace( /[\s\n\r\t]+/gm, ' ' ).replace( /"/, '' ) ); + } + return JSON.parse( resp.responseText ); +}; + +// =================================================================== MISC +API.prototype.isEncodedId = function isEncodedId( id ){ + if( typeof id !== 'string' ){ return false; } + if( id.match( /[g-zG-Z]/ ) ){ return false; } + return ( id.length === this.encodedIdExpectedLength ); +}; + +// ------------------------------------------------------------------- is type or throw err +API.prototype.ensureId = function ensureId( id ){ + if( !this.isEncodedId( id ) ){ + throw new APIError( 'ID is not a valid encoded id: ' + id ); + } + return id; +}; + +API.prototype.ensureObject = function ensureObject( obj ){ + if( !utils.isObject( obj ) ){ + throw new APIError( 'Not a valid object: ' + obj ); + } + return obj; +}; + +// ------------------------------------------------------------------- jquery +// using jq for the ajax in this module - that's why these are here +//TODO:?? could go in spaceghost +API.prototype.hasJQuery = function hasJQuery(){ + return this.spaceghost.evaluate( function pageHasJQuery(){ + var has = false; + try { + has = typeof ( jQuery + '' ) === 'string'; + } catch( err ){} + return has; + }); +}; + +API.prototype.ensureJQuery = function ensureJQuery(){ + if( !this.hasJQuery() ){ + var absLoc = this.spaceghost.options.scriptDir + this.jQueryLocation, + injected = this.spaceghost.page.injectJs( absLoc ); + if( !injected ){ + throw new APIError( 'Could not inject jQuery' ); + } + } +}; + + +// =================================================================== HISTORIES +var HistoriesAPI = function HistoriesAPI( api ){ + this.api = api; +}; +HistoriesAPI.prototype.toString = function toString(){ + return this.api + '.HistoriesAPI'; +}; + +// ------------------------------------------------------------------- +HistoriesAPI.prototype.urlTpls = { + index : 'api/histories', + show : 'api/histories/%s', + create : 'api/histories', + delete_ : 'api/histories/%s', + undelete: 'api/histories/deleted/%s/undelete' +}; + +HistoriesAPI.prototype.index = function index( deleted ){ + this.api.spaceghost.info( 'history.index: ' + (( deleted )?( 'w deleted' ):( '(wo deleted)' )) ); + + deleted = deleted || false; + return this.api._ajax( this.urlTpls.index, { + data : { deleted: deleted } + }); +}; + +HistoriesAPI.prototype.show = function show( id, deleted ){ + this.api.spaceghost.info( 'history.show: ' + [ id, (( deleted )?( 'w deleted' ):( '' )) ] ); + + id = ( id === 'most_recently_used' )?( id ):( this.api.ensureId( id ) ); + deleted = deleted || false; + return this.api._ajax( utils.format( this.urlTpls.show, id ), { + data : { deleted: deleted } + }); +}; + +HistoriesAPI.prototype.create = function create( payload ){ + this.api.spaceghost.info( 'history.create: ' + this.api.spaceghost.jsonStr( payload ) ); + + // py.payload <-> ajax.data + payload = this.api.ensureObject( payload ); + return this.api._ajax( utils.format( this.urlTpls.create ), { + type : 'POST', + data : payload + }); +}; + +HistoriesAPI.prototype.delete_ = function delete_( id, purge ){ + this.api.spaceghost.info( 'history.delete: ' + [ id, (( purge )?( '(purge!)' ):( '' )) ] ); + + // py.payload <-> ajax.data + var payload = ( purge )?({ purge: true }):({}); + return this.api._ajax( utils.format( this.urlTpls.delete_, this.api.ensureId( id ) ), { + type : 'DELETE', + data : payload + }); +}; + +HistoriesAPI.prototype.undelete = function undelete( id ){ + //throw ( 'unimplemented' ); + this.api.spaceghost.info( 'history.undelete: ' + id ); + + return this.api._ajax( utils.format( this.urlTpls.undelete, this.api.ensureId( id ) ), { + type : 'POST' + }); +}; + + +// =================================================================== HDAS +var HDAAPI = function HDAAPI( api ){ + this.api = api; +}; +HDAAPI.prototype.toString = function toString(){ + return this.api + '.HDAAPI'; +}; + +// ------------------------------------------------------------------- +HDAAPI.prototype.urlTpls = { + index : 'api/histories/%s/contents', + show : 'api/histories/%s/contents/%s', + create : 'api/histories/%s/contents'//, + // not implemented + //delete_ : 'api/histories/%s', + //undelete: 'api/histories/deleted/%s/undelete' +}; + +HDAAPI.prototype.index = function index( historyId, ids ){ + this.api.spaceghost.info( 'history.index: ' + [ historyId, ids ] ); + var data = {}; + if( ids ){ + ids = ( utils.isArray( ids ) )?( ids.join( ',' ) ):( ids ); + data.ids = ids; + } + + return this.api._ajax( utils.format( this.urlTpls.index, this.api.ensureId( historyId ) ), { + data : data + }); +}; + +HDAAPI.prototype.show = function show( historyId, id, deleted ){ + this.api.spaceghost.info( 'history.show: ' + [ id, (( deleted )?( 'w deleted' ):( '' )) ] ); + + id = ( id === 'most_recently_used' )?( id ):( this.api.ensureId( id ) ); + deleted = deleted || false; + return this.api._ajax( utils.format( this.urlTpls.show, id ), { + data : { deleted: deleted } + }); +}; + +HDAAPI.prototype.create = function create( historyId, payload ){ + this.api.spaceghost.info( 'history.create: ' + this.api.spaceghost.jsonStr( payload ) ); + + // py.payload <-> ajax.data + payload = this.api.ensureObject( payload ); + return this.api._ajax( utils.format( this.urlTpls.create ), { + type : 'POST', + data : payload + }); +}; + This diff is so big that we needed to truncate the remainder. Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: dannon: Fix broken tool_dependency_util -- was missing galaxy's 'util' import as well as 'shutil'.
by commits-noreply＠bitbucket.org 21 Mar '13

21 Mar '13

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/24d068f7a44d/ Changeset: 24d068f7a44d Branch: next-stable User: dannon Date: 2013-03-21 18:45:12 Summary: Fix broken tool_dependency_util -- was missing galaxy's 'util' import as well as 'shutil'. Affected #: 1 file diff -r 0cd8201b4b4848002adbdbcac647d76f112db4d8 -r 24d068f7a44dab10f8574d15aa4063ccdb0edb22 lib/tool_shed/util/tool_dependency_util.py --- a/lib/tool_shed/util/tool_dependency_util.py +++ b/lib/tool_shed/util/tool_dependency_util.py @@ -1,13 +1,15 @@ -import os, logging +import logging +import os +import shutil + +from galaxy import eggs +eggs.require( 'elementtree' ) +from elementtree import ElementTree, ElementInclude + from galaxy.model.orm import and_ import tool_shed.util.shed_util_common as suc -from galaxy import eggs -import pkg_resources - -pkg_resources.require( 'elementtree' ) -from elementtree import ElementTree, ElementInclude -from elementtree.ElementTree import Element, SubElement +from galaxy import util log = logging.getLogger( __name__ ) Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: 2 new changesets
by commits-noreply＠bitbucket.org 21 Mar '13

21 Mar '13

2 new commits in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/0cd8201b4b48/ changeset: 0cd8201b4b48 branch: next-stable user: carlfeberhard date: 2013-03-21 18:55:36 summary: history, view: fix show_deleted/hide_deleted affected #: 1 file diff -r 55a425f1f2f363f6e9fe94ab539bb58daeb8d136 -r 0cd8201b4b4848002adbdbcac647d76f112db4d8 templates/webapps/galaxy/history/view.mako --- a/templates/webapps/galaxy/history/view.mako +++ b/templates/webapps/galaxy/history/view.mako @@ -98,9 +98,9 @@ <a href="${get_history_link( history )}">${_('refresh')}</a> | %endif %if show_deleted: - <a href="${h.url_for(controller='history', id=trans.security.encode_id(history.id), show_deleted=False, use_panels=use_panels )}">${_('hide deleted')}</a> | + <a href="${h.url_for(controller='history', action='view', id=trans.security.encode_id(history.id), show_deleted=False, use_panels=use_panels )}">${_('hide deleted')}</a> | %else: - <a href="${h.url_for(controller='history', id=trans.security.encode_id(history.id), show_deleted=True, use_panels=use_panels )}">${_('show deleted')}</a> | + <a href="${h.url_for(controller='history', action='view', id=trans.security.encode_id(history.id), show_deleted=True, use_panels=use_panels )}">${_('show deleted')}</a> | %endif <a href="#" class="toggle">collapse all</a></div> https://bitbucket.org/galaxy/galaxy-central/commits/b79e2393c8ad/ changeset: b79e2393c8ad user: carlfeberhard date: 2013-03-21 18:56:18 summary: merge stable affected #: 1 file diff -r 903c3458a6783a878d5e31c6574cec550711e88a -r b79e2393c8ad8c88bda682e0767f572c0c09e26a templates/webapps/galaxy/history/view.mako --- a/templates/webapps/galaxy/history/view.mako +++ b/templates/webapps/galaxy/history/view.mako @@ -98,9 +98,9 @@ <a href="${get_history_link( history )}">${_('refresh')}</a> | %endif %if show_deleted: - <a href="${h.url_for(controller='history', id=trans.security.encode_id(history.id), show_deleted=False, use_panels=use_panels )}">${_('hide deleted')}</a> | + <a href="${h.url_for(controller='history', action='view', id=trans.security.encode_id(history.id), show_deleted=False, use_panels=use_panels )}">${_('hide deleted')}</a> | %else: - <a href="${h.url_for(controller='history', id=trans.security.encode_id(history.id), show_deleted=True, use_panels=use_panels )}">${_('show deleted')}</a> | + <a href="${h.url_for(controller='history', action='view', id=trans.security.encode_id(history.id), show_deleted=True, use_panels=use_panels )}">${_('show deleted')}</a> | %endif <a href="#" class="toggle">collapse all</a></div> Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: dannon: Fix broken tool_dependency_util -- was missing galaxy's 'util' import as well as 'shutil'.
by commits-noreply＠bitbucket.org 21 Mar '13

21 Mar '13

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/903c3458a678/ changeset: 903c3458a678 user: dannon date: 2013-03-21 18:45:12 summary: Fix broken tool_dependency_util -- was missing galaxy's 'util' import as well as 'shutil'. affected #: 1 file diff -r 0ce683ac64b95a2d6f6738407e8f69c7087a6b30 -r 903c3458a6783a878d5e31c6574cec550711e88a lib/tool_shed/util/tool_dependency_util.py --- a/lib/tool_shed/util/tool_dependency_util.py +++ b/lib/tool_shed/util/tool_dependency_util.py @@ -1,13 +1,15 @@ -import os, logging +import logging +import os +import shutil + +from galaxy import eggs +eggs.require( 'elementtree' ) +from elementtree import ElementTree, ElementInclude + from galaxy.model.orm import and_ import tool_shed.util.shed_util_common as suc -from galaxy import eggs -import pkg_resources - -pkg_resources.require( 'elementtree' ) -from elementtree import ElementTree, ElementInclude -from elementtree.ElementTree import Element, SubElement +from galaxy import util log = logging.getLogger( __name__ ) Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0