February 2015 - galaxy-commits - lists.galaxyproject.org

commit/galaxy-central: jmchilton: API endpoint for reloading tool data tables (from Bjoern).
by commits-noreply＠bitbucket.org 03 Feb '15

03 Feb '15

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/bcaffc4a0a61/ Changeset: bcaffc4a0a61 User: jmchilton Date: 2015-02-03 17:44:39+00:00 Summary: API endpoint for reloading tool data tables (from Bjoern). The endpoint isn't "RESTful" IMO - but it does match what we are doing for tools - so it is consistent which is more important. Affected #: 2 files diff -r 944074958b72d49757455a07a503024e631d15a6 -r bcaffc4a0a61d3d803cee45435548d0db0e59e54 lib/galaxy/webapps/galaxy/api/tool_data.py --- a/lib/galaxy/webapps/galaxy/api/tool_data.py +++ b/lib/galaxy/webapps/galaxy/api/tool_data.py @@ -28,6 +28,19 @@ @web.require_admin @expose_api + def reload( self, trans, id, **kwd ): + """ + GET /api/tool_data/{id}/reload + + Reloads a tool_data table. + """ + decoded_tool_data_id = id + data_table = trans.app.tool_data_tables.data_tables.get(decoded_tool_data_id) + return data_table.reload_from_files() + + + @web.require_admin + @expose_api def delete( self, trans, id, **kwd ): """ DELETE /api/tool_data/{id} diff -r 944074958b72d49757455a07a503024e631d15a6 -r bcaffc4a0a61d3d803cee45435548d0db0e59e54 lib/galaxy/webapps/galaxy/buildapp.py --- a/lib/galaxy/webapps/galaxy/buildapp.py +++ b/lib/galaxy/webapps/galaxy/buildapp.py @@ -174,6 +174,7 @@ webapp.mapper.resource( 'tool_data', 'tool_data', path_prefix='/api' ) webapp.mapper.connect( '/api/tool_data/{id:.+?}/fields/{value:.+?}/files/{path:.+?}', action='download_field_file', controller="tool_data" ) webapp.mapper.connect( '/api/tool_data/{id:.+?}/fields/{value:.+?}', action='show_field', controller="tool_data" ) + webapp.mapper.connect( '/api/tool_data/{id:.+?}/reload', action='reload', controller="tool_data" ) webapp.mapper.resource( 'dataset_collection', 'dataset_collections', path_prefix='/api/') webapp.mapper.resource( 'sample', 'samples', path_prefix='/api' ) webapp.mapper.resource( 'request', 'requests', path_prefix='/api' ) Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

[galaxyproject/usegalaxy-playbook] 047bb6: update test
by GitHub 03 Feb '15

03 Feb '15

Branch: refs/heads/master Home: https://github.com/galaxyproject/usegalaxy-playbook Commit: 047bb69d7ea1b8a1649f571b28906a34481d2610 https://github.com/galaxyproject/usegalaxy-playbook/commit/047bb69d7ea1b8a1… Author: martenson <cech.marten(a)gmail.com> Date: 2015-02-03 (Tue, 03 Feb 2015) Changed paths: M stage/group_vars/all.yml Log Message: ----------- update test

1 0

[galaxyproject/ansible-galaxy] 7ed9bb: Implement option to download Galaxy (instead of cl...
by GitHub 03 Feb '15

03 Feb '15

Branch: refs/heads/master Home: https://github.com/galaxyproject/ansible-galaxy Commit: 7ed9bbfd1b3351f1f300843183898db54f07038c https://github.com/galaxyproject/ansible-galaxy/commit/7ed9bbfd1b3351f1f300… Author: John Chilton <jmchilton(a)gmail.com> Date: 2015-01-29 (Thu, 29 Jan 2015) Changed paths: M defaults/main.yml M tasks/clone.yml A tasks/download.yml M tasks/main.yml A tasks/virtualenv.yml Log Message: ----------- Implement option to download Galaxy (instead of cloning it). Until we have debs this is... a... way to get Galaxy without the whole history for throw away machines like Docker - where keeping the size of the image small is more important than being able to update the Galaxy found inside. Commit: 47c4fd6714f70a69a4cd04acec45569c0613f512 https://github.com/galaxyproject/ansible-galaxy/commit/47c4fd6714f70a69a4cd… Author: John Chilton <jmchilton(a)gmail.com> Date: 2015-02-03 (Tue, 03 Feb 2015) Changed paths: M defaults/main.yml M tasks/clone.yml A tasks/download.yml M tasks/main.yml A tasks/virtualenv.yml Log Message: ----------- Merge pull request #3 from jmchilton/download_option Implement option to download Galaxy (instead of cloning it). Compare: https://github.com/galaxyproject/ansible-galaxy/compare/c9559f7d9c4c...47c4…

1 0

commit/galaxy-central: jmchilton: Small cleanup to lib/galaxy/tools/evaluation.py.
by commits-noreply＠bitbucket.org 03 Feb '15

03 Feb '15

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/944074958b72/ Changeset: 944074958b72 User: jmchilton Date: 2015-02-03 17:22:57+00:00 Summary: Small cleanup to lib/galaxy/tools/evaluation.py. Eliminate the absolute path call outside the compute_environment - since we wouldn't want to take the absolute path on a *nix box for a Windows path and because it is already being absolute-itized inside of the the ComputeEnviornment abstraction. Eliminate TODO comment that has already been TODONE. Affected #: 1 file diff -r 99a7a07c6ffba2948b2e1197a3a33554f60126ba -r 944074958b72d49757455a07a503024e631d15a6 lib/galaxy/tools/evaluation.py --- a/lib/galaxy/tools/evaluation.py +++ b/lib/galaxy/tools/evaluation.py @@ -353,8 +353,7 @@ # More convienent access to app.config.new_file_path; we don't need to # wrap a string, but this method of generating additional datasets # should be considered DEPRECATED - # TODO: path munging for cluster/dataset server relocatability - param_dict['__new_file_path__'] = os.path.abspath(self.compute_environment.new_file_path()) + param_dict['__new_file_path__'] = self.compute_environment.new_file_path() # The following points to location (xxx.loc) files which are pointers # to locally cached data param_dict['__tool_data_path__'] = param_dict['GALAXY_DATA_INDEX_DIR'] = self.app.config.tool_data_path Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: jmchilton: Drop test that is failing due to improved HDA accessibility consistency.
by commits-noreply＠bitbucket.org 03 Feb '15

03 Feb '15

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/99a7a07c6ffb/ Changeset: 99a7a07c6ffb User: jmchilton Date: 2015-02-03 17:20:41+00:00 Summary: Drop test that is failing due to improved HDA accessibility consistency. Affected #: 1 file diff -r ddbe0d12bc86da406bb96213e23b8ddb273ed669 -r 99a7a07c6ffba2948b2e1197a3a33554f60126ba test/api/test_dataset_collections.py --- a/test/api/test_dataset_collections.py +++ b/test/api/test_dataset_collections.py @@ -105,7 +105,9 @@ ) create_response = self._post( "dataset_collections", payload ) - self._assert_status_code_is( create_response, 403 ) + # TODO: re-enable once there is a way to restrict access + # to this dataset via the API. + #self._assert_status_code_is( create_response, 403 ) def test_enforces_unique_names( self ): element_identifiers = self.dataset_collection_populator.list_identifiers( self.history_id ) Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: jmchilton: Expose $__tool_directory__ to tools.
by commits-noreply＠bitbucket.org 03 Feb '15

03 Feb '15

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/ddbe0d12bc86/ Changeset: ddbe0d12bc86 User: jmchilton Date: 2015-02-03 15:29:34+00:00 Summary: Expose $__tool_directory__ to tools. Repeatedly one wants to do things like create symbolic links and then call a helper script - the 'interpreter' tag doesn't allow shell commands before calling a helper script so they could not be used in this fashion. The previous pattern for doing this was then to use the ToolShed only 'set_environment' requirement tag. These were onerous to setup and the resulting tools were no longer portable to non-ToolShed installed contexts - I believe this variant is more robust and elegant. More information https://trello.com/c/0pgF5PBQ and here https://trello.com/c/XK5SqE1i. Affected #: 5 files diff -r eb26a96483238cd353c3fc1bdce9699382dfadb6 -r ddbe0d12bc86da406bb96213e23b8ddb273ed669 lib/galaxy/jobs/__init__.py --- a/lib/galaxy/jobs/__init__.py +++ b/lib/galaxy/jobs/__init__.py @@ -1783,7 +1783,11 @@ @abstractmethod def new_file_path( self ): - """ Location to dump new files for this job on remote server. """ + """ Absolute path to dump new files for this job on compute server. """ + + @abstractmethod + def tool_directory( self ): + """ Absolute path to tool files for this job on compute server. """ @abstractmethod def version_path( self ): @@ -1836,6 +1840,9 @@ def version_path( self ): return self.job_wrapper.get_version_string_path() + def tool_directory( self ): + return os.path.abspath(self.job_wrapper.tool.tool_dir) + class NoopQueue( object ): """ diff -r eb26a96483238cd353c3fc1bdce9699382dfadb6 -r ddbe0d12bc86da406bb96213e23b8ddb273ed669 lib/galaxy/jobs/runners/pulsar.py --- a/lib/galaxy/jobs/runners/pulsar.py +++ b/lib/galaxy/jobs/runners/pulsar.py @@ -708,3 +708,6 @@ def unstructured_path_rewriter( self ): return self.rewriter + + def tool_directory( self ): + return self._tool_dir diff -r eb26a96483238cd353c3fc1bdce9699382dfadb6 -r ddbe0d12bc86da406bb96213e23b8ddb273ed669 lib/galaxy/tools/evaluation.py --- a/lib/galaxy/tools/evaluation.py +++ b/lib/galaxy/tools/evaluation.py @@ -345,6 +345,7 @@ if table_name in self.app.tool_data_tables: return self.app.tool_data_tables[ table_name ].get_entry( query_attr, query_val, return_attr ) + param_dict['__tool_directory__'] = self.compute_environment.tool_directory() param_dict['__get_data_table_entry__'] = get_data_table_entry # We add access to app here, this allows access to app.config, etc diff -r eb26a96483238cd353c3fc1bdce9699382dfadb6 -r ddbe0d12bc86da406bb96213e23b8ddb273ed669 test/functional/tools/samples_tool_conf.xml --- a/test/functional/tools/samples_tool_conf.xml +++ b/test/functional/tools/samples_tool_conf.xml @@ -32,6 +32,7 @@ <tool file="validation_repeat.xml" /><tool file="identifier_single.xml" /><tool file="identifier_multiple.xml" /> + <tool file="tool_directory.xml" /><tool file="collection_paired_test.xml" /><tool file="collection_nested_test.xml" /><tool file="collection_mixed_param.xml" /> diff -r eb26a96483238cd353c3fc1bdce9699382dfadb6 -r ddbe0d12bc86da406bb96213e23b8ddb273ed669 test/functional/tools/tool_directory.xml --- /dev/null +++ b/test/functional/tools/tool_directory.xml @@ -0,0 +1,20 @@ +<tool id="tool_directory" name="tool_directory"> + <command> + cp $__tool_directory__/tool_directory.xml output1 + </command> + <inputs> + <param type="integer" name="ignored" label="Ignored" value="0" /> + </inputs> + <outputs> + <data name="output1" type="xml" from_work_dir="output1" /> + </outputs> + <tests> + <test> + <output name="output1"> + <assert_contents> + <has_text text="QUINE" /> + </assert_contents> + </output> + </test> + </tests> +</tool> Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: carlfeberhard: Browser tests: remove starting slash from workflow return url comparisons
by commits-noreply＠bitbucket.org 03 Feb '15

03 Feb '15

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/eb26a9648323/ Changeset: eb26a9648323 User: carlfeberhard Date: 2015-02-03 14:00:51+00:00 Summary: Browser tests: remove starting slash from workflow return url comparisons Affected #: 1 file diff -r adcb595753b02cc9757a8f36a34898d4ea2233e7 -r eb26a96483238cd353c3fc1bdce9699382dfadb6 test/casperjs/api-workflow-tests.js --- a/test/casperjs/api-workflow-tests.js +++ b/test/casperjs/api-workflow-tests.js @@ -55,7 +55,7 @@ "uploaded workflow defaults to un-published: " + returned.published ); this.test.assert( utils.isArray( returned.tags ) && returned.tags.length === 0, "upload returned an empty tag array: " + this.jsonStr( returned.tags ) ); - this.test.assert( returned.url === '/' + utils.format( this.api.workflows.urlTpls.show, returned.id ), + this.test.assert( returned.url === utils.format( this.api.workflows.urlTpls.show, returned.id ), "url matches the show url: " + returned.url ); @@ -79,7 +79,7 @@ "workflow is un-published: " + firstWorkflow.published ); this.test.assert( utils.isArray( firstWorkflow.tags ) && firstWorkflow.tags.length === 0, "tag array is empty: " + this.jsonStr( firstWorkflow.tags ) ); - this.test.assert( firstWorkflow.url === '/' + utils.format( this.api.workflows.urlTpls.show, firstWorkflow.id ), + this.test.assert( firstWorkflow.url === utils.format( this.api.workflows.urlTpls.show, firstWorkflow.id ), "url matches the show url: " + firstWorkflow.url ); @@ -99,7 +99,7 @@ "workflow is un-published: " + workflowShow.published ); this.test.assert( utils.isArray( workflowShow.tags ) && workflowShow.tags.length === 0, "tag array is empty: " + this.jsonStr( workflowShow.tags ) ); - this.test.assert( workflowShow.url === '/' + utils.format( this.api.workflows.urlTpls.show, workflowShow.id ), + this.test.assert( workflowShow.url === utils.format( this.api.workflows.urlTpls.show, workflowShow.id ), "url matches the show url: " + workflowShow.url ); this.test.comment( 'inputs from show should be an object (and, in this case, empty)' ); Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: natefoo: Update tag latest_2015.01.13 for changeset 5e4060f5ac7a
by commits-noreply＠bitbucket.org 03 Feb '15

03 Feb '15

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/097bbb3b7d32/ Changeset: 097bbb3b7d32 Branch: stable User: natefoo Date: 2015-02-03 14:00:27+00:00 Summary: Update tag latest_2015.01.13 for changeset 5e4060f5ac7a Affected #: 1 file diff -r 5e4060f5ac7a3c11d4681bc80f94c620df9d0479 -r 097bbb3b7d3246faaa5188a1fc2a79b01630025c .hgtags --- a/.hgtags +++ b/.hgtags @@ -22,4 +22,4 @@ 2092948937ac30ef82f71463a235c66d34987088 release_2014.10.06 782fa60fc65488aea0c618d723e9a63d42caf865 latest_2014.10.06 2e8dd2949dd3eee0f56f9a3a5ebf1b2baca24aee release_2015.01.13 -fd75aaee91cf3e8a0916689dfea72e0c752c447c latest_2015.01.13 +5e4060f5ac7a3c11d4681bc80f94c620df9d0479 latest_2015.01.13 Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: carlfeberhard: Fix to 8e5b9ca: implement _future_expose_api_anonymous_and_sessionless to allow endpoints to be access without user or session (fixes access to api/authenticate)
by commits-noreply＠bitbucket.org 03 Feb '15

03 Feb '15

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/adcb595753b0/ Changeset: adcb595753b0 User: carlfeberhard Date: 2015-02-02 23:57:54+00:00 Summary: Fix to 8e5b9ca: implement _future_expose_api_anonymous_and_sessionless to allow endpoints to be access without user or session (fixes access to api/authenticate) Affected #: 3 files diff -r 147c8b343d4ae09d4ae71d424037e45f06bd8e9e -r adcb595753b02cc9757a8f36a34898d4ea2233e7 lib/galaxy/web/__init__.py --- a/lib/galaxy/web/__init__.py +++ b/lib/galaxy/web/__init__.py @@ -18,6 +18,7 @@ from framework.decorators import _future_expose_api_anonymous from framework.decorators import _future_expose_api_raw from framework.decorators import _future_expose_api_raw_anonymous +from framework.decorators import _future_expose_api_anonymous_and_sessionless from framework.formbuilder import form from framework.formbuilder import FormBuilder diff -r 147c8b343d4ae09d4ae71d424037e45f06bd8e9e -r adcb595753b02cc9757a8f36a34898d4ea2233e7 lib/galaxy/web/framework/decorators.py --- a/lib/galaxy/web/framework/decorators.py +++ b/lib/galaxy/web/framework/decorators.py @@ -196,7 +196,7 @@ # ----------------------------------------------------------------------------- (new) api decorators # TODO: rename as expose_api and make default. -def _future_expose_api( func, to_json=True, user_required=True ): +def _future_expose_api( func, to_json=True, user_required=True, user_or_session_required=True ): """ Expose this function via the API. """ @@ -206,10 +206,11 @@ # TODO: Document this branch, when can this happen, # I don't understand it. return __api_error_response( trans, err_msg=trans.error_message ) + # error if user required and anon # error if anon and no session if ( ( trans.anonymous and user_required ) - or ( trans.anonymous and not trans.galaxy_session ) ): + or ( trans.anonymous and user_or_session_required and not trans.galaxy_session ) ): error_code = error_codes.USER_NO_API_KEY # Use error codes default error message. err_msg = "API authentication required for this request" @@ -325,17 +326,20 @@ return dumps( error_dict ) -# TODO: rename as expose_api and make default. def _future_expose_api_anonymous( func, to_json=True ): """ Expose this function via the API but don't require a set user. """ return _future_expose_api( func, to_json=to_json, user_required=False ) +def _future_expose_api_anonymous_and_sessionless( func, to_json=True ): + """ + Expose this function via the API but don't require a user or a galaxy_session. + """ + return _future_expose_api( func, to_json=to_json, user_required=False, user_or_session_required=False ) def _future_expose_api_raw( func ): return _future_expose_api( func, to_json=False, user_required=True ) - def _future_expose_api_raw_anonymous( func ): return _future_expose_api( func, to_json=False, user_required=False ) diff -r 147c8b343d4ae09d4ae71d424037e45f06bd8e9e -r adcb595753b02cc9757a8f36a34898d4ea2233e7 lib/galaxy/webapps/galaxy/api/authenticate.py --- a/lib/galaxy/webapps/galaxy/api/authenticate.py +++ b/lib/galaxy/webapps/galaxy/api/authenticate.py @@ -14,7 +14,7 @@ from base64 import b64decode from urllib import unquote -from galaxy.web import _future_expose_api_anonymous as expose_api_anonymous +from galaxy.web import _future_expose_api_anonymous_and_sessionless as expose_api_anonymous_and_sessionless from galaxy.managers import api_keys from galaxy import exceptions from galaxy.web.base.controller import BaseAPIController @@ -29,7 +29,7 @@ super( AuthenticationController, self ).__init__( app ) self.api_keys_manager = api_keys.ApiKeyManager( app ) - @expose_api_anonymous + @expose_api_anonymous_and_sessionless def get_api_key( self, trans, **kwd ): """ def get_api_key( self, trans, **kwd ) Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0

commit/galaxy-central: carlfeberhard: API, folders & folder_contents: use _future_expose_api on NotImplemented routes
by commits-noreply＠bitbucket.org 03 Feb '15

03 Feb '15

1 new commit in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/147c8b343d4a/ Changeset: 147c8b343d4a User: carlfeberhard Date: 2015-02-02 23:01:44+00:00 Summary: API, folders & folder_contents: use _future_expose_api on NotImplemented routes Affected #: 3 files diff -r 1d57664c6d9c4875e5c818220fa8199cf3967de0 -r 147c8b343d4ae09d4ae71d424037e45f06bd8e9e lib/galaxy/webapps/galaxy/api/folder_contents.py --- a/lib/galaxy/webapps/galaxy/api/folder_contents.py +++ b/lib/galaxy/webapps/galaxy/api/folder_contents.py @@ -315,14 +315,14 @@ else: raise exceptions.MalformedId( 'Malformed folder id ( %s ) specified, unable to decode.' % str( encoded_folder_id ) ) - @web.expose_api + @expose_api def show( self, trans, id, library_id, **kwd ): """ GET /api/folders/{encoded_folder_id}/ """ raise exceptions.NotImplemented( 'Showing the library folder content is not implemented here.' ) - @web.expose_api + @expose_api def update( self, trans, id, library_id, payload, **kwd ): """ PUT /api/folders/{encoded_folder_id}/contents diff -r 1d57664c6d9c4875e5c818220fa8199cf3967de0 -r 147c8b343d4ae09d4ae71d424037e45f06bd8e9e lib/galaxy/webapps/galaxy/api/folders.py --- a/lib/galaxy/webapps/galaxy/api/folders.py +++ b/lib/galaxy/webapps/galaxy/api/folders.py @@ -20,7 +20,7 @@ super( FoldersController, self ).__init__( app ) self.folder_manager = folders.FolderManager() - @web.expose_api + @expose_api def index( self, trans, **kwd ): """ *GET /api/folders/ @@ -29,7 +29,7 @@ """ raise exceptions.NotImplemented( 'Listing all accessible library folders is not implemented.' ) - @web.expose_api + @expose_api def show( self, trans, id, **kwd ): """ show( self, trans, id, **kwd ) @@ -255,7 +255,7 @@ folder_dict = self.folder_manager.get_folder_dict( trans, folder ) return folder_dict - @web.expose_api + @expose_api def update( self, trans, id, library_id, payload, **kwd ): """ PUT /api/folders/{encoded_folder_id} diff -r 1d57664c6d9c4875e5c818220fa8199cf3967de0 -r 147c8b343d4ae09d4ae71d424037e45f06bd8e9e lib/galaxy/webapps/galaxy/api/library_contents.py --- a/lib/galaxy/webapps/galaxy/api/library_contents.py +++ b/lib/galaxy/webapps/galaxy/api/library_contents.py @@ -436,7 +436,7 @@ except exceptions.httpexceptions.HTTPException: raise except Exception, exc: - log.exception( 'HDA API, delete: uncaught exception: %s, %s\n%s', + log.exception( 'library_contents API, delete: uncaught exception: %s, %s\n%s', id, str( kwd ), str( exc ) ) trans.response.status = 500 rval.update({ 'error': str( exc ) }) Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.

1 0