9 new commits in galaxy-central: https://bitbucket.org/galaxy/galaxy-central/commits/52351e7d6433/ Changeset: 52351e7d6433 User: jmchilton Date: 2013-03-26 16:30:46 Summary: Fix typo related to lwr in job_conf.xml.sample_advanced. Affected #: 1 file diff -r 504264153fe1804c409c775be2c6db3f160b8fe2 -r 52351e7d6433973cac61a9822f20559560e10491 job_conf.xml.sample_advanced --- a/job_conf.xml.sample_advanced +++ b/job_conf.xml.sample_advanced @@ -7,7 +7,7 @@ <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner"/><plugin id="pbs" type="runner" load="galaxy.jobs.runners.pbs:PBSJobRunner" workers="2"/><plugin id="drmaa" type="runner" load="galaxy.jobs.runners.drmaa:DRMAARunner"/> - <plugin id="lwr" type="runner" load="galaxy.jobs.runners.lwr.LwrJobRunner" /><!-- https://lwr.readthedocs.org --> + <plugin id="lwr" type="runner" load="galaxy.jobs.runners.lwr:LwrJobRunner" /><!-- https://lwr.readthedocs.org --><plugin id="cli" type="runner" load="galaxy.jobs.runners.cli:ShellJobRunner" /><plugin id="condor" type="runner" load="galaxy.jobs.runners.condor:CondorJobRunner" /></plugins> https://bitbucket.org/galaxy/galaxy-central/commits/7aba79e8068c/ Changeset: 7aba79e8068c User: jmchilton Date: 2013-03-26 17:16:30 Summary: Fix returning destination id from dynamic job runners, JobMapper was using self.app.job_config but it didn't have access to app, now JobWrapper is passing job_config into JobMapper which in turn is storing it. Affected #: 2 files diff -r 52351e7d6433973cac61a9822f20559560e10491 -r 7aba79e8068c658fe7a1a7f412f999ac276c1940 lib/galaxy/jobs/__init__.py --- a/lib/galaxy/jobs/__init__.py +++ b/lib/galaxy/jobs/__init__.py @@ -580,7 +580,7 @@ self.tool_provided_job_metadata = None # Wrapper holding the info required to restore and clean up from files used for setting metadata externally self.external_output_metadata = metadata.JobExternalOutputMetadataWrapper( job ) - self.job_runner_mapper = JobRunnerMapper( self, queue.dispatcher.url_to_destination ) + self.job_runner_mapper = JobRunnerMapper( self, queue.dispatcher.url_to_destination, self.app.job_config ) self.params = None if job.params: self.params = from_json_string( job.params ) diff -r 52351e7d6433973cac61a9822f20559560e10491 -r 7aba79e8068c658fe7a1a7f412f999ac276c1940 lib/galaxy/jobs/mapper.py --- a/lib/galaxy/jobs/mapper.py +++ b/lib/galaxy/jobs/mapper.py @@ -21,9 +21,10 @@ (in the form of job_wrappers) to job runner url strings. """ - def __init__( self, job_wrapper, url_to_destination ): + def __init__( self, job_wrapper, url_to_destination, job_config ): self.job_wrapper = job_wrapper self.url_to_destination = url_to_destination + self.job_config = job_config self.rule_modules = self.__get_rule_modules( ) def __get_rule_modules( self ): @@ -135,7 +136,7 @@ if '://' in rval: return self.__convert_url_to_destination(rval) else: - return self.app.job_config.get_destination(rval) + return self.job_config.get_destination(rval) elif isinstance(rval, galaxy.jobs.JobDestination): # If the function generated a JobDestination, we'll use that # destination directly. However, for advanced job limiting, a https://bitbucket.org/galaxy/galaxy-central/commits/150660f80ca1/ Changeset: 150660f80ca1 User: jmchilton Date: 2013-03-26 17:18:36 Summary: Slight optimization in JobMapper, no need to prefetch rules modules and hit the disk for every job mapper if 99.9% of them are never going to use rules. Affected #: 1 file diff -r 7aba79e8068c658fe7a1a7f412f999ac276c1940 -r 150660f80ca10c1d31b0d60ca2e9ae3f9fa8dbee lib/galaxy/jobs/mapper.py --- a/lib/galaxy/jobs/mapper.py +++ b/lib/galaxy/jobs/mapper.py @@ -25,7 +25,6 @@ self.job_wrapper = job_wrapper self.url_to_destination = url_to_destination self.job_config = job_config - self.rule_modules = self.__get_rule_modules( ) def __get_rule_modules( self ): unsorted_module_names = self.__get_rule_module_names( ) @@ -119,7 +118,7 @@ def __last_rule_module_with_function( self, function_name ): # self.rule_modules is sorted in reverse order, so find first # wiht function - for rule_module in self.rule_modules: + for rule_module in self.__get_rule_modules( ): if hasattr( rule_module, function_name ): return rule_module return None https://bitbucket.org/galaxy/galaxy-central/commits/9bc03c707d69/ Changeset: 9bc03c707d69 User: jmchilton Date: 2013-03-26 17:29:23 Summary: Cleanup/reworking of logic in JobMapper for handling multiple possible output types coming from dynamic rules. This is a little more pythonic in that it is making fewer assumptions about types. This is also slightly less code, reduces the number of return statements, etc.... Also removed the the #TODO: Test extensively, this has now been done. Affected #: 1 file diff -r 150660f80ca10c1d31b0d60ca2e9ae3f9fa8dbee -r 9bc03c707d69c165b07dcdc07ea9d60efd11d74a lib/galaxy/jobs/mapper.py --- a/lib/galaxy/jobs/mapper.py +++ b/lib/galaxy/jobs/mapper.py @@ -128,27 +128,14 @@ if expand_type == "python": expand_function_name = self.__determine_expand_function_name( destination ) expand_function = self.__get_expand_function( expand_function_name ) - rval = self.__invoke_expand_function( expand_function ) - # TODO: test me extensively - if isinstance(rval, basestring): - # If the function returned a string, check if it's a URL, convert if necessary - if '://' in rval: - return self.__convert_url_to_destination(rval) + job_destination = self.__invoke_expand_function( expand_function ) + if not isinstance(job_destination, galaxy.jobs.JobDestination): + job_destination_rep = str(job_destination) # Should be either id or url + if '://' in job_destination_rep: + job_destination = self.__convert_url_to_destination(job_destination_rep) else: - return self.job_config.get_destination(rval) - elif isinstance(rval, galaxy.jobs.JobDestination): - # If the function generated a JobDestination, we'll use that - # destination directly. However, for advanced job limiting, a - # function may want to set the JobDestination's 'tags' - # attribute so that limiting can be done on a destination tag. - #id_or_tag = rval.get('id') - #if rval.get('tags', None): - # # functions that are generating destinations should only define one tag - # id_or_tag = rval.get('tags')[0] - #return id_or_tag, rval - return rval - else: - raise Exception( 'Dynamic function returned a value that could not be understood: %s' % rval ) + job_destination = self.job_config.get_destination(job_destination_rep) + return job_destination elif expand_type is None: raise Exception( 'Dynamic function type not specified (hint: add <param id="type">python</param> to your <destination>)' ) else: https://bitbucket.org/galaxy/galaxy-central/commits/7cf608b34231/ Changeset: 7cf608b34231 User: jmchilton Date: 2013-03-26 19:14:50 Summary: Dynamic job runner will take id of the tool as the python function if one is not specified. This extends that logic to handle newer tools (i.e. tool shed tools) that may have multiple ids. The most specific id with a corresponding function name will be used. Affected #: 2 files Diff not available. https://bitbucket.org/galaxy/galaxy-central/commits/14a5aecf14cf/ Changeset: 14a5aecf14cf User: jmchilton Date: 2013-03-26 21:24:25 Summary: Specify `python` as default dynamic job runner type - simplifies configuration and documentation and other types cheetah, XML, etc... do not seem to be on the roadmap. Affected #: 1 file Diff not available. https://bitbucket.org/galaxy/galaxy-central/commits/bdc4af97dbed/ Changeset: bdc4af97dbed User: jmchilton Date: 2013-03-27 21:19:57 Summary: Bug fix for recent changes to the LWR server's handling of config files. Affected #: 1 file Diff not available. https://bitbucket.org/galaxy/galaxy-central/commits/f63846588983/ Changeset: f63846588983 User: jmchilton Date: 2013-03-27 21:27:38 Summary: Small documentation update to reflect changes in 14a5aec. Affected #: 1 file Diff not available. https://bitbucket.org/galaxy/galaxy-central/commits/cb25513c63cd/ Changeset: cb25513c63cd User: natefoo Date: 2013-04-05 23:05:41 Summary: Merged in jmchilton/galaxy-central-multi-input-tool-fixes-2 (pull request #143) Affected #: 5 files Diff not available. Repository URL: https://bitbucket.org/galaxy/galaxy-central/ -- This is a commit notification from bitbucket.org. You are receiving this because you have the service enabled, addressing the recipient of this email.