BugFix for dynamic_options ParamValueFilter to run IUC SnpEff in a workflow
The current code in: lib/galaxy/tools/parameters/dynamic_options.py only searches the top layer of the dict to find the dependency value. A fix is provide in pull request: #343: Need to traverse the other_value dict to find dependencies for ParamValueFilter in<https://bitbucket.org/galaxy/galaxy-central/pull-request/343/need-to-traverse-the-other_value-dict-to> SnpEff tool_config <inputs> <param format="vcf,tabular,pileup,bed" name="input" type="data" label="Sequence changes (SNPs, MNPs, InDels)"/> ... <conditional name="snpDb"> <param name="genomeSrc" type="select" label="Genome source"> <option value="cached">Locally installed reference genome</option> <option value="history">Reference genome from your history</option> <option value="named">Named on demand</option> </param> <when value="cached"> <param name="genomeVersion" type="select" label="Genome"> <!--GENOME DESCRIPTION--> <options from_data_table="snpeff_genomedb"> <filter type="unique_value" column="0" /> </options> </param> <param name="extra_annotations" type="select" display="checkboxes" multiple="true" label="Additional Annotations"> <help>These are available for only a few genomes</help> <options from_data_table="snpeff_annotations"> <filter type="param_value" ref="genomeVersion" key="genome" column="0" /> <filter type="unique_value" column="1" /> </options> </param> When running workflow: input.vcf -> SnpEff The values in ParamValueFilter filter_options function: |>>>| ||self*.*ref_name|| |'genomeVersion' ||>>>| ||other_values|| |{u'spliceSiteSize': '1', u'filterHomHet': 'no_filter', u'outputFormat': 'vcf', u'filterOut': None, u'inputFormat': 'vcf', u'filterIn': 'no_filter', u'udLength': '5000', u'generate_stats': True, u'noLog': True, u'chr': 'None', u'intervals': None, u'snpDb': {'extra_annotations': None, 'regulation': None, 'genomeVersion': 'GRCh37.71', 'genomeSrc': 'cached', '__current_case__': 0}, u'offset': '', u'input': <galaxy.tools.parameters.basic.DummyDataset object at 0x11451b8d0>, u'transcripts': None, u'annotations': ['-canon', '-lof', '-onlyReg']} || Since |||'genomeVersion' isn't in the keys of other_values, but rather in ||||||other_values['|||||||||snpDb'] this failed the assertion: |||||||||| assert self.ref_name in other_values, "Required dependency '%s' not found in incoming values" % self.ref_name Pull request |||||343: |||||<https://bitbucket.org/galaxy/galaxy-central/pull-request/343/need-to-traverse-the-other_value-dict-to> |||||$ hg diff lib/galaxy/tools/parameters/dynamic_options.py diff -r 95517f976cca lib/galaxy/tools/parameters/dynamic_options.py --- a/lib/galaxy/tools/parameters/dynamic_options.py Thu Feb 27 16:56:25 2014 -0500 +++ b/lib/galaxy/tools/parameters/dynamic_options.py Fri Feb 28 11:37:04 2014 -0600 @@ -177,8 +177,27 @@ return self.ref_name def filter_options( self, options, trans, other_values ): if trans is not None and trans.workflow_building_mode: return [] - assert self.ref_name in other_values, "Required dependency '%s' not found in incoming values" % self.ref_name - ref = other_values.get( self.ref_name, None ) + ## Depth first traversal to find the value for a dependency + def get_dep_value(param_name, dep_name, cur_val, layer): + dep_val = cur_val + if isinstance(layer, dict ): + if dep_name in layer: + dep_val = layer[dep_name] + if param_name in layer: + return dep_val + else: + for l in layer.itervalues(): + dep_val = get_dep_value(param_name, dep_name, dep_val, l) + if dep_val: + break + elif isinstance( layer, list): + for l in layer: + dep_val = get_dep_value(param_name, dep_name, dep_val, l) + if dep_val: + break + return None + ref = get_dep_value(self.dynamic_option.tool_param.name, self.ref_name, None, other_values) + assert not ref, "Required dependency '%s' not found in incoming values" % self.ref_name for ref_attribute in self.ref_attribute: if not hasattr( ref, ref_attribute ): return [] #ref does not have attribute, so we cannot filter, return empty list -- James E. Johnson, Minnesota Supercomputing Institute, University of Minnesota
Hey JJ, Thanks for the bug report. I can confirm the issue. I think the problem probably is that the other_values thing you are printing out there is very different when rendering tools (it is the value of things at that depth of the tool state tree) versus workflows (in which it is the global state of the tree from the top). The tool variant is probably the "right" approach since they work and has the nice advantage of avoiding ambiguities that arise otherwise. https://bitbucket.org/galaxy/galaxy-central/pull-request/349/bring-workflow-... I have opened a pull request with an attempt to bring workflows in line with tools - it seems to fix snpEff for me locally - can you confirm? Any chance this also solves your other problem with the XY plotting tool (Pull Request #336)? -John On Fri, Feb 28, 2014 at 12:12 PM, Jim Johnson <johns198@umn.edu> wrote:
The current code in: lib/galaxy/tools/parameters/dynamic_options.py only searches the top layer of the dict to find the dependency value.
A fix is provide in pull request: #343: Need to traverse the other_value dict to find dependencies for ParamValueFilter in
SnpEff tool_config
<inputs> <param format="vcf,tabular,pileup,bed" name="input" type="data" label="Sequence changes (SNPs, MNPs, InDels)"/> ... <conditional name="snpDb"> <param name="genomeSrc" type="select" label="Genome source"> <option value="cached">Locally installed reference genome</option> <option value="history">Reference genome from your history</option> <option value="named">Named on demand</option> </param> <when value="cached"> <param name="genomeVersion" type="select" label="Genome"> <!--GENOME DESCRIPTION--> <options from_data_table="snpeff_genomedb"> <filter type="unique_value" column="0" /> </options> </param> <param name="extra_annotations" type="select" display="checkboxes" multiple="true" label="Additional Annotations"> <help>These are available for only a few genomes</help> <options from_data_table="snpeff_annotations"> <filter type="param_value" ref="genomeVersion" key="genome" column="0" /> <filter type="unique_value" column="1" /> </options> </param>
When running workflow: input.vcf -> SnpEff
The values in ParamValueFilter filter_options function:
self.ref_name 'genomeVersion'
other_values {u'spliceSiteSize': '1', u'filterHomHet': 'no_filter', u'outputFormat': 'vcf', u'filterOut': None, u'inputFormat': 'vcf', u'filterIn': 'no_filter', u'udLength': '5000', u'generate_stats': True, u'noLog': True, u'chr': 'None', u'intervals': None, u'snpDb': {'extra_annotations': None, 'regulation': None, 'genomeVersion': 'GRCh37.71', 'genomeSrc': 'cached', '__current_case__': 0}, u'offset': '', u'input': <galaxy.tools.parameters.basic.DummyDataset object at 0x11451b8d0>, u'transcripts': None, u'annotations': ['-canon', '-lof', '-onlyReg']}
Since 'genomeVersion' isn't in the keys of other_values, but rather in other_values['snpDb'] this failed the assertion: assert self.ref_name in other_values, "Required dependency '%s' not found in incoming values" % self.ref_name
Pull request 343:
$ hg diff lib/galaxy/tools/parameters/dynamic_options.py diff -r 95517f976cca lib/galaxy/tools/parameters/dynamic_options.py --- a/lib/galaxy/tools/parameters/dynamic_options.py Thu Feb 27 16:56:25 2014 -0500 +++ b/lib/galaxy/tools/parameters/dynamic_options.py Fri Feb 28 11:37:04 2014 -0600 @@ -177,8 +177,27 @@ return self.ref_name def filter_options( self, options, trans, other_values ): if trans is not None and trans.workflow_building_mode: return [] - assert self.ref_name in other_values, "Required dependency '%s' not found in incoming values" % self.ref_name - ref = other_values.get( self.ref_name, None ) + ## Depth first traversal to find the value for a dependency + def get_dep_value(param_name, dep_name, cur_val, layer): + dep_val = cur_val + if isinstance(layer, dict ): + if dep_name in layer: + dep_val = layer[dep_name] + if param_name in layer: + return dep_val + else: + for l in layer.itervalues(): + dep_val = get_dep_value(param_name, dep_name, dep_val, l) + if dep_val: + break + elif isinstance( layer, list): + for l in layer: + dep_val = get_dep_value(param_name, dep_name, dep_val, l) + if dep_val: + break + return None + ref = get_dep_value(self.dynamic_option.tool_param.name, self.ref_name, None, other_values) + assert not ref, "Required dependency '%s' not found in incoming values" % self.ref_name for ref_attribute in self.ref_attribute: if not hasattr( ref, ref_attribute ): return [] #ref does not have attribute, so we cannot filter, return empty list
-- James E. Johnson, Minnesota Supercomputing Institute, University of Minnesota
Nice work John. This fixed issues for running workflows for both SnpEff and XYPlot. Please reject my pull requests : #336 and #343 in favor of #349 Thanks, JJ On 3/18/14, 11:44 AM, John Chilton wrote:
Hey JJ,
Thanks for the bug report. I can confirm the issue. I think the problem probably is that the other_values thing you are printing out there is very different when rendering tools (it is the value of things at that depth of the tool state tree) versus workflows (in which it is the global state of the tree from the top). The tool variant is probably the "right" approach since they work and has the nice advantage of avoiding ambiguities that arise otherwise.
https://bitbucket.org/galaxy/galaxy-central/pull-request/349/bring-workflow-...
I have opened a pull request with an attempt to bring workflows in line with tools - it seems to fix snpEff for me locally - can you confirm? Any chance this also solves your other problem with the XY plotting tool (Pull Request #336)?
-John
On Fri, Feb 28, 2014 at 12:12 PM, Jim Johnson <johns198@umn.edu> wrote:
The current code in: lib/galaxy/tools/parameters/dynamic_options.py only searches the top layer of the dict to find the dependency value.
A fix is provide in pull request: #343: Need to traverse the other_value dict to find dependencies for ParamValueFilter in
SnpEff tool_config
<inputs> <param format="vcf,tabular,pileup,bed" name="input" type="data" label="Sequence changes (SNPs, MNPs, InDels)"/> ... <conditional name="snpDb"> <param name="genomeSrc" type="select" label="Genome source"> <option value="cached">Locally installed reference genome</option> <option value="history">Reference genome from your history</option> <option value="named">Named on demand</option> </param> <when value="cached"> <param name="genomeVersion" type="select" label="Genome"> <!--GENOME DESCRIPTION--> <options from_data_table="snpeff_genomedb"> <filter type="unique_value" column="0" /> </options> </param> <param name="extra_annotations" type="select" display="checkboxes" multiple="true" label="Additional Annotations"> <help>These are available for only a few genomes</help> <options from_data_table="snpeff_annotations"> <filter type="param_value" ref="genomeVersion" key="genome" column="0" /> <filter type="unique_value" column="1" /> </options> </param>
When running workflow: input.vcf -> SnpEff
The values in ParamValueFilter filter_options function:
self.ref_name 'genomeVersion'
other_values {u'spliceSiteSize': '1', u'filterHomHet': 'no_filter', u'outputFormat': 'vcf', u'filterOut': None, u'inputFormat': 'vcf', u'filterIn': 'no_filter', u'udLength': '5000', u'generate_stats': True, u'noLog': True, u'chr': 'None', u'intervals': None, u'snpDb': {'extra_annotations': None, 'regulation': None, 'genomeVersion': 'GRCh37.71', 'genomeSrc': 'cached', '__current_case__': 0}, u'offset': '', u'input': <galaxy.tools.parameters.basic.DummyDataset object at 0x11451b8d0>, u'transcripts': None, u'annotations': ['-canon', '-lof', '-onlyReg']}
Since 'genomeVersion' isn't in the keys of other_values, but rather in other_values['snpDb'] this failed the assertion: assert self.ref_name in other_values, "Required dependency '%s' not found in incoming values" % self.ref_name
Pull request 343:
$ hg diff lib/galaxy/tools/parameters/dynamic_options.py diff -r 95517f976cca lib/galaxy/tools/parameters/dynamic_options.py --- a/lib/galaxy/tools/parameters/dynamic_options.py Thu Feb 27 16:56:25 2014 -0500 +++ b/lib/galaxy/tools/parameters/dynamic_options.py Fri Feb 28 11:37:04 2014 -0600 @@ -177,8 +177,27 @@ return self.ref_name def filter_options( self, options, trans, other_values ): if trans is not None and trans.workflow_building_mode: return [] - assert self.ref_name in other_values, "Required dependency '%s' not found in incoming values" % self.ref_name - ref = other_values.get( self.ref_name, None ) + ## Depth first traversal to find the value for a dependency + def get_dep_value(param_name, dep_name, cur_val, layer): + dep_val = cur_val + if isinstance(layer, dict ): + if dep_name in layer: + dep_val = layer[dep_name] + if param_name in layer: + return dep_val + else: + for l in layer.itervalues(): + dep_val = get_dep_value(param_name, dep_name, dep_val, l) + if dep_val: + break + elif isinstance( layer, list): + for l in layer: + dep_val = get_dep_value(param_name, dep_name, dep_val, l) + if dep_val: + break + return None + ref = get_dep_value(self.dynamic_option.tool_param.name, self.ref_name, None, other_values) + assert not ref, "Required dependency '%s' not found in incoming values" % self.ref_name for ref_attribute in self.ref_attribute: if not hasattr( ref, ref_attribute ): return [] #ref does not have attribute, so we cannot filter, return empty list
-- James E. Johnson, Minnesota Supercomputing Institute, University of Minnesota
-- James E. Johnson, Minnesota Supercomputing Institute, University of Minnesota
participants (2)
-
Jim Johnson -
John Chilton