Reduce logging on missing tool XML file
Hi all, I've got a patch to reduce the noise in the log output when a tool XML file is missing, e.g. galaxy.tools ERROR 2010-09-29 11:34:43,111 error reading tool from path: ./tools/data_source/hapmapmart.xml Traceback (most recent call last): File "/Users/xxx/repositories/galaxy-central/lib/galaxy/tools/__init__.py", line 75, in load_tool tool = self.load_tool( path ) File "/Users/xxx/repositories/galaxy-central/lib/galaxy/tools/__init__.py", line 133, in load_tool tree = util.parse_xml( config_file ) File "/Users/xxx/repositories/galaxy-central/lib/galaxy/util/__init__.py", line 102, in parse_xml tree = ElementTree.parse(fname) File "/Users/xxx/repositories/galaxy-central/eggs/elementtree-1.2.6_20050316-py2.5.egg/elementtree/ElementTree.py", line 859, in parse tree.parse(source, parser) File "/Users/xxx/repositories/galaxy-central/eggs/elementtree-1.2.6_20050316-py2.5.egg/elementtree/ElementTree.py", line 576, in parse source = open(source, "rb") IOError: [Errno 2] No such file or directory: './tools/data_source/hapmapmart.xml' becomes: galaxy.tools ERROR 2010-09-29 11:36:03,982 Tool file missing: ./tools/data_source/hapmapmart.xml Changeset here: http://bitbucket.org/peterjc/galaxy-central/changeset/5c9046c8bb08 Peter
On Wed, Sep 29, 2010 at 11:38 AM, Peter <peter@maubp.freeserve.co.uk> wrote:
Hi all,
I've got a patch to reduce the noise in the log output when a tool XML file is missing, e.g.
Here is a follow up patch for the remaining two common error tracebacks I see with a default galaxy-central installation. First, missing loc files in dynamic options: galaxy.tools ERROR 2010-09-29 11:36:04,606 error reading tool from path: ./tools/human_genome_variation/funDo.xml Traceback (most recent call last): File "/Users/xxx/repositories/galaxy-central/lib/galaxy/tools/__init__.py", line 75, in load_tool tool = self.load_tool( path ) File "/Users/xxx/repositories/galaxy-central/lib/galaxy/tools/__init__.py", line 146, in load_tool return ToolClass( config_file, root, self.app ) File "/Users/xxx/repositories/galaxy-central/lib/galaxy/tools/__init__.py", line 288, in __init__ self.parse( root ) File "/Users/xxx/repositories/galaxy-central/lib/galaxy/tools/__init__.py", line 381, in parse self.parse_inputs( root ) File "/Users/xxx/repositories/galaxy-central/lib/galaxy/tools/__init__.py", line 468, in parse_inputs display, inputs = self.parse_input_page( page, enctypes ) File "/Users/xxx/repositories/galaxy-central/lib/galaxy/tools/__init__.py", line 607, in parse_input_page inputs = self.parse_input_elem( input_elem, enctypes ) File "/Users/xxx/repositories/galaxy-central/lib/galaxy/tools/__init__.py", line 687, in parse_input_elem param = self.parse_param_elem( elem, enctypes, context ) File "/Users/xxx/repositories/galaxy-central/lib/galaxy/tools/__init__.py", line 697, in parse_param_elem param = ToolParameter.build( self, input_elem ) File "/Users/xxx/repositories/galaxy-central/lib/galaxy/tools/parameters/basic.py", line 148, in build return parameter_types[param_type]( tool, param ) File "/Users/xxx/repositories/galaxy-central/lib/galaxy/tools/parameters/basic.py", line 497, in __init__ self.options = dynamic_options.DynamicOptions( options, self ) File "/Users/xxx/repositories/galaxy-central/lib/galaxy/tools/parameters/dynamic_options.py", line 426, in __init__ self.file_fields = self.parse_file_fields( open( data_file ) ) IOError: [Errno 2] No such file or directory: '/Users/xxx/repositories/galaxy-central/tool-data/funDo.loc' becomes: galaxy.tools ERROR 2010-09-29 11:51:19,663 Tool ./tools/human_genome_variation/funDo.xml missing data file: /Users/xxx/repositories/galaxy-central/tool-data/funDo.loc Likewise for missing loc files in validation, this: galaxy.tools ERROR 2010-09-29 11:59:23,003 error reading tool from path: ./tools/human_genome_variation/sift.xml Traceback (most recent call last): File "/Users/xxx/repositories/galaxy-central/lib/galaxy/tools/__init__.py", line 77, in load_tool tool = self.load_tool( path ) File "/Users/xxx/repositories/galaxy-central/lib/galaxy/tools/__init__.py", line 150, in load_tool return ToolClass( config_file, root, self.app ) File "/Users/xxx/repositories/galaxy-central/lib/galaxy/tools/__init__.py", line 292, in __init__ self.parse( root ) File "/Users/xxx/repositories/galaxy-central/lib/galaxy/tools/__init__.py", line 385, in parse self.parse_inputs( root ) File "/Users/xxx/repositories/galaxy-central/lib/galaxy/tools/__init__.py", line 472, in parse_inputs display, inputs = self.parse_input_page( page, enctypes ) File "/Users/xxx/repositories/galaxy-central/lib/galaxy/tools/__init__.py", line 611, in parse_input_page inputs = self.parse_input_elem( input_elem, enctypes ) File "/Users/xxx/repositories/galaxy-central/lib/galaxy/tools/__init__.py", line 691, in parse_input_elem param = self.parse_param_elem( elem, enctypes, context ) File "/Users/xxx/repositories/galaxy-central/lib/galaxy/tools/__init__.py", line 701, in parse_param_elem param = ToolParameter.build( self, input_elem ) File "/Users/xxx/repositories/galaxy-central/lib/galaxy/tools/parameters/basic.py", line 153, in build return parameter_types[param_type]( tool, param ) File "/Users/xxx/repositories/galaxy-central/lib/galaxy/tools/parameters/basic.py", line 1152, in __init__ ToolParameter.__init__( self, tool, elem ) File "/Users/xxx/repositories/galaxy-central/lib/galaxy/tools/parameters/basic.py", line 46, in __init__ self.validators.append( validation.Validator.from_element( self, elem ) ) File "/Users/xxx/repositories/galaxy-central/lib/galaxy/tools/parameters/validation.py", line 23, in from_element return validator_types[type].from_element( param, elem ) File "/Users/xxx/repositories/galaxy-central/lib/galaxy/tools/parameters/validation.py", line 251, in from_element return cls( filename, metadata_name, metadata_column, message, line_startswith ) File "/Users/xxx/repositories/galaxy-central/lib/galaxy/tools/parameters/validation.py", line 256, in __init__ for line in open( filename ): IOError: [Errno 2] No such file or directory: '/Users/pjcock/repositories/galaxy-central/tool-data/sift_db.loc' becomes: galaxy.tools ERROR 2010-09-29 12:03:13,383 Tool ./tools/human_genome_variation/sift.xml missing data file: /Users/xxx/repositories/galaxy-central/tool-data/sift_db.loc In order to do this, I introduced a new exception - however I'm not sure which module it would be best to define this in. I hope something based on this patch will be committed: http://bitbucket.org/peterjc/galaxy-central/changeset/35344d053267 Peter
Peter;
I've got a patch to reduce the noise in the log output when a tool XML file is missing, e.g.
Here is a follow up patch for the remaining two common error tracebacks I see with a default galaxy-central installation. First, missing loc files in dynamic options:
Is it a good thing to be making this messages less noisy? Your proposal makes them easier to miss amongst all of the logging messages that are displayed when Galaxy starts up. These are issues that you'd want to identify and fix, instead of getting confused later wondering why a tool doesn't show up in the menu or have data files associated with it. Brad
On Wed, Sep 29, 2010 at 12:45 PM, Brad Chapman <chapmanb@50mail.com> wrote:
Peter;
I've got a patch to reduce the noise in the log output when a tool XML file is missing, e.g.
Here is a follow up patch for the remaining two common error tracebacks I see with a default galaxy-central installation. First, missing loc files in dynamic options:
Is it a good thing to be making this messages less noisy? Your proposal makes them easier to miss amongst all of the logging messages that are displayed when Galaxy starts up. These are issues that you'd want to identify and fix, instead of getting confused later wondering why a tool doesn't show up in the menu or have data files associated with it.
Hi Brad, They are still logged at ERROR priority, so if you are sending the logging somewhere central nothing important is lost. In these specific examples the traceback really is just noise. Rather than making the errors stand out by printing tracebacks, I would suggest the default Galaxy behaviour of printing the logging to stdout should ignore the DEBUG log entries - they make up the bulk of the output and (thus far) are not of interest to me, and really do hide any errors. Peter
Peter, fortunately you may be able to adjust that in your universe_wsgi.ini by changing log_level from the out of the box default of DEBUG to something less chatty like ERROR: # Specifies the factory for the universe WSGI application paste.app_factory = galaxy.web.buildapp:app_factory log_level = DEBUG On Wed, Sep 29, 2010 at 10:33 AM, Peter <peter@maubp.freeserve.co.uk> wrote:
On Wed, Sep 29, 2010 at 12:45 PM, Brad Chapman <chapmanb@50mail.com> wrote:
Peter;
I've got a patch to reduce the noise in the log output when a tool XML file is missing, e.g.
Here is a follow up patch for the remaining two common error tracebacks I see with a default galaxy-central installation. First, missing loc files in dynamic options:
Is it a good thing to be making this messages less noisy? Your proposal makes them easier to miss amongst all of the logging messages that are displayed when Galaxy starts up. These are issues that you'd want to identify and fix, instead of getting confused later wondering why a tool doesn't show up in the menu or have data files associated with it.
participants (3)
-
Brad Chapman -
Peter -
Ross