Re: [galaxy-dev] [patch] Job Runner Eggs

John Chilton wrote:

Hello,

Currently lib/galaxy/config.py requires that custom job runners

be paired with exactly one egg for fetching, this seems less than ideal since runners may need 0 eggs (as is the case for a runner I am implementing) or multiple eggs. Attached is a minor patch that addresses this. Please consider it for addition to galaxy-central.

Thanks John,

I'm actually going to hold off on applying this. The reason is that I'd like for job runners to be more truly pluggable, since right now adding new ones requires adding them to config.py and jobs/__init__.py. A runner should really only need to be added to the start_job_runners option in the config to be started, and the eggs can be handled in the runner itself.

This is on my list of things to fix, and shouldn't take too long. I've created an issue to track it here:

https://bitbucket.org/galaxy/galaxy-central/issue/449/remove-per-runner-code...

--nate

Thanks for your time, John

---

John Chilton Software Developer University of Minnesota Supercomputing Institute Office: 612-625-0917 Cell: 612-226-9223 E-Mail: chilton@msi.umn.edu

exporting patch: # HG changeset patch # User John Chilton jmchilton@gmail.com # Date 1294805231 21600 # Node ID edc242c07824ea900ab72dbaf22f338b1288d275 # Parent b5c64c4db5eadaed6a06e99960152cd4965fb184 Minor refactoring of the way lib/galaxy/config.py configures eggs for custom job runners meant to increase flexibility for new job runners.

diff -r b5c64c4db5ea -r edc242c07824 lib/galaxy/config.py --- a/lib/galaxy/config.py Tue Jan 11 20:25:19 2011 -0500 +++ b/lib/galaxy/config.py Tue Jan 11 22:07:11 2011 -0600 @@ -185,14 +185,15 @@ raise ConfigurationError("File not found: %s" % path ) # Check job runners so the admin can scramble dependent egg. if self.start_job_runners is not None:

•        runner_to_egg = dict( pbs = 'pbs_python', sge = 'DRMAA_python', drmaa = 'drmaa' )
  

•        runner_to_eggs = dict( pbs = ['pbs_python'], sge = ['DRMAA_python'], drmaa = ['drmaa'] )
         for runner in self.start_job_runners.split( ',' ):
  

•            try:
  

•                pkg_resources.require( runner_to_egg[runner] )
  

•            except eggs.EggNotFetchable, e:
  

•                raise eggs.EggNotFetchable( 'You must scramble the %s egg to use the %s job runner.  Instructions are available at:\n  http://bitbucket.org/galaxy/galaxy-central/wiki/Config/Cluster' % ( runner_to_egg[runner], runner ) )
  

•            except KeyError:
  

•            if not runner in runner_to_eggs:
                 raise Exception( 'No such job runner: %s.  Please double-check the value of start_job_runners in universe_wsgi.ini' % runner )
  

•            for egg in runner_to_eggs[runner]:
  

•                try:
  

•                    pkg_resources.require( egg )
  

•                except eggs.EggNotFetchable, e:
  

•                    raise eggs.EggNotFetchable( 'You must scramble the %s egg to use the %s job runner.  Instructions are available at:\n  http://bitbucket.org/galaxy/galaxy-central/wiki/Config/Cluster' % ( egg, runner ) )
     # Check for deprecated options.
     for key in self.config_dict.keys():
         if key in self.deprecated_options:
  

---

galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev