Re: [galaxy-dev] Bug in handling user job limits (due to dynamic job runner code?)
Hi Lance
I was just running into the same issue as part of upgrading our
production server from release 2012_10_05 to 2013_02_08 (unfortunately,
I didn't catch it last week, when I did our test server, as the job
limit was not set there).
I added a little comment to your trello "ticket", but I thought it might
be useful to know for a the wider community. The user job limit has been
very handy for us. And I am sure other Galaxy installations rely on this
as well.
Regards, Hans-Rudolf
On 01/18/2013 11:28 PM, Lance Parsons wrote:
Just an update on this issue. Upon further investigation, it looks
like
the dynamic job runner code in commit 6f3b4e8 broke this. I haven't been
able to parse through everything going on yet to propose a fix, but I've
discovered two things:
1) Moving the self.__clear_user_job_count() call outside of the if
block, does indeed fix that issue
2) Fixing that leads the __check_user_jobs method to fail since it does
not check `self.track_jobs_in_database`, but instead assumes that jobs
are in the database, resulting in a different exception regarding
invalid columns.
Lance Parsons wrote:
> I submitted a trello "ticket" [(https://trello.com/c/6vxkqdjT)
> regarding this, but wanted to make sure I brought it to someone's
> attention (it's causing me some queue issues with my instance of galaxy).
>
> When registered_user_job_limit and anonymous_user_job_limit are set in
> universe.wsgi jobs cannot be run, instead the following error occurs:
>
> galaxy.jobs.handler ERROR 2012-12-04 12:44:51,869 failure running job
> 22396
> Traceback (most recent call last):
> File "/data/local/galaxy/galaxy-prod/lib/galaxy/jobs/handler.py",
> line 183, in __monitor_step
> job_state = self.__check_if_ready_to_run( job )
> File "/data/local/galaxy/galaxy-prod/lib/galaxy/jobs/handler.py",
> line 253, in __check_if_ready_to_run
> state = self.__check_user_jobs( job )
> File "/data/local/galaxy/galaxy-prod/lib/galaxy/jobs/handler.py",
> line 274, in __check_user_jobs
> if not self.user_job_count:
> AttributeError: 'JobHandlerQueue' object has no attribute
> 'user_job_count'
>
> Commenting out the lines for registered_user_job_limit and
> anonymous_user_job_limit in universe.wsgi allows job to be queue one
> again.
> It looks like this is due to the fact that
> `self.__clear_user_job_count()` on line 159 of `handler.py` is only
> called when jobs are tracked in the database. If jobs are not tracked
> in the database (as in my case), the error occurs. Perhaps the fix
> would be to simply move the call outside the `if` block.
>
> It appears this was broken in the 2012-11-13 revision (73e05bc).
>
>