On 19/03/10 12:51, Nate Coraor wrote:
Chris Cole wrote:
Hi,
Occasionally, we see jobs that are constantly in the 'job running' state in the history. This is despite the fact that the job completed correctly. I can also see in the paster.log file that Galaxy has picked up on the fact that the job ended, but it just hasn't updated the UI. Clicking the eye for the 'running' history items does show the results of the run.
Hi Nate, Sorry for not getting back earlier. Been waiting for another one to occur.
Is there anything unusual displayed in the paster log when this happens?
Not as far I can see. This is the progress from paster.log of a job that has finished, but is still 'running' in the history: galaxy.jobs DEBUG 2010-03-26 13:43:27,245 job 1493 put in policy queue galaxy.jobs.schedulingpolicy.roundrobin DEBUG 2010-03-26 13:43:27,245 RoundRobin queue: retrieving job from job queue for session = 369 galaxy.jobs DEBUG 2010-03-26 13:43:27,245 dispatching job 1493 to sge runner 10.31.3.203 - - [26/Mar/2010:13:43:26 +0100] "POST /tool_runner/index HTTP/1.1" 200 - "http://wsdev.compbio.dundee.ac.uk:3216/tool_runner/rerun?id=1533" "Moz illa/5.0 (X11; U; Linux i686; en-GB; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2" 10.31.3.203 - - [26/Mar/2010:13:43:27 +0100] "GET /history HTTP/1.1" 200 - "http://wsdev.compbio.dundee.ac.uk:3216/tool_runner/index" "Mozilla/5.0 (X11; U; L inux i686; en-GB; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2" galaxy.jobs.runners.sge DEBUG 2010-03-26 13:43:28,624 (1493) submitting file /homes/www-galaxy/galaxy_devel/database/pbs/galaxy_1493.sh galaxy.jobs.runners.sge DEBUG 2010-03-26 13:43:28,625 (1493) command is: python /homes/www-galaxy/galaxy_devel/tools/sr_mapping/bowtie_wrapper.py --thre ads="4" --dataType="solexa" --output=/homes/www-galaxy/galaxy_devel/database/files/001/dataset_1619.dat --suppressHeader=false --genomeSource =indexed --snpphred="None" --snpfrac="None" --keepends="None" --ref=/db/bowtie/dicty_genomic --indexSettings="None" --iautoB="None " --ipacked="None" --ibmax="None" --ibmaxdivn="None" --idcv="None" --inodc="None" --inoref="None" --ioffrate="None" - -iftab="None" --intoa="None" --iendian="None" --iseed="None" --icutoff="None" --paired=single --input1=/homes/www-galaxy/galaxy_ devel/database/files/001/dataset_1593.dat --input2="None" --params=full --skip=0 --alignLimit=-1 --trimH=0 --trimL=0 --mismatchSeed=2 --mismatchQual=70 --seedLen=18 --rounding=noRound --maqSoapAlign=-1 --tryHard=noTryHard --valAlign=1 --allValAligns=doAllValAligns --suppressAlign=40 --best=doBest --maxBacktracks=800 --strata=doStrata --offrate=-1 -- seed=-1 --minInsert="None" --maxInsert="None" --mateOrient="None" --maxAlignAttempt="None" --forwardAlign="None" --reverseAlign ="None" galaxy.jobs.runners.sge DEBUG 2010-03-26 13:43:28,640 (1493) queued in 64bit-pri.q queue as 789899 galaxy.jobs DEBUG 2010-03-26 13:43:28,793 job 1493 dispatched galaxy.jobs.runners.sge DEBUG 2010-03-26 13:43:29,775 (1493/789899) state change: job is queued and waiting to be scheduled 10.31.3.203 - - [26/Mar/2010:13:43:32 +0100] "POST /root/history_item_updates HTTP/1.1" 200 - "http://wsdev.compbio.dundee.ac.uk:3216/history" "Mozilla/5.0 ( X11; U; Linux i686; en-GB; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2" 10.31.3.203 - - [26/Mar/2010:13:43:35 +0100] "POST /root/history_item_updates HTTP/1.1" 200 - "http://wsdev.compbio.dundee.ac.uk:3216/history" "Mozilla/5.0 ( X11; U; Linux i686; en-GB; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2" galaxy.jobs.runners.sge DEBUG 2010-03-26 13:43:37,790 (1493/789899) state change: job is running 10.31.3.203 - - [26/Mar/2010:13:43:38 +0100] "POST /root/history_item_updates HTTP/1.1" 200 - "http://wsdev.compbio.dundee.ac.uk:3216/history" "Mozilla/5.0 ( X11; U; Linux i686; en-GB; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2" galaxy.jobs.schedulingpolicy.roundrobin DEBUG 2010-03-26 13:43:38,846 RoundRobin queue clean up: Removed job queue entry from dictionary for session = 369 THen a while later... galaxy.jobs.runners.sge DEBUG 2010-03-26 13:49:06,988 (1493/789899) state change: job finished normally galaxy.jobs DEBUG 2010-03-26 13:49:07,424 job 1493 ended Yet, here I am at 14:34 and the job is still 'running' in the history. Any ideas? Chris p.s. Typical! Just as I was about to click send, it turned green... Still, is there any reason for the delay?