Galaxy: problem with job status update
Hello, I have installed an instance of Galaxy on our server. I configured it to run the jobs on our cluster (SGE). Everything works just fine. I can run NGS jobs that run on our cluster nodes and return results to Galaxy. But, sometimes, I have a problem with the job status that is not updated! It can be flagged as "waiting" and never run, or, even worst, stay flagged as "running". When I check the logs I can see that the job has finished normally [1]. As the step is not flagged as "done" I cannot go further with my workflow. Any help would be welcomed! Thanks for your great work with Galaxy! Best regards, Tanguy LE CARROUR [1] Galaxy logs sample: galaxy.jobs DEBUG 2010-08-25 11:59:45,727 dispatching job 8 to sge runner galaxy.jobs.runners.sge DEBUG 2010-08-25 11:59:47,935 (8) submitting file /srv/GT/www/htdocs/Galaxy/new_galaxy/database/pbs/galaxy_8.sh galaxy.jobs.runners.sge DEBUG 2010-08-25 11:59:47,936 (8) command is: python /srv/GT/www/htdocs/Galaxy/new_galaxy/tools/samtools/sam_to_bam.py --input1=/srv/GT/www/htdocs/Galaxy/new_galaxy/database/files/000/dataset_9.dat --dbkey=hg19 --ref_file="None" --output1=/srv/GT/www/htdocs/Galaxy/new_galaxy/database/files/000/dataset_10.dat --index_dir=/srv/GT/www/htdocs/Galaxy/new_galaxy/tool-data galaxy.jobs.runners.sge DEBUG 2010-08-25 11:59:47,941 (8) queued in ims.q queue as 202466 galaxy.jobs.runners.sge DEBUG 2010-08-25 11:59:48,590 (8/202466) state change: job is queued and waiting to be scheduled galaxy.jobs.runners.sge DEBUG 2010-08-25 11:59:54,594 (8/202466) state change: job is running galaxy.jobs.runners.sge DEBUG 2010-08-25 12:02:58,979 (8/202466) state change: process status cannot be determined galaxy.jobs.runners.sge DEBUG 2010-08-25 12:02:59,979 (8/202466) state change: job finished normally -- Tanguy LE CARROUR Functional Genomics Center Zurich ETH Zurich / University of Zurich Winterthurerstrasse 190 8057 Zurich Switzerland
Tanguy Le Carrour wrote:
Hello,
I have installed an instance of Galaxy on our server. I configured it to run the jobs on our cluster (SGE). Everything works just fine. I can run NGS jobs that run on our cluster nodes and return results to Galaxy.
But, sometimes, I have a problem with the job status that is not updated! It can be flagged as "waiting" and never run, or, even worst, stay flagged as "running". When I check the logs I can see that the job has finished normally [1]. As the step is not flagged as "done" I cannot go further with my workflow.
Any help would be welcomed!
Thanks for your great work with Galaxy! Best regards,
Tanguy LE CARROUR
[1] Galaxy logs sample: galaxy.jobs DEBUG 2010-08-25 11:59:45,727 dispatching job 8 to sge runner galaxy.jobs.runners.sge DEBUG 2010-08-25 11:59:47,935 (8) submitting file /srv/GT/www/htdocs/Galaxy/new_galaxy/database/pbs/galaxy_8.sh galaxy.jobs.runners.sge DEBUG 2010-08-25 11:59:47,936 (8) command is: python /srv/GT/www/htdocs/Galaxy/new_galaxy/tools/samtools/sam_to_bam.py --input1=/srv/GT/www/htdocs/Galaxy/new_galaxy/database/files/000/dataset_9.dat --dbkey=hg19 --ref_file="None" --output1=/srv/GT/www/htdocs/Galaxy/new_galaxy/database/files/000/dataset_10.dat --index_dir=/srv/GT/www/htdocs/Galaxy/new_galaxy/tool-data galaxy.jobs.runners.sge DEBUG 2010-08-25 11:59:47,941 (8) queued in ims.q queue as 202466 galaxy.jobs.runners.sge DEBUG 2010-08-25 11:59:48,590 (8/202466) state change: job is queued and waiting to be scheduled galaxy.jobs.runners.sge DEBUG 2010-08-25 11:59:54,594 (8/202466) state change: job is running
Hi Tanguy, On jobs for which this happens, does this line:
galaxy.jobs.runners.sge DEBUG 2010-08-25 12:02:58,979 (8/202466) state change: process status cannot be determined
always appear? If so, does it appear on any jobs that DO finish correctly? Also, we have recently deprecated the sge job runner in favor of the more generic drmaa job runner. Please see the documentation at: http://bitbucket.org/galaxy/galaxy-central/wiki/Config/Cluster Thanks, --nate
galaxy.jobs.runners.sge DEBUG 2010-08-25 12:02:59,979 (8/202466) state change: job finished normally
-- Tanguy LE CARROUR
Functional Genomics Center Zurich ETH Zurich / University of Zurich Winterthurerstrasse 190 8057 Zurich Switzerland
------------------------------------------------------------------------
_______________________________________________ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
participants (2)
-
Nate Coraor
-
Tanguy Le Carrour