Hi guys, I'm running a galaxy cloudman instance and running the usual tophat->cufflinks->cuffdiff workflow from RNAseq data.I am using a m2.4xlarge as a master node, and autoscaling from 0-4 workers of the m2.xlarge type.
I have gotten the error: An error occurred running this job: Job output not returned from cluster
when running fasta groomer, tophat, and now cufflinks.
Following up troubleshooting from other people in the mailing list, I have set a new line in universe_wsgi.ini of retry_job_output_collection=30
Unfortunately, this does not seem to have fixed the problem.
The stdout is blank, and stderr gives Job output not returned from cluster
Under manage jobs in the admin panel, it lists 4 out of the 6 jobs as currently running. What is confusing is that of the 4 "running," one has already returned the error in the user dataset panel and yet is still listed as running.
From the SGE log, I see these errors:
03/11/2013 14:32:52|worker|ip-10-159-47-223|W|job 42.1 failed on host ip-10-30-130-84.ec2.internal before writing exit_status because: shepherd exited with exit status 19: before writing exit_status
03/11/2013 14:32:52|worker|ip-10-159-47-223|W|job 43.1 failed on host ip-10-30-130-84.ec2.internal before writing exit_status because: shepherd exited with exit status 19: before writing exit_status
03/11/2013 14:39:41|worker|ip-10-159-47-223|E|adminhost "ip-10-30-130-84.ec2.internal" already exists
03/11/2013 14:40:07|worker|ip-10-159-47-223|E|exechost "ip-10-30-130-84.ec2.internal" already exists
03/11/2013 14:50:54|worker|ip-10-159-47-223|E|adminhost "ip-10-30-130-84.ec2.internal" already exists
03/11/2013 14:50:55|worker|ip-10-159-47-223|E|exechost "ip-10-30-130-84.ec2.internal" already exists
Does anyone have any idea how to solve this error? It has removed my ability to use workflows completely and I still have not been able to run a single analysis to completion due to it.
Thanks for any insight anyone provide!
Brian