Chorny, Ilya wrote:
So it looks like for some reason galaxy is not picking up the information in the .e file. Not sure why. I even switched it back to writing the files to the pbs directory and it still did not pick up the .e files. BWA was core dumping but I was not given the error message in galaxy.
Okay, this is pretty odd, I'll try to see what's going on here. If you figure anything out, please let me know.
BTW, I find it nice to be able to see the .o and .e files. The way you have it coded the .o and .e are written out to the job_working_directory and then disappear at the end of the run. But the way I had it coded the pbs directory is world read writable. Is there an issue with having the pbs directory be world read write?
I imagine for some environments, it's preferable to avoid world-writable directories. Also, going forward, containing all of a job's filesystem data to a single directory will be important for running on multiple clusters without access to a single shared filesystem mounted at the same path. There's an issue in bitbucket somewhere regarding saving standard out and standard error for all jobs, which would solve the problem of it being in the working directory. For debugging, if you comment out shutil.rmtree( self.working_directory ) in jobs.cleanup(), it'll leave everything behind. --nate
Thanks,
Ilya
-----Original Message----- From: galaxy-dev-bounces@lists.bx.psu.edu [mailto:galaxy-dev-bounces@lists.bx.psu.edu] On Behalf Of Chorny, Ilya Sent: Monday, October 31, 2011 8:37 PM To: Nate Coraor (nate@bx.psu.edu) Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Re Acutal user code
It seems to be working but when I run BWA or Bowtie it finishes green but the files are empty. Doesn't give me much information. I also tried fastqc and it ran just fine.
-----Original Message----- From: Nate Coraor (nate@bx.psu.edu) [mailto:nate@bx.psu.edu] Sent: Monday, October 31, 2011 1:21 PM To: Chorny, Ilya Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: Re Acutal user code
Chorny, Ilya wrote:
Sweet!! Can't wait to see it.
Have a look:
https://bitbucket.org/natefoo/galaxy-central-ichorny/
Caveats:
* job_working_directory has to be in a shared filesystem * outputs_to_working_directory must be True * set_metadata_externally must be True
One thing that would be good from a security standpoint would be if the external chown script took a Galaxy Job ID for its argument instead of a path, and then looked up the working directory path from the ID. That way the script couldn't be used maliciously to chown any file.
--nate
Thanks,
Ilya
-----Original Message----- From: Nate Coraor (nate@bx.psu.edu) [mailto:nate@bx.psu.edu] Sent: Thursday, October 27, 2011 1:24 PM To: Chorny, Ilya Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: Re Acutal user code
Yup, cleaning up some things that were accidentally lost in merges. I added another bitbucket repo so as not to mess up yours.
Chorny, Ilya wrote:
Hi Nate,
Any updates?
Thanks,
Ilya
-----Original Message----- From: galaxy-dev-bounces@lists.bx.psu.edu [mailto:galaxy-dev-bounces@lists.bx.psu.edu] On Behalf Of Chorny, Ilya Sent: Friday, October 21, 2011 2:24 PM To: Nate Coraor (nate@bx.psu.edu) Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Re Acutal user code
One more thing there are some changes in lib/galaxy/tools/__init__.py and lib/galaxy/model/__init__.py that are not related to actual user. They look like they are toolshed related and may have been changed because to version of the file that I copied over was older then galaxy-central. Probably should have done a merge first. I can redo it if you like. Let me know.
Ilya
-----Original Message----- From: Chorny, Ilya Sent: Friday, October 21, 2011 2:11 PM To: Chorny, Ilya; Nate Coraor (nate@bx.psu.edu) Cc: galaxy-dev@lists.bx.psu.edu Subject: RE: Re Acutal user code
Sorry, the link was wrong.
https://bitbucket.org/ichorny/galaxy-central-ilya2/
-----Original Message----- From: galaxy-dev-bounces@lists.bx.psu.edu [mailto:galaxy-dev-bounces@lists.bx.psu.edu] On Behalf Of Chorny, Ilya Sent: Friday, October 21, 2011 2:07 PM To: Nate Coraor (nate@bx.psu.edu) Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Re Acutal user code
https://bitbucket.org/ichorny/galaxy-central-ilya2/pull-request/new
everything but the tophat/cufflinks wrapper.
Thanks,
Ilya
-----Original Message----- From: Nate Coraor (nate@bx.psu.edu) [mailto:nate@bx.psu.edu] Sent: Friday, October 21, 2011 12:56 PM To: Chorny, Ilya Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: Re Acutal user code
Chorny, Ilya wrote:
Nate,
I have been trying to figure out how to make the changes you suggested (i.e. write output files to the job_working_directory) and then move them over to files but I am having a difficult time navigating the code. If you could point me in the right direction, that would be helpful. I know the lib/galaxy/models/__init__.py initializes the output files but I don't think that is the right place to make the change. This change probably needs to be done in the Job_Wrapper but most of the code is very cryptic to me. Any guidance would be greatly appreciated. These changes should not take very long to implement once I get a handle on the code.
Hi Ilya,
I'm having a lot of difficulty separating out the actual user code from the other code in your fork. Could you create a fork with just this code?
Thanks, --nate
Thanks,
Ilya
Ilya Chorny Ph.D. Bioinformatics Scientist I Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Work: 858.202.4582 Email: ichorny@illumina.com<mailto:ichorny@illumina.com> Website: www.illumina.com<http://www.illumina.com>
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: