Hi Jeremy, The symlink in the cuffcompare wrapper was causing galaxy to crash because I run as the actual user and have to chmod the job_working directory at the end so Galaxy can clean it up. Turns out is seems like the symlink is not needed. Am I missing something. See below. Your code: for i, arg in enumerate( args ): input_file_name = "./input%i" % ( i+1 ) os.symlink( arg, input_file_name ) cmd += "%s " % input_file_name My code: for i, arg in enumerate( args ): cmd += arg Ilya Chorny Ph.D. Bioinformatics Scientist I Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Work: 858.202.4582 Email: ichorny@illumina.com<mailto:ichorny@illumina.com> Website: www.illumina.com<http://www.illumina.com>
Ok, I figured out why you need the symlink. Can you add an unlink after the process completes? i.e for i, arg in enumerate( args ): input_file_name = "./input%i" % ( i+1 ) os.unlink(input_file_name) From: galaxy-dev-bounces@lists.bx.psu.edu [mailto:galaxy-dev-bounces@lists.bx.psu.edu] On Behalf Of Chorny, Ilya Sent: Wednesday, September 07, 2011 9:18 AM To: galaxy-dev@lists.bx.psu.edu Subject: [galaxy-dev] cuffcompare wrapper Hi Jeremy, The symlink in the cuffcompare wrapper was causing galaxy to crash because I run as the actual user and have to chmod the job_working directory at the end so Galaxy can clean it up. Turns out is seems like the symlink is not needed. Am I missing something. See below. Your code: for i, arg in enumerate( args ): input_file_name = "./input%i" % ( i+1 ) os.symlink( arg, input_file_name ) cmd += "%s " % input_file_name My code: for i, arg in enumerate( args ): cmd += arg Ilya Chorny Ph.D. Bioinformatics Scientist I Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Work: 858.202.4582 Email: ichorny@illumina.com<mailto:ichorny@illumina.com> Website: www.illumina.com<http://www.illumina.com>
Ilya, A search of the Galaxy codebase indicates that thirteen tools use symlinks (e.g. GATK, Sicer, Picard, Cuff*, Bowtie), so the changes required to support this new code are significant. (Changes would also likely be needed for tools in the tool shed.) Also, asking tool wrappers to delete symlinks would be an idiosyncratic requirement as tools assume they have a temporary working directory at their disposal. For these reasons, it seems best to have the tool framework clean up symlinks as necessary to support the new code. Best, J. On Sep 7, 2011, at 2:28 PM, Chorny, Ilya wrote:
Ok, I figured out why you need the symlink.
Can you add an unlink after the process completes? i.e
for i, arg in enumerate( args ): input_file_name = "./input%i" % ( i+1 ) os.unlink(input_file_name)
From: galaxy-dev-bounces@lists.bx.psu.edu [mailto:galaxy-dev-bounces@lists.bx.psu.edu] On Behalf Of Chorny, Ilya Sent: Wednesday, September 07, 2011 9:18 AM To: galaxy-dev@lists.bx.psu.edu Subject: [galaxy-dev] cuffcompare wrapper
Hi Jeremy,
The symlink in the cuffcompare wrapper was causing galaxy to crash because I run as the actual user and have to chmod the job_working directory at the end so Galaxy can clean it up. Turns out is seems like the symlink is not needed. Am I missing something. See below.
Your code: for i, arg in enumerate( args ): input_file_name = "./input%i" % ( i+1 ) os.symlink( arg, input_file_name ) cmd += "%s " % input_file_name
My code: for i, arg in enumerate( args ): cmd += arg
Ilya Chorny Ph.D. Bioinformatics Scientist I Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Work: 858.202.4582 Email: ichorny@illumina.com Website: www.illumina.com
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
It's not an issue in the tmp dir but in the job_working_directory. I run most of those other tools with no problems. I don't think we should make it a requirement across the board and I think we can come up with alternative ways to clean up the job_working_directory. I am hoping that you could add the symlink to the cuffcompare wrapper as it is the only one where the symlink causes me a problem as far as I have tested. We don't want to have our code base differ to much from galaxy-central. Thanks, Ilya From: Jeremy Goecks [mailto:jeremy.goecks@emory.edu] Sent: Wednesday, September 07, 2011 6:26 PM To: Chorny, Ilya Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] cuffcompare wrapper Ilya, A search of the Galaxy codebase indicates that thirteen tools use symlinks (e.g. GATK, Sicer, Picard, Cuff*, Bowtie), so the changes required to support this new code are significant. (Changes would also likely be needed for tools in the tool shed.) Also, asking tool wrappers to delete symlinks would be an idiosyncratic requirement as tools assume they have a temporary working directory at their disposal. For these reasons, it seems best to have the tool framework clean up symlinks as necessary to support the new code. Best, J. On Sep 7, 2011, at 2:28 PM, Chorny, Ilya wrote: Ok, I figured out why you need the symlink. Can you add an unlink after the process completes? i.e for i, arg in enumerate( args ): input_file_name = "./input%i" % ( i+1 ) os.unlink(input_file_name) From: galaxy-dev-bounces@lists.bx.psu.edu<mailto:galaxy-dev-bounces@lists.bx.psu.edu> [mailto:galaxy-dev-bounces@lists.bx.psu.edu]<mailto:[mailto:galaxy-dev-bounces@lists.bx.psu.edu]> On Behalf Of Chorny, Ilya Sent: Wednesday, September 07, 2011 9:18 AM To: galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu> Subject: [galaxy-dev] cuffcompare wrapper Hi Jeremy, The symlink in the cuffcompare wrapper was causing galaxy to crash because I run as the actual user and have to chmod the job_working directory at the end so Galaxy can clean it up. Turns out is seems like the symlink is not needed. Am I missing something. See below. Your code: for i, arg in enumerate( args ): input_file_name = "./input%i" % ( i+1 ) os.symlink( arg, input_file_name ) cmd += "%s " % input_file_name My code: for i, arg in enumerate( args ): cmd += arg Ilya Chorny Ph.D. Bioinformatics Scientist I Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Work: 858.202.4582 Email: ichorny@illumina.com<mailto:ichorny@illumina.com> Website: www.illumina.com<http://www.illumina.com/> ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
I'm confused. Why would the symlink cause problems for Cuffcompare but not for other tools that use symlinks (including Cufflinks and Cuffdiff)? J. On Sep 8, 2011, at 1:43 PM, Chorny, Ilya wrote:
It’s not an issue in the tmp dir but in the job_working_directory. I run most of those other tools with no problems. I don’t think we should make it a requirement across the board and I think we can come up with alternative ways to clean up the job_working_directory. I am hoping that you could add the symlink to the cuffcompare wrapper as it is the only one where the symlink causes me a problem as far as I have tested. We don’t want to have our code base differ to much from galaxy-central.
Thanks,
Ilya
From: Jeremy Goecks [mailto:jeremy.goecks@emory.edu] Sent: Wednesday, September 07, 2011 6:26 PM To: Chorny, Ilya Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] cuffcompare wrapper
Ilya,
A search of the Galaxy codebase indicates that thirteen tools use symlinks (e.g. GATK, Sicer, Picard, Cuff*, Bowtie), so the changes required to support this new code are significant. (Changes would also likely be needed for tools in the tool shed.) Also, asking tool wrappers to delete symlinks would be an idiosyncratic requirement as tools assume they have a temporary working directory at their disposal.
For these reasons, it seems best to have the tool framework clean up symlinks as necessary to support the new code.
Best, J.
On Sep 7, 2011, at 2:28 PM, Chorny, Ilya wrote:
Ok, I figured out why you need the symlink.
Can you add an unlink after the process completes? i.e
for i, arg in enumerate( args ): input_file_name = "./input%i" % ( i+1 ) os.unlink(input_file_name)
From: galaxy-dev-bounces@lists.bx.psu.edu [mailto:galaxy-dev-bounces@lists.bx.psu.edu] On Behalf Of Chorny, Ilya Sent: Wednesday, September 07, 2011 9:18 AM To: galaxy-dev@lists.bx.psu.edu Subject: [galaxy-dev] cuffcompare wrapper
Hi Jeremy,
The symlink in the cuffcompare wrapper was causing galaxy to crash because I run as the actual user and have to chmod the job_working directory at the end so Galaxy can clean it up. Turns out is seems like the symlink is not needed. Am I missing something. See below.
Your code: for i, arg in enumerate( args ): input_file_name = "./input%i" % ( i+1 ) os.symlink( arg, input_file_name ) cmd += "%s " % input_file_name
My code: for i, arg in enumerate( args ): cmd += arg
Ilya Chorny Ph.D. Bioinformatics Scientist I Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Work: 858.202.4582 Email: ichorny@illumina.com Website: www.illumina.com
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
participants (2)
-
Chorny, Ilya -
Jeremy Goecks