samtools version detection issue
Hi all. I'm sorry if this has been answered before, but I've searched and cannot find a solution other than "add SAMtools to your PATH", which I already have done. I can invoke samtools from BASH while logged in as the "galaxy" user I set up according to the production installation guide. When I try to add .bam files to a shared data library, I get the following message: Traceback (most recent call last):
File "/home/galaxy/galaxy-dist/tools/data_source/upload.py", line 394, in __main__() File "/home/galaxy/galaxy-dist/tools/data_source/upload.py", line 386, in __main__ add_file( dataset, registry, json_file, output_path ) File "/home/galaxy/galaxy-dist/tools/data_source/upload.py", line 327, in add_file if link_data_only == 'copy_files' and datatype.dataset_content_needs_grooming( output_path ): File "/home/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 79, in dataset_content_needs_grooming version = self._get_samtools_version() File "/home/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 63, in _get_samtools_version output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE, stdout=subprocess.PIPE ).communicate()[1] File "/usr/lib64/python2.6/subprocess.py", line 639, in __init__ errread, errwrite) File "/usr/lib64/python2.6/subprocess.py", line 1220, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory
I've taken a peek at the "dataset_content_needs_grooming" function and it looks like there is an OS call that executes "$ samtools" and splits the resulting string to access the version number of samtools (why couldn't the developers add a --version flag?). It seems to me like samtools cannot be executed, but I can't think of any reason why. Invoking samtools as user "galaxy" (with the user's associated PATH) gives: Program: samtools (Tools for alignments in the SAM format)
Version: 0.1.12a (r862) Usage: samtools <command> [options] Command: view SAM<->BAM conversion sort sort alignment file pileup generate pileup output mpileup multi-way pileup faidx index/extract FASTA tview text alignment viewer index index alignment idxstats BAM index stats (r595 or later) fixmate fix mate information glfview print GLFv3 file flagstat simple stats calmd recalculate MD/NM tags and '=' bases merge merge sorted alignments rmdup remove PCR duplicates reheader replace BAM header
Does anyone know where I've gone wrong? -- Matt Shirley Ph.D Candidate - BCMB Pevsner Lab <http://pevsnerlab.kennedykrieger.org/> Johns Hopkins Medicine
On Jan 18, 2012, at 5:28 PM, Matt Shirley wrote:
Hi all. I'm sorry if this has been answered before, but I've searched and cannot find a solution other than "add SAMtools to your PATH", which I already have done. I can invoke samtools from BASH while logged in as the "galaxy" user I set up according to the production installation guide. When I try to add .bam files to a shared data library, I get the following message:
Traceback (most recent call last): File "/home/galaxy/galaxy-dist/tools/data_source/upload.py", line 394, in __main__() File "/home/galaxy/galaxy-dist/tools/data_source/upload.py", line 386, in __main__ add_file( dataset, registry, json_file, output_path ) File "/home/galaxy/galaxy-dist/tools/data_source/upload.py", line 327, in add_file if link_data_only == 'copy_files' and datatype.dataset_content_needs_grooming( output_path ): File "/home/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 79, in dataset_content_needs_grooming version = self._get_samtools_version() File "/home/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 63, in _get_samtools_version output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE, stdout=subprocess.PIPE ).communicate()[1] File "/usr/lib64/python2.6/subprocess.py", line 639, in __init__ errread, errwrite) File "/usr/lib64/python2.6/subprocess.py", line 1220, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory
I've taken a peek at the "dataset_content_needs_grooming" function and it looks like there is an OS call that executes "$ samtools" and splits the resulting string to access the version number of samtools (why couldn't the developers add a --version flag?). It seems to me like samtools cannot be executed, but I can't think of any reason why. Invoking samtools as user "galaxy" (with the user's associated PATH) gives:
Program: samtools (Tools for alignments in the SAM format) Version: 0.1.12a (r862) Usage: samtools <command> [options] Command: view SAM<->BAM conversion sort sort alignment file pileup generate pileup output mpileup multi-way pileup faidx index/extract FASTA tview text alignment viewer index index alignment idxstats BAM index stats (r595 or later) fixmate fix mate information glfview print GLFv3 file flagstat simple stats calmd recalculate MD/NM tags and '=' bases merge merge sorted alignments rmdup remove PCR duplicates reheader replace BAM header
Does anyone know where I've gone wrong?
Hi Matt, Are you running tools on a cluster? If running locally, are you starting Galaxy by hand (i.e. run.sh) from that same shell in which you can run samtools? --nate
-- Matt Shirley Ph.D Candidate - BCMB Pevsner Lab Johns Hopkins Medicine
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Hi, I'm not sure if this is related or not but I'm also noticed a similar problem where the program is not found in the path, but it exists. I have it submitted as an issue , but it could be related here so mentioned here. The problem started when I recently upgraded my Galaxy installation(It was running in cluster with SGE, in Red hat linux). Currently, I'm running Galaxy with drmaa in a cluster in Red Hat linux, where the web server is running in one Server, and the execution is being performed in another. I noticed that programs like ClustalW, dialign-tx etc, which uses a wrapper, is giving error as follows: sh: dialign-tx: command not found The original command line in the wrapper is as follows: command = "dialign-tx %s/dialign-tx %s %s -g%s -n%s -D %s %s" % (GALAXY_DATA_INDEX_DIR, options, sensitivity, max_frag_length, length_low_score_pos, input_file_name, output_file) Even though, the programs are present in the Execution node's path, and if I add, explicit path, like "/usr/local/bin" in the command as bellow: command = "/usr/local/bin/dialign-tx %s/dialign-tx %s %s -g%s -n%s -D %s %s" % (GALAXY_DATA_INDEX_DIR, options, sensitivity, max_frag_length, length_low_score_pos, input_file_name, output_file) Then the programs works fine without any error. I compared, the wrapper files with my old installation and the current installation, and both are 100% same. So, I guess the path to the executable is missing from the environment. I searched a lot, but couldn't find any solution to rectify this issue. Can someone please point out where I need to change to avoid adding explicit path to each programs. Thanks in advance. With Regards, ---------------- Ambarish Biswas, University of Otago Department of Biochemistry, Dunedin, New Zealand, On Fri, Jan 20, 2012 at 11:07 AM, Nate Coraor <nate@bx.psu.edu> wrote:
On Jan 18, 2012, at 5:28 PM, Matt Shirley wrote:
Hi all. I'm sorry if this has been answered before, but I've searched and cannot find a solution other than "add SAMtools to your PATH", which I already have done. I can invoke samtools from BASH while logged in as the "galaxy" user I set up according to the production installation guide. When I try to add .bam files to a shared data library, I get the following message:
Traceback (most recent call last): File "/home/galaxy/galaxy-dist/tools/data_source/upload.py", line 394, in __main__() File "/home/galaxy/galaxy-dist/tools/data_source/upload.py", line 386, in __main__ add_file( dataset, registry, json_file, output_path ) File "/home/galaxy/galaxy-dist/tools/data_source/upload.py", line 327, in add_file if link_data_only == 'copy_files' and datatype.dataset_content_needs_grooming( output_path ): File "/home/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 79, in dataset_content_needs_grooming version = self._get_samtools_version() File "/home/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 63, in _get_samtools_version output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE, stdout=subprocess.PIPE ).communicate()[1] File "/usr/lib64/python2.6/subprocess.py", line 639, in __init__ errread, errwrite) File "/usr/lib64/python2.6/subprocess.py", line 1220, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory
I've taken a peek at the "dataset_content_needs_grooming" function and it looks like there is an OS call that executes "$ samtools" and splits the resulting string to access the version number of samtools (why couldn't the developers add a --version flag?). It seems to me like samtools cannot be executed, but I can't think of any reason why. Invoking samtools as user "galaxy" (with the user's associated PATH) gives:
Program: samtools (Tools for alignments in the SAM format) Version: 0.1.12a (r862) Usage: samtools <command> [options] Command: view SAM<->BAM conversion sort sort alignment file pileup generate pileup output mpileup multi-way pileup faidx index/extract FASTA tview text alignment viewer index index alignment idxstats BAM index stats (r595 or later) fixmate fix mate information glfview print GLFv3 file flagstat simple stats calmd recalculate MD/NM tags and '=' bases merge merge sorted alignments rmdup remove PCR duplicates reheader replace BAM header
Does anyone know where I've gone wrong?
Hi Matt,
Are you running tools on a cluster? If running locally, are you starting Galaxy by hand (i.e. run.sh) from that same shell in which you can run samtools?
--nate
-- Matt Shirley Ph.D Candidate - BCMB Pevsner Lab Johns Hopkins Medicine
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
On Jan 20, 2012, at 12:12 AM, ambarish biswas wrote:
Hi, I'm not sure if this is related or not but I'm also noticed a similar problem where the program is not found in the path, but it exists. I have it submitted as an issue , but it could be related here so mentioned here. The problem started when I recently upgraded my Galaxy installation(It was running in cluster with SGE, in Red hat linux). Currently, I'm running Galaxy with drmaa in a cluster in Red Hat linux, where the web server is running in one Server, and the execution is being performed in another. I noticed that programs like ClustalW, dialign-tx etc, which uses a wrapper, is giving error as follows: sh: dialign-tx: command not found The original command line in the wrapper is as follows: command = "dialign-tx %s/dialign-tx %s %s -g%s -n%s -D %s %s" % (GALAXY_DATA_INDEX_DIR, options, sensitivity, max_frag_length, length_low_score_pos, input_file_name, output_file)
Even though, the programs are present in the Execution node's path, and if I add, explicit path, like "/usr/local/bin" in the command as bellow: command = "/usr/local/bin/dialign-tx %s/dialign-tx %s %s -g%s -n%s -D %s %s" % (GALAXY_DATA_INDEX_DIR, options, sensitivity, max_frag_length, length_low_score_pos, input_file_name, output_file)
Then the programs works fine without any error.
I compared, the wrapper files with my old installation and the current installation, and both are 100% same. So, I guess the path to the executable is missing from the environment. I searched a lot, but couldn't find any solution to rectify this issue. Can someone please point out where I need to change to avoid adding explicit path to each programs. Thanks in advance.
I replied to your bitbucket ticket first, just in case you don't see it, here it is: https://bitbucket.org/galaxy/galaxy-central/issue/702/program-command-not-fo... --nate
With Regards, ---------------- Ambarish Biswas, University of Otago Department of Biochemistry, Dunedin, New Zealand,
On Fri, Jan 20, 2012 at 11:07 AM, Nate Coraor <nate@bx.psu.edu> wrote:
On Jan 18, 2012, at 5:28 PM, Matt Shirley wrote:
Hi all. I'm sorry if this has been answered before, but I've searched and cannot find a solution other than "add SAMtools to your PATH", which I already have done. I can invoke samtools from BASH while logged in as the "galaxy" user I set up according to the production installation guide. When I try to add .bam files to a shared data library, I get the following message:
Traceback (most recent call last): File "/home/galaxy/galaxy-dist/tools/data_source/upload.py", line 394, in __main__() File "/home/galaxy/galaxy-dist/tools/data_source/upload.py", line 386, in __main__ add_file( dataset, registry, json_file, output_path ) File "/home/galaxy/galaxy-dist/tools/data_source/upload.py", line 327, in add_file if link_data_only == 'copy_files' and datatype.dataset_content_needs_grooming( output_path ): File "/home/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 79, in dataset_content_needs_grooming version = self._get_samtools_version() File "/home/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 63, in _get_samtools_version output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE, stdout=subprocess.PIPE ).communicate()[1] File "/usr/lib64/python2.6/subprocess.py", line 639, in __init__ errread, errwrite) File "/usr/lib64/python2.6/subprocess.py", line 1220, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory
I've taken a peek at the "dataset_content_needs_grooming" function and it looks like there is an OS call that executes "$ samtools" and splits the resulting string to access the version number of samtools (why couldn't the developers add a --version flag?). It seems to me like samtools cannot be executed, but I can't think of any reason why. Invoking samtools as user "galaxy" (with the user's associated PATH) gives:
Program: samtools (Tools for alignments in the SAM format) Version: 0.1.12a (r862) Usage: samtools <command> [options] Command: view SAM<->BAM conversion sort sort alignment file pileup generate pileup output mpileup multi-way pileup faidx index/extract FASTA tview text alignment viewer index index alignment idxstats BAM index stats (r595 or later) fixmate fix mate information glfview print GLFv3 file flagstat simple stats calmd recalculate MD/NM tags and '=' bases merge merge sorted alignments rmdup remove PCR duplicates reheader replace BAM header
Does anyone know where I've gone wrong?
Hi Matt,
Are you running tools on a cluster? If running locally, are you starting Galaxy by hand (i.e. run.sh) from that same shell in which you can run samtools?
--nate
-- Matt Shirley Ph.D Candidate - BCMB Pevsner Lab Johns Hopkins Medicine
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
participants (3)
-
ambarish biswas -
Matt Shirley -
Nate Coraor