Error! Sam-to-bam samtools command not found
Hi, I installed a local galaxy on Centos 6 and I followed the instructions on how to install samtools/bwa/bowtie found here http://wiki.galaxyproject.org/Admin/NGS%20Local%20Setup. I setup the correct env. for these programs correctly. Since I have no issues with the bwa and bowtie so I assumed I got everything correctly but that was not true since I keep getting the following error when I try Sam-To-Bam... ............. galaxy.jobs.runners.local DEBUG 2013-02-05 10:37:36,053 execution of external set_meta for job 62 finished galaxy.jobs DEBUG 2013-02-05 10:37:36,070 Tool did not define exit code or stdio handling; checking stderr for success galaxy.datatypes.metadata DEBUG 2013-02-05 10:37:36,101 setting metadata externally failed for HistoryDatasetAssociation 76: Error Setting BAM Metadata: /bin/sh: samtools: command not found galaxy.jobs.runners.local ERROR 2013-02-05 10:37:36,134 Job wrapper finish method failed Traceback (most recent call last): File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/runners/local.py", line 156, in run_job job_wrapper.finish( stdout, stderr, exit_code ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 412, in finish dataset.datatype.set_meta( dataset, overwrite = False ) #call datatype.set_meta directly for the initial set_meta call during dataset creation File "/usr/local/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 213, in set_meta raise Exception, "Error Setting BAM Metadata: %s" % stderr Exception: Error Setting BAM Metadata: /bin/sh: samtools: command not found galaxy.datatypes.metadata DEBUG 2013-02-05 10:37:36,239 Cleaning up external metadata files ............ * Samtools is in the path: >echo $PATH /usr/local/galaxy/galaxy_env/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/java/latest/bin:/usr/local/scripts:/usr/local/galaxy/software/samtools/0.1.16/bin/:/usr/local/galaxy/software/bwa/0.2.6/bin/:/usr/local/galaxy/software/bowtie/0.12.7/bin/:/usr/local/galaxy/software/fastx/0.0.13/bin/:/usr/local/galaxy/software/ncbi-blast/2.2.27/bin/:/usr/local/galaxy/software/lastz-distrib/1.02.00/bin/ * I setup the env.sh as recommended. >more //usr/local/galaxy/software/samtools/0.1.16/env.sh #Configure PATH to Samtools binaries PATH="/usr/local/galaxy/software/samtools/0.1.16/bin/:$PATH" export PATH * And here what I have in the bin:> ls -lh /usr/local/galaxy/software/samtools/0.1.16/bin/ total 1.5M -rwxr-xr-x 1 galaxy galaxy 2.4K Jan 29 12:53 blast2sam.pl -rwxr-xr-x 1 galaxy galaxy 2.1K Jan 29 12:53 bowtie2sam.pl -rwxr-xr-x 1 galaxy galaxy 19K Jan 29 12:53 export2sam.pl -rwxr-xr-x 1 galaxy galaxy 4.9K Jan 29 12:53 interpolate_sam.pl -rwxr-xr-x 1 galaxy galaxy 5.9K Jan 29 12:53 novo2sam.pl -rwxr-xr-x 1 galaxy galaxy 2.0K Jan 29 12:53 psl2sam.pl -rwxr-xr-x 1 galaxy galaxy 7.7K Jan 29 12:53 sam2vcf.pl -rwxrwxr-x 1 galaxy galaxy 1.4M Jan 29 12:53 samtools -rwxr-xr-x 1 galaxy galaxy 14K Jan 29 12:53 samtools.pl -rwxr-xr-x 1 galaxy galaxy 2.7K Jan 29 12:53 soap2sam.pl -rwxr-xr-x 1 galaxy galaxy 2.7K Jan 29 12:53 wgsim_eval.pl -rwxr-xr-x 1 galaxy galaxy 2.4K Jan 29 12:53 zoom2sam.pl * I could ran the samtools from anywhere in the system including from "galaxy" account. * I looked at this thread which the guy has similar problem but he was trying to set it up in a cluster and installing it on Ubunto. I read the whole thing and tried everything in there but no luck.... http://gmod.827538.n3.nabble.com/Exception-Error-Setting-BAM-Metadata-bin-sh... * I have installed R and Python/rpy and set their env. as required (I THINK) assuming this might be a cause but I do not see anything in the error log that will indicate that... Can anyone help in debugging this issue please? Thanks hakeem
Hakeem, What's going on here is that the BAM metadata setting function doesn't use the normal dependency methods and thus doesn't read 'env.sh' (which does look correct), requiring 'samtools' to be available on the path of the galaxy user. How are you executing galaxy itself? Are you positive that at launch, samtools is in the PATH? One simple way to inject this basic dependency would be to add it to the PATH in run.sh itself, as galaxy loads. -Dannon On Feb 5, 2013, at 12:17 PM, Hakeem Almabrazi <halmabrazi@idtdna.com> wrote:
Hi,
I installed a local galaxy on Centos 6 and I followed the instructions on how to install samtools/bwa/bowtie found herehttp://wiki.galaxyproject.org/Admin/NGS%20Local%20Setup.
I setup the correct env. for these programs correctly. Since I have no issues with the bwa and bowtie so I assumed I got everything correctly but that was not true since I keep getting the following error when I try Sam-To-Bam… …………. galaxy.jobs.runners.local DEBUG 2013-02-05 10:37:36,053 execution of external set_meta for job 62 finished galaxy.jobs DEBUG 2013-02-05 10:37:36,070 Tool did not define exit code or stdio handling; checking stderr for success galaxy.datatypes.metadata DEBUG 2013-02-05 10:37:36,101 setting metadata externally failed for HistoryDatasetAssociation 76: Error Setting BAM Metadata: /bin/sh: samtools: command not found galaxy.jobs.runners.local ERROR 2013-02-05 10:37:36,134 Job wrapper finish method failed Traceback (most recent call last): File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/runners/local.py", line 156, in run_job job_wrapper.finish( stdout, stderr, exit_code ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 412, in finish dataset.datatype.set_meta( dataset, overwrite = False ) #call datatype.set_meta directly for the initial set_meta call during dataset creation File "/usr/local/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 213, in set_meta raise Exception, "Error Setting BAM Metadata: %s" % stderr Exception: Error Setting BAM Metadata: /bin/sh: samtools: command not found galaxy.datatypes.metadata DEBUG 2013-02-05 10:37:36,239 Cleaning up external metadata files …………
· Samtools is in the path: >echo $PATH /usr/local/galaxy/galaxy_env/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/java/latest/bin:/usr/local/scripts:/usr/local/galaxy/software/samtools/0.1.16/bin/:/usr/local/galaxy/software/bwa/0.2.6/bin/:/usr/local/galaxy/software/bowtie/0.12.7/bin/:/usr/local/galaxy/software/fastx/0.0.13/bin/:/usr/local/galaxy/software/ncbi-blast/2.2.27/bin/:/usr/local/galaxy/software/lastz-distrib/1.02.00/bin/
· I setup the env.sh as recommended. >more //usr/local/galaxy/software/samtools/0.1.16/env.sh #Configure PATH to Samtools binaries PATH="/usr/local/galaxy/software/samtools/0.1.16/bin/:$PATH" export PATH
· And here what I have in the bin:> ls -lh /usr/local/galaxy/software/samtools/0.1.16/bin/ total 1.5M -rwxr-xr-x 1 galaxy galaxy 2.4K Jan 29 12:53 blast2sam.pl -rwxr-xr-x 1 galaxy galaxy 2.1K Jan 29 12:53 bowtie2sam.pl -rwxr-xr-x 1 galaxy galaxy 19K Jan 29 12:53 export2sam.pl -rwxr-xr-x 1 galaxy galaxy 4.9K Jan 29 12:53 interpolate_sam.pl -rwxr-xr-x 1 galaxy galaxy 5.9K Jan 29 12:53 novo2sam.pl -rwxr-xr-x 1 galaxy galaxy 2.0K Jan 29 12:53 psl2sam.pl -rwxr-xr-x 1 galaxy galaxy 7.7K Jan 29 12:53 sam2vcf.pl -rwxrwxr-x 1 galaxy galaxy 1.4M Jan 29 12:53 samtools -rwxr-xr-x 1 galaxy galaxy 14K Jan 29 12:53 samtools.pl -rwxr-xr-x 1 galaxy galaxy 2.7K Jan 29 12:53 soap2sam.pl -rwxr-xr-x 1 galaxy galaxy 2.7K Jan 29 12:53 wgsim_eval.pl -rwxr-xr-x 1 galaxy galaxy 2.4K Jan 29 12:53 zoom2sam.pl
· I could ran the samtools from anywhere in the system including from “galaxy” account.
· I looked at this thread which the guy has similar problem but he was trying to set it up in a cluster and installing it on Ubunto. I read the whole thing and tried everything in there but no luck…. http://gmod.827538.n3.nabble.com/Exception-Error-Setting-BAM-Metadata-bin-sh...
· I have installed R and Python/rpy and set their env. as required (I THINK) assuming this might be a cause but I do not see anything in the error log that will indicate that…
Can anyone help in debugging this issue please?
Thanks
hakeem
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Thank you Dannon for pointing me to the right direction. I have been using a script suggested in one of this forum to start it "service galaxy start/stop". I never thought that would be a source of issue since it is calling "run.sh" script anyway. However, I just tried "sh run.sh" and I did not see this issue at all. Here is part of the /etc/init.d/galaxy script I am using to start and stop the server ......... #!/bin/bash SERVICE_NAME=galaxy RUN_AS=galaxy RUN_IN=/usr/local/galaxy/galaxy-dist #--- main actions start() { echo "Starting $SERVICE_NAME... " cmd="cd $RUN_IN && sh run.sh --daemon" case "$(id -un)" in $RUN_AS) eval "$cmd" ;; root) su - $RUN_AS -c "$cmd" ;; *) echo "*** ERROR *** must be $RUN_AS or root in order to control this service" >&2 exit 1 esac echo "...done." } ................... So what does that mean? Is there something wrong with this script? Thank you... -----Original Message----- From: Dannon Baker [mailto:dannonbaker@me.com] Sent: Tuesday, February 05, 2013 11:37 AM To: Hakeem Almabrazi Cc: Galaxy Dev Subject: Re: [galaxy-dev] Error! Sam-to-bam samtools command not found Hakeem, What's going on here is that the BAM metadata setting function doesn't use the normal dependency methods and thus doesn't read 'env.sh' (which does look correct), requiring 'samtools' to be available on the path of the galaxy user. How are you executing galaxy itself? Are you positive that at launch, samtools is in the PATH? One simple way to inject this basic dependency would be to add it to the PATH in run.sh itself, as galaxy loads. -Dannon On Feb 5, 2013, at 12:17 PM, Hakeem Almabrazi <halmabrazi@idtdna.com> wrote:
Hi,
I installed a local galaxy on Centos 6 and I followed the instructions on how to install samtools/bwa/bowtie found herehttp://wiki.galaxyproject.org/Admin/NGS%20Local%20Setup.
I setup the correct env. for these programs correctly. Since I have no issues with the bwa and bowtie so I assumed I got everything correctly but that was not true since I keep getting the following error when I try Sam-To-Bam... ............. galaxy.jobs.runners.local DEBUG 2013-02-05 10:37:36,053 execution of external set_meta for job 62 finished galaxy.jobs DEBUG 2013-02-05 10:37:36,070 Tool did not define exit code or stdio handling; checking stderr for success galaxy.datatypes.metadata DEBUG 2013-02-05 10:37:36,101 setting metadata externally failed for HistoryDatasetAssociation 76: Error Setting BAM Metadata: /bin/sh: samtools: command not found galaxy.jobs.runners.local ERROR 2013-02-05 10:37:36,134 Job wrapper finish method failed Traceback (most recent call last): File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/runners/local.py", line 156, in run_job job_wrapper.finish( stdout, stderr, exit_code ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 412, in finish dataset.datatype.set_meta( dataset, overwrite = False ) #call datatype.set_meta directly for the initial set_meta call during dataset creation File "/usr/local/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 213, in set_meta raise Exception, "Error Setting BAM Metadata: %s" % stderr Exception: Error Setting BAM Metadata: /bin/sh: samtools: command not found galaxy.datatypes.metadata DEBUG 2013-02-05 10:37:36,239 Cleaning up external metadata files ............
* Samtools is in the path: >echo $PATH /usr/local/galaxy/galaxy_env/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/java/latest/bin:/usr/local/scripts:/usr/local/galaxy/software/samtools/0.1.16/bin/:/usr/local/galaxy/software/bwa/0.2.6/bin/:/usr/local/galaxy/software/bowtie/0.12.7/bin/:/usr/local/galaxy/software/fastx/0.0.13/bin/:/usr/local/galaxy/software/ncbi-blast/2.2.27/bin/:/usr/local/galaxy/software/lastz-distrib/1.02.00/bin/
* I setup the env.sh as recommended. >more //usr/local/galaxy/software/samtools/0.1.16/env.sh #Configure PATH to Samtools binaries PATH="/usr/local/galaxy/software/samtools/0.1.16/bin/:$PATH" export PATH
* And here what I have in the bin:> ls -lh /usr/local/galaxy/software/samtools/0.1.16/bin/ total 1.5M -rwxr-xr-x 1 galaxy galaxy 2.4K Jan 29 12:53 blast2sam.pl -rwxr-xr-x 1 galaxy galaxy 2.1K Jan 29 12:53 bowtie2sam.pl -rwxr-xr-x 1 galaxy galaxy 19K Jan 29 12:53 export2sam.pl -rwxr-xr-x 1 galaxy galaxy 4.9K Jan 29 12:53 interpolate_sam.pl -rwxr-xr-x 1 galaxy galaxy 5.9K Jan 29 12:53 novo2sam.pl -rwxr-xr-x 1 galaxy galaxy 2.0K Jan 29 12:53 psl2sam.pl -rwxr-xr-x 1 galaxy galaxy 7.7K Jan 29 12:53 sam2vcf.pl -rwxrwxr-x 1 galaxy galaxy 1.4M Jan 29 12:53 samtools -rwxr-xr-x 1 galaxy galaxy 14K Jan 29 12:53 samtools.pl -rwxr-xr-x 1 galaxy galaxy 2.7K Jan 29 12:53 soap2sam.pl -rwxr-xr-x 1 galaxy galaxy 2.7K Jan 29 12:53 wgsim_eval.pl -rwxr-xr-x 1 galaxy galaxy 2.4K Jan 29 12:53 zoom2sam.pl
* I could ran the samtools from anywhere in the system including from "galaxy" account.
* I looked at this thread which the guy has similar problem but he was trying to set it up in a cluster and installing it on Ubunto. I read the whole thing and tried everything in there but no luck.... http://gmod.827538.n3.nabble.com/Exception-Error-Setting-BAM-Metadata-bin-sh...
* I have installed R and Python/rpy and set their env. as required (I THINK) assuming this might be a cause but I do not see anything in the error log that will indicate that...
Can anyone help in debugging this issue please?
Thanks
hakeem
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
Probably nothing wrong with the script, but my hunch is that the original $PATH setting you did for the galaxy user doesn't persist to this script's environment because it uses a non-login shell. I'm definitely not a CentOS expert, but there's probably another file used for settings like this in non-login shells. You could also just set those variables in that /etc/init.d/galaxy script, I'm betting. -Dannon On Feb 5, 2013, at 12:55 PM, Hakeem Almabrazi <halmabrazi@idtdna.com> wrote:
Thank you Dannon for pointing me to the right direction. I have been using a script suggested in one of this forum to start it "service galaxy start/stop". I never thought that would be a source of issue since it is calling "run.sh" script anyway. However, I just tried "sh run.sh" and I did not see this issue at all.
Here is part of the /etc/init.d/galaxy script I am using to start and stop the server
......... #!/bin/bash
SERVICE_NAME=galaxy RUN_AS=galaxy RUN_IN=/usr/local/galaxy/galaxy-dist
#--- main actions
start() { echo "Starting $SERVICE_NAME... " cmd="cd $RUN_IN && sh run.sh --daemon" case "$(id -un)" in $RUN_AS) eval "$cmd" ;; root) su - $RUN_AS -c "$cmd" ;; *) echo "*** ERROR *** must be $RUN_AS or root in order to control this service" >&2 exit 1 esac echo "...done." }
...................
So what does that mean? Is there something wrong with this script?
Thank you...
-----Original Message----- From: Dannon Baker [mailto:dannonbaker@me.com] Sent: Tuesday, February 05, 2013 11:37 AM To: Hakeem Almabrazi Cc: Galaxy Dev Subject: Re: [galaxy-dev] Error! Sam-to-bam samtools command not found
Hakeem,
What's going on here is that the BAM metadata setting function doesn't use the normal dependency methods and thus doesn't read 'env.sh' (which does look correct), requiring 'samtools' to be available on the path of the galaxy user. How are you executing galaxy itself? Are you positive that at launch, samtools is in the PATH? One simple way to inject this basic dependency would be to add it to the PATH in run.sh itself, as galaxy loads.
-Dannon
On Feb 5, 2013, at 12:17 PM, Hakeem Almabrazi <halmabrazi@idtdna.com> wrote:
Hi,
I installed a local galaxy on Centos 6 and I followed the instructions on how to install samtools/bwa/bowtie found herehttp://wiki.galaxyproject.org/Admin/NGS%20Local%20Setup.
I setup the correct env. for these programs correctly. Since I have no issues with the bwa and bowtie so I assumed I got everything correctly but that was not true since I keep getting the following error when I try Sam-To-Bam... ............. galaxy.jobs.runners.local DEBUG 2013-02-05 10:37:36,053 execution of external set_meta for job 62 finished galaxy.jobs DEBUG 2013-02-05 10:37:36,070 Tool did not define exit code or stdio handling; checking stderr for success galaxy.datatypes.metadata DEBUG 2013-02-05 10:37:36,101 setting metadata externally failed for HistoryDatasetAssociation 76: Error Setting BAM Metadata: /bin/sh: samtools: command not found galaxy.jobs.runners.local ERROR 2013-02-05 10:37:36,134 Job wrapper finish method failed Traceback (most recent call last): File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/runners/local.py", line 156, in run_job job_wrapper.finish( stdout, stderr, exit_code ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 412, in finish dataset.datatype.set_meta( dataset, overwrite = False ) #call datatype.set_meta directly for the initial set_meta call during dataset creation File "/usr/local/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 213, in set_meta raise Exception, "Error Setting BAM Metadata: %s" % stderr Exception: Error Setting BAM Metadata: /bin/sh: samtools: command not found galaxy.datatypes.metadata DEBUG 2013-02-05 10:37:36,239 Cleaning up external metadata files ............
* Samtools is in the path: >echo $PATH /usr/local/galaxy/galaxy_env/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/java/latest/bin:/usr/local/scripts:/usr/local/galaxy/software/samtools/0.1.16/bin/:/usr/local/galaxy/software/bwa/0.2.6/bin/:/usr/local/galaxy/software/bowtie/0.12.7/bin/:/usr/local/galaxy/software/fastx/0.0.13/bin/:/usr/local/galaxy/software/ncbi-blast/2.2.27/bin/:/usr/local/galaxy/software/lastz-distrib/1.02.00/bin/
* I setup the env.sh as recommended. >more //usr/local/galaxy/software/samtools/0.1.16/env.sh #Configure PATH to Samtools binaries PATH="/usr/local/galaxy/software/samtools/0.1.16/bin/:$PATH" export PATH
* And here what I have in the bin:> ls -lh /usr/local/galaxy/software/samtools/0.1.16/bin/ total 1.5M -rwxr-xr-x 1 galaxy galaxy 2.4K Jan 29 12:53 blast2sam.pl -rwxr-xr-x 1 galaxy galaxy 2.1K Jan 29 12:53 bowtie2sam.pl -rwxr-xr-x 1 galaxy galaxy 19K Jan 29 12:53 export2sam.pl -rwxr-xr-x 1 galaxy galaxy 4.9K Jan 29 12:53 interpolate_sam.pl -rwxr-xr-x 1 galaxy galaxy 5.9K Jan 29 12:53 novo2sam.pl -rwxr-xr-x 1 galaxy galaxy 2.0K Jan 29 12:53 psl2sam.pl -rwxr-xr-x 1 galaxy galaxy 7.7K Jan 29 12:53 sam2vcf.pl -rwxrwxr-x 1 galaxy galaxy 1.4M Jan 29 12:53 samtools -rwxr-xr-x 1 galaxy galaxy 14K Jan 29 12:53 samtools.pl -rwxr-xr-x 1 galaxy galaxy 2.7K Jan 29 12:53 soap2sam.pl -rwxr-xr-x 1 galaxy galaxy 2.7K Jan 29 12:53 wgsim_eval.pl -rwxr-xr-x 1 galaxy galaxy 2.4K Jan 29 12:53 zoom2sam.pl
* I could ran the samtools from anywhere in the system including from "galaxy" account.
* I looked at this thread which the guy has similar problem but he was trying to set it up in a cluster and installing it on Ubunto. I read the whole thing and tried everything in there but no luck.... http://gmod.827538.n3.nabble.com/Exception-Error-Setting-BAM-Metadata-bin-sh...
* I have installed R and Python/rpy and set their env. as required (I THINK) assuming this might be a cause but I do not see anything in the error log that will indicate that...
Can anyone help in debugging this issue please?
Thanks
hakeem
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
participants (2)
-
Dannon Baker -
Hakeem Almabrazi