Re: [galaxy-dev] Error while installing DESeq-hts
I know this is a very old entry to the mailing list, but I thought this might be useful for some people. Recently, I set up DESeq-hts and its siblings on our galaxy instance. To succeed, I performed the steps below. @vipin: would you consider revising the tool_dependencies.xml for these tools to take these steps into account? If you're interested, I might compose a preliminary version. I believe that most, if not all of these settings can be automated using the tool_dependencies files. I did not manage to get DEXSeq working, as it throws errors on the 'estimateDispersions' step in R. I'm not sure how to solve this, as it gives me the same errors using the demonstration data from the pasilla library, following the manual steps. (R 3.0.2, bioconductor 2.13, DEXSeq 1.8). Any hints on that are welcome. Best, Geert ## GALAXY PACKAGES IN TOOLSHED ####################### - install the DESeq-hts tools from the main toolshed (cec4b4fb30be from vipints) - install R into galaxy (I used package_R_3_0_2 from iuc (50f7e1e71271)) - install package_scipi and package_numpi (from iuc) - install package_samtools-0.1.18 (I used devteam/package_samtools_0_1_18/c0f72bdba484) - install htseq_count package (I used d5edaf8dc974 from lparsons) ## INSTALL DESeq bioconductor package ######################### - open R from the commandline (using the R version installed earlier !), install bioconductor package 'DESeq' : - source("http://bioconductor.org/biocLite.R") - biocLite('DESeq') - biocLite('DESeq2') - biocLite('DEXSeq') (needed extra ubuntu package libcurl4-gnutls-dev for this) ## complie samtools from source (temporary) ############################# - download samtools 0.1.18 source code into a temporary location and unpack - compile with : "make CXXFLAGS=-fPIC CFLAGS=-fPIC CPPFLAGS=-fPIC" ## install octave ########### - OS-dependent, ubuntu: apt-get install octave ## set up DESeq-hts-1.0 ############### - go to /<path_to_shed_tools>/toolshed.g2.bx.psu.edu/repos/vipints/deseq_hts/cec4b4fb30be/deseq_hts/deseq-hts_1.0 - run : "setup_deseq-hts.sh" and provide paths as requested. - NOTE: set samtools path to the source compiled version for now (shed version fails to find sam.h)!! - NOTE: also add NumPy path to the SCIPY_PATH variable - rest: example (available in bin/deseq_config.sh): export LD_LIBRARY_PATH= export ENVIRONMENT=galaxy export DESEQ_VERSION=1.12.1 export DESEQ_PATH=/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/vipints/deseq_hts/cec4b4fb30be/deseq_hts/deseq-hts_1.0 export DESEQ_SRC_PATH=/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/vipints/deseq_hts/cec4b4fb30be/deseq_hts/deseq-hts_1.0/src export DESEQ_BIN_PATH=/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/vipints/deseq_hts/cec4b4fb30be/deseq_hts/deseq-hts_1.0/bin export INTERPRETER=octave export MATLAB_BIN_PATH= export MATLAB_MEX_PATH= export MATLAB_INCLUDE_DIR= export OCTAVE_BIN_PATH=/opt/software/octave-3.8.0/bin/octave export OCTAVE_MKOCT=/opt/software/octave-3.8.0/bin/mkoctfile export SAMTOOLS_DIR=/galaxy/packages_sources/samtools-0.1.18_fPIC #export SAMTOOLS_DIR=/galaxy/galaxy_tool_binaries/samtools/0.1.18/devteam/package_samtools_0_1_18/c0f72bdba484/bin/ export PYTHON_PATH=/galaxy/galaxy_env/bin/python export SCIPY_PATH=/galaxy/galaxy_tool_binaries/scipy/0.12.0/iuc/package_scipy_0_12/984d208b0808/lib/python/:/galaxy/galaxy_tool_binaries/numpy/1.7.1/iuc/package_numpy_1_7/ef12a3a11d5b/lib/python/ ## USE THE TOOLSHED VERSION, The one you installed bioclite packages with. export R_PATH=/galaxy/galaxy_tool_binaries/R_3_0_2/3.0.2/iuc/package_r_3_0_2/50f7e1e71271/R - enter mex directory, run "make octave" - swap SAMTOOLS_DIR in bin/deseq_config.sh to point to the toolshed version of samtools 0.1.18 ## RUN DESeq-hts-1.0 TEST ################## enter src directory, run test from the toolconf xml file: "./deseq-hts.sh ../test_data/deseq_c_elegans_WS200-I-regions.gff3 ../test_data/deseq_c_elegans_WS200-I-regions_deseq.txt ../test_data/genes.mat ../test_data/deseq_c_elegans_WS200-I-regions-SRX001872.bam ../test_data/deseq_c_elegans_WS200-I-regions-SRX001875.bam" ## set up DESeq-hts-2.0 ############### - go to /<path_to_shed_tools>/toolshed.g2.bx.psu.edu/repos/vipints/deseq_hts/cec4b4fb30be/deseq_hts/deseq-hts_2.0 - similar to 1.0: setup config file, - point to src compiled version of samtools - add scipy and numpy to scipi_path variable - use the toolshed version of R - in mex, run "make octave" - swap samtools path IMPORTANT: ---------------- - open "src/deseq2-hts.sh" in a text editor and add an extra line after line 38 (echo load the genome annotation....): export PYTHONPATH=$PYTHONPATH:${SCIPY_PATH} - in the same file (src/deseq2-hts.sh), append line 70 (../bin/get_read_counts...) with 2>&1 . This prevents some octave warnings from crashing the job (missing docstring warnings) Test-run in galaxy seems to work. ## set up DEXSeq-hts-1.0 ############### go to /<path_to_shed_tools>/toolshed.g2.bx.psu.edu/repos/vipints/deseq_hts/cec4b4fb30be/deseq_hts/dexseq_1.0 run setup_dexseq-hts.sh - set samtools path - set python_path : I use the galaxy python version (/galaxy/galaxy_env/bin/python), see development docs - set pythonpath: Make sure to include both HTSeq AND numpy here ! : export PYTHONPATH=/galaxy/galaxy_tool_binaries/htseq/0.5.4p5/lparsons/htseq_count/3ffe4e2572a7/lib/python/HTSeq-0.5.4p5-py2.6-linux-x86_64.egg:/galaxy/galaxy_tool_binaries/numpy/1.7.1/iuc/package_numpy_1_7/ef12a3a11d5b/lib/python:$PYTHONPATH - set R_path : I use toolshed version package_r_3_0_2 from iuc (with the installed bioclite packages from above) => FAILS : crashes on estimateDispersions / fitDispersionFunction step in run_DEXseq.R -- Geert Vandeweyer, Ph.D. Department of Medical Genetics University of Antwerp Prins Boudewijnlaan 43 2650 Edegem Belgium Tel: +32 (0)3 275 97 56 E-mail:geert.vandeweyer@ua.ac.be http://ua.ac.be/cognitivegenetics http://www.linkedin.com/in/geertvandeweyer
Hi Geert and Vipin, I might also consider the DESeq package and the wrappers from galaxytools if you plan to update your wrappers. https://github.com/bgruening/galaxytools/tree/master/orphan_tool_dependencie... @Geert: impressive documentation! Thanks a lot for sharing! Bjoern Am 29.07.2014 um 15:54 schrieb Geert Vandeweyer:
I know this is a very old entry to the mailing list, but I thought this might be useful for some people.
Recently, I set up DESeq-hts and its siblings on our galaxy instance. To succeed, I performed the steps below.
@vipin: would you consider revising the tool_dependencies.xml for these tools to take these steps into account? If you're interested, I might compose a preliminary version. I believe that most, if not all of these settings can be automated using the tool_dependencies files.
I did not manage to get DEXSeq working, as it throws errors on the 'estimateDispersions' step in R. I'm not sure how to solve this, as it gives me the same errors using the demonstration data from the pasilla library, following the manual steps. (R 3.0.2, bioconductor 2.13, DEXSeq 1.8). Any hints on that are welcome.
Best,
Geert
## GALAXY PACKAGES IN TOOLSHED ####################### - install the DESeq-hts tools from the main toolshed (cec4b4fb30be from vipints) - install R into galaxy (I used package_R_3_0_2 from iuc (50f7e1e71271)) - install package_scipi and package_numpi (from iuc) - install package_samtools-0.1.18 (I used devteam/package_samtools_0_1_18/c0f72bdba484) - install htseq_count package (I used d5edaf8dc974 from lparsons)
## INSTALL DESeq bioconductor package ######################### - open R from the commandline (using the R version installed earlier !), install bioconductor package 'DESeq' : - source("http://bioconductor.org/biocLite.R") - biocLite('DESeq') - biocLite('DESeq2') - biocLite('DEXSeq') (needed extra ubuntu package libcurl4-gnutls-dev for this)
## complie samtools from source (temporary) ############################# - download samtools 0.1.18 source code into a temporary location and unpack - compile with : "make CXXFLAGS=-fPIC CFLAGS=-fPIC CPPFLAGS=-fPIC"
## install octave ########### - OS-dependent, ubuntu: apt-get install octave
## set up DESeq-hts-1.0 ############### - go to /<path_to_shed_tools>/toolshed.g2.bx.psu.edu/repos/vipints/deseq_hts/cec4b4fb30be/deseq_hts/deseq-hts_1.0
- run : "setup_deseq-hts.sh" and provide paths as requested. - NOTE: set samtools path to the source compiled version for now (shed version fails to find sam.h)!! - NOTE: also add NumPy path to the SCIPY_PATH variable - rest: example (available in bin/deseq_config.sh): export LD_LIBRARY_PATH= export ENVIRONMENT=galaxy export DESEQ_VERSION=1.12.1 export DESEQ_PATH=/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/vipints/deseq_hts/cec4b4fb30be/deseq_hts/deseq-hts_1.0
export DESEQ_SRC_PATH=/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/vipints/deseq_hts/cec4b4fb30be/deseq_hts/deseq-hts_1.0/src
export DESEQ_BIN_PATH=/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/vipints/deseq_hts/cec4b4fb30be/deseq_hts/deseq-hts_1.0/bin
export INTERPRETER=octave export MATLAB_BIN_PATH= export MATLAB_MEX_PATH= export MATLAB_INCLUDE_DIR= export OCTAVE_BIN_PATH=/opt/software/octave-3.8.0/bin/octave export OCTAVE_MKOCT=/opt/software/octave-3.8.0/bin/mkoctfile export SAMTOOLS_DIR=/galaxy/packages_sources/samtools-0.1.18_fPIC #export SAMTOOLS_DIR=/galaxy/galaxy_tool_binaries/samtools/0.1.18/devteam/package_samtools_0_1_18/c0f72bdba484/bin/
export PYTHON_PATH=/galaxy/galaxy_env/bin/python export SCIPY_PATH=/galaxy/galaxy_tool_binaries/scipy/0.12.0/iuc/package_scipy_0_12/984d208b0808/lib/python/:/galaxy/galaxy_tool_binaries/numpy/1.7.1/iuc/package_numpy_1_7/ef12a3a11d5b/lib/python/
## USE THE TOOLSHED VERSION, The one you installed bioclite packages with. export R_PATH=/galaxy/galaxy_tool_binaries/R_3_0_2/3.0.2/iuc/package_r_3_0_2/50f7e1e71271/R
- enter mex directory, run "make octave" - swap SAMTOOLS_DIR in bin/deseq_config.sh to point to the toolshed version of samtools 0.1.18
## RUN DESeq-hts-1.0 TEST ################## enter src directory, run test from the toolconf xml file: "./deseq-hts.sh ../test_data/deseq_c_elegans_WS200-I-regions.gff3 ../test_data/deseq_c_elegans_WS200-I-regions_deseq.txt ../test_data/genes.mat ../test_data/deseq_c_elegans_WS200-I-regions-SRX001872.bam ../test_data/deseq_c_elegans_WS200-I-regions-SRX001875.bam"
## set up DESeq-hts-2.0 ############### - go to /<path_to_shed_tools>/toolshed.g2.bx.psu.edu/repos/vipints/deseq_hts/cec4b4fb30be/deseq_hts/deseq-hts_2.0
- similar to 1.0: setup config file, - point to src compiled version of samtools - add scipy and numpy to scipi_path variable - use the toolshed version of R - in mex, run "make octave" - swap samtools path
IMPORTANT: ---------------- - open "src/deseq2-hts.sh" in a text editor and add an extra line after line 38 (echo load the genome annotation....): export PYTHONPATH=$PYTHONPATH:${SCIPY_PATH} - in the same file (src/deseq2-hts.sh), append line 70 (../bin/get_read_counts...) with 2>&1 . This prevents some octave warnings from crashing the job (missing docstring warnings)
Test-run in galaxy seems to work.
## set up DEXSeq-hts-1.0 ############### go to /<path_to_shed_tools>/toolshed.g2.bx.psu.edu/repos/vipints/deseq_hts/cec4b4fb30be/deseq_hts/dexseq_1.0
run setup_dexseq-hts.sh - set samtools path - set python_path : I use the galaxy python version (/galaxy/galaxy_env/bin/python), see development docs - set pythonpath: Make sure to include both HTSeq AND numpy here ! : export PYTHONPATH=/galaxy/galaxy_tool_binaries/htseq/0.5.4p5/lparsons/htseq_count/3ffe4e2572a7/lib/python/HTSeq-0.5.4p5-py2.6-linux-x86_64.egg:/galaxy/galaxy_tool_binaries/numpy/1.7.1/iuc/package_numpy_1_7/ef12a3a11d5b/lib/python:$PYTHONPATH
- set R_path : I use toolshed version package_r_3_0_2 from iuc (with the installed bioclite packages from above)
=> FAILS : crashes on estimateDispersions / fitDispersionFunction step in run_DEXseq.R
Hey Greet and Bjoern, Sorry for the troubles with dependencies during the automated installation procedure! I will add tool dependency file, when I get some time during this weekend :) Vipin | Rätsch Lab On Tue, Jul 29, 2014 at 10:06 AM, Björn Grüning <bjoern.gruening@gmail.com> wrote:
Hi Geert and Vipin,
I might also consider the DESeq package and the wrappers from galaxytools if you plan to update your wrappers.
https://github.com/bgruening/galaxytools/tree/master/ orphan_tool_dependencies/package_deseq2_1_2_10
@Geert: impressive documentation! Thanks a lot for sharing! Bjoern
Am 29.07.2014 um 15:54 schrieb Geert Vandeweyer:
I know this is a very old entry to the mailing list, but I thought this
might be useful for some people.
Recently, I set up DESeq-hts and its siblings on our galaxy instance. To succeed, I performed the steps below.
@vipin: would you consider revising the tool_dependencies.xml for these tools to take these steps into account? If you're interested, I might compose a preliminary version. I believe that most, if not all of these settings can be automated using the tool_dependencies files.
I did not manage to get DEXSeq working, as it throws errors on the 'estimateDispersions' step in R. I'm not sure how to solve this, as it gives me the same errors using the demonstration data from the pasilla library, following the manual steps. (R 3.0.2, bioconductor 2.13, DEXSeq 1.8). Any hints on that are welcome.
Best,
Geert
## GALAXY PACKAGES IN TOOLSHED ####################### - install the DESeq-hts tools from the main toolshed (cec4b4fb30be from vipints) - install R into galaxy (I used package_R_3_0_2 from iuc (50f7e1e71271)) - install package_scipi and package_numpi (from iuc) - install package_samtools-0.1.18 (I used devteam/package_samtools_0_1_18/c0f72bdba484) - install htseq_count package (I used d5edaf8dc974 from lparsons)
## INSTALL DESeq bioconductor package ######################### - open R from the commandline (using the R version installed earlier !), install bioconductor package 'DESeq' : - source("http://bioconductor.org/biocLite.R") - biocLite('DESeq') - biocLite('DESeq2') - biocLite('DEXSeq') (needed extra ubuntu package libcurl4-gnutls-dev for this)
## complie samtools from source (temporary) ############################# - download samtools 0.1.18 source code into a temporary location and unpack - compile with : "make CXXFLAGS=-fPIC CFLAGS=-fPIC CPPFLAGS=-fPIC"
## install octave ########### - OS-dependent, ubuntu: apt-get install octave
## set up DESeq-hts-1.0 ############### - go to /<path_to_shed_tools>/toolshed.g2.bx.psu.edu/repos/vipints/ deseq_hts/cec4b4fb30be/deseq_hts/deseq-hts_1.0
- run : "setup_deseq-hts.sh" and provide paths as requested. - NOTE: set samtools path to the source compiled version for now (shed version fails to find sam.h)!! - NOTE: also add NumPy path to the SCIPY_PATH variable - rest: example (available in bin/deseq_config.sh): export LD_LIBRARY_PATH= export ENVIRONMENT=galaxy export DESEQ_VERSION=1.12.1 export DESEQ_PATH=/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/ vipints/deseq_hts/cec4b4fb30be/deseq_hts/deseq-hts_1.0
export DESEQ_SRC_PATH=/galaxy/shed_tools/toolshed.g2.bx.psu.edu/ repos/vipints/deseq_hts/cec4b4fb30be/deseq_hts/deseq-hts_1.0/src
export DESEQ_BIN_PATH=/galaxy/shed_tools/toolshed.g2.bx.psu.edu/ repos/vipints/deseq_hts/cec4b4fb30be/deseq_hts/deseq-hts_1.0/bin
export INTERPRETER=octave export MATLAB_BIN_PATH= export MATLAB_MEX_PATH= export MATLAB_INCLUDE_DIR= export OCTAVE_BIN_PATH=/opt/software/octave-3.8.0/bin/octave export OCTAVE_MKOCT=/opt/software/octave-3.8.0/bin/mkoctfile export SAMTOOLS_DIR=/galaxy/packages_ sources/samtools-0.1.18_fPIC #export SAMTOOLS_DIR=/galaxy/galaxy_tool_binaries/samtools/0.1.18/ devteam/package_samtools_0_1_18/c0f72bdba484/bin/
export PYTHON_PATH=/galaxy/galaxy_env/bin/python export SCIPY_PATH=/galaxy/galaxy_tool_binaries/scipy/0.12.0/ iuc/package_scipy_0_12/984d208b0808/lib/python/:/ galaxy/galaxy_tool_binaries/numpy/1.7.1/iuc/package_numpy_ 1_7/ef12a3a11d5b/lib/python/
## USE THE TOOLSHED VERSION, The one you installed bioclite packages with. export R_PATH=/galaxy/galaxy_tool_binaries/R_3_0_2/3.0.2/iuc/ package_r_3_0_2/50f7e1e71271/R
- enter mex directory, run "make octave" - swap SAMTOOLS_DIR in bin/deseq_config.sh to point to the toolshed version of samtools 0.1.18
## RUN DESeq-hts-1.0 TEST ################## enter src directory, run test from the toolconf xml file: "./deseq-hts.sh ../test_data/deseq_c_elegans_WS200-I-regions.gff3 ../test_data/deseq_c_elegans_WS200-I-regions_deseq.txt ../test_data/genes.mat ../test_data/deseq_c_elegans_WS200-I-regions-SRX001872.bam ../test_data/deseq_c_elegans_WS200-I-regions-SRX001875.bam"
## set up DESeq-hts-2.0 ############### - go to /<path_to_shed_tools>/toolshed.g2.bx.psu.edu/repos/vipints/ deseq_hts/cec4b4fb30be/deseq_hts/deseq-hts_2.0
- similar to 1.0: setup config file, - point to src compiled version of samtools - add scipy and numpy to scipi_path variable - use the toolshed version of R - in mex, run "make octave" - swap samtools path
IMPORTANT: ---------------- - open "src/deseq2-hts.sh" in a text editor and add an extra line after line 38 (echo load the genome annotation....): export PYTHONPATH=$PYTHONPATH:${SCIPY_PATH} - in the same file (src/deseq2-hts.sh), append line 70 (../bin/get_read_counts...) with 2>&1 . This prevents some octave warnings from crashing the job (missing docstring warnings)
Test-run in galaxy seems to work.
## set up DEXSeq-hts-1.0 ############### go to /<path_to_shed_tools>/toolshed.g2.bx.psu.edu/repos/vipints/ deseq_hts/cec4b4fb30be/deseq_hts/dexseq_1.0
run setup_dexseq-hts.sh - set samtools path - set python_path : I use the galaxy python version (/galaxy/galaxy_env/bin/python), see development docs - set pythonpath: Make sure to include both HTSeq AND numpy here ! : export PYTHONPATH=/galaxy/galaxy_tool_binaries/htseq/0.5.4p5/ lparsons/htseq_count/3ffe4e2572a7/lib/python/HTSeq- 0.5.4p5-py2.6-linux-x86_64.egg:/galaxy/galaxy_tool_ binaries/numpy/1.7.1/iuc/package_numpy_1_7/ef12a3a11d5b/lib/python:$ PYTHONPATH
- set R_path : I use toolshed version package_r_3_0_2 from iuc (with the installed bioclite packages from above)
=> FAILS : crashes on estimateDispersions / fitDispersionFunction step in run_DEXseq.R
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
participants (3)
-
Björn Grüning
-
Geert Vandeweyer
-
Vipin TS