Hi dev team, I have a very big problem since I updated to galaxy 17.05 with deeptools from toolshed. Due to a bug in galaxy 17.05 with toolshed installed tools (reported in this issue https://github.com/galaxyproject/galaxy/issues/4591), I re-installed deeptools to try to resolve it. Not resolved really but it's another problem. I temporarly bypassed it. Here is my problem : I've installed deeptools from toolshed and when we try to use "compute matrix" for example, we have this error message : Fatal error: Exit code 1 () Traceback (most recent call last): File "/usr/local/bin/computeMatrix", line 4, in <module> from deeptools.computeMatrix import main ImportError: No module named deeptools.computeMatrix It seems that wrapper are installed but not the tool "deeptools". For more information, here it's the way for the installation : search and install via toolshed Install "suite_deeptools" (version 2.5.1.1.0 : 3d68b716965a) Info in Manage tool dependencies : version=2.5.1, resolver=Conda, green check symbol in the line with a list of all deeptools. Galaxy 17.05, commit cfabe37 (21 sept 2017) I've tried to uninstall/reinstall dependencies with conda but without success. Have you any idea to install the deeptools please ? Thanks. Julie
This looks like the issue I’ve recently seen as well - I just discussed this with @bgruening on gitter - https://gitter.im/galaxy-iuc/iuc. I’m using a work-around of creating the __deeptools conda env manually. In my case, I’m not installing the whole suite, just the bamCoverage tool from the TS - https://toolshed.g2.bx.psu.edu/view/bgruening/deeptools_bam_coverage/5d11599.... My work-around is to create the conda env manually: $conda create -n __deeptools@2.5.1 deep tools=2.5.1 This env is created in the conda environment outside of Galaxy, so I just copy it to the conda environment I’ve configured for Galaxy. This works, but it seems there is some issue with the installation from the TS, so perhaps a more ling term fix is needed. On Oct 2, 2017, at 8:59 AM, julie dubois <dubjulie@gmail.com<mailto:dubjulie@gmail.com>> wrote: Hi dev team, I have a very big problem since I updated to galaxy 17.05 with deeptools from toolshed. Due to a bug in galaxy 17.05 with toolshed installed tools (reported in this issue https://github.com/galaxyproject/galaxy/issues/4591), I re-installed deeptools to try to resolve it. Not resolved really but it's another problem. I temporarly bypassed it. Here is my problem : I've installed deeptools from toolshed and when we try to use "compute matrix" for example, we have this error message : Fatal error: Exit code 1 () Traceback (most recent call last): File "/usr/local/bin/computeMatrix", line 4, in <module> from deeptools.computeMatrix import main ImportError: No module named deeptools.computeMatrix It seems that wrapper are installed but not the tool "deeptools". For more information, here it's the way for the installation : search and install via toolshed Install "suite_deeptools" (version 2.5.1.1.0 : 3d68b716965a) Info in Manage tool dependencies : version=2.5.1, resolver=Conda, green check symbol in the line with a list of all deeptools. Galaxy 17.05, commit cfabe37 (21 sept 2017) I've tried to uninstall/reinstall dependencies with conda but without success. Have you any idea to install the deeptools please ? Thanks. Julie ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
Dear Greg, I read your discussion with @bgruening and I think it's not exactly the same. Because in my case, The executable IS present in my galaxy-dist/tool_dependency/_conda/envs/__deeptools@2.5.1/bin . The problem is far away in the execution when the tool (computeMatrix in my example) try to load the python module deeptools.computeMatrix. So is there a solution to discover where is the problem ? Thanks 2017-10-02 15:16 GMT+02:00 Von Kuster, Greg <ghv2@psu.edu>:
This looks like the issue I’ve recently seen as well - I just discussed this with @bgruening on gitter - https://gitter.im/galaxy-iuc/iuc. I’m using a work-around of creating the __deeptools conda env manually. In my case, I’m not installing the whole suite, just the bamCoverage tool from the TS - https://toolshed.g2.bx.psu.edu/view/bgruening/deeptools_bam_coverage/5d11599.... My work-around is to create the conda env manually:
$conda create -n __deeptools@2.5.1 deep tools=2.5.1
This env is created in the conda environment outside of Galaxy, so I just copy it to the conda environment I’ve configured for Galaxy. This works, but it seems there is some issue with the installation from the TS, so perhaps a more ling term fix is needed.
On Oct 2, 2017, at 8:59 AM, julie dubois <dubjulie@gmail.com> wrote:
Hi dev team,
I have a very big problem since I updated to galaxy 17.05 with deeptools from toolshed.
Due to a bug in galaxy 17.05 with toolshed installed tools (reported in this issue https://github.com/galaxyproject/galaxy/issues/4591), I re-installed deeptools to try to resolve it. Not resolved really but it's another problem. I temporarly bypassed it.
Here is my problem : I've installed deeptools from toolshed and when we try to use "compute matrix" for example, we have this error message :
Fatal error: Exit code 1 () Traceback (most recent call last): File "/usr/local/bin/computeMatrix", line 4, in <module> from deeptools.computeMatrix import main ImportError: No module named deeptools.computeMatrix
It seems that wrapper are installed but not the tool "deeptools".
For more information, here it's the way for the installation : search and install via toolshed Install "suite_deeptools" (version 2.5.1.1.0 : 3d68b716965a) Info in Manage tool dependencies : version=2.5.1, resolver=Conda, green check symbol in the line with a list of all deeptools.
Galaxy 17.05, commit cfabe37 (21 sept 2017)
I've tried to uninstall/reinstall dependencies with conda but without success.
Have you any idea to install the deeptools please ?
Thanks. Julie ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
Hi Julie, any chance you are setting PYTHONPATH before starting galaxy's job handlers (a common place would the supervisor config, in case you're using that)? This will interfere with all python modules that are installed via conda. Best, Marius On 3 October 2017 at 09:13, julie dubois <dubjulie@gmail.com> wrote:
Dear Greg, I read your discussion with @bgruening and I think it's not exactly the same. Because in my case, The executable IS present in my galaxy-dist/tool_dependency/_conda/envs/__deeptools@2.5.1/bin . The problem is far away in the execution when the tool (computeMatrix in my example) try to load the python module deeptools.computeMatrix. So is there a solution to discover where is the problem ?
Thanks
This looks like the issue I’ve recently seen as well - I just discussed
2017-10-02 15:16 GMT+02:00 Von Kuster, Greg <ghv2@psu.edu>: this
with @bgruening on gitter - https://gitter.im/galaxy-iuc/iuc. I’m using a work-around of creating the __deeptools conda env manually. In my case, I’m not installing the whole suite, just the bamCoverage tool from the TS - https://toolshed.g2.bx.psu.edu/view/bgruening/deeptools_ bam_coverage/5d11599b8a7d. My work-around is to create the conda env manually:
$conda create -n __deeptools@2.5.1 deep tools=2.5.1
This env is created in the conda environment outside of Galaxy, so I just copy it to the conda environment I’ve configured for Galaxy. This works, but it seems there is some issue with the installation from the TS, so perhaps a more ling term fix is needed.
On Oct 2, 2017, at 8:59 AM, julie dubois <dubjulie@gmail.com> wrote:
Hi dev team,
I have a very big problem since I updated to galaxy 17.05 with deeptools from toolshed.
Due to a bug in galaxy 17.05 with toolshed installed tools (reported in this issue https://github.com/galaxyproject/galaxy/issues/4591), I re-installed deeptools to try to resolve it. Not resolved really but it's another problem. I temporarly bypassed it.
Here is my problem : I've installed deeptools from toolshed and when we try to use "compute matrix" for example, we have this error message :
Fatal error: Exit code 1 () Traceback (most recent call last): File "/usr/local/bin/computeMatrix", line 4, in <module> from deeptools.computeMatrix import main ImportError: No module named deeptools.computeMatrix
It seems that wrapper are installed but not the tool "deeptools".
For more information, here it's the way for the installation : search and install via toolshed Install "suite_deeptools" (version 2.5.1.1.0 : 3d68b716965a) Info in Manage tool dependencies : version=2.5.1, resolver=Conda, green check symbol in the line with a list of all deeptools.
Galaxy 17.05, commit cfabe37 (21 sept 2017)
I've tried to uninstall/reinstall dependencies with conda but without success.
Have you any idea to install the deeptools please ?
Thanks. Julie ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
Also upon re-reading I see that your dependency is being picked up from /usr/local/bin, which indicates that this isn't using conda at all. A small excerpt from the logs when you try to run the tool may be helpful to understand what is going on there. Best, Marius On 3 October 2017 at 09:29, Marius van den Beek <m.vandenbeek@gmail.com> wrote:
Hi Julie,
any chance you are setting PYTHONPATH before starting galaxy's job handlers (a common place would the supervisor config, in case you're using that)? This will interfere with all python modules that are installed via conda.
Best, Marius
On 3 October 2017 at 09:13, julie dubois <dubjulie@gmail.com> wrote:
Dear Greg, I read your discussion with @bgruening and I think it's not exactly the same. Because in my case, The executable IS present in my galaxy-dist/tool_dependency/_conda/envs/__deeptools@2.5.1/bin . The problem is far away in the execution when the tool (computeMatrix in my example) try to load the python module deeptools.computeMatrix. So is there a solution to discover where is the problem ?
Thanks
This looks like the issue I’ve recently seen as well - I just discussed
2017-10-02 15:16 GMT+02:00 Von Kuster, Greg <ghv2@psu.edu>: this
with @bgruening on gitter - https://gitter.im/galaxy-iuc/iuc. I’m using a work-around of creating the __deeptools conda env manually. In my case, I’m not installing the whole suite, just the bamCoverage tool from the TS - https://toolshed.g2.bx.psu.edu/view/bgruening/deeptools_bam_ coverage/5d11599b8a7d. My work-around is to create the conda env manually:
$conda create -n __deeptools@2.5.1 deep tools=2.5.1
This env is created in the conda environment outside of Galaxy, so I just copy it to the conda environment I’ve configured for Galaxy. This works, but it seems there is some issue with the installation from the TS, so perhaps a more ling term fix is needed.
On Oct 2, 2017, at 8:59 AM, julie dubois <dubjulie@gmail.com> wrote:
Hi dev team,
I have a very big problem since I updated to galaxy 17.05 with deeptools from toolshed.
Due to a bug in galaxy 17.05 with toolshed installed tools (reported in this issue https://github.com/galaxyproject/galaxy/issues/4591), I re-installed deeptools to try to resolve it. Not resolved really but it's another problem. I temporarly bypassed it.
Here is my problem : I've installed deeptools from toolshed and when we try to use "compute matrix" for example, we have this error message :
Fatal error: Exit code 1 () Traceback (most recent call last): File "/usr/local/bin/computeMatrix", line 4, in <module> from deeptools.computeMatrix import main ImportError: No module named deeptools.computeMatrix
It seems that wrapper are installed but not the tool "deeptools".
For more information, here it's the way for the installation : search and install via toolshed Install "suite_deeptools" (version 2.5.1.1.0 : 3d68b716965a) Info in Manage tool dependencies : version=2.5.1, resolver=Conda, green check symbol in the line with a list of all deeptools.
Galaxy 17.05, commit cfabe37 (21 sept 2017)
I've tried to uninstall/reinstall dependencies with conda but without success.
Have you any idea to install the deeptools please ?
Thanks. Julie ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
When I run computeMatrix, I have on ly this in the log file : 192.168.000.000 - - [03/oct./2017:10:16:07 +0200] "POST /galaxy/api/tools/deeptools_compute_matrix/build HTTP/1.1" 200 - "http://192.168.100.000:8080/galaxy?job_id=7e6a9fe53e29fc38&__identifer=tmmenbexqep" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:37.0) Gecko/20100101 Firefox/37.0" On the job working directory I have this in tool_script.sh : #!/bin/bash # The following block can be used by the job system # to ensure this script is runnable before actually attempting # to run it. if [ -n "$ABC_TEST_JOB_SCRIPT_INTEGRITY_XYZ" ]; then exit 42 fi export PATH=$PATH:'/data/galaxy-dist/tool_dependency/_conda/envs/__deeptools@2.5.1/bin' ; computeMatrix --version > /data/galaxy-dist/database/tmp/GALAXY_VERSION_STRING_38413 2>&1; ln -f -s "/data/galaxy-dist/database/files/080/dataset_80817.dat" "MACS2narrow_SamToBam_Bowtie_mapped_readsTrim__0.bw" && ln -f -s "/data/galaxy-dist/database/files/080/dataset_80593.dat" "MACS2narrow_SamToBam_Bowtie_mapped_readsTrim__0.bed" && computeMatrix scale-regions --regionsFileName 'MACS2narrow_SamToBam_Bowtie_mapped_readsTrim__0.bed' --scoreFileName 'MACS2narrow_SamToBam_Bowtie_mapped_readsTrim__0.bw' --outFileName '/data/galaxy-dist/database/files/080/dataset_80995.dat' --numberOfProcessors "${GALAXY_SLOTS:-4}" --regionBodyLength 500 It seems to export the good path for executable. So I don't understand. In the galaxy_38413.sh : #!/bin/bash # The following block can be used by the job system # to ensure this script is runnable before actually attempting # to run it. if [ -n "$ABC_TEST_JOB_SCRIPT_INTEGRITY_XYZ" ]; then exit 42 fi GALAXY_SLOTS="1"; export GALAXY_SLOTS; export GALAXY_SLOTS PRESERVE_GALAXY_ENVIRONMENT="True" GALAXY_LIB="/data/galaxy-dist/lib" if [ "$GALAXY_LIB" != "None" -a "$PRESERVE_GALAXY_ENVIRONMENT" = "True" ]; then if [ -n "$PYTHONPATH" ]; then PYTHONPATH="$GALAXY_LIB:$PYTHONPATH" else PYTHONPATH="$GALAXY_LIB" fi export PYTHONPATH fi GALAXY_VIRTUAL_ENV="/data/galaxy-dist/.venv" if [ "$GALAXY_VIRTUAL_ENV" != "None" -a -z "$VIRTUAL_ENV" \ -a -f "$GALAXY_VIRTUAL_ENV/bin/activate" -a "$PRESERVE_GALAXY_ENVIRONMENT" = "True" ]; then . "$GALAXY_VIRTUAL_ENV/bin/activate" fi echo "$GALAXY_SLOTS" > '/data/galaxy-dist/database/job_working_directory/038/38413/__instrument_core_galaxy_slots' date +"%s" > /data/galaxy-dist/database/job_working_directory/038/38413/__instrument_core_epoch_start cd /data/galaxy-dist/database/job_working_directory/038/38413 rm -rf working; mkdir -p working; cd working; /data/galaxy-dist/database/job_working_directory/038/38413/tool_script.sh; return_code=$?; cd '/data/galaxy-dist/database/job_working_directory/038/38413'; if [ "$GALAXY_LIB" != "None" ]; then if [ -n "$PYTHONPATH" ]; then PYTHONPATH="$GALAXY_LIB:$PYTHONPATH" else PYTHONPATH="$GALAXY_LIB" fi export PYTHONPATH fi if [ "$GALAXY_VIRTUAL_ENV" != "None" -a -z "$VIRTUAL_ENV" -a -f "$GALAXY_VIRTUAL_ENV/bin/activate" ]; then . "$GALAXY_VIRTUAL_ENV/bin/activate" fi GALAXY_PYTHON=`command -v python` export PATH=$PATH:'/data/galaxy-dist/tool_dependency/_conda/envs/__samtools@1.3.1/bin' ; python "/data/galaxy-dist/database/job_working_directory/038/38413/set_metadata_c9AkjG.py" "/data/galaxy-dist/database/job_working_directory/038/38413/registry.xml" "/data/galaxy-dist/database/job_working_directory/038/38413/working/galaxy.json" "/data/galaxy-dist/database/job_working_directory/038/38413/metadata_in_HistoryDatasetAssociation_72822_Eqsgna,/data/galaxy-dist/database/job_working_directory/038/38413/metadata_kwds_HistoryDatasetAssociation_72822_3VQ3be,/data/galaxy-dist/database/job_working_directory/038/38413/metadata_out_HistoryDatasetAssociation_72822_TQg8Pq,/data/galaxy-dist/database/job_working_directory/038/38413/metadata_results_HistoryDatasetAssociation_72822_ctopJa,/data/galaxy-dist/database/files/080/dataset_80995.dat,/data/galaxy-dist/database/job_working_directory/038/38413/metadata_override_HistoryDatasetAssociation_72822_Izp4CZ" 5242880; sh -c "exit $return_code" echo $? > /data/galaxy-dist/database/job_working_directory/038/38413/galaxy_38413.ec date +"%s" > /data/galaxy-dist/database/job_working_directory/038/38413/__instrument_core_epoch_end May be my galaxy.ini in not well configured for conda. For the moment concerning conda in galaxy.ini, I have these lines uncommented : preserve_python_environment = legacy_and_local dependency_resolvers_config_file = config/dependency_resolvers_conf.xml 2017-10-03 9:53 GMT+02:00 Marius van den Beek <m.vandenbeek@gmail.com>:
Also upon re-reading I see that your dependency is being picked up from /usr/local/bin, which indicates that this isn't using conda at all. A small excerpt from the logs when you try to run the tool may be helpful to understand what is going on there.
Best, Marius
On 3 October 2017 at 09:29, Marius van den Beek <m.vandenbeek@gmail.com> wrote:
Hi Julie,
any chance you are setting PYTHONPATH before starting galaxy's job handlers (a common place would the supervisor config, in case you're using that)? This will interfere with all python modules that are installed via conda.
Best, Marius
On 3 October 2017 at 09:13, julie dubois <dubjulie@gmail.com> wrote:
Dear Greg, I read your discussion with @bgruening and I think it's not exactly the same. Because in my case, The executable IS present in my galaxy-dist/tool_dependency/_conda/envs/__deeptools@2.5.1/bin . The problem is far away in the execution when the tool (computeMatrix in my example) try to load the python module deeptools.computeMatrix. So is there a solution to discover where is the problem ?
Thanks
2017-10-02 15:16 GMT+02:00 Von Kuster, Greg <ghv2@psu.edu>:
This looks like the issue I’ve recently seen as well - I just discussed this with @bgruening on gitter - https://gitter.im/galaxy-iuc/iuc. I’m using a work-around of creating the __deeptools conda env manually. In my case, I’m not installing the whole suite, just the bamCoverage tool from the TS -
https://toolshed.g2.bx.psu.edu/view/bgruening/deeptools_bam_coverage/5d11599.... My work-around is to create the conda env manually:
$conda create -n __deeptools@2.5.1 deep tools=2.5.1
This env is created in the conda environment outside of Galaxy, so I just copy it to the conda environment I’ve configured for Galaxy. This works, but it seems there is some issue with the installation from the TS, so perhaps a more ling term fix is needed.
On Oct 2, 2017, at 8:59 AM, julie dubois <dubjulie@gmail.com> wrote:
Hi dev team,
I have a very big problem since I updated to galaxy 17.05 with deeptools from toolshed.
Due to a bug in galaxy 17.05 with toolshed installed tools (reported in this issue https://github.com/galaxyproject/galaxy/issues/4591), I re-installed deeptools to try to resolve it. Not resolved really but it's another problem. I temporarly bypassed it.
Here is my problem : I've installed deeptools from toolshed and when we try to use "compute matrix" for example, we have this error message :
Fatal error: Exit code 1 () Traceback (most recent call last): File "/usr/local/bin/computeMatrix", line 4, in <module> from deeptools.computeMatrix import main ImportError: No module named deeptools.computeMatrix
It seems that wrapper are installed but not the tool "deeptools".
For more information, here it's the way for the installation : search and install via toolshed Install "suite_deeptools" (version 2.5.1.1.0 : 3d68b716965a) Info in Manage tool dependencies : version=2.5.1, resolver=Conda, green check symbol in the line with a list of all deeptools.
Galaxy 17.05, commit cfabe37 (21 sept 2017)
I've tried to uninstall/reinstall dependencies with conda but without success.
Have you any idea to install the deeptools please ?
Thanks. Julie ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
So the `preserve_python_environment = legacy_and_local` setting is causing this for you. Now normally deeptools is not a local tool, but I understand that because of https://github.com/galaxyproject/galaxy/issues/4591, you have manipulated the shed_tool_conf.xml file, and it is now being considered as a local tool. In any case this setting should be set to legacy_and_local only if you experience difficulties when it is not set, that is why `legacy` is the default. On 3 October 2017 at 10:30, julie dubois <dubjulie@gmail.com> wrote:
When I run computeMatrix, I have on ly this in the log file :
192.168.000.000 - - [03/oct./2017:10:16:07 +0200] "POST /galaxy/api/tools/deeptools_compute_matrix/build HTTP/1.1" 200 - "http://192.168.100.000:8080/galaxy?job_id=7e6a9fe53e29fc38&__identifer= tmmenbexqep" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:37.0) Gecko/20100101 Firefox/37.0"
On the job working directory I have this in tool_script.sh : #!/bin/bash
# The following block can be used by the job system # to ensure this script is runnable before actually attempting # to run it. if [ -n "$ABC_TEST_JOB_SCRIPT_INTEGRITY_XYZ" ]; then exit 42 fi export PATH=$PATH:'/data/galaxy-dist/tool_dependency/_conda/envs/__ deeptools@2.5.1/bin' ; computeMatrix --version > /data/galaxy-dist/database/tmp/GALAXY_VERSION_STRING_38413 2>&1; ln -f -s "/data/galaxy-dist/database/files/080/dataset_80817.dat" "MACS2narrow_SamToBam_Bowtie_mapped_readsTrim__0.bw" && ln -f -s "/data/galaxy-dist/database/files/080/dataset_80593.dat" "MACS2narrow_SamToBam_Bowtie_mapped_readsTrim__0.bed" && computeMatrix scale-regions --regionsFileName 'MACS2narrow_SamToBam_Bowtie_mapped_readsTrim__0.bed' --scoreFileName 'MACS2narrow_SamToBam_Bowtie_mapped_readsTrim__0.bw' --outFileName '/data/galaxy-dist/database/files/080/dataset_80995.dat' --numberOfProcessors "${GALAXY_SLOTS:-4}" --regionBodyLength 500
It seems to export the good path for executable. So I don't understand.
In the galaxy_38413.sh : #!/bin/bash
# The following block can be used by the job system # to ensure this script is runnable before actually attempting # to run it. if [ -n "$ABC_TEST_JOB_SCRIPT_INTEGRITY_XYZ" ]; then exit 42 fi
GALAXY_SLOTS="1"; export GALAXY_SLOTS; export GALAXY_SLOTS PRESERVE_GALAXY_ENVIRONMENT="True" GALAXY_LIB="/data/galaxy-dist/lib" if [ "$GALAXY_LIB" != "None" -a "$PRESERVE_GALAXY_ENVIRONMENT" = "True" ]; then if [ -n "$PYTHONPATH" ]; then PYTHONPATH="$GALAXY_LIB:$PYTHONPATH" else PYTHONPATH="$GALAXY_LIB" fi export PYTHONPATH fi
GALAXY_VIRTUAL_ENV="/data/galaxy-dist/.venv" if [ "$GALAXY_VIRTUAL_ENV" != "None" -a -z "$VIRTUAL_ENV" \ -a -f "$GALAXY_VIRTUAL_ENV/bin/activate" -a "$PRESERVE_GALAXY_ENVIRONMENT" = "True" ]; then . "$GALAXY_VIRTUAL_ENV/bin/activate" fi echo "$GALAXY_SLOTS" > '/data/galaxy-dist/database/job_working_directory/038/ 38413/__instrument_core_galaxy_slots' date +"%s" > /data/galaxy-dist/database/job_working_directory/038/ 38413/__instrument_core_epoch_start cd /data/galaxy-dist/database/job_working_directory/038/38413 rm -rf working; mkdir -p working; cd working; /data/galaxy-dist/database/job_working_directory/038/38413/tool_script.sh; return_code=$?; cd '/data/galaxy-dist/database/job_working_directory/038/38413'; if [ "$GALAXY_LIB" != "None" ]; then if [ -n "$PYTHONPATH" ]; then PYTHONPATH="$GALAXY_LIB:$PYTHONPATH" else PYTHONPATH="$GALAXY_LIB" fi export PYTHONPATH fi if [ "$GALAXY_VIRTUAL_ENV" != "None" -a -z "$VIRTUAL_ENV" -a -f "$GALAXY_VIRTUAL_ENV/bin/activate" ]; then . "$GALAXY_VIRTUAL_ENV/bin/activate" fi GALAXY_PYTHON=`command -v python` export PATH=$PATH:'/data/galaxy-dist/tool_dependency/_conda/envs/__ samtools@1.3.1/bin' ; python "/data/galaxy-dist/database/job_working_directory/038/ 38413/set_metadata_c9AkjG.py" "/data/galaxy-dist/database/job_working_directory/038/38413/registry.xml" "/data/galaxy-dist/database/job_working_directory/038/ 38413/working/galaxy.json" "/data/galaxy-dist/database/job_working_directory/038/38413/metadata_in_ HistoryDatasetAssociation_72822_Eqsgna,/data/galaxy- dist/database/job_working_directory/038/38413/metadata_kwds_ HistoryDatasetAssociation_72822_3VQ3be,/data/galaxy- dist/database/job_working_directory/038/38413/metadata_ out_HistoryDatasetAssociation_72822_TQg8Pq,/data/galaxy- dist/database/job_working_directory/038/38413/metadata_results_ HistoryDatasetAssociation_72822_ctopJa,/data/galaxy- dist/database/files/080/dataset_80995.dat,/data/galaxy-dist/database/job_ working_directory/038/38413/metadata_override_HistoryDatasetAssociation_ 72822_Izp4CZ" 5242880; sh -c "exit $return_code" echo $? > /data/galaxy-dist/database/job_working_directory/038/38413/ galaxy_38413.ec date +"%s" > /data/galaxy-dist/database/job_working_directory/038/ 38413/__instrument_core_epoch_end
May be my galaxy.ini in not well configured for conda. For the moment concerning conda in galaxy.ini, I have these lines uncommented :
preserve_python_environment = legacy_and_local dependency_resolvers_config_file = config/dependency_resolvers_conf.xml
2017-10-03 9:53 GMT+02:00 Marius van den Beek <m.vandenbeek@gmail.com>:
Also upon re-reading I see that your dependency is being picked up from /usr/local/bin, which indicates that this isn't using conda at all. A small excerpt from the logs when you try to run the tool may be helpful to understand what is going on there.
Best, Marius
On 3 October 2017 at 09:29, Marius van den Beek <m.vandenbeek@gmail.com> wrote:
Hi Julie,
any chance you are setting PYTHONPATH before starting galaxy's job handlers (a common place would the supervisor config, in case you're
using
that)? This will interfere with all python modules that are installed via conda.
Best, Marius
On 3 October 2017 at 09:13, julie dubois <dubjulie@gmail.com> wrote:
Dear Greg, I read your discussion with @bgruening and I think it's not exactly the same. Because in my case, The executable IS present in my galaxy-dist/tool_dependency/_conda/envs/__deeptools@2.5.1/bin . The problem is far away in the execution when the tool (computeMatrix in my example) try to load the python module deeptools.computeMatrix. So is there a solution to discover where is the problem ?
Thanks
2017-10-02 15:16 GMT+02:00 Von Kuster, Greg <ghv2@psu.edu>:
This looks like the issue I’ve recently seen as well - I just
discussed
this with @bgruening on gitter - https://gitter.im/galaxy-iuc/iuc. I’m using a work-around of creating the __deeptools conda env manually. In my case, I’m not installing the whole suite, just the bamCoverage tool from the TS -
https://toolshed.g2.bx.psu.edu/view/bgruening/deeptools_ bam_coverage/5d11599b8a7d. My work-around is to create the conda env manually:
$conda create -n __deeptools@2.5.1 deep tools=2.5.1
This env is created in the conda environment outside of Galaxy, so I just copy it to the conda environment I’ve configured for Galaxy. This works, but it seems there is some issue with the installation from the TS, so perhaps a more ling term fix is needed.
On Oct 2, 2017, at 8:59 AM, julie dubois <dubjulie@gmail.com> wrote:
Hi dev team,
I have a very big problem since I updated to galaxy 17.05 with deeptools from toolshed.
Due to a bug in galaxy 17.05 with toolshed installed tools (reported in this issue https://github.com/galaxyproject/galaxy/issues/4591), I re-installed deeptools to try to resolve it. Not resolved really but it's another problem. I temporarly bypassed it.
Here is my problem : I've installed deeptools from toolshed and when we try to use "compute matrix" for example, we have this error message :
Fatal error: Exit code 1 () Traceback (most recent call last): File "/usr/local/bin/computeMatrix", line 4, in <module> from deeptools.computeMatrix import main ImportError: No module named deeptools.computeMatrix
It seems that wrapper are installed but not the tool "deeptools".
For more information, here it's the way for the installation : search and install via toolshed Install "suite_deeptools" (version 2.5.1.1.0 : 3d68b716965a) Info in Manage tool dependencies : version=2.5.1, resolver=Conda, green check symbol in the line with a list of all deeptools.
Galaxy 17.05, commit cfabe37 (21 sept 2017)
I've tried to uninstall/reinstall dependencies with conda but without success.
Have you any idea to install the deeptools please ?
Thanks. Julie ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
OK, so I've modified my galaxy.ini to set `preserve_python_environment= legacy` Then I 'm going back to the good shed_tool_conf.xml file : it means no modification in this file regarding the original. I've restarted galaxy and the error traced in https://github.com/galaxyproject/galaxy/issues/4591 reappears. So, I'm answering you in https://github.com/galaxyproject/galaxy/issues/4591 and may be the problem here will disappear. Thanks again for your help 2017-10-03 10:41 GMT+02:00 Marius van den Beek <m.vandenbeek@gmail.com>:
So the `preserve_python_environment = legacy_and_local` setting is causing this for you. Now normally deeptools is not a local tool, but I understand that because of https://github.com/galaxyproject/galaxy/issues/4591, you have manipulated the shed_tool_conf.xml file, and it is now being considered as a local tool. In any case this setting should be set to legacy_and_local only if you experience difficulties when it is not set, that is why `legacy` is the default.
On 3 October 2017 at 10:30, julie dubois <dubjulie@gmail.com> wrote:
When I run computeMatrix, I have on ly this in the log file :
192.168.000.000 - - [03/oct./2017:10:16:07 +0200] "POST /galaxy/api/tools/deeptools_compute_matrix/build HTTP/1.1" 200 -
"http://192.168.100.000:8080/galaxy?job_id=7e6a9fe53e29fc38&__identifer=tmmenbexqep" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:37.0) Gecko/20100101 Firefox/37.0"
On the job working directory I have this in tool_script.sh : #!/bin/bash
# The following block can be used by the job system # to ensure this script is runnable before actually attempting # to run it. if [ -n "$ABC_TEST_JOB_SCRIPT_INTEGRITY_XYZ" ]; then exit 42 fi export PATH=$PATH:'/data/galaxy-dist/tool_dependency/_conda/envs/__deeptools@2.5.1/bin' ; computeMatrix --version > /data/galaxy-dist/database/tmp/GALAXY_VERSION_STRING_38413 2>&1; ln -f -s "/data/galaxy-dist/database/files/080/dataset_80817.dat" "MACS2narrow_SamToBam_Bowtie_mapped_readsTrim__0.bw" && ln -f -s "/data/galaxy-dist/database/files/080/dataset_80593.dat" "MACS2narrow_SamToBam_Bowtie_mapped_readsTrim__0.bed" && computeMatrix scale-regions --regionsFileName 'MACS2narrow_SamToBam_Bowtie_mapped_readsTrim__0.bed' --scoreFileName 'MACS2narrow_SamToBam_Bowtie_mapped_readsTrim__0.bw' --outFileName '/data/galaxy-dist/database/files/080/dataset_80995.dat' --numberOfProcessors "${GALAXY_SLOTS:-4}" --regionBodyLength 500
It seems to export the good path for executable. So I don't understand.
In the galaxy_38413.sh : #!/bin/bash
# The following block can be used by the job system # to ensure this script is runnable before actually attempting # to run it. if [ -n "$ABC_TEST_JOB_SCRIPT_INTEGRITY_XYZ" ]; then exit 42 fi
GALAXY_SLOTS="1"; export GALAXY_SLOTS; export GALAXY_SLOTS PRESERVE_GALAXY_ENVIRONMENT="True" GALAXY_LIB="/data/galaxy-dist/lib" if [ "$GALAXY_LIB" != "None" -a "$PRESERVE_GALAXY_ENVIRONMENT" = "True" ]; then if [ -n "$PYTHONPATH" ]; then PYTHONPATH="$GALAXY_LIB:$PYTHONPATH" else PYTHONPATH="$GALAXY_LIB" fi export PYTHONPATH fi
GALAXY_VIRTUAL_ENV="/data/galaxy-dist/.venv" if [ "$GALAXY_VIRTUAL_ENV" != "None" -a -z "$VIRTUAL_ENV" \ -a -f "$GALAXY_VIRTUAL_ENV/bin/activate" -a "$PRESERVE_GALAXY_ENVIRONMENT" = "True" ]; then . "$GALAXY_VIRTUAL_ENV/bin/activate" fi echo "$GALAXY_SLOTS" >
'/data/galaxy-dist/database/job_working_directory/038/38413/__instrument_core_galaxy_slots' date +"%s" > /data/galaxy-dist/database/job_working_directory/038/38413/__instrument_core_epoch_start cd /data/galaxy-dist/database/job_working_directory/038/38413 rm -rf working; mkdir -p working; cd working; /data/galaxy-dist/database/job_working_directory/038/38413/tool_script.sh; return_code=$?; cd '/data/galaxy-dist/database/job_working_directory/038/38413'; if [ "$GALAXY_LIB" != "None" ]; then if [ -n "$PYTHONPATH" ]; then PYTHONPATH="$GALAXY_LIB:$PYTHONPATH" else PYTHONPATH="$GALAXY_LIB" fi export PYTHONPATH fi if [ "$GALAXY_VIRTUAL_ENV" != "None" -a -z "$VIRTUAL_ENV" -a -f "$GALAXY_VIRTUAL_ENV/bin/activate" ]; then . "$GALAXY_VIRTUAL_ENV/bin/activate" fi GALAXY_PYTHON=`command -v python` export PATH=$PATH:'/data/galaxy-dist/tool_dependency/_conda/envs/__samtools@1.3.1/bin' ; python "/data/galaxy-dist/database/job_working_directory/038/38413/set_metadata_c9AkjG.py" "/data/galaxy-dist/database/job_working_directory/038/38413/registry.xml"
"/data/galaxy-dist/database/job_working_directory/038/38413/working/galaxy.json"
"/data/galaxy-dist/database/job_working_directory/038/38413/metadata_in_HistoryDatasetAssociation_72822_Eqsgna,/data/galaxy-dist/database/job_working_directory/038/38413/metadata_kwds_HistoryDatasetAssociation_72822_3VQ3be,/data/galaxy-dist/database/job_working_directory/038/38413/metadata_out_HistoryDatasetAssociation_72822_TQg8Pq,/data/galaxy-dist/database/job_working_directory/038/38413/metadata_results_HistoryDatasetAssociation_72822_ctopJa,/data/galaxy-dist/database/files/080/dataset_80995.dat,/data/galaxy-dist/database/job_working_directory/038/38413/metadata_override_HistoryDatasetAssociation_72822_Izp4CZ" 5242880; sh -c "exit $return_code" echo $? > /data/galaxy-dist/database/job_working_directory/038/38413/galaxy_38413.ec date +"%s" > /data/galaxy-dist/database/job_working_directory/038/38413/__instrument_core_epoch_end
May be my galaxy.ini in not well configured for conda. For the moment concerning conda in galaxy.ini, I have these lines uncommented :
preserve_python_environment = legacy_and_local dependency_resolvers_config_file = config/dependency_resolvers_conf.xml
2017-10-03 9:53 GMT+02:00 Marius van den Beek <m.vandenbeek@gmail.com>:
Also upon re-reading I see that your dependency is being picked up from /usr/local/bin, which indicates that this isn't using conda at all. A small excerpt from the logs when you try to run the tool may be helpful to understand what is going on there.
Best, Marius
On 3 October 2017 at 09:29, Marius van den Beek <m.vandenbeek@gmail.com> wrote:
Hi Julie,
any chance you are setting PYTHONPATH before starting galaxy's job handlers (a common place would the supervisor config, in case you're using that)? This will interfere with all python modules that are installed via conda.
Best, Marius
On 3 October 2017 at 09:13, julie dubois <dubjulie@gmail.com> wrote:
Dear Greg, I read your discussion with @bgruening and I think it's not exactly the same. Because in my case, The executable IS present in my galaxy-dist/tool_dependency/_conda/envs/__deeptools@2.5.1/bin . The problem is far away in the execution when the tool (computeMatrix in my example) try to load the python module deeptools.computeMatrix. So is there a solution to discover where is the problem ?
Thanks
2017-10-02 15:16 GMT+02:00 Von Kuster, Greg <ghv2@psu.edu>:
This looks like the issue I’ve recently seen as well - I just discussed this with @bgruening on gitter - https://gitter.im/galaxy-iuc/iuc. I’m using a work-around of creating the __deeptools conda env manually. In my case, I’m not installing the whole suite, just the bamCoverage tool from the TS -
https://toolshed.g2.bx.psu.edu/view/bgruening/deeptools_bam_coverage/5d11599.... My work-around is to create the conda env manually:
$conda create -n __deeptools@2.5.1 deep tools=2.5.1
This env is created in the conda environment outside of Galaxy, so I just copy it to the conda environment I’ve configured for Galaxy. This works, but it seems there is some issue with the installation from the TS, so perhaps a more ling term fix is needed.
On Oct 2, 2017, at 8:59 AM, julie dubois <dubjulie@gmail.com> wrote:
Hi dev team,
I have a very big problem since I updated to galaxy 17.05 with deeptools from toolshed.
Due to a bug in galaxy 17.05 with toolshed installed tools (reported in this issue https://github.com/galaxyproject/galaxy/issues/4591), I re-installed deeptools to try to resolve it. Not resolved really but it's another problem. I temporarly bypassed it.
Here is my problem : I've installed deeptools from toolshed and when we try to use "compute matrix" for example, we have this error message :
Fatal error: Exit code 1 () Traceback (most recent call last): File "/usr/local/bin/computeMatrix", line 4, in <module> from deeptools.computeMatrix import main ImportError: No module named deeptools.computeMatrix
It seems that wrapper are installed but not the tool "deeptools".
For more information, here it's the way for the installation : search and install via toolshed Install "suite_deeptools" (version 2.5.1.1.0 : 3d68b716965a) Info in Manage tool dependencies : version=2.5.1, resolver=Conda, green check symbol in the line with a list of all deeptools.
Galaxy 17.05, commit cfabe37 (21 sept 2017)
I've tried to uninstall/reinstall dependencies with conda but without success.
Have you any idea to install the deeptools please ?
Thanks. Julie ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/
participants (3)
-
julie dubois -
Marius van den Beek -
Von Kuster, Greg