Re: [galaxy-dev] Conda problems: Failed to activate conda environment
Dear Gerhard, I don't know about the python error.. could you tell us the output of `locale` when logged in as the user running the Galaxy server? For the second problem, I remember that I also have a similar problem. For some reasons some symlinks in the conda environment are not created. you can check if in the respective environment's bin directory symlinks to the conda, activate and deactivate binaries/scripts exist. If not you can fix the missing links (I run the following after each tool installation) ``` for env in "{{ galaxy_server_dir }}"/database/dependencies/_conda/envs/*; do for link in conda activate deactivate; do \[ -h $env/bin/$link \] || ln -s "{{ galaxy_server_dir }}"/database/dependencies/_conda/bin/$link $env/bin/$link; done done ``` Hope this helps. Cheers, Matthias On 05.07.2018 18:00, galaxy-dev-request@lists.galaxyproject.org wrote:
Message: 1 Date: Thu, 5 Jul 2018 08:14:35 +0000 From: "Van Wageningen, GERHARD [gerhardv@sun.ac.za]" <gerhardv@sun.ac.za> To:"galaxy-dev@lists.galaxyproject.org" <galaxy-dev@lists.galaxyproject.org> Subject: [galaxy-dev] Conda problems: Failed to activate conda environment Message-ID: <AM0PR07MB4354EEAD1AFB5809409F9F3692400@AM0PR07MB4354.eurprd07.prod.outlook.com>
Content-Type: text/plain; charset="iso-8859-1"
Hi All
I have recently started learning the Galaxy setup starting with an upgrade. Made some headway but I was wondering if someone could comment on a problem that has emerged. The system is a Galaxy 18.05 installation (upgraded from 17.01) on an Altair PBS scheduler with drmaa setup (working)
Conda problem: Failed to activate conda environment.
Conda works during tool installation or reinstallation.
Galaxy tools have been updated/reinstalled so they all look "green" in manage tools.
In Manage Dependencies there are a number of "Dependency resolved but version not found" warnings, otherwise the deps look clean.
When running tools in Galaxy errors occur, after the jobs have correctly deployed on the cluster.
The galaxy application raised errors like the two below.
"Failed to activate conda environment! Error was: Fatal Python error: Py_Initialize: Unable to get the locale encoding File "/home/.galaxy/tooldeps/python/2.7/iuc/package_python_2_7/8b09fe018cac/lib/python2.7/encodings/__init__.py", line 123 raise C"
"
Failed to activate conda environment! Error was: CondaEnvironmentError: Environment error: Cannot activate environment bash. User does not have write access for conda symlinks.
"
Galaxy started and stopped using supervisord with config below:
-- ------------------------------------------- Matthias Bernt Bioinformatics Service Molekulare Systembiologie (MOLSYB) Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/ Helmholtz Centre for Environmental Research GmbH - UFZ Permoserstraße 15, 04318 Leipzig, Germany Phone +49 341 235 482296, m.bernt@ufz.de, www.ufz.de Sitz der Gesellschaft/Registered Office: Leipzig Registergericht/Registration Office: Amtsgericht Leipzig Handelsregister Nr./Trade Register Nr.: B 4703 Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig Wilfried Kraus Wissenschaftlicher Geschäftsführer/Scientific Managing Director: Prof. Dr. Dr. h.c. Georg Teutsch Administrative Geschäftsführerin/ Administrative Managing Director: Prof. Dr. Heike Graßmann -------------------------------------------
Hi Matthias - thanx for the missing link tip - that fixes that ... Locale as Galaxy user is [galaxy@storage1 ~]$ locale LANG=en_US.UTF-8 LC_CTYPE="en_US.UTF-8" LC_NUMERIC="en_US.UTF-8" LC_TIME="en_US.UTF-8" LC_COLLATE="en_US.UTF-8" LC_MONETARY="en_US.UTF-8" LC_MESSAGES="en_US.UTF-8" LC_PAPER="en_US.UTF-8" LC_NAME="en_US.UTF-8" LC_ADDRESS="en_US.UTF-8" LC_TELEPHONE="en_US.UTF-8" LC_MEASUREMENT="en_US.UTF-8" LC_IDENTIFICATION="en_US.UTF-8" LC_ALL= ________________________________ From: Matthias Bernt <m.bernt@ufz.de> Sent: 06 July 2018 10:15:12 To: Van Wageningen, GERHARD [gerhardv@sun.ac.za] Cc: galaxy-dev@lists.galaxyproject.org Subject: Re: Conda problems: Failed to activate conda environment Dear Gerhard, I don't know about the python error.. could you tell us the output of `locale` when logged in as the user running the Galaxy server? For the second problem, I remember that I also have a similar problem. For some reasons some symlinks in the conda environment are not created. you can check if in the respective environment's bin directory symlinks to the conda, activate and deactivate binaries/scripts exist. If not you can fix the missing links (I run the following after each tool installation) ``` for env in "{{ galaxy_server_dir }}"/database/dependencies/_conda/envs/*; do for link in conda activate deactivate; do \[ -h $env/bin/$link \] || ln -s "{{ galaxy_server_dir }}"/database/dependencies/_conda/bin/$link $env/bin/$link; done done ``` Hope this helps. Cheers, Matthias On 05.07.2018 18:00, galaxy-dev-request@lists.galaxyproject.org wrote:
Message: 1 Date: Thu, 5 Jul 2018 08:14:35 +0000 From: "Van Wageningen, GERHARD [gerhardv@sun.ac.za]" <gerhardv@sun.ac.za> To:"galaxy-dev@lists.galaxyproject.org" <galaxy-dev@lists.galaxyproject.org> Subject: [galaxy-dev] Conda problems: Failed to activate conda environment Message-ID: <AM0PR07MB4354EEAD1AFB5809409F9F3692400@AM0PR07MB4354.eurprd07.prod.outlook.com>
Content-Type: text/plain; charset="iso-8859-1"
Hi All
I have recently started learning the Galaxy setup starting with an upgrade. Made some headway but I was wondering if someone could comment on a problem that has emerged. The system is a Galaxy 18.05 installation (upgraded from 17.01) on an Altair PBS scheduler with drmaa setup (working)
Conda problem: Failed to activate conda environment.
Conda works during tool installation or reinstallation.
Galaxy tools have been updated/reinstalled so they all look "green" in manage tools.
In Manage Dependencies there are a number of "Dependency resolved but version not found" warnings, otherwise the deps look clean.
When running tools in Galaxy errors occur, after the jobs have correctly deployed on the cluster.
The galaxy application raised errors like the two below.
"Failed to activate conda environment! Error was: Fatal Python error: Py_Initialize: Unable to get the locale encoding File "/home/.galaxy/tooldeps/python/2.7/iuc/package_python_2_7/8b09fe018cac/lib/python2.7/encodings/__init__.py", line 123 raise C"
"
Failed to activate conda environment! Error was: CondaEnvironmentError: Environment error: Cannot activate environment bash. User does not have write access for conda symlinks.
"
Galaxy started and stopped using supervisord with config below:
-- ------------------------------------------- Matthias Bernt Bioinformatics Service Molekulare Systembiologie (MOLSYB) Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/ Helmholtz Centre for Environmental Research GmbH - UFZ Permoserstraße 15, 04318 Leipzig, Germany Phone +49 341 235 482296, m.bernt@ufz.de, www.ufz.de Sitz der Gesellschaft/Registered Office: Leipzig Registergericht/Registration Office: Amtsgericht Leipzig Handelsregister Nr./Trade Register Nr.: B 4703 Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig Wilfried Kraus Wissenschaftlicher Geschäftsführer/Scientific Managing Director: Prof. Dr. Dr. h.c. Georg Teutsch Administrative Geschäftsführerin/ Administrative Managing Director: Prof. Dr. Heike Graßmann ------------------------------------------- [http://cdn.sun.ac.za/100/ProductionFooter.jpg]<http://www.sun.ac.za/english/Pages/Water-crisis.aspx> The integrity and confidentiality of this email is governed by these terms. Disclaimer<http://www.sun.ac.za/emaildisclaimer> Die integriteit en vertroulikheid van hierdie e-pos word deur die volgende bepalings gereël. Vrywaringsklousule<http://www.sun.ac.za/emaildisclaimer>
Hi Gerhard, wonderful. For the other problem: you seem to use a tool that still uses a toolshed package (python in this case) instead of the conda dependency. Can you make sure that you ensure that conda is used for dependency resolution? Otherwise, which tool are you trying to install? Cheers, Matthias On 06.07.2018 10:38, Van Wageningen, GERHARD [gerhardv@sun.ac.za] wrote:
Hi Matthias - thanx for the missing link tip - that fixes that ...
Locale as Galaxy user is
[galaxy@storage1 ~]$ locale LANG=en_US.UTF-8 LC_CTYPE="en_US.UTF-8" LC_NUMERIC="en_US.UTF-8" LC_TIME="en_US.UTF-8" LC_COLLATE="en_US.UTF-8" LC_MONETARY="en_US.UTF-8" LC_MESSAGES="en_US.UTF-8" LC_PAPER="en_US.UTF-8" LC_NAME="en_US.UTF-8" LC_ADDRESS="en_US.UTF-8" LC_TELEPHONE="en_US.UTF-8" LC_MEASUREMENT="en_US.UTF-8" LC_IDENTIFICATION="en_US.UTF-8" LC_ALL=
------------------------------------------------------------------------ *From:* Matthias Bernt <m.bernt@ufz.de> *Sent:* 06 July 2018 10:15:12 *To:* Van Wageningen, GERHARD [gerhardv@sun.ac.za] *Cc:* galaxy-dev@lists.galaxyproject.org *Subject:* Re: Conda problems: Failed to activate conda environment Dear Gerhard,
I don't know about the python error.. could you tell us the output of `locale` when logged in as the user running the Galaxy server?
For the second problem, I remember that I also have a similar problem. For some reasons some symlinks in the conda environment are not created.
you can check if in the respective environment's bin directory symlinks to the conda, activate and deactivate binaries/scripts exist.
If not you can fix the missing links (I run the following after each tool installation)
``` for env in "{{ galaxy_server_dir }}"/database/dependencies/_conda/envs/*; do for link in conda activate deactivate; do \[ -h $env/bin/$link \] || ln -s "{{ galaxy_server_dir }}"/database/dependencies/_conda/bin/$link $env/bin/$link; done done ```
Hope this helps.
Cheers, Matthias
On 05.07.2018 18:00, galaxy-dev-request@lists.galaxyproject.org wrote:
Message: 1 Date: Thu, 5 Jul 2018 08:14:35 +0000 From: "Van Wageningen, GERHARD [gerhardv@sun.ac.za]" <gerhardv@sun.ac.za> To:"galaxy-dev@lists.galaxyproject.org" <galaxy-dev@lists.galaxyproject.org> Subject: [galaxy-dev] Conda problems: Failed to activate conda environment Message-ID: <AM0PR07MB4354EEAD1AFB5809409F9F3692400@AM0PR07MB4354.eurprd07.prod.outlook.com>
Content-Type: text/plain; charset="iso-8859-1"
Hi All
I have recently started learning the Galaxy setup starting with an upgrade. Made some headway but I was wondering if someone could comment on a problem that has emerged. The system is a Galaxy 18.05 installation (upgraded from 17.01) on an Altair PBS scheduler with drmaa setup (working)
Conda problem: Failed to activate conda environment.
Conda works during tool installation or reinstallation.
Galaxy tools have been updated/reinstalled so they all look "green" in manage tools.
In Manage Dependencies there are a number of "Dependency resolved but version not found" warnings, otherwise the deps look clean.
When running tools in Galaxy errors occur, after the jobs have correctly deployed on the cluster.
The galaxy application raised errors like the two below.
"Failed to activate conda environment! Error was: Fatal Python error: Py_Initialize: Unable to get the locale encoding File "/home/.galaxy/tooldeps/python/2.7/iuc/package_python_2_7/8b09fe018cac/lib/python2.7/encodings/__init__.py", line 123 raise C"
"
Failed to activate conda environment! Error was: CondaEnvironmentError: Environment error: Cannot activate environment bash. User does not have write access for conda symlinks.
"
Galaxy started and stopped using supervisord with config below:
--
------------------------------------------- Matthias Bernt Bioinformatics Service Molekulare Systembiologie (MOLSYB) Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/ Helmholtz Centre for Environmental Research GmbH - UFZ Permoserstraße 15, 04318 Leipzig, Germany Phone +49 341 235 482296, m.bernt@ufz.de, www.ufz.de
Sitz der Gesellschaft/Registered Office: Leipzig Registergericht/Registration Office: Amtsgericht Leipzig Handelsregister Nr./Trade Register Nr.: B 4703 Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig Wilfried Kraus Wissenschaftlicher Geschäftsführer/Scientific Managing Director: Prof. Dr. Dr. h.c. Georg Teutsch Administrative Geschäftsführerin/ Administrative Managing Director: Prof. Dr. Heike Graßmann ------------------------------------------- <http://www.sun.ac.za/english/Pages/Water-crisis.aspx>
The integrity and confidentiality of this email is governed by these terms. Disclaimer <http://www.sun.ac.za/emaildisclaimer > Die integriteit en vertroulikheid van hierdie e-pos word deur die volgende bepalings gereël. Vrywaringsklousule <http://www.sun.ac.za/emaildisclaimer >
-- ------------------------------------------- Matthias Bernt Bioinformatics Service Molekulare Systembiologie (MOLSYB) Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/ Helmholtz Centre for Environmental Research GmbH - UFZ Permoserstraße 15, 04318 Leipzig, Germany Phone +49 341 235 482296, m.bernt@ufz.de, www.ufz.de Sitz der Gesellschaft/Registered Office: Leipzig Registergericht/Registration Office: Amtsgericht Leipzig Handelsregister Nr./Trade Register Nr.: B 4703 Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig Wilfried Kraus Wissenschaftlicher Geschäftsführer/Scientific Managing Director: Prof. Dr. Dr. h.c. Georg Teutsch Administrative Geschäftsführerin/ Administrative Managing Director: Prof. Dr. Heike Graßmann -------------------------------------------
The python error occurs with the bowtie2 installation. Other tools give other errors ... I think the soft links may have sorted out a lot of that. ...still checking. Björn Grüning suggested that I look at the sequence of tool dep lookups. So: dependency_resolvers_conf.xml looks like this (below)... and the galaxy.ini file points to the tooldeps folder in the usual way. I renamed the dependency_resolvers_conf.xml file to force the installation to use the defaults and reinstalled the bowtie2 package. Conda kicked in and did some work. Now it seems as though it is generating errors based on the contents of the dataset ... so I am guessing that there is something was wrong or missing with the dependency_resolvers_conf.xml that was there. Still going through the other tools to test them. Thanks for your help. dependency_resolvers_conf.xml file on system <dependency_resolvers> <!-- the default configuration, first look for dependencies installed from the toolshed --> <tool_shed_packages /> <!-- then look for env.sh files in directories according to the "galaxy packages" schema. These resolvers can take a base_path attribute to specify where to look for package definitions, but by default look in the directory specified by tool_dependency_dir in Galaxy's config/galaxy.ini --> <galaxy_packages /> <galaxy_packages versionless="true" /> <conda /> <conda versionless="true" /> <!-- Example configuration of modules dependency resolver, uses Environment Modules --> <!-- <modules modulecmd="/opt/Modules/3.2.9/bin/modulecmd" /> <modules modulecmd="/opt/Modules/3.2.9/bin/modulecmd" versionless="true" default_indicator="default" /> Attributes are: * modulecmd - path to modulecmd * versionless - default: false - whether to resolve tools using a version number or not * find_by - directory or avail - use the DirectoryModuleChecker or AvailModuleChecker * prefetch - default: true - in the AvailModuleChecker prefetch module info with 'module avail' * default_indicator - default: '(default)' - what indicate to the AvailModuleChecker that a module is the default version --> <!-- other resolvers <tool_shed_tap /> <homebrew /> --> </dependency_resolvers> New error message from Bowtie2 after disabling local dedependency_resolvers_conf.xml file Fatal error: Exit code 127 () Building a SMALL index Reference file does not seem to be a FASTA file Error: Encountered internal Bowtie 2 exception (#1) Command: /home/.galaxy/tooldeps/_conda/envs/mulled-v1-7576289d51ff5aefa21c8a2af901e1a61eb245f4e5fd66ae894d120dd35ff8f6/bin/bowtie2-build-s --wrapper basic-0 --threads 8 /home/.galaxy/database/files/013/dataset_13148.dat genome Deleting "genome.3.bt2" file written during aborted indexing attempt. Deleting "genome.4.bt2" file written during aborted indexing attempt. Deleting "genome.1.bt2" file written during aborted indexing attempt. Deleting "genome.2.bt2" file written during aborted indexing attempt. /home/.galaxy/database/jobs_directory/007/7436/tool_script.sh: line 41: samtools: command not found (ERR): "genome" does not exist or is not a Bowtie 2 index Exiting now ... ________________________________ From: Matthias Bernt <m.bernt@ufz.de> Sent: 06 July 2018 10:42:12 To: Van Wageningen, GERHARD [gerhardv@sun.ac.za] Cc: galaxy-dev@lists.galaxyproject.org Subject: Re: Conda problems: Failed to activate conda environment Hi Gerhard, wonderful. For the other problem: you seem to use a tool that still uses a toolshed package (python in this case) instead of the conda dependency. Can you make sure that you ensure that conda is used for dependency resolution? Otherwise, which tool are you trying to install? Cheers, Matthias On 06.07.2018 10:38, Van Wageningen, GERHARD [gerhardv@sun.ac.za] wrote:
Hi Matthias - thanx for the missing link tip - that fixes that ...
Locale as Galaxy user is
[galaxy@storage1 ~]$ locale LANG=en_US.UTF-8 LC_CTYPE="en_US.UTF-8" LC_NUMERIC="en_US.UTF-8" LC_TIME="en_US.UTF-8" LC_COLLATE="en_US.UTF-8" LC_MONETARY="en_US.UTF-8" LC_MESSAGES="en_US.UTF-8" LC_PAPER="en_US.UTF-8" LC_NAME="en_US.UTF-8" LC_ADDRESS="en_US.UTF-8" LC_TELEPHONE="en_US.UTF-8" LC_MEASUREMENT="en_US.UTF-8" LC_IDENTIFICATION="en_US.UTF-8" LC_ALL=
------------------------------------------------------------------------ *From:* Matthias Bernt <m.bernt@ufz.de> *Sent:* 06 July 2018 10:15:12 *To:* Van Wageningen, GERHARD [gerhardv@sun.ac.za] *Cc:* galaxy-dev@lists.galaxyproject.org *Subject:* Re: Conda problems: Failed to activate conda environment Dear Gerhard,
I don't know about the python error.. could you tell us the output of `locale` when logged in as the user running the Galaxy server?
For the second problem, I remember that I also have a similar problem. For some reasons some symlinks in the conda environment are not created.
you can check if in the respective environment's bin directory symlinks to the conda, activate and deactivate binaries/scripts exist.
If not you can fix the missing links (I run the following after each tool installation)
``` for env in "{{ galaxy_server_dir }}"/database/dependencies/_conda/envs/*; do for link in conda activate deactivate; do \[ -h $env/bin/$link \] || ln -s "{{ galaxy_server_dir }}"/database/dependencies/_conda/bin/$link $env/bin/$link; done done ```
Hope this helps.
Cheers, Matthias
On 05.07.2018 18:00, galaxy-dev-request@lists.galaxyproject.org wrote:
Message: 1 Date: Thu, 5 Jul 2018 08:14:35 +0000 From: "Van Wageningen, GERHARD [gerhardv@sun.ac.za]" <gerhardv@sun.ac.za> To:"galaxy-dev@lists.galaxyproject.org" <galaxy-dev@lists.galaxyproject.org> Subject: [galaxy-dev] Conda problems: Failed to activate conda environment Message-ID: <AM0PR07MB4354EEAD1AFB5809409F9F3692400@AM0PR07MB4354.eurprd07.prod.outlook.com>
Content-Type: text/plain; charset="iso-8859-1"
Hi All
I have recently started learning the Galaxy setup starting with an upgrade. Made some headway but I was wondering if someone could comment on a problem that has emerged. The system is a Galaxy 18.05 installation (upgraded from 17.01) on an Altair PBS scheduler with drmaa setup (working)
Conda problem: Failed to activate conda environment.
Conda works during tool installation or reinstallation.
Galaxy tools have been updated/reinstalled so they all look "green" in manage tools.
In Manage Dependencies there are a number of "Dependency resolved but version not found" warnings, otherwise the deps look clean.
When running tools in Galaxy errors occur, after the jobs have correctly deployed on the cluster.
The galaxy application raised errors like the two below.
"Failed to activate conda environment! Error was: Fatal Python error: Py_Initialize: Unable to get the locale encoding File "/home/.galaxy/tooldeps/python/2.7/iuc/package_python_2_7/8b09fe018cac/lib/python2.7/encodings/__init__.py", line 123 raise C"
"
Failed to activate conda environment! Error was: CondaEnvironmentError: Environment error: Cannot activate environment bash. User does not have write access for conda symlinks.
"
Galaxy started and stopped using supervisord with config below:
--
------------------------------------------- Matthias Bernt Bioinformatics Service Molekulare Systembiologie (MOLSYB) Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/ Helmholtz Centre for Environmental Research GmbH - UFZ Permoserstraße 15, 04318 Leipzig, Germany Phone +49 341 235 482296, m.bernt@ufz.de, www.ufz.de
Sitz der Gesellschaft/Registered Office: Leipzig Registergericht/Registration Office: Amtsgericht Leipzig Handelsregister Nr./Trade Register Nr.: B 4703 Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig Wilfried Kraus Wissenschaftlicher Geschäftsführer/Scientific Managing Director: Prof. Dr. Dr. h.c. Georg Teutsch Administrative Geschäftsführerin/ Administrative Managing Director: Prof. Dr. Heike Graßmann ------------------------------------------- <http://www.sun.ac.za/english/Pages/Water-crisis.aspx>
The integrity and confidentiality of this email is governed by these terms. Disclaimer <http://www.sun.ac.za/emaildisclaimer > Die integriteit en vertroulikheid van hierdie e-pos word deur die volgende bepalings gereël. Vrywaringsklousule <http://www.sun.ac.za/emaildisclaimer >
-- ------------------------------------------- Matthias Bernt Bioinformatics Service Molekulare Systembiologie (MOLSYB) Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/ Helmholtz Centre for Environmental Research GmbH - UFZ Permoserstraße 15, 04318 Leipzig, Germany Phone +49 341 235 482296, m.bernt@ufz.de, www.ufz.de Sitz der Gesellschaft/Registered Office: Leipzig Registergericht/Registration Office: Amtsgericht Leipzig Handelsregister Nr./Trade Register Nr.: B 4703 Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig Wilfried Kraus Wissenschaftlicher Geschäftsführer/Scientific Managing Director: Prof. Dr. Dr. h.c. Georg Teutsch Administrative Geschäftsführerin/ Administrative Managing Director: Prof. Dr. Heike Graßmann ------------------------------------------- [http://cdn.sun.ac.za/100/ProductionFooter.jpg]<http://www.sun.ac.za/english/Pages/Water-crisis.aspx> The integrity and confidentiality of this email is governed by these terms. Disclaimer<http://www.sun.ac.za/emaildisclaimer> Die integriteit en vertroulikheid van hierdie e-pos word deur die volgende bepalings gereël. Vrywaringsklousule<http://www.sun.ac.za/emaildisclaimer>
participants (2)
-
Matthias Bernt -
Van Wageningen, GERHARD [gerhardv＠sun.ac.za]