Problem with installing tools using Conda
galaxy_handler%(process_num)s.log
Dear Team, I have problem with the installation of tools using conda. I am getting following error message when i try to install tools from galaxy. I tried to install these tools manually with the same conda used by the galaxy and it worked. Can anyone help us to solve this issue ? Traceback (most recent call last): File "/xxxx/xxxxx/xxxx-xxxx/conda//bin/conda", line 4, in <module> from conda.cli.main import main ImportError: No module named conda.cli.main galaxy.tools.deps.resolvers.conda DEBUG 2017-03-02 17:35:49,759 Removing failed conda install of samtools, version '1.3.1' Traceback (most recent call last): File "/xxxx/xxxxx/xxxx-xxxx/conda//bin/conda", line 4, in <module> from conda.cli.main import main ImportError: No module named conda.cli.main galaxy.tools.deps.resolvers.conda DEBUG 2017-03-02 17:35:49,941 Removing failed conda install of samtools, version 'None' Traceback (most recent call last): File "/xxxx/xxxxx/xxxx-xxxx/conda//bin/conda", line 4, in <module> from conda.cli.main import main ImportError: No module named conda.cli.main galaxy.tools.deps.resolvers.conda DEBUG 2017-03-02 17:35:50,142 Removing failed conda install of bwa, version '0.7.15' Traceback (most recent call last): File "/xxxx/xxxxx/xxxx-xxxx/conda//bin/conda", line 4, in <module> from conda.cli.main import main ImportError: No module named conda.cli.main We are running galaxy-16.07 behind the proxy using nginx, uwsgi and supervisor in scientific linux 7. Below are the details of galaxy.ini and supervisor configuration. *### galaxy.ini * conda_prefix = /xxxx/xxxxx/xxxx-xxxx/conda/ #conda_exec = conda_debug = False conda_ensure_channels = r,bioconda,iuc #conda_auto_install = False #conda_verbose_install_check=False conda_auto_init = True *### supervisor* [supervisord] [program:galaxy_web] command = $GALAXY_PATH/.venv/bin/uwsgi --virtualenv $GALAXY_PATH/.venv --ini-paste $GALAXY_PATH/config/galaxy.ini --logdate --master --processes 3 --threads 6 --logto $GALAXY_PATH/logs/galaxy.uwsgi.log --socket 127.0.0.1:4001 --pythonpath lib --stats 127.0.0.1:9191 directory = $GALAXY_PATH umask = 022 autostart = true autorestart = true startsecs = 20 user = galaxy environment = PATH=$GALAXY_PATH/.venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin,PYTHONHOME=$GALAXY_PATH/.venv,DRMAA_LIBRARY_PATH=/usr/local/lib/libdrmaa.so numprocs = 1 stoplight = INT startretries = 15 process_name = web%(process_num)s [program:handler] command = $GALAXY_PATH/.venv/bin/python ./lib/galaxy/main.py -c $GALAXY_PATH/config/galaxy.ini --server-name=handler%(process_num)s --log-file=$GALAXY_PATH/logs/ directory = $GALAXY_PATH process_name = handler%(process_num)s numprocs = 3 umask = 022 autostart = true autorestart = true startsecs = 20 user = galaxy environment = PATH=$GALAXY_PATH/.venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin,PYTHONHOME=$GALAXY_PATH/.venv,DRMAA_LIBRARY_PATH=/usr/local/lib/libdrmaa.so startretries = 15 -- *Thanks and Regards,* *Ashok Vardharajan*
I've never seen anything like this before - I am sorry. Hopefully someone else will have a more concrete idea. Does that Conda executable work on the command line? I'd try removing the whole Conda folder and rebuilding it. Is it possible the install was corrupted at some point - maybe during install? The just released Galaxy 17.01 has much better logging of Conda problems and a script that can be used outside of Galaxy to explicitly setup Conda. -John On Thu, Mar 2, 2017 at 11:47 AM, Ashok Varadharajan <ashvark@gmail.com> wrote:
Dear Team,
I have problem with the installation of tools using conda. I am getting following error message when i try to install tools from galaxy. I tried to install these tools manually with the same conda used by the galaxy and it worked. Can anyone help us to solve this issue ?
Traceback (most recent call last): File "/xxxx/xxxxx/xxxx-xxxx/conda//bin/conda", line 4, in <module> from conda.cli.main import main ImportError: No module named conda.cli.main galaxy.tools.deps.resolvers.conda DEBUG 2017-03-02 17:35:49,759 Removing failed conda install of samtools, version '1.3.1' Traceback (most recent call last): File "/xxxx/xxxxx/xxxx-xxxx/conda//bin/conda", line 4, in <module> from conda.cli.main import main ImportError: No module named conda.cli.main galaxy.tools.deps.resolvers.conda DEBUG 2017-03-02 17:35:49,941 Removing failed conda install of samtools, version 'None' Traceback (most recent call last): File "/xxxx/xxxxx/xxxx-xxxx/conda//bin/conda", line 4, in <module> from conda.cli.main import main ImportError: No module named conda.cli.main galaxy.tools.deps.resolvers.conda DEBUG 2017-03-02 17:35:50,142 Removing failed conda install of bwa, version '0.7.15' Traceback (most recent call last): File "/xxxx/xxxxx/xxxx-xxxx/conda//bin/conda", line 4, in <module> from conda.cli.main import main ImportError: No module named conda.cli.main
We are running galaxy-16.07 behind the proxy using nginx, uwsgi and supervisor in scientific linux 7. Below are the details of galaxy.ini and supervisor configuration.
### galaxy.ini
conda_prefix = /xxxx/xxxxx/xxxx-xxxx/conda/ #conda_exec = conda_debug = False conda_ensure_channels = r,bioconda,iuc #conda_auto_install = False #conda_verbose_install_check=False conda_auto_init = True
### supervisor
[supervisord]
[program:galaxy_web] command = $GALAXY_PATH/.venv/bin/uwsgi --virtualenv $GALAXY_PATH/.venv --ini-paste $GALAXY_PATH/config/galaxy.ini --logdate --master --processes 3 --threads 6 --logto $GALAXY_PATH/logs/galaxy.uwsgi.log --socket 127.0.0.1:4001 --pythonpath lib --stats 127.0.0.1:9191 directory = $GALAXY_PATH umask = 022 autostart = true autorestart = true startsecs = 20 user = galaxy environment = PATH=$GALAXY_PATH/.venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin,PYTHONHOME=$GALAXY_PATH/.venv,DRMAA_LIBRARY_PATH=/usr/local/lib/libdrmaa.so numprocs = 1 stoplight = INT startretries = 15 process_name = web%(process_num)s
galaxy_handler%(process_num)s.log
[program:handler] command = $GALAXY_PATH/.venv/bin/python ./lib/galaxy/main.py -c $GALAXY_PATH/config/galaxy.ini --server-name=handler%(process_num)s --log-file=$GALAXY_PATH/logs/ directory = $GALAXY_PATH process_name = handler%(process_num)s numprocs = 3 umask = 022 autostart = true autorestart = true startsecs = 20 user = galaxy environment = PATH=$GALAXY_PATH/.venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin,PYTHONHOME=$GALAXY_PATH/.venv,DRMAA_LIBRARY_PATH=/usr/local/lib/libdrmaa.so startretries = 15
-- Thanks and Regards, Ashok Vardharajan
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hi Ashok, this seems to be a problem with your conda installation. See for example here: https://github.com/conda/conda/issues/2463 Can you update your conda manually? Cheers, Bjoern Am 02.03.2017 um 17:47 schrieb Ashok Varadharajan:
ImportError: No module named conda.cli.main
Thanks John and Bjoern for your reply. Conda is working properly when i trigger it manually as a galaxy user. (example: $conda_prefix/conda install bwa -c bioconda) I tried to remove the conda_prefix directory and it was reinstalled after restarting the galaxy. But still I am getting the same issue when installing tools from the toolshed. I also tried updating the conda using "conda update conda" but didnt help. Do i have to adjust the PYTHONPATH in my supervisor configuration ? Regards, Ashok On Thu, Mar 2, 2017 at 10:14 PM, Björn Grüning <bjoern.gruening@gmail.com> wrote:
Hi Ashok,
this seems to be a problem with your conda installation.
See for example here: https://github.com/conda/conda/issues/2463 Can you update your conda manually?
Cheers, Bjoern
Am 02.03.2017 um 17:47 schrieb Ashok Varadharajan:
ImportError: No module named conda.cli.main
-- *Thanks and Regards,* *Ashok Varadharajan*
Hi Ashok, You should not set the PYTHONPATH in supervisor. That is indeed going to conflict with conda. If you are using uwsgi you can use something like this: [program:galaxy_web] command = /bioinfo/guests/mvandenb/galaxy/.venv/bin/uwsgi --virtualenv /bioinfo/guests/mvandenb/galaxy/.venv --ini-paste /bioinfo/guests/mvandenb/gx/config/galaxy.ini --logdate --master --processes 2 --threads 2 --logto /home/galaxy/galaxy/uwsgi.log --socket 127.0.0.1:4001 --pythonpath lib --stats 127.0.0.1:9191 directory = /bioinfo/guests/mvandenb/galaxy umask = 022 autostart = true autorestart = true startsecs = 20 user = galaxy environment = /bioinfo/guests/mvandenb/galaxy/.venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin numprocs = 1 stopsignal = INT startretries = 15 Best, Marius On 3 March 2017 at 10:29, Ashok Varadharajan <ashvark@gmail.com> wrote:
Thanks John and Bjoern for your reply.
Conda is working properly when i trigger it manually as a galaxy user. (example: $conda_prefix/conda install bwa -c bioconda)
I tried to remove the conda_prefix directory and it was reinstalled after restarting the galaxy. But still I am getting the same issue when installing tools from the toolshed.
I also tried updating the conda using "conda update conda" but didnt help.
Do i have to adjust the PYTHONPATH in my supervisor configuration ?
Regards, Ashok
On Thu, Mar 2, 2017 at 10:14 PM, Björn Grüning <bjoern.gruening@gmail.com> wrote:
Hi Ashok,
this seems to be a problem with your conda installation.
See for example here: https://github.com/conda/conda/issues/2463 Can you update your conda manually?
Cheers, Bjoern
Am 02.03.2017 um 17:47 schrieb Ashok Varadharajan:
ImportError: No module named conda.cli.main
-- *Thanks and Regards,* *Ashok Varadharajan*
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
galaxy_handler%(process_num)s.log
Hi Marius, Thanks for your reply. Below is my supervisor configuration and it looks similar to yours except "PYTHONHOME" in the environment section. Do you think that would be the issue ? [supervisord] [program:galaxy_web] command = $GALAXY_PATH/.venv/bin/uwsgi --virtualenv $GALAXY_PATH/.venv --ini-paste $GALAXY_PATH/config/galaxy.ini --logdate --master --processes 3 --threads 6 --logto $GALAXY_PATH/logs/galaxy.uwsgi.log --socket 127.0.0.1:4001 --pythonpath lib --stats 127.0.0.1:9191 directory = $GALAXY_PATH umask = 022 autostart = true autorestart = true startsecs = 20 user = galaxy environment = PATH=$GALAXY_PATH/.venv/bin:/ usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin, *PYTHONHOME=$GALAXY_PATH/.venv*,DRMAA_LIBRARY_PATH=/usr/local/ lib/libdrmaa.so numprocs = 1 stoplight = INT startretries = 15 process_name = web%(process_num)s [program:handler] command = $GALAXY_PATH/.venv/bin/python ./lib/galaxy/main.py -c $GALAXY_PATH/config/galaxy.ini --server-name=handler%(process_num)s --log-file=$GALAXY_PATH/logs/ directory = $GALAXY_PATH process_name = handler%(process_num)s numprocs = 3 umask = 022 autostart = true autorestart = true startsecs = 20 user = galaxy environment = PATH=$GALAXY_PATH/.venv/bin:/ usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin, *PYTHONHOME=$GALAXY_PATH/.venv*,DRMAA_LIBRARY_PATH=/usr/local/ lib/libdrmaa.so startretries = 15 Thanks Ashok On Fri, Mar 3, 2017 at 10:37 AM, Marius van den Beek <m.vandenbeek@gmail.com
wrote:
Hi Ashok,
You should not set the PYTHONPATH in supervisor. That is indeed going to conflict with conda. If you are using uwsgi you can use something like this:
[program:galaxy_web] command = /bioinfo/guests/mvandenb/galaxy/.venv/bin/uwsgi --virtualenv /bioinfo/guests/mvandenb/galaxy/.venv --ini-paste /bioinfo/guests/mvandenb/gx/config/galaxy.ini --logdate --master --processes 2 --threads 2 --logto /home/galaxy/galaxy/uwsgi.log --socket 127.0.0.1:4001 --pythonpath lib --stats 127.0.0.1:9191 directory = /bioinfo/guests/mvandenb/galaxy umask = 022 autostart = true autorestart = true startsecs = 20 user = galaxy environment = /bioinfo/guests/mvandenb/galaxy/.venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin numprocs = 1 stopsignal = INT startretries = 15
Best, Marius
On 3 March 2017 at 10:29, Ashok Varadharajan <ashvark@gmail.com> wrote:
Thanks John and Bjoern for your reply.
Conda is working properly when i trigger it manually as a galaxy user. (example: $conda_prefix/conda install bwa -c bioconda)
I tried to remove the conda_prefix directory and it was reinstalled after restarting the galaxy. But still I am getting the same issue when installing tools from the toolshed.
I also tried updating the conda using "conda update conda" but didnt help.
Do i have to adjust the PYTHONPATH in my supervisor configuration ?
Regards, Ashok
On Thu, Mar 2, 2017 at 10:14 PM, Björn Grüning <bjoern.gruening@gmail.com
wrote:
Hi Ashok,
this seems to be a problem with your conda installation.
See for example here: https://github.com/conda/conda/issues/2463 Can you update your conda manually?
Cheers, Bjoern
Am 02.03.2017 um 17:47 schrieb Ashok Varadharajan:
ImportError: No module named conda.cli.main
-- *Thanks and Regards,* *Ashok Varadharajan*
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
-- *Thanks and Regards,* *Ashok Varadharajan*
Yep: PYTHONHOME=~/.venv conda --help Traceback (most recent call last): File "/home/marius/conda/bin/conda", line 3, in <module> from conda.cli import main ImportError: No module named conda.cli On 3 March 2017 at 11:04, Ashok Varadharajan <ashvark@gmail.com> wrote:
Hi Marius,
Thanks for your reply.
Below is my supervisor configuration and it looks similar to yours except "PYTHONHOME" in the environment section. Do you think that would be the issue ?
[supervisord]
[program:galaxy_web] command = $GALAXY_PATH/.venv/bin/uwsgi --virtualenv $GALAXY_PATH/.venv --ini-paste $GALAXY_PATH/config/galaxy.ini --logdate --master --processes 3 --threads 6 --logto $GALAXY_PATH/logs/galaxy.uwsgi.log --socket 127.0.0.1:4001 --pythonpath lib --stats 127.0.0.1:9191 directory = $GALAXY_PATH umask = 022 autostart = true autorestart = true startsecs = 20 user = galaxy environment = PATH=$GALAXY_PATH/.venv/bin:/u sr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin, *PYTHONHOME=$GALAXY_PATH/.venv*,DRMAA_LIBRARY_PATH=/usr/local/l ib/libdrmaa.so numprocs = 1 stoplight = INT startretries = 15 process_name = web%(process_num)s
galaxy_handler%(process_num)s.log
[program:handler] command = $GALAXY_PATH/.venv/bin/python ./lib/galaxy/main.py -c $GALAXY_PATH/config/galaxy.ini --server-name=handler%(process_num)s --log-file=$GALAXY_PATH/logs/ directory = $GALAXY_PATH process_name = handler%(process_num)s numprocs = 3 umask = 022 autostart = true autorestart = true startsecs = 20 user = galaxy environment = PATH=$GALAXY_PATH/.venv/bin:/u sr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin, *PYTHONHOME=$GALAXY_PATH/.venv*,DRMAA_LIBRARY_PATH=/usr/local/l ib/libdrmaa.so startretries = 15
Thanks Ashok
On Fri, Mar 3, 2017 at 10:37 AM, Marius van den Beek < m.vandenbeek@gmail.com> wrote:
Hi Ashok,
You should not set the PYTHONPATH in supervisor. That is indeed going to conflict with conda. If you are using uwsgi you can use something like this:
[program:galaxy_web] command = /bioinfo/guests/mvandenb/galaxy/.venv/bin/uwsgi --virtualenv /bioinfo/guests/mvandenb/galaxy/.venv --ini-paste /bioinfo/guests/mvandenb/gx/config/galaxy.ini --logdate --master --processes 2 --threads 2 --logto /home/galaxy/galaxy/uwsgi.log --socket 127.0.0.1:4001 --pythonpath lib --stats 127.0.0.1:9191 directory = /bioinfo/guests/mvandenb/galaxy umask = 022 autostart = true autorestart = true startsecs = 20 user = galaxy environment = /bioinfo/guests/mvandenb/galaxy/.venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin numprocs = 1 stopsignal = INT startretries = 15
Best, Marius
On 3 March 2017 at 10:29, Ashok Varadharajan <ashvark@gmail.com> wrote:
Thanks John and Bjoern for your reply.
Conda is working properly when i trigger it manually as a galaxy user. (example: $conda_prefix/conda install bwa -c bioconda)
I tried to remove the conda_prefix directory and it was reinstalled after restarting the galaxy. But still I am getting the same issue when installing tools from the toolshed.
I also tried updating the conda using "conda update conda" but didnt help.
Do i have to adjust the PYTHONPATH in my supervisor configuration ?
Regards, Ashok
On Thu, Mar 2, 2017 at 10:14 PM, Björn Grüning < bjoern.gruening@gmail.com> wrote:
Hi Ashok,
this seems to be a problem with your conda installation.
See for example here: https://github.com/conda/conda/issues/2463 Can you update your conda manually?
Cheers, Bjoern
Am 02.03.2017 um 17:47 schrieb Ashok Varadharajan:
ImportError: No module named conda.cli.main
-- *Thanks and Regards,* *Ashok Varadharajan*
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
-- *Thanks and Regards,* *Ashok Varadharajan*
participants (4)
-
Ashok Varadharajan -
Björn Grüning -
John Chilton -
Marius van den Beek