Am 10.07.2018 um 08:54 schrieb Van Wageningen, GERHARD [gerhardv@sun.ac.za]:
> So those files are "just not there" due to a faulty install?

Seems so.
You can try for hisat as well.
The questions is why :(

> And the
> repair is to delete the __toolname.x.x.x folders from the _conda/envs
> folder and force conda to reinstall them?


Yes.

>
> Where can I look for the channel priorities and is there a standard rule
> for them?

https://github.com/galaxyproject/galaxy/blob/dev/config/galaxy.yml.sample#L240

Cheers,
Bjoern

>
> Thanks for your input - must appreciated!
>
>
> gerhard
>
> ------------------------------------------------------------------------
> *From:* Björn Grüning <bjoern.gruening@gmail.com>
> *Sent:* 10 July 2018 08:49:04
> *To:* Van Wageningen, GERHARD [gerhardv@sun.ac.za];
> galaxy-dev@lists.galaxyproject.org
> *Subject:* Re: [galaxy-dev] Conda problems: Failed to activate conda
> environment
> Which means that your conda envs are not what they are supposed to be.
>
> You can fix this by removing the folder and let Galaxy re-created them,
> but you should figure out why they are "empty". Maybe a wrong channel
> priority or network issues during installation?
>
> Hope that helps.
> You could also from insite the env, do a `conda install ...`
>
>> Hi - tx for (fast) reply.  Ok ... with this result ...
>>
>>
>> (/home/.galaxy/tooldeps/_conda/envs/__trim-galore@0.4.3) [root@storage1
>> /home/.galaxy/tooldeps/_conda/envs]$ trim_galore --version
>> -bash: trim_galore: command not found
>>
>> Environment is:
>>
>> (/home/.galaxy/tooldeps/_conda/envs/__trim-galore@0.4.3) [root@storage1
>> /home/.galaxy/tooldeps/_conda/envs]$ env
>> XDG_SESSION_ID=45352
>> HOSTNAME=storage1.hpc
>> SELINUX_ROLE_REQUESTED=
>> TERM=xterm-256color
>> SHELL=/bin/bash
>> HISTSIZE=1000
>> SSH_CLIENT=10.55.40.3 55438 22
>> PERL5LIB=/root/perl5/lib/perl5:/usr/local/lib/perl5:
>> SELINUX_USE_CURRENT_RANGE=
>> QTDIR=/usr/lib64/qt-3.3
>> OLDPWD=/home/.galaxy/tooldeps/_conda/envs/__hisat2@2.1.0
>> QTINC=/usr/lib64/qt-3.3/include
>> PERL_MB_OPT=--install_base /root/perl5
>> SSH_TTY=/dev/pts/2
>> QT_GRAPHICSSYSTEM_CHECKED=1
>> USER=root
>> LS_COLORS=rs=0:di=38;5;27:ln=38;5;51:mh=44;38;5;15:pi=40;38;5;11:so=38;5;13:do=38;5;5:bd=48;5;232;38;5;11:cd=48;5;232;38;5;3:or=48;5;232;38;5;9:mi=05;48;5;232;38;5;15:su=48;5;196;38;5;15:sg=48;5;11;38;5;16:ca=48;5;196;38;5;226:tw=48;5;10;38;5;16:ow=48;5;10;38;5;21:st=48;5;21;38;5;15:ex=38;5;34:*.tar=38;5;9:*.tgz=38;5;9:*.arc=38;5;9:*.arj=38;5;9:*.taz=38;5;9:*.lha=38;5;9:*.lz4=38;5;9:*.lzh=38;5;9:*.lzma=38;5;9:*.tlz=38;5;9:*.txz=38;5;9:*.tzo=38;5;9:*.t7z=38;5;9:*.zip=38;5;9:*.z=38;5;9:*.Z=38;5;9:*.dz=38;5;9:*.gz=38;5;9:*.lrz=38;5;9:*.lz=38;5;9:*.lzo=38;5;9:*.xz=38;5;9:*.bz2=38;5;9:*.bz=38;5;9:*.tbz=38;5;9:*.tbz2=38;5;9:*.tz=38;5;9:*.deb=38;5;9:*.rpm=38;5;9:*.jar=38;5;9:*.war=38;5;9:*.ear=38;5;9:*.sar=38;5;9:*.rar=38;5;9:*.alz=38;5;9:*.ace=38;5;9:*.zoo=38;5;9:*.cpio=38;5;9:*.7z=38;5;9:*.rz=38;5;9:*.cab=38;5;9:*.jpg=38;5;13:*.jpeg=38;5;13:*.gif=38;5;13:*.bmp=38;5;13:*.pbm=38;5;13:*.pgm=38;5;13:*.ppm=38;5;13:*.tga=38;5;13:*.xbm=38;5;13:*.xpm=38;5;13:*.tif=38;5;13:*.tiff=38;5;13:*.png=38;5;13:*.svg=38;5;13:*.svgz=38;5;13:*.mng=38;5;13:*.pcx=38;5;13:*.mov=38;5;13:*.mpg=38;5;13:*.mpeg=38;5;13:*.m2v=38;5;13:*.mkv=38;5;13:*.webm=38;5;13:*.ogm=38;5;13:*.mp4=38;5;13:*.m4v=38;5;13:*.mp4v=38;5;13:*.vob=38;5;13:*.qt=38;5;13:*.nuv=38;5;13:*.wmv=38;5;13:*.asf=38;5;13:*.rm=38;5;13:*.rmvb=38;5;13:*.flc=38;5;13:*.avi=38;5;13:*.fli=38;5;13:*.flv=38;5;13:*.gl=38;5;13:*.dl=38;5;13:*.xcf=38;5;13:*.xwd=38;5;13:*.yuv=38;5;13:*.cgm=38;5;13:*.emf=38;5;13:*.axv=38;5;13:*.anx=38;5;13:*.ogv=38;5;13:*.ogx=38;5;13:*.aac=38;5;45:*.au=38;5;45:*.flac=38;5;45:*.mid=38;5;45:*.midi=38;5;45:*.mka=38;5;45:*.mp3=38;5;45:*.mpc=38;5;45:*.ogg=38;5;45:*.ra=38;5;45:*.wav=38;5;45:*.axa=38;5;45:*.oga=38;5;45:*.spx=38;5;45:*.xspf=38;5;45:
>> MAIL=/var/spool/mail/root
>> PATH=/home/.galaxy/tooldeps/_conda/envs/__trim-galore@0.4.3/bin:/usr/lib64/qt-3.3/bin:/root/perl5/bin:/opt/pbs/default/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/opt/puppetlabs/bin:/root/bin
>> CONDA_PATH_BACKUP=/usr/lib64/qt-3.3/bin:/root/perl5/bin:/opt/pbs/default/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/opt/puppetlabs/bin:/root/bin
>> CONDA_PREFIX=/home/.galaxy/tooldeps/_conda/envs/__trim-galore@0.4.3
>> CONDA=/home/.galaxy/tooldeps/_conda
>> PWD=/home/.galaxy/tooldeps/_conda/envs
>> LANG=en_ZA.UTF-8
>> MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/apps/etc/modulefiles
>> LOADEDMODULES=
>> KDEDIRS=/usr
>> PS1=(/home/.galaxy/tooldeps/_conda/envs/__trim-galore@0.4.3)
>> \[\e[1;31;40m\][\u@\h \w]$\[\e[0m\]
>> SELINUX_LEVEL_REQUESTED=
>> HISTCONTROL=ignoredups
>> SHLVL=1
>> HOME=/root
>> CONDA_PS1_BACKUP=\[\e[1;31;40m\][\u@\h \w]$\[\e[0m\]
>> PERL_LOCAL_LIB_ROOT=:/root/perl5
>> LOGNAME=root
>> QTLIB=/usr/lib64/qt-3.3/lib
>> SSH_CONNECTION=10.55.40.3 55438 10.55.40.1 22
>> MODULESHOME=/usr/share/Modules
>> CONDA_DEFAULT_ENV=/home/.galaxy/tooldeps/_conda/envs/__trim-galore@0.4.3
>> LESSOPEN=||/usr/bin/lesspipe.sh %s
>> XDG_RUNTIME_DIR=/run/user/0
>> QT_PLUGIN_PATH=/usr/lib64/kde4/plugins:/usr/lib/kde4/plugins
>> PERL_MM_OPT=INSTALL_BASE=/root/perl5
>> XAUTHORITY=/tmp/.Xauthority.root
>> BASH_FUNC_module()=() {  eval `/usr/bin/modulecmd bash $*`
>> }
>> _=/usr/bin/env
>> (/home/.galaxy/tooldeps/_conda/envs/__trim-galore@0.4.3) [root@storage1
>> /home/.galaxy/tooldeps/_conda/envs]$
>>
>>
>> ------------------------------------------------------------------------
>> *From:* Björn Grüning <bjoern.gruening@gmail.com>
>> *Sent:* 10 July 2018 08:38:15
>> *To:* Van Wageningen, GERHARD [gerhardv@sun.ac.za];
>> galaxy-dev@lists.galaxyproject.org
>> *Subject:* Re: [galaxy-dev] Conda problems: Failed to activate conda
>> environment
>> Hi,
>>
>> can you do:
>>
>>    . /home/.galaxy/tooldeps/_conda/bin/activate
>> '/home/.galaxy/tooldeps/_conda/envs/__trim-galore@0.4.3
>>
>> And see if you can execute trim_galore from the commandline and get the
>> help?
>>
>> Ciao,
>> Bjoern
>>
>> Am 10.07.2018 um 08:34 schrieb Van Wageningen, GERHARD [gerhardv@sun.ac.za]:
>>> Hi Bjoern, thanks for the pointer ... We are running the default
>>> dependency_resolvers_conf.xml now and that, along with the symlinks hint
>>> has helped with a lot of the tool dep. issues.  But there is still a
>>> pathing issue we are stuck on.
>>>
>>> Take for example the trim_galore tool.
>>>
>>> The error is
>>>
>>> /home/.galaxy/database/jobs_directory/007/7480/tool_script.sh: line 25:
>>> trim_galore: command not found
>>>
>>> And the script is:
>>>
>>> #!/bin/bash
>>>
>>> # The following block can be used by the job system
>>> # to ensure this script is runnable before actually attempting
>>> # to run it.
>>> if [ -n "$ABC_TEST_JOB_SCRIPT_INTEGRITY_XYZ" ]; then
>>>      exit 42
>>> fi
>>> [ "$CONDA_DEFAULT_ENV" =
>>> "/home/.galaxy/tooldeps/_conda/envs/__trim-galore@0.4.3" ] ||
>>> MAX_TRIES=3
>>> COUNT=0
>>> while [ $COUNT -lt $MAX_TRIES ]; do
>>>      . /home/.galaxy/tooldeps/_conda/bin/activate
>>> '/home/.galaxy/tooldeps/_conda/envs/__trim-galore@0.4.3' >
>>> conda_activate.log 2>&1
>>>      if [ $? -eq 0 ];then
>>>          break
>>>      else
>>>          let COUNT=COUNT+1
>>>          if [ $COUNT -eq $MAX_TRIES ];then
>>>              echo "Failed to activate conda environment! Error was:"
>>>              cat conda_activate.log
>>>              exit 1
>>>          fi
>>>          sleep 10s
>>>      fi
>>> done ; trim_galore --version >
>>> /scratch2/galaxy/tmp/GALAXY_VERSION_STRING_7477 2>&1; ln -s
>>> '/home/.galaxy/database/files/013/dataset_13139.dat' input_1.fastq && 
>>> trim_galore  --phred33    --output_dir ./      input_1.fastq 
>>> --dont_gzip  && if [ -f input_1_trimmed.fq.gz ] ; then mv
>>> input_1_trimmed.fq.gz input_1_trimmed.fq ; fi && if [ -f
>>> input_1_val_1.fq.gz ] ; then mv input_1_val_1.fq.gz input_1_val_1.fq ;
>>> fi && if [ -f input_2_val_2.fq.gz ] ; then mv input_2_val_2.fq.gz
>>> input_2_val_2.fq ; fi && if [ -f input_1_unpaired_1.fq.gz ] ; then mv
>>> input_1_unpaired_1.fq.gz input_1_unpaired_1.fq ; fi && if [ -f
>>> input_2_unpaired_2.fq.gz ] ; then mv input_2_unpaired_2.fq.gz
>>> input_2_unpaired_2.fq ; fi
>>>
>>>
>>>
>>> In fact most of the tools do this post-upgrade - where they cannot find
>>> a file.
>>>
>>> Here is the trace from hisat2 - it fails in a similar way so we conclude
>>> that there is a common underlying fault here that is affecting the
>>> pathing of the scripts.
>>>
>>> Any ideas how we should go about finding this error?
>>>
>>> Thanks, Gerhard
>>>
>>> PS Here is the hisat2 output.
>>>
>>> #!/bin/bash
>>>
>>> # The following block can be used by the job system
>>>
>>> # to ensure this script is runnable before actually attempting
>>>
>>> # to run it.
>>>
>>> if [ -n "$ABC_TEST_JOB_SCRIPT_INTEGRITY_XYZ" ]; then
>>>
>>>      exit 42
>>>
>>> fi
>>>
>>> [ "$CONDA_DEFAULT_ENV" =
>>> "/home/.galaxy/tooldeps/_conda/envs/mulled-v1-3b104c294f65450b09ba89d24826c61eddd553d789c9f1ae48a29715de1b7426"
>>> ] ||
>>>
>>> MAX_TRIES=3
>>>
>>> COUNT=0
>>>
>>> while [ $COUNT -lt $MAX_TRIES ]; do
>>>
>>>      . /home/.galaxy/tooldeps/_conda/bin/activate
>>> '/home/.galaxy/tooldeps/_conda/envs/mulled-v1-3b104c294f65450b09ba89d24826c61eddd553d789c9f1ae48a29715de1b7426'
>>>  > conda_activate.log 2>&1
>>>
>>>      if [ $? -eq 0 ];then
>>>
>>>          break
>>>
>>>      else
>>>
>>>          let COUNT=COUNT+1
>>>
>>>          if [ $COUNT -eq $MAX_TRIES ];then
>>>
>>>              echo "Failed to activate conda environment! Error was:"
>>>
>>>              cat conda_activate.log
>>>
>>>              exit 1
>>>
>>>          fi
>>>
>>>          sleep 10s
>>>
>>>      fi
>>>
>>> done ; [ "$CONDA_DEFAULT_ENV" =
>>> "/home/.galaxy/tooldeps/_conda/envs/mulled-v1-3b104c294f65450b09ba89d24826c61eddd553d789c9f1ae48a29715de1b7426"
>>> ] ||
>>>
>>> MAX_TRIES=3
>>>
>>> COUNT=0
>>>
>>> while [ $COUNT -lt $MAX_TRIES ]; do
>>>
>>>      . /home/.galaxy/tooldeps/_conda/bin/activate
>>> '/home/.galaxy/tooldeps/_conda/envs/mulled-v1-3b104c294f65450b09ba89d24826c61eddd553d789c9f1ae48a29715de1b7426'
>>>  > conda_activate.log 2>&1
>>>
>>>      if [ $? -eq 0 ];then
>>>
>>>          break
>>>
>>>      else
>>>
>>>          let COUNT=COUNT+1
>>>
>>>          if [ $COUNT -eq $MAX_TRIES ];then
>>>
>>>              echo "Failed to activate conda environment! Error was:"
>>>
>>>              cat conda_activate.log
>>>
>>>              exit 1
>>>
>>>          fi
>>>
>>>          sleep 10s
>>>
>>>      fi
>>>
>>> done ; hisat2 --version >
>>> /scratch2/galaxy/tmp/GALAXY_VERSION_STRING_7484 2>&1; set -o | grep -q
>>> pipefail && set -o pipefail;   ln -s
>>> '/home/.galaxy/database/files/013/dataset_13248.dat' genome.fa &&
>>> _hisat2-build_ -p ${GALAXY_SLOTS:-1} genome.fa genome &&        ln -f -s
>>> '/home/.galaxy/database/files/013/dataset_13238.dat' input_f.fastq && 
>>> ln -f -s '/home/.galaxy/database/files/013/dataset_13243.dat'
>>> input_r.fastq &&     hisat2  -p ${GALAXY_SLOTS:-1}  -x 'genome'     -1
>>> 'input_f.fastq' -2 'input_r.fastq'                               |
>>> samtools sort - -@ ${GALAXY_SLOTS:-1} -l 6 -o
>>> '/home/.galaxy/database/jobs_directory/007/7484/galaxy_dataset_19227.dat'
>>>
>>> The job fails when it cannot find “hisat2-build”.
>>>
>>> Fatal error: Exit code 127 ()
>>>
>>> /home/.galaxy/database/jobs_directory/007/7484/tool_script.sh: line 41:
>>> hisat2-build: command not found
>>>
>>> The PATH variable should be exported with installation
>>> (/home/.galaxy/tooldeps/hisat/2.0.3/iuc/package_hisat_2_0_3/dc03603edd43/env.sh):
>>>
>>> PATH=/home/.galaxy/tooldeps/hisat/2.0.3/iuc/package_hisat_2_0_3/dc03603edd43/bin:$PATH;
>>> export PATH
>>>
>>> HISAT2_ROOT_DIR=/home/.galaxy/tooldeps/hisat/2.0.3/iuc/package_hisat_2_0_3/dc03603edd43;
>>> export HISAT2_ROOT_DIR
>>>
>>> HISAT_ROOT_DIR=/home/.galaxy/tooldeps/hisat/2.0.3/iuc/package_hisat_2_0_3/dc03603edd43;
>>> export HISAT_ROOT_DIR
>>>
>>> The conda environment with dependencies seems to be there:
>>>
>>> /home/.galaxy/tooldeps/_conda/envs/mulled-v1-3b104c294f65450b09ba89d24826c61eddd553d789c9f1ae48a29715de1b7426/bin
>>>
>>>
>>>
>>>
>>> Björn Grüning <bjoern.gruening@gmail.com>;
>>> galaxy-dev@lists.galaxyproject.org
>>>
>>> ------------------------------------------------------------------------
>>> *From:* Björn Grüning <bjoern.gruening@gmail.com>
>>> *Sent:* 05 July 2018 22:00:58
>>> *To:* Van Wageningen, GERHARD [gerhardv@sun.ac.za];
>>> galaxy-dev@lists.galaxyproject.org
>>> *Subject:* Re: [galaxy-dev] Conda problems: Failed to activate conda
>>> environment
>>> Hi Gerhard,
>>>
>>> it seems your tool is picking up the old tool_dependency version and not
>>> the conda version of python.
>>>
>>> Have a look at
>>> https://github.com/galaxyproject/galaxy/blob/dev/config/dependency_resolvers_conf.xml.sample
>
>>
>>>
>>> <https://github.com/galaxyproject/galaxy/blob/dev/config/dependency_resolvers_conf.xml.sample>
>>>       
>>> galaxyproject/galaxy
>>> <https://github.com/galaxyproject/galaxy/blob/dev/config/dependency_resolvers_conf.xml.sample>
>>> github.com
>>> galaxy - Data intensive science for everyone.
>>>
>>>
>>>
>>> to configure the order of dependency resolution in Galaxy.
>>>
>>> Cheers,
>>> Bjoern
>>>
>>>> Hi All
>>>>
>>>> I have recently started learning the Galaxy setup starting with an upgrade.
>>>> Made some headway but I was wondering if someone could comment on a
>>>> problem that has emerged.
>>>> The system is a Galaxy 18.05 installation (upgraded from 17.01) on an
>>>> Altair PBS scheduler with drmaa setup (working)
>>>>
>>>> Conda problem: Failed to activate conda environment.
>>>>
>>>> Conda works during tool installation or reinstallation.
>>>>
>>>> Galaxy tools have been updated/reinstalled so they all look "green" in
>>>> manage tools.
>>>>
>>>> In Manage Dependencies there are a number of "Dependency resolved but
>>>> version not found" warnings, otherwise the deps look clean.
>>>>
>>>> When running tools in Galaxy errors occur, after the jobs have correctly
>>>> deployed on the cluster.
>>>>
>>>>
>>>> The galaxy application raised errors like the two below.
>>>>
>>>> "Failed to activate conda environment! Error was:
>>>> Fatal Python error: Py_Initialize: Unable to get the locale encoding
>>>>    File
>>>> "/home/.galaxy/tooldeps/python/2.7/iuc/package_python_2_7/8b09fe018cac/lib/python2.7/encodings/__init__.py",
>>>> line 123
>>>>      raise C"
>>>>
>>>> "
>>>>
>>>> Failed to activate conda environment! Error was: CondaEnvironmentError:
>>>> Environment error: Cannot activate environment bash. User does not have
>>>> write access for conda symlinks.
>>>>
>>>>
>>>> "
>>>>
>>>>
>>>> Galaxy started and stopped using supervisord with config below:
>>>>
>>>>
>>>> Thanks, hopefully someone has seen this before ...
>>>>
>>>>
>>>> regards, Gerhard
>>>>
>>>>
>>>>
>>>> [program:web]
>>>> command         = uwsgi --plugin python --virtualenv /home/.galaxy/.venv
>>>> --ini-paste /home/.galaxy/config/galaxy.ini
>>>> directory       = /home/.galaxy
>>>> umask           = 022
>>>> autostart       = true
>>>> autorestart     = true
>>>> startsecs       = 10
>>>> user            = galaxy
>>>> numprocs        = 1
>>>> stopsignal      = INT
>>>>
>>>> [program:handler]
>>>> command         = python ./scripts/galaxy-main -c
>>>> /home/.galaxy/config/galaxy.ini --server-name=handler%(process_num)s
>>>> directory       = /home/.galaxy
>>>> process_name    = handler%(process_num)s
>>>> numprocs        = 1
>>>> umask           = 022
>>>> autostart       = true
>>>> autorestart     = true
>>>> startsecs       = 10
>>>> user            = galaxy
>>>> environment     =
>>>> VIRTUAL_ENV="/home/.galaxy/.venv",PATH="/home/.galaxy/.venv/bin:%(ENV_PATH)s",http_proxy="http://storage2.eth.hpc:3128/",https_proxy="http://storage2.eth.hpc:3128/"
>>>> stdout_logfile  = /home/.galaxy/handler%(process_num)s.log
>>>> redirect_stderr = true
>>>>
>>>> [group:galaxy]
>>>> programs        = handler, web
>>>>
>>>> <http://www.sun.ac.za/english/Pages/Water-crisis.aspx>
>>>>
>>>> The integrity and confidentiality of this email is governed by these
>>>> terms. Disclaimer <http://www.sun.ac.za/emaildisclaimer >
>>>> Die integriteit en vertroulikheid van hierdie e-pos word deur die
>>>> volgende bepalings gereël. Vrywaringsklousule
>>>> <http://www.sun.ac.za/emaildisclaimer >
>>>>
>>>>
>>>> ___________________________________________________________
>>>> Please keep all replies on the list by using "reply all"
>>>> in your mail client.  To manage your subscriptions to this
>>>> and other Galaxy lists, please use the interface at:
>>>>    https://lists.galaxyproject.org/
>>>>
>>>> To search Galaxy mailing lists use the unified search at:
>>>>    http://galaxyproject.org/search/
>>>>
>>> <http://www.sun.ac.za/english/Pages/Water-crisis.aspx>
>>>
>>> The integrity and confidentiality of this email is governed by these
>>> terms. Disclaimer <http://www.sun.ac.za/emaildisclaimer >
>>> Die integriteit en vertroulikheid van hierdie e-pos word deur die
>>> volgende bepalings gereël. Vrywaringsklousule
>>> <http://www.sun.ac.za/emaildisclaimer >
>> <http://www.sun.ac.za/english/Pages/Water-crisis.aspx>
>>
>> The integrity and confidentiality of this email is governed by these
>> terms. Disclaimer <http://www.sun.ac.za/emaildisclaimer >
>> Die integriteit en vertroulikheid van hierdie e-pos word deur die
>> volgende bepalings gereël. Vrywaringsklousule
>> <http://www.sun.ac.za/emaildisclaimer >
> <http://www.sun.ac.za/english/Pages/Water-crisis.aspx>
>
> The integrity and confidentiality of this email is governed by these
> terms. Disclaimer <http://www.sun.ac.za/emaildisclaimer >
> Die integriteit en vertroulikheid van hierdie e-pos word deur die
> volgende bepalings gereël. Vrywaringsklousule
> <http://www.sun.ac.za/emaildisclaimer >