Hi Peter,

Recent galaxy releases are using the `hg` command that should be automatically installed along with other galaxy dependencies. If you're running galaxy in a virtualenv then that virtualenv should have the `hg` script in the bin folder. Depending on how you start galaxy you may need to add the virtualenv's `bin` folder to the `PATH`.

Hope that helps, Marius

On 25 June 2018 at 07:51, Peter Briggs peter.briggs@manchester.ac.uk wrote:

Dear devs

I've encountered what appears to be a subtle bug with release 18.05, which breaks the installation of tools from the toolshed, and appears to be a result of not having mercurial (hg) available in /usr/bin on the system that Galaxy is installed on (in this case Scientific Linux 6.5).

When attempting to install a tool (e.g. devteam/fastqc) from the main toolshed via the admin interface, after clicking "install" the tool installation status goes immediately to "Error". The tool repository isn't cloned to "shed_tools" and no dependencies are installed.

I've been unable to find any error messages in the logs. However, attempting to install via the API does return the message:

Error cloning repository: [Errno 2] No such file or directory

which comes from the "clone_repository" function in lib/tool_shed/util/hg_util.py (when something goes wrong with the "hg clone ..." command).

Installing Mercurial 1.3 via yum on the server and attempting tool installation again gives a slightly different error via the API:

Error cloning repository: Command '['hg', 'clone', '-r', u'17', u' https://toolshed.g2.bx.psu.edu/repos/devteam/fastqc', u'/XXXXXXXXXXXXXX/shed_tools/toolshed.g2.bx.psu.edu/repos/ devteam/fastqc/c15237684a01/fastqc']' returned non-zero exit status 255 Output was: abort: No such file or directory: /XXXXXXXXXXXXXXXXXX/shed_tools/ toolshed.g2.bx.psu.edu/repos/devteam/fastqc/c15237684a01/fastqc

Uninstalling the system Mercurial and instead installing version 3.7.3 and making a link from /usr/bin/hg seems to fix the problem, and tools can be installed without problems.

I couldn't find any evidence of this being reported before, and I don't know if I've missed some configuration detail which means that Galaxy isn't picking up hg from its virtualenv instead of /usr/bin.

Has anyone else encountered this problem? and is there a fix/workaround for it (other than horrible links to /usr/bin/hg)?

Any advice gratefully received!

Best wishes

Peter

-- Peter Briggs peter.briggs@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482

---

Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/

Hi all,

Just wanted to append my experience with this issue, since I just recently was up against the same thing and spent a great deal of time on it. In my case, I was attempting to use conda for framework dependencies. Hg was properly installed inside the conda env, but was not being seen during tool shed installation. Since there was no error, it was difficult to determine what to do next. I didn’t have the good idea of trying to install a tool shed tool via API, which it sounds like was what gave Peter the necessary info to properly debug this. So, I went ahead and added the galaxy conda env to my PATH, a non-optimal temporary fix. I’ll probably return to using virtualenvs to prevent other mysterious issues from arising.

Thanks, John

From: galaxy-dev galaxy-dev-bounces@lists.galaxyproject.org on behalf of Peter Briggs peter.briggs@manchester.ac.uk Date: Tuesday, June 26, 2018 at 5:13 AM To: Marius van den Beek m.vandenbeek@gmail.com Cc: "galaxy-dev@lists.galaxyproject.org" galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Unable to install tools from toolshed/mercurial issue in release_18.05 instance

Hello Marius

Thanks for the reply. This installation is using supervisor with uwsgi to run Galaxy using a virtualenv, and I did wonder why my install wasn't picking up Mercurial from there.

Based on your suggestion it turned out that I needed to add an 'environment' parameter like e.g.:

environment = VIRTUAL_ENV="/srv/galaxy/venv",PATH="/srv/galaxy/venv/bin:%(ENV_PATH)s" to the supervisor .ini file in order to get the virtualenv on the PATH.

(I thought I'd tried this before without success but it works now, so I must have done something wrong previously - possibly a type in a path.)

(Also, a comment re the documentation: in the "Scaling and Load Balancing" section of the docs, the 'environment' parameter is only mentioned for handler processes and doesn't appear in the example config for web processes:

https://docs.galaxyproject.org/en/master/admin/scaling.html#supervisord

and since I don't have any handlers (I'm using the uwsgi + mules strategy) it didn't occur to me that I would need it. It might be useful if the docs clarified that 'environment' is also required for the web processes in this case.)

Thanks again for your help,

Best wishes

Peter

-- Peter Briggs peter.briggs@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482

________________________________ From: Marius van den Beek [m.vandenbeek@gmail.com] Sent: Monday, June 25, 2018 5:30 PM To: Peter Briggs Cc: galaxy-dev@lists.galaxyproject.org Subject: Re: [galaxy-dev] Unable to install tools from toolshed/mercurial issue in release_18.05 instance Hi Peter,

Recent galaxy releases are using the `hg` command that should be automatically installed along with other galaxy dependencies. If you're running galaxy in a virtualenv then that virtualenv should have the `hg` script in the bin folder. Depending on how you start galaxy you may need to add the virtualenv's `bin` folder to the `PATH`.

Hope that helps, Marius

On 25 June 2018 at 07:51, Peter Briggs <peter.briggs@manchester.ac.ukmailto:peter.briggs@manchester.ac.uk> wrote: Dear devs

I've encountered what appears to be a subtle bug with release 18.05, which breaks the installation of tools from the toolshed, and appears to be a result of not having mercurial (hg) available in /usr/bin on the system that Galaxy is installed on (in this case Scientific Linux 6.5).

When attempting to install a tool (e.g. devteam/fastqc) from the main toolshed via the admin interface, after clicking "install" the tool installation status goes immediately to "Error". The tool repository isn't cloned to "shed_tools" and no dependencies are installed.

I've been unable to find any error messages in the logs. However, attempting to install via the API does return the message:

Error cloning repository: [Errno 2] No such file or directory

which comes from the "clone_repository" function in lib/tool_shed/util/hg_util.py (when something goes wrong with the "hg clone ..." command).

Installing Mercurial 1.3 via yum on the server and attempting tool installation again gives a slightly different error via the API:

Error cloning repository: Command '['hg', 'clone', '-r', u'17', u'https://toolshed.g2.bx.psu.edu/repos/devteam/fastqc', u'/XXXXXXXXXXXXXX/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/fastqc/c15237684a01/fastqc'http://toolshed.g2.bx.psu.edu/repos/devteam/fastqc/c15237684a01/fastqc']' returned non-zero exit status 255 Output was: abort: No such file or directory: /XXXXXXXXXXXXXXXXXX/shed_tools/toolshed.g2.bx.psu.edu/repos/devteam/fastqc/c15237684a01/fastqchttp://toolshed.g2.bx.psu.edu/repos/devteam/fastqc/c15237684a01/fastqc

Uninstalling the system Mercurial and instead installing version 3.7.3 and making a link from /usr/bin/hg seems to fix the problem, and tools can be installed without problems.

I couldn't find any evidence of this being reported before, and I don't know if I've missed some configuration detail which means that Galaxy isn't picking up hg from its virtualenv instead of /usr/bin.

Has anyone else encountered this problem? and is there a fix/workaround for it (other than horrible links to /usr/bin/hg)?

Any advice gratefully received!

Best wishes

Peter

-- Peter Briggs peter.briggs@manchester.ac.ukmailto:peter.briggs@manchester.ac.uk Bioinformatics Core Facility University of Manchester B.1083 Michael Smith Bldg Tel: (0161) 2751482

___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/