I wrote something in Python a while back (which I've attached) to parse a shed_tool_conf.xml file and generate a shell script to install those tools on a different server. The shell script uses Galaxy's scripts/api/install_tool_shed_repositories.py to install the tools and ends up looking something like this:


#!/bin/sh
set -eux

python ./scripts/api/install_tool_shed_repositories.py --local $GALAXY_INSTALL_URL --api $GALAXY_INSTALL_KEY --tool-deps --repository-deps --url https://toolshed.g2.bx.psu.edu --owner devteam --name fasta_to_tabular --revision 9d189d08f2ad --panel-section-id convert
python ./scripts/api/install_tool_shed_repositories.py --local $GALAXY_INSTALL_URL --api $GALAXY_INSTALL_KEY --tool-deps --repository-deps --url https://toolshed.g2.bx.psu.edu --owner devteam --name tabular_to_fasta --revision 0b4e36026794 --panel-section-id convert
... etc


All you have to do is set those two environment variables appropriately and run the shell script from your Galaxy root directory. I did end up having to modify our nginx config to increase the proxy read timeout for the /api location to a much larger value (I used 3600s) to give the tool dependencies enough time to build, though.

Hope you find it useful.


Cheers,

Brian


On Wed, Nov 23, 2016 at 7:32 AM, Peter Briggs <peter.briggs@manchester.ac.uk> wrote:
Hi Peter

Thanks for your comments - at least it doesn't sound like I'm missing the "perfect" way to migrate the installed tools. In the longer term moving towards bioconda dependency resolution will probably be more stable.

Also re breaking tool repos: even repos that aren't updated on the toolshed can break over time, for example if the URL for an executable moves, or if an implicit Python dependency is updated and breaks an install (these are both things that I'm seeing in my tests).

Thanks again

Best wishes

Peter


On 23/11/16 11:40, Peter Cock wrote:
On Wed, Nov 23, 2016 at 10:49 AM, Peter Briggs
<peter.briggs@manchester.ac.uk> wrote:
Hello Evan, Hans-Rudolf

I'm just in the middle of doing a similar migration for our local production
server, and Hans-Rudolf's advice seems sound to me. Definitely moving the
core Galaxy server has been relatively straightforward.

However: for the installed tools, I'm not sure that changing the paths in
the env.sh files is sufficient - in our installation the absolute paths
seemed to be baked into a lot of other files under the 'tool_dependencies'
directory - including things like compiled files (e.g. static and shared
libraries). So for many of the tools I wouldn't feel confident that they
would still work after the move.

Yes, I've seen that with various third party tool installations (out side of
Galaxy), so sadly you cannot in general move the files to a different path :(

I don't know if we can do anything about this directly in Galaxy, or even
in BioConda?

My plan has been to reinstall each of the tools from the toolshed (i.e.
uninstall via the admin interface then reinstall the same tool repository
revision(s) using the API), but I don't feel able to recommend this approach
either as in my testing this has also had problems - ranging from some tool
revisions no longer being available, through to more serious issues (such as
tool dependencies which used to work but since become broken). I figured I'd
just have to knuckle down and work through each problem as I encountered it.

One simple reason for this is stale URLs, where Galaxy's cache can help
but is another step for tool wrapper authors to do when setting up a new
Galaxy dependency: https://github.com/galaxyproject/cargo-port

However, even well intentioned Tool Shed updates could also break things :(

If anyone else has experiences with these kinds of migrations then I'm also
very interested to know what worked (and what didn't)!

Sorry - the closest I've been is setting up a new Galaxy server in
parallel with our old server, and manually installing "missing" tools
via the Tool Shed.

Peter


--
Peter Briggs peter.briggs@manchester.ac.uk
Bioinformatics Core Facility University of Manchester
B.1083 Michael Smith Bldg Tel: (0161) 2751482
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/mailinglists/



--
Brian Claywell | programmer/analyst
Matsen Group   | matsen.fredhutch.org
Fred Hutchinson Cancer Research Center