Creation of a backup/migrate script for data/users
Hi, I would like to create a code to backup and migrate(*) galaxy datas, based on the api and/or with the script I did here: https://github.com/remyd1/galaxy_debug_params/blob/master/source/Templates/p... To do that, I would use the galaxy model, to retrieve everything in the old one and recreate everything in the new one. However, I would like to be able to do that without my web server running. That is to say, a "cold" migration to avoid any current task to write additionnal things into the database. Obviously, that is a problem for the api because this one use the web server. Could you give some advice to do that ? Should I recreate a code from 0 ? Could I use the code in the api ? Is there a way to create a code that could use the api differently (recreate a copy of the api without the web server) ? Kind regards, Remy (*) this is important; that means you could import data from an old galaxy to a recent one (or the opposite); a simple backup of files and a dump of the db is not enough.
This is a tough question - I suspect it will be difficult no matter how you proceed. The API objects don't map 100% directly to database objects and may not have complete coverage - but using the database directly will require one to learn a lot of Galaxy's data model I suspect. A middle ground - would be to just use the Galaxy code and ORM layer without running a Galaxy server (see scripts/db_shell.py for instance). My gut tells me it is probably be easiest (least difficult anyway) to target the database directly - but this could be wrong. -John On Wed, Nov 19, 2014 at 6:21 AM, Rémy Dernat <remy.d1@gmail.com> wrote:
Hi,
I would like to create a code to backup and migrate(*) galaxy datas, based on the api and/or with the script I did here: https://github.com/remyd1/galaxy_debug_params/blob/master/source/Templates/p...
To do that, I would use the galaxy model, to retrieve everything in the old one and recreate everything in the new one.
However, I would like to be able to do that without my web server running. That is to say, a "cold" migration to avoid any current task to write additionnal things into the database. Obviously, that is a problem for the api because this one use the web server.
Could you give some advice to do that ? Should I recreate a code from 0 ? Could I use the code in the api ? Is there a way to create a code that could use the api differently (recreate a copy of the api without the web server) ?
Kind regards,
Remy
(*) this is important; that means you could import data from an old galaxy to a recent one (or the opposite); a simple backup of files and a dump of the db is not enough.
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hello, Thanks to the helpfull message of John, I am now able to backup and retrieve my users from a galaxy to another without using directly the database (by using the galaxy model and sql alchemy). I am trying to do the same thing for workflows, libraries, datasets and histories. The repository is here: https://github.com/remyd1/galaxy_backup Any interest/participation is welcome. Best Regards Remy 2014-11-21 17:50 GMT+01:00 John Chilton <jmchilton@gmail.com>:
This is a tough question - I suspect it will be difficult no matter how you proceed. The API objects don't map 100% directly to database objects and may not have complete coverage - but using the database directly will require one to learn a lot of Galaxy's data model I suspect.
A middle ground - would be to just use the Galaxy code and ORM layer without running a Galaxy server (see scripts/db_shell.py for instance).
My gut tells me it is probably be easiest (least difficult anyway) to target the database directly - but this could be wrong.
-John
Hi,
I would like to create a code to backup and migrate(*) galaxy datas,
On Wed, Nov 19, 2014 at 6:21 AM, Rémy Dernat <remy.d1@gmail.com> wrote: based
on the api and/or with the script I did here:
https://github.com/remyd1/galaxy_debug_params/blob/master/source/Templates/p...
To do that, I would use the galaxy model, to retrieve everything in the
one and recreate everything in the new one.
However, I would like to be able to do that without my web server running. That is to say, a "cold" migration to avoid any current task to write additionnal things into the database. Obviously, that is a problem for
old the
api because this one use the web server.
Could you give some advice to do that ? Should I recreate a code from 0 ? Could I use the code in the api ? Is there a way to create a code that could use the api differently (recreate a copy of the api without the web server) ?
Kind regards,
Remy
(*) this is important; that means you could import data from an old galaxy to a recent one (or the opposite); a simple backup of files and a dump of the db is not enough.
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
participants (2)
-
John Chilton -
Rémy Dernat