cleanup_datasets.py with postgresql 8.1
Posting to galaxy-dev@bx.psu.edu to give the question better exposure to the development community. Please _remove the galaxy-user@bx.psu.edu mailing list from all replies_ - we want to avoid double posts.
For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
On 9/26/13 7:29 AM, Malcolm Tobias wrote:
All,
My understanding is that the cleanup_datasets.py under scripts/cleanup_datasets should be compatible with 'older' versions of postgresql. I'm running 8.1 under CentOS 5. When I attempt to run the scripts, it fails to clean any data sets. From the logs, I'm noticing messages like:
database_connection contains an unknown SQLAlchemy database dialect: postgresql
In my universe_wsgi.ini I've defined the database_connection as:
database_connection = postgresql://galaxy:********@localhost:5432/galaxydb
and am confident it's working as Galaxy works fine, other than this problem with deleting older datasets.
=9.1. I noticed yet another version of cleanup_datasets.py under cron which looks like it might be compatible with postgres, but am leary of
The pgcleanup.py script isn't an option, as it requires postgresql trying it as I'm not sure what it means by "1: database directory to clean". I'm not sure what directory to point this to, nor am I confident that just deleting data from the directory is the safest thing.
Any suggestions on how to proceed?
Thanks!
Malcolm
--
Malcolm Tobias
314.362.1594
___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists, please use the interface at:
To search Galaxy mailing lists use the unified search at:
-- Jennifer Hillman-Jackson http://galaxyproject.org
Any feedback on this? Our local instance of Galaxy is ready to "go live" and this is one of the remaining issues keeping us from more wide-spread usage. Can anyone confirm that I should be using scripts/cleanup_datasets/cleanup_datasets.py when I'm using postgresql? Are there any limits on the versions of postgresql? Thanks in advance, Malcolm On Thursday 26 September 2013 17:16:16 Jennifer Jackson wrote:
Posting to galaxy-dev@bx.psu.edu<mailto:galaxy-dev@bx.psu.edu> to give the question better exposure to the development community. Please remove the galaxy-user@bx.psu.edu<mailto:galaxy-user@bx.psu.edu> mailing list from all replies - we want to avoid double posts.
For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
On 9/26/13 7:29 AM, Malcolm Tobias wrote:
All,
My understanding is that the cleanup_datasets.py under scripts/cleanup_datasets should be compatible with 'older' versions of postgresql. I'm running 8.1 under CentOS 5. When I attempt to run the scripts, it fails to clean any data sets. From the logs, I'm noticing messages like:
database_connection contains an unknown SQLAlchemy database dialect: postgresql
In my universe_wsgi.ini I've defined the database_connection as:
database_connection = postgresql://galaxy:********@localhost:5432/galaxydb
and am confident it's working as Galaxy works fine, other than this problem with deleting older datasets.
The pgcleanup.py script isn't an option, as it requires postgresql >=9.1. I noticed yet another version of cleanup_datasets.py under cron which looks like it might be compatible with postgres, but am leary of trying it as I'm not sure what it means by "1: database directory to clean". I'm not sure what directory to point this to, nor am I confident that just deleting data from the directory is the safest thing.
Any suggestions on how to proceed?
Thanks!
Malcolm
--
Malcolm Tobias
314.362.1594
___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists, please use the interface at:
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
-- Jennifer Hillman-Jackson http://galaxyproject.org
-- Malcolm Tobias 314.362.1594
I've managed to resolve this issue. I'm not sure how I ended up with this issue, but in case anyone else get's hit by it, here's what I've learned: -At version 0.6, SQLAlchemy changed the format for connecting to postgresql: [postgresql] The "postgres" dialect is now named "postgresql" -I somehow managed to get multiple eggs pointing to different versions of SQLAlchemy: [galaxy@login002 galaxy-dist]$ ls eggs/SQLAlchemy-0. SQLAlchemy-0.5.6_dev_r6498-py2.5.egg/ SQLAlchemy-0.5.6_dev_r6498-py2.7.egg/ SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg -During my early stages of rolling out Galaxy, I managed to use the 0.7.9 version of SQLAlchemy which forced me to change my database_connection syntax: [galaxy@login002 galaxy-dist]$ ./run.sh --daemon -this updates some of the "eggs" -it doesn't start; check the logs: [galaxy@login002 galaxy-dist]$ tail paster.log ... ile__', None ), self.config.database_engine_options, app=self ) File "/home/galaxy/galaxy-dist/lib/galaxy/model/migrate/check.py", line 107, in create_or_verify_database % ( db_schema.version, migrate_repository.versions.latest, config_arg ) ) Exception: Your database has version '114' but this code expects version '115'. Please backup your database and then migrate the schema by running 'sh manage_db.sh upgrade'. Removing PID file paster.pid [galaxy@login002 galaxy-dist]$ sh manage_db.sh upgrade /home/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/url.py:105: SADeprecationWarning: The SQLAlchemy PostgreSQL dialect has been renamed from 'postgres' to 'postgresql'. The new URL format is postgresql[+driver]://<user>:<pass>@<host>/<dbname> -at that point I changed my database_connection from: database_connection = postgres:// to: database_connection = postgresql:// and everything appeared to be happy. Everything that is except for the cleanup script. -I've now changed everything to: database_connection = postgres:// and everything appears to be happy. I really don't understand how this came out. I was using postgresql:// for quite a while without issues, so it would seem that I could *not* have been using 0.5.6 of SQLAlchemy unless this was modified by the Galaxy community. Cheers, Malcolm On Thursday 26 September 2013 17:16:16 Jennifer Jackson wrote:
Posting to galaxy-dev@bx.psu.edu<mailto:galaxy-dev@bx.psu.edu> to give the question better exposure to the development community. Please remove the galaxy-user@bx.psu.edu<mailto:galaxy-user@bx.psu.edu> mailing list from all replies - we want to avoid double posts.
For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
On 9/26/13 7:29 AM, Malcolm Tobias wrote:
All,
My understanding is that the cleanup_datasets.py under scripts/cleanup_datasets should be compatible with 'older' versions of postgresql. I'm running 8.1 under CentOS 5. When I attempt to run the scripts, it fails to clean any data sets. From the logs, I'm noticing messages like:
database_connection contains an unknown SQLAlchemy database dialect: postgresql
In my universe_wsgi.ini I've defined the database_connection as:
database_connection = postgresql://galaxy:********@localhost:5432/galaxydb
and am confident it's working as Galaxy works fine, other than this problem with deleting older datasets.
The pgcleanup.py script isn't an option, as it requires postgresql >=9.1. I noticed yet another version of cleanup_datasets.py under cron which looks like it might be compatible with postgres, but am leary of trying it as I'm not sure what it means by "1: database directory to clean". I'm not sure what directory to point this to, nor am I confident that just deleting data from the directory is the safest thing.
Any suggestions on how to proceed?
Thanks!
Malcolm
--
Malcolm Tobias
314.362.1594
___________________________________________________________ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using "reply all" in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
http://lists.bx.psu.edu/listinfo/galaxy-dev
To manage your subscriptions to this and other Galaxy lists, please use the interface at:
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
-- Jennifer Hillman-Jackson http://galaxyproject.org
-- Malcolm Tobias 314.362.1594
participants (2)
-
Jennifer Jackson -
Malcolm Tobias