Actually, the example in my previous letter wasn't correct - I have tried to run run.sh and manage_db.sh scripts several times after creation of novel db, and the copy-paste is the result of the last attempt. So, indeed, the db galaxy was created in the first running, while warning messages from the last running. But now I drop the db again, and the issue is reproduced right after creation of the fresh db: [galaxy@cluster galaxy-dist]$ sh run.sh <~! several messages are omitted !~> migrate.versioning.script.base DEBUG 2013-09-11 18:33:49,426 Loading script lib/galaxy/model/migrate/versions/0113_update_migrate_tools_table.py... migrate.versioning.script.base DEBUG 2013-09-11 18:33:49,426 Script lib/galaxy/model/migrate/versions/0113_update_migrate_tools_table.py loaded successfully migrate.versioning.script.base DEBUG 2013-09-11 18:33:49,426 Loading script lib/galaxy/model/migrate/versions/0114_update_migrate_tools_table_again.py... migrate.versioning.script.base DEBUG 2013-09-11 18:33:49,426 Script lib/galaxy/model/migrate/versions/0114_update_migrate_tools_table_again.py loaded successfully migrate.versioning.script.base DEBUG 2013-09-11 18:33:49,426 Loading script lib/galaxy/model/migrate/versions/0115_longer_user_password_field.py... migrate.versioning.script.base DEBUG 2013-09-11 18:33:49,426 Script lib/galaxy/model/migrate/versions/0115_longer_user_password_field.py loaded successfully migrate.versioning.repository DEBUG 2013-09-11 18:33:49,427 Repository lib/galaxy/model/migrate loaded successfully migrate.versioning.repository DEBUG 2013-09-11 18:33:49,427 Config: OrderedDict([('db_settings', OrderedDict([('__name__', 'db_settings'), ('repository_id', 'Galaxy'), ('version_table', 'migrate_version'), ('required_dbs', '[]')]))]) galaxy.model.migrate.check DEBUG 2013-09-11 18:33:49,433 psycopg2 egg successfully loaded for postgres dialect /export/apps/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/url.py:105: SADeprecationWarning: The SQLAlchemy PostgreSQL dialect has been renamed from 'postgres' to 'postgresql'. The new URL format is postgresql[+driver]://<user>:<pass>@<host>/<dbname> galaxy.model.migrate.check INFO 2013-09-11 18:33:49,447 No database, initializing galaxy.model.migrate.check INFO 2013-09-11 18:33:49,477 Migrating 0 -> 1... galaxy.model.migrate.check INFO 2013-09-11 18:33:49,712 galaxy.model.migrate.check INFO 2013-09-11 18:33:49,712 Migrating 1 -> 2... galaxy.model.migrate.check INFO 2013-09-11 18:33:50,013 galaxy.model.migrate.check INFO 2013-09-11 18:33:50,014 Migrating 2 -> 3... galaxy.model.migrate.check INFO 2013-09-11 18:33:50,803 galaxy.model.migrate.check INFO 2013-09-11 18:33:50,803 Migrating 3 -> 4... galaxy.model.migrate.check INFO 2013-09-11 18:33:51,502 galaxy.model.migrate.check INFO 2013-09-11 18:33:51,502 Migrating 4 -> 5... galaxy.model.custom_types DEBUG 2013-09-11 18:33:51,556 Fixing a discrepancy concerning deleted shared history items. galaxy.model.custom_types DEBUG 2013-09-11 18:33:51,583 0 items affected, and restored. galaxy.model.custom_types DEBUG 2013-09-11 18:33:51,583 Time elapsed: 0.0264229774475 galaxy.model.custom_types DEBUG 2013-09-11 18:33:51,583 Fixing a discrepancy concerning cleaning up deleted history items shared before HDAs. galaxy.model.custom_types DEBUG 2013-09-11 18:33:51,586 0 items affected, and restored. galaxy.model.custom_types DEBUG 2013-09-11 18:33:51,586 Time elapsed: 0.00265407562256 galaxy.model.migrate.check INFO 2013-09-11 18:33:51,589 galaxy.model.migrate.check INFO 2013-09-11 18:33:51,589 Migrating 5 -> 6... galaxy.model.migrate.check INFO 2013-09-11 18:33:52,334 ======================================== galaxy.model.migrate.check INFO 2013-09-11 18:33:52,334 This migration script changes certain values in the history_dataset_association.extension galaxy.model.migrate.check INFO 2013-09-11 18:33:52,334 column, specifically 'qual' is chaged to be 'qual454'. galaxy.model.migrate.check INFO 2013-09-11 18:33:52,334 ======================================== galaxy.model.migrate.check INFO 2013-09-11 18:33:52,334 galaxy.model.migrate.check INFO 2013-09-11 18:33:52,335 Migrating 6 -> 7... galaxy.model.migrate.check INFO 2013-09-11 18:33:53,053 ======================================== galaxy.model.migrate.check INFO 2013-09-11 18:33:53,053 This migration script creates the new history_user_share_association table, and adds galaxy.model.migrate.check INFO 2013-09-11 18:33:53,053 a new boolean type column to the history table. This provides support for sharing galaxy.model.migrate.check INFO 2013-09-11 18:33:53,053 histories in the same way that workflows are shared. galaxy.model.migrate.check INFO 2013-09-11 18:33:53,053 ======================================== galaxy.model.migrate.check INFO 2013-09-11 18:33:53,053 galaxy.model.migrate.check INFO 2013-09-11 18:33:53,054 Migrating 7 -> 8... galaxy.model.migrate.check INFO 2013-09-11 18:33:53,969 ======================================== galaxy.model.migrate.check INFO 2013-09-11 18:33:53,970 This migration script adds the following new tables for supporting Galaxy forms: galaxy.model.migrate.check INFO 2013-09-11 18:33:53,970 1) form_definition_current galaxy.model.migrate.check INFO 2013-09-11 18:33:53,970 2) form_definition galaxy.model.migrate.check INFO 2013-09-11 18:33:53,970 3) form_values galaxy.model.migrate.check INFO 2013-09-11 18:33:53,970 4) request_type galaxy.model.migrate.check INFO 2013-09-11 18:33:53,970 5) request galaxy.model.migrate.check INFO 2013-09-11 18:33:53,970 6) sample galaxy.model.migrate.check INFO 2013-09-11 18:33:53,970 7) sample_state galaxy.model.migrate.check INFO 2013-09-11 18:33:53,971 8) sample_event galaxy.model.migrate.check INFO 2013-09-11 18:33:53,971 ======================================== galaxy.model.migrate.check INFO 2013-09-11 18:33:53,971 galaxy.model.migrate.check INFO 2013-09-11 18:33:53,971 Migrating 8 -> 9... galaxy.model.migrate.check INFO 2013-09-11 18:33:54,784 ======================================== galaxy.model.migrate.check INFO 2013-09-11 18:33:54,784 This migration script adds a new column to 2 tables: galaxy.model.migrate.check INFO 2013-09-11 18:33:54,785 1) a new boolean type column named 'submitted' to the 'request' table galaxy.model.migrate.check INFO 2013-09-11 18:33:54,785 2) a new string type column named 'bar_code' to the 'sample' table galaxy.model.migrate.check INFO 2013-09-11 18:33:54,785 ======================================== galaxy.model.migrate.check INFO 2013-09-11 18:33:54,785 galaxy.model.migrate.check INFO 2013-09-11 18:33:54,785 Migrating 9 -> 10... galaxy.model.migrate.check INFO 2013-09-11 18:33:55,607 ======================================== galaxy.model.migrate.check INFO 2013-09-11 18:33:55,607 This migration script adds the history_dataset_association_display_at_authorization table, which galaxy.model.migrate.check INFO 2013-09-11 18:33:55,608 allows 'private' datasets to be displayed at external sites without making them public. galaxy.model.migrate.check INFO 2013-09-11 18:33:55,608 galaxy.model.migrate.check INFO 2013-09-11 18:33:55,608 If using mysql, this script will display the following error, which is corrected in the next migration galaxy.model.migrate.check INFO 2013-09-11 18:33:55,608 script: history_dataset_association_display_at_authorization table failed: (OperationalError) galaxy.model.migrate.check INFO 2013-09-11 18:33:55,608 (1059, 'Identifier name 'ix_history_dataset_association_display_at_authorization_update_time' galaxy.model.migrate.check INFO 2013-09-11 18:33:55,608 is too long. galaxy.model.migrate.check INFO 2013-09-11 18:33:55,608 ======================================== galaxy.model.migrate.check INFO 2013-09-11 18:33:55,608 galaxy.model.migrate.check INFO 2013-09-11 18:33:55,609 Migrating 10 -> 11... galaxy.model.migrate.check INFO 2013-09-11 18:33:55,617 ======================================== galaxy.model.migrate.check INFO 2013-09-11 18:33:55,617 This script fixes a problem introduced in the previous migration script ( 9->10 ). MySQL galaxy.model.migrate.check INFO 2013-09-11 18:33:55,617 has a name length limit and thus the index 'ix_hdadaa_history_dataset_association_id' has galaxy.model.migrate.check INFO 2013-09-11 18:33:55,618 to be manually created. galaxy.model.migrate.check INFO 2013-09-11 18:33:55,618 ======================================== galaxy.model.migrate.check INFO 2013-09-11 18:33:55,618 galaxy.model.migrate.check INFO 2013-09-11 18:33:55,618 Migrating 11 -> 12... galaxy.model.migrate.check INFO 2013-09-11 18:33:56,467 ======================================== galaxy.model.migrate.check INFO 2013-09-11 18:33:56,467 This script adds a new user_address table that is currently only used with sample requests, where galaxy.model.migrate.check INFO 2013-09-11 18:33:56,467 a user can select from a list of his addresses to associate with the request. This script also galaxy.model.migrate.check INFO 2013-09-11 18:33:56,467 drops the request.submitted column which was boolean and replaces it with a request.state column galaxy.model.migrate.check INFO 2013-09-11 18:33:56,467 which is a string, allowing for more flexibility with request states. galaxy.model.migrate.check INFO 2013-09-11 18:33:56,467 ======================================== galaxy.model.migrate.check INFO 2013-09-11 18:33:56,467 galaxy.model.migrate.check INFO 2013-09-11 18:33:56,468 Migrating 12 -> 13... galaxy.model.migrate.check INFO 2013-09-11 18:33:57,362 ======================================== galaxy.model.migrate.check INFO 2013-09-11 18:33:57,362 This migration script eliminates all of the tables that were used for the 1st version of the galaxy.model.migrate.check INFO 2013-09-11 18:33:57,362 library templates where template fields and contents were each stored as a separate table row galaxy.model.migrate.check INFO 2013-09-11 18:33:57,362 in various library item tables. All of these tables are dropped in this script, eliminating all galaxy.model.migrate.check INFO 2013-09-11 18:33:57,362 existing template data. A total of 14 existing tables are dropped. galaxy.model.migrate.check INFO 2013-09-11 18:33:57,362 galaxy.model.migrate.check INFO 2013-09-11 18:33:57,362 We're now basing library templates on Galaxy forms, so field contents are stored as a jsonified galaxy.model.migrate.check INFO 2013-09-11 18:33:57,362 list in the form_values table. This script introduces the following 3 new association tables: galaxy.model.migrate.check INFO 2013-09-11 18:33:57,363 1) library_info_association galaxy.model.migrate.check INFO 2013-09-11 18:33:57,363 2) library_folder_info_association galaxy.model.migrate.check INFO 2013-09-11 18:33:57,363 3) library_dataset_dataset_info_association galaxy.model.migrate.check INFO 2013-09-11 18:33:57,363 galaxy.model.migrate.check INFO 2013-09-11 18:33:57,363 If using mysql, this script will throw an (OperationalError) exception due to a long index name galaxy.model.migrate.check INFO 2013-09-11 18:33:57,363 on the library_dataset_dataset_info_association table, which is OK because the script creates galaxy.model.migrate.check INFO 2013-09-11 18:33:57,363 an index with a shortened name. galaxy.model.migrate.check INFO 2013-09-11 18:33:57,363 ======================================== galaxy.model.migrate.check INFO 2013-09-11 18:33:57,364 galaxy.model.migrate.check INFO 2013-09-11 18:33:57,364 Migrating 13 -> 14... galaxy.model.migrate.check INFO 2013-09-11 18:33:58,217 galaxy.model.migrate.check INFO 2013-09-11 18:33:58,217 Migration script to add support for "Pages". galaxy.model.migrate.check INFO 2013-09-11 18:33:58,217 1) Creates Page and PageRevision tables galaxy.model.migrate.check INFO 2013-09-11 18:33:58,217 2) Adds username column to User table galaxy.model.migrate.check INFO 2013-09-11 18:33:58,217 galaxy.model.migrate.check INFO 2013-09-11 18:33:58,217 Traceback (most recent call last): File "/export/apps/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py", line 35, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/export/apps/galaxy/galaxy-dist/lib/galaxy/app.py", line 52, in __init__ create_or_verify_database( db_url, kwargs.get( 'global_conf', {} ).get( '__file__', None ), self.config.database_engine_options, app=self ) File "/export/apps/galaxy/galaxy-dist/lib/galaxy/model/migrate/check.py", line 66, in create_or_verify_database migrate_to_current_version( engine, db_schema ) File "/export/apps/galaxy/galaxy-dist/lib/galaxy/model/migrate/check.py", line 124, in migrate_to_current_version schema.runchange( ver, change, changeset.step ) File "/export/apps/galaxy/galaxy-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/schema.py", line 91, in runchange change.run(self.engine, step) File "/export/apps/galaxy/galaxy-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/script/py.py", line 145, in run script_func(engine) File "lib/galaxy/model/migrate/versions/0014_pages.py", line 67, in upgrade col.create( User_table, index_name='ix_user_username', unique_name='username' ) File "/export/apps/galaxy/galaxy-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/schema.py", line 533, in create engine.execute(stmt) File "/export/apps/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py", line 2446, in execute return connection.execute(statement, *multiparams, **params) File "/export/apps/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py", line 1449, in execute params) File "/export/apps/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py", line 1584, in _execute_clauseelement compiled_sql, distilled_params File "/export/apps/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py", line 1698, in _execute_context context) File "/export/apps/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py", line 1691, in _execute_context context) File "/export/apps/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/default.py", line 331, in do_execute cursor.execute(statement, parameters) ProgrammingError: (ProgrammingError) column "username" is of type character varying but expression is of type boolean HINT: You will need to rewrite or cast the expression. 'UPDATE galaxy_user SET username=%(username)s' {'username': False} 2013/9/11 Nate Coraor <nate@bx.psu.edu>
On Sep 11, 2013, at 9:16 AM, hogart wrote:
Hi Nate, Yes, db schema is old, but the upgrading (manage_db.sh upgrade) to the current version resulted to the error, as mentioned above. I don't remember the release number of the previous version of Galaxy, but it was installed in the March this year. A didn't restored the dump yet, now I am playinig with the newly created database. In my newbie point of view, it seems like, that dropping of the old database (drop database galaxy; as superuser) was incomplete, - is it possible? Thanks for letting me know, I will try to upgrade the PostgeSQL (the current version is default version of CentOS 5.6).
Sorry, I missed the bit where you were starting with an empty database and not reloading your dump of the old one.
It does seem possible that your database is not empty on startup, as the username column should not already exist at the time that this migration script runs. When you drop and then create the database, Galaxy should create all the necessary tables and run through all of the migration scripts the first time that you start it. Is it doing this? If it thinks that the database is already at revision 13, I suspect the database is not empty when it starts up. Check your database_connection string in universe_wsgi.ini and make sure it is pointed at the correct database.
--nate