Hi Nate, I do not create any tables manually. I simply create the database and setup users and roles. I do the following: initdb D /local/postgresql/data pg_ctl D /local/postgresql/data I /local/postgresql/pgsql.log start createdb galaxy_1 Sudo su - postgres pgsql galaxy_1 CREATE USER galaxy CREATE USER galaxyftp ALTER ROLE galaxyftp PASSWORD 'dbpassword' GRANT SELECT ON galaxy_user TO galaxyftp (Note: this fails prior to launching galaxy server for the first time) I start galaxy and run 'sh manage_db.sh upgrade'. 0 -> 1... done 1 -> 2... done 2 -> 3... done 3 -> 4... done 4 -> 5... done 5 -> 6... ======================================== This migration script changes certain values in the history_dataset_association.extension column, specifically 'qual' is chaged to be 'qual454'. ======================================== /hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/SQLAlchemy-0.5.6_dev_ r6498-py2.6.egg/sqlalchemy/sql/compiler.py:336: SAWarning: The SQLAlchemy psycopg2 dialect now automatically escapes '%' in text() expressions to '%%'. BIND_PARAMS.sub(do_bindparam, self.post_process_text(textclause.text)) done 6 -> 7... ======================================== This migration script creates the new history_user_share_association table, and adds a new boolean type column to the history table. This provides support for sharing histories in the same way that workflows are shared. ======================================== done 7 -> 8... ======================================== This migration script adds the following new tables for supporting Galaxy forms: 1) form_definition_current 2) form_definition 3) form_values 4) request_type 5) request 6) sample 7) sample_state 8) sample_event ======================================== done 8 -> 9... ======================================== This migration script adds a new column to 2 tables: 1) a new boolean type column named 'submitted' to the 'request' table 2) a new string type column named 'bar_code' to the 'sample' table ======================================== done 9 -> 10... ======================================== This migration script adds the history_dataset_association_display_at_authorization table, which allows 'private' datasets to be displayed at external sites without making them public. If using mysql, this script will display the following error, which is corrected in the next migration script: history_dataset_association_display_at_authorization table failed: (OperationalError) (1059, 'Identifier name 'ix_history_dataset_association_display_at_authorization_update_time' is too long. ======================================== done 10 -> 11... ======================================== This script fixes a problem introduced in the previous migration script ( 9->10 ). MySQL has a name length limit and thus the index 'ix_hdadaa_history_dataset_association_id' has to be manually created. ======================================== done 11 -> 12... ======================================== This script adds a new user_address table that is currently only used with sample requests, where a user can select from a list of his addresses to associate with the request. This script also drops the request.submitted column which was boolean and replaces it with a request.state column which is a string, allowing for more flexibility with request states. ======================================== 0012_user_address DEBUG 2011-12-14 15:15:12,085 Adding column 'deleted' to request_type table failed: (ProgrammingError) column "deleted" of relation "request_type" already exists '\nALTER TABLE "request_type" ADD deleted BOOLEAN' {} 0012_user_address DEBUG 2011-12-14 15:15:12,085 Adding column 'deleted' to request_type table failed: (ProgrammingError) column "deleted" of relation "request_type" already exists '\nALTER TABLE "request_type" ADD deleted BOOLEAN' {} done 12 -> 13... ======================================== This migration script eliminates all of the tables that were used for the 1st version of the library templates where template fields and contents were each stored as a separate table row in various library item tables. All of these tables are dropped in this script, eliminating all existing template data. A total of 14 existing tables are dropped. We're now basing library templates on Galaxy forms, so field contents are stored as a jsonified list in the form_values table. This script introduces the following 3 new association tables: 1) library_info_association 2) library_folder_info_association 3) library_dataset_dataset_info_association If using mysql, this script will throw an (OperationalError) exception due to a long index name on the library_dataset_dataset_info_association table, which is OK because the script creates an index with a shortened name. ======================================== done 13 -> 14... Migration script to add support for "Pages". 1) Creates Page and PageRevision tables 2) Adds username column to User table done 14 -> 15... This migration script adds the tables necessary to support tagging of histories, datasets, and history-dataset associations (user views of datasets). If using mysql, this script will display the following error, which is corrected in the next migration script: history_dataset_association_tag_association table failed: (OperationalError) (1059, 'Identifier name 'ix_history_dataset_association_tag_association_history_dataset_association _id' is too long) (ProgrammingError) relation "tag" already exists '\nCREATE TABLE tag (\n\tid SERIAL NOT NULL, \n\ttype INTEGER, \n\tparent_id INTEGER, \n\tname VARCHAR(255), \n\tPRIMARY KEY (id), \n\t FOREIGN KEY(parent_id) REFERENCES tag (id), \n\t UNIQUE (name)\n)\n\n' {} done 15 -> 16... This script fixes a problem introduced in 0015_tagging.py. MySQL has a name length limit and thus the index 'ix_hda_ta_history_dataset_association_id' has to be manually created. done 16 -> 17... ======================================== This script adds 3 indexes to table columns: library_folder.name, library_dataset.name, library_dataset_dataset_association.name. ======================================== 0017_library_item_indexes DEBUG 2011-12-14 15:15:15,400 Adding index 'ix_library_folder_name' to library_folder table failed: (ProgrammingError) relation "ix_library_folder_name" already exists 'CREATE INDEX ix_library_folder_name ON library_folder (name)' {} 0017_library_item_indexes DEBUG 2011-12-14 15:15:15,400 Adding index 'ix_library_folder_name' to library_folder table failed: (ProgrammingError) relation "ix_library_folder_name" already exists 'CREATE INDEX ix_library_folder_name ON library_folder (name)' {} 0017_library_item_indexes DEBUG 2011-12-14 15:15:15,413 Adding index 'ix_library_dataset_name' to library_dataset table failed: (ProgrammingError) relation "ix_library_dataset_name" already exists 'CREATE INDEX ix_library_dataset_name ON library_dataset (name)' {} 0017_library_item_indexes DEBUG 2011-12-14 15:15:15,413 Adding index 'ix_library_dataset_name' to library_dataset table failed: (ProgrammingError) relation "ix_library_dataset_name" already exists 'CREATE INDEX ix_library_dataset_name ON library_dataset (name)' {} done 17 -> 18... This migration script provides support for (a) ordering tags by recency and (b) tagging pages. This script deletes all existing tags. done 18 -> 19... ======================================== This script creates a request.folder_id column which is a foreign key to the library_folder table. This also adds a 'type' and 'layout' column to the form_definition table. ======================================== 0019_request_library_folder DEBUG 2011-12-14 15:15:16,933 Adding column 'type' to form_definition table failed: (ProgrammingError) column "type" of relation "form_definition" already exists '\nALTER TABLE "form_definition" ADD type VARCHAR(255)' {} 0019_request_library_folder DEBUG 2011-12-14 15:15:16,933 Adding column 'type' to form_definition table failed: (ProgrammingError) column "type" of relation "form_definition" already exists '\nALTER TABLE "form_definition" ADD type VARCHAR(255)' {} 0019_request_library_folder DEBUG 2011-12-14 15:15:16,934 Adding column 'layout' to form_definition table failed: (ProgrammingError) column "layout" of relation "form_definition" already exists '\nALTER TABLE "form_definition" ADD layout BYTEA' {} 0019_request_library_folder DEBUG 2011-12-14 15:15:16,934 Adding column 'layout' to form_definition table failed: (ProgrammingError) column "layout" of relation "form_definition" already exists '\nALTER TABLE "form_definition" ADD layout BYTEA' {} done 19 -> 20... ======================================== This script creates a job_to_output_library_dataset table for allowing library uploads to run as regular jobs. To support this, a library_folder_id column is added to the job table, and library_folder/output_library_datasets relations are added to the Job object. An index is also added to the dataset.state column. ======================================== done 20 -> 21... This migration script adds a user preferences table to Galaxy. done 21 -> 22... Migration script to add support for storing visualizations. 1) Creates Visualization and VisualizationRevision tables done 22 -> 23... Migration script to add columns for tracking whether pages are deleted and publicly accessible. done 23 -> 24... Remove unique constraint from page slugs to allow creating a page with the same slug as a deleted page. done 24 -> 25... ======================================== This script adds a foreign key to the form_values table in the galaxy_user table ======================================== done 25 -> 26... ======================================== This script adds tables needed for Galaxy cloud functionality. ======================================== done 26 -> 27... ======================================== This migration script adds the request_event table and removes the state field in the request table ======================================== done 27 -> 28... ======================================== This script adds the filename_override_metadata column to the JobExternalOutputMetadata table, allowing existing metadata files to be written when using external metadata and a cluster set up with read-only access to database/files ======================================== done 28 -> 29... This migration script adds a user actions table to Galaxy. done 29 -> 30... Migration script to add column for a history slug. done 30 -> 31... Migration script to (a) add and populate necessary columns for doing community tagging of histories, datasets, and pages and (b) add table for doing individual and community tagging of workflows. SQLite does not support 'ALTER TABLE ADD FOREIGN KEY', so this script will generate error messages when run against SQLite; however, script does execute successfully against SQLite. done 31 -> 32... Migration script to add slug column for stored workflow. done 32 -> 33... Migration script to add necessary columns for distinguishing between viewing/importing and publishing histories, workflows, and pages. Script adds published column to histories and workflows and importable column to pages. done 33 -> 34... Migration script to create a table for page-user share association. done 34 -> 35... Migration script to (a) create tables for annotating objects and (b) create tags for workflow steps. done 35 -> 36... Migration script to add a deleted column to the following tables: library_info_association, library_folder_info_association, library_dataset_dataset_info_association. done 36 -> 37... This migration script removes the library_id & folder_id fields in the 'request' table and adds the same to the 'sample' table. This also adds a 'datatx' column to request_type table to store the sequencer login information. Finally, this adds a 'dataset_files' column to the sample table. done 37 -> 38... Migration script to add an inheritable column to the following tables: library_info_association, library_folder_info_association. Also, in case of sqlite check if the previous migration script deleted the request table and if so, restore the table. done 38 -> 39... Migration script to add a synopsis column to the library table. Traceback (most recent call last): File "./scripts/manage_db.py", line 63, in <module> main( repository=repo, url=db_url ) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/sqlalchemy_migrate-0 .5.4-py2.6.egg/migrate/versioning/shell.py", line 150, in main ret = command_func(**kwargs) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/sqlalchemy_migrate-0 .5.4-py2.6.egg/migrate/versioning/api.py", line 221, in upgrade return _migrate(url, repository, version, upgrade=True, err=err, **opts) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/sqlalchemy_migrate-0 .5.4-py2.6.egg/migrate/versioning/api.py", line 349, in _migrate schema.runchange(ver, change, changeset.step) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/sqlalchemy_migrate-0 .5.4-py2.6.egg/migrate/versioning/schema.py", line 184, in runchange change.run(self.engine, step) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/sqlalchemy_migrate-0 .5.4-py2.6.egg/migrate/versioning/script/py.py", line 101, in run func() File "lib/galaxy/model/migrate/versions/0039_add_synopsis_column_to_library_tabl e.py", line 20, in upgrade c.create( Library_table ) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/sqlalchemy_migrate-0 .5.4-py2.6.egg/migrate/changeset/schema.py", line 365, in create engine._run_visitor(visitorcallable, self, *args, **kwargs) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/SQLAlchemy-0.5.6_dev _r6498-py2.6.egg/sqlalchemy/engine/base.py", line 1158, in _run_visitor visitorcallable(self.dialect, conn, **kwargs).traverse(element) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/SQLAlchemy-0.5.6_dev _r6498-py2.6.egg/sqlalchemy/sql/visitors.py", line 89, in traverse return traverse(obj, self.__traverse_options__, self._visitor_dict) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/SQLAlchemy-0.5.6_dev _r6498-py2.6.egg/sqlalchemy/sql/visitors.py", line 200, in traverse return traverse_using(iterate(obj, opts), obj, visitors) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/SQLAlchemy-0.5.6_dev _r6498-py2.6.egg/sqlalchemy/sql/visitors.py", line 194, in traverse_using meth(target) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/sqlalchemy_migrate-0 .5.4-py2.6.egg/migrate/changeset/ansisql.py", line 97, in visit_column self.execute() File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/SQLAlchemy-0.5.6_dev _r6498-py2.6.egg/sqlalchemy/engine/base.py", line 1812, in execute return self.connection.execute(self.buffer.getvalue()) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/SQLAlchemy-0.5.6_dev _r6498-py2.6.egg/sqlalchemy/engine/base.py", line 824, in execute return Connection.executors[c](self, object, multiparams, params) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/SQLAlchemy-0.5.6_dev _r6498-py2.6.egg/sqlalchemy/engine/base.py", line 888, in _execute_text return self.__execute_context(context) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/SQLAlchemy-0.5.6_dev _r6498-py2.6.egg/sqlalchemy/engine/base.py", line 896, in __execute_context self._cursor_execute(context.cursor, context.statement, context.parameters[0], context=context) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/SQLAlchemy-0.5.6_dev _r6498-py2.6.egg/sqlalchemy/engine/base.py", line 950, in _cursor_execute self._handle_dbapi_exception(e, statement, parameters, cursor, context) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/SQLAlchemy-0.5.6_dev _r6498-py2.6.egg/sqlalchemy/engine/base.py", line 931, in _handle_dbapi_exception raise exc.DBAPIError.instance(statement, parameters, e, connection_invalidated=is_disconnect) sqlalchemy.exc.ProgrammingError: (ProgrammingError) column "synopsis" of relation "library" already exists '\nALTER TABLE "library" ADD synopsis TEXT' {} Thanks, Iry On 1/3/12 11:41 AM, "Nate Coraor" <nate@bx.psu.edu> wrote:
On Dec 21, 2011, at 2:49 PM, Iry Witham wrote:
I have installed a new Galaxy server utilizing the November 18,2011 distribution on a SLES 11 VM and cannot get it to launch cleanly. I am running postgreSQL v. 8.3.9 to create my database. Once the database is created and running I launch my Galaxy server and within moments all of the PID fail except for the reports_webapp.pid. When I tail the logs I find the following:
tail reports_webapp.log - galaxy.webapps.reports.buildapp DEBUG 2011-12-21 14:20:54,179 Enabling 'httpexceptions' middleware galaxy.webapps.reports.buildapp DEBUG 2011-12-21 14:20:54,194 Enabling 'recursive' middleware
/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/Paste-1.6-py2.6.egg /paste/exceptions/serial_number_generator.py:11: DeprecationWarning: the md5 module is deprecated; use hashlib instead import md5 galaxy.webapps.reports.buildapp DEBUG 2011-12-21 14:20:54,272 Enabling 'error' middleware galaxy.webapps.reports.buildapp DEBUG 2011-12-21 14:20:54,277 Enabling 'trans logger' middleware galaxy.webapps.reports.buildapp DEBUG 2011-12-21 14:20:54,277 Enabling 'config' middleware galaxy.webapps.reports.buildapp DEBUG 2011-12-21 14:20:54,294 Enabling 'x-forwarded-host' middleware Starting server in PID 20411. serving on 0.0.0.0:9001 view at http://127.0.0.1:9001
tail web0.log - galaxy.model.migrate.check DEBUG 2011-12-21 14:20:54,793 psycopg2 egg successfully loaded for postgres dialect Traceback (most recent call last): File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/lib/galaxy/web/buildapp .py", line 82, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/lib/galaxy/app.py", line 39, in __init__ create_or_verify_database( db_url, kwargs.get( 'global_conf', {} ).get( '__file__', None ), self.config.database_engine_options ) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/lib/galaxy/model/migrat e/check.py", line 105, in create_or_verify_database % ( db_schema.version, migrate_repository.versions.latest, config_arg ) ) Exception: Your database has version '38' but this code expects version '85'. Please backup your database and then migrate the schema by running 'sh manage_db.sh -c ./universe_wsgi.webapp.ini upgrade'. Removing PID file web0.pid
tail runner0.log - galaxy.model.migrate.check DEBUG 2011-12-21 14:20:54,789 psycopg2 egg successfully loaded for postgres dialect Traceback (most recent call last): File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/lib/galaxy/web/buildapp .py", line 82, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/lib/galaxy/app.py", line 39, in __init__ create_or_verify_database( db_url, kwargs.get( 'global_conf', {} ).get( '__file__', None ), self.config.database_engine_options ) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/lib/galaxy/model/migrat e/check.py", line 105, in create_or_verify_database % ( db_schema.version, migrate_repository.versions.latest, config_arg ) ) Exception: Your database has version '38' but this code expects version '85'. Please backup your database and then migrate the schema by running 'sh manage_db.sh -c ./universe_wsgi.runner.ini upgrade'. Removing PID file runner0.pid
sh manage_db.sh upgrade -
38 -> 39...
Migration script to add a synopsis column to the library table.
Traceback (most recent call last): File "./scripts/manage_db.py", line 63, in <module> main( repository=repo, url=db_url ) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/sqlalchemy_migrate -0.5.4-py2.6.egg/migrate/versioning/shell.py", line 150, in main ret = command_func(**kwargs) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/sqlalchemy_migrate -0.5.4-py2.6.egg/migrate/versioning/api.py", line 221, in upgrade return _migrate(url, repository, version, upgrade=True, err=err, **opts) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/sqlalchemy_migrate -0.5.4-py2.6.egg/migrate/versioning/api.py", line 349, in _migrate schema.runchange(ver, change, changeset.step) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/sqlalchemy_migrate -0.5.4-py2.6.egg/migrate/versioning/schema.py", line 184, in runchange change.run(self.engine, step) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/sqlalchemy_migrate -0.5.4-py2.6.egg/migrate/versioning/script/py.py", line 101, in run func() File "lib/galaxy/model/migrate/versions/0039_add_synopsis_column_to_library_ta ble.py", line 20, in upgrade c.create( Library_table ) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/sqlalchemy_migrate -0.5.4-py2.6.egg/migrate/changeset/schema.py", line 365, in create engine._run_visitor(visitorcallable, self, *args, **kwargs) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/SQLAlchemy-0.5.6_d ev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 1158, in _run_visitor visitorcallable(self.dialect, conn, **kwargs).traverse(element) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/SQLAlchemy-0.5.6_d ev_r6498-py2.6.egg/sqlalchemy/sql/visitors.py", line 89, in traverse return traverse(obj, self.__traverse_options__, self._visitor_dict) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/SQLAlchemy-0.5.6_d ev_r6498-py2.6.egg/sqlalchemy/sql/visitors.py", line 200, in traverse return traverse_using(iterate(obj, opts), obj, visitors) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/SQLAlchemy-0.5.6_d ev_r6498-py2.6.egg/sqlalchemy/sql/visitors.py", line 194, in traverse_using meth(target) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/sqlalchemy_migrate -0.5.4-py2.6.egg/migrate/changeset/ansisql.py", line 97, in visit_column self.execute() File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/SQLAlchemy-0.5.6_d ev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 1812, in execute return self.connection.execute(self.buffer.getvalue()) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/SQLAlchemy-0.5.6_d ev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 824, in execute return Connection.executors[c](self, object, multiparams, params) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/SQLAlchemy-0.5.6_d ev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 888, in _execute_text return self.__execute_context(context) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/SQLAlchemy-0.5.6_d ev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 896, in __execute_context self._cursor_execute(context.cursor, context.statement, context.parameters[0], context=context) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/SQLAlchemy-0.5.6_d ev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 950, in _cursor_execute self._handle_dbapi_exception(e, statement, parameters, cursor, context) File "/hpcdata/galaxy-dev/galaxy-setup/galaxy-dist-jax/eggs/SQLAlchemy-0.5.6_d ev_r6498-py2.6.egg/sqlalchemy/engine/base.py", line 931, in _handle_dbapi_exception raise exc.DBAPIError.instance(statement, parameters, e, connection_invalidated=is_disconnect) sqlalchemy.exc.ProgrammingError: (ProgrammingError) column "synopsis" of relation "library" already exists '\nALTER TABLE "library" ADD synopsis TEXT' {}
I have dropped the database multiple times and recreated it and intentionally left out steps when creating DB users and roles and the manage_db.sh fails at different spots. However, I cannot get past 38 -> 39. Any suggestions?
Hi Iry,
Can you clarify whether you are manually creating any tables? You only need to create the database itself, Galaxy handles all of the tables.
If you're not creating any tables, what is the output of the very first run of Galaxy on an empty database?
--nate
Thanks,
IRy ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: