Problem upgrading DB from 73 to 75 (on SQLite)
Hi all, I've just updated my Mac development machine with the latest galaxy-central, and run.sh tells me the database needs to be updated from 73 to 75. Note this is running on SQLite. Running manage_db.sh upgrade gives an ImportError: No module named migrate.versioning.shell I just grabbed a fresh hg clone and it worked, but in case there is an obvious bug lurking here, the full output is below (and I'll keep the broken install on disk a bit longer in case there is anything else you need to know). Regards, Peter -- $ ./run.sh python path is: /Users/pjcock/repositories/galaxy-central/eggs/numpy-1.3.0-py2.5-macosx-10.3-fat-ucs2.egg, /Users/pjcock/repositories/galaxy-central/eggs/pysam-0.4.1_kanwei_ae2bd50d9945-py2.5-macosx-10.3-fat-ucs2.egg, /Users/pjcock/repositories/galaxy-central/eggs/Whoosh-0.3.18-py2.5.egg, /Users/pjcock/repositories/galaxy-central/eggs/pycrypto-2.0.1-py2.5-macosx-10.3-fat-ucs2.egg, /Users/pjcock/repositories/galaxy-central/eggs/python_lzo-1.08_2.03_static-py2.5-macosx-10.3-fat-ucs2.egg, /Users/pjcock/repositories/galaxy-central/eggs/bx_python-0.7.0_14b6a6c95da6-py2.5-macosx-10.3-fat-ucs2.egg, /Users/pjcock/repositories/galaxy-central/eggs/amqplib-0.6.1-py2.5.egg, /Users/pjcock/repositories/galaxy-central/eggs/pexpect-2.4-py2.5.egg, /Users/pjcock/repositories/galaxy-central/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.5.egg, /Users/pjcock/repositories/galaxy-central/eggs/Babel-0.9.4-py2.5.egg, /Users/pjcock/repositories/galaxy-central/eggs/Beaker-1.4-py2.5.egg, /Users/pjcock/repositories/galaxy-central/eggs/Mako-0.2.5-py2.5.egg, /Users/pjcock/repositories/galaxy-central/eggs/WebHelpers-0.2-py2.5.egg, /Users/pjcock/repositories/galaxy-central/eggs/simplejson-2.1.1-py2.5-macosx-10.3-fat-ucs2.egg, /Users/pjcock/repositories/galaxy-central/eggs/wchartype-0.1-py2.5.egg, /Users/pjcock/repositories/galaxy-central/eggs/elementtree-1.2.6_20050316-py2.5.egg, /Users/pjcock/repositories/galaxy-central/eggs/docutils-0.7-py2.5.egg, /Users/pjcock/repositories/galaxy-central/eggs/WebOb-0.8.5-py2.5.egg, /Users/pjcock/repositories/galaxy-central/eggs/Routes-1.12.3-py2.5.egg, /Users/pjcock/repositories/galaxy-central/eggs/Cheetah-2.2.2-py2.5-macosx-10.3-fat-ucs2.egg, /Users/pjcock/repositories/galaxy-central/eggs/PasteDeploy-1.3.3-py2.5.egg, /Users/pjcock/repositories/galaxy-central/eggs/PasteScript-1.7.3-py2.5.egg, /Users/pjcock/repositories/galaxy-central/eggs/Paste-1.6-py2.5.egg, /Users/pjcock/repositories/galaxy-central/lib, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/setuptools-0.6c8-py2.5.egg, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/MySQL_python-1.2.2-py2.5-macosx-10.3-i386.egg, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/TurboGears-1.0.7-py2.5.egg, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/TurboKid-1.0.4-py2.5.egg, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/TurboJson-1.1.4-py2.5.egg, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/TurboCheetah-1.0-py2.5.egg, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/RuleDispatch-0.5a0.dev_r2306-py2.5-macosx-10.5-i386.egg, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/FormEncode-1.1-py2.5.egg, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/DecoratorTools-1.7-py2.5.egg, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/configobj-4.5.3-py2.5.egg, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/CherryPy-2.3.0-py2.5.egg, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/kid-0.9.6-py2.5.egg, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/PyProtocols-1.0a0dev_r2302-py2.5-macosx-10.3-i386.egg, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/Genshi-0.5.1-py2.5-macosx-10.3-i386.egg, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/pydot-1.0.2-py2.5.egg, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/logilab_common-0.48.0-py2.5.egg, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/logilab_astng-0.19.3-py2.5.egg, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/pylint-0.19.0-py2.5.egg, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/nose-0.11.3-py2.5.egg, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/buildbot_slave-0.8.2-py2.5.egg, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/matplotlib-1.0.1_r0-py2.6-macosx-10.3-fat.egg, /usr/local/lib/wxPython-unicode-2.8.10.1/lib/python2.5/site-packages, /usr/local/lib/wxPython-unicode-2.8.10.1/lib/python2.5/site-packages/wx-2.8-mac-unicode, /Library/Frameworks/Python.framework/Versions/2.5/lib/python25.zip, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-darwin, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-mac, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-mac/lib-scriptpackages, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-tk, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-dynload, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/Numeric, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/PIL, /usr/local/lib/wxPython-unicode-2.8.10.1/lib/python2.5, /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,494 Loading datatypes from datatypes_conf.xml galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,499 Loaded display application 'ucsc_bam' for datatype 'bam', inherit=False galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,504 Loaded display application 'ensembl_bam' for datatype 'bam', inherit=False galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,505 Loaded display application 'genetrack_interval' for datatype 'bed', inherit=False galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,509 Loaded display application 'ucsc_bigbed' for datatype 'bigbed', inherit=False galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,513 Loaded display application 'ucsc_bigwig' for datatype 'bigwig', inherit=False galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,526 Loaded display application 'ensembl_gff' for datatype 'gff', inherit=True galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,527 Loaded display application 'genetrack_interval' for datatype 'interval', inherit=True galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,537 Loaded display application 'ensembl_interval' for datatype 'interval', inherit=True galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,542 Loaded display application 'gbrowse_interval_as_bed' for datatype 'interval', inherit=True galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,546 Loaded display application 'ucsc_maf_customtrack' for datatype 'mafcustomtrack', inherit=False galaxy.datatypes.assembly DEBUG 2011-02-21 15:19:10,548 Velvet log info JJ __init__ galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,559 Loaded sniffer for datatype: galaxy.datatypes.binary:Bam galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,559 Loaded sniffer for datatype: galaxy.datatypes.binary:Sff galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,559 Loaded sniffer for datatype: galaxy.datatypes.xml:BlastXml galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,559 Loaded sniffer for datatype: galaxy.datatypes.sequence:Maf galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,559 Loaded sniffer for datatype: galaxy.datatypes.sequence:Lav galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,559 Loaded sniffer for datatype: galaxy.datatypes.sequence:csFasta galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,559 Loaded sniffer for datatype: galaxy.datatypes.qualityscore:QualityScoreSOLiD galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,560 Loaded sniffer for datatype: galaxy.datatypes.qualityscore:QualityScore454 galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,560 Loaded sniffer for datatype: galaxy.datatypes.sequence:Fasta galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,560 Loaded sniffer for datatype: galaxy.datatypes.sequence:Fastq galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,560 Loaded sniffer for datatype: galaxy.datatypes.interval:Wiggle galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,561 Loaded sniffer for datatype: galaxy.datatypes.images:Html galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,561 Loaded sniffer for datatype: galaxy.datatypes.images:Pdf galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,561 Loaded sniffer for datatype: galaxy.datatypes.sequence:Axt galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,561 Loaded sniffer for datatype: galaxy.datatypes.interval:Bed galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,561 Loaded sniffer for datatype: galaxy.datatypes.interval:CustomTrack galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,562 Loaded sniffer for datatype: galaxy.datatypes.interval:Gtf galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,562 Loaded sniffer for datatype: galaxy.datatypes.interval:Gff galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,562 Loaded sniffer for datatype: galaxy.datatypes.interval:Gff3 galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,562 Loaded sniffer for datatype: galaxy.datatypes.tabular:Pileup galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,562 Loaded sniffer for datatype: galaxy.datatypes.interval:Interval galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,563 Loaded sniffer for datatype: galaxy.datatypes.tabular:Sam galaxy.datatypes.registry DEBUG 2011-02-21 15:19:10,563 Loaded sniffer for datatype: galaxy.datatypes.tabular:Vcf galaxy.model.migrate.check DEBUG 2011-02-21 15:19:10,606 pysqlite>=2 egg successfully loaded for sqlite dialect Traceback (most recent call last): File "/Users/pjcock/repositories/galaxy-central/lib/galaxy/web/buildapp.py", line 82, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/Users/pjcock/repositories/galaxy-central/lib/galaxy/app.py", line 32, in __init__ create_or_verify_database( db_url, self.config.database_engine_options ) File "/Users/pjcock/repositories/galaxy-central/lib/galaxy/model/migrate/check.py", line 102, in create_or_verify_database % ( db_schema.version, migrate_repository.versions.latest ) ) Exception: Your database has version '73' but this code expects version '75'. Please backup your database and then migrate the schema by running 'sh manage_db.sh upgrade'. $ sh manage_db.sh upgrade Traceback (most recent call last): File "./scripts/manage_db.py", line 12, in <module> from migrate.versioning.shell import main ImportError: No module named migrate.versioning.shell In case it is of use (it was asked for on what looked like a similar issue), $ ls lib/galaxy/model/migrate/versions/ 0001_initial_tables.py 0038_add_inheritable_column_to_library_template_assoc_tables.py 0001_initial_tables.pyc 0038_add_inheritable_column_to_library_template_assoc_tables.pyc 0002_metadata_file_table.py 0039_add_synopsis_column_to_library_table.py 0002_metadata_file_table.pyc 0039_add_synopsis_column_to_library_table.pyc 0003_security_and_libraries.py 0040_page_annotations.py 0003_security_and_libraries.pyc 0040_page_annotations.pyc 0004_indexes_and_defaults.py 0041_workflow_invocation.py 0004_indexes_and_defaults.pyc 0041_workflow_invocation.pyc 0005_cleanup_datasets_fix.py 0042_workflow_invocation_fix.py 0005_cleanup_datasets_fix.pyc 0042_workflow_invocation_fix.pyc 0006_change_qual_datatype.py 0043_visualization_sharing_tagging_annotating.py 0006_change_qual_datatype.pyc 0043_visualization_sharing_tagging_annotating.pyc 0007_sharing_histories.py 0044_add_notify_column_to_request_table.py 0007_sharing_histories.pyc 0044_add_notify_column_to_request_table.pyc 0008_galaxy_forms.py 0045_request_type_permissions_table.py 0008_galaxy_forms.pyc 0045_request_type_permissions_table.pyc 0009_request_table.py 0046_post_job_actions.py 0009_request_table.pyc 0046_post_job_actions.pyc 0010_hda_display_at_authz_table.py 0047_job_table_user_id_column.py 0010_hda_display_at_authz_table.pyc 0047_job_table_user_id_column.pyc 0011_v0010_mysql_index_fix.py 0048_dataset_instance_state_column.py 0011_v0010_mysql_index_fix.pyc 0048_dataset_instance_state_column.pyc 0012_user_address.py 0049_api_keys_table.py 0012_user_address.pyc 0049_api_keys_table.pyc 0013_change_lib_item_templates_to_forms.py 0050_drop_cloud_tables.py 0013_change_lib_item_templates_to_forms.pyc 0050_drop_cloud_tables.pyc 0014_pages.py 0051_imported_col_for_jobs_table.py 0014_pages.pyc 0051_imported_col_for_jobs_table.pyc 0015_tagging.py 0052_sample_dataset_table.py 0015_tagging.pyc 0052_sample_dataset_table.pyc 0016_v0015_mysql_index_fix.py 0053_item_ratings.py 0016_v0015_mysql_index_fix.pyc 0053_item_ratings.pyc 0017_library_item_indexes.py 0054_visualization_dbkey.py 0017_library_item_indexes.pyc 0054_visualization_dbkey.pyc 0018_ordered_tags_and_page_tags.py 0055_add_pja_assoc_for_jobs.py 0018_ordered_tags_and_page_tags.pyc 0055_add_pja_assoc_for_jobs.pyc 0019_request_library_folder.py 0056_workflow_outputs.py 0019_request_library_folder.pyc 0056_workflow_outputs.pyc 0020_library_upload_job.py 0057_request_notify.py 0020_library_upload_job.pyc 0057_request_notify.pyc 0021_user_prefs.py 0058_history_import_export.py 0021_user_prefs.pyc 0058_history_import_export.pyc 0022_visualization_tables.py 0059_sample_dataset_file_path.py 0022_visualization_tables.pyc 0059_sample_dataset_file_path.pyc 0023_page_published_and_deleted_columns.py 0060_history_archive_import.py 0023_page_published_and_deleted_columns.pyc 0060_history_archive_import.pyc 0024_page_slug_unique_constraint.py 0061_tasks.py 0024_page_slug_unique_constraint.pyc 0061_tasks.pyc 0025_user_info.py 0062_user_openid_table.py 0025_user_info.pyc 0062_user_openid_table.pyc 0026_cloud_tables.py 0063_sequencer_table.py 0026_cloud_tables.pyc 0063_sequencer_table.pyc 0027_request_events.py 0064_add_run_and_sample_run_association_tables.py 0027_request_events.pyc 0064_add_run_and_sample_run_association_tables.pyc 0028_external_metadata_file_override.py 0065_add_name_to_form_fields_and_values.py 0028_external_metadata_file_override.pyc 0065_add_name_to_form_fields_and_values.pyc 0029_user_actions.py 0066_deferred_job_and_transfer_job_tables.py 0029_user_actions.pyc 0066_deferred_job_and_transfer_job_tables.pyc 0030_history_slug_column.py 0067_populate_sequencer_table.py 0030_history_slug_column.pyc 0067_populate_sequencer_table.pyc 0031_community_and_workflow_tags.py 0068_rename_sequencer_to_external_services.py 0031_community_and_workflow_tags.pyc 0068_rename_sequencer_to_external_services.pyc 0032_stored_workflow_slug_column.py 0069_rename_sequencer_form_type.py 0032_stored_workflow_slug_column.pyc 0069_rename_sequencer_form_type.pyc 0033_published_cols_for_histories_and_workflows.py 0070_add_info_column_to_deferred_job_table.py 0033_published_cols_for_histories_and_workflows.pyc 0070_add_info_column_to_deferred_job_table.pyc 0034_page_user_share_association.py 0071_add_history_and_workflow_to_sample.py 0034_page_user_share_association.pyc 0071_add_history_and_workflow_to_sample.pyc 0035_item_annotations_and_workflow_step_tags.py 0072_add_pid_and_socket_columns_to_transfer_job_table.py 0035_item_annotations_and_workflow_step_tags.pyc 0072_add_pid_and_socket_columns_to_transfer_job_table.pyc 0036_add_deleted_column_to_library_template_assoc_tables.py 0073_add_ldda_to_implicit_conversion_table.py 0036_add_deleted_column_to_library_template_assoc_tables.pyc 0073_add_ldda_to_implicit_conversion_table.pyc 0037_samples_library.py 0074_add_purged_column_to_library_dataset_table.py 0037_samples_library.pyc 0075_add_subindex_column_to_run_table.py peter-cocks-mac-pro:galaxy-central pjcock$
On Mon, Feb 21, 2011 at 3:41 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
Hi all,
I've just updated my Mac development machine with the latest galaxy-central, and run.sh tells me the database needs to be updated from 73 to 75. Note this is running on SQLite. Running manage_db.sh upgrade gives an ImportError: No module named migrate.versioning.shell
I just grabbed a fresh hg clone and it worked, but in case there is an obvious bug lurking here, the full output is below (and I'll keep the broken install on disk a bit longer in case there is anything else you need to know).
Regards,
Peter
I just got the same issue with 75 to 76 on this machine :( $ ./run.sh ... galaxy.model.migrate.check DEBUG 2011-03-11 11:09:40,719 pysqlite>=2 egg successfully loaded for sqlite dialect Traceback (most recent call last): File "/Users/pjcock/repositories/galaxy-central/lib/galaxy/web/buildapp.py", line 82, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) File "/Users/pjcock/repositories/galaxy-central/lib/galaxy/app.py", line 32, in __init__ create_or_verify_database( db_url, self.config.database_engine_options ) File "/Users/pjcock/repositories/galaxy-central/lib/galaxy/model/migrate/check.py", line 102, in create_or_verify_database % ( db_schema.version, migrate_repository.versions.latest ) ) Exception: Your database has version '75' but this code expects version '76'. Please backup your database and then migrate the schema by running 'sh manage_db.sh upgrade'. then: $ sh manage_db.sh upgrade Traceback (most recent call last): File "./scripts/manage_db.py", line 12, in <module> from migrate.versioning.shell import main ImportError: No module named migrate.versioning.shell Peter
Peter Cock wrote:
$ sh manage_db.sh upgrade Traceback (most recent call last): File "./scripts/manage_db.py", line 12, in <module> from migrate.versioning.shell import main ImportError: No module named migrate.versioning.shell
This is a bug, but as a workaround try the following: python -ES ./scripts/manage_db.py upgrade
Peter ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
On Tue, Mar 15, 2011 at 3:18 PM, Nate Coraor <nate@bx.psu.edu> wrote:
Peter Cock wrote:
$ sh manage_db.sh upgrade Traceback (most recent call last): File "./scripts/manage_db.py", line 12, in <module> from migrate.versioning.shell import main ImportError: No module named migrate.versioning.shell
This is a bug, but as a workaround try the following:
python -ES ./scripts/manage_db.py upgrade
Hi Nate, The workaround failed for me, $ python -ES ./scripts/manage_db.py upgrade Traceback (most recent call last): File "./scripts/manage_db.py", line 12, in <module> from migrate.versioning.shell import main File "/Users/pjcock/repositories/galaxy-central-stuck-db-75/eggs/sqlalchemy_migrate-0.5.4-py2.6.egg/migrate/versioning/shell.py", line 7, in <module> from migrate.versioning.base import * File "/Users/pjcock/repositories/galaxy-central-stuck-db-75/eggs/sqlalchemy_migrate-0.5.4-py2.6.egg/migrate/versioning/base/__init__.py", line 5, in <module> from const import databases,operations File "/Users/pjcock/repositories/galaxy-central-stuck-db-75/eggs/sqlalchemy_migrate-0.5.4-py2.6.egg/migrate/versioning/base/const.py", line 7, in <module> from sqlalchemy.util import OrderedDict ImportError: No module named sqlalchemy.util There's probably some PYTHONPATH or something solution to this... I don't use eggs enough to see it immediately though. Is there a bug open for the upgrade issue? Thanks, Peter
Peter Cock wrote:
On Tue, Mar 15, 2011 at 3:18 PM, Nate Coraor <nate@bx.psu.edu> wrote:
Peter Cock wrote:
$ sh manage_db.sh upgrade Traceback (most recent call last): File "./scripts/manage_db.py", line 12, in <module> from migrate.versioning.shell import main ImportError: No module named migrate.versioning.shell
This is a bug, but as a workaround try the following:
python -ES ./scripts/manage_db.py upgrade
Hi Nate,
The workaround failed for me,
$ python -ES ./scripts/manage_db.py upgrade Traceback (most recent call last): File "./scripts/manage_db.py", line 12, in <module> from migrate.versioning.shell import main File "/Users/pjcock/repositories/galaxy-central-stuck-db-75/eggs/sqlalchemy_migrate-0.5.4-py2.6.egg/migrate/versioning/shell.py", line 7, in <module> from migrate.versioning.base import * File "/Users/pjcock/repositories/galaxy-central-stuck-db-75/eggs/sqlalchemy_migrate-0.5.4-py2.6.egg/migrate/versioning/base/__init__.py", line 5, in <module> from const import databases,operations File "/Users/pjcock/repositories/galaxy-central-stuck-db-75/eggs/sqlalchemy_migrate-0.5.4-py2.6.egg/migrate/versioning/base/const.py", line 7, in <module> from sqlalchemy.util import OrderedDict ImportError: No module named sqlalchemy.util
There's probably some PYTHONPATH or something solution to this... I don't use eggs enough to see it immediately though.
Hmm, okay. Try the following: python -ES ./scripts/fetch_eggs.py And then again: python -ES ./scripts/manage_db.py upgrade
Is there a bug open for the upgrade issue?
No, I think this is isolated to something going on in your specific python environment. If I can reproduce it then I can work on a fix. --nate
Thanks,
Peter
On Wed, Mar 16, 2011 at 2:06 PM, Nate Coraor <nate@bx.psu.edu> wrote:
Peter Cock wrote:
On Tue, Mar 15, 2011 at 3:18 PM, Nate Coraor <nate@bx.psu.edu> wrote:
Hi Nate,
The workaround failed for me,
$ python -ES ./scripts/manage_db.py upgrade Traceback (most recent call last): File "./scripts/manage_db.py", line 12, in <module> from migrate.versioning.shell import main File "/Users/pjcock/repositories/galaxy-central-stuck-db-75/eggs/sqlalchemy_migrate-0.5.4-py2.6.egg/migrate/versioning/shell.py", line 7, in <module> from migrate.versioning.base import * File "/Users/pjcock/repositories/galaxy-central-stuck-db-75/eggs/sqlalchemy_migrate-0.5.4-py2.6.egg/migrate/versioning/base/__init__.py", line 5, in <module> from const import databases,operations File "/Users/pjcock/repositories/galaxy-central-stuck-db-75/eggs/sqlalchemy_migrate-0.5.4-py2.6.egg/migrate/versioning/base/const.py", line 7, in <module> from sqlalchemy.util import OrderedDict ImportError: No module named sqlalchemy.util
There's probably some PYTHONPATH or something solution to this... I don't use eggs enough to see it immediately though.
Hmm, okay. Try the following:
python -ES ./scripts/fetch_eggs.py
And then again:
python -ES ./scripts/manage_db.py upgrade
Is there a bug open for the upgrade issue?
No, I think this is isolated to something going on in your specific python environment. If I can reproduce it then I can work on a fix.
--nate
$ python -ES ./scripts/fetch_eggs.py Fetched http://eggs.g2.bx.psu.edu/Mako/Mako-0.2.5-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/pysam/pysam-0.4.1_kanwei_ae2bd50d9945-py2.6-macosx... Fetched http://eggs.g2.bx.psu.edu/Babel/Babel-0.9.4-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/Whoosh/Whoosh-0.3.18-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/Tempita/Tempita-0.1-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/Cheetah/Cheetah-2.2.2-py2.6-macosx-10.6-universal-... Fetched http://eggs.g2.bx.psu.edu/lrucache/lrucache-0.2-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/NoseHTML/NoseHTML-0.4.1-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/pexpect/pexpect-2.4-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/amqplib/amqplib-0.6.1-py2.6.egg Warning: decorator (a dependent egg of sqlalchemy-migrate) cannot be fetched Fetched http://eggs.g2.bx.psu.edu/bx_python/bx_python-0.7.0_14b6a6c95da6-py2.6-macos... Fetched http://eggs.g2.bx.psu.edu/PasteDeploy/PasteDeploy-1.3.3-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/WebHelpers/WebHelpers-0.2-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/docutils/docutils-0.7-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/numpy/numpy-1.3.0-py2.6-macosx-10.6-universal-ucs2... Fetched http://eggs.g2.bx.psu.edu/pysqlite/pysqlite-2.5.6_3.6.17_static-py2.6-macosx... Fetched http://eggs.g2.bx.psu.edu/Beaker/Beaker-1.4-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/SVGFig/SVGFig-1.1.6-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/SQLAlchemy/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/simplejson/simplejson-2.1.1-py2.6-macosx-10.6-univ... Fetched http://eggs.g2.bx.psu.edu/NoseTestDiff/NoseTestDiff-0.1-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/python_lzo/python_lzo-1.08_2.03_static-py2.6-macos... Fetched http://eggs.g2.bx.psu.edu/wchartype/wchartype-0.1-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/twill/twill-0.9-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/Routes/Routes-1.12.3-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/elementtree/elementtree-1.2.6_20050316-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/decorator/decorator-3.1.2-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/GeneTrack/GeneTrack-2.0.0_beta_1_dev_48da9e998f0ca... Fetched http://eggs.g2.bx.psu.edu/pycrypto/pycrypto-2.0.1-py2.6-macosx-10.6-universa... Fetched http://eggs.g2.bx.psu.edu/WebError/WebError-0.8a-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/Paste/Paste-1.6-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/wsgiref/wsgiref-0.1.2-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/python_daemon/python_daemon-1.5.5-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/nose/nose-0.11.1-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/WebOb/WebOb-0.8.5-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/PasteScript/PasteScript-1.7.3-py2.6.egg $ python -ES ./scripts/manage_db.py upgrade 75 -> 76... This migration script fixes the data corruption caused in the form_values table (content json field) by migrate script 65. No corrupted rows found. done Job done - thanks! This also worked on the old install I had stuck on DB version 73. Does this help explain what the root cause of the problem is? Peter
Peter Cock wrote:
$ python -ES ./scripts/fetch_eggs.py Fetched http://eggs.g2.bx.psu.edu/Mako/Mako-0.2.5-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/pysam/pysam-0.4.1_kanwei_ae2bd50d9945-py2.6-macosx... Fetched http://eggs.g2.bx.psu.edu/Babel/Babel-0.9.4-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/Whoosh/Whoosh-0.3.18-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/Tempita/Tempita-0.1-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/Cheetah/Cheetah-2.2.2-py2.6-macosx-10.6-universal-... Fetched http://eggs.g2.bx.psu.edu/lrucache/lrucache-0.2-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/NoseHTML/NoseHTML-0.4.1-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/pexpect/pexpect-2.4-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/amqplib/amqplib-0.6.1-py2.6.egg Warning: decorator (a dependent egg of sqlalchemy-migrate) cannot be fetched Fetched http://eggs.g2.bx.psu.edu/bx_python/bx_python-0.7.0_14b6a6c95da6-py2.6-macos... Fetched http://eggs.g2.bx.psu.edu/PasteDeploy/PasteDeploy-1.3.3-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/WebHelpers/WebHelpers-0.2-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/docutils/docutils-0.7-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/numpy/numpy-1.3.0-py2.6-macosx-10.6-universal-ucs2... Fetched http://eggs.g2.bx.psu.edu/pysqlite/pysqlite-2.5.6_3.6.17_static-py2.6-macosx... Fetched http://eggs.g2.bx.psu.edu/Beaker/Beaker-1.4-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/SVGFig/SVGFig-1.1.6-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/SQLAlchemy/SQLAlchemy-0.5.6_dev_r6498-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/simplejson/simplejson-2.1.1-py2.6-macosx-10.6-univ... Fetched http://eggs.g2.bx.psu.edu/NoseTestDiff/NoseTestDiff-0.1-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/python_lzo/python_lzo-1.08_2.03_static-py2.6-macos... Fetched http://eggs.g2.bx.psu.edu/wchartype/wchartype-0.1-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/twill/twill-0.9-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/Routes/Routes-1.12.3-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/elementtree/elementtree-1.2.6_20050316-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/decorator/decorator-3.1.2-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/GeneTrack/GeneTrack-2.0.0_beta_1_dev_48da9e998f0ca... Fetched http://eggs.g2.bx.psu.edu/pycrypto/pycrypto-2.0.1-py2.6-macosx-10.6-universa... Fetched http://eggs.g2.bx.psu.edu/WebError/WebError-0.8a-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/Paste/Paste-1.6-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/wsgiref/wsgiref-0.1.2-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/python_daemon/python_daemon-1.5.5-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/nose/nose-0.11.1-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/WebOb/WebOb-0.8.5-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/PasteScript/PasteScript-1.7.3-py2.6.egg
Were there really no eggs, or did you remove your eggs directory prior to running this fetch?
$ python -ES ./scripts/manage_db.py upgrade 75 -> 76...
This migration script fixes the data corruption caused in the form_values table (content json field) by migrate script 65.
No corrupted rows found. done
Job done - thanks! This also worked on the old install I had stuck on DB version 73. Does this help explain what the root cause of the problem is?
Depending on the answer to the above question, it means there is probably (still) a bug in the egg conflict/resolve code somewhere. --nate
Peter
On Thu, Mar 17, 2011 at 7:47 PM, Nate Coraor <nate@bx.psu.edu> wrote:
Peter Cock wrote:
$ python -ES ./scripts/fetch_eggs.py Fetched http://eggs.g2.bx.psu.edu/Mako/Mako-0.2.5-py2.6.egg ... Fetched http://eggs.g2.bx.psu.edu/PasteScript/PasteScript-1.7.3-py2.6.egg
Were there really no eggs, or did you remove your eggs directory prior to running this fetch?
I can't tell you if there were no eggs (I'm fairly sure there were some), but I hadn't removed any deliberately. Might that have happened via hg update clean?
$ python -ES ./scripts/manage_db.py upgrade 75 -> 76...
This migration script fixes the data corruption caused in the form_values table (content json field) by migrate script 65.
No corrupted rows found. done
Job done - thanks! This also worked on the old install I had stuck on DB version 73. Does this help explain what the root cause of the problem is?
Depending on the answer to the above question, it means there is probably (still) a bug in the egg conflict/resolve code somewhere.
I will be in touch if this happens again (and it probably will next time the database schema is updated), but if you want I could try and reproduce it by doing a clean checkout of an old revision, running galaxy, then try to update it to the current code. Regards, Peter
Peter Cock wrote:
Were there really no eggs, or did you remove your eggs directory prior to running this fetch?
I can't tell you if there were no eggs (I'm fairly sure there were some), but I hadn't removed any deliberately. Might that have happened via hg update clean?
I'm pretty sure hg update clean will only remove changes of tracked files, not untracked ones like the eggs.
Depending on the answer to the above question, it means there is probably (still) a bug in the egg conflict/resolve code somewhere.
I will be in touch if this happens again (and it probably will next time the database schema is updated), but if you want I could try and reproduce it by doing a clean checkout of an old revision, running galaxy, then try to update it to the current code.
If you know the exact revision to start with to reproduce it, that'd certainly help. Thanks, --nate
Regards,
Peter
participants (2)
-
Nate Coraor -
Peter Cock