$GALAXY_TEST_DB_TEMPLATE failing with KeyError: <VerNum(127)>
Hi all, I've been using the SQLite template database feature to speed up functional testing. Rather than starting for the initial schema and applying all the incremental changes (as done when installing a fresh Galaxy instance), this lets the test suite start with a recent version of the database, by setting the environment variable $GALAXY_TEST_DB_TEMPLATE to a URL to fetch a template database, e.g. from: https://github.com/jmchilton/galaxy-downloads Database schema version 120 was working, changing to 127 broke: https://github.com/peterjc/pico_galaxy/commit/412d3fe88b8011f0b690956d129fd7... TravisCI using v120, worked: https://travis-ci.org/peterjc/pico_galaxy/builds/52308064 TravisCI using v127, broke: https://travis-ci.org/peterjc/pico_galaxy/builds/52398778 galaxy.model.migrate.check DEBUG 2015-02-27 10:05:33,274 pysqlite>=2 egg successfully loaded for sqlite dialect Traceback (most recent call last): File "./scripts/functional_tests.py", line 602, in <module> sys.exit( main() ) File "./scripts/functional_tests.py", line 409, in main app = UniverseApplication( **kwargs ) File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/lib/galaxy/app.py", line 49, in __init__ self._configure_models( check_migrate_databases=True, check_migrate_tools=check_migrate_tools, config_file=config_file ) File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/lib/galaxy/config.py", line 779, in _configure_models create_or_verify_database( db_url, config_file, self.config.database_engine_options, app=self ) File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/lib/galaxy/model/migrate/check.py", line 61, in create_or_verify_database migrate() File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/lib/galaxy/model/migrate/check.py", line 57, in migrate migrate_to_current_version( engine, db_schema ) File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/lib/galaxy/model/migrate/check.py", line 117, in migrate_to_current_version changeset = schema.changeset( None ) File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/schema.py", line 80, in changeset changeset = self.repository.changeset(database, start_ver, version) File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/repository.py", line 225, in changeset changes = [self.version(v).script(database, op) for v in versions] File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/repository.py", line 189, in version return self.versions.version(*p, **k) File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/version.py", line 140, in version return self.versions[VerNum(vernum)] KeyError: <VerNum(127)> This is now testing against the (beta) Galaxy on GitHub: https://github.com/galaxyproject/galaxy Is it possible this is out of synch with the SQLite templates John posts here? https://github.com/jmchilton/galaxy-downloads Peter
Hey Peter, That database is pretty new (not yet in stable) - but you are tracking the master branch of galaxy on github in your tests (which is sortof like stable). You will want to either track the dev branch (so you can continue to catch all the errors right away :)) or track master and use an older database. Someday soon I will move this logic into Galaxy's run_tests.sh so that Galaxy always knows what version of the database it needs and we don't have to deal with these version incompatibilities. Sorry for the confusion. Thanks, -John On Fri, Feb 27, 2015 at 9:07 AM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
Hi all,
I've been using the SQLite template database feature to speed up functional testing. Rather than starting for the initial schema and applying all the incremental changes (as done when installing a fresh Galaxy instance), this lets the test suite start with a recent version of the database, by setting the environment variable $GALAXY_TEST_DB_TEMPLATE to a URL to fetch a template database, e.g. from:
https://github.com/jmchilton/galaxy-downloads
Database schema version 120 was working, changing to 127 broke:
https://github.com/peterjc/pico_galaxy/commit/412d3fe88b8011f0b690956d129fd7...
TravisCI using v120, worked: https://travis-ci.org/peterjc/pico_galaxy/builds/52308064
TravisCI using v127, broke: https://travis-ci.org/peterjc/pico_galaxy/builds/52398778
galaxy.model.migrate.check DEBUG 2015-02-27 10:05:33,274 pysqlite>=2 egg successfully loaded for sqlite dialect Traceback (most recent call last): File "./scripts/functional_tests.py", line 602, in <module> sys.exit( main() ) File "./scripts/functional_tests.py", line 409, in main app = UniverseApplication( **kwargs ) File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/lib/galaxy/app.py", line 49, in __init__ self._configure_models( check_migrate_databases=True, check_migrate_tools=check_migrate_tools, config_file=config_file ) File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/lib/galaxy/config.py", line 779, in _configure_models create_or_verify_database( db_url, config_file, self.config.database_engine_options, app=self ) File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/lib/galaxy/model/migrate/check.py", line 61, in create_or_verify_database migrate() File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/lib/galaxy/model/migrate/check.py", line 57, in migrate migrate_to_current_version( engine, db_schema ) File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/lib/galaxy/model/migrate/check.py", line 117, in migrate_to_current_version changeset = schema.changeset( None ) File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/schema.py", line 80, in changeset changeset = self.repository.changeset(database, start_ver, version) File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/repository.py", line 225, in changeset changes = [self.version(v).script(database, op) for v in versions] File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/repository.py", line 189, in version return self.versions.version(*p, **k) File "/home/travis/build/peterjc/pico_galaxy/galaxy-master/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/version.py", line 140, in version return self.versions[VerNum(vernum)] KeyError: <VerNum(127)>
This is now testing against the (beta) Galaxy on GitHub: https://github.com/galaxyproject/galaxy
Is it possible this is out of synch with the SQLite templates John posts here? https://github.com/jmchilton/galaxy-downloads
Peter
On Fri, Feb 27, 2015 at 2:15 PM, John Chilton <jmchilton@gmail.com> wrote:
Hey Peter,
That database is pretty new (not yet in stable) - but you are tracking the master branch of galaxy on github in your tests (which is sortof like stable). You will want to either track the dev branch (so you can continue to catch all the errors right away :)) or track master and use an older database.
Ah - that makes sense - wrongly assumed the master was the default and development branch. (This probably also explains why the galaxy_blast tests stopped working, installing the blast.py custom datatypes was failing.)
Someday soon I will move this logic into Galaxy's run_tests.sh so that Galaxy always knows what version of the database it needs and we don't have to deal with these version incompatibilities. Sorry for the confusion.
Yes please - an explicit error message for "Your database schema (version XXX) is too new for this version of Galaxy" should do the trick (not just for run_tests.sh but one once in the database code to cover all of Galaxy). Thanks, Peter
participants (2)
-
John Chilton -
Peter Cock