nglims/removing multiplexing
Hi, I would like to remove the "multiplexed" menu from the nglims sample information form. How would I do that? In our lab, we don't want submitters making their own libraries since they might accidentally use the same barcodes or somehow mess up others' libraries. We want to remove that option altogether. Actually along the same lines, we would like to remove all "services needed" check boxes too and simply have all of the data entry on one page. Thank you for your help. -- Lee Katz, Ph.D.
Lee;
Hi, I would like to remove the "multiplexed" menu from the nglims sample information form. How would I do that? In our lab, we don't want submitters making their own libraries since they might accidentally use the same barcodes or somehow mess up others' libraries. We want to remove that option altogether.
Good idea. I added a flag to the nglims.yaml file so you can specify whether barcode multiplexing is available or not: https://bitbucket.org/chapmanb/galaxy-central/src/62f7dc3864535e68af7aa2a7ef... setting `available: false` will turn off the option for the user to select and add this. If you update from the latest in bitbucket it will have this change. I also synchronized with the most recent galaxy-dist, so an update will pull in those changes as well.
Actually along the same lines, we would like to remove all "services needed" check boxes too and simply have all of the data entry on one page.
Moving to a single page entry would be a big change from the current wizard approach. You can control what services are available by default using the `default` flag in the nglims.yaml: https://bitbucket.org/chapmanb/galaxy-central/src/62f7dc3864535e68af7aa2a7ef... Currently sequencing is the only one selected by default but you can add others if they're commonly used to make the submission process easier for users. Hope this helps, Brad
Thanks Brad! I updated with hg pull -u and then ran "sh manage_db.sh upgrade" but I have an error. Could you help? I backed up my database before I ran the upgrade script, so I can go back in time. (sorry about the wall of text) 109 -> 110... /home/galaxy/bin/chapman-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'bytes' Add UUID column to dataset table done 110 -> 111... Add support for job destinations to the job table done 111 -> 112... Migration script to add the data_manager_history_association table and data_manager_job_association. 0112_add_data_manager_history_association_and_data_manager_job_association_tables DEBUG 2013-07-01 14:46:25,549 Created data_manager_history_association table 0112_add_data_manager_history_association_and_data_manager_job_association_tables DEBUG 2013-07-01 14:46:25,549 Created data_manager_history_association table Created data_manager_history_association table 0112_add_data_manager_history_association_and_data_manager_job_association_tables DEBUG 2013-07-01 14:46:25,890 Created data_manager_job_association table 0112_add_data_manager_history_association_and_data_manager_job_association_tables DEBUG 2013-07-01 14:46:25,890 Created data_manager_job_association table Created data_manager_job_association table done 112 -> 113... Migration script to update the migrate_tools.repository_path column to point to the new location lib/tool_shed/galaxy_install/migrate. done 113 -> 114... Migration script to update the migrate_tools.repository_path column to point to the new location lib/tool_shed/galaxy_install/migrate. done 114 -> 115... Altering password column failed Traceback (most recent call last): File "lib/galaxy/model/migrate/versions/0115_longer_user_password_field.py", line 15, in upgrade user.c.password.alter(type=String(255)) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/schema.py", line 491, in alter return alter_column(self, *p, **k) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/schema.py", line 136, in alter_column engine._run_visitor(visitorcallable, delta) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 2302, in _run_visitor conn._run_visitor(visitorcallable, element, **kwargs) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1972, in _run_visitor **kwargs).traverse_single(element) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/ansisql.py", line 53, in traverse_single ret = super(AlterTableVisitor, self).traverse_single(elem) File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/visitors.py", line 106, in traverse_single return meth(obj, **kw) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/databases/sqlite.py", line 53, in visit_column self.recreate_table(table,column,delta) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/databases/sqlite.py", line 36, in recreate_table self.execute() File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/ansisql.py", line 42, in execute return self.connection.execute(self.buffer.getvalue()) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1449, in execute params) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1628, in _execute_text statement, parameters File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1698, in _execute_context context) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1691, in _execute_context context) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/default.py", line 331, in do_execute cursor.execute(statement, parameters) OperationalError: (OperationalError) there is already another table or index with this name: migration_tmp u'ALTER TABLE galaxy_user RENAME TO migration_tmp' () done 115 -> 116... Tables for a next generation sequencing lims tied to requests. Traceback (most recent call last): File "./scripts/manage_db.py", line 65, in <module> main( repository=repo, url=db_url ) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/shell.py", line 207, in main ret = command_func(**kwargs) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/api.py", line 186, in upgrade return _migrate(url, repository, version, upgrade=True, err=err, **opts) File "<string>", line 2, in _migrate File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/util/__init__.py", line 159, in with_engine return f(*a, **kw) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/api.py", line 366, in _migrate schema.runchange(ver, change, changeset.step) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/schema.py", line 91, in runchange change.run(self.engine, step) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/script/py.py", line 145, in run script_func(engine) File "lib/galaxy/model/migrate/versions/0116_seq_lims.py", line 52, in upgrade for table in _get_tables(metadata): File "lib/galaxy/model/migrate/versions/0116_seq_lims.py", line 34, in _get_tables primary_key = True), File "build/bdist.linux-x86_64/egg/sqlalchemy/schema.py", line 305, in __new__ sqlalchemy.exc.InvalidRequestError: Table 'sample_request_map' is already defined for this MetaData instance. Specify 'extend_existing=True' to redefine options and columns on an existing Table object. Error in sys.excepthook: Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/apport_python_hook.py", line 64, in apport_excepthook from apport.fileutils import likely_packaged, get_recent_crashes ImportError: No module named apport.fileutils Original exception was: Traceback (most recent call last): File "./scripts/manage_db.py", line 65, in <module> main( repository=repo, url=db_url ) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/shell.py", line 207, in main ret = command_func(**kwargs) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/api.py", line 186, in upgrade return _migrate(url, repository, version, upgrade=True, err=err, **opts) File "<string>", line 2, in _migrate File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/util/__init__.py", line 159, in with_engine return f(*a, **kw) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/api.py", line 366, in _migrate schema.runchange(ver, change, changeset.step) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/schema.py", line 91, in runchange change.run(self.engine, step) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/script/py.py", line 145, in run script_func(engine) File "lib/galaxy/model/migrate/versions/0116_seq_lims.py", line 52, in upgrade for table in _get_tables(metadata): File "lib/galaxy/model/migrate/versions/0116_seq_lims.py", line 34, in _get_tables primary_key = True), File "build/bdist.linux-x86_64/egg/sqlalchemy/schema.py", line 305, in __new__ sqlalchemy.exc.InvalidRequestError: Table 'sample_request_map' is already defined for this MetaData instance. Specify 'extend_existing=True' to redefine options and columns on an existing Table object. On Mon, Jul 1, 2013 at 12:46 PM, Brad Chapman <chapmanb@50mail.com> wrote:
Lee;
Hi, I would like to remove the "multiplexed" menu from the nglims sample information form. How would I do that? In our lab, we don't want submitters making their own libraries since they might accidentally use the same barcodes or somehow mess up others' libraries. We want to remove that option altogether.
Good idea. I added a flag to the nglims.yaml file so you can specify whether barcode multiplexing is available or not:
https://bitbucket.org/chapmanb/galaxy-central/src/62f7dc3864535e68af7aa2a7ef...
setting `available: false` will turn off the option for the user to select and add this.
If you update from the latest in bitbucket it will have this change. I also synchronized with the most recent galaxy-dist, so an update will pull in those changes as well.
Actually along the same lines, we would like to remove all "services needed" check boxes too and simply have all of the data entry on one page.
Moving to a single page entry would be a big change from the current wizard approach. You can control what services are available by default using the `default` flag in the nglims.yaml:
https://bitbucket.org/chapmanb/galaxy-central/src/62f7dc3864535e68af7aa2a7ef...
Currently sequencing is the only one selected by default but you can add others if they're commonly used to make the submission process easier for users.
Hope this helps, Brad
-- Lee Katz, Ph.D.
Lee;
Thanks Brad! I updated with hg pull -u and then ran "sh manage_db.sh upgrade" but I have an error. Could you help? I backed up my database before I ran the upgrade script, so I can go back in time.
I think there is something funky going on with the version of SQLAlchemy-migrate currently in use in Galaxy, at least on SQLite databases. It creates temporary tables but then they are not getting cleaned up. The best fix is to reset yourself back to the original version (minus 1 to take into account the LIMS tables): sh manage_db.sh downgrade --version=108 Then redo the upgrade: sh manage_db.sh uprade Anytime you run into the intermittent error, you can manually fix with: $ sqlite3 database/universe.sqlite sqlite> drop table migration_tmp; Keep your backup around and you can do this with the new version. Sorry for the ugly hack but that was a reasonable workaround. There is some discussion on the mailing list about this previously but I don't think there was a good conclusion: http://lists.bx.psu.edu/pipermail/galaxy-dev/2013-May/014570.html Hope this cleans it up for you, Brad
(sorry about the wall of text)
109 -> 110... /home/galaxy/bin/chapman-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'bytes'
Add UUID column to dataset table
done 110 -> 111...
Add support for job destinations to the job table
done 111 -> 112...
Migration script to add the data_manager_history_association table and data_manager_job_association.
0112_add_data_manager_history_association_and_data_manager_job_association_tables DEBUG 2013-07-01 14:46:25,549 Created data_manager_history_association table 0112_add_data_manager_history_association_and_data_manager_job_association_tables DEBUG 2013-07-01 14:46:25,549 Created data_manager_history_association table Created data_manager_history_association table 0112_add_data_manager_history_association_and_data_manager_job_association_tables DEBUG 2013-07-01 14:46:25,890 Created data_manager_job_association table 0112_add_data_manager_history_association_and_data_manager_job_association_tables DEBUG 2013-07-01 14:46:25,890 Created data_manager_job_association table Created data_manager_job_association table done 112 -> 113...
Migration script to update the migrate_tools.repository_path column to point to the new location lib/tool_shed/galaxy_install/migrate.
done 113 -> 114...
Migration script to update the migrate_tools.repository_path column to point to the new location lib/tool_shed/galaxy_install/migrate.
done 114 -> 115... Altering password column failed Traceback (most recent call last): File "lib/galaxy/model/migrate/versions/0115_longer_user_password_field.py", line 15, in upgrade user.c.password.alter(type=String(255)) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/schema.py", line 491, in alter return alter_column(self, *p, **k) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/schema.py", line 136, in alter_column engine._run_visitor(visitorcallable, delta) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 2302, in _run_visitor conn._run_visitor(visitorcallable, element, **kwargs) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1972, in _run_visitor **kwargs).traverse_single(element) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/ansisql.py", line 53, in traverse_single ret = super(AlterTableVisitor, self).traverse_single(elem) File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/visitors.py", line 106, in traverse_single return meth(obj, **kw) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/databases/sqlite.py", line 53, in visit_column self.recreate_table(table,column,delta) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/databases/sqlite.py", line 36, in recreate_table self.execute() File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/ansisql.py", line 42, in execute return self.connection.execute(self.buffer.getvalue()) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1449, in execute params) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1628, in _execute_text statement, parameters File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1698, in _execute_context context) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1691, in _execute_context context) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/default.py", line 331, in do_execute cursor.execute(statement, parameters) OperationalError: (OperationalError) there is already another table or index with this name: migration_tmp u'ALTER TABLE galaxy_user RENAME TO migration_tmp' () done 115 -> 116...
Tables for a next generation sequencing lims tied to requests.
Traceback (most recent call last): File "./scripts/manage_db.py", line 65, in <module> main( repository=repo, url=db_url ) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/shell.py", line 207, in main ret = command_func(**kwargs) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/api.py", line 186, in upgrade return _migrate(url, repository, version, upgrade=True, err=err, **opts) File "<string>", line 2, in _migrate File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/util/__init__.py", line 159, in with_engine return f(*a, **kw) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/api.py", line 366, in _migrate schema.runchange(ver, change, changeset.step) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/schema.py", line 91, in runchange change.run(self.engine, step) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/script/py.py", line 145, in run script_func(engine) File "lib/galaxy/model/migrate/versions/0116_seq_lims.py", line 52, in upgrade for table in _get_tables(metadata): File "lib/galaxy/model/migrate/versions/0116_seq_lims.py", line 34, in _get_tables primary_key = True), File "build/bdist.linux-x86_64/egg/sqlalchemy/schema.py", line 305, in __new__ sqlalchemy.exc.InvalidRequestError: Table 'sample_request_map' is already defined for this MetaData instance. Specify 'extend_existing=True' to redefine options and columns on an existing Table object. Error in sys.excepthook: Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/apport_python_hook.py", line 64, in apport_excepthook from apport.fileutils import likely_packaged, get_recent_crashes ImportError: No module named apport.fileutils
Original exception was: Traceback (most recent call last): File "./scripts/manage_db.py", line 65, in <module> main( repository=repo, url=db_url ) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/shell.py", line 207, in main ret = command_func(**kwargs) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/api.py", line 186, in upgrade return _migrate(url, repository, version, upgrade=True, err=err, **opts) File "<string>", line 2, in _migrate File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/util/__init__.py", line 159, in with_engine return f(*a, **kw) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/api.py", line 366, in _migrate schema.runchange(ver, change, changeset.step) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/schema.py", line 91, in runchange change.run(self.engine, step) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/script/py.py", line 145, in run script_func(engine) File "lib/galaxy/model/migrate/versions/0116_seq_lims.py", line 52, in upgrade for table in _get_tables(metadata): File "lib/galaxy/model/migrate/versions/0116_seq_lims.py", line 34, in _get_tables primary_key = True), File "build/bdist.linux-x86_64/egg/sqlalchemy/schema.py", line 305, in __new__ sqlalchemy.exc.InvalidRequestError: Table 'sample_request_map' is already defined for this MetaData instance. Specify 'extend_existing=True' to redefine options and columns on an existing Table object.
On Mon, Jul 1, 2013 at 12:46 PM, Brad Chapman <chapmanb@50mail.com> wrote:
Lee;
Hi, I would like to remove the "multiplexed" menu from the nglims sample information form. How would I do that? In our lab, we don't want submitters making their own libraries since they might accidentally use the same barcodes or somehow mess up others' libraries. We want to remove that option altogether.
Good idea. I added a flag to the nglims.yaml file so you can specify whether barcode multiplexing is available or not:
https://bitbucket.org/chapmanb/galaxy-central/src/62f7dc3864535e68af7aa2a7ef...
setting `available: false` will turn off the option for the user to select and add this.
If you update from the latest in bitbucket it will have this change. I also synchronized with the most recent galaxy-dist, so an update will pull in those changes as well.
Actually along the same lines, we would like to remove all "services needed" check boxes too and simply have all of the data entry on one page.
Moving to a single page entry would be a big change from the current wizard approach. You can control what services are available by default using the `default` flag in the nglims.yaml:
https://bitbucket.org/chapmanb/galaxy-central/src/62f7dc3864535e68af7aa2a7ef...
Currently sequencing is the only one selected by default but you can add others if they're commonly used to make the submission process easier for users.
Hope this helps, Brad
-- Lee Katz, Ph.D.
It's that previous versions of sqlalchemy-migrate didn't clean up these tables under sqlite and didn't mind overwriting them if they existed, but the new version both does clean up after itself, and expects the table to not exist, failing miserably if it does. I think there's a Trello card for fixing this somewhere, but yes, the temporary workaround is to just drop the migration_tmp table if this happens and migrate again. On Jul 1, 2013, at 3:16 PM, Brad Chapman <chapmanb@50mail.com> wrote:
Lee;
Thanks Brad! I updated with hg pull -u and then ran "sh manage_db.sh upgrade" but I have an error. Could you help? I backed up my database before I ran the upgrade script, so I can go back in time.
I think there is something funky going on with the version of SQLAlchemy-migrate currently in use in Galaxy, at least on SQLite databases. It creates temporary tables but then they are not getting cleaned up.
The best fix is to reset yourself back to the original version (minus 1 to take into account the LIMS tables):
sh manage_db.sh downgrade --version=108
Then redo the upgrade:
sh manage_db.sh uprade
Anytime you run into the intermittent error, you can manually fix with:
$ sqlite3 database/universe.sqlite sqlite> drop table migration_tmp;
Keep your backup around and you can do this with the new version. Sorry for the ugly hack but that was a reasonable workaround. There is some discussion on the mailing list about this previously but I don't think there was a good conclusion:
http://lists.bx.psu.edu/pipermail/galaxy-dev/2013-May/014570.html
Hope this cleans it up for you, Brad
(sorry about the wall of text)
109 -> 110... /home/galaxy/bin/chapman-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'bytes'
Add UUID column to dataset table
done 110 -> 111...
Add support for job destinations to the job table
done 111 -> 112...
Migration script to add the data_manager_history_association table and data_manager_job_association.
0112_add_data_manager_history_association_and_data_manager_job_association_tables DEBUG 2013-07-01 14:46:25,549 Created data_manager_history_association table 0112_add_data_manager_history_association_and_data_manager_job_association_tables DEBUG 2013-07-01 14:46:25,549 Created data_manager_history_association table Created data_manager_history_association table 0112_add_data_manager_history_association_and_data_manager_job_association_tables DEBUG 2013-07-01 14:46:25,890 Created data_manager_job_association table 0112_add_data_manager_history_association_and_data_manager_job_association_tables DEBUG 2013-07-01 14:46:25,890 Created data_manager_job_association table Created data_manager_job_association table done 112 -> 113...
Migration script to update the migrate_tools.repository_path column to point to the new location lib/tool_shed/galaxy_install/migrate.
done 113 -> 114...
Migration script to update the migrate_tools.repository_path column to point to the new location lib/tool_shed/galaxy_install/migrate.
done 114 -> 115... Altering password column failed Traceback (most recent call last): File "lib/galaxy/model/migrate/versions/0115_longer_user_password_field.py", line 15, in upgrade user.c.password.alter(type=String(255)) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/schema.py", line 491, in alter return alter_column(self, *p, **k) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/schema.py", line 136, in alter_column engine._run_visitor(visitorcallable, delta) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 2302, in _run_visitor conn._run_visitor(visitorcallable, element, **kwargs) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1972, in _run_visitor **kwargs).traverse_single(element) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/ansisql.py", line 53, in traverse_single ret = super(AlterTableVisitor, self).traverse_single(elem) File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/visitors.py", line 106, in traverse_single return meth(obj, **kw) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/databases/sqlite.py", line 53, in visit_column self.recreate_table(table,column,delta) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/databases/sqlite.py", line 36, in recreate_table self.execute() File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/ansisql.py", line 42, in execute return self.connection.execute(self.buffer.getvalue()) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1449, in execute params) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1628, in _execute_text statement, parameters File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1698, in _execute_context context) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1691, in _execute_context context) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/default.py", line 331, in do_execute cursor.execute(statement, parameters) OperationalError: (OperationalError) there is already another table or index with this name: migration_tmp u'ALTER TABLE galaxy_user RENAME TO migration_tmp' () done 115 -> 116...
Tables for a next generation sequencing lims tied to requests.
Traceback (most recent call last): File "./scripts/manage_db.py", line 65, in <module> main( repository=repo, url=db_url ) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/shell.py", line 207, in main ret = command_func(**kwargs) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/api.py", line 186, in upgrade return _migrate(url, repository, version, upgrade=True, err=err, **opts) File "<string>", line 2, in _migrate File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/util/__init__.py", line 159, in with_engine return f(*a, **kw) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/api.py", line 366, in _migrate schema.runchange(ver, change, changeset.step) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/schema.py", line 91, in runchange change.run(self.engine, step) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/script/py.py", line 145, in run script_func(engine) File "lib/galaxy/model/migrate/versions/0116_seq_lims.py", line 52, in upgrade for table in _get_tables(metadata): File "lib/galaxy/model/migrate/versions/0116_seq_lims.py", line 34, in _get_tables primary_key = True), File "build/bdist.linux-x86_64/egg/sqlalchemy/schema.py", line 305, in __new__ sqlalchemy.exc.InvalidRequestError: Table 'sample_request_map' is already defined for this MetaData instance. Specify 'extend_existing=True' to redefine options and columns on an existing Table object. Error in sys.excepthook: Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/apport_python_hook.py", line 64, in apport_excepthook from apport.fileutils import likely_packaged, get_recent_crashes ImportError: No module named apport.fileutils
Original exception was: Traceback (most recent call last): File "./scripts/manage_db.py", line 65, in <module> main( repository=repo, url=db_url ) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/shell.py", line 207, in main ret = command_func(**kwargs) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/api.py", line 186, in upgrade return _migrate(url, repository, version, upgrade=True, err=err, **opts) File "<string>", line 2, in _migrate File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/util/__init__.py", line 159, in with_engine return f(*a, **kw) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/api.py", line 366, in _migrate schema.runchange(ver, change, changeset.step) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/schema.py", line 91, in runchange change.run(self.engine, step) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/script/py.py", line 145, in run script_func(engine) File "lib/galaxy/model/migrate/versions/0116_seq_lims.py", line 52, in upgrade for table in _get_tables(metadata): File "lib/galaxy/model/migrate/versions/0116_seq_lims.py", line 34, in _get_tables primary_key = True), File "build/bdist.linux-x86_64/egg/sqlalchemy/schema.py", line 305, in __new__ sqlalchemy.exc.InvalidRequestError: Table 'sample_request_map' is already defined for this MetaData instance. Specify 'extend_existing=True' to redefine options and columns on an existing Table object.
On Mon, Jul 1, 2013 at 12:46 PM, Brad Chapman <chapmanb@50mail.com> wrote:
Lee;
Hi, I would like to remove the "multiplexed" menu from the nglims sample information form. How would I do that? In our lab, we don't want submitters making their own libraries since they might accidentally use the same barcodes or somehow mess up others' libraries. We want to remove that option altogether.
Good idea. I added a flag to the nglims.yaml file so you can specify whether barcode multiplexing is available or not:
https://bitbucket.org/chapmanb/galaxy-central/src/62f7dc3864535e68af7aa2a7ef...
setting `available: false` will turn off the option for the user to select and add this.
If you update from the latest in bitbucket it will have this change. I also synchronized with the most recent galaxy-dist, so an update will pull in those changes as well.
Actually along the same lines, we would like to remove all "services needed" check boxes too and simply have all of the data entry on one page.
Moving to a single page entry would be a big change from the current wizard approach. You can control what services are available by default using the `default` flag in the nglims.yaml:
https://bitbucket.org/chapmanb/galaxy-central/src/62f7dc3864535e68af7aa2a7ef...
Currently sequencing is the only one selected by default but you can add others if they're commonly used to make the submission process easier for users.
Hope this helps, Brad
-- Lee Katz, Ph.D.
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Thank you. I feel like this next error might be my fault (I don't know python very well). I think it is saying that I don't have a module, but it is installed via Ubuntu aptitude, so now I don't really know what to do next. I reinstalled the module like so: aptitude reinstall python-apport $ sh manage_db.sh downgrade --version=108 112 -> 111... Traceback (most recent call last): File "./scripts/manage_db.py", line 65, in <module> main( repository=repo, url=db_url ) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/shell.py", line 207, in main ret = command_func(**kwargs) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/api.py", line 202, in downgrade return _migrate(url, repository, version, upgrade=False, err=err, **opts) File "<string>", line 2, in _migrate File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/util/__init__.py", line 159, in with_engine return f(*a, **kw) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/api.py", line 366, in _migrate schema.runchange(ver, change, changeset.step) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/schema.py", line 91, in runchange change.run(self.engine, step) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/script/py.py", line 142, in run raise TypeError("upgrade/downgrade functions must accept engine" TypeError: upgrade/downgrade functions must accept engine parameter (since version 0.5.4) Error in sys.excepthook: Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/apport_python_hook.py", line 64, in apport_excepthook from apport.fileutils import likely_packaged, get_recent_crashes ImportError: No module named apport.fileutils Original exception was: Traceback (most recent call last): File "./scripts/manage_db.py", line 65, in <module> main( repository=repo, url=db_url ) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/shell.py", line 207, in main ret = command_func(**kwargs) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/api.py", line 202, in downgrade return _migrate(url, repository, version, upgrade=False, err=err, **opts) File "<string>", line 2, in _migrate File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/util/__init__.py", line 159, in with_engine return f(*a, **kw) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/api.py", line 366, in _migrate schema.runchange(ver, change, changeset.step) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/schema.py", line 91, in runchange change.run(self.engine, step) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/script/py.py", line 142, in run raise TypeError("upgrade/downgrade functions must accept engine" TypeError: upgrade/downgrade functions must accept engine parameter (since version 0.5.4) On Mon, Jul 1, 2013 at 4:14 PM, Dannon Baker <siryoplait@gmail.com> wrote:
It's that previous versions of sqlalchemy-migrate didn't clean up these tables under sqlite and didn't mind overwriting them if they existed, but the new version both does clean up after itself, and expects the table to not exist, failing miserably if it does.
I think there's a Trello card for fixing this somewhere, but yes, the temporary workaround is to just drop the migration_tmp table if this happens and migrate again.
On Jul 1, 2013, at 3:16 PM, Brad Chapman <chapmanb@50mail.com> wrote:
Lee;
Thanks Brad! I updated with hg pull -u and then ran "sh manage_db.sh upgrade" but I have an error. Could you help? I backed up my database before I ran the upgrade script, so I can go back in time.
I think there is something funky going on with the version of SQLAlchemy-migrate currently in use in Galaxy, at least on SQLite databases. It creates temporary tables but then they are not getting cleaned up.
The best fix is to reset yourself back to the original version (minus 1 to take into account the LIMS tables):
sh manage_db.sh downgrade --version=108
Then redo the upgrade:
sh manage_db.sh uprade
Anytime you run into the intermittent error, you can manually fix with:
$ sqlite3 database/universe.sqlite sqlite> drop table migration_tmp;
Keep your backup around and you can do this with the new version. Sorry for the ugly hack but that was a reasonable workaround. There is some discussion on the mailing list about this previously but I don't think there was a good conclusion:
http://lists.bx.psu.edu/pipermail/galaxy-dev/2013-May/014570.html
Hope this cleans it up for you, Brad
(sorry about the wall of text)
109 -> 110...
SAWarning: Did not recognize type 'BIGINT' of column 'bytes'
Add UUID column to dataset table
done 110 -> 111...
Add support for job destinations to the job table
done 111 -> 112...
Migration script to add the data_manager_history_association table and data_manager_job_association.
0112_add_data_manager_history_association_and_data_manager_job_association_tables
DEBUG 2013-07-01 14:46:25,549 Created data_manager_history_association
0112_add_data_manager_history_association_and_data_manager_job_association_tables
DEBUG 2013-07-01 14:46:25,549 Created data_manager_history_association
/home/galaxy/bin/chapman-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/dialects/sqlite/base.py:723: table table
Created data_manager_history_association table
0112_add_data_manager_history_association_and_data_manager_job_association_tables
DEBUG 2013-07-01 14:46:25,890 Created data_manager_job_association table
0112_add_data_manager_history_association_and_data_manager_job_association_tables
DEBUG 2013-07-01 14:46:25,890 Created data_manager_job_association table Created data_manager_job_association table done 112 -> 113...
Migration script to update the migrate_tools.repository_path column to point to the new location lib/tool_shed/galaxy_install/migrate.
done 113 -> 114...
Migration script to update the migrate_tools.repository_path column to point to the new location lib/tool_shed/galaxy_install/migrate.
done 114 -> 115... Altering password column failed Traceback (most recent call last): File "lib/galaxy/model/migrate/versions/0115_longer_user_password_field.py", line 15, in upgrade user.c.password.alter(type=String(255)) File
"/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/schema.py",
line 491, in alter return alter_column(self, *p, **k) File
"/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/schema.py",
line 136, in alter_column engine._run_visitor(visitorcallable, delta) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 2302, in _run_visitor conn._run_visitor(visitorcallable, element, **kwargs) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1972, in _run_visitor **kwargs).traverse_single(element) File
"/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/ansisql.py",
line 53, in traverse_single ret = super(AlterTableVisitor, self).traverse_single(elem) File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/visitors.py", line 106, in traverse_single return meth(obj, **kw) File
"/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/databases/sqlite.py",
line 53, in visit_column self.recreate_table(table,column,delta) File
"/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/databases/sqlite.py",
line 36, in recreate_table self.execute() File
"/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/ansisql.py",
line 42, in execute return self.connection.execute(self.buffer.getvalue()) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1449, in execute params) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1628, in _execute_text statement, parameters File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1698, in _execute_context context) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1691, in _execute_context context) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/default.py", line 331, in do_execute cursor.execute(statement, parameters) OperationalError: (OperationalError) there is already another table or index with this name: migration_tmp u'ALTER TABLE galaxy_user RENAME TO migration_tmp' () done 115 -> 116...
Tables for a next generation sequencing lims tied to requests.
Traceback (most recent call last): File "./scripts/manage_db.py", line 65, in <module> main( repository=repo, url=db_url ) File
"/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/shell.py",
line 207, in main ret = command_func(**kwargs) File
"/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/api.py",
line 186, in upgrade return _migrate(url, repository, version, upgrade=True, err=err, **opts) File "<string>", line 2, in _migrate File
"/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/util/__init__.py",
line 159, in with_engine return f(*a, **kw) File
"/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/api.py",
line 366, in _migrate schema.runchange(ver, change, changeset.step) File
"/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/schema.py",
line 91, in runchange change.run(self.engine, step) File
"/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/script/py.py",
line 145, in run script_func(engine) File "lib/galaxy/model/migrate/versions/0116_seq_lims.py", line 52, in upgrade for table in _get_tables(metadata): File "lib/galaxy/model/migrate/versions/0116_seq_lims.py", line 34, in _get_tables primary_key = True), File "build/bdist.linux-x86_64/egg/sqlalchemy/schema.py", line 305, in __new__ sqlalchemy.exc.InvalidRequestError: Table 'sample_request_map' is already defined for this MetaData instance. Specify 'extend_existing=True' to redefine options and columns on an existing Table object. Error in sys.excepthook: Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/apport_python_hook.py", line 64, in apport_excepthook from apport.fileutils import likely_packaged, get_recent_crashes ImportError: No module named apport.fileutils
Original exception was: Traceback (most recent call last): File "./scripts/manage_db.py", line 65, in <module> main( repository=repo, url=db_url ) File
"/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/shell.py",
line 207, in main ret = command_func(**kwargs) File
"/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/api.py",
line 186, in upgrade return _migrate(url, repository, version, upgrade=True, err=err, **opts) File "<string>", line 2, in _migrate File
"/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/util/__init__.py",
line 159, in with_engine return f(*a, **kw) File
"/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/api.py",
line 366, in _migrate schema.runchange(ver, change, changeset.step) File
"/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/schema.py",
line 91, in runchange change.run(self.engine, step) File
"/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/script/py.py",
line 145, in run script_func(engine) File "lib/galaxy/model/migrate/versions/0116_seq_lims.py", line 52, in upgrade for table in _get_tables(metadata): File "lib/galaxy/model/migrate/versions/0116_seq_lims.py", line 34, in _get_tables primary_key = True), File "build/bdist.linux-x86_64/egg/sqlalchemy/schema.py", line 305, in __new__ sqlalchemy.exc.InvalidRequestError: Table 'sample_request_map' is already defined for this MetaData instance. Specify 'extend_existing=True' to redefine options and columns on an existing Table object.
On Mon, Jul 1, 2013 at 12:46 PM, Brad Chapman <chapmanb@50mail.com> wrote:
Lee;
Hi, I would like to remove the "multiplexed" menu from the nglims
sample
information form. How would I do that? In our lab, we don't want submitters making their own libraries since they might accidentally use the same barcodes or somehow mess up others' libraries. We want to remove that option altogether.
Good idea. I added a flag to the nglims.yaml file so you can specify whether barcode multiplexing is available or not:
https://bitbucket.org/chapmanb/galaxy-central/src/62f7dc3864535e68af7aa2a7ef...
setting `available: false` will turn off the option for the user to select and add this.
If you update from the latest in bitbucket it will have this change. I also synchronized with the most recent galaxy-dist, so an update will pull in those changes as well.
Actually along the same lines, we would like to remove all "services needed" check boxes too and simply have all of the data entry on one page.
Moving to a single page entry would be a big change from the current wizard approach. You can control what services are available by default using the `default` flag in the nglims.yaml:
https://bitbucket.org/chapmanb/galaxy-central/src/62f7dc3864535e68af7aa2a7ef...
Currently sequencing is the only one selected by default but you can
add
others if they're commonly used to make the submission process easier for users.
Hope this helps, Brad
-- Lee Katz, Ph.D.
Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
-- Lee Katz, Ph.D.
Lee; Thanks for all the patience with this. I'll take the errors in order:
Original exception was: Traceback (most recent call last): [...] raise TypeError("upgrade/downgrade functions must accept engine" TypeError: upgrade/downgrade functions must accept engine parameter (since version 0.5.4)
This was a typo in the downgrade function for one of the patches. I pushed a fix so if you update from bitbucket things should work smoothly again (once we get to problem 2 below). For anyone with galaxy-dist commit privileges, here's the one liner: https://bitbucket.org/chapmanb/galaxy-central/commits/463852762a6eeadcf17731...
Thank you. I feel like this next error might be my fault (I don't know python very well). I think it is saying that I don't have a module, but it is installed via Ubuntu aptitude, so now I don't really know what to do next. I reinstalled the module like so: aptitude reinstall python-apport
$ sh manage_db.sh downgrade --version=108 112 -> 111... Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/apport_python_hook.py", line 64, in apport_excepthook from apport.fileutils import likely_packaged, get_recent_crashes ImportError: No module named apport.fileutils
The second error is complaining about this Ubuntu specific program after the initial error, so not sure what exactly happened here. It looks like there is a problem with this in Ubuntu 13.04: https://bugs.launchpad.net/ubuntu/+source/apport/+bug/1170479 Since this is an error report on top of your error report, I think it'll go away after you fix the initial cause by doing the update. Hope this gets it working smoothly for you, Brad
On Mon, Jul 1, 2013 at 8:34 PM, Brad Chapman <chapmanb@50mail.com> wrote:
This was a typo in the downgrade function for one of the patches. I pushed a fix so if you update from bitbucket things should work smoothly again (once we get to problem 2 below).
For anyone with galaxy-dist commit privileges, here's the one liner:
https://bitbucket.org/chapmanb/galaxy-central/commits/463852762a6eeadcf17731...
Thanks Brad, added to stable branch in https://bitbucket.org/galaxy/galaxy-central/commits/0a06df7da177a204c7f3800e...
Thanks! I did hg pull -u. For the database, I downgraded to 108 and then upgraded again but got this error when upgrading. Is it due to the Ubuntu bug? Or something else? Should I be sticking with a particular Hg version instead of the latest? $ sh manage_db.sh upgrade 115 -> 116... Tables for a next generation sequencing lims tied to requests. /home/galaxy/bin/chapman-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'bytes' Traceback (most recent call last): File "./scripts/manage_db.py", line 65, in <module> main( repository=repo, url=db_url ) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/shell.py", line 207, in main ret = command_func(**kwargs) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/api.py", line 186, in upgrade return _migrate(url, repository, version, upgrade=True, err=err, **opts) File "<string>", line 2, in _migrate File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/util/__init__.py", line 159, in with_engine return f(*a, **kw) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/api.py", line 366, in _migrate schema.runchange(ver, change, changeset.step) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/schema.py", line 91, in runchange change.run(self.engine, step) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/script/py.py", line 145, in run script_func(engine) File "lib/galaxy/model/migrate/versions/0116_seq_lims.py", line 52, in upgrade for table in _get_tables(metadata): File "lib/galaxy/model/migrate/versions/0116_seq_lims.py", line 34, in _get_tables primary_key = True), File "build/bdist.linux-x86_64/egg/sqlalchemy/schema.py", line 305, in __new__ sqlalchemy.exc.InvalidRequestError: Table 'sample_request_map' is already defined for this MetaData instance. Specify 'extend_existing=True' to redefine options and columns on an existing Table object. On Tue, Jul 2, 2013 at 10:59 AM, Dannon Baker <dannon.baker@gmail.com>wrote:
On Mon, Jul 1, 2013 at 8:34 PM, Brad Chapman <chapmanb@50mail.com> wrote:
This was a typo in the downgrade function for one of the patches. I pushed a fix so if you update from bitbucket things should work smoothly again (once we get to problem 2 below).
For anyone with galaxy-dist commit privileges, here's the one liner:
https://bitbucket.org/chapmanb/galaxy-central/commits/463852762a6eeadcf17731...
Thanks Brad, added to stable branch in https://bitbucket.org/galaxy/galaxy-central/commits/0a06df7da177a204c7f3800e...
-- Lee Katz, Ph.D.
Actually I didn't notice that I had an error in 114->115 in addition to 115->116 (below). So actually I just deleted the whole database, checked out a fresh version with hg, upgraded, upgraded the database with manage_db, and then ran the add_defaults script for nglims. Everything works and is fine, except I lost all my data. The entire nglims directory is backed up, but I don't have any way of retrieving the data easily in the working version. Do I need to remake all my groups, workflows, roles, and users? Actual analyses were all just test cases. original error when upgrading from 108 with the now broken version: 112 -> 113... Migration script to update the migrate_tools.repository_path column to point to the new location lib/tool_shed/galaxy_install/migrate. done 113 -> 114... Migration script to update the migrate_tools.repository_path column to point to the new location lib/tool_shed/galaxy_install/migrate. done 114 -> 115... Altering password column failed Traceback (most recent call last): File "lib/galaxy/model/migrate/versions/0115_longer_user_password_field.py", line 15, in upgrade user.c.password.alter(type=String(255)) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/schema.py", line 491, in alter return alter_column(self, *p, **k) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/schema.py", line 136, in alter_column engine._run_visitor(visitorcallable, delta) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 2302, in _run_visitor conn._run_visitor(visitorcallable, element, **kwargs) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1972, in _run_visitor **kwargs).traverse_single(element) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/ansisql.py", line 53, in traverse_sing le ret = super(AlterTableVisitor, self).traverse_single(elem) File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/visitors.py", line 106, in traverse_single return meth(obj, **kw) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/databases/sqlite.py", line 53, in visi t_column self.recreate_table(table,column,delta) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/databases/sqlite.py", line 36, in recr eate_table self.execute() File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/changeset/ansisql.py", line 42, in execute return self.connection.execute(self.buffer.getvalue()) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1449, in execute params) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1628, in _execute_text statement, parameters File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1698, in _execute_context context) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1691, in _execute_context context) File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/default.py", line 331, in do_execute cursor.execute(statement, parameters) OperationalError: (OperationalError) there is already another table or index with this name: migration_tmp u'ALTER TABLE galaxy_user R ENAME TO migration_tmp' () done 115 -> 116... Tables for a next generation sequencing lims tied to requests. Traceback (most recent call last): File "./scripts/manage_db.py", line 65, in <module> main( repository=repo, url=db_url ) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/shell.py", line 207, in main ret = command_func(**kwargs) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/api.py", line 186, in upgrade return _migrate(url, repository, version, upgrade=True, err=err, **opts) File "<string>", line 2, in _migrate File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/util/__init__.py", line 159, in with_ engine return f(*a, **kw) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/api.py", line 366, in _migrate schema.runchange(ver, change, changeset.step) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/schema.py", line 91, in runchange change.run(self.engine, step) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/script/py.py", line 145, in run script_func(engine) File "lib/galaxy/model/migrate/versions/0116_seq_lims.py", line 52, in upgrade for table in _get_tables(metadata): File "lib/galaxy/model/migrate/versions/0116_seq_lims.py", line 34, in _get_tables primary_key = True), File "build/bdist.linux-x86_64/egg/sqlalchemy/schema.py", line 305, in __new__ sqlalchemy.exc.InvalidRequestError: Table 'sample_request_map' is already defined for this MetaData instance. Specify 'extend_existin g=True' to redefine options and columns on an existing Table object. Error in sys.excepthook: Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/apport_python_hook.py", line 64, in apport_excepthook from apport.fileutils import likely_packaged, get_recent_crashes ImportError: No module named apport.fileutils Original exception was: Traceback (most recent call last): File "./scripts/manage_db.py", line 65, in <module> main( repository=repo, url=db_url ) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/shell.py", line 207, in main ret = command_func(**kwargs) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/api.py", line 186, in upgrade return _migrate(url, repository, version, upgrade=True, err=err, **opts) File "<string>", line 2, in _migrate File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/util/__init__.py", line 159, in with_ engine return f(*a, **kw) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/api.py", line 366, in _migrate schema.runchange(ver, change, changeset.step) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/schema.py", line 91, in runchange change.run(self.engine, step) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/script/py.py", line 145, in run script_func(engine) File "lib/galaxy/model/migrate/versions/0116_seq_lims.py", line 52, in upgrade for table in _get_tables(metadata): File "lib/galaxy/model/migrate/versions/0116_seq_lims.py", line 34, in _get_tables primary_key = True), File "build/bdist.linux-x86_64/egg/sqlalchemy/schema.py", line 305, in __new__ sqlalchemy.exc.InvalidRequestError: Table 'sample_request_map' is already defined for this MetaData instance. Specify 'extend_existin g=True' to redefine options and columns on an existing Table object. On Tue, Jul 2, 2013 at 12:28 PM, Lee Katz <lskatz@gmail.com> wrote:
Thanks! I did hg pull -u. For the database, I downgraded to 108 and then upgraded again but got this error when upgrading. Is it due to the Ubuntu bug? Or something else? Should I be sticking with a particular Hg version instead of the latest?
$ sh manage_db.sh upgrade 115 -> 116...
Tables for a next generation sequencing lims tied to requests.
/home/galaxy/bin/chapman-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/dialects/sqlite/base.py:723: SAWarning: Did not recognize type 'BIGINT' of column 'bytes' Traceback (most recent call last): File "./scripts/manage_db.py", line 65, in <module> main( repository=repo, url=db_url ) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/shell.py", line 207, in main ret = command_func(**kwargs) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/api.py", line 186, in upgrade return _migrate(url, repository, version, upgrade=True, err=err, **opts) File "<string>", line 2, in _migrate File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/util/__init__.py", line 159, in with_engine return f(*a, **kw) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/api.py", line 366, in _migrate schema.runchange(ver, change, changeset.step) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/schema.py", line 91, in runchange change.run(self.engine, step) File "/home/galaxy/bin/chapman-dist/eggs/sqlalchemy_migrate-0.7.2-py2.7.egg/migrate/versioning/script/py.py", line 145, in run script_func(engine) File "lib/galaxy/model/migrate/versions/0116_seq_lims.py", line 52, in upgrade for table in _get_tables(metadata): File "lib/galaxy/model/migrate/versions/0116_seq_lims.py", line 34, in _get_tables primary_key = True), File "build/bdist.linux-x86_64/egg/sqlalchemy/schema.py", line 305, in __new__ sqlalchemy.exc.InvalidRequestError: Table 'sample_request_map' is already defined for this MetaData instance. Specify 'extend_existing=True' to redefine options and columns on an existing Table object.
On Tue, Jul 2, 2013 at 10:59 AM, Dannon Baker <dannon.baker@gmail.com>wrote:
On Mon, Jul 1, 2013 at 8:34 PM, Brad Chapman <chapmanb@50mail.com> wrote:
This was a typo in the downgrade function for one of the patches. I pushed a fix so if you update from bitbucket things should work smoothly again (once we get to problem 2 below).
For anyone with galaxy-dist commit privileges, here's the one liner:
https://bitbucket.org/chapmanb/galaxy-central/commits/463852762a6eeadcf17731...
Thanks Brad, added to stable branch in https://bitbucket.org/galaxy/galaxy-central/commits/0a06df7da177a204c7f3800e...
-- Lee Katz, Ph.D.
-- Lee Katz, Ph.D.
Lee; I pushed a fix to the nglims fork on bitbucket which should resolve this errors. Apologies, the nglims error is a mistake on my part. The way Galaxy handles version updates changed a bit on the infrastructure side and I hadn't coded correctly to handle all the cases. You shouldn't need to restart and lose all your data. I think you're nearly there with the upgrade. More details below and thanks for all the patience.
114 -> 115... [...] OperationalError: (OperationalError) there is already another table or index with this name: migration_tmp u'ALTER TABLE galaxy_user R
ENAME TO migration_tmp' ()
This is the same error as before, so needs the same hack to work around: $ sqlite3 database/universe.sqlite sqlite> drop table migration_tmp; Sorry about that one, not sure exactly how to fix this more cleanly.
115 -> 116...
Tables for a next generation sequencing lims tied to requests. [...] sqlalchemy.exc.InvalidRequestError: Table 'sample_request_map' is already defined for this MetaData instance. Specify 'extend_existin
g=True' to redefine options and columns on an existing Table object.
This is my bug mentioned above, so should work with an updated checkout of the code. Hope this gets you there. Fingers crossed, Brad
It works! Thanks! And--getting back to the original request--it doesn't have the multiplexing button anymore after setting it as avalable: false. Awesome! On Tue, Jul 2, 2013 at 7:19 PM, Brad Chapman <chapmanb@50mail.com> wrote:
Lee; I pushed a fix to the nglims fork on bitbucket which should resolve this errors. Apologies, the nglims error is a mistake on my part. The way Galaxy handles version updates changed a bit on the infrastructure side and I hadn't coded correctly to handle all the cases.
You shouldn't need to restart and lose all your data. I think you're nearly there with the upgrade. More details below and thanks for all the patience.
114 -> 115... [...] OperationalError: (OperationalError) there is already another table or index with this name: migration_tmp u'ALTER TABLE galaxy_user R
ENAME TO migration_tmp' ()
This is the same error as before, so needs the same hack to work around:
$ sqlite3 database/universe.sqlite sqlite> drop table migration_tmp;
Sorry about that one, not sure exactly how to fix this more cleanly.
115 -> 116...
Tables for a next generation sequencing lims tied to requests. [...] sqlalchemy.exc.InvalidRequestError: Table 'sample_request_map' is already defined for this MetaData instance. Specify 'extend_existin
g=True' to redefine options and columns on an existing Table object.
This is my bug mentioned above, so should work with an updated checkout of the code.
Hope this gets you there. Fingers crossed, Brad
-- Lee Katz, Ph.D.
participants (4)
-
Brad Chapman -
Dannon Baker -
Dannon Baker -
Lee Katz