Thanks a lot!It seems our postgres DB got corrupted? Any suggestion on a fix?Dear list,
Yesterday, I noticed that the user quota setting on our Galaxy had a problem - the list of user email address is missing from the manage quota page. So I thought to give a Galaxy codebase update a try after backing up the Galaxy dir and the database.
Running hg head gives me:
==
postgres@ubuntu:/mnt/galaxyTools/galaxy-2013-05-13$ hg head
changeset: 10003:b4a373d86c51
tag: tip
parent: 10001:471484ff8be6
user: greg
date: Wed Jun 12 11:48:09 2013 -0400
summary: Add targets to Repository Actions menu items.
==
I updated the DB schema to support the code update. But the restart of Galaxy failed with an error in paster.log:====
Traceback (most recent call last):
File "/mnt/galaxyTools/galaxy-2013-05-13/lib/galaxy/webapps/galaxy/buildapp.py", line 35, in app_factory
app = UniverseApplication( global_conf = global_conf, **kwargs )
File "/mnt/galaxyTools/galaxy-2013-05-13/lib/galaxy/app.py", line 164, in __init__
self.job_manager = manager.JobManager( self )
File "/mnt/galaxyTools/galaxy-2013-05-13/lib/galaxy/jobs/manager.py", line 36, in __init__
self.job_handler.start()
File "/mnt/galaxyTools/galaxy-2013-05-13/lib/galaxy/jobs/handler.py", line 34, in start
self.job_queue.start()
File "/mnt/galaxyTools/galaxy-2013-05-13/lib/galaxy/jobs/handler.py", line 77, in start
self.__check_jobs_at_startup()
File "/mnt/galaxyTools/galaxy-2013-05-13/lib/galaxy/jobs/handler.py", line 125, in __check_jobs_at_startup
self.dispatcher.recover( job, job_wrapper )
File "/mnt/galaxyTools/galaxy-2013-05-13/lib/galaxy/jobs/handler.py", line 620, in recover
self.job_runners[runner_name].recover( job, job_wrapper )
File "/mnt/galaxyTools/galaxy-2013-05-13/lib/galaxy/jobs/runners/local.py", line 128, in recover
job_wrapper.change_state( model.Job.states.ERROR, info = "This job was killed when Galaxy was restarted. Please retry the job." )
File "/mnt/galaxyTools/galaxy-2013-05-13/lib/galaxy/jobs/__init__.py", line 824, in change_state
dataset.state = state
File "/mnt/galaxyTools/galaxy-2013-05-13/lib/galaxy/model/__init__.py", line 1163, in set_dataset_state
object_session( self ).flush() #flush here, because hda.flush() won't flush the Dataset object
File "/mnt/galaxyTools/galaxy-2013-05-13/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 1718, in flush
self._flush(objects)
File "/mnt/galaxyTools/galaxy-2013-05-13/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py", line 1789, in _flush
flush_context.execute()
File "/mnt/galaxyTools/galaxy-2013-05-13/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/unitofwork.py", line 331, in execute
rec.execute(self)
File "/mnt/galaxyTools/galaxy-2013-05-13/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/unitofwork.py", line 475, in execute
uow
File "/mnt/galaxyTools/galaxy-2013-05-13/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/persistence.py", line 59, in save_obj
mapper, table, update)
File "/mnt/galaxyTools/galaxy-2013-05-13/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/persistence.py", line 485, in _emit_update_statements
execute(statement, params)
File "/mnt/galaxyTools/galaxy-2013-05-13/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1449, in execute
params)
File "/mnt/galaxyTools/galaxy-2013-05-13/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1584, in _execute_clauseelement
compiled_sql, distilled_params
File "/mnt/galaxyTools/galaxy-2013-05-13/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1698, in _execute_context
context)
File "/mnt/galaxyTools/galaxy-2013-05-13/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py", line 1691, in _execute_context
context)
File "/mnt/galaxyTools/galaxy-2013-05-13/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/default.py", line 331, in do_execute
cursor.execute(statement, parameters)
InternalError: (InternalError) could not read block 326 in file "base/257536/259291": read only 0 of 8192 bytes
'UPDATE dataset SET update_time=%(update_time)s, state=%(state)s WHERE dataset.id = %(dataset_id)s' {'update_time': datetime.datetime(2013, 6, 17, 12, 42, 38, 145908), 'state': 'error', 'dataset_id': 68792}
==I went on to check the Galaxy database (in postgres) which confirmed the error in the paster.log.
==
galaxy=# select * from dataset where id = 68792; id | create_time | update_time | state | deleted | purged | purgable | external_filename | _extra_files_path | file_size | total_size | object_store_id | uuid
-------+----------------------------+----------------------------+-------+---------+--------+----------+-------------------+-------------------+-----------+------------+-----------------+------
68792 | 2013-06-12 06:55:17.608725 | 2013-06-12 06:55:19.396557 | error | f | f | t | | | | | |
(1 row)
galaxy=# UPDATE dataset SET update_time='2013-06-17 12:42:38.145908', state='error' WHERE dataset.id = 68792;
ERROR: could not read block 326 in file "base/257536/259291": read only 0 of 8192 bytes
galaxy=# UPDATE dataset SET state='error' WHERE dataset.id = 68792;
UPDATE 1
When I tried to restore the database from the dumped copy I made yesterday, I got the following error:
==
ERROR: insert or update on table "galaxy_session_to_history" violates foreign key constraint "galaxy_session_to_history_session_id_fkey"==
DETAIL: Key (session_id)=(956338) is not present in table "galaxy_session".
ERROR: insert or update on table "job" violates foreign key constraint "job_session_id_fkey"
DETAIL: Key (session_id)=(852060) is not present in table "galaxy_session".
ERROR: insert or update on table "workflow_step_connection" violates foreign key constraint "workflow_step_connection_input_step_id_fkey"
DETAIL: Key (input_step_id)=(18422) is not present in table "workflow_step".
ERROR: insert or update on table "workflow_step_connection" violates foreign key constraint "workflow_step_connection_output_step_id_fkey"
DETAIL: Key (output_step_id)=(18422) is not present in table "workflow_step".
Leon
--
Hailiang (Leon) Mei
Netherlands Bioinformatics Center
BioAssist NGS Taskforce
- http://ngs.nbic.nl
Skype: leon_mei Mobile: +31 6 41709231
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/