Really intensive database operations including datasest collections
but other things too like multi-running tools or workflows over many
individual datasets for instance - can very easily overwhelm the
default sqlite database. This is frustrating and shouldn't happen -
but it does unfortunately. I would recommend using a postgres database
when testing out dataset collections. The good news is that it is
easier than ever to get a fully fledged production-quality server
thanks to Bjoern's docker server
(https://github.com/bgruening/docker-galaxy-stable) - it comes bundled
with Postgres and Slurm so it should be able to handle the collection
operations. If you need to run Galaxy on a non-containerized server
(for instance because that is where the software is) more information
on setting up Galaxy can be found here
https://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer.
Here is a Trello card to track progress on the database optimization
efforts if you are interested https://trello.com/c/UPLsMKQI.
Very sorry.
On Mon, Jan 26, 2015 at 9:35 AM, Torsten Houwaart
<houwaart@informatik.uni-freiburg.de> wrote:
Hello Galaxy Devs,
I was using data collections (for the first time) for a new workflow of
ours
and I ran into this problem. There was no complaint by the
workflow-editor
and I could start the workflow but then <see below> happened.
If you need more information about the workflow or otherwise let me know.
Best,
Torsten H.
job traceback:
Traceback (most recent call last):
File
"/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/runners/__init__.py",
line 565, in finish_job
job_state.job_wrapper.finish( stdout, stderr, exit_code )
File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line
1250, in finish
self.sa_session.flush()
File
"/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/scoping.py",
line 114, in do
return getattr(self.registry(), name)(*args, **kwargs)
File
"/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/session.py",
line 1718, in flush
self._flush(objects)
File
"/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/session.py",
line 1789, in _flush
flush_context.execute()
File
"/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/unitofwork.py",
line 331, in execute
rec.execute(self)
File
"/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/unitofwork.py",
line 475, in execute
uow
File
"/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/persistence.py",
line 59, in save_obj
mapper, table, update)
File
"/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/orm/persistence.py",
line 485, in _emit_update_statements
execute(statement, params)
File
"/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py",
line 1449, in execute
params)
File
"/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py",
line 1584, in _execute_clauseelement
compiled_sql, distilled_params
File
"/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py",
line 1698, in _execute_context
context)
File
"/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/base.py",
line 1691, in _execute_context
context)
File
"/usr/local/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs2.egg/sqlalchemy/engine/default.py",
line 331, in do_execute
cursor.execute(statement, parameters)
DBAPIError: (TransactionRollbackError) deadlock detected
DETAIL: Process 3144 waits for ShareLock on transaction 2517124; blocked
by
process 3143.
Process 3143 waits for ShareLock on transaction 2517123; blocked by
process
3144.
HINT: See server log for query details.
'UPDATE workflow_invocation SET update_time=%(update_time)s WHERE
workflow_invocation.id = %(workflow_invocation_id)s' {'update_time':
datetime.datetime(2015, 1, 26, 14, 20, 4, 155440),
'workflow_invocation_id':
5454}
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
John Chilton
Torsten Houwaart