Object-Store, setting filetypes crashes Galaxy
Hi, I have configured to use the hierarchical object store but as soon as I try to reset the filetpye of a dataset Galaxy is crashing with: galaxy.objectstore DEBUG 2014-06-12 14:39:21,180 Using preferred backend 'files3' for creation of MetadataFile 5963 132.230.153.57 - - [12/Jun/2014:14:39:20 +0200] "POST /datasets/966f24627ef70c12/edit HTTP/1.1" 500 - " http://galaxy.uni-freiburg.de/datasets/966f24627ef70c12/edit" "Mozilla/5.0 (X11; Linux x86_64; rv:29.0) Gecko/20100101 Fire fox/29.0" Error - <type 'exceptions.OSError'>: [Errno 2] No such file or directory: 'database/tmp/metadata_temp_file_1xnGcE' URL: http://galaxy.uni-freiburg.de/datasets/966f24627ef70c12/edit File '/usr/local/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/usr/local/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/usr/local/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/usr/local/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '/usr/local/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 190 in handle_request body = method( trans, **kwargs ) File '/usr/local/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/dataset.py', line 295 in edit trans.app.datatypes_registry.set_external_metadata_tool.tool_action.execute( trans.app.datatypes_registry.set_external_metadata_tool, trans, incoming = { 'input1':data }, overwrite = False ) #overwrite is False as per existi ng behavior File '/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/actions/metadata.py', line 18 in execute overwrite, history, job_params ) File '/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/actions/metadata.py', line 79 in execute_via_app kwds = { 'overwrite' : overwrite } ) File '/usr/local/galaxy/galaxy-dist/lib/galaxy/datatypes/metadata.py', line 717 in setup_external_metadata shutil.copy( dataset.metadata.get( meta_key, None ).file_name, metadata_temp.file_name ) File '/usr/local/galaxy/galaxy-dist/lib/galaxy/datatypes/metadata.py', line 575 in file_name self._filename = abspath( tempfile.NamedTemporaryFile( dir = self.tmp_dir, prefix = "metadata_temp_file_" ).name ) File '/usr/local/python/2.7/lib/python2.7/tempfile.py', line 454 in NamedTemporaryFile (fd, name) = _mkstemp_inner(dir, prefix, suffix, flags) File '/usr/local/python/2.7/lib/python2.7/tempfile.py', line 235 in _mkstemp_inner fd = _os.open(file, flags, 0600) OSError: [Errno 2] No such file or directory: 'database/tmp/metadata_temp_file_1xnGcE' I have attached my object_store_conf.xml file. Thanks, Bjoern
Hi Björn, I believe the problem is most likely this bug: https://trello.com/c/OHlQoCrx If you symlink /usr/local/galaxy/galaxy-dist/database/tmp to somewhere writeable (e.g. what you have new_files_path set to), that'd confirm it. I'm going to take a look at fixing this today. --nate On Thu, Jun 12, 2014 at 8:43 AM, bjoern.gruening@googlemail.com < bjoern.gruening@gmail.com> wrote:
Hi,
I have configured to use the hierarchical object store but as soon as I try to reset the filetpye of a dataset Galaxy is crashing with:
galaxy.objectstore DEBUG 2014-06-12 14:39:21,180 Using preferred backend 'files3' for creation of MetadataFile 5963 132.230.153.57 - - [12/Jun/2014:14:39:20 +0200] "POST /datasets/966f24627ef70c12/edit HTTP/1.1" 500 - " http://galaxy.uni-freiburg.de/datasets/966f24627ef70c12/edit" "Mozilla/5.0 (X11; Linux x86_64; rv:29.0) Gecko/20100101 Fire fox/29.0" Error - <type 'exceptions.OSError'>: [Errno 2] No such file or directory: 'database/tmp/metadata_temp_file_1xnGcE' URL: http://galaxy.uni-freiburg.de/datasets/966f24627ef70c12/edit File '/usr/local/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/usr/local/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/usr/local/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/usr/local/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '/usr/local/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 190 in handle_request body = method( trans, **kwargs ) File '/usr/local/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/dataset.py', line 295 in edit
trans.app.datatypes_registry.set_external_metadata_tool.tool_action.execute( trans.app.datatypes_registry.set_external_metadata_tool, trans, incoming = { 'input1':data }, overwrite = False ) #overwrite is False as per existi ng behavior File '/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/actions/metadata.py', line 18 in execute overwrite, history, job_params ) File '/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/actions/metadata.py', line 79 in execute_via_app kwds = { 'overwrite' : overwrite } ) File '/usr/local/galaxy/galaxy-dist/lib/galaxy/datatypes/metadata.py', line 717 in setup_external_metadata shutil.copy( dataset.metadata.get( meta_key, None ).file_name, metadata_temp.file_name ) File '/usr/local/galaxy/galaxy-dist/lib/galaxy/datatypes/metadata.py', line 575 in file_name self._filename = abspath( tempfile.NamedTemporaryFile( dir = self.tmp_dir, prefix = "metadata_temp_file_" ).name ) File '/usr/local/python/2.7/lib/python2.7/tempfile.py', line 454 in NamedTemporaryFile (fd, name) = _mkstemp_inner(dir, prefix, suffix, flags) File '/usr/local/python/2.7/lib/python2.7/tempfile.py', line 235 in _mkstemp_inner fd = _os.open(file, flags, 0600) OSError: [Errno 2] No such file or directory: 'database/tmp/metadata_temp_file_1xnGcE'
I have attached my object_store_conf.xml file. Thanks, Bjoern
Hi Nate, we encountered an addition bug related to the object-store: Traceback (most recent call last): File "/usr/local/galaxy/galaxy- dist/lib/galaxy/jobs/runners/__init__.py", line 153, in prepare_job job_wrapper.prepare() File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 712, in prepare self.command_line, self.extra_filenames = tool_evaluator.build() File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/evaluation.py", line 348, in build self.__build_command_line( ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/evaluation.py", line 364, in __build_command_line command_line = fill_template( command, context=param_dict ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/util/template.py", line 9, in fill_template return str( Template( source=template_text, searchList=[context] ) ) File "/usr/local/galaxy/galaxy-dist/eggs/Cheetah-2.2.2-py2.7-linux-x86_64-ucs2.egg/Cheetah/Template.py", line 1004, in __str__ return getattr(self, mainMethName)() File "cheetah_DynamicallyCompiledCheetahTemplate_1402663002_65_42228.py", line 94, in respond File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/wrappers.py", line 203, in __getattr__ return getattr( self.dataset, key ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/model/__init__.py", line 1493, in extra_files_path return self.dataset.extra_files_path File "/usr/local/galaxy/galaxy-dist/lib/galaxy/model/__init__.py", line 1349, in extra_files_path return self.object_store.get_filename( self, dir_only=True, extra_dir=self._extra_files_path or "dataset_%d_files" % self.id ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/objectstore/__init__.py", line 416, in get_filename return self.__call_method('get_filename', obj, ObjectNotFound, True, **kwargs) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/objectstore/__init__.py", line 436, in __call_method % ( method, str( obj ), str( kwargs ) ) ) ObjectNotFound: objectstore, __call_method failed: get_filename on <galaxy.model.Dataset object at 0x7f7ee103fb90>, kwargs: {'dir_only': True, 'extra_dir': 'dataset_118453_files'} It should be reproducible with the latest Galaxy stable release and the latest NCBI Blast+ wrappers if you are trying to generate a Blast+ database. Thanks, Bjoern 2014-06-12 16:50 GMT+02:00 Björn Grüning <bjoern.gruening@gmail.com>:
Hi Nate,
yes that is fixing it for me!
Thanks! Bjoern
Am 12.06.2014 16:26, schrieb Nate Coraor:
writeable (e.g. what you have new_files_path set to), that'd confirm it.
I can confirm this is error. Switching the tool to use $outfile.files_path instead of $outfile.extra_files_path improves the situation in my testing. I thought for sure .extra_files_path had been deprecated in terms of files_path - but the only Trello card I could find indicates the opposite may be true - https://trello.com/c/ar9zF40y? I think that for some more exotic features such as parameter rewriting by setting outputs_to_working_directory (required for running such tools are real users I suspect) and the LWR, usage of files_path in tool wrappers will be reworked for the job and extra_files_path will not be. Hopefully someone with a longer history on the project will comment on usage of .files_path versus .extra_files_path. -John On Fri, Jun 13, 2014 at 7:43 AM, bjoern.gruening@googlemail.com <bjoern.gruening@gmail.com> wrote:
Hi Nate,
we encountered an addition bug related to the object-store:
Traceback (most recent call last): File "/usr/local/galaxy/galaxy- dist/lib/galaxy/jobs/runners/__init__.py", line 153, in prepare_job job_wrapper.prepare() File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 712, in prepare self.command_line, self.extra_filenames = tool_evaluator.build() File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/evaluation.py", line 348, in build self.__build_command_line( ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/evaluation.py", line 364, in __build_command_line command_line = fill_template( command, context=param_dict ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/util/template.py", line 9, in fill_template return str( Template( source=template_text, searchList=[context] ) ) File "/usr/local/galaxy/galaxy-dist/eggs/Cheetah-2.2.2-py2.7-linux-x86_64-ucs2.egg/Cheetah/Template.py", line 1004, in __str__ return getattr(self, mainMethName)() File "cheetah_DynamicallyCompiledCheetahTemplate_1402663002_65_42228.py", line 94, in respond File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/wrappers.py", line 203, in __getattr__ return getattr( self.dataset, key ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/model/__init__.py", line 1493, in extra_files_path return self.dataset.extra_files_path File "/usr/local/galaxy/galaxy-dist/lib/galaxy/model/__init__.py", line 1349, in extra_files_path return self.object_store.get_filename( self, dir_only=True, extra_dir=self._extra_files_path or "dataset_%d_files" % self.id ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/objectstore/__init__.py", line 416, in get_filename return self.__call_method('get_filename', obj, ObjectNotFound, True, **kwargs) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/objectstore/__init__.py", line 436, in __call_method % ( method, str( obj ), str( kwargs ) ) ) ObjectNotFound: objectstore, __call_method failed: get_filename on <galaxy.model.Dataset object at 0x7f7ee103fb90>, kwargs: {'dir_only': True, 'extra_dir': 'dataset_118453_files'}
It should be reproducible with the latest Galaxy stable release and the latest NCBI Blast+ wrappers if you are trying to generate a Blast+ database.
Thanks, Bjoern
2014-06-12 16:50 GMT+02:00 Björn Grüning <bjoern.gruening@gmail.com>:
Hi Nate,
yes that is fixing it for me!
Thanks! Bjoern
Am 12.06.2014 16:26, schrieb Nate Coraor:
writeable (e.g. what you have new_files_path set to), that'd confirm it.
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Nate, John, that bug bites me with so many tools. Next one is MACS2 Traceback (most recent call last): File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/runners/__init__.py", line 153, in prepare_job job_wrapper.prepare() File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 712, in prepare self.command_line, self.extra_filenames = tool_evaluator.build() File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/evaluation.py", line 348, in build self.__build_command_line( ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/evaluation.py", line 364, in __build_command_line command_line = fill_template( command, context=param_dict ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/util/template.py", line 9, in fill_template return str( Template( source=template_text, searchList=[context] ) ) File "/usr/local/galaxy/galaxy-dist/eggs/Cheetah-2.2.2-py2.7-linux-x86_64-ucs2.egg/Cheetah/Template.py", line 1004, in __str__ return getattr(self, mainMethName)() File "cheetah_DynamicallyCompiledCheetahTemplate_1403081606_9_13081.py", line 253, in respond File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/wrappers.py", line 203, in __getattr__ return getattr( self.dataset, key ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/model/__init__.py", line 1493, in extra_files_path return self.dataset.extra_files_path File "/usr/local/galaxy/galaxy-dist/lib/galaxy/model/__init__.py", line 1349, in extra_files_path return self.object_store.get_filename( self, dir_only=True, extra_dir=self._extra_files_path or "dataset_%d_files" % self.id ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/objectstore/__init__.py", line 416, in get_filename return self.__call_method('get_filename', obj, ObjectNotFound, True, **kwargs) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/objectstore/__init__.py", line 436, in __call_method % ( method, str( obj ), str( kwargs ) ) ) ObjectNotFound: objectstore, __call_method failed: get_filename on <galaxy.model.Dataset object at 0x7eff89510310>, kwargs: {'dir_only': True, 'extra_dir': 'dataset_118783_files'} Is there anything I can do? It is affecting our productive server right now. Any quick fix would be very much appreciated. The object store is not a beta feature or? I'm wondering who is using it at the moment? Cheers, Bjoern Am 16.06.2014 16:04, schrieb John Chilton:
I can confirm this is error. Switching the tool to use $outfile.files_path instead of $outfile.extra_files_path improves the situation in my testing.
I thought for sure .extra_files_path had been deprecated in terms of files_path - but the only Trello card I could find indicates the opposite may be true - https://trello.com/c/ar9zF40y? I think that for some more exotic features such as parameter rewriting by setting outputs_to_working_directory (required for running such tools are real users I suspect) and the LWR, usage of files_path in tool wrappers will be reworked for the job and extra_files_path will not be.
Hopefully someone with a longer history on the project will comment on usage of .files_path versus .extra_files_path.
-John
On Fri, Jun 13, 2014 at 7:43 AM, bjoern.gruening@googlemail.com <bjoern.gruening@gmail.com> wrote:
Hi Nate,
we encountered an addition bug related to the object-store:
Traceback (most recent call last): File "/usr/local/galaxy/galaxy- dist/lib/galaxy/jobs/runners/__init__.py", line 153, in prepare_job job_wrapper.prepare() File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 712, in prepare self.command_line, self.extra_filenames = tool_evaluator.build() File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/evaluation.py", line 348, in build self.__build_command_line( ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/evaluation.py", line 364, in __build_command_line command_line = fill_template( command, context=param_dict ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/util/template.py", line 9, in fill_template return str( Template( source=template_text, searchList=[context] ) ) File "/usr/local/galaxy/galaxy-dist/eggs/Cheetah-2.2.2-py2.7-linux-x86_64-ucs2.egg/Cheetah/Template.py", line 1004, in __str__ return getattr(self, mainMethName)() File "cheetah_DynamicallyCompiledCheetahTemplate_1402663002_65_42228.py", line 94, in respond File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/wrappers.py", line 203, in __getattr__ return getattr( self.dataset, key ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/model/__init__.py", line 1493, in extra_files_path return self.dataset.extra_files_path File "/usr/local/galaxy/galaxy-dist/lib/galaxy/model/__init__.py", line 1349, in extra_files_path return self.object_store.get_filename( self, dir_only=True, extra_dir=self._extra_files_path or "dataset_%d_files" % self.id ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/objectstore/__init__.py", line 416, in get_filename return self.__call_method('get_filename', obj, ObjectNotFound, True, **kwargs) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/objectstore/__init__.py", line 436, in __call_method % ( method, str( obj ), str( kwargs ) ) ) ObjectNotFound: objectstore, __call_method failed: get_filename on <galaxy.model.Dataset object at 0x7f7ee103fb90>, kwargs: {'dir_only': True, 'extra_dir': 'dataset_118453_files'}
It should be reproducible with the latest Galaxy stable release and the latest NCBI Blast+ wrappers if you are trying to generate a Blast+ database.
Thanks, Bjoern
2014-06-12 16:50 GMT+02:00 Björn Grüning <bjoern.gruening@gmail.com>:
Hi Nate,
yes that is fixing it for me!
Thanks! Bjoern
Am 12.06.2014 16:26, schrieb Nate Coraor:
writeable (e.g. what you have new_files_path set to), that'd confirm it.
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Bjoern, The following patch should put out the fire... https://bitbucket.org/galaxy/galaxy-central/commits/d781366bc120787e201b73a4... TL;DR - After this patch $output.extra_files_path is the same thing as $output.files_path - which is to say not broken in a bunch of corner cases and slightly less efficient in the most vanilla Galaxy deployments. $input.extra_files_path however is a bit tricky if the path doesn't exist (probably not the case with your deployment Bjoern) - for disk object stores it just returns the non-existent path - for the nested object store it throws an error and the tool won't run. I had a patch to synchronize the behavior (https://gist.github.com/jmchilton/d05bcf02874762cdb829) - but I cannot say for sure the current behavior isn't intentional (Dannon, Nate - want to weigh in on this?). -John On Thu, Jun 19, 2014 at 5:16 PM, Björn Grüning <bjoern.gruening@gmail.com> wrote:
Nate, John,
that bug bites me with so many tools. Next one is MACS2
Traceback (most recent call last): File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/runners/__init__.py", line 153, in prepare_job
job_wrapper.prepare() File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 712, in prepare self.command_line, self.extra_filenames = tool_evaluator.build() File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/evaluation.py", line 348, in build self.__build_command_line( ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/evaluation.py", line 364, in __build_command_line command_line = fill_template( command, context=param_dict ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/util/template.py", line 9, in fill_template return str( Template( source=template_text, searchList=[context] ) ) File "/usr/local/galaxy/galaxy-dist/eggs/Cheetah-2.2.2-py2.7-linux-x86_64-ucs2.egg/Cheetah/Template.py", line 1004, in __str__ return getattr(self, mainMethName)() File "cheetah_DynamicallyCompiledCheetahTemplate_1403081606_9_13081.py", line 253, in respond
File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/wrappers.py", line 203, in __getattr__ return getattr( self.dataset, key ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/model/__init__.py", line 1493, in extra_files_path return self.dataset.extra_files_path File "/usr/local/galaxy/galaxy-dist/lib/galaxy/model/__init__.py", line 1349, in extra_files_path return self.object_store.get_filename( self, dir_only=True, extra_dir=self._extra_files_path or "dataset_%d_files" % self.id ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/objectstore/__init__.py", line 416, in get_filename return self.__call_method('get_filename', obj, ObjectNotFound, True, **kwargs) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/objectstore/__init__.py", line 436, in __call_method % ( method, str( obj ), str( kwargs ) ) ) ObjectNotFound: objectstore, __call_method failed: get_filename on <galaxy.model.Dataset object at 0x7eff89510310>, kwargs: {'dir_only': True, 'extra_dir': 'dataset_118783_files'}
Is there anything I can do? It is affecting our productive server right now. Any quick fix would be very much appreciated. The object store is not a beta feature or? I'm wondering who is using it at the moment?
Cheers, Bjoern
Am 16.06.2014 16:04, schrieb John Chilton:
I can confirm this is error. Switching the tool to use $outfile.files_path instead of $outfile.extra_files_path improves the situation in my testing.
I thought for sure .extra_files_path had been deprecated in terms of files_path - but the only Trello card I could find indicates the opposite may be true - https://trello.com/c/ar9zF40y? I think that for some more exotic features such as parameter rewriting by setting outputs_to_working_directory (required for running such tools are real users I suspect) and the LWR, usage of files_path in tool wrappers will be reworked for the job and extra_files_path will not be.
Hopefully someone with a longer history on the project will comment on usage of .files_path versus .extra_files_path.
-John
On Fri, Jun 13, 2014 at 7:43 AM, bjoern.gruening@googlemail.com <bjoern.gruening@gmail.com> wrote:
Hi Nate,
we encountered an addition bug related to the object-store:
Traceback (most recent call last): File "/usr/local/galaxy/galaxy- dist/lib/galaxy/jobs/runners/__init__.py", line 153, in prepare_job job_wrapper.prepare() File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 712, in prepare self.command_line, self.extra_filenames = tool_evaluator.build() File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/evaluation.py", line 348, in build self.__build_command_line( ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/evaluation.py", line 364, in __build_command_line command_line = fill_template( command, context=param_dict ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/util/template.py", line 9, in fill_template return str( Template( source=template_text, searchList=[context] ) ) File
"/usr/local/galaxy/galaxy-dist/eggs/Cheetah-2.2.2-py2.7-linux-x86_64-ucs2.egg/Cheetah/Template.py", line 1004, in __str__ return getattr(self, mainMethName)() File "cheetah_DynamicallyCompiledCheetahTemplate_1402663002_65_42228.py", line 94, in respond File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/wrappers.py", line 203, in __getattr__ return getattr( self.dataset, key ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/model/__init__.py", line 1493, in extra_files_path return self.dataset.extra_files_path File "/usr/local/galaxy/galaxy-dist/lib/galaxy/model/__init__.py", line 1349, in extra_files_path return self.object_store.get_filename( self, dir_only=True, extra_dir=self._extra_files_path or "dataset_%d_files" % self.id ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/objectstore/__init__.py", line 416, in get_filename return self.__call_method('get_filename', obj, ObjectNotFound, True, **kwargs) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/objectstore/__init__.py", line 436, in __call_method % ( method, str( obj ), str( kwargs ) ) ) ObjectNotFound: objectstore, __call_method failed: get_filename on <galaxy.model.Dataset object at 0x7f7ee103fb90>, kwargs: {'dir_only': True, 'extra_dir': 'dataset_118453_files'}
It should be reproducible with the latest Galaxy stable release and the latest NCBI Blast+ wrappers if you are trying to generate a Blast+ database.
Thanks, Bjoern
2014-06-12 16:50 GMT+02:00 Björn Grüning <bjoern.gruening@gmail.com>:
Hi Nate,
yes that is fixing it for me!
Thanks! Bjoern
Am 12.06.2014 16:26, schrieb Nate Coraor:
writeable (e.g. what you have new_files_path set to), that'd confirm it.
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Hello all, This is resurrecting an old thread, but using $input.extra_files_path rather than $input.files_path explained an unfortunate regression in the BLAST+ wrappers reported earlier today: https://github.com/peterjc/galaxy_blast/issues/69 Thanks John for his detective work. Given $output.files_path is considered preferable over $output.extra_files_path it would be great to have the same alias available for input parameters for consistency (although as per John's old email below, this may be tricky). Regards, Peter On Fri, Jun 20, 2014 at 5:15 AM, John Chilton <jmchilton@gmail.com> wrote:
Bjoern,
The following patch should put out the fire...
https://bitbucket.org/galaxy/galaxy-central/commits/d781366bc120787e201b73a4...
TL;DR - After this patch $output.extra_files_path is the same thing as $output.files_path - which is to say not broken in a bunch of corner cases and slightly less efficient in the most vanilla Galaxy deployments.
$input.extra_files_path however is a bit tricky if the path doesn't exist (probably not the case with your deployment Bjoern) - for disk object stores it just returns the non-existent path - for the nested object store it throws an error and the tool won't run. I had a patch to synchronize the behavior (https://gist.github.com/jmchilton/d05bcf02874762cdb829) - but I cannot say for sure the current behavior isn't intentional (Dannon, Nate - want to weigh in on this?).
-John
participants (5)
-
bjoern.gruening＠googlemail.com -
Björn Grüning
-
John Chilton -
Nate Coraor -
Peter Cock