Bjoern, The following patch should put out the fire... https://bitbucket.org/galaxy/galaxy-central/commits/d781366bc120787e201b73a4... TL;DR - After this patch $output.extra_files_path is the same thing as $output.files_path - which is to say not broken in a bunch of corner cases and slightly less efficient in the most vanilla Galaxy deployments. $input.extra_files_path however is a bit tricky if the path doesn't exist (probably not the case with your deployment Bjoern) - for disk object stores it just returns the non-existent path - for the nested object store it throws an error and the tool won't run. I had a patch to synchronize the behavior (https://gist.github.com/jmchilton/d05bcf02874762cdb829) - but I cannot say for sure the current behavior isn't intentional (Dannon, Nate - want to weigh in on this?). -John On Thu, Jun 19, 2014 at 5:16 PM, Björn Grüning <bjoern.gruening@gmail.com> wrote:
Nate, John,
that bug bites me with so many tools. Next one is MACS2
Traceback (most recent call last): File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/runners/__init__.py", line 153, in prepare_job
job_wrapper.prepare() File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 712, in prepare self.command_line, self.extra_filenames = tool_evaluator.build() File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/evaluation.py", line 348, in build self.__build_command_line( ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/evaluation.py", line 364, in __build_command_line command_line = fill_template( command, context=param_dict ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/util/template.py", line 9, in fill_template return str( Template( source=template_text, searchList=[context] ) ) File "/usr/local/galaxy/galaxy-dist/eggs/Cheetah-2.2.2-py2.7-linux-x86_64-ucs2.egg/Cheetah/Template.py", line 1004, in __str__ return getattr(self, mainMethName)() File "cheetah_DynamicallyCompiledCheetahTemplate_1403081606_9_13081.py", line 253, in respond
File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/wrappers.py", line 203, in __getattr__ return getattr( self.dataset, key ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/model/__init__.py", line 1493, in extra_files_path return self.dataset.extra_files_path File "/usr/local/galaxy/galaxy-dist/lib/galaxy/model/__init__.py", line 1349, in extra_files_path return self.object_store.get_filename( self, dir_only=True, extra_dir=self._extra_files_path or "dataset_%d_files" % self.id ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/objectstore/__init__.py", line 416, in get_filename return self.__call_method('get_filename', obj, ObjectNotFound, True, **kwargs) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/objectstore/__init__.py", line 436, in __call_method % ( method, str( obj ), str( kwargs ) ) ) ObjectNotFound: objectstore, __call_method failed: get_filename on <galaxy.model.Dataset object at 0x7eff89510310>, kwargs: {'dir_only': True, 'extra_dir': 'dataset_118783_files'}
Is there anything I can do? It is affecting our productive server right now. Any quick fix would be very much appreciated. The object store is not a beta feature or? I'm wondering who is using it at the moment?
Cheers, Bjoern
Am 16.06.2014 16:04, schrieb John Chilton:
I can confirm this is error. Switching the tool to use $outfile.files_path instead of $outfile.extra_files_path improves the situation in my testing.
I thought for sure .extra_files_path had been deprecated in terms of files_path - but the only Trello card I could find indicates the opposite may be true - https://trello.com/c/ar9zF40y? I think that for some more exotic features such as parameter rewriting by setting outputs_to_working_directory (required for running such tools are real users I suspect) and the LWR, usage of files_path in tool wrappers will be reworked for the job and extra_files_path will not be.
Hopefully someone with a longer history on the project will comment on usage of .files_path versus .extra_files_path.
-John
On Fri, Jun 13, 2014 at 7:43 AM, bjoern.gruening@googlemail.com <bjoern.gruening@gmail.com> wrote:
Hi Nate,
we encountered an addition bug related to the object-store:
Traceback (most recent call last): File "/usr/local/galaxy/galaxy- dist/lib/galaxy/jobs/runners/__init__.py", line 153, in prepare_job job_wrapper.prepare() File "/usr/local/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py", line 712, in prepare self.command_line, self.extra_filenames = tool_evaluator.build() File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/evaluation.py", line 348, in build self.__build_command_line( ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/evaluation.py", line 364, in __build_command_line command_line = fill_template( command, context=param_dict ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/util/template.py", line 9, in fill_template return str( Template( source=template_text, searchList=[context] ) ) File
"/usr/local/galaxy/galaxy-dist/eggs/Cheetah-2.2.2-py2.7-linux-x86_64-ucs2.egg/Cheetah/Template.py", line 1004, in __str__ return getattr(self, mainMethName)() File "cheetah_DynamicallyCompiledCheetahTemplate_1402663002_65_42228.py", line 94, in respond File "/usr/local/galaxy/galaxy-dist/lib/galaxy/tools/wrappers.py", line 203, in __getattr__ return getattr( self.dataset, key ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/model/__init__.py", line 1493, in extra_files_path return self.dataset.extra_files_path File "/usr/local/galaxy/galaxy-dist/lib/galaxy/model/__init__.py", line 1349, in extra_files_path return self.object_store.get_filename( self, dir_only=True, extra_dir=self._extra_files_path or "dataset_%d_files" % self.id ) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/objectstore/__init__.py", line 416, in get_filename return self.__call_method('get_filename', obj, ObjectNotFound, True, **kwargs) File "/usr/local/galaxy/galaxy-dist/lib/galaxy/objectstore/__init__.py", line 436, in __call_method % ( method, str( obj ), str( kwargs ) ) ) ObjectNotFound: objectstore, __call_method failed: get_filename on <galaxy.model.Dataset object at 0x7f7ee103fb90>, kwargs: {'dir_only': True, 'extra_dir': 'dataset_118453_files'}
It should be reproducible with the latest Galaxy stable release and the latest NCBI Blast+ wrappers if you are trying to generate a Blast+ database.
Thanks, Bjoern
2014-06-12 16:50 GMT+02:00 Björn Grüning <bjoern.gruening@gmail.com>:
Hi Nate,
yes that is fixing it for me!
Thanks! Bjoern
Am 12.06.2014 16:26, schrieb Nate Coraor:
writeable (e.g. what you have new_files_path set to), that'd confirm it.
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/