Hey Nikos, I have been meaning to test tophat with the LWR but I haven't been able to get to it and I doubt I will before the Galaxy conference. The one more piece of advice I have (in addition to trying this out with galaxy-central) is that in the staging directory there are going to be some files with odd names (tmpXXXX) or something like that, these files should contain the standard error and standard output of the tophat run so they will hopefully provide some insight into what is going wrong. -John On Fri, Jun 21, 2013 at 8:21 AM, Nikos Sidiropoulos <nikos.sidiro@gmail.com> wrote:

Hi John

Thank you for the reply.

In the lwr_staging directory all the runs from tophat are kept in (separate folders for each run) which means that the job failed to finish.

In the paster.log the most common error is

HTTPNotFound: No file found with path lwr_staging/<uniquejobstring>/working/insertions.bed

That's the case when tophat's output is set at default and the outputs are under working/tophat_out/.

If I change this in the wrapper and redirect the output in working/ I get the following error.

Traceback (most recent call last): File "/hermes-storage/galaxy-dist/lib/galaxy/jobs/runners/lwr.py", line 135, in finish_job client.download_work_dir_output(source_file, job_wrapper.working_directory, output_file) File "/hermes-storage/galaxy-dist/lib/galaxy/jobs/runners/lwr_client/client.py", line 146, in download_work_dir_output self.__raw_download_output(name, self.job_id, "work_dir", output) File "/hermes-storage/galaxy-dist/lib/galaxy/jobs/runners/lwr_client/client.py", line 174, in __raw_download_output output_path=output_path) File "/hermes-storage/galaxy-dist/lib/galaxy/jobs/runners/lwr_ client/client.py", line 52, in __raw_execute response = self.transport.execute(url, data=data, input_path=input_path, output_path=output_path) File "/hermes-storage/galaxy-dist/lib/galaxy/jobs/runners/lwr_client/transport/standard.py", line 26, in execute with open(output_path, 'wb') as output:

TypeError: coercing to Unicode: need string or buffer, file found

or one time I got this

Groomed 750000 sanger reads into sanger reads. Based upon quality and sequence, the input data is valid for: solexa, sanger, illumina Input ASCII range: 'B'(66) - 'b'(98)

Input decimal range: 33 - 65

In the last two cases, no error was logged in lwr/paster.log

Bests, Nikos

2013/6/18 John Chilton <chilton@msi.umn.edu>

...

Hey Nikos,

Thanks for the interest in the LWR and frankly I am excited to hear that bowtie worked properly. Some of these core NGS tools that require implicit indices spanning multiple files are not really addressed by the LWR currently, it only knows how to stage files that actually appear in the command-line or a config file. I guess if you are just using files from the history this is not a problem, but it is something to be aware of if you plan to start using .loc files.

My first thought is that the LWR might not support 'from_work_dir' outputs in nested subdirectories. I will have to look more closely at that. Something that will help me though, is knowing if TopHat produced the outputs.

Can you look at the LWR staging directory and tell me if these files were created (by default when a job fails, it is not cleaned up this is configurable in galaxy-central but not in galaxy-dist right now)? Also, does the LWR have any useful logs available? These will likely be in paster.log in the LWR directory.

Also, the tip of galaxy-central includes improved error logging and exception handling so it is possible it would produce a better error message than this admittedly cryptic problem. I know it is not always possible to upgrade, but if it is easy it is something worth considering doing.

-John

On Tue, Jun 18, 2013 at 8:18 AM, Nikos Sidiropoulos <nikos.sidiro@gmail.com> wrote:

...

Hi

We are running the latest stable distribution on our server. We want tophat2 (along with other tools) to be executed on another machine.

LWR is setup based on the online documentation (http://wiki.galaxyproject.org/Admin/Config/LWR, https://lwr.readthedocs.org/en/latest/)

Bowtie2 is working as should but it looks like LWR is unable to return the outputs from tophat2.

In universe_wsgi.ini the runner is configured like this: [galaxy:tool_runners] bowtie2 = lwr://https://uniquestring@chaos:8913 tophat2 = lwr://https://uniquestring@chaos:8913

When running tophat2 the following error message is returned.

Traceback (most recent call last): File "/hermes-storage/galaxy-dist/lib/galaxy/jobs/runners/lwr.py", line 135, in finish_job client.download_work_dir_output(source_file, job_wrapper.working_directory, output_file) File

"/hermes-storage/galaxy-dist/lib/galaxy/jobs/runners/lwr_client/client.py", line 146, in download_work_dir_output self.__raw_download_output(name, self.job_id, "work_dir", output) File

"/hermes-storage/galaxy-dist/lib/galaxy/jobs/runners/lwr_client/client.py", line 174, in __raw_download_output output_path=output_path) File

"/hermes-storage/galaxy-dist/lib/galaxy/jobs/runners/lwr_client/client.py", line 52, in __raw_execute response = self.transport.execute(url, data=data, input_path=input_path, output_path=output_path) File

"/hermes-storage/galaxy-dist/lib/galaxy/jobs/runners/lwr_client/transport/standard.py", line 21, in execute response = self._url_open(request, data) File

"/hermes-storage/galaxy-dist/lib/galaxy/jobs/runners/lwr_client/transport/standard.py", line 12, in _url_open return urllib2.urlopen(request, data) File "/usr/lib/python2.7/urllib2.py", line 127, in urlopen return _opener.open(url, data, timeout) File "/usr/lib/python2.7/urllib2.py", line 407, in open response = meth(req, response) File "/usr/lib/python2.7/urllib2.py", line 520, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python2.7/urllib2.py", line 445, in error return self._call_chain(*args) File "/usr/lib/python2.7/urllib2.py", line 379, in _call_chain result = func(*args) File "/usr/lib/python2.7/urllib2.py", line 528, in http_error_default raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) HTTPError: HTTP Error 500: Internal Server Error

We tried to specify tophat's output folder either directly in lwr's working or output directory without success.

Is there a solution to this?

Bests, Nikos

___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/