Hi John 

Thank you for the reply. 

In the lwr_staging directory all the runs from tophat are kept in (separate folders for each run) which means that the job failed to finish.

In the paster.log the most common error is

HTTPNotFound: No file found with path lwr_staging/<uniquejobstring>/working/insertions.bed

That's the case when tophat's output is set at default and the outputs are under working/tophat_out/.

If I change this in the wrapper and redirect the output in working/ I get the following error.

Traceback (most recent call last):
  File "/hermes-storage/galaxy-dist/lib/galaxy/jobs/runners/lwr.py", line 135, in finish_job
    client.download_work_dir_output(source_file, job_wrapper.working_directory, output_file)
  File "/hermes-storage/galaxy-dist/lib/galaxy/jobs/runners/lwr_client/client.py", line 146, in download_work_dir_output
    self.__raw_download_output(name, self.job_id, "work_dir", output)
  File "/hermes-storage/galaxy-dist/lib/galaxy/jobs/runners/lwr_client/client.py", line 174, in __raw_download_output
    output_path=output_path)
  File "/hermes-storage/galaxy-dist/lib/galaxy/jobs/runners/lwr_client/client.py", line 52, in __raw_execute
    response = self.transport.execute(url, data=data, input_path=input_path, output_path=output_path)
  File "/hermes-storage/galaxy-dist/lib/galaxy/jobs/runners/lwr_client/transport/standard.py", line 26, in execute
    with open(output_path, 'wb') as output: 
TypeError: coercing to Unicode: need string or buffer, file found

or one time I got this

Groomed 750000 sanger reads into sanger reads.
Based upon quality and sequence, the input data is valid for: solexa, sanger, illumina
Input ASCII range: 'B'(66) - 'b'(98) 
Input decimal range: 33 - 65

In the last two cases, no error was logged in lwr/paster.log


Bests,
Nikos 


2013/6/18 John Chilton <chilton@msi.umn.edu>
Hey Nikos,

Thanks for the interest in the LWR and frankly I am excited to hear
that bowtie worked properly. Some of these core NGS tools that require
implicit indices spanning multiple files are not really addressed by
the LWR currently, it only knows how to stage files that actually
appear in the command-line or a config file. I guess if you are just
using files from the history this is not a problem, but it is
something to be aware of if you plan to start using .loc files.

My first thought is that the LWR might not support 'from_work_dir'
outputs in nested subdirectories. I will have to look more closely at
that. Something that will help me though, is knowing if TopHat
produced the outputs.

Can you look at the LWR staging directory and tell me if these files
were created (by default when a job fails, it is not cleaned up this
is configurable in galaxy-central but not in galaxy-dist right now)?
Also, does the LWR have any useful logs available? These will likely
be in paster.log in the LWR directory.

Also, the tip of galaxy-central includes improved error logging and
exception handling so it is possible it would produce a better error
message than this admittedly cryptic problem. I know it is not always
possible to upgrade, but if it is easy it is something worth
considering doing.

-John

On Tue, Jun 18, 2013 at 8:18 AM, Nikos Sidiropoulos
<nikos.sidiro@gmail.com> wrote:
> Hi
>
> We are running the latest stable distribution on our server. We want tophat2
> (along with other tools) to be executed on another machine.
>
> LWR is setup based on the online documentation
> (http://wiki.galaxyproject.org/Admin/Config/LWR,
> https://lwr.readthedocs.org/en/latest/)
>
> Bowtie2 is working as should but it looks like LWR is unable to return the
> outputs from tophat2.
>
> In universe_wsgi.ini the runner is configured like this:
> [galaxy:tool_runners]
> bowtie2 = lwr://https://uniquestring@chaos:8913
> tophat2 = lwr://https://uniquestring@chaos:8913
>
> When running tophat2 the following error message is returned.
>
> Traceback (most recent call last):
>   File "/hermes-storage/galaxy-dist/lib/galaxy/jobs/runners/lwr.py", line
> 135, in finish_job
>     client.download_work_dir_output(source_file,
> job_wrapper.working_directory, output_file)
>   File
> "/hermes-storage/galaxy-dist/lib/galaxy/jobs/runners/lwr_client/client.py",
> line 146, in download_work_dir_output
>     self.__raw_download_output(name, self.job_id, "work_dir", output)
>   File
> "/hermes-storage/galaxy-dist/lib/galaxy/jobs/runners/lwr_client/client.py",
> line 174, in __raw_download_output
>     output_path=output_path)
>   File
> "/hermes-storage/galaxy-dist/lib/galaxy/jobs/runners/lwr_client/client.py",
> line 52, in __raw_execute
>     response = self.transport.execute(url, data=data, input_path=input_path,
> output_path=output_path)
>   File
> "/hermes-storage/galaxy-dist/lib/galaxy/jobs/runners/lwr_client/transport/standard.py",
> line 21, in execute
>     response = self._url_open(request, data)
>   File
> "/hermes-storage/galaxy-dist/lib/galaxy/jobs/runners/lwr_client/transport/standard.py",
> line 12, in _url_open
>     return urllib2.urlopen(request, data)
>   File "/usr/lib/python2.7/urllib2.py", line 127, in urlopen
>     return _opener.open(url, data, timeout)
>   File "/usr/lib/python2.7/urllib2.py", line 407, in open
>     response = meth(req, response)
>   File "/usr/lib/python2.7/urllib2.py", line 520, in http_response
>     'http', request, response, code, msg, hdrs)
>   File "/usr/lib/python2.7/urllib2.py", line 445, in error
>     return self._call_chain(*args)
>   File "/usr/lib/python2.7/urllib2.py", line 379, in _call_chain
>     result = func(*args)
>   File "/usr/lib/python2.7/urllib2.py", line 528, in http_error_default
>     raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
> HTTPError: HTTP Error 500: Internal Server Error
>
>
> We tried to specify tophat's output folder either directly in lwr's working
> or output directory without success.
>
> Is there a solution to this?
>
> Bests,
> Nikos
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/