Thanks for your input and patience, Holger!

Please try the new version 0.4 of the weblogo wrapper in galaxy-central #5772 - it has additional error reporting that may help clarify dependency or other problems and let me know how you go?

On Tue, Jul 5, 2011 at 5:49 PM, Holger Klein h.klein@imb-mainz.de wrote:

Hi Ross,

thanks for taking care of this issue.

On 07/05/2011 12:31 AM, Ross wrote:

...

Is this error seen on Galaxy main or test? If so please share the history with me so I can see the input and reproduce what sounds like a wrapper error?

Otherwise, if this is on a private instance, and if the tool has never produced output successfully, then this may be a dependency installation problem - eg you may need to ensure that the weblogo3 executable is available and working correctly on the path used by your execution nodes. To assure yourself that your data works with the tool, please try running it on main using the same data, and let me know what you see?

in fact it's a private instance of galaxy, it's the latest version of galaxy-dist (hg summary: 5743:720455407d1c). The input data is fine, it's a clustalw alignment in fasta format which can be used by the weblogo module on galaxy main.

Maybe some background info on the weblogo installation helps: it's located below the tool_dependency_dir as defined in universe_wsgi.ini in weblogo/3.0 (with default as a link to 3.0). It contains the file env.sh which sets the PATH: export PATH=/home/galaxy/dependencies/weblogo/3.0:$PATH

Starting the weblogo executable with the galaxy virtualenv python seems to work (just tested --help), although it returns a warning:

~/python/bin/python ./weblogo --help/home/galaxy/python/lib/python2.6/site-packages/CoreBio-0.5.0-py2.6.egg/corebio/seq_io/_nexus/__init__.py:19: DeprecationWarning: the sets module is deprecated  import sets

I also tested putting a link to the weblogo executable in the PATH that's defined for the galaxy user (as opposed to the dependency dir mechanism, that I have to admit I don't fully understand yet), but that also doesn't work. Could this be an issue of PYTHONPATH needing to be adjusted?

Regards, Holger

...

Thanks again.

On Tue, Jul 5, 2011 at 12:34 AM, Holger Klein h.klein@imb-mainz.de wrote:

...

Dear all,

I have a problem with the weblogo tool. I have a clustalw alignment in fasta format that I'd like to visualize as a logo. The sequence logo module ends with a success (green box), the info tells me the amount and length of the input data. But the output is empty, there are no plots (no matter if I select jpg, png, pdf or text). The respective image can't be displayed "because it contains errors" or is empty in case of text.

I suspect that the actual call of the weblogo tool doesn't succeed, but I didn't figure out yet on how to check this. Does anybody have hints on where to look?

Cheers, Holger

Holger, thanks for finding those errors - I'll take a look shortly.

What's the 'dependency directory' ? I don't think the wrapper knows anything about it. The tool assumes that the weblogo script is on the path when the actual job starts executing - wherever that is.

If the weblogo script produces output it must be on the path after you source that script and working I think. So, the environment on the execution node must include the relevant path. Otherwise it won't work.

What path does the execution host get when a galaxy job is run? Does it include the right path to that weblogo script (marked executable)? Can the user each job runs as execute it?

On Thu, Jul 7, 2011 at 7:56 PM, Holger Klein h.klein@imb-mainz.de wrote:

Hi Ross,

On 07/07/2011 02:07 AM, Ross wrote:

...

Please try the new version 0.4 of the weblogo wrapper in galaxy-central #5772 - it has additional error reporting that may help clarify dependency or other problems and let me know how you go?

thanks, with the new version I get some more hints. It seems that there is a problem with the path.

Just having weblogo installed in the dependency directory and using the env.sh mechanism to set the path, the wrapper doesn't find the executable at all: --->%--- ## rgWebLogo3.py error - cannot locate the weblogo binary weblogo on the current path ## Please ensure it is installed and working from http://code.google.com/p/weblogo ---%<---

When I put a soft link to a directory which is in the galaxy user's static path, I get a different error: --->%--- Traceback (most recent call last):  File "/local/data/home/galaxy/galaxy-dist/tools/rgenetics/rgWebLogo3.py", line 156, in    checks,s = w.run()  File "/local/data/home/galaxy/galaxy-dist/tools/rgenetics/rgWebLogo3.py", line 127, in run    s = self.runCL()  File "/local/data/home/galaxy/galaxy-dist/tools/rgenetics/rgWebLogo3.py", line 47, in runCL    print >> sys.stderr, '## rgWebLogo3.py error - executing %s returned error code %d' % cl TypeError: not enough arguments for format string ---%<---

Line 47 seems to lack the variable for the return code, when changing the line to

print >> sys.stderr, '## rgWebLogo3.py error - executing %s returned error code %d' % (cl, rval)

I get the following message: --->%--- ## rgWebLogo3.py error - executing weblogo -F png -c auto -o /local/data/galaxy_files/000/dataset_304.dat -U bits -t "Galaxy-Rgenetics Sequence Logo" -f /local/data/galaxy_files/000/dataset_286.dat -s large returned error code 1 ## This may be a data problem or a tool dependency (weblogo) installation problem ## Please ensure weblogo is correctly installed and working on the command line -see http://code.google.com/p/weblogo ---%<---

So it still seems to boil down to my local weblogo installation. Sourcing the respective env.sh and executing the above command line, I get a valid png though (again with the warning mentioned before): --->%--- galaxy@imbc1:~/tmp/1$ source ~/dependencies/weblogo/default/env.sh galaxy@imbc1:~/tmp/1$ weblogo -F png -c auto -o /local/data/galaxy_files/000/dataset_304.dat -U bits -t "Galaxy-Rgenetics Sequence Logo" -f /local/data/galaxy_files/000/dataset_286.dat -s large /home/galaxy/python/lib/python2.6/site-packages/CoreBio-0.5.0-py2.6.egg/corebio/seq_io/_nexus/__init__.py:19: DeprecationWarning: the sets module is deprecated  import sets ---%<---

Could this be related to the way galaxy is setting the paths dynamically using the env.sh file? Do I have to adjust python paths in there as well?

Regards, Holger

...

On Tue, Jul 5, 2011 at 5:49 PM, Holger Klein h.klein@imb-mainz.de wrote:

...

Hi Ross,

thanks for taking care of this issue.

On 07/05/2011 12:31 AM, Ross wrote:

...

Is this error seen on Galaxy main or test? If so please share the history with me so I can see the input and reproduce what sounds like a wrapper error?

Otherwise, if this is on a private instance, and if the tool has never produced output successfully, then this may be a dependency installation problem - eg you may need to ensure that the weblogo3 executable is available and working correctly on the path used by your execution nodes. To assure yourself that your data works with the tool, please try running it on main using the same data, and let me know what you see?

in fact it's a private instance of galaxy, it's the latest version of galaxy-dist (hg summary: 5743:720455407d1c). The input data is fine, it's a clustalw alignment in fasta format which can be used by the weblogo module on galaxy main.

Maybe some background info on the weblogo installation helps: it's located below the tool_dependency_dir as defined in universe_wsgi.ini in weblogo/3.0 (with default as a link to 3.0). It contains the file env.sh which sets the PATH: export PATH=/home/galaxy/dependencies/weblogo/3.0:$PATH

Starting the weblogo executable with the galaxy virtualenv python seems to work (just tested --help), although it returns a warning:

~/python/bin/python ./weblogo --help/home/galaxy/python/lib/python2.6/site-packages/CoreBio-0.5.0-py2.6.egg/corebio/seq_io/_nexus/__init__.py:19: DeprecationWarning: the sets module is deprecated  import sets

I also tested putting a link to the weblogo executable in the PATH that's defined for the galaxy user (as opposed to the dependency dir mechanism, that I have to admit I don't fully understand yet), but that also doesn't work. Could this be an issue of PYTHONPATH needing to be adjusted?

Regards, Holger

...

Thanks again.

On Tue, Jul 5, 2011 at 12:34 AM, Holger Klein h.klein@imb-mainz.de wrote:

...

Dear all,

I have a problem with the weblogo tool. I have a clustalw alignment in fasta format that I'd like to visualize as a logo. The sequence logo module ends with a success (green box), the info tells me the amount and length of the input data. But the output is empty, there are no plots (no matter if I select jpg, png, pdf or text). The respective image can't be displayed "because it contains errors" or is empty in case of text.

I suspect that the actual call of the weblogo tool doesn't succeed, but I didn't figure out yet on how to check this. Does anybody have hints on where to look?

Cheers, Holger

-- Dr. Holger Klein Core Facility Bioinformatics Institute of Molecular Biology gGmbH (IMB) http://www.imb-mainz.de/ Tel: +49(6131) 39 21511

AFAIK, the requirements stuff is still work in progress?

Yes, of course you're right - there has to be a better way - particularly where there are complex inter-dependencies like weblogo/python/corebio

On Thu, Jul 7, 2011 at 10:41 PM, Holger Klein h.klein@imb-mainz.de wrote:

Hi Ross,

On 07/07/2011 12:38 PM, Ross wrote:

...

Holger, thanks for finding those errors - I'll take a look shortly.

What's the 'dependency directory' ? I don't think the wrapper knows anything about it.

it's defined in universe_wsgi.ini:

# The directory containing tool dependencies tool_dependency_dir = /local/data/home/galaxy/dependencies

We only recently installed galaxy locally, and Nate pointed me towards this way to handle external tool dependencies. This issue here is related: https://bitbucket.org/galaxy/galaxy-central/issue/82/fix-the-tag-set-in-the-...

Until now when tools didn't work with that mechanism (CCAT, clustalw) I simply put a link into a directory which is in galaxy's path.

...

The tool assumes that the weblogo script is on the path when the actual job starts executing - wherever that is.

If the weblogo script produces output it must be on the path after you source that script and working I think. So, the environment on the execution node must include the relevant path. Otherwise it won't work.

What path does the execution host get when a galaxy job is run? Does it include the right path to that weblogo script (marked executable)? Can the user each job runs as execute it?

I got a step further. The above mentioned mechanism with putting a link to the weblogo executable simply into the path didn't work, because the weblogo script uses the system-wide python (from #!/usr/bin/env python) which doesn't have the corebio module installed. When changing the interpreter to #!/home/galaxy/python/bin/python (galaxy-specific virtualenv) it works. Somehow I just assumed that with setting the PYTHON variable in the startup script would be sufficient.

So, now it works, but I wonder if there's a mechanism that is cleaner than hard-coding the python interpreter? Is there a way to tell galaxy or wrapper scripts to use a specific python version?

Regards, Holger

...

On Thu, Jul 7, 2011 at 7:56 PM, Holger Klein h.klein@imb-mainz.de wrote:

...

Hi Ross,

On 07/07/2011 02:07 AM, Ross wrote:

...

Please try the new version 0.4 of the weblogo wrapper in galaxy-central #5772 - it has additional error reporting that may help clarify dependency or other problems and let me know how you go?

thanks, with the new version I get some more hints. It seems that there is a problem with the path.

Just having weblogo installed in the dependency directory and using the env.sh mechanism to set the path, the wrapper doesn't find the executable at all: --->%--- ## rgWebLogo3.py error - cannot locate the weblogo binary weblogo on the current path ## Please ensure it is installed and working from http://code.google.com/p/weblogo ---%<---

When I put a soft link to a directory which is in the galaxy user's static path, I get a different error: --->%--- Traceback (most recent call last):  File "/local/data/home/galaxy/galaxy-dist/tools/rgenetics/rgWebLogo3.py", line 156, in    checks,s = w.run()  File "/local/data/home/galaxy/galaxy-dist/tools/rgenetics/rgWebLogo3.py", line 127, in run    s = self.runCL()  File "/local/data/home/galaxy/galaxy-dist/tools/rgenetics/rgWebLogo3.py", line 47, in runCL    print >> sys.stderr, '## rgWebLogo3.py error - executing %s returned error code %d' % cl TypeError: not enough arguments for format string ---%<---

Line 47 seems to lack the variable for the return code, when changing the line to

print >> sys.stderr, '## rgWebLogo3.py error - executing %s returned error code %d' % (cl, rval)

I get the following message: --->%--- ## rgWebLogo3.py error - executing weblogo -F png -c auto -o /local/data/galaxy_files/000/dataset_304.dat -U bits -t "Galaxy-Rgenetics Sequence Logo" -f /local/data/galaxy_files/000/dataset_286.dat -s large returned error code 1 ## This may be a data problem or a tool dependency (weblogo) installation problem ## Please ensure weblogo is correctly installed and working on the command line -see http://code.google.com/p/weblogo ---%<---

So it still seems to boil down to my local weblogo installation. Sourcing the respective env.sh and executing the above command line, I get a valid png though (again with the warning mentioned before): --->%--- galaxy@imbc1:~/tmp/1$ source ~/dependencies/weblogo/default/env.sh galaxy@imbc1:~/tmp/1$ weblogo -F png -c auto -o /local/data/galaxy_files/000/dataset_304.dat -U bits -t "Galaxy-Rgenetics Sequence Logo" -f /local/data/galaxy_files/000/dataset_286.dat -s large /home/galaxy/python/lib/python2.6/site-packages/CoreBio-0.5.0-py2.6.egg/corebio/seq_io/_nexus/__init__.py:19: DeprecationWarning: the sets module is deprecated  import sets ---%<---

Could this be related to the way galaxy is setting the paths dynamically using the env.sh file? Do I have to adjust python paths in there as well?

Regards, Holger

...

On Tue, Jul 5, 2011 at 5:49 PM, Holger Klein h.klein@imb-mainz.de wrote:

...

Hi Ross,

thanks for taking care of this issue.

On 07/05/2011 12:31 AM, Ross wrote:

...

Is this error seen on Galaxy main or test? If so please share the history with me so I can see the input and reproduce what sounds like a wrapper error?

Otherwise, if this is on a private instance, and if the tool has never produced output successfully, then this may be a dependency installation problem - eg you may need to ensure that the weblogo3 executable is available and working correctly on the path used by your execution nodes. To assure yourself that your data works with the tool, please try running it on main using the same data, and let me know what you see?

in fact it's a private instance of galaxy, it's the latest version of galaxy-dist (hg summary: 5743:720455407d1c). The input data is fine, it's a clustalw alignment in fasta format which can be used by the weblogo module on galaxy main.

Maybe some background info on the weblogo installation helps: it's located below the tool_dependency_dir as defined in universe_wsgi.ini in weblogo/3.0 (with default as a link to 3.0). It contains the file env.sh which sets the PATH: export PATH=/home/galaxy/dependencies/weblogo/3.0:$PATH

Starting the weblogo executable with the galaxy virtualenv python seems to work (just tested --help), although it returns a warning:

~/python/bin/python ./weblogo --help/home/galaxy/python/lib/python2.6/site-packages/CoreBio-0.5.0-py2.6.egg/corebio/seq_io/_nexus/__init__.py:19: DeprecationWarning: the sets module is deprecated  import sets

I also tested putting a link to the weblogo executable in the PATH that's defined for the galaxy user (as opposed to the dependency dir mechanism, that I have to admit I don't fully understand yet), but that also doesn't work. Could this be an issue of PYTHONPATH needing to be adjusted?

Regards, Holger

...

Thanks again.

On Tue, Jul 5, 2011 at 12:34 AM, Holger Klein h.klein@imb-mainz.de wrote: > Dear all, > > I have a problem with the weblogo tool. > I have a clustalw alignment in fasta format that I'd like to visualize > as a logo. The sequence logo module ends with a success (green box), the > info tells me the amount and length of the input data. But the output is > empty, there are no plots (no matter if I select jpg, png, pdf or text). > The respective image can't be displayed "because it contains errors" or > is empty in case of text. > > I suspect that the actual call of the weblogo tool doesn't succeed, but > I didn't figure out yet on how to check this. Does anybody have hints on > where to look? > > Cheers, > Holger >

-- Dr. Holger Klein Core Facility Bioinformatics Institute of Molecular Biology gGmbH (IMB) http://www.imb-mainz.de/ Tel: +49(6131) 39 21511

-- Dr. Holger Klein Core Facility Bioinformatics Institute of Molecular Biology gGmbH (IMB) http://www.imb-mainz.de/ Tel: +49(6131) 39 21511