Hi Peter,

In biolinux, I start galaxy by a clicking on an application icon.

If I open the terminal and run the suggested command: $ sh run.sh | grep -i blast

I get the following output (long):

$ sudo sh run.sh | grep -i blast
python path is: /usr/lib/galaxy-server/eggs/numpy-1.6.0-py2.7-linux-x86_64-ucs4.egg, /usr/lib/galaxy-server/eggs/pysam-0.4.2_kanwei_b10f6e722e9a-py2.7-linux-x86_64-ucs4.egg, /usr/lib/galaxy-server/eggs/boto-2.2.2-py2.7.egg, /usr/lib/galaxy-server/eggs/mercurial-2.1.2-py2.7-linux-x86_64-ucs4.egg, /usr/lib/galaxy-server/eggs/Fabric-1.4.2-py2.7.egg, /usr/lib/galaxy-server/eggs/ssh-1.7.14-py2.7.egg, /usr/lib/galaxy-server/eggs/Whoosh-0.3.18-py2.7.egg, /usr/lib/galaxy-server/eggs/pycrypto-2.5-py2.7-linux-x86_64-ucs4.egg, /usr/lib/galaxy-server/eggs/python_lzo-1.08_2.03_static-py2.7-linux-x86_64-ucs4.egg, /usr/lib/galaxy-server/eggs/bx_python-0.7.1_7b95ff194725-py2.7-linux-x86_64-ucs4.egg, /usr/lib/galaxy-server/eggs/amqplib-0.6.1-py2.7.egg, /usr/lib/galaxy-server/eggs/pexpect-2.4-py2.7.egg, /usr/lib/galaxy-server/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg, /usr/lib/galaxy-server/eggs/Babel-0.9.4-py2.7.egg, /usr/lib/galaxy-server/eggs/Mako-0.4.1-py2.7.egg, /usr/lib/galaxy-server/eggs/WebHelpers-0.2-py2.7.egg, /usr/lib/galaxy-server/eggs/simplejson-2.1.1-py2.7-linux-x86_64-ucs4.egg, /usr/lib/galaxy-server/eggs/wchartype-0.1-py2.7.egg, /usr/lib/galaxy-server/eggs/elementtree-1.2.6_20050316-py2.7.egg, /usr/lib/galaxy-server/eggs/docutils-0.7-py2.7.egg, /usr/lib/galaxy-server/eggs/WebOb-0.8.5-py2.7.egg, /usr/lib/galaxy-server/eggs/Routes-1.12.3-py2.7.egg, /usr/lib/galaxy-server/eggs/Cheetah-2.2.2-py2.7-linux-x86_64-ucs4.egg, /usr/lib/galaxy-server/eggs/PasteDeploy-1.3.3-py2.7.egg, /usr/lib/galaxy-server/eggs/PasteScript-1.7.3-py2.7.egg, /usr/lib/galaxy-server/eggs/Paste-1.6-py2.7.egg, /usr/lib/galaxy-server/lib, /usr/lib/python2.7, /usr/lib/python2.7/plat-linux2, /usr/lib/python2.7/lib-tk, /usr/lib/python2.7/lib-old, /usr/lib/python2.7/lib-dynload, /usr/local/lib/python2.7/dist-packages, /usr/lib/python2.7/dist-packages/PIL, /usr/lib/python2.7/dist-packages/gst-0.10, /usr/lib/python2.7/dist-packages/gtk-2.0, /usr/lib/pymodules/python2.7, /usr/lib/python2.7/dist-packages/ubuntu-sso-client, /usr/lib/python2.7/dist-packages
galaxy.datatypes.registry DEBUG 2013-05-01 19:58:26,358 Loaded sniffer for datatype 'galaxy.datatypes.xml:BlastXml'
galaxy.tools.data DEBUG 2013-05-01 19:58:26,371 Loaded tool data table 'blastdb'
galaxy.tools.data DEBUG 2013-05-01 19:58:26,372 Loaded tool data table 'blastdb_p'
galaxy.tools DEBUG 2013-05-01 19:58:30,808 Loading section: NCBI BLAST+
galaxy.tools DEBUG 2013-05-01 19:58:30,808 Loaded tool id: ncbi_blastn_wrapper, version: 0.0.11.
galaxy.tools DEBUG 2013-05-01 19:58:30,808 Loaded tool id: ncbi_blastp_wrapper, version: 0.0.11.
galaxy.tools DEBUG 2013-05-01 19:58:30,809 Loaded tool id: ncbi_blastx_wrapper, version: 0.0.11.
galaxy.tools DEBUG 2013-05-01 19:58:30,809 Loaded tool id: ncbi_tblastn_wrapper, version: 0.0.11.
galaxy.tools DEBUG 2013-05-01 19:58:30,809 Loaded tool id: ncbi_tblastx_wrapper, version: 0.0.11.
galaxy.tools DEBUG 2013-05-01 19:58:30,809 Loaded tool id: blastxml_to_tabular, version: 0.0.8.
galaxy.tools DEBUG 2013-05-01 19:58:30,813 Loaded tool id: megablast_wrapper, version: 1.2.0.
galaxy.tools DEBUG 2013-05-01 19:58:30,813 Loaded tool id: megablast_xml_parser, version: 1.0.0.
Traceback (most recent call last):
File "./scripts/paster.py", line 34, in <module>
    command.run()
File "/usr/lib/galaxy-server/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/command.py", line 84, in run
    invoke(command, command_name, options, args[1:])
File "/usr/lib/galaxy-server/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/command.py", line 123, in invoke
    exit_code = runner.run(args)
File "/usr/lib/galaxy-server/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/command.py", line 218, in run
    result = self.command()
File "/usr/lib/galaxy-server/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/serve.py", line 303, in command
    serve()
File "/usr/lib/galaxy-server/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/serve.py", line 287, in serve
    server(app)
File "/usr/lib/galaxy-server/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/deploy/loadwsgi.py", line 151, in server_wrapper
    **context.local_conf)
File "/usr/lib/galaxy-server/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/deploy/util/fixtypeerror.py", line 57, in fix_call
    val = callable(*args, **kw)
File "/usr/lib/galaxy-server/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py", line 1314, in server_runner
    serve(wsgi_app, **kwargs)
File "/usr/lib/galaxy-server/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py", line 1264, in serve
    threadpool_options=threadpool_options)
File "/usr/lib/galaxy-server/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py", line 1114, in __init__
    RequestHandlerClass, ssl_context)
File "/usr/lib/galaxy-server/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py", line 1094, in __init__
    RequestHandlerClass, ssl_context)
File "/usr/lib/galaxy-server/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py", line 358, in __init__
    HTTPServer.__init__(self, server_address, RequestHandlerClass)
File "/usr/lib/python2.7/SocketServer.py", line 408, in __init__
    self.server_bind()
File "/usr/lib/python2.7/BaseHTTPServer.py", line 108, in server_bind
    SocketServer.TCPServer.server_bind(self)
File "/usr/lib/python2.7/SocketServer.py", line 419, in server_bind
    self.socket.bind(self.server_address)
File "/usr/lib/python2.7/socket.py", line 224, in meth
    return getattr(self._sock,name)(*args)
socket.error: [Errno 98] Address already in use
----------------------------------------------------------------------------------------------

as might be expected from this output, it doesn't start up galaxy at localhost.

>check is the read permissions on the BLAST database

Well, from the terminal, I have access to all those files. The permissions are all the same, e.g.
-rwxrwx--- 1 root vboxsf 15721925 Apr 10 18:55 nr.00.phd

Within galaxy, I can open them with the get data menu command.

>... Galaxy log output

How is the best way to get that?

In other news, I was able to get galaxy working on the host OS X system. This took a long time, but finally, within virtualenv, and with many repeated attempts to get all the python egg dependencies, I got it up and running. In the process, I seem to have deleted some of the old python files in the OS X system (trying to force it to use the macports py27 installation). Perhaps the OS X instance will be OK but it does not come with the blast tools that I want, and I do not see an easy way to install them (I don't see an admin button to automatically install them).

Now that I've played with linux, I can see that it is not so bad to use. Apple seems to have dropped the ball with facilitating scientific computing. From your advice, it seems that instead of spending any more of my time trying to set up galaxy on my laptop, I should look around for ssh access to an existing galaxy server.

Regards, Mike DS

On Wed, May 1, 2013 at 7:26 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:

Hi Mike,

On Tue, Apr 30, 2013 at 11:17 PM, Mike Dyall-Smith
<mike.dyallsmith@gmail.com> wrote:
> Thanks Peter. My answers are below:
>
>>What query sequences are you using?
> I have just been using one fasta protein sequence.

OK - this and the fact it works on the host machine is good to know.

>>Meanwhile monitor the system with top
> Top says only a max of 27% cpu usage, but the linux screen eventually
> freezes, and I have to restart. I am not sure how to read out RAM and disk
> IO from top.

Linux 'top' does list memory usage, both per process and for
the system in the text at the top. There are other tools for
monitoring IO like iotop - but you could also probably just
watch the host Apple OS X System Monitor for this.

This does sound like the VM hasn't got enough RAM to run
BLASTP against NR efficiently.

>>grep blastp paster.log
> Tried that, and it says there is no such file or directory

In the Galaxy folder? Maybe the default log filename has
changed since I setup my machine... are you running Galaxy
as a daemon, or running run.sh at the terminal directly? If the
later, try something like this:

$ sh run.sh | grep -i blast

(If you don't get much output, adjust the logging level in the
universe_wgsi.ini configuration file.)

>>Could you try running BLAST from the host Mac OX
> Yes. And it works fine! I get a good match in a relatively short time.

That's progress - the hardware itself is capable :)

> I then made a very small protein database, checked it by commandline blastp
> in both host OS X and in guest linux, and it worked fine. Added it to the
> blastdb_p.loc file, restarted and saw it listed in galaxy. Tried to use it
> for a blastp, and got the same error as before with the huge NCBI nr
> database. So, it is not a matter of size....

There are clearly two separate issues, (1) getting BLAST to run nicely
on your VM - which I think is running out of RAM, and (2) sorting out
your Galaxy BLAST database configuration.

Something to check is the read permissions on the BLAST database
files (which Linux user are you running Galaxy as, and can that user
read the database files and their folder?).

I'm keen to see the Galaxy log output to see what exactly was the
command line being used to run BLAST, which would help with
debugging where the problem is.

> Thanks for your comments about RAM and blast searches. It gives me hope that
> I can get galaxy running usefully. I only chose biolinux because of the
> suite of programs and the apparent ease of use. The other reason was that I
> could not install galaxy on OS X (10.6). I get errors that others have noted
> on the discussion lists but no-one seems to have a solution for.

I used to do my Galaxy tool development on Mac OS X, but it didn't
work 100% right, and in any case many of the tools I wanted to
wrap and run within Galaxy were Linux only - so now I just ssh
into a Linux server to do Galaxy work. Given the main Galaxy
development and the Penn state Galaxy server all happens on
Linux, you'll have a much easier time under Linux than Mac OS X.

Regards,

Peter

--
_____________________________________
Mike Dyall-Smith, Ph.D.,