check is the read permissions on the BLAST database Well, from the terminal, I have access to all those files. The
Hi Peter, In biolinux, I start galaxy by a clicking on an application icon. If I open the terminal and run the suggested command: $ sh run.sh | grep -i blast I get the following output (long): $ sudo sh run.sh | grep -i blast python path is: /usr/lib/galaxy-server/eggs/numpy-1.6.0-py2.7-linux-x86_64-ucs4.egg, /usr/lib/galaxy-server/eggs/pysam-0.4.2_kanwei_b10f6e722e9a-py2.7-linux-x86_64-ucs4.egg, /usr/lib/galaxy-server/eggs/boto-2.2.2-py2.7.egg, /usr/lib/galaxy-server/eggs/mercurial-2.1.2-py2.7-linux-x86_64-ucs4.egg, /usr/lib/galaxy-server/eggs/Fabric-1.4.2-py2.7.egg, /usr/lib/galaxy-server/eggs/ssh-1.7.14-py2.7.egg, /usr/lib/galaxy-server/eggs/Whoosh-0.3.18-py2.7.egg, /usr/lib/galaxy-server/eggs/pycrypto-2.5-py2.7-linux-x86_64-ucs4.egg, /usr/lib/galaxy-server/eggs/python_lzo-1.08_2.03_static-py2.7-linux-x86_64-ucs4.egg, /usr/lib/galaxy-server/eggs/bx_python-0.7.1_7b95ff194725-py2.7-linux-x86_64-ucs4.egg, /usr/lib/galaxy-server/eggs/amqplib-0.6.1-py2.7.egg, /usr/lib/galaxy-server/eggs/pexpect-2.4-py2.7.egg, /usr/lib/galaxy-server/eggs/SQLAlchemy-0.5.6_dev_r6498-py2.7.egg, /usr/lib/galaxy-server/eggs/Babel-0.9.4-py2.7.egg, /usr/lib/galaxy-server/eggs/Mako-0.4.1-py2.7.egg, /usr/lib/galaxy-server/eggs/WebHelpers-0.2-py2.7.egg, /usr/lib/galaxy-server/eggs/simplejson-2.1.1-py2.7-linux-x86_64-ucs4.egg, /usr/lib/galaxy-server/eggs/wchartype-0.1-py2.7.egg, /usr/lib/galaxy-server/eggs/elementtree-1.2.6_20050316-py2.7.egg, /usr/lib/galaxy-server/eggs/docutils-0.7-py2.7.egg, /usr/lib/galaxy-server/eggs/WebOb-0.8.5-py2.7.egg, /usr/lib/galaxy-server/eggs/Routes-1.12.3-py2.7.egg, /usr/lib/galaxy-server/eggs/Cheetah-2.2.2-py2.7-linux-x86_64-ucs4.egg, /usr/lib/galaxy-server/eggs/PasteDeploy-1.3.3-py2.7.egg, /usr/lib/galaxy-server/eggs/PasteScript-1.7.3-py2.7.egg, /usr/lib/galaxy-server/eggs/Paste-1.6-py2.7.egg, /usr/lib/galaxy-server/lib, /usr/lib/python2.7, /usr/lib/python2.7/plat-linux2, /usr/lib/python2.7/lib-tk, /usr/lib/python2.7/lib-old, /usr/lib/python2.7/lib-dynload, /usr/local/lib/python2.7/dist-packages, /usr/lib/python2.7/dist-packages/PIL, /usr/lib/python2.7/dist-packages/gst-0.10, /usr/lib/python2.7/dist-packages/gtk-2.0, /usr/lib/pymodules/python2.7, /usr/lib/python2.7/dist-packages/ubuntu-sso-client, /usr/lib/python2.7/dist-packages galaxy.datatypes.registry DEBUG 2013-05-01 19:58:26,358 Loaded sniffer for datatype 'galaxy.datatypes.xml:BlastXml' galaxy.tools.data DEBUG 2013-05-01 19:58:26,371 Loaded tool data table 'blastdb' galaxy.tools.data DEBUG 2013-05-01 19:58:26,372 Loaded tool data table 'blastdb_p' galaxy.tools DEBUG 2013-05-01 19:58:30,808 Loading section: NCBI BLAST+ galaxy.tools DEBUG 2013-05-01 19:58:30,808 Loaded tool id: ncbi_blastn_wrapper, version: 0.0.11. galaxy.tools DEBUG 2013-05-01 19:58:30,808 Loaded tool id: ncbi_blastp_wrapper, version: 0.0.11. galaxy.tools DEBUG 2013-05-01 19:58:30,809 Loaded tool id: ncbi_blastx_wrapper, version: 0.0.11. galaxy.tools DEBUG 2013-05-01 19:58:30,809 Loaded tool id: ncbi_tblastn_wrapper, version: 0.0.11. galaxy.tools DEBUG 2013-05-01 19:58:30,809 Loaded tool id: ncbi_tblastx_wrapper, version: 0.0.11. galaxy.tools DEBUG 2013-05-01 19:58:30,809 Loaded tool id: blastxml_to_tabular, version: 0.0.8. galaxy.tools DEBUG 2013-05-01 19:58:30,813 Loaded tool id: megablast_wrapper, version: 1.2.0. galaxy.tools DEBUG 2013-05-01 19:58:30,813 Loaded tool id: megablast_xml_parser, version: 1.0.0. Traceback (most recent call last): File "./scripts/paster.py", line 34, in <module> command.run() File "/usr/lib/galaxy-server/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/command.py", line 84, in run invoke(command, command_name, options, args[1:]) File "/usr/lib/galaxy-server/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/command.py", line 123, in invoke exit_code = runner.run(args) File "/usr/lib/galaxy-server/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/command.py", line 218, in run result = self.command() File "/usr/lib/galaxy-server/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/serve.py", line 303, in command serve() File "/usr/lib/galaxy-server/eggs/PasteScript-1.7.3-py2.7.egg/paste/script/serve.py", line 287, in serve server(app) File "/usr/lib/galaxy-server/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/deploy/loadwsgi.py", line 151, in server_wrapper **context.local_conf) File "/usr/lib/galaxy-server/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/deploy/util/fixtypeerror.py", line 57, in fix_call val = callable(*args, **kw) File "/usr/lib/galaxy-server/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py", line 1314, in server_runner serve(wsgi_app, **kwargs) File "/usr/lib/galaxy-server/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py", line 1264, in serve threadpool_options=threadpool_options) File "/usr/lib/galaxy-server/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py", line 1114, in __init__ RequestHandlerClass, ssl_context) File "/usr/lib/galaxy-server/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py", line 1094, in __init__ RequestHandlerClass, ssl_context) File "/usr/lib/galaxy-server/eggs/Paste-1.6-py2.7.egg/paste/httpserver.py", line 358, in __init__ HTTPServer.__init__(self, server_address, RequestHandlerClass) File "/usr/lib/python2.7/SocketServer.py", line 408, in __init__ self.server_bind() File "/usr/lib/python2.7/BaseHTTPServer.py", line 108, in server_bind SocketServer.TCPServer.server_bind(self) File "/usr/lib/python2.7/SocketServer.py", line 419, in server_bind self.socket.bind(self.server_address) File "/usr/lib/python2.7/socket.py", line 224, in meth return getattr(self._sock,name)(*args) socket.error: [Errno 98] Address already in use ---------------------------------------------------------------------------------------------- as might be expected from this output, it doesn't start up galaxy at localhost. permissions are all the same, e.g. -rwxrwx--- 1 root vboxsf 15721925 Apr 10 18:55 nr.00.phd Within galaxy, I can open them with the get data menu command.
... Galaxy log output How is the best way to get that?
In other news, I was able to get galaxy working on the host OS X system. This took a long time, but finally, within virtualenv, and with many repeated attempts to get all the python egg dependencies, I got it up and running. In the process, I seem to have deleted some of the old python files in the OS X system (trying to force it to use the macports py27 installation). Perhaps the OS X instance will be OK but it does not come with the blast tools that I want, and I do not see an easy way to install them (I don't see an admin button to automatically install them). Now that I've played with linux, I can see that it is not so bad to use. Apple seems to have dropped the ball with facilitating scientific computing. From your advice, it seems that instead of spending any more of my time trying to set up galaxy on my laptop, I should look around for ssh access to an existing galaxy server. Regards, Mike DS On Wed, May 1, 2013 at 7:26 PM, Peter Cock <p.j.a.cock@googlemail.com>wrote:
Hi Mike,
On Tue, Apr 30, 2013 at 11:17 PM, Mike Dyall-Smith <mike.dyallsmith@gmail.com> wrote:
Thanks Peter. My answers are below:
What query sequences are you using? I have just been using one fasta protein sequence.
OK - this and the fact it works on the host machine is good to know.
Meanwhile monitor the system with top Top says only a max of 27% cpu usage, but the linux screen eventually freezes, and I have to restart. I am not sure how to read out RAM and disk IO from top.
Linux 'top' does list memory usage, both per process and for the system in the text at the top. There are other tools for monitoring IO like iotop - but you could also probably just watch the host Apple OS X System Monitor for this.
This does sound like the VM hasn't got enough RAM to run BLASTP against NR efficiently.
grep blastp paster.log Tried that, and it says there is no such file or directory
In the Galaxy folder? Maybe the default log filename has changed since I setup my machine... are you running Galaxy as a daemon, or running run.sh at the terminal directly? If the later, try something like this:
$ sh run.sh | grep -i blast
(If you don't get much output, adjust the logging level in the universe_wgsi.ini configuration file.)
Could you try running BLAST from the host Mac OX Yes. And it works fine! I get a good match in a relatively short time.
That's progress - the hardware itself is capable :)
I then made a very small protein database, checked it by commandline blastp in both host OS X and in guest linux, and it worked fine. Added it to the blastdb_p.loc file, restarted and saw it listed in galaxy. Tried to use it for a blastp, and got the same error as before with the huge NCBI nr database. So, it is not a matter of size....
There are clearly two separate issues, (1) getting BLAST to run nicely on your VM - which I think is running out of RAM, and (2) sorting out your Galaxy BLAST database configuration.
Something to check is the read permissions on the BLAST database files (which Linux user are you running Galaxy as, and can that user read the database files and their folder?).
I'm keen to see the Galaxy log output to see what exactly was the command line being used to run BLAST, which would help with debugging where the problem is.
Thanks for your comments about RAM and blast searches. It gives me hope that I can get galaxy running usefully. I only chose biolinux because of the suite of programs and the apparent ease of use. The other reason was that I could not install galaxy on OS X (10.6). I get errors that others have noted on the discussion lists but no-one seems to have a solution for.
I used to do my Galaxy tool development on Mac OS X, but it didn't work 100% right, and in any case many of the tools I wanted to wrap and run within Galaxy were Linux only - so now I just ssh into a Linux server to do Galaxy work. Given the main Galaxy development and the Penn state Galaxy server all happens on Linux, you'll have a much easier time under Linux than Mac OS X.
Regards,
Peter
-- _____________________________________ Mike Dyall-Smith, Ph.D.,