December 2011 - galaxy-dev - lists.galaxyproject.org

Galaxy web interface falls over when running TopHat
by David Matthews 13 Dec '11

13 Dec '11

Hi, We seem to have sorted out the problem of TopHat failing to run but now we have a new problem. When TopHat runs with a large genome (but not with small genomes), it finishes the run fine and all the data is there but the web interface falls over about 8 hours into the run and when we try to access the web based interface we get a Proxy Error. When we restart it all looks fine. This is the sort of errors our HPC people find: Exception happened during processing of request from ('XXX.XXX.XXX.XXX', 32960) Traceback (most recent call last): File "/gpfs/cluster/isys/galaxy/Galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/pa ste/httpserver.py", line 1053, in process_request_in_thread Unexpected exception in worker <function <lambda> at 0x201caed8> Traceback (most recent call last): File "/gpfs/cluster/isys/galaxy/Galaxy/galaxy-dist/eggs/Paste-1.6-py2.6.egg/pa ste/httpserver.py", line 863, in worker_thread_callback Unhandled exception in thread started by <bound method Thread.__bootstrap of <Th read(worker 118, stopped 1127647552)>> Traceback (most recent call last): Do you have any suggestions for what is happening? Best Wishes, David. __________________________________ Dr David A. Matthews Senior Lecturer in Virology Room E49 Department of Cellular and Molecular Medicine, School of Medical Sciences University Walk, University of Bristol Bristol. BS8 1TD U.K. Tel. +44 117 3312058 Fax. +44 117 3312091 D.A.Matthews(a)bristol.ac.uk

4 5

googlebot
by SHAUN WEBB 13 Dec '11

13 Dec '11

Hi, my paster.log is full of errors relating to a googlebot trying to access a library file. I noticed a robots.txt file in the static folder configured as such. User-agent: * Disallow: / I thought this would stop all bots accessing all files. Should this be configured differently? Thanks Shaun -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

3 3

Re: [galaxy-dev] administrator Interface.
by James Boocock 13 Dec '11

13 Dec '11

Hi, I have setup a galaxy production server and created a admins.txt file with comma separated account names. When running galaxy and logged into one of the activated accounts I cannot view the interface. Configs Listed Below Cheers James. #Config File file:universe config path:/home/galaxy/galaxy-dist/universe-wsgi. Config line related to the admins folder. admin_users = /home/galaxy/galaxy-dist/tools/SOER1000genes/galaxy/admins.txt #Admin.txt. file:admins.txt path:/home/galaxy/galaxy-dist/tools/SOER1000genes/galaxy/admins.txt #file contents# admin_users=sfk2001(a)gmail.com ## When I Log in to the account above in galaxy I cannot view the admin interface. I am confused as to what I have done wrong or possibly stupid.

2 1

Tools accessing web services
by Amelia Ireland 12 Dec '11

12 Dec '11

Apologies in advance for the newbie question. I have a script that uses various UniProt web services. The script sends off a request to the UniProt server, waits for it to be returned, and then writes the results to a file. This works fine as a standalone script, but when plugged into Galaxy, I get this error: ⇝ KeyError: 'GALAXY_URL' URL: http://127.0.0.1:8080/tool_runner/index Does Galaxy "intervene" in the operation of all tools that access outside web services, and if so, do I need to rewrite the tool as one that acts as a data source (like the UCSC browser, biomart, etc.)? The data sources documentation did not mention whether this was necessary or not. Thanks! -- Amelia Ireland Scientific Curator, Gene Ontology project http://www.berkeleybop.org || http://www.geneontology.org Preventing Euthanasia Through Rescue: http://petrescueinc.org

2 2

How and where to install tool dependencies
by Josh Nielsen 12 Dec '11

12 Dec '11

Hello, I have a question which I have not seen specifically addressed in the online Galaxy wiki documentation about how to integrate tools (dependencies) into Galaxy. I have implemented a locally managed instance of Galaxy that my business is using with our cluster and now have a freshly installed and configured instance of Galaxy running. It is bare-bones right now and I did not use mercurial to sync any existing files/directory structures. I have seen the page on external tool dependencies ( http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Dependencies) needed for Galaxy, but I am somewhat unsure where to place the tools to utilize them as intended (other than through trial & error). It appears that there are shell directories for the tools under ~/galaxy-dist/tools/ with basic wrapper scripts but without the corresponding executables (very few that I've noticed have the tools already in them). Is the intent to download the dependency tools and (building from source if necessary) take the binaries in those directories and copy them to their corresponding directory under ~/galaxy-dist/tools/? This seems to have worked with an error I first got when clipping a FASTQ file which reported that fastx_clipper was not a recognized command. So I downloaded the FASTX Toolkit, compiled the binaries, and copied only the binaries into the corresponding fastx tools directory. Would I do the same thing for TopHat and Cufflinks by taking all their binaries (combined) and copying them into ~/galaxy-dist/tools/ngs_rna/? Even if that is the case though, I have occasionally gotten errors about tools missing in completely different directories. One was for the FASTQ Groomer. One user saw this error in their browser (which for now is the only way I know to figure out where tools are *expected* to be): *File "/home/galaxy/galaxy-dist/tools/rgenetics/rgFastQC.py", line 141, in assert os.path.isfile(opts.executable),'##rgFastQC.py error - cannot find executable %s' % opts.executable AssertionError: ##rgFastQC.py error - cannot find executable /home/galaxy/galaxy-dist/tool-data/shared/jars/FastQC/fastqc * To fix this I downloaded the FastQC tar file from its webpage, unzipped it, and copied the fastqc binary/script to the "home/galaxy/galaxy -dist/tool-data/shared/jars/FastQC/*"* directory. I also had to mkdir FastQC/ under jars/ to place it there since it didn't already exist. Had I not been told the specific directory by the error I'm not sure how I would have intuitively known to place the binary there (unless I'm overlooking some critical documentation). And how do I know that other similar things are not missing which should be there? Can anyone shed some light on this please? Adding a brief page on the Galaxy wiki site under the Admin section about this would really help, even if it only showed an example for one or two specific tools. Thanks, Josh Nielsen

3 5

Galaxy on GPU
by Isabelle Phan 12 Dec '11

12 Dec '11

Hello, Can any applications in Galaxy use GPU hardware (e.g. for processing next gen data)? Thanks, Isabelle PS: From Enis's reply to a post from last year, it looked like the answer was no at the time On Fri, Dec 10, 2010 at 8:59 PM, <[hidden email]> wrote: ---------------------------- Original Message ---------------------------- Subject: Re: [galaxy-dev] Wiki erratum... From: "Richard Bruskiewich" <[hidden email]> Date: Fri, December 10, 2010 7:17 pm To: "Anton Nekrutenko" <[hidden email]> -------------------------------------------------------------------------- > Also, Amazon has so-called "cluster" instances, and now GPU cluster > instances. Again, the same idea applies: can specific tools be told to only > run on such a cluster instance? Further ahead, could Galaxy be configured > to automatically start/stop specific instances only when needed (including > cluster instances)? Because MPI-type jobs are the only true beneficiaries of the cluster instances, but only a handful of bioinformatics software are actually implemented using MPI and because those instances require a different AMI, we do not currently have support for that type of instances - maybe down the line. Nonetheless, in the coming new version Galaxy Cloud (currently being tested), the application will be able to automatically scale the size of the cluster based on the current workload. -- Isabelle Phan, DPhil Seattle Biomedical Research Institute +1(206)256 7113

2 1

Divide FASTQ file into paired and unpaired reads (version 0.0.4) - naming suffix not recognized
by Liisa Koski 12 Dec '11

12 Dec '11

Hi , Looks like the tool Divide FASTQ file into paired and unpaired reads is not recognizing my paired-end naming suffix @HWI-ST916:79:D04M5ACXX:1:1101:10000:100326 1:N:0:TGNCCA @HWI-ST916:79:D04M5ACXX:1:1101:10000:100326 2:N:0:TGNCCA Is there a patch for this or another tool/way I can divide this fastq file? Thanks in advance, Liisa

2 5

Dynamic Tool Parameter Lists?
by John Major 12 Dec '11

12 Dec '11

Hello Gal-Dev, I've been poking around looking for a way to define a tool and be able to define a dynamic set of values for a property dropdown. Say, I wanted to present a list of transcipt ID's in my tool page, with this list drawn from a database local to my system. Is there any mechanism to do this? thanks- john

2 1

How to pass files/data invisibly between tools?
by Steven Platt 12 Dec '11

12 Dec '11

Hi, We wrap our tools in perl and ruby scripts and have them execute in a randomly named folder. Things all work well, both in & out of workflow, until I want to merge the log files of workflow components as later tools don't know where the previous were running. Ideally I'd like to run all tools in the same (randomly named) folder and write all logs to one file. The first tool can easily name & create the folder but how do I invisibly pass this filepath to subsequent tools? I've managed to write the filepath as an output file & send it to the second tool in workflow, but this appears to require a txt data parameter that is visible to the user and is auto-populated from history if the tool is used out of workflow. Plus it's not the seamless passage of data I'm looking for. I'll obviously need to build in some code to accommodate running tools out of workflow i.e. when filepath = nil, but I think that'll be the easy part. Any ideas? Thanks Steve ----------------------------------------- ************************************************************************** The information contained in the EMail and any attachments is confidential and intended solely and for the attention and use of the named addressee(s). It may not be disclosed to any other person without the express authority of the HPA, or the intended recipient, or both. If you are not the intended recipient, you must not disclose, copy, distribute or retain this message or any part of it. This footnote also confirms that this EMail has been swept for computer viruses, but please re-sweep any attachments before opening or saving. HTTP://www.HPA.org.uk **************************************************************************

2 1

Error viewing BAM files in IGV
by Alexander Graf 12 Dec '11

12 Dec '11

Hello, I have recently updated our Galaxy dist and I'm running into problems viewing BAM files in IGV (v2.0.22), saying: Invalid GZIP header. I have configured the Apache as explained in the Wiki using this *http.conf* : ------------------------------------------------------------------------------------------------------------------------------------ <VirtualHost *:80> ServerName 127.0.0.1 RewriteEngine on RewriteRule ^/galaxy$ /galaxy/ [R] RewriteRule ^/galaxy/static/style/(.*) /opt/galaxy/static/june_2007_style/blue/$1 [L] RewriteRule ^/galaxy/static/scripts/(.*) /opt/galaxy/static/scripts/packed/$1 [L] RewriteRule ^/galaxy/static/(.*) /opt/galaxy/static/$1 [L] RewriteRule ^/galaxy/favicon.ico /opt/galaxy/static/favicon.ico [L] RewriteRule ^/galaxy/robots.txt /opt/galaxy/static/robots.txt [L] RewriteRule ^/galaxy(.*) http://localhost:8081$1 [P] <Proxy http://localhost:8081> Order deny,allow Allow from all </Proxy> <Location "/galaxy"> # Define the authentication method XSendFile on XSendFilePath / # Compress all uncompressed content.http.conf SetOutputFilter DEFLATE SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary SetEnvIfNoCase Request_URI \.(?:t?gz|zip|bz2)$ no-gzip dont-vary </Location> <Directory "/galaxy/static"> ExpiresActive On ExpiresDefault "access plus 6 hours" </Directory> </VirtualHost> ---------------------------------------------------------------------------------------------------------------------------------------- By having this parts changed in *universe_wsgi.ini*: ---------------------------------------------------------------------------------------------------------------------------------------- [server:main] use = egg:Paste#http port = 8081 host = 0.0.0.0 use_threadpool = True [filter:gzip] use = egg:Paste#gzip [filter:proxy-prefix] use = egg:PasteDeploy#prefix prefix = /galaxy [app:main] paste.app_factory = galaxy.web.buildapp:app_factory filter-with = proxy-prefix cookie_path = /galaxy apache_xsendfile = True upstream_gzip = False ---------------------------------------------------------------------------------------------------------------------------------------- The resulting *Galaxy error log* is: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- galaxy.web.framework DEBUG 2011-12-09 12:32:08,825 Error: this request returned None from get_history(): http://10.153.182.203/galaxy/root 10.163.241.110 - - [09/Dec/2011:12:32:08 +0200] "GET /galaxy/root?app_action=data&user_id=c9a3f3a19e75965d&app_name=igv_bam&link_name=local_default&action_param=galaxy_9b0f702d0207cd78.bam.bai&dataset_id=9b0f702d0207cd78 HTTP/1.1" 200 - "-" "IGV Version 2.0.22 (1360)11/29/2011 02:24 PM Java/1.6.0_22" 10.163.241.110 - - [09/Dec/2011:12:32:08 +0200] "HEAD /galaxy/display_application/9b0f702d0207cd78/igv_bam/local_default/c9a3f3a19e75965d/data/galaxy_9b0f702d0207cd78.bam HTTP/1.1" 302 - "-" "IGV Version 2.0.22 (1360)11/29/2011 02:24 PM Java/1.6.0_22" galaxy.web.framework DEBUG 2011-12-09 12:32:08,915 Error: this request returned None from get_history(): http://10.153.182.203/galaxy/root 10.163.241.110 - - [09/Dec/2011:12:32:08 +0200] "HEAD /galaxy/root?app_action=data&user_id=c9a3f3a19e75965d&app_name=igv_bam&link_name=local_default&action_param=galaxy_9b0f702d0207cd78.bam&dataset_id=9b0f702d0207cd78 HTTP/1.1" 200 - "-" "IGV Version 2.0.22 (1360)11/29/2011 02:24 PM Java/1.6.0_22" ---------------------------------------- Exception happened during processing of request from ('127.0.0.1', 52683) Traceback (most recent call last): File "/opt/galaxy/eggs/Paste-1.6-py2.6.egg/paste/httpserver.py", line 1053, in process_request_in_thread self.finish_request(request, client_address) File "/usr/lib/python2.6/SocketServer.py", line 322, in finish_request self.RequestHandlerClass(request, client_address, self) File "/usr/lib/python2.6/SocketServer.py", line 618, in __init__ self.finish() File "/usr/lib/python2.6/SocketServer.py", line 661, in finish self.wfile.flush() File "/usr/lib/python2.6/socket.py", line 297, in flush self._sock.sendall(buffer(data, write_offset, buffer_size)) error: [Errno 32] Broken pipe ---------------------------------------- --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Thanks in advance for your help Alex

2 2