March 2011 - galaxy-dev - lists.galaxyproject.org

Galaxy process
by SHAUN WEBB 09 Mar '11

09 Mar '11

Hi, since making the last update I have found some new warnings in my paster.log, it also seems as though the galaxy process starts to gather memory and eventually hang (35% of 64G memory). I've posted the entries below. If anyone could help me understand what is going on that would be great. Thanks. Shaun Webb paste.httpserver.ThreadPool INFO 2011-03-09 09:49:49,962 No idle tasks, and only 0 busy tasks; adding 5 more workers paste.httpserver.ThreadPool INFO 2011-03-09 09:49:58,754 No idle tasks, and only 4 busy tasks; adding 1 more workers paste.httpserver.ThreadPool INFO 2011-03-09 09:51:47,301 Culling 6 extra workers (5 idle workers present) paste.httpserver.ThreadPool INFO 2011-03-09 09:55:17,163 No idle tasks, and only 0 busy tasks; adding 5 more workers 129.215.14.72 - - [09/Mar/2011:09:48:40 +0100] "GET /history HTTP/1.1" 500 - "http://bifx-core.bio.ed.ac.uk:8080/" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 (.NET CLR 2.0.50727; .NET CLR 1.1.4322; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)" paste.httpserver.ThreadPool INFO 2011-03-09 10:13:16,956 Culling 5 extra workers (7 idle workers present) 212.183.140.59 - - [09/Mar/2011:10:13:17 +0100] "GET / HTTP/1.1" 200 - "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Chrome/9.0.597.102 Safari/534.13" paste.httpserver.ThreadPool INFO 2011-03-09 10:14:35,715 No idle tasks, and only 2 busy tasks; adding 3 more workers paste.httpserver.ThreadPool WARNING 2011-03-09 10:15:15,104 Thread 140283224094464 hung (working on task for 3096 seconds) ---------------------------------------- Exception happened during processing of request from ('212.183.140.59', 10871) Traceback (most recent call last): File "/usr/lib/python2.6/SocketServer.py", line 281, in _handle_request_noblock self.process_request(request, client_address) File "/storage/home/galaxy/galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/httpserver.py", line 1037, in process_request lambda: self.process_request_in_thread(request, client_address)) File "/storage/home/galaxy/galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/httpserver.py", line 617, in add_task self.kill_hung_threads() File "/storage/home/galaxy/galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/httpserver.py", line 778, in kill_hung_threads self.kill_worker(worker.thread_id) File "/storage/home/galaxy/galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/httpserver.py", line 705, in kill_worker killthread.async_raise(thread_id, SystemExit) File "/storage/home/galaxy/galaxy_dist/eggs/Paste-1.6-py2.6.egg/paste/util/killthread.py", line 22, in async_raise raise ValueError("invalid thread id") ValueError: invalid thread id ---------------------------------------- -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

1 0

Pass files as url
by Bonci, Timothy Daniel 08 Mar '11

08 Mar '11

I have an interactive applet that visualizes data produced by Galaxy. The applet, being run client side, does not have access to the file system of the server, so passing history files by reference (path) won't work. All the files are available directly to the browser through a url, but I can't figure out a way to get that url. Alternatively, If anyone could help me find the history.htm file internally I could have the program that preps the applet parse that for the urls. ---Tim Bonci

2 1

divide fq into 2
by Musa A. Hassan 08 Mar '11

08 Mar '11

Hi All, Is there anyone out there who know how I can divide an fq file containing illumina short reads randomly into 2 small files contaning approximately equal number of reads? I have a huge fq from the illumina high-seq platform, unfortunately, this file is huge and is causing all sorts of problems and I'd like to divide into to equal sizes(based on number of reads). Musa

2 1

Referencing data library directly from tool's input parameter dropdown.
by Rodriguez, Aaron (NIH/NCI) [C] 08 Mar '11

08 Mar '11

Hello all. First of all thanks to Galaxy developers and supporters for this great tool -- I'm experimenting with adding my custom tools to galaxy and I'm very impressed with how simple galaxy makes this process. Some of the tools that I'm adding to my local instance take reference files as a parameter. These files are often large and fairly static. I understand that I can create a data library where users can select the reference file needed and import into "history" for analysis. However, I would like to know if it's possible for a tool's input parameter (dropdown) to enumerate these files from a public data-library (to avoid the steps needed to import from library into history) and have that file pathname become the value for my parameter? As an alternative, would it be possible to load such a drop down from files (of a certain type) existing at some predefined system path? Thanks in advance, -AR

2 2

about bx-python and bigwig
by Davide Cittaro 08 Mar '11

08 Mar '11

Hi all, Probably this is not the best place to ask about bx-python, but since most of bx developers are galaxy developers, it may be worth trying... I just want to know the status of bigwig support in bx. I need to handle bigwig as input data and I previously wrapped kent binaries with python subprocess. I wanted to make it more pythonic and started reading bigwig docs. If bx already has support for an iterator similar to bx.wiggle.Reader (I need to transfer all the chromosome in another data structure) I won't read docs further :-) Thanks d /* Davide Cittaro, PhD Cogentech - Consortium for Genomic Technologies via adamello, 16 20139 Milano Italy tel.: +39(02)574303007 e-mail: davide.cittaro(a)ifom-ieo-campus.it */ -- /* Davide Cittaro Next Generation Sequencing and Affymetrix data analyst Cogentech - Consortium for genomic technologies via adamello, 16 20139 Milano Italy tel.: +39(02)574303007 e-mail: davide.cittaro(a)ifom-ieo-campus.it */

3 3

fastq filetypes
by Ryan Golhar 08 Mar '11

08 Mar '11

Is there a way to quickly determine what fastq filetype a particular fastq file is? I ask because I was recently given a fastq which which I incorrectly assumed to be fastqsanger. When I ran a fastx tool on the file, the tool seg faulted. It took me a while to figure out that the fastq type is not sanger. Is there a tool available that report what the encoding type is? -- CONFIDENTIALITY NOTICE: This email communication may contain private, confidential, or legally privileged information intended for the sole use of the designated and/or duly authorized recipient(s). If you are not the intended recipient or have received this email in error, please notify the sender immediately by email and permanently delete all copies of this email including all attachments without reading them. If you are the intended recipient, secure the contents in a manner that conforms to all applicable state and/or federal requirements related to privacy and confidentiality of such information.

2 1

Re: [galaxy-dev] SAM tools SAM to BAM
by Jennifer Jackson 07 Mar '11

07 Mar '11

Hi Christin, This error message means that the build assigned to your data is not loaded as a native database genome. This wiki page explains how to load genomes to local instances: https://bitbucket.org/galaxy/galaxy-central/wiki/DataIntegration Best, Jen Galaxy team ps. Apologies for the delay in reply, the question got a bit lost since it was not directed at one of our mailing lists. Using galaxy-dev(a)bx.psu.edu would be a good choice for questions like this about running local instances. On 2/17/11 8:22 AM, christin weinberg wrote: > Hey Jen, > > we have a local galaxy instance....now there is following error while > converting SAM to BAM > > Sequences are not currently available for the specified build. > > Thanks for your help! > > Best > Christin -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org

1 0

possible bug - user log in / browser cache
by Ryan Golhar 07 Mar '11

07 Mar '11

I have a local instance running. When I'm logged in as user1, log out, then try to log in as user2, I keep getting the "You have been logged out" message. Its as if the browser's cache is retaining some information. To get around this, after I log out as user1, I have to click on Analyze Data, then click on Log in to log in as user2. I'm running Safari on Mac, but have noticed this with Firefox as well.

3 3

upload large data file
by Yanji Xu 07 Mar '11

07 Mar '11

Dear Sir/Madam, I installed galaxy in my local server, then I tried to upload a 4.7 Gb fastq file into galaxy, but failed. Below is the error message. OverflowError: signed integer is greater than maximum How could I upload large data files into galaxy and process the data? Any information from you would be quite appreciated. Thanks, Dicty

3 2

file upload
by Musa A. Hassan 07 Mar '11

07 Mar '11

I am trying to use galaxy for the first time and am running into some early problems. While trying to upload my file which is basically an fq containing illumina short reads, the file won't upload. When I try the fastq extension of the same file, it does but then the file isn't recognized by the galaxy hence it's like I have no file uploaded. Does anyone have an idea what's going on here. I've looked at the short tutorial on galaxy and it seems my file should be fine. Musa

2 1