Hi Paul, Hi Peter

You might also wanna look at the 'FastqSampler' function in the Bioconductor 'ShortRead' package
http://bioconductor.org/packages/release/bioc/html/ShortRead.html

We are working (as part of our NGS pipeline redesign) on adding more Bioconductor functionalities to Galaxy. Unfortunately, it is very low on my pile of stuff to do, so it will take a while till it appears in the 'Tool Shed'.


Regards, Hans



On 11/08/2011 11:45 PM, Peter Cock wrote:

On Tue, Nov 8, 2011 at 10:26 PM, Austin Paul<austinpa@usc.edu>  wrote:

Hi Peter,

Thanks for the suggestion.  For example, I have a fastq file with 50 million

reads and I want to randomly select 5 million of them. It seems biopython

would very easily select a single or a handful of reads with the

Bio.SeqIO.index() function.  Would it also be able to do the job I am

interested in?

Austin

I think so, but you'd have to use Bio.SeqIO.index_db() which stores

the index in an SQLite dictionary rather than in memory which isn't

really viable here (unless you have a 64bit big memory machine?).

I don't think I've tried it with quite that many reads though...

Alternatively, if I understood her correctly, Jennifer pointed out you

can do this in Galaxy but it will take a lot of IO:

1. Convert FASTQ to tabular (4 lines per record ->  1 line per record)

2. Randomly select lines (each line is now a record so safe)

3. Convert tabular back to FASTQ

It should work though, and requires no additional programming.

Peter

___________________________________________________________

The Galaxy User list should be used for the discussion of

Galaxy analysis and other features on the public server

at usegalaxy.org.  Please keep all replies on the list by

using "reply all" in your mail client.  For discussion of

local Galaxy instances and the Galaxy source code, please

use the Galaxy Development list:

  http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists,

please use the interface at:

  http://lists.bx.psu.edu/

___________________________________________________________
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org.  Please keep all replies on the list by
using "reply all" in your mail client.  For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:

http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists,
please use the interface at:

http://lists.bx.psu.edu/