[Bioc-sig-seq] Memory bottlenecks while using ShortRead

[Steve Lianoglou]

Hi,

> More generally I'd like to implement something to process larger files.
> The conversations I've had have involved 'streams' where the file
> essentially gets processed in chunks. The user would have to provide a
> function that processed the appropriate data structure, e.g., an
> 'AlignedRead' or 'ShortReadQ' object, and possibly also a function that
> stitched the results from several chunks back together.

I think one could get a lot of mileage out of this. For instance, I
think for QA type of stuff, being able to rip through a fastq file and
process 1 fastq "object" (the 4 (or so) lines you get per read) would
be very handy.

-- 
Steve Lianoglou
Graduate Student: Computational Systems Biology
 | Memorial Sloan-Kettering Cancer Center
 | Weill Medical College of Cornell University
Contact Info: http://cbio.mskcc.org/~lianos/contact