[R] Huge data frames?

[Laurent Gautier]

On Wed, Aug 28, 2002 at 05:38:50AM +0200, Magnus Lie Hetland wrote:
> A friend of mine recently mentioned that he had painlessly imported a
> data file with 8 columns and 500,000 rows into matlab. When I tried
> the same thing in R (both Unix and Windows variants) I had little
> success. The Windows version hung for a very long time, until I
> eventually more or less ran out of virtual memory; I tried to set the
> proper memory allocations for the Unix version, but it never seemed
> satisfied :]
> 
> I used read.table -- should I have used something else? Is it even

try 'scan'

> possible to work with this large files? I assume a memory-mapped
> binary file would have been quite efficient (as opposed to an
> in-memory parsed text file) -- is something like that even possible in
> R?
> 

have a look at the package 'rhdf5' (at least at www.bioconductor.org,
not sure it's on CRAN). Not exactly what you describe but could be
relevant.



Regards,


L.


> -- 
> Magnus Lie Hetland                                  The Anygui Project
> http://hetland.org                                  http://anygui.org
> -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
> r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
> Send "info", "help", or "[un]subscribe"
> (in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
> _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._

-- 
--------------------------------------------------------------
Laurent Gautier			CBS, Building 208, DTU
PhD. Student			DK-2800 Lyngby,Denmark	
tel: +45 45 25 24 89		http://www.cbs.dtu.dk/laurent
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._