[R] Importing binary data

[Tony Plate]

Probably the simplest way to improve the speed of your code would be to 
write the data so that all the data in a column is contiguous.  Then you'll 
be able to read each column with a single call to readBin().

hope this helps,

Tony Plate

At Tuesday 04:02 AM 6/1/2004, Uli Tuerk wrote:

>Hi everybody!
>
>I've a large dataset, about 2 Mio entries of the format which I would like
>to import into a frame:
><integer><integer><float><string><float><string><string>
>
>Because to the huge data amount I've choosen a binary format instead
>of a text format when exporting from Matlab.
>My import function is attached below. It works fine for only some entries
>but is deadly slow when trying to read the complete set.
>
>Does anybody has some pointers for me for improving the import or handling
>such large data sets?
>
>Thanks in advance!
>
>Uli
>
>
>
>read.DET.data <- function ( f ) {
>         counter <- 1
>         spk.v <- c()
>         imp.v <- c()
>         score.v <- c()
>         th.v <- c()
>         ses.v <- c()
>         rec.v <- c()
>         type.v <- c()
>         fid <- file( f ,"rb")
>         tempi <- readBin(fid , integer(), size=1, signed=FALSE)
>         while ( length(tempi) != 0) {
>                 spk.v[ counter ] <- tempi
>                 imp.v[ counter ] <- readBin(fid, integer(), size=1, 
> signed=FALSE)
>                 score.v[ counter  ] <- readBin(fid, numeric(), size=4)
>                 type.v[ counter ] <- readBin(fid, character())
>                 th.v[ counter ] <- readBin(fid, numeric(), size=4)
>                 ses.v[ counter ] <- readBin(fid, character())
>                 rec.v[ counter ] <- readBin(fid, character())
>                 counter <- counter + 1
>                 tempi <- readBin(fid, integer(), size=1, signed=FALSE)
>         }
>         close( fid )
>         spkf <- factor ( spk.v )
>         impf <- factor ( imp.v )
>
>         det.f <- data.frame( spk=spkf, imp=impf, score=score.v, th=th.v, 
> ses=ses.v, rec=rec.v, type=type.v)
>
>         det.f
>}
>
>______________________________________________
>R-help at stat.math.ethz.ch mailing list
>https://www.stat.math.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html