[R] R_decompress1 and zlib

[Jan.Kleinn at partnerre.com]

Dear all,

I have a problem similar to what was already posted about two years ago
(http://tolstoy.newcastle.edu.au/R/help/05/01/10011.html): I have a binary
file, which contains a header and data compressed by zlib and written to
the file as unsigned character. Here is, what I try to do:

> zz <- file(in.file, 'rb')
> ## the header has to be read in three parts due to the different data
types
> header1 <- readBin(zz, integer(), 8, size = 4)
> header2 <- readBin(zz, numeric(), 5, size = 8)
> header3 <- readBin(zz, integer(), 14, size = 4)
> data.raw <- readBin(zz, raw(), 1e6)
> close(zz)
> ## it works fine up to here
> mydata <- .Call('R_decompress1', data.raw)
Error: cannot allocate vector of size 1976123 Kb
In addition: Warning messages:
1: Reached total allocation of 1534Mb: see help(memory.size)
2: Reached total allocation of 1534Mb: see help(memory.size)
> length(data.raw)
[1] 481

The .Call is, where R complains about not having enough memory.
Interestingly, when I generate some test-data, compress it, write it to a
file, read it from the file and decompress it, it works fine:
> t <- round(abs(rnorm(1000)), digits = 2)
> t.asc <- paste(t, collapse = ' ')
> t.raw <- charToRaw(t.asc)
> t.comp <- .Call('R_compress1', t.raw)
> ## write to file
> zz <- file('test.file', 'wb')
> writeBin(t.comp, zz)
> close(zz)
> ## read from file
> zz <- file('test.file', 'rb')
> comp.in <- readBin(zz, raw(), 1e6)
> close(zz)
> t.decomp <- .Call('R_decompress1', comp.in)
> out.asc <- rawToChar(t.decomp)
> out.num <- as.numeric(unlist(strsplit(out.asc, ' ')))
> length(comp.in)
[1] 1627

I don't understand, why the decompression of the 481 element long raw
vector out of my input file does not work, whereas the decompression works
fine for a raw vector of over 1000 elements I created myself. Could it be,
that my data is not properly zlib-compressed? Does anyone know, whether
there could be some other How could I check the integrity of the data? Any
ideas are greatly appreciated.

I'm working on Windows XP with R version 2.4.1 under ESS in Emacs (but I
suspect, this shouldn't matter):
> version
               _
platform       i386-pc-mingw32
arch           i386
os             mingw32
system         i386, mingw32
status
major          2
minor          4.1
year           2006
month          12
day            18
svn rev        40228
language       R
version.string R version 2.4.1 (2006-12-18)

Many thanks in advance and best regards, Jan



DISCLAIMER: This e-mail contains information solely intended...{{dropped}}