[R] read.spss question warning compression bias

[Marc Schwartz]

On Thu, 2003-12-11 at 09:44, Peter Flom wrote:
> Hello again
> 
> I have a file from SPSS in .sav format.
> 
> when I run
> 
> library(foreign)
> cvar<-as.data.frame(read.spss("c:\\NDRI\\cvar\\data\\cvar2rev3.sav"))
> 
> I get a warning
> 
> Warning message: 
> c:\NDRI\cvar\data\cvar2rev3.sav: Compression bias (0) is not the usual
> value of 100. 
> 
> The data appear to be OK, but I am concerned.  
> 
>  (I tried searching the archives and the documenation for data import
> export, but saw nothing).
> 
> 
> Thanks as always
> 
> Peter


The error message appears to be coming from sfm-read.c in the package.
The particular code is at line 682:

  ext->bias = hdr.bias;
  if (ext->bias != 100.0)
    warning("%s: Compression bias (%g) is not the usual "
	    "value of 100.", h->fn, ext->bias);

I have not used SPSS, but I presume that as with other applications, it
can save the data files in either compressed or non-compressed formats.

>From reading the code in the above source file, there appears to be a
SPSS header structure that contains information on the nature of the
file and its contents.

One of the entries in the structure indicates whether or not the data
file is compressed. That "flag" is then stored in:

  ext->compressed = hdr.compressed;

at line 669. That value is either 0 (FALSE) or 1 (TRUE).

That information comes into play later on at line 1393, which is the
read_compressed_data function.

It would seem that if the data file is not compressed, which I presume
is the case with your file, the check of the compression bias at line
682 is superfluous.

Thus, if my read is correct, the fix (if one is needed) would be to add
a check in advance of the bias check code:

  if (ext->compressed)
  {
    ext->bias = hdr.bias;
    if (ext->bias != 100.0)
      warning("%s: Compression bias (%g) is not the usual "
              "value of 100.", h->fn, ext->bias);
  }


An additional question might be, if the file is not compressed, what is
the default bias value set by SPSS? If it is 0, then the check is
meaningless. On the other hand, if the default value is 100, whether or
not the file is compressed, then the warning message would serve a
purpose in flagging the possibility of other issues. Reasonably, that
setting may be SPSS version specific.

Hopefully, I am close. If so, your data should be correct and the
warning is just that, a "warning" and not an "error".


HTH,

Marc Schwartz