[R] Memory error

[Andrew Perrin]

Zoinks, thanks! I will seek to pare that file down and try again.

Andy

----------------------------------------------------------------------
Andrew J Perrin - andrew_perrin (at) unc.edu - http://perrin.socsci.unc.edu
Assistant Professor of Sociology; Book Review Editor, _Social Forces_
University of North Carolina - CB#3210, Chapel Hill, NC 27599-3210 USA
New Book: http://www.press.uchicago.edu/cgi-bin/hfs.cgi/00/178592.ctl



On Thu, 8 Mar 2007, Uwe Ligges wrote:

>
>
> Andrew Perrin wrote:
>> Greetings-
>> 
>> Running R 2.4.0 under Debian Linux, I am getting a memory error trying to 
>> read a very large file:
>> 
>>> library(foreign)
>>> oldgrades.df <- read.spss('Individual grades with AI (Nov 7 
>>> 2006).sav',to.data.frame=TRUE)
>> Error: cannot allocate vector of size 10826 Kb
>> 
>> 
>> This file is, granted, quite large:
>> 
>> aperrin at perrin:/data0/grading$ ls -l
>> total 630304
>> -r-xr-xr-x 1 aperrin aperrin 271210015 2007-03-06 15:54 Individual grades 
>> with AI (Mar 2 2007).sav
>> -r-xr-xr-x 1 aperrin aperrin 353209140 2007-03-06 15:57 Individual grades 
>> with AI (Nov 7 2006).sav
>> 
>> 
>> ...but there ought to be plenty of resources. The machine is a dual-Xeon 
>> 2.8Ghz with 6GB of RAM and enormous swap. It's doing almost nothing else 
>> when I try the load, and at the moment it returned the error, this was the 
>> status of top:
>> 
>> Mem:   6750980k total,  4668388k used,  2082592k free,   141820k buffers
>> Swap: 19535032k total,        8k used, 19535024k free,   749244k cached
>>
>>    PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
>>   6168 aperrin   25   0 3015m 2.9g 2880 R  100 45.6   7:52.93 R
>
>
> You have hit the 3GB limit. Since your Xeon is probably a P4 architecture 
> (which means 32 bit), you cannot go on here. Probably it is the best idea to 
> put the data from SPSS into some database and use that database from R.
>
> Uwe Ligges
>
>
>
>> 
>> 
>> Background info:
>> aperrin at perrin:~$ uname -a
>> Linux perrin 2.6.18 #1 SMP Tue Feb 6 14:20:44 EST 2007 i686 GNU/Linux
>> 
>> aperrin at perrin:~$ R --version
>> R version 2.4.0 Patched (2006-11-25 r39997)
>> Copyright (C) 2006 The R Foundation for Statistical Computing
>> ISBN 3-900051-07-0
>> 
>> 
>> 
>> Any thoughts? I would be happy to compile R locally if that would help.
>> 
>> Andy
>> 
>> 
>> ----------------------------------------------------------------------
>> Andrew J Perrin - andrew_perrin (at) unc.edu - http://perrin.socsci.unc.edu
>> Assistant Professor of Sociology; Book Review Editor, _Social Forces_
>> University of North Carolina - CB#3210, Chapel Hill, NC 27599-3210 USA
>> New Book: http://www.press.uchicago.edu/cgi-bin/hfs.cgi/00/178592.ctl
>> 
>> ______________________________________________
>> R-help at stat.math.ethz.ch mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide 
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>