[R] Error: cannot allocate vector of size xxx

[David Winsemius]

On Sep 22, 2011, at 5:00 PM, Mario Montecinos Carvajal wrote:

> Michale and Paul
>
> Thanks for your quick response.
>
> Michael, I am running a 32bit version of R
>
>> sessionInfo()
> R version 2.11.1 (2010-05-31)
> i386-pc-mingw32
>
>
> Paul, the dimension of the Data Frame with I am workis is
> dim(d)
>> [1] 70174    11
> And the size of the file that contains the data is 2946 Kb

Not a very big dataset for a workspace that should be 2GB or more.

>
>
> The script that I use for manage the data is:
>
> #clean the workspace
> rm(list = ls(all = TRUE))

I hate it when people put that code in without a warning and  
preferably commented out. Some us boobs have been known to paste in  
code without mentally single stepping through it.

>
> #Increase the memory size
> memory.limit(size=4000)
>
> #Set Directory of WorkSpace where whe have the data
> setwd('C:/Users/XXX/XXX/R/XXXX')
>
> #load data
> dat <- read.table('dat_fin_age.txt')
>
> #delete empty registry
> d<-na.omit(dat)
> remove(dat) # remove dat (intermedia object) for release resource
>
> #Asigned the name to the variables in the data
> names(d)<-
> c 
> ('pesq 
> ','year 
> ','month','sex','length','weigth','mature','age','soi','temp','k')

Bad idea to use "length" as a variable name. It is a function name.

>
> #define the ages over the analysis is focuses.
> minage<-1 # minimum age
> maxage<-6 # maximum age

At this point you should have offered:

  str(d)

>
> # One of the diferent lm I have been tested
>
>

You are constructing a 5-way interaction that will have way more  
interactions than you will know what to do with. Why not start out  
with a simple model and move "up" from there? If these ages and years  
are integer valued then you may not have enough  data to support such  
a model with only 70k records.

What do these models tell you?

l6w.lin <- lm(length~as.factor(age) + as.factor(year) + as.factor(sex)  
+soi + k,

Or:

l6w.2way <- lm(length~ (as.factor(age) + as.factor(year) +  
as.factor(sex) +soi + k)^2 ,


> data=d[d$age
> %in% minage:maxage,])

A much safer way to handle this would be:

data=subset(d, age >= minage & age <= maxage)

%in% is a set operation and you are passing it a vector of integers  
rather than a range.


>
> Today I will prove your recommendation to use the package biglm

Should not be necessary. The problem lies elsewhere.

>
> Thanks to both for your Help
>
> Regards
>
> 2011/9/22 Paul Hiemstra <paul.hiemstra at knmi.nl>
>
>> On 09/22/2011 04:00 AM, R. Michael Weylandt
>> <michael.weylandt at gmail.com> wrote:
>>> Are you running a 32bit or 64bit version of R? Type sessionInfo()  
>>> to see.
>>>
>>> Michael
>>
>> ...in addition, how large is your dataset? Please provide us with a  
>> self
>> contained example which reproduces this problem. You could take a  
>> look
>> at the biglm package.
>>
>> regards,
>> Paul
>>
>>> On Sep 21, 2011, at 10:41 PM, Mario Montecinos Carvajal <
>> mariomontecinos at gmail.com> wrote:
>>>
>>>> Hi
>>>>
>>>> I am a new user of the mail list.
>>>>
>>>> Mi problem occurs when I try to test a lineal model (lm), becouse  
>>>> appear
>> the
>>>> messaje "Error: cannot allocate vector of size xxx"
>>>>
>>>> The data frame whit I am working, Have
>>>>
>>>>> dim(d)
>>>> [1] 70174    11
>>>>
>>>> and the function i am test is:
>>>>
>>>>
>> lm 
>> (length~as.factor(age)*as.factor(year)*as.factor(sex)*soi*k,data=d[d 
>> $age
>>>> %in% minage:maxage,])
>>>>
>>>> I tried with a diferent options for solve this problem, but any  
>>>> one give
>> me
>>>> results.
>>>>
>>>> I tried with:
>>>>
>>>> Change in Function: memory.limit(size=4000)
>>>>
>>>> Change the setting in Windows using  BCDEdit /set
>>>>
>>>> Change in the memory availability for R, change the path of the  
>>>> program
>>>> (suggested for other user)
>>>>
>>>> My computer have 4GB RAM, Have 20 GB of Virtual Memory, HDD 300  
>>>> GB with
>> 200
>>>> GB of free space and Windows 7 as OS and my R version is 2.11.1
>>>>
>>>> I need solve this problem, any help or suggestion will be very well
>>>> received.
>>>>
>>>> I've been thinking in change the OS, but this is may last option.
>>>>
>>>> Regards
>>>>
>>>> I read


David Winsemius, MD
West Hartford, CT