[R-sig-ME] suggestions for estimating confidence intervals with only 1.5 GB of ram?

Darren Norris doon75 at hotmail.com
Thu Mar 31 23:58:01 CEST 2011


Dear all,
Doug has previously explained how to approximate confidence intervals 
for parameters
( <AANLkTin+CRAH6EoeK9+-BS4+cwRr6sO-r4dgkRinqOjy at mail.gmail.com 
<mailto:AANLkTin+CRAH6EoeK9+-BS4+cwRr6sO-r4dgkRinqOjy at mail.gmail.com>> )
and Ben Bolker (GLMM wiki - http://glmm.wikidot.com/faq) and Mike 
Laurance (ezPredict in package "ez") have made functons / shared code  
for calculating and plotting fitted values from lmer models and intervals.
Just some examples of what I have found  - Fantastic! A big thank you 
for all the time taken. What more could I need?

The problem and reason for this email ( apologies for the length but I 
found it hard to explain) :
I would like to obtain (then plot) the fitted (?predicted - sorry never 
know correct term) response values and in an ideal world 95% confidence 
(or prediction) intervals for a lmer model.
Using Bens code example as adapted in Mikes "ezPredict" I run out of 
memory when trying to estimate confidence intervals (I think Ben may 
have given a health warning about this issue but I can't find the 
email). This is in no way a complaint / critisim but simply a request 
for help to make the best use of the resources at my disposal.

Why do I need a confidence interval? Because the "upper" values are 
biologically important - they will represent the minimum abundance of a 
turtle species.

The dataframe (link to data and code below) has 333 rows. I am trying to 
predict the response values with 7 factors (2 continuous and 5 
categorical - total of 24 parameters). In an ideal world I would get an 
extra 3 columns of prediction, lower interval, upper interval to add to 
my dataframe.

Is my model appropriate for deriving confidence intervals - i.e. even if 
I had 64 bit linux with 1TB of ram would it work?
Can anyone suggest alternatives for how I can obtain predictions and 95% 
confidence interval for this data set with a limit of 1.5GB of memory?

Thank you for any suggestions, more details below.
Darren

My windows 32 bit laptop has a memory limit of approx1.5 GB for R 
("sessionInfo()" below), I do not have finances to buy a new laptop.
The 64 bit windows 7 desktop in my laboratory has 8GB of ram for R. No 
more money to buy a new desktop and as is for use in a group, even 
installing a virtual linux distro is not an option.

My surprise is that even on the desktop with 8GB ram I ran out of memory.

I feel that I must be doing something wrong or "missing a trick" to 
obtain predictions and intervals for my model which is based on a 
relatively small amount of data.
I am aware that even with my "small" dataset the |"expand.grid|" 
function returns a "big amount" of data.
Is there any way to make the workflow / use of these functions (that Ben 
kindly shared here http://glmm.wikidot.com/faq section "Predictions 
and/or confidence (or prediction) intervals on predictions") work more 
efficiently?

The data is available at this link as a R workspace  
"DarrensSpace.RData"  (size is approx 70 kb): 
http://cid-f0a9fa3480208398.office.live.com/self.aspx/lmeData/DarrensSpace.RData

####   the random effect (1|yearMonth) is used to model serial temporal 
autocorrelation,
####   data is  observational & ecological so unbalanced
#### "yall " is abundance of the turtle species of interest
library("lme4") 
fmer2f<-lmer(yall~yearSeason+sun+total_precip_trip+mean_temp_trip+surveyArea+obscat+hour_period+(1|yearMonth),REML=FALSE,data=df.p)
library("ez")
er_pred<- ezPredict(fit = fmer2f) ## runs out of memory here

##### session info for the laptop I use
R version 2.12.1 (2010-12-16)
Platform: i386-pc-mingw32/i386 (32-bit)

locale:
[1] LC_COLLATE=English_United Kingdom.1252  LC_CTYPE=English_United 
Kingdom.1252    LC_MONETARY=English_United Kingdom.1252
[4] LC_NUMERIC=C                            LC_TIME=English_United 
Kingdom.1252   

attached base packages:
[1] grid      splines   stats     graphics  grDevices utils     
datasets  methods   base    

other attached packages:
 [1] ez_3.0-0           lme4_0.999375-37   Matrix_0.999375-46 
lattice_0.19-13    stringr_0.4        ggplot2_0.8.9      proto_0.3-8      
 [8] reshape_0.8.3      plyr_1.4           reshape2_1.1       
car_2.0-9          survival_2.36-2    nnet_7.3-1         MASS_7.3-9       

loaded via a namespace (and not attached):
[1] nlme_3.1-97   stats4_2.12.1




More information about the R-sig-mixed-models mailing list