[R] Mixed model

[Spencer Graves]

	  I often think carefully about what I want and store only that.  For 
example, I might do something like the following:

	  b1 <- coef(lme(...))
	  kb <- length(b1)
	  B <- array(NA, dim=c(nb, kb))
	  for(i in 1:nb){

		    B[i, ] <- coef(lme(...))	
	  }

	  With "fit[[i]] <- lme(...)", you store, as Doug said, "a copy of the 
model frame (the parts of Dataset that are needed to evaluate the model) 
plus a lot of other information)" for each pass through the loop.  Since 
you are doing a simulation, you probably only really care about a few 
numbers for each "i".  Identify those and only store those each time 
through the loop.

	  spencer graves
p.s.  Have you considered "simulate.lme"?

Stephen wrote:

> Hi
> Thank you for your comments.
> Yes you are correct its a very big data set.
> Perhaps I am best splitting it up and then importing to R.
> The reason for the loop is that I am conducting the equivalent of Split
> file in SPSS.
> Specifically, I am conducting the analysis for each value of on the
> grouping variable 'runnb'.
> If there is a less memory intensive way of doing this I'd appreciate
> knowing about it.
> Many Thanks and comments appreciated
> Regards
> Stephen
> 
> ________________________________
> 
> From: Douglas Bates [mailto:dmbates at gmail.com]
> Sent: Sun 26/06/2005 17:01
> To: Stephen
> Cc: r-help at stat.math.ethz.ch
> Subject: Re: [R] Mixed model
> 
> 
> 
> On 6/26/05, Stephen <szlevine at nana.co.il> wrote:
> 
>>
>>
>>Hi All,
>>
>>
>>
>>I am currently conducting a mixed model. I have 7 repeated measures on
> 
> a
> 
>>simulated clinical trial. If I understand the model correctly, the
>>outcome is the measure (as a factor) the predictors are clinical group
>>and trial (1-7). The fixed factors are the measure and group. The
> 
> random
> 
>>factors are the intercept and id and group.
>>
>>
>>
>>I tried using 2 functions to calculate mixed effects.
>>
>>Following previous correspondence .
>>
>>
>>
>>Dataset <- read.table("C:/Program
> 
> Files/R/rw2011/data/miss/model1a.dat",
> 
>>header=TRUE, sep="\t", na.strings="NA", dec=".", strip.white=TRUE)
>>
>>attach(Dataset)
>>
>>
>>
>>require (nlme)
>>
>>with(Dataset, table(runnb, id, grp))
>>
>>b.lvls <- table(Dataset$runnb)
>>
>>nb <- length(b.lvls)
>>
>>fit <- vector(mode="list", nb)
>>
>>
>>
>>for(i in 1:nb)
>>
>> fit[[i]]<- lme (trans1 ~ Index1 + grp,
>>
>>            random = ~ 1 | id / grp ,
>>
>>            data = Dataset,
>>
>>            na.action = "na.exclude")
>>
>>
>>
>>
>>
>>This (above) worked OK only I am having memory problems.
>>
>>I have a gig of RAM set at --sdi --max-mem-size=512M (complete version
>>below)
>>
>>I am wondering if running the file as a database be slower / faster?
>>
>>
>>
>>Then I read that lme4 does it quicker and more accurately
>>
>>so I thought that I should re-run the code but from the for line:
>>
>>
>>
>>
>>>for (i in 1:nb)
>>
>>+  fit[[i]]  <- lmer(trans1 ~ Index1 + grp + (1|id:grp) + (1|id),
>>
>>+ Dataset, na.action = na.exclude)
>>
>>
>>
>>Producing
>>
>>
>>
>>Error in lmer(trans1 ~ Index1 + grp + (1 | id:grp) + (1 | id),
> 
> Dataset,
> 
>>:
>>
>>        flist[[2]] must be a factor of length 200000
>>
>>In addition: Warning messages:
>>
>>1: numerical expression has 200000 elements: only the first used in:
>>id:grp
>>
>>2: numerical expression has 200000 elements: only the first used in:
>>id:grp
> 
> 
> Check
> 
> str(Dataset)
> 
> and, if necessary, convert id to a factor with
> 
> Dataset$id <- factor(Dataset$id)
> 
> 
> In is not surprising that you are running into memory problems.  Look
> at the size of one of the fitted objects from lme or from lmer.  They
> are very large because they contain a copy of the model frame (the
> parts of Dataset that are needed to evaluate the model) plus a lot of
> other information.  You have a large Dataset and you are saving
> multiple copies of it although I must admit that I don't understand
> why the calls to lme or lmer are in a loop.
> 
> 
> 
> ???? ?"? ???? ????
> http://mail.nana.co.il
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html

-- 
Spencer Graves, PhD
Senior Development Engineer
PDF Solutions, Inc.
333 West San Carlos Street Suite 700
San Jose, CA 95110, USA

spencer.graves at pdf.com
www.pdf.com <http://www.pdf.com>
Tel:  408-938-4420
Fax: 408-280-7915