[R] Memory issues..

[JFRI (Jesper Frickman)]

How much processing takes place before you get to the lme call? Maybe R
has just used up the memory on something else. I think there is a fair
amount of memory leak, as I get similar problems with my program. I use
R 1.8.0. My program goes as follows.

1. Use RODBC to get a data.frame containing assays to analyze (17 assays
are found).
2. Define an AnalyzeAssay(assay, suffix) function to do the following:
	a) Use RODBC to get data.
	b) Store dataset "limsdata" in workspace using the <<- operator
to avoid the following error in qqnorm.lme: Error in eval(expr, envir,
enclos) : Object "limsdata" not found, when I call it with a grouping
formula like: ~ resid(.) | ORDCURV.
	c) Call lme to analyze data.
	d) Produce some diagnostic plots. Record them by setting
record=TRUE on the trellis.device
	e) Save the plots on win.metafile using replayPlot(...)
	f) Save text to a file using sink(...)

3. Call the function for each assay using the code:

# Analyze each assay
for(i in 1:length(assays[,1]))
{
	writeLines(paste("Analyzing ", assays$DILUTION[i], " ",
assays$PROFNO[i], "...", sep=""))
	flush.console()
	AnalyzeAssay(assays$DILUTION[i], assays$PROFNO[i])

	# Clean up memory
	rm(limsdata)
	gc()
}

As you can see, I try to remove the dataset stored in workspace and then
call gc() to clean up my memory as I go.

Nevertheless, when I come to assay 11 out of 17, it stops with a memory
allocation error. I have to quit R, and start again with assay 11, then
it stops again with assay 15 and finally 17. The last assays have much
more data than the first ones, but all assays can be completed as long
as I keep restarting...

Maybe restarting the job can help you getting it done?

Cheers,
Jesper

-----Original Message-----
From: Rodrigo Abt [mailto:rodrigo.abt at sii.cl] 
Sent: Monday, November 10, 2003 11:02 AM
To: r-help at stat.math.ethz.ch
Subject: [R] Memory issues..


Hi dear R-listers, I'm trying to fit a 3-level model using lme in R. My
sample size is about 2965 and 3 factors:

year (5 levels), ssize (4 levels), condition (2 levels).

When I issue the following command:

>
lme(var~year*ssize*condition,random=~ssize+condition|subject,data=smp,me
thod
="ML")

I got the following error:

Error in logLik.lmeStructInt(lmeSt, lmePars) :
        Calloc could not allocate (65230 of 8) memory
In addition: Warning message:
Reached total allocation of 120Mb: see help(memory.size)

I'm currently using a Win2000 machine with 128Mb RAM and a 1.2 Gb
processor. My version of R is 1.7.1.

Thanks in advance,

Rodrigo Abt.
Department of Economic and Tributary Studies,
SII, Chile.