Hello R List,

I'm trying to run a linear mixed effects analysis on fairly large datasets with lmer (from the lme4 package) on a 32-bit Windows machine running XP with 3 GB of RAM. It's not working. (details below)

I've researched the ff and bigmemory packages, but it appears they won't handle the mixed mode dataset I'm analyzing. It has some character fields for the categorical variables. It's also not clear a linear mixed effect regression can be run with those packages. Can anyone point me to a lme solution on larger datasets that can address the 32-bit memory limitations?

The smallest dataframe I'm trying to run is about 120,000 observations and 7 variables, but I'd rather run a for loop script on 620K+ observations. I've seen several recommendations (Kabacoff in 'R in Action' and others) that when possible, run R in a 64-bit build. Problem is I'm on a deadline, and procuring a new computer takes time, and approvals up the food chain. Suggestions? Is 64-bit my only option?

Here is the information about what I'm running:

R version 2.14.0 (2011-10-31)
Platform: i386-pc-mingw32/i386 (32-bit)

locale:
[1] LC_COLLATE=English_United States.1252
[2] LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C
[5] LC_TIME=English_United States.1252

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods
[7] base

other attached packages:
[1] lme4_0.999999-0 Matrix_1.0-5    lattice_0.20-10
[4] RODBC_1.3-6

loaded via a namespace (and not attached):
[1] grid_2.14.0   nlme_3.1-104  stats4_2.14.0 tools_2.14.0

I run this code:

> install.packages("lme4")
> library(lme4)
> math07g4 <- sqlQuery(conn, "select ssid, ss_chg, campus2, district_id, pblack, pfreelnch, pmob
         FROM codemob0607ma WHERE grade2 = 4")
> attach(math07g4)
> fit07ma4 <- lmer(ss_chg ~ 1 + factor(campus2) + factor(district_id) + pblack + pfreelnch + pmob +
   (1 | campus2) + (1 | district_id), data=math07g4)

And I get this:

Error: cannot allocate vector of size 2.5 Gb
In addition: Warning messages:
1: In model.matrix.default(mt, mf, contrasts) :
  Reached total allocation of 2187Mb: see help(memory.size)
2: In model.matrix.default(mt, mf, contrasts) :
  Reached total allocation of 2187Mb: see help(memory.size)
3: In model.matrix.default(mt, mf, contrasts) :
  Reached total allocation of 2187Mb: see help(memory.size)
4: In model.matrix.default(mt, mf, contrasts) :
  Reached total allocation of 2187Mb: see help(memory.size)


Thanks in advance,

Ken

Ken McCall | Database reporter
Dayton Daily News<http://www.daytondailynews.com/> | 1611 S. Main St. Dayton, Ohio 45409
Ph: 937-000-000  Cell: 937-225-2393  Email: ken.mccall@coxinc.com<mailto:ken.mccall@coxinc.com>
COMPLETE. IN-DEPTH. DEPENDABLE.


	[[alternative HTML version deleted]]

