[R] repost: problems with lm for nested fixed-factor Anova (ANOVA I)

Richard M. Heiberger rmh at temple.edu
Thu Feb 12 18:18:53 CET 2009

tmp <- data.frame(y=rnorm(15000),
                  x1 <- factor(sample(48, 15000, replace=TRUE)),
                  z1 <- factor(sample(242, 15000, replace=TRUE)))
            tmp.aov <- aov(y ~ x1/z1, data=tmp)
## exceeds memory

tmp2 <- data.frame(y=rnorm(15000),
                   x1 <- factor(sample(48, 15000, replace=TRUE)),
                   z1 <- factor(sample(5, 15000, replace=TRUE)))
            tmp2.aov <- aov(y ~ x1/z1, data=tmp2)
## about 5 seconds

Use data.frames.  They make it easier to read.
Use aov() instead of lm().  It is the same arithmetic,
but the unneeded columns of X are handled more gracefully.

My guess is that your data has 100s of distinct values for z1.
Therefore excess space was allocated.  It is easier to understand with
distinct values of z1, but as you see it is costly in computer

You can force the actual numerical values of the second term to be
distinct across levels of x1 with the interaction() function.  Then
use the simpler model and let the linear dependencies work in your

            tmp.aov <- aov(y ~ x1 + interaction(x1, z1), data=tmp)
## about 6 seconds


More information about the R-help mailing list