[R] Slight discrepancy between predict.lm() and all.effects()

[Prof Brian Ripley]

On Mon, 27 Nov 2006, Michael Kubovy wrote:

> In the course of exploring response prediction, I stumbled upon a
> small discrepancy between the CIs produced by predict.lm() and
> all.effects()
>
> require(mlmRev)
> require(effects)
> hsb.lm <- lm(mAch ~ minrty * sector, Hsb82)
> hsb.new <- data.frame(
>     minrty = rep(c('No', 'Yes'), 2),
>     sector = rep(c('Public', 'Catholic'), each = 2))
> hsb.eff <- all.effects(hsb.lm)
> cbind(
>     hsb.new,
>     predict(hsb.lm, hsb.new, interval = 'confidence', type =
> 'response')
> )
> # the following lower and upper bounds differ starting with the
> fourth decimal place
> data.frame(
>     hsb.new,
>     fit = hsb.eff[[1]]$fit,
>     lwr = hsb.eff[[1]]$lower,
>     upr = hsb.eff[[1]]$upper
> )
>
> Is this due to rounding or algorithm?

Neither: a statistical methodology difference.

predict.lm uses a t reference distribution.  effect.lm uses a Normal 
reference distribution.  (My guess is because it is designed to work also 
with GLM fits, but perhaps John Fox can elaborate and consider using a t 
distribution for lm and gaussian glm models.)  Of course in this example 
there is no practical difference, but there could be in others.

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595