[R-sig-ME] Penalty = shrinkage = ?

[Douglas Bates]

On Thu, Nov 19, 2009 at 11:56 AM, Prew, Paul <Paul.Prew at ecolab.com> wrote:
> I found the comment below interesting in one of yesterday's threads, as I am currently analyzing a data set with a random effect fully nested within a fixed factor.  Could anyone elaborate on what is meant by "penalty on the random effect"? Is this also what is deemed "shrinkage"?  How does it work?  Thanks, Paul

Look at slide 25 in
http://lme4.r-forge.r-project.org/slides/2009-07-21-Seewiesen/5LongitudinalD.pdf

In this slide the parameter estimates that you would have gotten by
fitting each subject's data separately are compared with the estimates
from a mixed-effects model with random effects for slope and
intercept.  The effective slope and intercept for each subject is
shrunk toward the population-wide estimate compared to the
within-subject estimate.  John Tukey referred to this as "borrowing
strength" from the population.

The extent of the shrinkage is controlled by the variance-covariance
matrix of the random effects.  A large variance results in parameter
estimates that are close to the within-subject estimates.  In terms of
the discussion on fidelity to the data versus model complexity in
another thread, such a model has high complexity and high fidelity.
The opposite case, very low variance for the random effects provides a
low complexity model but with correspondingly low fidelity to the
data.

Slide 26 in that presentation shows that the subjects whose data is
rather noisy, and hence whose within-subject parameter estimates are
poorly determined (330 or 331), have their coefficients "shrunk" more
than those whose data determines the within-subject estimates very
well (309 or 349).

> "I understand that when a random effect is fully
> nested within a fixed effect, the penalty on the random effect
> resolves the singularity and allows estimation of both. (That is, if
> appropriate, you could model depfemr as a fixed effect?)"
>
>
>
> Paul Prew   ▪  Statistician
> 651-795-5942   ▪   fax 651-204-7504
> Ecolab Research Center   ▪  Mail Stop ESC-F4412-A
> 655 Lone Oak Drive   ▪   Eagan, MN 55121-1560
>
>
>
> CONFIDENTIALITY NOTICE:
> This e-mail communication and any attachments may contain proprietary and privileged information for the use of the designated recipients named above.
> Any unauthorized review, use, disclosure or distribution is prohibited.
> If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message.
>
>
>        [[alternative HTML version deleted]]
>
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>
>