[R-sig-ME] quick question regarding your "residual" command for your lmer command - please help me

[David Atkins]

Yugo--

Most of what you write below is related to general statistical questions 
about mixed models, and hence would be more appropriate for a general 
stats listserv or local stat support (you know, there are some good stat 
consulting facilities at UW...).

With respect to the following specific question:

 >> > But my question then is, is this variance estimate the "within group
 >> > variance" explained in different textbooks of multilevel analysis? 
  But

Yes, the "Residual" variance reported by summary() form either lmer() or 
nlme() is what variously gets called "within group" or "level-1" error / 
variance.  For example, we can use this (from a random-intercept) model 
as our estimate of the within-group variance to calculate the intraclass 
correlation coefficient.

As a more general comment, you might check out UCLA's stats webpages, 
which have many helpful examples of stats procedures (including R and 
including mixed models aka multilevel models aka hierarchical linear 
models).  For example, there is R code to accompany Singer and Willett's 
book on longitudinal data analysis, which is one of the best 
introductions I know of for social science types:

http://www.ats.ucla.edu/stat/examples/alda.htm

Hope that helps.

cheers, Dave
-- 
Dave Atkins, PhD
University of Washington
datkins at u.washington.edu
http://depts.washington.edu/cshrb/

August 1 - October 30:

Universitat Zurich
Psychologisches Institut
Klinische Psychologie
Binzmuhlestrasse 14/23
CH-8050 Zurich

+41 44 635 71 75

Again, I will defer to the R-Sig-Mixed-Models mailing list.

On Wed, Aug 29, 2012 at 6:52 AM, Yugo Nakamura <yugonakamura25 at 
gmail.com> wrote:
 > To Professor Bates,
 >
 > Thank you for your reply and for adding me on to the list.  I will check
 > your commands again.
 >
 > To shorten my question, I essentially get confused when researches 
study the
 > level 1 residuals pooled across the groups and not within groups 
separately.
 >
 > I just do not see why and what this analysis entail or suggest for the
 > multilevel analysis and assumptions that do not take the grouping into
 > consideration.
 >
 > Your thoughts and expertise will be deeply appreciated.
 >
 > Above
 >
 >
 > On Tue, Aug 28, 2012 at 11:25 PM, Douglas Bates <bates at 
stat.wisc.edu> wrote:
 >>
 >> I have taken the liberty of cc:'ing the
 >> R-SIG-Mixed-Models at R-project.org mailing list on this reply.  Many of
 >> those who read that list will be able to help you, often more quickly
 >> than I am able to do.
 >>
 >> Your question is a bit confusing in that you say you are using lmer
 >> and then quote the documentation for residuals.lme.  The nlme package,
 >> containing lme and supporting methods, and the lme4 package,
 >> containing lmer, are different.  You should not expect the
 >> documentation for one to apply to the other.
 >>
 >> On Tue, Aug 28, 2012 at 8:20 AM, Yugo Nakamura <yugonakamura25 at 
gmail.com>
 >> wrote:
 >> > To Professor Bates,
 >> >
 >> > I am terribly sorry for this impromptu email.  I have been using your
 >> > lmer
 >> > command for my dissertation (at the University of Washington) but I am
 >> > having some difficulty to fully understand the outputs.  Your 
expertise
 >> > will
 >> > be deeply appreciated.  I would like to thank you in advance for your
 >> > time.
 >> >
 >> > My question pertains to the Residual command of the lmer and the
 >> > variance
 >> > estimates of the level 1 residuals in the  lmer output.  I am not
 >> > certain as
 >> > to how these figures are calculated and what these estimates really
 >> > imply.
 >> >
 >> > The definition of the residual are as follow.
 >> >
 >> > residuals.lme {nlme}
 >> > The residuals at level i are obtained by subtracting the fitted levels
 >> > at
 >> > that level from the response vector (and dividing by the estimated
 >> > within-group standard error, if type="pearson"). The fitted values at
 >> > level
 >> > i are obtained by adding together the population fitted values (based
 >> > only
 >> > on the fixed effects estimates) and the estimated contributions of the
 >> > random effects to the fitted values at grouping levels less or 
equal to
 >> > i
 >> >
 >> > 
http://stat.ethz.ch/R-manual/R-patched/library/nlme/html/residuals.lme.html
 >> >
 >> > Is this definition saying that, the residuals are defined by 
subtracting
 >> > the
 >> > fitted values from the fixed effects and the empirical Bayes 
estimate of
 >> > the
 >> > level 2 residuals?
 >> >
 >> > I calculated the variance of these residuals and it gave me the same
 >> > estimate of the variance estimate of the "Residuals" (level 1) in the
 >> > lmer
 >> > output.
 >> >
 >> > But my question then is, is this variance estimate the "within group
 >> > variance" explained in different textbooks of multilevel analysis? 
  But
 >> > if
 >> > so I find it slightly too big..  I have learned that within group
 >> > variance
 >> > are normally distributed with mean zero and constant variance for each
 >> > and
 >> > every group.   That is, the variance is calculated within/for each and
 >> > every
 >> > group and this variance is constant across all groups (quite strong
 >> > assumption).
 >> >
 >> > But the variance estimate and the residual command above seems to give
 >> > us
 >> > the pooled residuals regardless of the groups.  That is, the 
variance is
 >> > the
 >> > estimate of the variance across all the residuals 
regardless/ignoring of
 >> > the
 >> > group.   Is this so?   If this is the case, isn't this variance 
the sum
 >> > of
 >> > all the within group variances (simply because the sum of normal
 >> > distribution is also a normal distribution with the mean and variance
 >> > also
 >> > summed)?  Thus, to get a within group variance (for each group) I 
should
 >> > divide your Residual variance estimate by the number of groups?
 >> >
 >> > I hope I was able to make sense.  It will be great if you could 
share me
 >> > your expertise.    If there is a link that describes all the 
details of
 >> > your
 >> > command, that will be very helpful as well.
 >> >
 >> > Thank you so much in advance for your time and cooperation!
 >> >
 >> > yours,
 >> >
 >> > Yugo Nakamura
 >> >
 >> >
 >> >
 >> >
 >
 >