[R-sig-ME] sandwich variance estimation using glmer?

Fri Nov 5 17:12:15 CET 2010

On 11/4/10 3:06 PM, Andrew Robinson wrote:
> That's an interesting point, Dave.  Surely, however, there are better
> model diagnostics?  By 'better' I mean more likely to pinpoint the
> source of lack of fit within a model?
>
> Personally I wouldn't rely on the concordance between robust and
> asymptotic SE as an indicator of model appropriateness.  I don't see how
> it adds anything important to the diagnostic process.
>
> 1) if the match is bad, I'll examine the diagnostics, but
>
> 2) if the match is good, I'll examine the diagnostics anyway, as part of
> due diligence.

Andrew--

You are, of course, absolutely correct.  I use R almost exclusively, and 
for the most part, not having robust SE doesn't really cause any 
problems for me (in general, or with respect to checking diagnostics... 
actually, I love it!).

 From my own experience, I have seen the HLM's software use of both be 
helpful in the following way: I have several times had colleagues / 
students come to me with output from HLM, noting the discrepancy between 
SE and wondering what this means?  In essence, b/c that software puts 
both of them in front of you, it's hard to miss when they are different 
(esp. since p-values will then be radically different).

These times tend to be "teachable moments" in terms of what it is saying 
about the model and data (and fit).  More often that not, the outcome 
has been some form of count variable (which lurk quite commonly in 
psychological/psychiatric waters), and thus we can talk about 
distributional properties of the outcome vs. model, etc. (And I think 
the mean-variance relationship of the Poisson tends to lead to large 
adjustments with the robust SE.)

I would in no way suggest that lme4 should follow suit with the HLM 
software, and I honestly doubt whether it would serve a similar purpose, 
as more novice statistical users tend to be intimidated by R (given lack 
of menus, GUI, etc.).

As I mentioned before, what I *would* be interested in are robust 
approaches a la incorporating t-distribution in prior or likelihood.  I 
am currently working with a colleague on an analysis of a small number 
of groups, where we have discused this -- he's exploring WinBUGS as an 
option at present (to follow-up on our analyses with glmer and MCMCglmm).

For what it's worth (and realizing it ain't all that much about R...).

cheers, Dave

>
> Also I don't know of any theory that suggests that deviation between
> robust and asymptotic standard error estimates *always* indicates a
> model problem.  If anyone does, I'm happy to learn about it.
>
> On the other hand, robust SE are (likely to be) larger than asymptotic
> SE.  So, if I see a deviation, it could be because (a) there's a problem
> with my model, and (b) the model is fine but I'm paying a price for
> robustness.
>
> All of which raises an interesting question: if we observe a deviation
> between the estimate and hunt through the model diagnostics to find a
> cause, but can't --- then what do we do?
>
> Cheers
>
> Andrew
>
>
> On Fri, Nov 5, 2010 at 2:59 AM, David Atkins <datkins at u.washington.edu
> <mailto:datkins at u.washington.edu>> wrote:
>
>
>     One slightly different perspective on robust SE in mixed models:
>
>     The place where I have seen these used regularly is in the HLM
>     software (popular in education and psychology circles).  HLM
>     *always* reports both "standard" and robust SE.
>
>     What I find interesting is that if you read Raudenbush and Bryk (and
>     the HLM manual), they suggest using the robust SE as a model
>     diagnostic (my term).  That is, when there is a discrepancy between
>     SE, they rightly note that something is amiss, and you should do
>     further detective work related to the random-effects specification.
>
>     That seems like a very valid use of robust SE, though I fully
>     acknowledge such info (ie, model isn't fitting well) could be got
>     other ways.
>
>     [BTW, I'd love to see other robust approaches, such as t-distributed
>     error and/or priors, but as Ben notes that's an awfully high bar to
>     implement -- either in lmer or MCMCglmm.  The heavy package is an
>     initial attempt, but seems to be "stalled out" at the moment.]
>
>     For what it's worth.
>
>     cheers, Dave
>
>
>
>     Harold wrote:
>
>     Let me push on this just a bit to spark further discussion. The OP
>     was interested in robust standard errors given misspecification in
>     the likelihood. So, one possible avenue was to explore Huber-White
>     standard errors, or the sandwich estimator, to account for this
>     misspecification and obtain "better" standard errors, but still use
>     the point estimates of the fixed effects as given.
>
>     Some discussion on this has noted that the misspecification occurs
>     in many ways, sometimes given that distributional assumptions were
>     not met. Let's assume someone was willing and skilled to code up the
>     HW as a possible solution within lmer to account for not meeting
>     certain distributional assumptions.
>
>     My question is now why not directly code up models that permit for
>     different distributional assumptions, such as t-distributions of
>     residuals (random effects) or whatever the case might be? In other
>     words, why not write code that addresses the problems directly
>     (misspecification of the likelihood) rather than focusing on HW
>     estimates.
>
>     Isn't it a better use of time and energy to focus on properly
>     specifying the likelihood and estimating parameters from that model
>     rather than HW?
>
>     --
>     Dave Atkins, PhD
>     Research Associate Professor
>     Department of Psychiatry and Behavioral Science
>     University of Washington
>     datkins at u.washington.edu <mailto:datkins at u.washington.edu>
>
>     Center for the Study of Health and Risk Behaviors (CSHRB)
>     1100 NE 45th Street, Suite 300
>     Seattle, WA  98105
>     206-616-3879
>     http://depts.washington.edu/cshrb/
>     (Mon-Wed)
>
>     Center for Healthcare Improvement, for Addictions, Mental Illness,
>       Medically Vulnerable Populations (CHAMMP)
>     325 9th Avenue, 2HH-15
>     Box 359911
>     Seattle, WA 98104
>     http://www.chammp.org
>     (Thurs)
>
>     _______________________________________________
>     R-sig-mixed-models at r-project.org
>     <mailto:R-sig-mixed-models at r-project.org> mailing list
>     https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>
>
>
>
> --
> Andrew Robinson
> Program Manager, ACERA
> Senior Lecturer in Applied Statistics                      Tel:
> +61-3-8344-6410
> Department of Mathematics and Statistics            Fax: +61-3-8344 4599
> University of Melbourne, VIC 3010 Australia
> Email: a.robinson at ms.unimelb.edu.au
> <mailto:a.robinson at ms.unimelb.edu.au>    Website:
> http://www.ms.unimelb.edu.au
>
>