[R-sig-ME] Assumptions of random effects for unbiased estimates

[Poe, John]

Laura,

I think we might be talking past each other somewhat on the FE vs RE
discussion. An RE model that has only level one (time or group varying)
covariates and a latent variable for the expected value of group membership
on Y is problematic for the reasons that you are talking about. But mixed
effects models aren't typically the same specification as the FE model or
the RE model. They either use group/grand mean centering of variables to
instrument the problem or random coefficients to stop making the assumption
of no correlation altogether.

The primary benefit of a mixed effects approach is that you have a better
handle on all of the factors that are influencing Y and you can make decent
predictions about cases given a particular context. By using the fixed
effects model you are literally making the choice not to care about things
that you know influence Y.

My usual advice if you don't care about between group effects or
cross-level effects AND you don't care about making predictions about
individuals or groups then fixed effects models that difference out between
group variability are fine for linear models. Given Malcolm's working paper
I might reevaluate that recommendation going forward.

On point 1 that you raise about credibility of the argument that group mean
centered models actually do fix the problems associated with random effects
correlation:

   - There's not actually a credibility problem here. There is a
   specification test in econometrics that is analogous to a Hausman test but
   compares fixed effects estimates to a hierarchical linear model. It's
   called a Mundlak specification test
   <http://blog.stata.com/2015/10/29/fixed-effects-or-random-effects-the-mundlak-approach/>.
   So you can demonstrate that it's not a problem.
   - The typical response when this test shows that there is still a
   violation of the no correlation between a random effect and a level 1
   variable assumption is to stop making that assumption and use a random
   coefficients model. The Bell and Jones paper does a good job of working
   through the logic here.

On your point about the comment :"But group-mean centering can also be done
with random effects models, with the same benefit you get with fixed
effects models (isolation of the within effects), while still allowing for
estimation of the between relationships"

   - Group mean centering of variables tends to instrument the endogeneity
   between the RE and the level one variables so that the correlation isn't an
   issue anymore. In the basic RE model without level 2 covariates and without
   centering you have a problem. But that model doesn't tend to get used much
   without criticism in political science, psychology, or economics (I'm not
   an ecologist so I can't say for practice in your field). It seems to be
   something of a straw-man in my experience.

On point 2

Based on John Poe's response and example with the income, I think that is
> an argument that the model identification is wrong if you don't allow mean
> income versus deviations from mean income to have different effects on consumption,
> rather than an argument that RE solve the problem of unobserved
> heterogeneity more credibly than FE. This is a point about model specification
> rather than dealing with unobservable heterogeneity.


   - All unobserved heterogeneity problems are about model specification.
   It's an omitted variable bias problem. Random effects models are literally
   just a version of the model with an additional latent variable for the
   expected value of group membership. That latent variable can be generated
   directly from coefficients off of dummy variables so they are esentially
   reparamaterizations of one another.
   - In a random effects setting you are including a new latent variable
   and in a random coefficients model you are adding an interaction between
   the random effect and the variable
   - In most cases you can deal with the endogeneity issue by including
   group means and then taking their deviations. When you can't do that, you
   can use dummy variables or the group-level expectations for a random effect
   and then interact that with the endogenous covariate.

 On point 3 for FE in nonlinear models

   - The basic issue is that in high dimensional problems the root finding
   algorithms for nonlinear MLE tend to start to give biased answers as the
   mode diverges away from the mean. This divergence between the mode and the
   mean works as a function of the number of groups and the size of the groups
   so that if T grows to infinity then there's no bias problem. I haven't seen
   solid research on how unbalanced group size influences this but my
   intuition says that it's likely to increase the bias problem. You get bias
   in as few as ten dimensions and there's really no way to fix it with convex
   optimization. So if you want to include dummy variables in something like a
   logit you either need to use some version of expectation maximization or
   integrate over the data then take the jacobian and hessian directly. At
   that point you know where the mean is and you don't have to rely on an
   optimizer so you will get unbiased results. That's why it's not an issue in
   Poisson or a linear model. You can just calculate the derivatives directly.

As far as citations on incidental parameters bias:

   - Lancaster, Tony. 2000. "The incidental parameter problem since 1948."
   Journal of Econometrics 95 (2):391-413.
   - Katz, Ethan. 2001. "Bias in conditional and unconditional fixed
   effects logit estimation." Political Analysis 9 (4):379-84.
   - Hahn, J., & Newey, W. (2004). Jackknife and analytical bias reduction
   for nonlinear panel models. Econometrica, 72(4), 1295-1319.
   - Greene, William. 2004. "The behaviour of the maximum likelihood
   estimator of limited dependent variable models in the presence of fixed
   effects." The Econometrics Journal 7 (1):98-119.
   - Bill Greene's chapter in Baltagi, B. H. (2014). *The Oxford Handbook
   of Panel Data*. Oxford University Press, USA.
   - Note that one of his recommendations is just to use group mean
      centering and the mixed effects framework
   - Beck, Nathaniel. 2015. Estimating grouped data models with a binary
   dependent variable and fixed effects: What are the issues? Paper read at
   annual meeting of the Society for Political Methodology, July

On the bias variance trade-off question I think it's mostly just that
economists aren't interested in prediction as much as estimating average
causal effects. They don't seem to care if their results are applicable to
any particular case so long as they describe the average effect of X on Y
well.


On Wed, Oct 12, 2016 at 9:37 AM, Laura Dee <ledee at umn.edu> wrote:

> Dear all,
> Thanks all - very interesting and helpful responses. I think I should have
> been clearer with my question: in my case, the unobserved heterogeneity
> between groups as not being of interest to study (but something to be
> controlled for to isolate the effects of other x_ij's). Also I'm using a
> linear model setting. The paper Jake and Malcolm sent was very help to lay
> out these issues and suggest the within and between RE model when you want
> to be studying the between group variation. And, I'll go through John Poe's
> slides in detail.
>
> There are four points that have emerged from this discussion that I think
> are worth teasing apart:
>
> *1)*  Interest in studying the mean effects and how they differ between
> groups, which FE do not allow because they remove the mean effect. However,
> even with RE and the ability to study those between differences, you still
> have the challenge of credibly identifying the mean effects of income on
> y_ij -- and whether you have ruled out/controlled for other factors that
> vary cross-sectionally. RE do not solve this issue but are preferred
> because the mean effect between groups is what is of interest. Therfore,
> one is willing to accept some bias in the estimates if there are other
> unobserved variables that vary cross-sectionally and influence the outcome.
>
> Further, Malcolm, I agree that both FE and RE both try to account for a
> group mean but not this statement because of the assumption of RE: "But
> group-mean centering can also be done with random effects models, with the
> same benefit you get with fixed effects models (isolation of the within
> effects), while still allowing for estimation of the between relationships"
> However, estimates are unbiased if FE are correlated with the error term,
> which is not the case for RE. Though, agreed, if it is the between group
> variation that is of interest, then it does not make sense to use a FE
> model and there is too much focus on bias over other issues (i.e.,
> estimating the effect of interest).
>
> *Question: *in the Bell & Jones paper that Jake sent, they present
> the Plümper and Troeger’s (2007) fixed effects vector decomposition. Is
> that used often? I don't think it has made it's way to ecology.
>
> *2)* Based on John Poe's response and example with the income, I think that
> is an argument that the model identification is wrong if you don't allow
> mean income versus deviations from mean income to have different effects on
> consumption, rather than an argument that RE solve the problem of
> unobserved heterogeneity more credibly than FE. This is a point about model
> specification rather than dealing with unobservable heterogeneity.
>
> *3) *Agreed that FE are biased with some forms of non-linear models. Could
> anyone send me some more recent papers on this topic?
>
> *4) *Ben raised the issue of a bias-variance trade-off, which is a good
> point and economists seem to focus more (and maybe too much) on bias.
> However, with enough observations, it's less of a trade-off.
>
> Many thanks to everyone,
> Laura
>
>
>
> On Wed, Oct 12, 2016 at 3:12 AM, Malcolm Fairbrother <
> M.Fairbrother at bristol.ac.uk> wrote:
>
> > As others have said, there are rather peculiar inconsistencies between
> > what the methodological literature knows and what empirical economists
> > actually do.
> >
> > I think the paper Jake cited (by my colleagues at Bristol/Sheffield) is
> > indeed one of the most useful on all this. The published version of the
> > paper is at: http://dx.doi.org/10.1017/psrm.2014.7
> >
> > The following working paper (by them and me) takes up similar themes:
> > https://www.researchgate.net/publication/299604336_Fixed_
> > and_Random_effects_making_an_informed_choice
> >
> > One of the additional limitations we note here with fixed effects models
> > (using a simulation study) is that they can be anti-conservative, in the
> > sense that the SEs they return are too small if the data are generated
> from
> > a random slopes model.
> >
> > In brief, Laura, fixed effects models only estimate within-group
> > relationships, whereas random effects (AKA multilevel, mixed) models can
> > estimate within- and between-group relationships. The estimation of fixed
> > effects models implicitly entails group mean centering (though the models
> > are typically written out as though unit dummies are estimated). But
> > group-mean centering can also be done with random effects models, with
> the
> > same benefit you get with fixed effects models (isolation of the within
> > effects), while still allowing for estimation of the between
> relationships.
> > You might have less confidence that the between component of some x_ij is
> > uncorrelated with the unit error term, but it is still possible for the
> > within (group-mean-centered) component to be correlated with the
> > observation-level error term. So I would agree that bias is worth
> thinking
> > about, but using fixed effects is no more helpful than random effects as
> a
> > solution to the problem.
> >
> > Hope that's useful,
> > Malcolm
> >
> >
> > Dr Malcolm Fairbrother
> > Reader in Global Policy and Politics
> > School of Geographical Sciences  •  Cabot Institute  •  Centre for
> > Multilevel Modelling
> > University of Bristol
> >
> >
> >
> >
> > Date: Tue, 11 Oct 2016 20:49:45 -0500
> >> From: Jake Westfall <jake.a.westfall at gmail.com>
> >> To: r-sig-mixed-models at r-project.org
> >> Subject: Re: [R-sig-ME] Assumptions of random effects for unbiased
> >>         estimates
> >> Message-ID:
> >>         <CAE9_Wg6+ZFXh-9on=nmuUwLKO6ScXjMRfbgf4y+XpGNhVAwJqA at mail.gm
> >> ail.com>
> >> Content-Type: text/plain; charset="UTF-8"
> >>
> >>
> >> What a nice contribution from John!
> >>
> >> Jake
> >>
> >> On Tue, Oct 11, 2016 at 8:11 PM, Poe, John <jdpo223 at g.uky.edu> wrote:
> >>
> >> > My reading of modern work by panel data econometricians is that they
> >> seem
> >> > very fine with the use of mixed effects models that properly
> >> differentiate
> >> > effects at different levels of analysis and the tools to do so have
> >> existed
> >> > in that literature since the early 1980s. They have been borrowing
> >> heavily
> >> > from the mixed effects literature in designing econometric models and
> >> talk
> >> > about them in panel data textbooks. This hasn't typically filtered
> down
> >> to
> >> > applied economists who tend to misunderstand what other fields do
> >> because
> >> > other fields just tend to talk about them differently.
> >> >
> >> > The short version:
> >> > Everyone in the mixed effects literature just uses group/grand mean
> >> > centering and random coefficients to deal with endogeneity bias. If
> you
> >> are
> >> > an economist and someone outside of econ says mixed effects models you
> >> > should think *correlated random effects models* and not *random
> effects
> >> > models*.
> >> >
> >> > The long version:
> >> > Economists are pretty afraid error structures that are correlated with
> >> > independent variables in general and have built up pretty elaborate
> >> > statistical models to deal with the problem. In panel data, this
> >> manifests
> >> > itself as wanting to avoid confounding effects at different levels of
> >> > analysis so that within group varying effects are segregated from
> >> between
> >> > group varying effects. It can also happen when you are omitting higher
> >> > level random effects
> >> > <http://methods.johndavidpoe.com/2016/09/09/independence-acr
> >> oss-levels-in-mixed-effects-models/>
> >> > and they are distorting the structure of the random effects that you
> are
> >> > including. This is generally a good thing as you want to be able to
> test
> >> > hypotheses at specific levels of analysis without confounding.
> >> >
> >> > It's a big enough theoretical concern in the discipline that they
> >> usually
> >> > just want to remove all between group effects from the data as a
> >> *default* to
> >> > get level one effects because it is simpler and more fool proof than
> >> > dealing with the problem in a mixed effects setting. It's so pervasive
> >> that
> >> > they are often socialized into not designing hypotheses for any
> between
> >> > group or cross-level variation and just focus on within group (time
> >> > varying) variability when at all possible (what economists call
> *within
> >> > effects*).
> >> >
> >> > What economists refer to as fixed effects models just difference out
> all
> >> > between group variation so that it cannot contaminate within group
> >> effects
> >> > (bias level one coefficients). It's the equivalent to including group
> >> > indicator variables in the model instead of a random effect and just
> >> > accepting that you can't make substantive inferences about anything at
> >> the
> >> > group level (what economists call *between effects*).
> >> >
> >> > The typical conventional wisdom in applied econometrics is to use a
> >> > Hausman test which is a generic test comparing coefficients between a
> >> > random effects model (with no level 2 covariates) and a model with all
> >> > between group variability removed from the data. If there are
> >> differences
> >> > between the two, then they prefer to go with the latter. This is bad
> >> > practice according to econometrics textbooks but applied people don't
> >> seem
> >> > to care (Baltagi 2013 ch 4.3). This only makes sense if you don't care
> >> > about group invariant variables that only differ crosssectionally
> and/or
> >> > you think of their effects as contamination. Panel data econometrics
> >> > textbooks tend to argue for a wider range of options here but in
> >> practice
> >> > not that many economists seem to use them.
> >> >
> >> > There's an alternative framework in econ for dealing with this problem
> >> > that they call a Mundlak device (Mundlak 1978) or correlated random
> >> effects
> >> > models (Baltagi Handbook of Panel Data 2014 ch 6.3.3 or really any
> panel
> >> > data textbook) which is equivalent to a hierarchical linear model with
> >> > group mean centering for level-one variables. This approach is used in
> >> > econometrics by some pretty standard advanced panel data models (e.g.
> >> > Hausman-Taylor and Arellano Bond). The other alternative that is
> >> advocated
> >> > by panel data econometricians but doesn't seem to have filtered down
> to
> >> > rank and file economists is to use random coefficients models and just
> >> > allow the random effects to be correlated with level one variables
> >> (Hsiao
> >> > 2014 chapter 6 and most of his other written work).
> >> >
> >> > It is important to understand that efficiency isn't the primary reason
> >> for
> >> > use of a mixed effects model over a fixed effects model for most
> >> research.
> >> > A common reason to use a mixed effects model is that you have
> hypotheses
> >> > about variables operating at higher levels of analysis or cross-level
> >> > interactions and those questions cannot be answered by fixed effects
> >> panel
> >> > models that have removed all between group variability from the
> >> analysis.
> >> > You are sacrificing the ability to test group variant hypotheses by
> >> using a
> >> > basic fixed effects model over a mixed effects model. For nonlinear
> >> models
> >> > like a logistic regression it can also be very difficult to use an
> >> unbiased
> >> > fixed effects model (though there are ways in a panel setting e.g.
> Hahn
> >> and
> >> > Newy 2004) and trivial to use a mixed effects model.
> >> >
> >> > Panel data econometricians almost always talk about typical practice
> >> among
> >> > applied economists using fixed effects as flawed (see Baltagi 2013 ch.
> >> > 4.3). Mark Nerlov's 2000 History of Panel Data Econometrics is my
> >> favorite
> >> > example:
> >> >
> >> > The absurdity of the contention that possible correlation between some
> >> of
> >> >> the observed explanatory variables and the individual-specific
> >> component of
> >> >> the disturbance is a ground for using fixed effects should be clear
> >> from
> >> >> the following example: Consider a panel of households with data on
> >> >> consumption and income. We are trying to estimate a consumption
> >> function.
> >> >> Income varies across households and over time. The variation across
> >> >> households is related to ability of the main earner and other
> household
> >> >> specific factors which vary little over time, that is to say, reflect
> >> >> mainly differences in permanent income. Such permanent differences in
> >> >> income are widely believed to be the source of most differences in
> >> >> consumption both crosssectionally and over time, whereas, variations
> of
> >> >> income over time are likely to be mostly transitory and unrelated to
> >> >> consumption in most categories. Yet, fixed-effects regressions are
> >> >> equivalent to using only this variation and discarding the
> information
> >> on
> >> >> the consumption-income relationship contained the cross-section
> >> variation
> >> >> among the household means.
> >> >
> >> >
> >> > See the last couple of pages of this lecture
> >> > <http://www.johndavidpoe.com/wp-content/uploads/2012/09/Blal
> >> ock-Lecture.pdf> for
> >>
> >> > the citations in the econometrics and multilevel literature that I
> >> > referenced.
> >> >
> >> >
> >> >
> >> > On Tue, Oct 11, 2016 at 3:32 PM, Jake Westfall <
> >> jake.a.westfall at gmail.com>
> >> > wrote:
> >> >
> >> >> Hi Laura and Ben,
> >> >>
> >> >> I like this paper on this topic:
> >> >> http://psych.colorado.edu/~westfaja/FixedvsRandom.pdf
> >> >>
> >> >> What it comes down to essentially is that if the cluster effects are
> >> >> correlated with the "time-varying" (i.e., within-cluster varying) X
> >> >> predictor -- so that, for example, some clusters have high means on X
> >> and
> >> >> others have low means on X -- then there is the possibility that the
> >> >> average within-cluster effect (which is what the fixed effect model
> >> >> estimates) differs from the overall effect of X, not conditional on
> the
> >> >> clusters. An extreme example of this is Simpson's paradox. Now since
> >> the
> >> >> estimate from the random-effects model can be seen as a weighted
> >> average
> >> >> of
> >> >> these two effects, it will generally be pulled to some extent away
> from
> >> >> the
> >> >> fixed-effect estimate toward the unconditional estimate, which is the
> >> bias
> >> >> that econometricians fret about. However, if the cluster effects are
> >> not
> >> >> correlated with X, so that each cluster has the same mean on X, then
> >> this
> >> >> situation is not possible, so the random-effect model will give the
> >> same
> >> >> unbiased estimate as the fixed-effect model.
> >> >>
> >> >> A simple solution to this problem is to retain the random-effect
> model,
> >> >> but
> >> >> to split the predictor X into two components, one representing the
> >> >> within-cluster variation of X and the other representing the
> >> >> between-cluster variation of X, and estimate separate slopes for
> these
> >> two
> >> >> effects. One can even test whether these two slopes differ from each
> >> >> other,
> >> >> which is conceptually similar to what the Hausman test does. As
> >> described
> >> >> in the paper linked above, the estimate of the within-cluster
> >> component of
> >> >> the X effect equals the estimate one would obtain from a fixed-effect
> >> >> model.
> >> >>
> >> >> As for the original question, I can't speak for common practice in
> >> >> ecology,
> >> >> but I suspect it may be like it is in my home field of psychology,
> >> where
> >> >> we
> >> >> do worry about this issue (to some extent), but we discuss it using
> >> >> completely different language. That is, we discuss it in terms of
> >> whether
> >> >> there are different effects of the predictor at the within-cluster
> and
> >> >> between-cluster levels, and how our model might account for that.
> >> >>
> >> >> Jake
> >> >>
> >> >> On Tue, Oct 11, 2016 at 1:50 PM, Ben Bolker <bbolker at gmail.com>
> wrote:
> >> >>
> >> >> >
> >> >> >   I didn't respond to this offline, as it took me a while even to
> >> start
> >> >> > to come up to speed on the question.  Random effects are indeed
> >> defined
> >> >> > from *very* different points of view in the two communities
> >> >> > ([bio]statistical vs. econometric); I'm sure there are points of
> >> >> > contact, but I've been having a hard time getting my head around it
> >> all.
> >> >> >
> >> >> > Econometric definition:
> >> >> >
> >> >> > The wikipedia page <https://en.wikipedia.org/wiki
> >> /Random_effects_model>
> >> >> > and CrossValidated question
> >> >> > <http://stats.stackexchange.com/questions/66161/why-do-
> >> >> > random-effect-models-require-the-effects-to-be-uncorrelated-
> >> >> with-the-inpu>
> >> >> > were both helpful for me.
> >> >> >
> >> >> >  In the (bio)statistical world fixed and random effects are usually
> >> >> > justified practically in terms of shrinkage estimators, or
> >> >> > philosophically in terms of random draws from an exchangeable set
> of
> >> >> > levels: e.g. see
> >> >> > <http://stats.stackexchange.com/questions/4700/what-is-
> >> >> > the-difference-between-fixed-effect-random-effect-and-mixed-
> >> >> effect-mode/>
> >> >> > for links.
> >> >> >
> >> >> >   I don't think I can really write an answer yet.  I'm still trying
> >> to
> >> >> > understand at an intuitive or heuristic level what it means for
> >> >> > Cov(x_it,c_i)=0, where x_it is a set of explanatory variables over
> >> time
> >> >> > for an individual subject and c_i is the conditional mode (=BLUP in
> >> >> > linear mixed-model-land) for the deviation of the individual i from
> >> the
> >> >> > population mean ... or more particularly what it means for that
> >> >> > condition to be violated, which is the point at which fixed effects
> >> >> > would become preferred.
> >> >> >
> >> >> >   As a side note, some statisticians (Andrew Gelman is the one who
> >> >> > springs to mind) have commented on the possible overemphasis on
> bias.
> >> >> > (All else being equal unbiased estimators are preferred to biased
> >> >> > estimators but all else is not always equal). Two examples: (1)
> >> >> > penalized estimators such as lasso/ridge regression (closely
> related
> >> to
> >> >> > mixed models) give biased parameter estimates with lower mean
> squared
> >> >> > error. (2) When estimating variability, one has to choose a
> >> particular
> >> >> > scale (variance, standard error, log(standard error), etc.) on
> which
> >> one
> >> >> > would prefer to get an unbiased answer.
> >> >> >
> >> >> > On 16-10-11 12:02 PM, Laura Dee wrote:
> >> >> > > Dear all,
> >> >> > > Random effects are more efficient estimators ? however they come
> at
> >>
> >> >> the
> >> >> > > cost of the assumption that the random effect is not correlated
> >> with
> >> >> the
> >> >> > > included explanatory variables. Otherwise, using random effects
> >> leads
> >> >> to
> >> >> > > biased estimates (e.g., as laid out in Woolridge
> >> >> > > <https://faculty.fuqua.duke.edu/~moorman/Wooldridge,%20FE%20
> >> >> and%20RE.pdf
> >> >> > >'s
> >> >> > > Econometrics text). This assumption is a strong one for many
> >> >> > > observational datasets, and most analyses in economics do not use
> >> >> random
> >> >> > > effects for this reason. *Is there a reason why observational
> >> >> ecological
> >> >> > > datasets would be fundamentally different that I am missing? Why
> is
> >> >> this
> >> >> > > important assumption (to have unbiased estimates from random
> >> effects)
> >> >> > > not emphasized in ecology? *
> >> >> > >
> >> >> > > Thanks!
> >> >> > >
> >> >> > > Laura
> >> >> > >
> >> >> > > --
> >> >> > > Laura Dee
> >> >> > > Post-doctoral Associate
> >> >> > > University of Minnesota
> >> >> > > ledee at umn.edu <mailto:ledee at umn.edu>
> >> >> > > lauraedee.com <http://lauraedee.com>
> >> >> >
> >> >> > _______________________________________________
> >> >> > R-sig-mixed-models at r-project.org mailing list
> >> >> > https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
> >> >> >
> >> >>
> >> >>         [[alternative HTML version deleted]]
> >> >>
> >> >> _______________________________________________
> >> >> R-sig-mixed-models at r-project.org mailing list
> >> >> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> >
> >> >
> >> >
> >> >
> >> > Thanks,
> >> > John
> >> >
> >> >
> >> > John Poe
> >> > Doctoral Candidate
> >> > Department of Political Science
> >> > Research Methodologist
> >> > UK Center for Public Health Services & Systems Research
> >> > University of Kentucky
> >> > 111 Washington Avenue, Room 203a
> >> > Lexington, KY 40536
> >> > www.johndavidpoe.com
> >> >
> >>
> >>         [[alternative HTML version deleted]]
> >>
> >>
> >>
> >> ------------------------------
> >>
> >> Message: 2
> >> Date: Tue, 11 Oct 2016 22:47:41 -0400
> >> From: "Poe, John" <jdpo223 at g.uky.edu>
> >> To: Jake Westfall <jake.a.westfall at gmail.com>
> >> Cc: r-sig-mixed-models at r-project.org
> >> Subject: Re: [R-sig-ME] Assumptions of random effects for unbiased
> >>         estimates
> >> Message-ID:
> >>         <CAFW8ByrhGyML6DE=dMnmNm7xSeWB6zBDgvR_HaDy2Vnn53hnPQ at mail.gm
> >> ail.com>
> >> Content-Type: text/plain; charset="UTF-8"
> >>
> >>
> >> Thanks Jake!
> >>
> >> On Oct 11, 2016 9:50 PM, "Jake Westfall" <jake.a.westfall at gmail.com>
> >> wrote:
> >>
> >> > What a nice contribution from John!
> >> >
> >> > Jake
> >> >
> >> > On Tue, Oct 11, 2016 at 8:11 PM, Poe, John <jdpo223 at g.uky.edu> wrote:
> >> >
> >> > > My reading of modern work by panel data econometricians is that they
> >> seem
> >> > > very fine with the use of mixed effects models that properly
> >> > differentiate
> >> > > effects at different levels of analysis and the tools to do so have
> >> > existed
> >> > > in that literature since the early 1980s. They have been borrowing
> >> > heavily
> >> > > from the mixed effects literature in designing econometric models
> and
> >> > talk
> >> > > about them in panel data textbooks. This hasn't typically filtered
> >> down
> >> > to
> >> > > applied economists who tend to misunderstand what other fields do
> >> because
> >> > > other fields just tend to talk about them differently.
> >> > >
> >> > > The short version:
> >> > > Everyone in the mixed effects literature just uses group/grand mean
> >> > > centering and random coefficients to deal with endogeneity bias. If
> >> you
> >> > are
> >> > > an economist and someone outside of econ says mixed effects models
> you
> >> > > should think *correlated random effects models* and not *random
> >> effects
> >> > > models*.
> >> > >
> >> > > The long version:
> >> > > Economists are pretty afraid error structures that are correlated
> with
> >> > > independent variables in general and have built up pretty elaborate
> >> > > statistical models to deal with the problem. In panel data, this
> >> > manifests
> >> > > itself as wanting to avoid confounding effects at different levels
> of
> >> > > analysis so that within group varying effects are segregated from
> >> between
> >> > > group varying effects. It can also happen when you are omitting
> higher
> >> > > level random effects
> >> > > <http://methods.johndavidpoe.com/2016/09/09/independence-
> >> > across-levels-in-mixed-effects-models/>
> >> > > and they are distorting the structure of the random effects that you
> >> are
> >> > > including. This is generally a good thing as you want to be able to
> >> test
> >> > > hypotheses at specific levels of analysis without confounding.
> >> > >
> >> > > It's a big enough theoretical concern in the discipline that they
> >> usually
> >> > > just want to remove all between group effects from the data as a
> >> > *default* to
> >> > > get level one effects because it is simpler and more fool proof than
> >> > > dealing with the problem in a mixed effects setting. It's so
> pervasive
> >> > that
> >> > > they are often socialized into not designing hypotheses for any
> >> between
> >> > > group or cross-level variation and just focus on within group (time
> >> > > varying) variability when at all possible (what economists call
> >> *within
> >> > > effects*).
> >> > >
> >> > > What economists refer to as fixed effects models just difference out
> >> all
> >> > > between group variation so that it cannot contaminate within group
> >> > effects
> >> > > (bias level one coefficients). It's the equivalent to including
> group
> >> > > indicator variables in the model instead of a random effect and just
> >> > > accepting that you can't make substantive inferences about anything
> at
> >> > the
> >> > > group level (what economists call *between effects*).
> >> > >
> >> > > The typical conventional wisdom in applied econometrics is to use a
> >> > > Hausman test which is a generic test comparing coefficients between
> a
> >> > > random effects model (with no level 2 covariates) and a model with
> all
> >> > > between group variability removed from the data. If there are
> >> differences
> >> > > between the two, then they prefer to go with the latter. This is bad
> >> > > practice according to econometrics textbooks but applied people
> don't
> >> > seem
> >> > > to care (Baltagi 2013 ch 4.3). This only makes sense if you don't
> care
> >> > > about group invariant variables that only differ crosssectionally
> >> and/or
> >> > > you think of their effects as contamination. Panel data econometrics
> >> > > textbooks tend to argue for a wider range of options here but in
> >> practice
> >> > > not that many economists seem to use them.
> >> > >
> >> > > There's an alternative framework in econ for dealing with this
> problem
> >> > > that they call a Mundlak device (Mundlak 1978) or correlated random
> >> > effects
> >> > > models (Baltagi Handbook of Panel Data 2014 ch 6.3.3 or really any
> >> panel
> >> > > data textbook) which is equivalent to a hierarchical linear model
> with
> >> > > group mean centering for level-one variables. This approach is used
> in
> >> > > econometrics by some pretty standard advanced panel data models
> (e.g.
> >> > > Hausman-Taylor and Arellano Bond). The other alternative that is
> >> > advocated
> >> > > by panel data econometricians but doesn't seem to have filtered down
> >> to
> >> > > rank and file economists is to use random coefficients models and
> just
> >> > > allow the random effects to be correlated with level one variables
> >> (Hsiao
> >> > > 2014 chapter 6 and most of his other written work).
> >> > >
> >> > > It is important to understand that efficiency isn't the primary
> reason
> >> > for
> >> > > use of a mixed effects model over a fixed effects model for most
> >> > research.
> >> > > A common reason to use a mixed effects model is that you have
> >> hypotheses
> >> > > about variables operating at higher levels of analysis or
> cross-level
> >> > > interactions and those questions cannot be answered by fixed effects
> >> > panel
> >> > > models that have removed all between group variability from the
> >> analysis.
> >> > > You are sacrificing the ability to test group variant hypotheses by
> >> > using a
> >> > > basic fixed effects model over a mixed effects model. For nonlinear
> >> > models
> >> > > like a logistic regression it can also be very difficult to use an
> >> > unbiased
> >> > > fixed effects model (though there are ways in a panel setting e.g.
> >> Hahn
> >> > and
> >> > > Newy 2004) and trivial to use a mixed effects model.
> >> > >
> >> > > Panel data econometricians almost always talk about typical practice
> >> > among
> >> > > applied economists using fixed effects as flawed (see Baltagi 2013
> ch.
> >> > > 4.3). Mark Nerlov's 2000 History of Panel Data Econometrics is my
> >> > favorite
> >> > > example:
> >> > >
> >> > > The absurdity of the contention that possible correlation between
> >> some of
> >> > >> the observed explanatory variables and the individual-specific
> >> > component of
> >> > >> the disturbance is a ground for using fixed effects should be clear
> >> from
> >> > >> the following example: Consider a panel of households with data on
> >> > >> consumption and income. We are trying to estimate a consumption
> >> > function.
> >> > >> Income varies across households and over time. The variation across
> >> > >> households is related to ability of the main earner and other
> >> household
> >> > >> specific factors which vary little over time, that is to say,
> reflect
> >> > >> mainly differences in permanent income. Such permanent differences
> in
> >> > >> income are widely believed to be the source of most differences in
> >> > >> consumption both crosssectionally and over time, whereas,
> variations
> >> of
> >> > >> income over time are likely to be mostly transitory and unrelated
> to
> >> > >> consumption in most categories. Yet, fixed-effects regressions are
> >> > >> equivalent to using only this variation and discarding the
> >> information
> >> > on
> >> > >> the consumption-income relationship contained the cross-section
> >> > variation
> >> > >> among the household means.
> >> > >
> >> > >
> >> > > See the last couple of pages of this lecture
> >> > > <http://www.johndavidpoe.com/wp-content/uploads/2012/09/
> >> > Blalock-Lecture.pdf> for
> >> > > the citations in the econometrics and multilevel literature that I
> >> > > referenced.
> >> > >
> >> > >
> >> > >
> >> > > On Tue, Oct 11, 2016 at 3:32 PM, Jake Westfall <
> >> > jake.a.westfall at gmail.com>
> >> > > wrote:
> >> > >
> >> > >> Hi Laura and Ben,
> >> > >>
> >> > >> I like this paper on this topic:
> >> > >> http://psych.colorado.edu/~westfaja/FixedvsRandom.pdf
> >> > >>
> >> > >> What it comes down to essentially is that if the cluster effects
> are
> >> > >> correlated with the "time-varying" (i.e., within-cluster varying) X
> >> > >> predictor -- so that, for example, some clusters have high means
> on X
> >> > and
> >> > >> others have low means on X -- then there is the possibility that
> the
> >> > >> average within-cluster effect (which is what the fixed effect model
> >> > >> estimates) differs from the overall effect of X, not conditional on
> >> the
> >> > >> clusters. An extreme example of this is Simpson's paradox. Now
> since
> >> the
> >> > >> estimate from the random-effects model can be seen as a weighted
> >> average
> >> > >> of
> >> > >> these two effects, it will generally be pulled to some extent away
> >> from
> >> > >> the
> >> > >> fixed-effect estimate toward the unconditional estimate, which is
> the
> >> > bias
> >> > >> that econometricians fret about. However, if the cluster effects
> are
> >> not
> >> > >> correlated with X, so that each cluster has the same mean on X,
> then
> >> > this
> >> > >> situation is not possible, so the random-effect model will give the
> >> same
> >> > >> unbiased estimate as the fixed-effect model.
> >> > >>
> >> > >> A simple solution to this problem is to retain the random-effect
> >> model,
> >> > >> but
> >> > >> to split the predictor X into two components, one representing the
> >> > >> within-cluster variation of X and the other representing the
> >> > >> between-cluster variation of X, and estimate separate slopes for
> >> these
> >> > two
> >> > >> effects. One can even test whether these two slopes differ from
> each
> >> > >> other,
> >> > >> which is conceptually similar to what the Hausman test does. As
> >> > described
> >> > >> in the paper linked above, the estimate of the within-cluster
> >> component
> >> > of
> >> > >> the X effect equals the estimate one would obtain from a
> fixed-effect
> >> > >> model.
> >> > >>
> >> > >> As for the original question, I can't speak for common practice in
> >> > >> ecology,
> >> > >> but I suspect it may be like it is in my home field of psychology,
> >> where
> >> > >> we
> >> > >> do worry about this issue (to some extent), but we discuss it using
> >> > >> completely different language. That is, we discuss it in terms of
> >> > whether
> >> > >> there are different effects of the predictor at the within-cluster
> >> and
> >> > >> between-cluster levels, and how our model might account for that.
> >> > >>
> >> > >> Jake
> >> > >>
> >> > >> On Tue, Oct 11, 2016 at 1:50 PM, Ben Bolker <bbolker at gmail.com>
> >> wrote:
> >> > >>
> >> > >> >
> >> > >> >   I didn't respond to this offline, as it took me a while even to
> >> > start
> >> > >> > to come up to speed on the question.  Random effects are indeed
> >> > defined
> >> > >> > from *very* different points of view in the two communities
> >> > >> > ([bio]statistical vs. econometric); I'm sure there are points of
> >> > >> > contact, but I've been having a hard time getting my head around
> it
> >> > all.
> >> > >> >
> >> > >> > Econometric definition:
> >> > >> >
> >> > >> > The wikipedia page <https://en.wikipedia.org/
> >> > wiki/Random_effects_model>
> >> > >> > and CrossValidated question
> >> > >> > <http://stats.stackexchange.com/questions/66161/why-do-
> >> > >> > random-effect-models-require-the-effects-to-be-uncorrelated-
> >> > >> with-the-inpu>
> >> > >> > were both helpful for me.
> >> > >> >
> >> > >> >  In the (bio)statistical world fixed and random effects are
> usually
> >> > >> > justified practically in terms of shrinkage estimators, or
> >> > >> > philosophically in terms of random draws from an exchangeable set
> >> of
> >> > >> > levels: e.g. see
> >> > >> > <http://stats.stackexchange.com/questions/4700/what-is-
> >> > >> > the-difference-between-fixed-effect-random-effect-and-mixed-
> >> > >> effect-mode/>
> >> > >> > for links.
> >> > >> >
> >> > >> >   I don't think I can really write an answer yet.  I'm still
> >> trying to
> >> > >> > understand at an intuitive or heuristic level what it means for
> >> > >> > Cov(x_it,c_i)=0, where x_it is a set of explanatory variables
> over
> >> > time
> >> > >> > for an individual subject and c_i is the conditional mode (=BLUP
> in
> >> > >> > linear mixed-model-land) for the deviation of the individual i
> from
> >> > the
> >> > >> > population mean ... or more particularly what it means for that
> >> > >> > condition to be violated, which is the point at which fixed
> effects
> >> > >> > would become preferred.
> >> > >> >
> >> > >> >   As a side note, some statisticians (Andrew Gelman is the one
> who
> >> > >> > springs to mind) have commented on the possible overemphasis on
> >> bias.
> >> > >> > (All else being equal unbiased estimators are preferred to biased
> >> > >> > estimators but all else is not always equal). Two examples: (1)
> >> > >> > penalized estimators such as lasso/ridge regression (closely
> >> related
> >> > to
> >> > >> > mixed models) give biased parameter estimates with lower mean
> >> squared
> >> > >> > error. (2) When estimating variability, one has to choose a
> >> particular
> >> > >> > scale (variance, standard error, log(standard error), etc.) on
> >> which
> >> > one
> >> > >> > would prefer to get an unbiased answer.
> >> > >> >
> >> > >> > On 16-10-11 12:02 PM, Laura Dee wrote:
> >> > >> > > Dear all,
> >> > >> > > Random effects are more efficient estimators ? however they
> come
> >> at
> >>
> >> > >> the
> >> > >> > > cost of the assumption that the random effect is not correlated
> >> with
> >> > >> the
> >> > >> > > included explanatory variables. Otherwise, using random effects
> >> > leads
> >> > >> to
> >> > >> > > biased estimates (e.g., as laid out in Woolridge
> >> > >> > > <https://faculty.fuqua.duke.edu/~moorman/Wooldridge,%20FE%20
> >> > >> and%20RE.pdf
> >> > >> > >'s
> >> > >> > > Econometrics text). This assumption is a strong one for many
> >> > >> > > observational datasets, and most analyses in economics do not
> use
> >> > >> random
> >> > >> > > effects for this reason. *Is there a reason why observational
> >> > >> ecological
> >> > >> > > datasets would be fundamentally different that I am missing?
> Why
> >> is
> >> > >> this
> >> > >> > > important assumption (to have unbiased estimates from random
> >> > effects)
> >> > >> > > not emphasized in ecology? *
> >> > >> > >
> >> > >> > > Thanks!
> >> > >> > >
> >> > >> > > Laura
> >> > >> > >
> >> > >> > > --
> >> > >> > > Laura Dee
> >> > >> > > Post-doctoral Associate
> >> > >> > > University of Minnesota
> >> > >> > > ledee at umn.edu <mailto:ledee at umn.edu>
> >> > >> > > lauraedee.com <http://lauraedee.com>
> >> > >> >
> >> > >> > _______________________________________________
> >> > >> > R-sig-mixed-models at r-project.org mailing list
> >> > >> > https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
> >> > >> >
> >> > >>
> >> > >>         [[alternative HTML version deleted]]
> >> > >>
> >> > >> _______________________________________________
> >> > >> R-sig-mixed-models at r-project.org mailing list
> >> > >> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
> >> > >>
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > Thanks,
> >> > > John
> >> > >
> >> > >
> >> > > John Poe
> >> > > Doctoral Candidate
> >> > > Department of Political Science
> >> > > Research Methodologist
> >> > > UK Center for Public Health Services & Systems Research
> >> > > University of Kentucky
> >> > > 111 Washington Avenue, Room 203a
> >> > > Lexington, KY 40536
> >> > > www.johndavidpoe.com
> >>
> >
>
>
> --
> Laura Dee
> Post-doctoral Associate
> University of Minnesota
> ledee at umn.edu
> lauraedee.com
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>



-- 




Thanks,
John


John Poe
Doctoral Candidate
Department of Political Science
Research Methodologist
UK Center for Public Health Services & Systems Research
University of Kentucky
111 Washington Avenue, Room 203a
Lexington, KY 40536
www.johndavidpoe.com

	[[alternative HTML version deleted]]