[R-sig-ME] Assumptions of random effects for unbiased estimates
Laura Dee
ledee at umn.edu
Wed Oct 12 18:03:14 CEST 2016
Thank you, John, for taking the time to write such a detailed response.
Much appreciated.
Laura
On Wednesday, October 12, 2016, Poe, John <jdpo223 at g.uky.edu> wrote:
> Laura,
>
> I think we might be talking past each other somewhat on the FE vs RE
> discussion. An RE model that has only level one (time or group varying)
> covariates and a latent variable for the expected value of group membership
> on Y is problematic for the reasons that you are talking about. But mixed
> effects models aren't typically the same specification as the FE model or
> the RE model. They either use group/grand mean centering of variables to
> instrument the problem or random coefficients to stop making the assumption
> of no correlation altogether.
>
> The primary benefit of a mixed effects approach is that you have a better
> handle on all of the factors that are influencing Y and you can make decent
> predictions about cases given a particular context. By using the fixed
> effects model you are literally making the choice not to care about things
> that you know influence Y.
>
> My usual advice if you don't care about between group effects or
> cross-level effects AND you don't care about making predictions about
> individuals or groups then fixed effects models that difference out between
> group variability are fine for linear models. Given Malcolm's working paper
> I might reevaluate that recommendation going forward.
>
> On point 1 that you raise about credibility of the argument that group
> mean centered models actually do fix the problems associated with random
> effects correlation:
>
> - There's not actually a credibility problem here. There is a
> specification test in econometrics that is analogous to a Hausman test but
> compares fixed effects estimates to a hierarchical linear model. It's
> called a Mundlak specification test
> <http://blog.stata.com/2015/10/29/fixed-effects-or-random-effects-the-mundlak-approach/>.
> So you can demonstrate that it's not a problem.
> - The typical response when this test shows that there is still a
> violation of the no correlation between a random effect and a level 1
> variable assumption is to stop making that assumption and use a random
> coefficients model. The Bell and Jones paper does a good job of working
> through the logic here.
>
> On your point about the comment :"But group-mean centering can also be
> done with random effects models, with the same benefit you get with fixed
> effects models (isolation of the within effects), while still allowing for
> estimation of the between relationships"
>
> - Group mean centering of variables tends to instrument the
> endogeneity between the RE and the level one variables so that the
> correlation isn't an issue anymore. In the basic RE model without level 2
> covariates and without centering you have a problem. But that model doesn't
> tend to get used much without criticism in political science, psychology,
> or economics (I'm not an ecologist so I can't say for practice in your
> field). It seems to be something of a straw-man in my experience.
>
> On point 2
>
> Based on John Poe's response and example with the income, I think that is
>> an argument that the model identification is wrong if you don't allow mean
>> income versus deviations from mean income to have different effects on consumption,
>> rather than an argument that RE solve the problem of unobserved
>> heterogeneity more credibly than FE. This is a point about model specification
>> rather than dealing with unobservable heterogeneity.
>
>
> - All unobserved heterogeneity problems are about model specification.
> It's an omitted variable bias problem. Random effects models are literally
> just a version of the model with an additional latent variable for the
> expected value of group membership. That latent variable can be generated
> directly from coefficients off of dummy variables so they are esentially
> reparamaterizations of one another.
> - In a random effects setting you are including a new latent variable
> and in a random coefficients model you are adding an interaction between
> the random effect and the variable
> - In most cases you can deal with the endogeneity issue by including
> group means and then taking their deviations. When you can't do that, you
> can use dummy variables or the group-level expectations for a random effect
> and then interact that with the endogenous covariate.
>
> On point 3 for FE in nonlinear models
>
> - The basic issue is that in high dimensional problems the root
> finding algorithms for nonlinear MLE tend to start to give biased answers
> as the mode diverges away from the mean. This divergence between the mode
> and the mean works as a function of the number of groups and the size of
> the groups so that if T grows to infinity then there's no bias problem. I
> haven't seen solid research on how unbalanced group size influences this
> but my intuition says that it's likely to increase the bias problem. You
> get bias in as few as ten dimensions and there's really no way to fix it
> with convex optimization. So if you want to include dummy variables in
> something like a logit you either need to use some version of expectation
> maximization or integrate over the data then take the jacobian and hessian
> directly. At that point you know where the mean is and you don't have to
> rely on an optimizer so you will get unbiased results. That's why it's not
> an issue in Poisson or a linear model. You can just calculate the
> derivatives directly.
>
> As far as citations on incidental parameters bias:
>
> - Lancaster, Tony. 2000. "The incidental parameter problem since
> 1948." Journal of Econometrics 95 (2):391-413.
> - Katz, Ethan. 2001. "Bias in conditional and unconditional fixed
> effects logit estimation." Political Analysis 9 (4):379-84.
> - Hahn, J., & Newey, W. (2004). Jackknife and analytical bias
> reduction for nonlinear panel models. Econometrica, 72(4), 1295-1319.
> - Greene, William. 2004. "The behaviour of the maximum likelihood
> estimator of limited dependent variable models in the presence of fixed
> effects." The Econometrics Journal 7 (1):98-119.
> - Bill Greene's chapter in Baltagi, B. H. (2014). *The Oxford Handbook
> of Panel Data*. Oxford University Press, USA.
> - Note that one of his recommendations is just to use group mean
> centering and the mixed effects framework
> - Beck, Nathaniel. 2015. Estimating grouped data models with a binary
> dependent variable and fixed effects: What are the issues? Paper read at
> annual meeting of the Society for Political Methodology, July
>
> On the bias variance trade-off question I think it's mostly just that
> economists aren't interested in prediction as much as estimating average
> causal effects. They don't seem to care if their results are applicable to
> any particular case so long as they describe the average effect of X on Y
> well.
>
>
> On Wed, Oct 12, 2016 at 9:37 AM, Laura Dee <ledee at umn.edu
> <javascript:_e(%7B%7D,'cvml','ledee at umn.edu');>> wrote:
>
>> Dear all,
>> Thanks all - very interesting and helpful responses. I think I should have
>> been clearer with my question: in my case, the unobserved heterogeneity
>> between groups as not being of interest to study (but something to be
>> controlled for to isolate the effects of other x_ij's). Also I'm using a
>> linear model setting. The paper Jake and Malcolm sent was very help to lay
>> out these issues and suggest the within and between RE model when you want
>> to be studying the between group variation. And, I'll go through John
>> Poe's
>> slides in detail.
>>
>> There are four points that have emerged from this discussion that I think
>> are worth teasing apart:
>>
>> *1)* Interest in studying the mean effects and how they differ between
>> groups, which FE do not allow because they remove the mean effect.
>> However,
>> even with RE and the ability to study those between differences, you still
>> have the challenge of credibly identifying the mean effects of income on
>> y_ij -- and whether you have ruled out/controlled for other factors that
>> vary cross-sectionally. RE do not solve this issue but are preferred
>> because the mean effect between groups is what is of interest. Therfore,
>> one is willing to accept some bias in the estimates if there are other
>> unobserved variables that vary cross-sectionally and influence the
>> outcome.
>>
>> Further, Malcolm, I agree that both FE and RE both try to account for a
>> group mean but not this statement because of the assumption of RE: "But
>> group-mean centering can also be done with random effects models, with the
>> same benefit you get with fixed effects models (isolation of the within
>> effects), while still allowing for estimation of the between
>> relationships"
>> However, estimates are unbiased if FE are correlated with the error term,
>> which is not the case for RE. Though, agreed, if it is the between group
>> variation that is of interest, then it does not make sense to use a FE
>> model and there is too much focus on bias over other issues (i.e.,
>> estimating the effect of interest).
>>
>> *Question: *in the Bell & Jones paper that Jake sent, they present
>> the Plümper and Troeger’s (2007) fixed effects vector decomposition. Is
>> that used often? I don't think it has made it's way to ecology.
>>
>> *2)* Based on John Poe's response and example with the income, I think
>> that
>> is an argument that the model identification is wrong if you don't allow
>> mean income versus deviations from mean income to have different effects
>> on
>> consumption, rather than an argument that RE solve the problem of
>> unobserved heterogeneity more credibly than FE. This is a point about
>> model
>> specification rather than dealing with unobservable heterogeneity.
>>
>> *3) *Agreed that FE are biased with some forms of non-linear models. Could
>> anyone send me some more recent papers on this topic?
>>
>> *4) *Ben raised the issue of a bias-variance trade-off, which is a good
>> point and economists seem to focus more (and maybe too much) on bias.
>> However, with enough observations, it's less of a trade-off.
>>
>> Many thanks to everyone,
>> Laura
>>
>>
>>
>> On Wed, Oct 12, 2016 at 3:12 AM, Malcolm Fairbrother <
>> M.Fairbrother at bristol.ac.uk
>> <javascript:_e(%7B%7D,'cvml','M.Fairbrother at bristol.ac.uk');>> wrote:
>>
>> > As others have said, there are rather peculiar inconsistencies between
>> > what the methodological literature knows and what empirical economists
>> > actually do.
>> >
>> > I think the paper Jake cited (by my colleagues at Bristol/Sheffield) is
>> > indeed one of the most useful on all this. The published version of the
>> > paper is at: http://dx.doi.org/10.1017/psrm.2014.7
>> >
>> > The following working paper (by them and me) takes up similar themes:
>> > https://www.researchgate.net/publication/299604336_Fixed_
>> > and_Random_effects_making_an_informed_choice
>> >
>> > One of the additional limitations we note here with fixed effects models
>> > (using a simulation study) is that they can be anti-conservative, in the
>> > sense that the SEs they return are too small if the data are generated
>> from
>> > a random slopes model.
>> >
>> > In brief, Laura, fixed effects models only estimate within-group
>> > relationships, whereas random effects (AKA multilevel, mixed) models can
>> > estimate within- and between-group relationships. The estimation of
>> fixed
>> > effects models implicitly entails group mean centering (though the
>> models
>> > are typically written out as though unit dummies are estimated). But
>> > group-mean centering can also be done with random effects models, with
>> the
>> > same benefit you get with fixed effects models (isolation of the within
>> > effects), while still allowing for estimation of the between
>> relationships.
>> > You might have less confidence that the between component of some x_ij
>> is
>> > uncorrelated with the unit error term, but it is still possible for the
>> > within (group-mean-centered) component to be correlated with the
>> > observation-level error term. So I would agree that bias is worth
>> thinking
>> > about, but using fixed effects is no more helpful than random effects
>> as a
>> > solution to the problem.
>> >
>> > Hope that's useful,
>> > Malcolm
>> >
>> >
>> > Dr Malcolm Fairbrother
>> > Reader in Global Policy and Politics
>> > School of Geographical Sciences • Cabot Institute • Centre for
>> > Multilevel Modelling
>> > University of Bristol
>> >
>> >
>> >
>> >
>> > Date: Tue, 11 Oct 2016 20:49:45 -0500
>> >> From: Jake Westfall <jake.a.westfall at gmail.com
>> <javascript:_e(%7B%7D,'cvml','jake.a.westfall at gmail.com');>>
>> >> To: r-sig-mixed-models at r-project.org
>> <javascript:_e(%7B%7D,'cvml','r-sig-mixed-models at r-project.org');>
>> >> Subject: Re: [R-sig-ME] Assumptions of random effects for unbiased
>> >> estimates
>> >> Message-ID:
>> >> <CAE9_Wg6+ZFXh-9on=nmuUwLKO6ScXjMRfbgf4y+XpGNhVAwJqA at mail.gm
>> <javascript:_e(%7B%7D,'cvml','nmuUwLKO6ScXjMRfbgf4y%2BXpGNhVAwJqA at mail.gm');>
>> >> ail.com>
>> >> Content-Type: text/plain; charset="UTF-8"
>> >>
>> >>
>> >> What a nice contribution from John!
>> >>
>> >> Jake
>> >>
>> >> On Tue, Oct 11, 2016 at 8:11 PM, Poe, John <jdpo223 at g.uky.edu
>> <javascript:_e(%7B%7D,'cvml','jdpo223 at g.uky.edu');>> wrote:
>> >>
>> >> > My reading of modern work by panel data econometricians is that they
>> >> seem
>> >> > very fine with the use of mixed effects models that properly
>> >> differentiate
>> >> > effects at different levels of analysis and the tools to do so have
>> >> existed
>> >> > in that literature since the early 1980s. They have been borrowing
>> >> heavily
>> >> > from the mixed effects literature in designing econometric models and
>> >> talk
>> >> > about them in panel data textbooks. This hasn't typically filtered
>> down
>> >> to
>> >> > applied economists who tend to misunderstand what other fields do
>> >> because
>> >> > other fields just tend to talk about them differently.
>> >> >
>> >> > The short version:
>> >> > Everyone in the mixed effects literature just uses group/grand mean
>> >> > centering and random coefficients to deal with endogeneity bias. If
>> you
>> >> are
>> >> > an economist and someone outside of econ says mixed effects models
>> you
>> >> > should think *correlated random effects models* and not *random
>> effects
>> >> > models*.
>> >> >
>> >> > The long version:
>> >> > Economists are pretty afraid error structures that are correlated
>> with
>> >> > independent variables in general and have built up pretty elaborate
>> >> > statistical models to deal with the problem. In panel data, this
>> >> manifests
>> >> > itself as wanting to avoid confounding effects at different levels of
>> >> > analysis so that within group varying effects are segregated from
>> >> between
>> >> > group varying effects. It can also happen when you are omitting
>> higher
>> >> > level random effects
>> >> > <http://methods.johndavidpoe.com/2016/09/09/independence-acr
>> >> oss-levels-in-mixed-effects-models/>
>> >> > and they are distorting the structure of the random effects that you
>> are
>> >> > including. This is generally a good thing as you want to be able to
>> test
>> >> > hypotheses at specific levels of analysis without confounding.
>> >> >
>> >> > It's a big enough theoretical concern in the discipline that they
>> >> usually
>> >> > just want to remove all between group effects from the data as a
>> >> *default* to
>> >> > get level one effects because it is simpler and more fool proof than
>> >> > dealing with the problem in a mixed effects setting. It's so
>> pervasive
>> >> that
>> >> > they are often socialized into not designing hypotheses for any
>> between
>> >> > group or cross-level variation and just focus on within group (time
>> >> > varying) variability when at all possible (what economists call
>> *within
>> >> > effects*).
>> >> >
>> >> > What economists refer to as fixed effects models just difference out
>> all
>> >> > between group variation so that it cannot contaminate within group
>> >> effects
>> >> > (bias level one coefficients). It's the equivalent to including group
>> >> > indicator variables in the model instead of a random effect and just
>> >> > accepting that you can't make substantive inferences about anything
>> at
>> >> the
>> >> > group level (what economists call *between effects*).
>> >> >
>> >> > The typical conventional wisdom in applied econometrics is to use a
>> >> > Hausman test which is a generic test comparing coefficients between a
>> >> > random effects model (with no level 2 covariates) and a model with
>> all
>> >> > between group variability removed from the data. If there are
>> >> differences
>> >> > between the two, then they prefer to go with the latter. This is bad
>> >> > practice according to econometrics textbooks but applied people don't
>> >> seem
>> >> > to care (Baltagi 2013 ch 4.3). This only makes sense if you don't
>> care
>> >> > about group invariant variables that only differ crosssectionally
>> and/or
>> >> > you think of their effects as contamination. Panel data econometrics
>> >> > textbooks tend to argue for a wider range of options here but in
>> >> practice
>> >> > not that many economists seem to use them.
>> >> >
>> >> > There's an alternative framework in econ for dealing with this
>> problem
>> >> > that they call a Mundlak device (Mundlak 1978) or correlated random
>> >> effects
>> >> > models (Baltagi Handbook of Panel Data 2014 ch 6.3.3 or really any
>> panel
>> >> > data textbook) which is equivalent to a hierarchical linear model
>> with
>> >> > group mean centering for level-one variables. This approach is used
>> in
>> >> > econometrics by some pretty standard advanced panel data models (e.g.
>> >> > Hausman-Taylor and Arellano Bond). The other alternative that is
>> >> advocated
>> >> > by panel data econometricians but doesn't seem to have filtered down
>> to
>> >> > rank and file economists is to use random coefficients models and
>> just
>> >> > allow the random effects to be correlated with level one variables
>> >> (Hsiao
>> >> > 2014 chapter 6 and most of his other written work).
>> >> >
>> >> > It is important to understand that efficiency isn't the primary
>> reason
>> >> for
>> >> > use of a mixed effects model over a fixed effects model for most
>> >> research.
>> >> > A common reason to use a mixed effects model is that you have
>> hypotheses
>> >> > about variables operating at higher levels of analysis or cross-level
>> >> > interactions and those questions cannot be answered by fixed effects
>> >> panel
>> >> > models that have removed all between group variability from the
>> >> analysis.
>> >> > You are sacrificing the ability to test group variant hypotheses by
>> >> using a
>> >> > basic fixed effects model over a mixed effects model. For nonlinear
>> >> models
>> >> > like a logistic regression it can also be very difficult to use an
>> >> unbiased
>> >> > fixed effects model (though there are ways in a panel setting e.g.
>> Hahn
>> >> and
>> >> > Newy 2004) and trivial to use a mixed effects model.
>> >> >
>> >> > Panel data econometricians almost always talk about typical practice
>> >> among
>> >> > applied economists using fixed effects as flawed (see Baltagi 2013
>> ch.
>> >> > 4.3). Mark Nerlov's 2000 History of Panel Data Econometrics is my
>> >> favorite
>> >> > example:
>> >> >
>> >> > The absurdity of the contention that possible correlation between
>> some
>> >> of
>> >> >> the observed explanatory variables and the individual-specific
>> >> component of
>> >> >> the disturbance is a ground for using fixed effects should be clear
>> >> from
>> >> >> the following example: Consider a panel of households with data on
>> >> >> consumption and income. We are trying to estimate a consumption
>> >> function.
>> >> >> Income varies across households and over time. The variation across
>> >> >> households is related to ability of the main earner and other
>> household
>> >> >> specific factors which vary little over time, that is to say,
>> reflect
>> >> >> mainly differences in permanent income. Such permanent differences
>> in
>> >> >> income are widely believed to be the source of most differences in
>> >> >> consumption both crosssectionally and over time, whereas,
>> variations of
>> >> >> income over time are likely to be mostly transitory and unrelated to
>> >> >> consumption in most categories. Yet, fixed-effects regressions are
>> >> >> equivalent to using only this variation and discarding the
>> information
>> >> on
>> >> >> the consumption-income relationship contained the cross-section
>> >> variation
>> >> >> among the household means.
>> >> >
>> >> >
>> >> > See the last couple of pages of this lecture
>> >> > <http://www.johndavidpoe.com/wp-content/uploads/2012/09/Blal
>> >> ock-Lecture.pdf> for
>> >>
>> >> > the citations in the econometrics and multilevel literature that I
>> >> > referenced.
>> >> >
>> >> >
>> >> >
>> >> > On Tue, Oct 11, 2016 at 3:32 PM, Jake Westfall <
>> >> jake.a.westfall at gmail.com
>> <javascript:_e(%7B%7D,'cvml','jake.a.westfall at gmail.com');>>
>> >> > wrote:
>> >> >
>> >> >> Hi Laura and Ben,
>> >> >>
>> >> >> I like this paper on this topic:
>> >> >> http://psych.colorado.edu/~westfaja/FixedvsRandom.pdf
>> >> >>
>> >> >> What it comes down to essentially is that if the cluster effects are
>> >> >> correlated with the "time-varying" (i.e., within-cluster varying) X
>> >> >> predictor -- so that, for example, some clusters have high means on
>> X
>> >> and
>> >> >> others have low means on X -- then there is the possibility that the
>> >> >> average within-cluster effect (which is what the fixed effect model
>> >> >> estimates) differs from the overall effect of X, not conditional on
>> the
>> >> >> clusters. An extreme example of this is Simpson's paradox. Now since
>> >> the
>> >> >> estimate from the random-effects model can be seen as a weighted
>> >> average
>> >> >> of
>> >> >> these two effects, it will generally be pulled to some extent away
>> from
>> >> >> the
>> >> >> fixed-effect estimate toward the unconditional estimate, which is
>> the
>> >> bias
>> >> >> that econometricians fret about. However, if the cluster effects are
>> >> not
>> >> >> correlated with X, so that each cluster has the same mean on X, then
>> >> this
>> >> >> situation is not possible, so the random-effect model will give the
>> >> same
>> >> >> unbiased estimate as the fixed-effect model.
>> >> >>
>> >> >> A simple solution to this problem is to retain the random-effect
>> model,
>> >> >> but
>> >> >> to split the predictor X into two components, one representing the
>> >> >> within-cluster variation of X and the other representing the
>> >> >> between-cluster variation of X, and estimate separate slopes for
>> these
>> >> two
>> >> >> effects. One can even test whether these two slopes differ from each
>> >> >> other,
>> >> >> which is conceptually similar to what the Hausman test does. As
>> >> described
>> >> >> in the paper linked above, the estimate of the within-cluster
>> >> component of
>> >> >> the X effect equals the estimate one would obtain from a
>> fixed-effect
>> >> >> model.
>> >> >>
>> >> >> As for the original question, I can't speak for common practice in
>> >> >> ecology,
>> >> >> but I suspect it may be like it is in my home field of psychology,
>> >> where
>> >> >> we
>> >> >> do worry about this issue (to some extent), but we discuss it using
>> >> >> completely different language. That is, we discuss it in terms of
>> >> whether
>> >> >> there are different effects of the predictor at the within-cluster
>> and
>> >> >> between-cluster levels, and how our model might account for that.
>> >> >>
>> >> >> Jake
>> >> >>
>> >> >> On Tue, Oct 11, 2016 at 1:50 PM, Ben Bolker <bbolker at gmail.com
>> <javascript:_e(%7B%7D,'cvml','bbolker at gmail.com');>> wrote:
>> >> >>
>> >> >> >
>> >> >> > I didn't respond to this offline, as it took me a while even to
>> >> start
>> >> >> > to come up to speed on the question. Random effects are indeed
>> >> defined
>> >> >> > from *very* different points of view in the two communities
>> >> >> > ([bio]statistical vs. econometric); I'm sure there are points of
>> >> >> > contact, but I've been having a hard time getting my head around
>> it
>> >> all.
>> >> >> >
>> >> >> > Econometric definition:
>> >> >> >
>> >> >> > The wikipedia page <https://en.wikipedia.org/wiki
>> >> /Random_effects_model>
>> >> >> > and CrossValidated question
>> >> >> > <http://stats.stackexchange.com/questions/66161/why-do-
>> >> >> > random-effect-models-require-the-effects-to-be-uncorrelated-
>> >> >> with-the-inpu>
>> >> >> > were both helpful for me.
>> >> >> >
>> >> >> > In the (bio)statistical world fixed and random effects are
>> usually
>> >> >> > justified practically in terms of shrinkage estimators, or
>> >> >> > philosophically in terms of random draws from an exchangeable set
>> of
>> >> >> > levels: e.g. see
>> >> >> > <http://stats.stackexchange.com/questions/4700/what-is-
>> >> >> > the-difference-between-fixed-effect-random-effect-and-mixed-
>> >> >> effect-mode/>
>> >> >> > for links.
>> >> >> >
>> >> >> > I don't think I can really write an answer yet. I'm still
>> trying
>> >> to
>> >> >> > understand at an intuitive or heuristic level what it means for
>> >> >> > Cov(x_it,c_i)=0, where x_it is a set of explanatory variables over
>> >> time
>> >> >> > for an individual subject and c_i is the conditional mode (=BLUP
>> in
>> >> >> > linear mixed-model-land) for the deviation of the individual i
>> from
>> >> the
>> >> >> > population mean ... or more particularly what it means for that
>> >> >> > condition to be violated, which is the point at which fixed
>> effects
>> >> >> > would become preferred.
>> >> >> >
>> >> >> > As a side note, some statisticians (Andrew Gelman is the one who
>> >> >> > springs to mind) have commented on the possible overemphasis on
>> bias.
>> >> >> > (All else being equal unbiased estimators are preferred to biased
>> >> >> > estimators but all else is not always equal). Two examples: (1)
>> >> >> > penalized estimators such as lasso/ridge regression (closely
>> related
>> >> to
>> >> >> > mixed models) give biased parameter estimates with lower mean
>> squared
>> >> >> > error. (2) When estimating variability, one has to choose a
>> >> particular
>> >> >> > scale (variance, standard error, log(standard error), etc.) on
>> which
>> >> one
>> >> >> > would prefer to get an unbiased answer.
>> >> >> >
>> >> >> > On 16-10-11 12:02 PM, Laura Dee wrote:
>> >> >> > > Dear all,
>> >> >> > > Random effects are more efficient estimators ? however they
>> come at
>> >>
>> >> >> the
>> >> >> > > cost of the assumption that the random effect is not correlated
>> >> with
>> >> >> the
>> >> >> > > included explanatory variables. Otherwise, using random effects
>> >> leads
>> >> >> to
>> >> >> > > biased estimates (e.g., as laid out in Woolridge
>> >> >> > > <https://faculty.fuqua.duke.edu/~moorman/Wooldridge,%20FE%20
>> >> >> and%20RE.pdf
>> >> >> > >'s
>> >> >> > > Econometrics text). This assumption is a strong one for many
>> >> >> > > observational datasets, and most analyses in economics do not
>> use
>> >> >> random
>> >> >> > > effects for this reason. *Is there a reason why observational
>> >> >> ecological
>> >> >> > > datasets would be fundamentally different that I am missing?
>> Why is
>> >> >> this
>> >> >> > > important assumption (to have unbiased estimates from random
>> >> effects)
>> >> >> > > not emphasized in ecology? *
>> >> >> > >
>> >> >> > > Thanks!
>> >> >> > >
>> >> >> > > Laura
>> >> >> > >
>> >> >> > > --
>> >> >> > > Laura Dee
>> >> >> > > Post-doctoral Associate
>> >> >> > > University of Minnesota
>> >> >> > > ledee at umn.edu <javascript:_e(%7B%7D,'cvml','ledee at umn.edu');>
>> <mailto:ledee at umn.edu <javascript:_e(%7B%7D,'cvml','ledee at umn.edu');>>
>> >> >> > > lauraedee.com <http://lauraedee.com>
>> >> >> >
>> >> >> > _______________________________________________
>> >> >> > R-sig-mixed-models at r-project.org
>> <javascript:_e(%7B%7D,'cvml','R-sig-mixed-models at r-project.org');>
>> mailing list
>> >> >> > https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>> >> >> >
>> >> >>
>> >> >> [[alternative HTML version deleted]]
>> >> >>
>> >> >> _______________________________________________
>> >> >> R-sig-mixed-models at r-project.org
>> <javascript:_e(%7B%7D,'cvml','R-sig-mixed-models at r-project.org');>
>> mailing list
>> >> >> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>> >> >>
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > Thanks,
>> >> > John
>> >> >
>> >> >
>> >> > John Poe
>> >> > Doctoral Candidate
>> >> > Department of Political Science
>> >> > Research Methodologist
>> >> > UK Center for Public Health Services & Systems Research
>> >> > University of Kentucky
>> >> > 111 Washington Avenue, Room 203a
>> >> > Lexington, KY 40536
>> >> > www.johndavidpoe.com
>> >> >
>> >>
>> >> [[alternative HTML version deleted]]
>> >>
>> >>
>> >>
>> >> ------------------------------
>> >>
>> >> Message: 2
>> >> Date: Tue, 11 Oct 2016 22:47:41 -0400
>> >> From: "Poe, John" <jdpo223 at g.uky.edu
>> <javascript:_e(%7B%7D,'cvml','jdpo223 at g.uky.edu');>>
>> >> To: Jake Westfall <jake.a.westfall at gmail.com
>> <javascript:_e(%7B%7D,'cvml','jake.a.westfall at gmail.com');>>
>> >> Cc: r-sig-mixed-models at r-project.org
>> <javascript:_e(%7B%7D,'cvml','r-sig-mixed-models at r-project.org');>
>> >> Subject: Re: [R-sig-ME] Assumptions of random effects for unbiased
>> >> estimates
>> >> Message-ID:
>> >> <CAFW8ByrhGyML6DE=dMnmNm7xSeWB6zBDgvR_HaDy2Vnn53hnPQ at mail.gm
>> <javascript:_e(%7B%7D,'cvml','dMnmNm7xSeWB6zBDgvR_HaDy2Vnn53hnPQ at mail.gm');>
>> >> ail.com>
>> >> Content-Type: text/plain; charset="UTF-8"
>> >>
>> >>
>> >> Thanks Jake!
>> >>
>> >> On Oct 11, 2016 9:50 PM, "Jake Westfall" <jake.a.westfall at gmail.com
>> <javascript:_e(%7B%7D,'cvml','jake.a.westfall at gmail.com');>>
>> >> wrote:
>> >>
>> >> > What a nice contribution from John!
>> >> >
>> >> > Jake
>> >> >
>> >> > On Tue, Oct 11, 2016 at 8:11 PM, Poe, John <jdpo223 at g.uky.edu
>> <javascript:_e(%7B%7D,'cvml','jdpo223 at g.uky.edu');>> wrote:
>> >> >
>> >> > > My reading of modern work by panel data econometricians is that
>> they
>> >> seem
>> >> > > very fine with the use of mixed effects models that properly
>> >> > differentiate
>> >> > > effects at different levels of analysis and the tools to do so have
>> >> > existed
>> >> > > in that literature since the early 1980s. They have been borrowing
>> >> > heavily
>> >> > > from the mixed effects literature in designing econometric models
>> and
>> >> > talk
>> >> > > about them in panel data textbooks. This hasn't typically filtered
>> >> down
>> >> > to
>> >> > > applied economists who tend to misunderstand what other fields do
>> >> because
>> >> > > other fields just tend to talk about them differently.
>> >> > >
>> >> > > The short version:
>> >> > > Everyone in the mixed effects literature just uses group/grand mean
>> >> > > centering and random coefficients to deal with endogeneity bias. If
>> >> you
>> >> > are
>> >> > > an economist and someone outside of econ says mixed effects models
>> you
>> >> > > should think *correlated random effects models* and not *random
>> >> effects
>> >> > > models*.
>> >> > >
>> >> > > The long version:
>> >> > > Economists are pretty afraid error structures that are correlated
>> with
>> >> > > independent variables in general and have built up pretty elaborate
>> >> > > statistical models to deal with the problem. In panel data, this
>> >> > manifests
>> >> > > itself as wanting to avoid confounding effects at different levels
>> of
>> >> > > analysis so that within group varying effects are segregated from
>> >> between
>> >> > > group varying effects. It can also happen when you are omitting
>> higher
>> >> > > level random effects
>> >> > > <http://methods.johndavidpoe.com/2016/09/09/independence-
>> >> > across-levels-in-mixed-effects-models/>
>> >> > > and they are distorting the structure of the random effects that
>> you
>> >> are
>> >> > > including. This is generally a good thing as you want to be able to
>> >> test
>> >> > > hypotheses at specific levels of analysis without confounding.
>> >> > >
>> >> > > It's a big enough theoretical concern in the discipline that they
>> >> usually
>> >> > > just want to remove all between group effects from the data as a
>> >> > *default* to
>> >> > > get level one effects because it is simpler and more fool proof
>> than
>> >> > > dealing with the problem in a mixed effects setting. It's so
>> pervasive
>> >> > that
>> >> > > they are often socialized into not designing hypotheses for any
>> >> between
>> >> > > group or cross-level variation and just focus on within group (time
>> >> > > varying) variability when at all possible (what economists call
>> >> *within
>> >> > > effects*).
>> >> > >
>> >> > > What economists refer to as fixed effects models just difference
>> out
>> >> all
>> >> > > between group variation so that it cannot contaminate within group
>> >> > effects
>> >> > > (bias level one coefficients). It's the equivalent to including
>> group
>> >> > > indicator variables in the model instead of a random effect and
>> just
>> >> > > accepting that you can't make substantive inferences about
>> anything at
>> >> > the
>> >> > > group level (what economists call *between effects*).
>> >> > >
>> >> > > The typical conventional wisdom in applied econometrics is to use a
>> >> > > Hausman test which is a generic test comparing coefficients
>> between a
>> >> > > random effects model (with no level 2 covariates) and a model with
>> all
>> >> > > between group variability removed from the data. If there are
>> >> differences
>> >> > > between the two, then they prefer to go with the latter. This is
>> bad
>> >> > > practice according to econometrics textbooks but applied people
>> don't
>> >> > seem
>> >> > > to care (Baltagi 2013 ch 4.3). This only makes sense if you don't
>> care
>> >> > > about group invariant variables that only differ crosssectionally
>> >> and/or
>> >> > > you think of their effects as contamination. Panel data
>> econometrics
>> >> > > textbooks tend to argue for a wider range of options here but in
>> >> practice
>> >> > > not that many economists seem to use them.
>> >> > >
>> >> > > There's an alternative framework in econ for dealing with this
>> problem
>> >> > > that they call a Mundlak device (Mundlak 1978) or correlated random
>> >> > effects
>> >> > > models (Baltagi Handbook of Panel Data 2014 ch 6.3.3 or really any
>> >> panel
>> >> > > data textbook) which is equivalent to a hierarchical linear model
>> with
>> >> > > group mean centering for level-one variables. This approach is
>> used in
>> >> > > econometrics by some pretty standard advanced panel data models
>> (e.g.
>> >> > > Hausman-Taylor and Arellano Bond). The other alternative that is
>> >> > advocated
>> >> > > by panel data econometricians but doesn't seem to have filtered
>> down
>> >> to
>> >> > > rank and file economists is to use random coefficients models and
>> just
>> >> > > allow the random effects to be correlated with level one variables
>> >> (Hsiao
>> >> > > 2014 chapter 6 and most of his other written work).
>> >> > >
>> >> > > It is important to understand that efficiency isn't the primary
>> reason
>> >> > for
>> >> > > use of a mixed effects model over a fixed effects model for most
>> >> > research.
>> >> > > A common reason to use a mixed effects model is that you have
>> >> hypotheses
>> >> > > about variables operating at higher levels of analysis or
>> cross-level
>> >> > > interactions and those questions cannot be answered by fixed
>> effects
>> >> > panel
>> >> > > models that have removed all between group variability from the
>> >> analysis.
>> >> > > You are sacrificing the ability to test group variant hypotheses by
>> >> > using a
>> >> > > basic fixed effects model over a mixed effects model. For nonlinear
>> >> > models
>> >> > > like a logistic regression it can also be very difficult to use an
>> >> > unbiased
>> >> > > fixed effects model (though there are ways in a panel setting e.g.
>> >> Hahn
>> >> > and
>> >> > > Newy 2004) and trivial to use a mixed effects model.
>> >> > >
>> >> > > Panel data econometricians almost always talk about typical
>> practice
>> >> > among
>> >> > > applied economists using fixed effects as flawed (see Baltagi 2013
>> ch.
>> >> > > 4.3). Mark Nerlov's 2000 History of Panel Data Econometrics is my
>> >> > favorite
>> >> > > example:
>> >> > >
>> >> > > The absurdity of the contention that possible correlation between
>> >> some of
>> >> > >> the observed explanatory variables and the individual-specific
>> >> > component of
>> >> > >> the disturbance is a ground for using fixed effects should be
>> clear
>> >> from
>> >> > >> the following example: Consider a panel of households with data on
>> >> > >> consumption and income. We are trying to estimate a consumption
>> >> > function.
>> >> > >> Income varies across households and over time. The variation
>> across
>> >> > >> households is related to ability of the main earner and other
>> >> household
>> >> > >> specific factors which vary little over time, that is to say,
>> reflect
>> >> > >> mainly differences in permanent income. Such permanent
>> differences in
>> >> > >> income are widely believed to be the source of most differences in
>> >> > >> consumption both crosssectionally and over time, whereas,
>> variations
>> >> of
>> >> > >> income over time are likely to be mostly transitory and unrelated
>> to
>> >> > >> consumption in most categories. Yet, fixed-effects regressions are
>> >> > >> equivalent to using only this variation and discarding the
>> >> information
>> >> > on
>> >> > >> the consumption-income relationship contained the cross-section
>> >> > variation
>> >> > >> among the household means.
>> >> > >
>> >> > >
>> >> > > See the last couple of pages of this lecture
>> >> > > <http://www.johndavidpoe.com/wp-content/uploads/2012/09/
>> >> > Blalock-Lecture.pdf> for
>> >> > > the citations in the econometrics and multilevel literature that I
>> >> > > referenced.
>> >> > >
>> >> > >
>> >> > >
>> >> > > On Tue, Oct 11, 2016 at 3:32 PM, Jake Westfall <
>> >> > jake.a.westfall at gmail.com
>> <javascript:_e(%7B%7D,'cvml','jake.a.westfall at gmail.com');>>
>> >> > > wrote:
>> >> > >
>> >> > >> Hi Laura and Ben,
>> >> > >>
>> >> > >> I like this paper on this topic:
>> >> > >> http://psych.colorado.edu/~westfaja/FixedvsRandom.pdf
>> >> > >>
>> >> > >> What it comes down to essentially is that if the cluster effects
>> are
>> >> > >> correlated with the "time-varying" (i.e., within-cluster varying)
>> X
>> >> > >> predictor -- so that, for example, some clusters have high means
>> on X
>> >> > and
>> >> > >> others have low means on X -- then there is the possibility that
>> the
>> >> > >> average within-cluster effect (which is what the fixed effect
>> model
>> >> > >> estimates) differs from the overall effect of X, not conditional
>> on
>> >> the
>> >> > >> clusters. An extreme example of this is Simpson's paradox. Now
>> since
>> >> the
>> >> > >> estimate from the random-effects model can be seen as a weighted
>> >> average
>> >> > >> of
>> >> > >> these two effects, it will generally be pulled to some extent away
>> >> from
>> >> > >> the
>> >> > >> fixed-effect estimate toward the unconditional estimate, which is
>> the
>> >> > bias
>> >> > >> that econometricians fret about. However, if the cluster effects
>> are
>> >> not
>> >> > >> correlated with X, so that each cluster has the same mean on X,
>> then
>> >> > this
>> >> > >> situation is not possible, so the random-effect model will give
>> the
>> >> same
>> >> > >> unbiased estimate as the fixed-effect model.
>> >> > >>
>> >> > >> A simple solution to this problem is to retain the random-effect
>> >> model,
>> >> > >> but
>> >> > >> to split the predictor X into two components, one representing the
>> >> > >> within-cluster variation of X and the other representing the
>> >> > >> between-cluster variation of X, and estimate separate slopes for
>> >> these
>> >> > two
>> >> > >> effects. One can even test whether these two slopes differ from
>> each
>> >> > >> other,
>> >> > >> which is conceptually similar to what the Hausman test does. As
>> >> > described
>> >> > >> in the paper linked above, the estimate of the within-cluster
>> >> component
>> >> > of
>> >> > >> the X effect equals the estimate one would obtain from a
>> fixed-effect
>> >> > >> model.
>> >> > >>
>> >> > >> As for the original question, I can't speak for common practice in
>> >> > >> ecology,
>> >> > >> but I suspect it may be like it is in my home field of psychology,
>> >> where
>> >> > >> we
>> >> > >> do worry about this issue (to some extent), but we discuss it
>> using
>> >> > >> completely different language. That is, we discuss it in terms of
>> >> > whether
>> >> > >> there are different effects of the predictor at the within-cluster
>> >> and
>> >> > >> between-cluster levels, and how our model might account for that.
>> >> > >>
>> >> > >> Jake
>> >> > >>
>> >> > >> On Tue, Oct 11, 2016 at 1:50 PM, Ben Bolker <bbolker at gmail.com
>> <javascript:_e(%7B%7D,'cvml','bbolker at gmail.com');>>
>> >> wrote:
>> >> > >>
>> >> > >> >
>> >> > >> > I didn't respond to this offline, as it took me a while even
>> to
>> >> > start
>> >> > >> > to come up to speed on the question. Random effects are indeed
>> >> > defined
>> >> > >> > from *very* different points of view in the two communities
>> >> > >> > ([bio]statistical vs. econometric); I'm sure there are points of
>> >> > >> > contact, but I've been having a hard time getting my head
>> around it
>> >> > all.
>> >> > >> >
>> >> > >> > Econometric definition:
>> >> > >> >
>> >> > >> > The wikipedia page <https://en.wikipedia.org/
>> >> > wiki/Random_effects_model>
>> >> > >> > and CrossValidated question
>> >> > >> > <http://stats.stackexchange.com/questions/66161/why-do-
>> >> > >> > random-effect-models-require-the-effects-to-be-uncorrelated-
>> >> > >> with-the-inpu>
>> >> > >> > were both helpful for me.
>> >> > >> >
>> >> > >> > In the (bio)statistical world fixed and random effects are
>> usually
>> >> > >> > justified practically in terms of shrinkage estimators, or
>> >> > >> > philosophically in terms of random draws from an exchangeable
>> set
>> >> of
>> >> > >> > levels: e.g. see
>> >> > >> > <http://stats.stackexchange.com/questions/4700/what-is-
>> >> > >> > the-difference-between-fixed-effect-random-effect-and-mixed-
>> >> > >> effect-mode/>
>> >> > >> > for links.
>> >> > >> >
>> >> > >> > I don't think I can really write an answer yet. I'm still
>> >> trying to
>> >> > >> > understand at an intuitive or heuristic level what it means for
>> >> > >> > Cov(x_it,c_i)=0, where x_it is a set of explanatory variables
>> over
>> >> > time
>> >> > >> > for an individual subject and c_i is the conditional mode
>> (=BLUP in
>> >> > >> > linear mixed-model-land) for the deviation of the individual i
>> from
>> >> > the
>> >> > >> > population mean ... or more particularly what it means for that
>> >> > >> > condition to be violated, which is the point at which fixed
>> effects
>> >> > >> > would become preferred.
>> >> > >> >
>> >> > >> > As a side note, some statisticians (Andrew Gelman is the one
>> who
>> >> > >> > springs to mind) have commented on the possible overemphasis on
>> >> bias.
>> >> > >> > (All else being equal unbiased estimators are preferred to
>> biased
>> >> > >> > estimators but all else is not always equal). Two examples: (1)
>> >> > >> > penalized estimators such as lasso/ridge regression (closely
>> >> related
>> >> > to
>> >> > >> > mixed models) give biased parameter estimates with lower mean
>> >> squared
>> >> > >> > error. (2) When estimating variability, one has to choose a
>> >> particular
>> >> > >> > scale (variance, standard error, log(standard error), etc.) on
>> >> which
>> >> > one
>> >> > >> > would prefer to get an unbiased answer.
>> >> > >> >
>> >> > >> > On 16-10-11 12:02 PM, Laura Dee wrote:
>> >> > >> > > Dear all,
>> >> > >> > > Random effects are more efficient estimators ? however they
>> come
>> >> at
>> >>
>> >> > >> the
>> >> > >> > > cost of the assumption that the random effect is not
>> correlated
>> >> with
>> >> > >> the
>> >> > >> > > included explanatory variables. Otherwise, using random
>> effects
>> >> > leads
>> >> > >> to
>> >> > >> > > biased estimates (e.g., as laid out in Woolridge
>> >> > >> > > <https://faculty.fuqua.duke.edu/~moorman/Wooldridge,%20FE%20
>> >> > >> and%20RE.pdf
>> >> > >> > >'s
>> >> > >> > > Econometrics text). This assumption is a strong one for many
>> >> > >> > > observational datasets, and most analyses in economics do not
>> use
>> >> > >> random
>> >> > >> > > effects for this reason. *Is there a reason why observational
>> >> > >> ecological
>> >> > >> > > datasets would be fundamentally different that I am missing?
>> Why
>> >> is
>> >> > >> this
>> >> > >> > > important assumption (to have unbiased estimates from random
>> >> > effects)
>> >> > >> > > not emphasized in ecology? *
>> >> > >> > >
>> >> > >> > > Thanks!
>> >> > >> > >
>> >> > >> > > Laura
>> >> > >> > >
>> >> > >> > > --
>> >> > >> > > Laura Dee
>> >> > >> > > Post-doctoral Associate
>> >> > >> > > University of Minnesota
>> >> > >> > > ledee at umn.edu <javascript:_e(%7B%7D,'cvml','ledee at umn.edu');>
>> <mailto:ledee at umn.edu <javascript:_e(%7B%7D,'cvml','ledee at umn.edu');>>
>> >> > >> > > lauraedee.com <http://lauraedee.com>
>> >> > >> >
>> >> > >> > _______________________________________________
>> >> > >> > R-sig-mixed-models at r-project.org
>> <javascript:_e(%7B%7D,'cvml','R-sig-mixed-models at r-project.org');>
>> mailing list
>> >> > >> > https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>> >> > >> >
>> >> > >>
>> >> > >> [[alternative HTML version deleted]]
>> >> > >>
>> >> > >> _______________________________________________
>> >> > >> R-sig-mixed-models at r-project.org
>> <javascript:_e(%7B%7D,'cvml','R-sig-mixed-models at r-project.org');>
>> mailing list
>> >> > >> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>> >> > >>
>> >> > >
>> >> > >
>> >> > >
>> >> > > --
>> >> > >
>> >> > >
>> >> > >
>> >> > >
>> >> > > Thanks,
>> >> > > John
>> >> > >
>> >> > >
>> >> > > John Poe
>> >> > > Doctoral Candidate
>> >> > > Department of Political Science
>> >> > > Research Methodologist
>> >> > > UK Center for Public Health Services & Systems Research
>> >> > > University of Kentucky
>> >> > > 111 Washington Avenue, Room 203a
>> >> > > Lexington, KY 40536
>> >> > > www.johndavidpoe.com
>> >>
>> >
>>
>>
>> --
>> Laura Dee
>> Post-doctoral Associate
>> University of Minnesota
>> ledee at umn.edu <javascript:_e(%7B%7D,'cvml','ledee at umn.edu');>
>> lauraedee.com
>>
>> [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-mixed-models at r-project.org
>> <javascript:_e(%7B%7D,'cvml','R-sig-mixed-models at r-project.org');>
>> mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>
>
>
>
> --
>
>
>
>
> Thanks,
> John
>
>
> John Poe
> Doctoral Candidate
> Department of Political Science
> Research Methodologist
> UK Center for Public Health Services & Systems Research
> University of Kentucky
> 111 Washington Avenue, Room 203a
> Lexington, KY 40536
> www.johndavidpoe.com
>
--
Laura Dee
Post-doctoral Associate
University of Minnesota
ledee at umn.edu
lauraedee.com
[[alternative HTML version deleted]]
More information about the R-sig-mixed-models
mailing list