[R-meta] clubSandwich for variance components

[James Pustejovsky]

Hi Stefanou,

This is a good question, about which there's currently limited evidence.
robumeta can estimate two different working models, the correlated effects
model and the hierarchical effects model:

robumeta's correlated effects model has a single variance component,
tau-squared, estimated using the method of moments (MoM). Section 5.4.4 of
the supplementary materials of my paper on "expanding the range of working
models" (https://osf.io/nyv4u/) has some simulation results on the
performance of the MoM estimator compared to the REML estimator of the more
general CHE working model. We found that both estimators had similar
accuracy (as measured by of root mean squared error) when the CE model is
correctly specified or mildly mis-specified. (The MoM estimator is less
biased but has higher variance than the REML estimator.) In the presence of
within-study heterogeneity, however, the REML estimator from the more
general CHE working model is more accurate--sometimes substantially so.

robumeta's hierarchical effects model has two variance components,
tau-squared (between-study) and omega-squared (within-study), again
estimated using MoM. We did not study this estimator in the paper, but my
sense is that its performance would be similar to the MoM for the
correlated effects model, or perhaps worse. These MoM estimators are not
constructed in an efficient way and so will likely have higher variance
than the REML estimators of the same working model. Admittedly, I'm
speculating here, so it would definitely be helpful to have some simulation
evidence with head-to-head comparisons between them (if anyone's lurking on
the listserv looking for research ideas!).

James

On Mon, Aug 30, 2021 at 7:24 PM Stefanou Revesz <stefanourevesz using gmail.com>
wrote:

> Dear James,
>
> Thank you very much. This is very helpful to know, almost any thread I
> looked at in the archives talked about the systematic bias in the
> estimates of variance components and the necessity of using RVE to
> account for that, and yet I couldn't see how they are corrected. (this
> can easily create a lot of confusion among substantively oriented
> meta-analysts).
>
> Just curious, do Tau and Omega estimates given by robumeta (I'm aware
> of its underlying limitations in terms of modeling dependence etc.)
> are themselves free from systematic bias (I think they are estimated
> using closed-form equations)?
>
> Thank you for this very important clarification,
> Stefanou
>
> On Mon, Aug 30, 2021 at 6:54 PM James Pustejovsky <jepusto using gmail.com>
> wrote:
> >
> > Hi Stefanou,
> >
> > clubSandwich provides standard errors, hypothesis tests, and confidence
> intervals for regression coefficient estimates (for meta-regressions and an
> array of other types of linear, generalized linear, and linear mixed
> effects models). It does *not* adjust variance component estimates (such as
> between-study heterogeneity estimates) not does it provide robust
> confidence intervals for variance components. The issues you asked about
> are challenging and relatively un-studied problems, and so unfortunately I
> don’t have any suggestions about alternative packages to look at.
> >
> > James
> >
> > > On Aug 30, 2021, at 5:18 PM, Stefanou Revesz <stefanourevesz using gmail.com>
> wrote:
> > >
> > > Dear Colleagues,
> > >
> > > I have recently learned about the RVE estimation and the usefulness of
> > > the clubSandwich package in this regard.
> > >
> > > I read the documentation of the package, but I was unable to determine
> > > which function or functions in the package provide the adjusted
> > > estimates of variance components and possibly their confidence
> > > intervals (if I'm not mistaken, these are the ones that are often
> > > unreliable if left unadjusted)?
> > >
> > > Thank you for your assistance,
> > > Stefanou
> > >
> > > _______________________________________________
> > > R-sig-meta-analysis mailing list
> > > R-sig-meta-analysis using r-project.org
> > > https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>

	[[alternative HTML version deleted]]