[R-meta] clubSandwich for rma.uni() models

[Luke Martinez]

Hi James,

This is an amazing answer. I guess the 3 items you have listed are
ordered in terms of applicability.

Almost always (1) is readily applicable and relevant. With (2),
determining "bit of dependency" is a bit difficult or at least
involves some trial and error. With (3), one has to put in some effort
in determining the amount of endogeneity and even after that it's a
bit of challenge to abandon the idea of rma.mv() that can itself be
corrected for misspecification in favor of an rma() idea especially if
the data structure cries for an rma.mv() model.

Related to all of this, @Wolfgang asked a question a while back
(https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2021-September/003189.html)
regarding if: *"the degree of discrepancy between the standard and the
cluster-robust Wald-test could be used as a rough measure [of the]
extent the working model is reasonable"*

It seems the answer is yes according to
(https://doi.org/10.1016/j.csda.2003.08.006 ; p.439):

"[R]obust standard errors [are] an indicator for possible
misspecification of the model or its assumptions. If the robust
standard errors are much different from the asymptotic standard
errors, this should be interpreted as a warning sign that some
important assumption is violated."

Any thoughts?
Luke

On Mon, Nov 29, 2021 at 3:27 PM James Pustejovsky <jepusto using gmail.com> wrote:
>
> Hi Luke,
>
> cluster-robust variance estimation methods are relevant to rma.uni()
> models for a few reasons:
> 1. If you cluster by row, as in vcovCR(rma_uni_fit, cluster =
> dat$es_ID, type = "CR2"), you get heteroskedasticity-robust standard
> errors. This can be useful if the sampling variances used in
> estimating the random effects model could be systematically
> inaccurate/wrong or just because, in practice, the sampling variances
> are usually estimated rather than known exactly. Sidik & Jonkman
> provide a thorough rationale and description in this paper:
> Sidik, K., & Jonkman, J. N. (2006). Robust variance estimation for
> random effects meta-analysis. Computational Statistics & Data
> Analysis, 50(12), 3681-3701.
> 2. Perhaps you have a dataset with a little bit of dependency (say,
> just a few studies that report multiple effect size estimates) but you
> don't want to go to the trouble of modeling it all and you'd rather
> just ignore the dependencies. Instead of sticking your fingers in your
> ears and going "la la la", you could fit the model with rma.uni
> (ignoring the dependencies) but then use cluster-robust standard
> errors to acknowledge the possibility that not all of the effect sizes
> are independent.
> 3. Perhaps you have some other reason to fit a univariate (or
> "marginal") model to a dataset that has some dependency structure to
> it. For instance, multivariate random effects models involve (tacit)
> assumptions about independence between random effects and predictors
> and independence between random effects and structural features of the
> data (such as sampling variances or number of effect sizes per study).
> Using a multivariate model when those assumptions are violated can
> lead to systematically biased estimates of average effects, and
> perhaps there's a situation where using a univariate model would avoid
> those assumptions and produce unbiased estimates. In such a situation,
> it would make sense to cluster the standard errors to account for
> dependence in the effect size estimates.
>
> James
>
> On Mon, Nov 29, 2021 at 12:09 AM Luke Martinez <martinezlukerm using gmail.com> wrote:
> >
> > Dear Meta Experts,
> >
> > (A) My understanding has been that the sandwich estimators are only
> > relevant to rma.mv() models where the structure of `V=` and/or
> > `random=` is suspected to be misspecified (hence SE of fixed effects
> > may be inaccurate).
> >
> > (B) My understanding has also been that the sandwich estimators use
> > the highest clustering variable in purely nested models to compute the
> > dfs needed for fixed effects' p-value calculations.
> >
> > rma.uni() models may loosely meet the (B) requirement. But it is not
> > obvious to me how such models may meet the (A) requirement.
> >
> > Thus, how is clubSandwich:::vcovCR.rma.uni() relevant to rma.uni() models?
> >
> > Thanks,
> > Luke
> >
> > _______________________________________________
> > R-sig-meta-analysis mailing list
> > R-sig-meta-analysis using r-project.org
> > https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis