[R-meta] clubSandwich for variance components

[Stefanou Revesz]

Very helpful, thank you very much, James. Unlike many others, I didn't
use robumeta much because its correlated effects model essentially was
a 2-level meta-regression model that allowed for observed estimates to
covary within each study. This was not as nearly enough for the messy
longitudinal studies that I dealt with.

Speaking of, a while ago, I asked a question
(https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2021-August/003098.html)
regarding potential design-related moderators (e.g., in terms of
quality of treatment effects, threats to internal validity, ranking of
design's quality etc.) that can distinguish between 3 longitudinal
designs for meta-analysis (purely by the way the 3 designs are set
up).

In case it is of interest to you, I would highly appreciate it if you
could offer your thoughts on it there.

Once again, many thanks,
Stefanou

On Mon, Aug 30, 2021 at 9:35 PM James Pustejovsky <jepusto using gmail.com> wrote:
>
> Hi Stefanou,
>
> This is a good question, about which there's currently limited evidence. robumeta can estimate two different working models, the correlated effects model and the hierarchical effects model:
>
> robumeta's correlated effects model has a single variance component, tau-squared, estimated using the method of moments (MoM). Section 5.4.4 of the supplementary materials of my paper on "expanding the range of working models" (https://osf.io/nyv4u/) has some simulation results on the performance of the MoM estimator compared to the REML estimator of the more general CHE working model. We found that both estimators had similar accuracy (as measured by of root mean squared error) when the CE model is correctly specified or mildly mis-specified. (The MoM estimator is less biased but has higher variance than the REML estimator.) In the presence of within-study heterogeneity, however, the REML estimator from the more general CHE working model is more accurate--sometimes substantially so.
>
> robumeta's hierarchical effects model has two variance components, tau-squared (between-study) and omega-squared (within-study), again estimated using MoM. We did not study this estimator in the paper, but my sense is that its performance would be similar to the MoM for the correlated effects model, or perhaps worse. These MoM estimators are not constructed in an efficient way and so will likely have higher variance than the REML estimators of the same working model. Admittedly, I'm speculating here, so it would definitely be helpful to have some simulation evidence with head-to-head comparisons between them (if anyone's lurking on the listserv looking for research ideas!).
>
> James
>
> On Mon, Aug 30, 2021 at 7:24 PM Stefanou Revesz <stefanourevesz using gmail.com> wrote:
>>
>> Dear James,
>>
>> Thank you very much. This is very helpful to know, almost any thread I
>> looked at in the archives talked about the systematic bias in the
>> estimates of variance components and the necessity of using RVE to
>> account for that, and yet I couldn't see how they are corrected. (this
>> can easily create a lot of confusion among substantively oriented
>> meta-analysts).
>>
>> Just curious, do Tau and Omega estimates given by robumeta (I'm aware
>> of its underlying limitations in terms of modeling dependence etc.)
>> are themselves free from systematic bias (I think they are estimated
>> using closed-form equations)?
>>
>> Thank you for this very important clarification,
>> Stefanou
>>
>> On Mon, Aug 30, 2021 at 6:54 PM James Pustejovsky <jepusto using gmail.com> wrote:
>> >
>> > Hi Stefanou,
>> >
>> > clubSandwich provides standard errors, hypothesis tests, and confidence intervals for regression coefficient estimates (for meta-regressions and an array of other types of linear, generalized linear, and linear mixed effects models). It does *not* adjust variance component estimates (such as between-study heterogeneity estimates) not does it provide robust confidence intervals for variance components. The issues you asked about are challenging and relatively un-studied problems, and so unfortunately I don’t have any suggestions about alternative packages to look at.
>> >
>> > James
>> >
>> > > On Aug 30, 2021, at 5:18 PM, Stefanou Revesz <stefanourevesz using gmail.com> wrote:
>> > >
>> > > Dear Colleagues,
>> > >
>> > > I have recently learned about the RVE estimation and the usefulness of
>> > > the clubSandwich package in this regard.
>> > >
>> > > I read the documentation of the package, but I was unable to determine
>> > > which function or functions in the package provide the adjusted
>> > > estimates of variance components and possibly their confidence
>> > > intervals (if I'm not mistaken, these are the ones that are often
>> > > unreliable if left unadjusted)?
>> > >
>> > > Thank you for your assistance,
>> > > Stefanou
>> > >
>> > > _______________________________________________
>> > > R-sig-meta-analysis mailing list
>> > > R-sig-meta-analysis using r-project.org
>> > > https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis