[R-meta] Outlier and influence cases 3-level meta-analysis

[Viechtbauer, Wolfgang (SP)]

Dear Daniel,

No fudamental issues, except that I have not yet implemented all of this for 'rma.mv' objects. Assuming that you have installed the 'devel' version of metafor (https://wviechtb.github.io/metafor/#installation), things like rstandard(), rstudent(), cooks.distance(), and dfbetas() are however already available to you, which are some of the main tools for detecting outliers and/or influential cases anyway (covariance ratios would also be nice, but not yet available). There is also hatvalues() and weights(), but these may be a bit less accessible.

One issue in more complex models is the question at which 'level' we want to detect outliers / influential cases. For rma.mv objects, things like rstandard() and cooks.distance() have a 'cluster' argument, which can be used to specify a grouping variable. For example:

dat <- dat.konstantopoulos2011
res <- rma.mv(yi, vi, random = ~ 1 | district/school, data=dat)
cooks.distance(res)

will compute the Cook's distances for each individual estimate, while

cooks.distance(res, cluster=dat$district)

will compute the Cook's distances for each level of 'district'. Or easier for interpretation:

plot(cooks.distance(res), type="o", pch=19)

suggests maybe 2 somewhat influential estimates and

plot(cooks.distance(res, cluster=dat$district), type="o", pch=19)

one influential district (probably the one with those 2 estimates, but I haven't checked).

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>Behalf Of daniel.f.gucciardi using gmail.com
>Sent: Wednesday, 17 March, 2021 9:44
>To: r-sig-meta-analysis using r-project.org
>Subject: [R-meta] Outlier and influence cases 3-level meta-analysis
>
>Hi all,
>
>My colleagues and I have conducted a 3-level meta-analysis using metafor in
>which we account for multiple outcomes from individual studies. As part of
>our sensitivity analyses, we examined outlier and influential cases using
>the recommendations of Viechtbauer and Cheung
>(https://onlinelibrary.wiley.com/doi/10.1002/jrsm.11), which seems to be a
>common approach. Are there any issues in applying their framework for
>two-level meta-analytic models to three-level models? If so, I'd appreciate
>your direction to any literature that might help me appreciate these issues
>please.
>
>Regards,
>
>Daniel