[R-meta] Credibility intervals with RVE

[Fabian Schellhaas]

Hi James,

Thanks a lot for the quick response -- very helpful as always! We already
report sensitivity analyses across different levels of rho (under others),
so I'll just add deltas for the upper and lower bounds of the credibility
interval.

Cheers,
Fabian


On Tue, Apr 9, 2019 at 4:25 PM James Pustejovsky <jepusto using gmail.com> wrote:

> Fabian,
>
> a) Your approach is exactly what I would do (and have done in the past).
> b) You are exactly right about the limitations of the approach. With RVE,
> it's recommended to conduct sensitivity analyses. You could do the same
> thing here, and report how the prediction interval changes depending on the
> assumed value of r.
>
> James
>
> On Tue, Apr 9, 2019 at 1:44 PM Fabian Schellhaas <
> fabian.schellhaas using yale.edu> wrote:
>
>> Hi all,
>>
>> Credibility intervals can be useful because – in contrast to confidence
>> intervals – they consider the heterogeneity of synthesized effects and
>> make
>> predictions about the range of values within which the true effect of
>> future studies will likely fall (e.g., IntHout et al., 2016).
>>
>> It is straightforward enough to calculate credibility intervals for rma
>> and
>> rma.mv objects in metafor using the `predict` function. However, (a) how
>> would one compute credibility intervals for multivariate multilevel
>> meta-analytic models with cluster-robust variance estimation, and (b) is
>> doing so even meaningful, given that the covariance matrix has only been
>> aproximated? I included more details about both questions below.
>>
>> Thanks a lot!
>> Fabian
>>
>> -----
>>
>> Regarding question (a), I would compute the credibility interval for a
>> simplified example as follows:
>>
>> # fit model
>> vcv <- clubSandwich::impute_covariance_matrix(vi = data$vi, cluster =
>> data$cluster_id, r = 0.7)
>> m <- metafor::rma.mv(yi, vcv, random = ~ 1 | cluster_id/es_id, data =
>> data)
>> robu <- clubSandwich::coef_test(m, vcov = "CR2")
>>
>> # compute 95% credibility interval
>> robu$beta - 1.96*sqrt(sum(m$sigma2) + robu$SE^2)
>> robu$beta + 1.96*sqrt(sum(m$sigma2) + robu$SE^2)
>>
>> Regarding question (b), since V in this model is just a rough
>> approximation
>> of the empirical covariance structure, the variance components of the
>> model
>> are also just an approximation as well. Even when using cluster-robust
>> standard errors for the computation of the credibility intervals, we still
>> use the model's approximated variance components, and thus the credibility
>> interval provides just a rough guess of the range within which future
>> studies will fall.
>>
>>         [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-meta-analysis mailing list
>> R-sig-meta-analysis using r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>
>

	[[alternative HTML version deleted]]