[R-meta] Follow-up: Interpreting variance components in rma.mv

[James Pustejovsky]

Hi Yuhang,

The probability calculations are not correct here because the SD you're
using does not apply to gains. For the model you've specified:
y_ij = b1 * Cat1_ij + b2 * Cat2_ij + b3 * Cat1_ij x Time1_ij + b4 * Cat2_ij
x Time2_ij + u_j + v_ij + e_ij,
where Var(u_j) = tau^2, Var(v_ij) = omega^2, and Var(e_ij) = V_ij (the
known sampling variance).

Now consider a new study j* that reports effects of type Cat1 at both time0
(i = 1) and time1 (i = 2), the true effect size parameters would be:
y_1j* = b1 + u_j* + v_1j*
y_2j* = b1 + b3 + u_j* + v_2j*
and therefore the gain score would be
y_2j* - y_1j* = b3 + v_2j* - v_1j*.

Under the assumptions of your model,
E(y_2j* - y_1j*) = b3
Var(y_2j* - y_1j*) = Var(v_2j*) + Var(v_1j*) = 2 * omega^2,
So you would need to calculate the prediction using an SD of sqrt(2) *
omega.

One thing to emphasize here is that these calculations hinge on the model
being appropriately specified. If you've got the random effect structure
wrong, then the probability calculation could be completely off.

Another way to approach this prediction would be to do a meta-analysis of
the gain scores directly (i.e., take the effect sizes from time-1 minus
those from time-0 and use that in a basic random effects meta-analysis).
You could then do the probability calculation in the usual way (as you did
in your earlier post).

James

On Wed, Aug 24, 2022 at 10:50 PM Yuhang Hu <yh342 using nau.edu> wrote:

> Hello All,
>
> I wanted to ask a follow-up on my previous post (
> https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2022-August/004139.html
> ).
>
> I'm currently fitting the following model (cat_mod = categorical mod):
>
> rma.mv(yi ~ 0 + cat_mod * time + covariates, random = ~ 1 | study/effect)
>
> with a total heterogeneity in sd unit = 0.699.
>
> "cat_mod" levels' means at time0 are very different from each other. As
> such, I have computed the gains (i.e., time1 - time0) for each level of
> cat_mod:
>
> Gain (cat1) = 0.27
> Gain (cat2) = 0.33
>
> ***Question: I wonder whether I can say the following or not?***
>
> "The probability that a gain from time0 to time1 in cat1 is 0 or larger is:
> pnorm(0,.27, .699, lower.tail = FALSE)
> > [1] 0.650
>
> "The probability that a gain from time0 to time1 in cat2 is 0 or larger is:
> pnorm(0,.33, .699, lower.tail = FALSE)
> > [1] 0.68
>
> Thank you for your attention.
>
> Yuhang Hu
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> To manage your subscription to this mailing list, go to:
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>

	[[alternative HTML version deleted]]