[R-meta] Follow-up: Interpreting variance components in rma.mv
yh342 @end|ng |rom n@u@edu
Sat Aug 27 05:38:52 CEST 2022
Thank you very much for your clear response.
As another alternative, if I define my model as:
rma.mv(yi ~ 0 + cat_mod * time + covariates, random = list(~ time | study,
~1| effect), struc = "UN")
Then, let's say we get the following var-covar matrix for the true effects
at our time points in each study averaged across all studies:
time0 0.6 0.2
time1 0.2 1.2
with rho(time0, time1) = 1.00 (I know 1.00 is odd but in my case this is
In addition, a typical study's own heterogeneity, lets say, is estimated to
If Gain (cat1) = 0.27; Gain (cat2) = 0.33, then Var(Gain) from time0 to
time1 at the study level will be:
Var(Gain) = sqrt( Var(time0) + Var(time1) - 2 * rho(time0, time1) *
(sqrt(Var[time0]) * sqrt(Var[time1])) )
***Question: So now should we add Var(effects) i.e., omega in your
notation, to the above equation or it needs to be further modified?
On Fri, Aug 26, 2022 at 7:38 AM James Pustejovsky <jepusto using gmail.com> wrote:
> Hi Yuhang,
> The probability calculations are not correct here because the SD you're
> using does not apply to gains. For the model you've specified:
> y_ij = b1 * Cat1_ij + b2 * Cat2_ij + b3 * Cat1_ij x Time1_ij + b4 *
> Cat2_ij x Time2_ij + u_j + v_ij + e_ij,
> where Var(u_j) = tau^2, Var(v_ij) = omega^2, and Var(e_ij) = V_ij (the
> known sampling variance).
> Now consider a new study j* that reports effects of type Cat1 at both
> time0 (i = 1) and time1 (i = 2), the true effect size parameters would be:
> y_1j* = b1 + u_j* + v_1j*
> y_2j* = b1 + b3 + u_j* + v_2j*
> and therefore the gain score would be
> y_2j* - y_1j* = b3 + v_2j* - v_1j*.
> Under the assumptions of your model,
> E(y_2j* - y_1j*) = b3
> Var(y_2j* - y_1j*) = Var(v_2j*) + Var(v_1j*) = 2 * omega^2,
> So you would need to calculate the prediction using an SD of sqrt(2) *
> One thing to emphasize here is that these calculations hinge on the model
> being appropriately specified. If you've got the random effect structure
> wrong, then the probability calculation could be completely off.
> Another way to approach this prediction would be to do a meta-analysis of
> the gain scores directly (i.e., take the effect sizes from time-1 minus
> those from time-0 and use that in a basic random effects meta-analysis).
> You could then do the probability calculation in the usual way (as you did
> in your earlier post).
> On Wed, Aug 24, 2022 at 10:50 PM Yuhang Hu <yh342 using nau.edu> wrote:
>> Hello All,
>> I wanted to ask a follow-up on my previous post (
>> I'm currently fitting the following model (cat_mod = categorical mod):
>> rma.mv(yi ~ 0 + cat_mod * time + covariates, random = ~ 1 | study/effect)
>> with a total heterogeneity in sd unit = 0.699.
>> "cat_mod" levels' means at time0 are very different from each other. As
>> such, I have computed the gains (i.e., time1 - time0) for each level of
>> Gain (cat1) = 0.27
>> Gain (cat2) = 0.33
>> ***Question: I wonder whether I can say the following or not?***
>> "The probability that a gain from time0 to time1 in cat1 is 0 or larger
>> pnorm(0,.27, .699, lower.tail = FALSE)
>> >  0.650
>> "The probability that a gain from time0 to time1 in cat2 is 0 or larger
>> pnorm(0,.33, .699, lower.tail = FALSE)
>> >  0.68
>> Thank you for your attention.
>> Yuhang Hu
>> [[alternative HTML version deleted]]
>> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
>> To manage your subscription to this mailing list, go to:
Yuhang Hu (She/Her/Hers)
Ph.D. Student in Applied Linguistics
Department of English
Northern Arizona University
[[alternative HTML version deleted]]
More information about the R-sig-meta-analysis