[R-meta] Interpreting variance components in rma.mv

[James Pustejovsky]

Hi Yuhang,

I am just not sure whether what you're describing has a legitimate
statistical interpretation. I think the calculations you've described would
have to be examined through simulation studies in order to understand their
statistical properties.

James

On Mon, Aug 22, 2022 at 1:06 AM Yuhang Hu <yh342 using nau.edu> wrote:

> And James to add some clarity to my previous email, I can imagine 3
> situations for calculating that probability using:
>
> (1) Estimates of average effect and total variation in sd unit (Kind of
> like point estimate for probability)
>
> (2) Lower CI limit of average effect and upper CI limit of total variation
> in sd unit (Kind of like lower limit for probability)
>
> (3) Upper CI limit of average effect and lower CI limit of total variation
> in sd unit (Kind of like upper limit for probability)
>
> I wonder how legitimate this proposal might sound to you? Thanks!
>
> Yuhang
>
> On Sun, Aug 21, 2022 at 9:28 PM Yuhang Hu <yh342 using nau.edu> wrote:
>
>> Dear James,
>>
>> Thank you. You noted that using estimates of average effect and total
>> variation (in sd unit) ignores the fact that these quantities are
>> themselves estimates and not fixed values.
>>
>> But can't we use the lower and upper limits of these estimates' own CIs
>> to obtain the a range to supplement A in the following calculations?
>>
>> A:
>> pnorm(0, average_effect, average_total_variation, lower.tail = FALSE)
>>
>> B:
>> pnorm(0, lower_average_effect, lower_total_variation, lower.tail = FALSE)
>>
>> pnorm(0, upper_average_effect, upper_total_variation, lower.tail = FALSE)
>>
>> Best,
>> Yuhang
>>
>> On Sun, Aug 21, 2022 at 12:08 PM James Pustejovsky <jepusto using gmail.com>
>> wrote:
>>
>>> Hi Yuhang,
>>>
>>> But is it appropriate to assume that true effects' dispersion at time 0
>>>> and time 1 is exactly the same (equality of variances across time points)?
>>>>
>>>
>>> The model you've fit assumes that the variances are equal across time
>>> points. Whether this assumption is appropriate is an empirical question and
>>> something you'll need to gauge for yourself. You could probe it by, for
>>> example, fitting a model that allows the variance components to differ by
>>> time point:
>>> rma.mv(yi ~ 0 + cat_mod * time + covariates, random = ~ time |
>>> study/effect, struct = "UN")
>>> And then comparing the fit of this model to the fit of the model that
>>> assumes compound symmetry (i.e., your initial model).
>>>
>>> James
>>>
>>
>>
>> --
>> Yuhang Hu (She/Her/Hers)
>> Ph.D. Student in Applied Linguistics
>> Department of English
>> Northern Arizona University
>>
>
>
> --
> Yuhang Hu (She/Her/Hers)
> Ph.D. Student in Applied Linguistics
> Department of English
> Northern Arizona University
>

	[[alternative HTML version deleted]]