[R-meta] About how to interpret the following P-value from the result

[Michael Dewey]

Dear Nick

Comment in-line

On 28/10/2023 10:08, 英文科陳品誠 via R-sig-meta-analysis wrote:
> I'm currently performing a meta-analysis on the relationship between
> "Digital game-based learning" and "English vocabulary acquisition". And I'm
> using the random effect model for moderator analysis since the
> heterogeneity effect is quite high. The following is the code I use for
> performing the moderator analysis for one moderator in my dataset, "device
> used":
> 
>> mod.Device <- rma.mv(yi = y, V = v,+                        slab = studyID, data = DG,+                        random = ~ 1 | studyID/effectsizeID,+                        test = "t", method = "REML",+                        mods = ~ Device)> mod.Device
> 
> and here is the result:
> 
> 
> 
> Multivariate Meta-Analysis Model (k = 29; method: REML)
> 
> Variance Components:
> 
>              estim    sqrt  nlvls  fixed                factor
> sigma^2.1  0.0663  0.2575     28     no               studyID
> sigma^2.2  0.0000  0.0000     29     no  studyID/effectsizeID
> 
> Test for Residual Heterogeneity:
> QE(df = 25) = 44.8744, p-val = 0.0086
> 
> Test of Moderators (coefficients 2:4):
> F(df1 = 3, df2 = 25) = 0.8975, p-val = 0.4563
> 
> Model Results:
> 
>             estimate      se     tval  df    pval    ci.lb   ci.ub
> intrcpt      0.7746  0.1008   7.6856  25  <.0001   0.5670  0.9822  ***
> DeviceMD     0.0898  0.1681   0.5343  25  0.5978  -0.2564  0.4361
> DeviceN/A   -0.2275  0.2338  -0.9732  25  0.3398  -0.7091  0.2540
> DeviceV/A    0.2968  0.2936   1.0107  25  0.3219  -0.3080  0.9015
> 
> intrcpt    = computer
> 
> DeviceMD   = mobile devices
> 
> DeviceN/A  = not available
> 
> DeviceV/A  = VR or AR devices
> ---
> 
> and this is what I'm a little bit confused on. Can anyone help me clarify
> whether my explanation is correct or wrong?
> (1) the p-value under "*Test of moderators*" which is 0.4563, does this
> mean that this moderator (device used) does not have a significant impact
> on English vocabulary learning using digital game-based learning? (Since
> the p-value here is larger than 0.05)

Yes, that is what it implies.

> (2) the p-value of "*intrcpt*" which is <.0001, does this mean that using
> computers to learn English vocabulary still has a significant difference
> among the four types of devices?
> 

No, that is the estimated value of the vocabulary score for the fourth 
category of your moderator variable. It is unlikely that it has any 
scientific value.

Michael

> 陳品誠 (Nick Chen)
> Email: t571 using wlgsh.tp.edu.tw <t5741 using wlgsh.tp.edu.tw>
> 
> 	[[alternative HTML version deleted]]
> 
> _______________________________________________
> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> To manage your subscription to this mailing list, go to:
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> 

-- 
Michael