[R-meta] About how to interpret the following P-value from the result

[Nick Chen]

Dear Micheal,

     Thank you for the clarification. This has bugged me for quite a long
time. Then one follow-up question, is there any test that can be done to
show which of the element in the subgroup (here in this case, intercpt,
DeviceMD, DeviceN/A, DeviceV/A) has significant difference than the others?

Nick

Michael Dewey via R-sig-meta-analysis <r-sig-meta-analysis using r-project.org>於
2023年10月28日 週六，下午8:38寫道：

> Dear Nick
>
> Comment in-line
>
> On 28/10/2023 10:08, 英文科陳品誠 via R-sig-meta-analysis wrote:
> > I'm currently performing a meta-analysis on the relationship between
> > "Digital game-based learning" and "English vocabulary acquisition". And
> I'm
> > using the random effect model for moderator analysis since the
> > heterogeneity effect is quite high. The following is the code I use for
> > performing the moderator analysis for one moderator in my dataset,
> "device
> > used":
> >
> >> mod.Device <- rma.mv(yi = y, V = v,+                        slab =
> studyID, data = DG,+                        random = ~ 1 |
> studyID/effectsizeID,+                        test = "t", method =
> "REML",+                        mods = ~ Device)> mod.Device
> >
> > and here is the result:
> >
> >
> >
> > Multivariate Meta-Analysis Model (k = 29; method: REML)
> >
> > Variance Components:
> >
> >              estim    sqrt  nlvls  fixed                factor
> > sigma^2.1  0.0663  0.2575     28     no               studyID
> > sigma^2.2  0.0000  0.0000     29     no  studyID/effectsizeID
> >
> > Test for Residual Heterogeneity:
> > QE(df = 25) = 44.8744, p-val = 0.0086
> >
> > Test of Moderators (coefficients 2:4):
> > F(df1 = 3, df2 = 25) = 0.8975, p-val = 0.4563
> >
> > Model Results:
> >
> >             estimate      se     tval  df    pval    ci.lb   ci.ub
> > intrcpt      0.7746  0.1008   7.6856  25  <.0001   0.5670  0.9822  ***
> > DeviceMD     0.0898  0.1681   0.5343  25  0.5978  -0.2564  0.4361
> > DeviceN/A   -0.2275  0.2338  -0.9732  25  0.3398  -0.7091  0.2540
> > DeviceV/A    0.2968  0.2936   1.0107  25  0.3219  -0.3080  0.9015
> >
> > intrcpt    = computer
> >
> > DeviceMD   = mobile devices
> >
> > DeviceN/A  = not available
> >
> > DeviceV/A  = VR or AR devices
> > ---
> >
> > and this is what I'm a little bit confused on. Can anyone help me clarify
> > whether my explanation is correct or wrong?
> > (1) the p-value under "*Test of moderators*" which is 0.4563, does this
> > mean that this moderator (device used) does not have a significant impact
> > on English vocabulary learning using digital game-based learning? (Since
> > the p-value here is larger than 0.05)
>
> Yes, that is what it implies.
>
> > (2) the p-value of "*intrcpt*" which is <.0001, does this mean that using
> > computers to learn English vocabulary still has a significant difference
> > among the four types of devices?
> >
>
> No, that is the estimated value of the vocabulary score for the fourth
> category of your moderator variable. It is unlikely that it has any
> scientific value.
>
> Michael
>
> > 陳品誠 (Nick Chen)
> > Email: t571 using wlgsh.tp.edu.tw <t5741 using wlgsh.tp.edu.tw>
> >
> >       [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> > To manage your subscription to this mailing list, go to:
> > https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> >
>
> --
> Michael
>
> _______________________________________________
> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> To manage your subscription to this mailing list, go to:
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>

	[[alternative HTML version deleted]]