[R-meta] Hypothesis Testing of R2 in metafor

[Viechtbauer, Wolfgang (NP)]

Dear Zhouhan,

Yes, one could interpret the omnibus test of all coefficients as a test of R^2 = 0. They do not match up as nicely though as in regular regression, where there is a direct relationship between the F-statistic and R^2, namely:

R^2 = F / (F + (n-p)/(p-1))

where n is the sample size and p is the number of coefficients. This formula doesn't apply to mixed-effects meta-regression models and, for example, cases can arise where R^2 = 0 while the omnibus test statistic is greater than 0. But yes, in principle, if one cannot reject the null hypothesis that all coefficients (except for the intercept) are equal to 0, then this would imply that one cannot reject R^2 = 0.

Best,
Wolfgang

> -----Original Message-----
> From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> On Behalf
> Of Zhouhan Jin via R-sig-meta-analysis
> Sent: Tuesday, May 28, 2024 18:28
> To: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis using r-
> project.org>; Michael Dewey <lists using dewey.myzen.co.uk>
> Cc: Zhouhan Jin <zjin65 using uwo.ca>
> Subject: Re: [R-meta] Hypothesis Testing of R2 in metafor
>
> Thanks for the link regarding the CIs for R^2.
>
> But my question was: can one, in principle, take the p-value used to test the
> null that all model coefs are zero to also serve as a null hypothesis test of
> R^2 being zero or there is a separate hypothesis test for R^2 (again in
> principle)?
>
> In fact, is such a principle used in meta-analysis as I (think) it is used in
> regular regression (below)?
>
> summary(lm(mpg ~ cyl, data= mtcars))$fstatistic
>
> Best wishes,
>
> Zhouhan
> On May 28, 2024 at 06:32 -0400, Michael Dewey <lists using dewey.myzen.co.uk>, wrote:
>
> Dear Zhouhan
>
> Does https://www.metafor-project.org/doku.php/tips:ci_for_r2 help?
>
> Michael
>
> On 28/05/2024 04:34, Zhouhan Jin via R-sig-meta-analysis wrote:
> Hello Colleagues,
>
> My understanding is that it is common to report the p-value for the omnibus null
> hypothesis that all the coefficients in a model are zero, next to an R^2 value
> for an rma.mv()/rma() model.
>
> If the above assumption is correct, then, should one also take that p-value to
> be a null hypothesis test of R^2 being zero or there is a separate test for R^2?
>
> Thanks in advance.
>
> Best wishes,
>
> Zhouhan