[R-meta] extracting data from multi-way ANOVAs

[Ilya Fischhoff]

Hello everyone,

I'm new to this listserv, and excited to learn about it. My apologies
if this question has been addressed before and I arrived too late to
read it.

I'm working on a meta-analysis for which I would like to use data as
reported in (repeated measures) multi-way ANOVAs or mixed effects
models. In these papers, means and variances of groups are not
available. For papers reporting results of one-way ANOVAs, I've been
using this formula relating the absolute value of Cohen's d to F ratio
and sample sizes: |d| = sqrt(F*(nt + nc)/(nt*nc)), where nt and nc are
treatment and control sample sizes. (Source: Koricheva, J., et al.
(2013). Handbook of meta-analysis in ecology and evolution, Princeton
University Press. p. 200.)

Does this same formula apply to the F ratio and sample sizes in a two-
or three-way ANOVA, or a linear mixed effects model?

I noticed, using synthetic data, that the F value resulting from a
one-way ANOVA differ from the F value for the same factor in a two-way
ANOVA. This gave me pause in applying the same formula to multi-way
ANOVAs. A typical paper reports, for each variable and interactions in
a two- or three-way ANOVA or linear mixed effects model, the degrees
of freedom, F value, and P value. Here is an example of two-way ANOVA
results from a paper that examined effects of food and infection on
NH4 release by Daphnia (aquatic organisms):

Food C:P ratio
F[2,42] = 0.044, P = 0.96

Infection
F[1,42] = 1.92, P = 0.17

Food X Infection
F[2,42] = 2.59, P = 0.088

(Source: Narr, C. F. and P. C. Frost (2015). "Does infection tilt the
scales? Disease effects on the mass balance of an invertebrate
nutrient recycler." Oecologia 179(4): 969-979.)

For this meta-analysis we are interested in the effect of infection,
not in the other factor (food) or their interaction. In the
meta-analysis, we are analyzing absolute values of d, so we do not
need to know the direction of the effect. Some papers we find also
report MS and SS, and/or test statistics for the error term.

We have been using equations reported in this paper
(https://www.bmj.com/content/343/bmj.d2090) to estimate variance in d
based on the P value.

I'd really appreciate feedback (and references, if possible) on
whether the formula at the top can be applied to multi-way ANOVAs or
mixed effects models. If not, guidance on correct equations for
estimating d would be much appreciated! It would be fantastic to be
able to use data from many more papers.

Thank you for reading and for any advice!
Best,
Ilya