[R-meta] Questions regarding REML and FE models and R^2 calculation in metafor

[Nevo Sagi]

Dear list members,

I conducted a meta-analysis on the role of climate in mediating a specific
ecological process, using the *metafor *package in R.
This is actually a meta-regression, using the rma.mv function, with
*temperature *and *precipitation *as moderators, along with some other
moderators related to experimental design. I also use reference as a random
effect ('random = ~1|*Reference'*), as some references include more than
one experiment.

*1. FE vs REML model:*
After reading Wolfgang Viechtbauer's blog post
<https://wviechtb.github.io/metafor/reference/misc-models.html> on the
differences between fixed-effects and random-effects models in the
*metafor *package, I decided to use the FE method, because the studies I
gathered are not a random sample of the population of hypothetical studies.
Instead, the sample is biased by underrepresentation of some climates and
overrepresentation of others.
I wonder whether my interpretation of the difference between FE and REML
models is correct, and would like to get some feedback on it.

*2. R^2 calculation:*
Reviewers of my manuscript required that I provide R-squared values for
each of the climate moderators.
Using the *metafor *package, only rma.uni models (where random variables
cannot be specified) provide R^2 estimation.
In a previous conversation in this mailing list, Wolfgang indicated that
pseudo-R^2 can be calculated based on the variance (sigma2) reported by
models including and excluding the moderator in question:
*(res0$sigma2 - res1$sigma2) / res0$sigma2*
*where 'res0' is the model without coefficients and 'res1' the model with.*

I have two problems with this solution:
1. FE models do not provide variance components (sigma2). Therefore, the
pseudo R-squared can be calculated only for REML models. I guess this can
be explained by the nature of the models, which I don't fully understand.
2. When using REML models and performing the above calculation, I get weird
results. For example, one of the pseudo R^2 values was above 1. This cannot
mean that the moderator explained more than 100% of the variance in the
effect size. How comparable is this pseudo R^2 for the standard R^2 of
simpler models?


To conclude, I will be glad to get feedback on both problems:
1. Should I use a random-effect or fixed-effect model?
2. How do I get a reliable R^2 or an alternative measure of goodness of fit
for single-moderator models that include a random structure and a sampling
variance?


Thank you very much,

Nevo Sagi


-- 
Dr. Nevo Sagi

Prof. Dror Hawlena's Risk-Management Ecology Lab
Department of Ecology, Evolution & Behavior
The Alexander Silberman Institute of Life Sciences
The Hebrew University of Jerusalem
Edmond J. Safra Campus at Givat Ram, Jerusalem 9190401, Israel.

	[[alternative HTML version deleted]]