[R-meta] variance explained by fixed & random effects
Viechtbauer, Wolfgang (SP)
wo||g@ng@v|echtb@uer @end|ng |rom m@@@tr|chtun|ver@|ty@n|
Wed Mar 13 18:00:34 CET 2019
You forgot some parts in your code, so to make this example reproducible, let's use this model:
dat <- dat.konstantopoulos2011
res <- rma.mv(yi, vi, mods = ~ factor(year), random = ~ 1 | district/school, data=dat)
And I would consider schools nested within districts here, but this isn't pertinent to the issue.
One way of estimating an R^2-type measure for the fixed effects is to fit the reduced model without the predictor and then compute the proportional reduction in the variance components:
res0 <- rma.mv(yi, vi, random = ~ 1 | district/school, data=dat)
pmax(0, (res0$sigma2 - res$sigma2) / res0$sigma2) * 100
max(0, (sum(res0$sigma2) - sum(res$sigma2)) / sum(res0$sigma2)) * 100
One can compute this per component or overall. There is no guarantee that the value is >= 0, so I use pmax()/max() to set negative values to 0. One could debate whether it even makes sense here to estimate how much of the school-level heterogeneity is accounted for by 'year' (since this is a district-level predictor), but that's another issue.
I am not sure how to think of the question how much variance is explained by a random effect here. One could ask how much of the total heterogeneity is due to district- and schol-level variance, which would be:
res0$sigma2 / sum(res0$sigma2) * 100
Or one could ask how much of the unaccounted for heterogeneity (so heterogeneity not accounted for by 'year') is due to district- and schol-level variance, which would be:
res$sigma2 / sum(res$sigma2) * 100
Or one could ask how much of the total variance (so heterogenity + sampling variance) is due to district- and schol-level variance, which would be:
k <- res$k
wi <- 1/dat$vi
s2 <- (k-1) * sum(wi) / (sum(wi)^2 - sum(wi^2))
res0$sigma2 / (sum(res0$sigma2) + s2) * 100
(here I am using the Higgins & Thompson, 2002, definition of a 'typical' sampling variance).
Or one could ask how much of the unaccounted for variance (so unaccounted for heterogenity + sampling variance) is due to district- and schol-level variance, which would be:
res1$sigma2 / (sum(res1$sigma2) + s2) * 100
These are at least some possibilities.
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On Behalf Of Theresa Stratmann
Sent: Wednesday, 13 March, 2019 17:29
To: r-sig-meta-analysis using r-project.org
Subject: [R-meta] variance explained by fixed & random effects
Dear Meta-Analysis Community,
I am using the metafor package to help me summarize some data on habitat selection. I have a question whose answer I have not found yet on the useful metafor website, and was hoping that someone could help me (or point me to an online resource I missed).
As part of the description of my meta-analysis I would like to explain how much of the variation in habitat selection is due to my fixed effects (season) versus my random effects (individual & year).
For a normal mixed-effects model I know how to compute the marginal and conditional R^2 (Nakagawa & Schielzeth 2013), but I am wondering how to do this correctly for a meta-analysis, or maybe, better said, talk about variance explained by the fixed and random effects.
First I looked into I^2 and the post on this for random effect models. But then I read more from Borenstein et al. (https://www.meta-analysis-workshops.com/download/common-mistakes1.pdf) and was not quiet sure anymore if this is what I want (I^2 = proportion of observed variance that reflects variance in the true effects and not the sampling error ... so this seems more of a summary for the entire model (?), which also needs to be treated with care and presented properly).
Then I found this question on this mailing list: https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2017-September/000232.html ...but cannot really get this code to produce anything sensible, because I guess I am not clear on the formula for res0 vs. res1.
So I will give a different example, which closely approximates what I am trying to do (a model with fixed & random effects):
dat <- dat.konstantopoulos2011
res <- rma.mv(yi, vi, mods = ~ factor(year), list( ~ 1 | district , ~ 1|school) data=dat)
How would you calculate / correctly describe variance explained by the fixed effects (year)? By the random effects (district, school)? I am not tied to a particular statistic, just want to make sure that whatever I do is an honest representation of my results and gives readers (& myself) the information they need to interpret the results.
I am new to meta-analyses and this is just a small part of my project, so I have been able to do some reading, but not an in-depth dive. Therefore I would appreciate any help you can offer.
B.S. Ecology, The University of Georgia
M.S. Wildlife & Fisheries Biology, Clemson University
Senckenberg Biodiversity and Climate Research Center
More information about the R-sig-meta-analysis