[R-meta] I^2 results with small sampling variance

[Martineau, Roger]

Dear metafor users,

I downloaded the data set (i.e., p154-dataset.csv) from  Assinka and Wibbelink (2016) available from http://www.tqmp.org/RegularArticles/vol12-3/p154/p154.pdf


The data set contains 100 studies and I compared I^2 statistics from the model with no moderator (overall) and the one with 2 significant moderators (multiplemoderator).


1)      Model overall (Listing 5 in the publication) and I^2 statistics:

overall <- rma.mv(y, v, random = list(~ 1 | effectsizeID, ~ 1 | studyID), tdist=TRUE, data=dataset)

> # Calculation of I^2 statistic
> W <- diag(1/overall$vi)
> X <- model.matrix(overall)
> P <- W - W %*% X %*% solve(t(X) %*% W %*% X) %*% t(X) %*% W
> 100 * sum(overall$sigma2) / # I^2 statistic
+   (sum(overall$sigma2) + (overall$k-overall$p)/sum(diag(P)))
[1] 93.058
> # Separation of total variance by cluster
> 100 * overall$sigma2 / (sum(overall$sigma2) + (overall$k-overall$p)/sum(diag(P)))
[1] 34.754 58.304


>




2)      Model multiplemoderator (Listing 5 in the publication) and I^2 statistics:

multiplemoderator <- rma.mv(y, v, mods = ~ pyear + typeovert + typecovert,
                            random =list(~ 1 | effectsizeID, ~ 1 | studyID),
                            tdist=TRUE, data=dataset)

> # Calculation of I^2 statistic
> W <- diag(1/multiplemoderator$vi)
> X <- model.matrix(multiplemoderator)
> P <- W - W %*% X %*% solve(t(X) %*% W %*% X) %*% t(X) %*% W
> 100 * sum(multiplemoderator$sigma2) / # I^2 statistic
+   (sum(multiplemoderator$sigma2) + (multiplemoderator$k-multiplemoderator$p)/sum(diag(P)))
[1] 90.993
> # Separation of total variance by cluster
> 100 * multiplemoderator$sigma2 / (sum(multiplemoderator$sigma2) + (multiplemoderator$k-multiplemoderator$p)/sum(diag(P)))
[1] 33.079 57.913


>



Questions:

The sampling variance is small at 7% to 11% in these models.
What is the correct interpretation of I^2 results? And for the slight reduction in sampling variance despite the addition of significant moderators?
I get similar results from my own data set with a large number of studies despite the addition of several significant moderators in the model.

Thanks in advance,

Roger :)

Roger Martineau, mv Ph.D.
Centre de recherche et de développement
sur le bovin laitier et le porc
Agriculture et agroalimentaire Canada/Agriculture and Agri-Food Canada
Téléphone/Telephone: 819-780-7319
Télécopieur/Facsimile: 819-564-5507
2000, Rue Collège / 2000, College Street
Sherbrooke (Québec)  J1M 0C8
Canada
roger.martineau at agr.gc.ca<mailto:roger.martineau at agr.gc.ca>



	[[alternative HTML version deleted]]