[R-meta] Error in weight effect in rma

[Viechtbauer, Wolfgang (SP)]

Hi Maria,

The point sizes are based only on the diagonal of the weight matrix. However, in such models, the weighting is more complex as described here:

https://www.metafor-project.org/doku.php/tips:weights_in_rma.mv_models

which covers this specific model (for other types of rma.mv() models, the weighting structure/matrix is of course different). So you might want to examine the 'row-sum weights' and you could even use them to specify the point sizes (via the psize argument) if you like.

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>Behalf Of Wiertsema, M.
>Sent: Tuesday, 07 December, 2021 14:21
>To: r-sig-meta-analysis using r-project.org
>Subject: [R-meta] Error in weight effect in rma
>
>Hi everyone,
>
>I am currently working on a meta-analysis on the longitudinal
>association between bullying (predictor) and social status (outcome).
>
>Because we have multiple effects within one study, we used the
>random-effects model for the forest plots. I have plotted the forest
>plot, with effects based on Fisher's z, with the following code:
># random-effects model
>ram2 <- rma.mv(yi, vi, random = ~ 1 | Sampleshort/Effectsize,
>data=dat2, slab=paste(Effectsize, AuthorsR, YEAR, Country, plotn,
>TypebullyingR, AgeBullyR, sep="; "), method="REML")
>print(ram2, digits=3)
>
>#check profile plots
>
>par(mfrow=c(2,1))
>profile(ram2, sigma2=1)
>profile(ram2, sigma2=2)
>
>#forest plots
>forest(ram2, xlim=c(-0.85, 1.20), at=(c(-0.3, -0.2, -0.1, 0, 0.1, 0.2,
>0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1)),
>       rows=c(1:31),
>       xlab="Fisher's Z", mlab="")
>
>### add text with Q-value, dfs, and p-value
>text(-0.85, -1, pos=4, cex=1, bquote(paste("Multilevel RE Model for
>All Studies (Q = ",
>                                           .(formatC(ram2$QE,
>digits=2, format="f")), ", df = ",
>                                           .(ram2$k - ram2$p),
>                                           ", p = ",
>.(formatC(ram2$QEp, digits=2, format="f")),
>                                           ")")))
>
>### switch to bold font
>par(font=2)
>
>### add column headings to the plot
>text((-0.43), 32.5, "Effect number, Study, Country, Sample size, Type
>of bullying, & Age bully")
>text((1.015), 32.5, "Fisher's z correlations [95% CI]")
>
>So far, so good.
>
>However, when interpreting the forest plot we noticed something strange:
>There are a couple of effect sizes, from two different samples, which
>we expected to have a greater weight based on their CI and based on
>the fact that those were the only effect sizes based on a sample. I
>included the forest plot as an appendix, I hope you can see it.
>
>For effect number 49 and 56, it would make sense to have a greater
>weight based on their CI's, [0.45-0.57] and [0.36-0.44] respectively,
>thus relatively small. For effect number 50 and 51, which are both
>based on the same sample, the CI's are [0.24-0.36] and [0.34-0.46], so
>the same interval as effect number 49 and even larger than effect
>number 56. However, those two effects have a bigger weight compared to
>effect number 49 and 46.
>
>Additionally, the Cooks distance plot does recognize effect number 49
>as an outlier, but based on the weight of that effect size in the
>forest plot you would not expect that.
>
>Am I missing something or is something weird going on in my code?
>
>Please let me know if anything is unclear.
>Many thanks in advance!
>
>Best,
>Maria
>
>--
>PhD student
>University of Groningen, Faculty of Behavioural and Social Sciences
>Grote Rozenstraat 38, 9712 TJ Groningen
>email: m.wiertsema using rug.nl | phone: 050 - 363 3182