[R-meta] Appending a risk-of-bias traffic-light plot to a 'three-level' forest plot

[Joshua Bernal]

Greetings Dr. Viechtbauer and members,

I ran a three-level meta-analysis with rma.mv (due to multiple
measures of a construct within-studies). I seek help on how to append
a risk-of-bias traffic-light plot to an extension of the three-level
forest plot, please find the query details below...

Goal A (metafor):
Create the three-level forest plot with each study's "aggregated"
effect estimate and CI.
(Source A: stat.ethz.ch/pipermail/r-sig-meta-analysis/2019-February/001423.html)

Goal A Issues:
1) Source A presents 2 ways to obtain "aggregated" effect sizes, with
the second way getting "exactly the same results as those from the
full model". Because the present rma.mv analysis uses the
t-distribution (test = "t"), the p-values and summary CIs obtained are
slightly different/larger. Is it possible to get the exact same
results as when test = "t" for the rma.mv model? To see what this
looks like, please find the metafor reprex below.
2) After running the reprex, you will see that Study 6 contributes one
effect size only, its CI from the "standard" rma.mv forest plot (where
all effect sizes in each study are shown) vs its CI from the
"aggregated" forest plot are different. If the argument
"tau2=res$sigma2[2]" is omitted from the code of the "aggregated"
forest plot, then the obtained summary CI for Study 6 then matches
with that of the "standard" plot. Which would be considered the
correct approach in this case?

metafor reprex:
###
library(metafor)

dat <- structure(list(author = c("Study 1", "Study 1", "Study 1", "Study 1",
"Study 1", "Study 2", "Study 2", "Study 2", "Study 2", "Study 3",
"Study 3", "Study 4", "Study 4", "Study 5", "Study 5", "Study 6"
), study = c(1, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 4, 4, 5, 5, 6),
    outcome = c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14,
    15, 16), yi = c(0.52, 0.256, 0.5003, 0.3131, -0.1019, 0.3591,
    0.0341, 0.0357, 0.3942, -0.1503, 0.2671, 0.131, 0.1664, 1.1761,
    0.5799, 0.406), vi = c(0.00926, 0.00199, 0.01171, 0.0171,
    0.00524, 0.0712, 0.0971, 0.0823, 0.0912, 0.1204, 0.3536,
    0.0655, 0.0607, 0.9571, 0.6617, 0.5139)), row.names = c(NA,
16L), class = "data.frame")

res <- rma.mv(yi, vi, random = ~ 1 | author/outcome, data = dat, test
= "t", method = "REML", slab = author)
res
forest(res)

sav <- lapply(split(dat, dat$author), function(x) {
    res <- rma(yi, vi, data=x, tau2=res$sigma2[2])
    return(c(coef(res), vcov(res)))
})

sav <- data.frame(do.call(rbind, sav))
names(sav) <- c("yi", "vi")

forest(sav$yi, sav$vi, slab=rownames(sav))

rma(yi, vi, data=sav)
res
###

Goal B (robvis, development version 0.3.0.900):
Create a risk-of-bias traffic-light plot using the "Generic" (or aka
"ROB1") template.
(Source B: https://github.com/mcguinlu/robvis  ("Generic")  OR
cran.r-project.org/web/packages/robvis/vignettes/Introduction_to_robvis.html#rob1
 ("ROB1"))

Goal B Issues:
After running the robvis reprex below, you will see that an extra
judgment label ("Critical" or dark red circle with exclamation mark)
is automatically added/cannot be removed (although only 5 judgment
labels are relevant in the dataset, I am forced to specify 6 judgement
labels). I am aware there is an argument "judgement_levels" under
development, but do not know if it is relevant and if so how to take
it from here...

robvis reprex:
###
library(robvis)

rp <- structure(list(Study = c("Study 1", "Study 2", "Study 3", "Study 4",
"Study 5", "Study 6"), Bias.due.to.confounding. = c("Not applicable",
"Not applicable", "Not applicable", "Some concerns", "Low", "No information"
), Bias.in.selection.of.participants.into.the.study. = c("Not applicable",
"Not applicable", "Not applicable", "Some concerns", "Low", "Low"
), Bias.in.classification.of.interventions. = c("Not applicable",
"Not applicable", "Not applicable", "Some concerns", "Low", "Low"
), Bias.arising.from.the.randomization.process. = c("Some concerns",
"Low", "Low", "Not applicable", "Not applicable", "Not applicable"
), Bias.due.to.deviations.from.intended.intervention. = c("Low",
"Some concerns", "Low", "Low", "Some concerns", "Low"),
Bias.due.to.missing.outcome.data. = c("Some concerns",
"Low", "No information", "Some concerns", "Low", "No information"
), Bias.in.measurement.of.the.outcome. = c("Some concerns", "Low",
"Some concerns", "Some concerns", "Low", "Some concerns"),
Bias.due.to.selection.of.the.reported.result. = c("Some concerns",
"Low", "High", "Some concerns", "Low", "High"), Overall = c("Some concerns",
"Low", "High", "Some concerns", "Low", "High")), row.names = c(NA,
6L), class = "data.frame")

bor <- rob_traffic_light(rp, tool="Generic", colour="cochrane",
psize=10, overall=TRUE, judgement_labels=c("","High","Some
concerns","Low","No information","Not applicable"))

bor
###

Goal C:
Append the risk-of-bias traffic-light plot (from Goal B) to the forest
plot (from Goal A).
(Source C: mcguinlu.github.io/robvis/articles/metafor.html#appending-risk-of-bias-plots-to-forest-plots-1)

Goal C Issues:
Because the present analysis considers the bias domains of both
randomized and non-randomized controlled intervention studies and risk
of bias was respectively assessed by ROB2 and ROBINS-I, the "Generic"
template is used because it allows modifying the number of domains so
as to include the relevant domains of both tools in the traffic-light
plot. However, according to "rob_tools(forest = TRUE)", robvis
currently supports only "ROB2" and "ROBINS-I" templates for the
"rob_append_to_forest()" function.

Other sub-goals:
- Modify text of header from "Estimate [95% CI]" to "SMD [95% CI]".
- Modify text of x-axis label from "Observed Outcome" to "Standardized
Mean Difference".
- To the right of "SMD [95% CI]", add a column with the header "No. of
effect sizes" to display the number of effect sizes within each study.

Many thanks in advance for your help...

Sincerely,
Josh