[R-meta] Extracting pooled rma.mv model summary statistics after multiple imputation

[Thomas Swanton]

Dear Wolfgang,

Thank you for your reply, and my apologies for the delayed response – I was on leave for a few weeks.

Below I provide a minimal and fully reproducible example using the Bangert-Drowns et al. sample data following your blog post at https://www.metafor-project.org/doku.php/tips:multiple_imputation_with_mice_and_metafor

I am conducting a three-level mixed-effects meta-regression with multiple effect sizes (level 2; defined in the below example by "id") nested within studies (level 3; defined in the below example by "author").

Where I am stuck is: how do I get the portion of the output about variance components, heterogeneity, and the test for moderators after using multiple imputation? The pooling gives me the model results (estimate, std error, t statistic, df, p-value), but how do I pool the portion of output that is normally above "Model Results"?

In the last line of the code (fit$analyses[1]), I can access this for each imputed dataset, but I am unsure how to access the pooled summary statistics.

I hope this clarifies where I am stuck. I would be very grateful for your advice.

Many thanks,

Tom Swanton
PhD Candidate
School of Psychology, The University of Sydney


# load packages

library(metafor)
library(mice)

# use data from meta-analysis by Bangert-Drowns et al. (2004)

dat <- dat.bangertdrowns2004

# keep variables needed for analysis

dat <- dat[c("id", "author", "yi", "vi", "length", "wic", "feedback", "info", "pers", "imag", "meta")]

# turn dummy variables into factors

dat$wic <- factor(dat$wic)
dat$feedback <- factor(dat$feedback)
dat$info  <- factor(dat$info)
dat$pers  <- factor(dat$pers)
dat$imag  <- factor(dat$imag)
dat$meta  <- factor(dat$meta)

# set up predictor matrix for imputations

predMatrix <- make.predictorMatrix(dat)
predMatrix[,"id"] <- 0 # don't use effect size id for imputing
predMatrix[,"author"] <- 0 # don't use author for imputing
predMatrix[,"vi"] <- 0 # don't use vi for imputing
predMatrix["id",] <- 0 # don't impute id (no NA's)
predMatrix["author",] <- 0 # don't impute author (no NA's)
predMatrix["yi",] <- 0 # don't impute yi (no NA's)
predMatrix["vi",] <- 0 # don't impute vi (no NA's)
predMatrix

# specify imputation method

impMethod <- make.method(dat)
impMethod

# generate multiple imputations

imp <- mice(dat, print=FALSE, m=20, predictorMatrix=predMatrix, method=impMethod, seed=1234)

# fit model of interest to each of the 20 imputed datasets

fit <- with(imp,
            rma.mv(yi = yi,
                   V = vi,
                   slab = author,
                   random = ~ 1 | author/id,
                   test = "t",
                   method = "REML",
                   mods = ~ length + wic + feedback + info + pers + imag + meta))

# pool and round results

pool <- summary(pool(fit))
pool[-1] <- round(pool[-1], digits=4)
pool

# view output from first imputed dataset

fit$analyses[1]



From: Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
Sent: Saturday, 16 October 2021 12:18 AM
To: Thomas Swanton <thomas.swanton using sydney.edu.au>; r-sig-meta-analysis using r-project.org <r-sig-meta-analysis using r-project.org>
Subject: RE: Extracting pooled rma.mv model summary statistics after multiple imputation 
 
Hi Tom,

Could you provide a minimal and fully reproducible example (see: https://protect-au.mimecast.com/s/-pXHCnx1jni7Z8PPPu9Oavy?domain=stat.ethz.ch) that can be used to discuss this? Right now, it's not entirely clear to me where exactly you are stuck.

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>Behalf Of Thomas Swanton
>Sent: Friday, 15 October, 2021 2:02
>To: r-sig-meta-analysis using r-project.org
>Subject: [R-meta] Extracting pooled rma.mv model summary statistics after
>multiple imputation
>
>Dear colleagues,
>
>Is anyone able to help me understand how to extract pooled rma.mv model summary
>statistics after applying multiple imputation using mice please?
>
>I am conducting a three-level mixed effects multiple meta-regression using the
>rma.mv function in metafor, and have applied multiple imputation using mice
>following the instructions available here: https://www.metafor-
>project.org/doku.php/tips:multiple_imputation_with_mice_and_metafor
>
>I have the table of pooled model coefficients as in the worked example, but I
>haven't been able to work out how to extract pooled model summary statistics
>(e.g., tau-squared, Q statistic).
>
>Are you able to help me understand how to pool them across all the imputed
>datasets please?
>
>Many thanks,
>Tom
>
>Thomas Swanton, PhD Candidate
>School of Psychology, The University of Sydney, Australia
>thomas.swanton using sydney.edu.au
>
>Tom Swanton | PhD Candidate and Research
> Assistant
>
>The University of Sydney
>
>Faculty of Science, School of Psychology, Brain & Mind Centre, Gambling Treatment
>& Research Clinic
>
>Rm 204, Level 2, 94 Mallett St, Camperdown | NSW | 2050
>
>+61 432 891 501
>
>thomas.swanton using sydney.edu.au | sydney.edu.au
>
>Twitter: @TomSwanton |
> ResearchGate: researchgate.net/profile/Thomas-Swanton
>
>Editorial Assistant, International Gambling Studies
>
>We acknowledge the tradition of custodianship and law of the Country on which the
>University
> of Sydney campuses stand. We pay our respects to those who have cared and
>continue to care for Country.