[R-meta] Extracting pooled rma.mv model summary statistics after multiple imputation

Fri Dec 10 02:50:50 CET 2021

Hi Wolfgang,

Thanks again for your helpful response.

If I may, I have another question – how can I calculate the pooled variance inflation factors for each moderator to test for collinearity (again, after using mice for multiple imputation)?

Many thanks,
Tom

From: Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
Sent: Sunday, 14 November 2021 1:58 AM
To: Thomas Swanton <thomas.swanton using sydney.edu.au>; r-sig-meta-analysis using r-project.org <r-sig-meta-analysis using r-project.org>
Subject: RE: Extracting pooled rma.mv model summary statistics after multiple imputation 

Hi Thomas,

Thanks for the example. This leads to an error though in this step:

> pool <- summary(pool(fit))
Error in h(simpleError(msg, call)) :                                                                                     
  error in evaluating the argument 'object' in selecting a method for function 'summary': the condition has length > 1

To be precise, by default, you will just get a bunch of warnings, but I use:

Sys.setenv("_R_CHECK_LENGTH_1_CONDITION_"="true")

which ends up switching the warnings here to errors.

The problem really comes from:
> pool(fit)
Error in if (x$ddf > 0) qt(level/2, df = x$ddf, lower.tail = FALSE) else NA :                                            
  the condition has length > 1 

And this error comes from a bug in 'broom' which is used by 'mice'. The problem is in tidy.rma(), namely this line:

https://protect-au.mimecast.com/s/GOuDCjZ1N7inN41D0uW2cOp?domain=github.com

crit <- if (x$ddf > 0) qt(level/2, df=x$ddf, lower.tail=FALSE) else NA

In rma.mv() models with test="t", x$ddf is a vector, not a single value, and hence we get an if() conditions that has length > 1. I will report this to the broom authors.

As for your actual question, see this:

https://protect-au.mimecast.com/s/Py6MCk81N9tOGy3AvsVz45E?domain=stat.ethz.ch

So you could do:

round(apply(sapply(fit$analyses, \(x) x$sigma2), 1, mean), 4)
format.pval(median(sapply(fit$analyses, \(x) x$QEp)))
format.pval(median(sapply(fit$analyses, \(x) x$QMp)))

Best,
Wolfgang

>-----Original Message-----
>From: Thomas Swanton [mailto:thomas.swanton using sydney.edu.au]
>Sent: Thursday, 11 November, 2021 1:18
>To: Viechtbauer, Wolfgang (SP); r-sig-meta-analysis using r-project.org
>Subject: Re: Extracting pooled rma.mv model summary statistics after multiple
>imputation
>
>Dear Wolfgang,
>
>Thank you for your reply, and my apologies for the delayed response - I was on
>leave for a few weeks.
>
>Below I provide a minimal and fully reproducible example using the Bangert-Drowns
>et al. sample data following your blog post at https://www.metafor-
>project.org/doku.php/tips:multiple_imputation_with_mice_and_metafor
>
>I am conducting a three-level mixed-effects meta-regression with multiple effect
>sizes (level 2; defined in the below example by "id") nested within studies
>(level 3; defined in the below example by "author").
>
>Where I am stuck is: how do I get the portion of the output about variance
>components, heterogeneity, and the test for moderators after using multiple
>imputation? The pooling gives me the model results (estimate, std error, t
>statistic, df, p-value), but how do I pool the portion of output that is normally
>above "Model Results"?
>
>In the last line of the code (fit$analyses[1]), I can access this for each
>imputed dataset, but I am unsure how to access the pooled summary statistics.
>
>I hope this clarifies where I am stuck. I would be very grateful for your advice.
>
>Many thanks,
>
>Tom Swanton
>PhD Candidate
>School of Psychology, The University of Sydney
>
># load packages
>
>library(metafor)
>library(mice)
>
># use data from meta-analysis by Bangert-Drowns et al. (2004)
>
>dat <- dat.bangertdrowns2004
>
># keep variables needed for analysis
>
>dat <- dat[c("id", "author", "yi", "vi", "length", "wic", "feedback", "info",
>"pers", "imag", "meta")]
>
># turn dummy variables into factors
>
>dat$wic <- factor(dat$wic)
>dat$feedback <- factor(dat$feedback)
>dat$info  <- factor(dat$info)
>dat$pers  <- factor(dat$pers)
>dat$imag  <- factor(dat$imag)
>dat$meta  <- factor(dat$meta)
>
># set up predictor matrix for imputations
>
>predMatrix <- make.predictorMatrix(dat)
>predMatrix[,"id"] <- 0 # don't use effect size id for imputing
>predMatrix[,"author"] <- 0 # don't use author for imputing
>predMatrix[,"vi"] <- 0 # don't use vi for imputing
>predMatrix["id",] <- 0 # don't impute id (no NA's)
>predMatrix["author",] <- 0 # don't impute author (no NA's)
>predMatrix["yi",] <- 0 # don't impute yi (no NA's)
>predMatrix["vi",] <- 0 # don't impute vi (no NA's)
>predMatrix
>
># specify imputation method
>
>impMethod <- make.method(dat)
>impMethod
>
># generate multiple imputations
>
>imp <- mice(dat, print=FALSE, m=20, predictorMatrix=predMatrix, method=impMethod,
>seed=1234)
>
># fit model of interest to each of the 20 imputed datasets
>
>fit <- with(imp,
>            rma.mv(yi = yi,
>                   V = vi,
>                   slab = author,
>                   random = ~ 1 | author/id,
>                   test = "t",
>                   method = "REML",
>                   mods = ~ length + wic + feedback + info + pers + imag + meta))
>
># pool and round results
>
>pool <- summary(pool(fit))
>pool[-1] <- round(pool[-1], digits=4)
>pool
>
># view output from first imputed dataset
>
>fit$analyses[1]
>
>From: Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
>Sent: Saturday, 16 October 2021 12:18 AM
>To: Thomas Swanton <thomas.swanton using sydney.edu.au>; r-sig-meta-analysis using r-
>project.org <r-sig-meta-analysis using r-project.org>
>Subject: RE: Extracting pooled rma.mv model summary statistics after multiple
>imputation
>
>Hi Tom,
>
>Could you provide a minimal and fully reproducible example (see: https://protect-
>au.mimecast.com/s/-pXHCnx1jni7Z8PPPu9Oavy?domain=stat.ethz.ch) that can be used
>to discuss this? Right now, it's not entirely clear to me where exactly you are
>stuck.
>
>Best,
>Wolfgang
>
>>-----Original Message-----
>>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>>Behalf Of Thomas Swanton
>>Sent: Friday, 15 October, 2021 2:02
>>To: r-sig-meta-analysis using r-project.org
>>Subject: [R-meta] Extracting pooled rma.mv model summary statistics after
>>multiple imputation
>>
>>Dear colleagues,
>>
>>Is anyone able to help me understand how to extract pooled rma.mv model summary
>>statistics after applying multiple imputation using mice please?
>>
>>I am conducting a three-level mixed effects multiple meta-regression using the
>>rma.mv function in metafor, and have applied multiple imputation using mice
>>following the instructions available here: https://www.metafor-
>>project.org/doku.php/tips:multiple_imputation_with_mice_and_metafor
>>
>>I have the table of pooled model coefficients as in the worked example, but I
>>haven't been able to work out how to extract pooled model summary statistics
>>(e.g., tau-squared, Q statistic).
>>
>>Are you able to help me understand how to pool them across all the imputed
>>datasets please?
>>
>>Many thanks,
>>Tom
>>
>>Thomas Swanton, PhD Candidate
>>School of Psychology, The University of Sydney, Australia
>>thomas.swanton using sydney.edu.au
>>
>>Tom Swanton | PhD Candidate and Research
>> Assistant
>>
>>The University of Sydney
>>
>>Faculty of Science, School of Psychology, Brain & Mind Centre, Gambling
>Treatment
>>& Research Clinic
>>
>>Rm 204, Level 2, 94 Mallett St, Camperdown | NSW | 2050
>>
>>+61 432 891 501
>>
>>thomas.swanton using sydney.edu.au | sydney.edu.au
>>
>>Twitter: @TomSwanton |
>> ResearchGate: researchgate.net/profile/Thomas-Swanton
>>
>>Editorial Assistant, International Gambling Studies
>>
>>We acknowledge the tradition of custodianship and law of the Country on which
>the
>>University
>> of Sydney campuses stand. We pay our respects to those who have cared and
>>continue to care for Country.