[R-meta] Extracting pooled rma.mv model summary statistics after multiple imputation

[Viechtbauer, Wolfgang (SP)]

Hi Tom,

I don't know if people ever pool VIFs. But I suppose one could just their average across models.

vif() in metafor provides the VIFs. Just use lapply() to get those across fit$analyses elements and then average per moderator.

Best,
Wolfgang

>-----Original Message-----
>From: Thomas Swanton [mailto:thomas.swanton using sydney.edu.au]
>Sent: Friday, 10 December, 2021 2:51
>To: Viechtbauer, Wolfgang (SP); r-sig-meta-analysis using r-project.org
>Subject: Re: Extracting pooled rma.mv model summary statistics after multiple
>imputation
>
>Hi Wolfgang,
>
>Thanks again for your helpful response.
>
>If I may, I have another question - how can I calculate the pooled variance
>inflation factors for each moderator to test for collinearity (again, after using
>mice for multiple imputation)?
>
>Many thanks,
>Tom
>
>
>From: Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
>Sent: Sunday, 14 November 2021 1:58 AM
>To: Thomas Swanton <thomas.swanton using sydney.edu.au>; r-sig-meta-analysis using r-
>project.org <r-sig-meta-analysis using r-project.org>
>Subject: RE: Extracting pooled rma.mv model summary statistics after multiple
>imputation
>
>Hi Thomas,
>
>Thanks for the example. This leads to an error though in this step:
>
>> pool <- summary(pool(fit))
>Error in h(simpleError(msg, call))
>:
>
>  error in evaluating the argument 'object' in selecting a method for function
>'summary': the condition has length > 1
>
>To be precise, by default, you will just get a bunch of warnings, but I use:
>
>Sys.setenv("_R_CHECK_LENGTH_1_CONDITION_"="true")
>
>which ends up switching the warnings here to errors.
>
>The problem really comes from:
>> pool(fit)
>Error in if (x$ddf > 0) qt(level/2, df = x$ddf, lower.tail = FALSE) else NA
>:
>  the condition has length > 1
>
>And this error comes from a bug in 'broom' which is used by 'mice'. The problem
>is in tidy.rma(), namely this line:
>
>https://protect-au.mimecast.com/s/GOuDCjZ1N7inN41D0uW2cOp?domain=github.com
>
>crit <- if (x$ddf > 0) qt(level/2, df=x$ddf, lower.tail=FALSE) else NA
>
>In rma.mv() models with test="t", x$ddf is a vector, not a single value, and
>hence we get an if() conditions that has length > 1. I will report this to the
>broom authors.
>
>As for your actual question, see this:
>
>https://protect-au.mimecast.com/s/Py6MCk81N9tOGy3AvsVz45E?domain=stat.ethz.ch
>
>So you could do:
>
>round(apply(sapply(fit$analyses, \(x) x$sigma2), 1, mean), 4)
>format.pval(median(sapply(fit$analyses, \(x) x$QEp)))
>format.pval(median(sapply(fit$analyses, \(x) x$QMp)))
>
>Best,
>Wolfgang
>
>>-----Original Message-----
>>From: Thomas Swanton [mailto:thomas.swanton using sydney.edu.au]
>>Sent: Thursday, 11 November, 2021 1:18
>>To: Viechtbauer, Wolfgang (SP); r-sig-meta-analysis using r-project.org
>>Subject: Re: Extracting pooled rma.mv model summary statistics after multiple
>>imputation
>>
>>Dear Wolfgang,
>>
>>Thank you for your reply, and my apologies for the delayed response - I was on
>>leave for a few weeks.
>>
>>Below I provide a minimal and fully reproducible example using the Bangert-
>Drowns
>>et al. sample data following your blog post at https://www.metafor-
>>project.org/doku.php/tips:multiple_imputation_with_mice_and_metafor
>>
>>I am conducting a three-level mixed-effects meta-regression with multiple effect
>>sizes (level 2; defined in the below example by "id") nested within studies
>>(level 3; defined in the below example by "author").
>>
>>Where I am stuck is: how do I get the portion of the output about variance
>>components, heterogeneity, and the test for moderators after using multiple
>>imputation? The pooling gives me the model results (estimate, std error, t
>>statistic, df, p-value), but how do I pool the portion of output that is
>normally
>>above "Model Results"?
>>
>>In the last line of the code (fit$analyses[1]), I can access this for each
>>imputed dataset, but I am unsure how to access the pooled summary statistics.
>>
>>I hope this clarifies where I am stuck. I would be very grateful for your
>advice.
>>
>>Many thanks,
>>
>>Tom Swanton
>>PhD Candidate
>>School of Psychology, The University of Sydney
>>
>># load packages
>>
>>library(metafor)
>>library(mice)
>>
>># use data from meta-analysis by Bangert-Drowns et al. (2004)
>>
>>dat <- dat.bangertdrowns2004
>>
>># keep variables needed for analysis
>>
>>dat <- dat[c("id", "author", "yi", "vi", "length", "wic", "feedback", "info",
>>"pers", "imag", "meta")]
>>
>># turn dummy variables into factors
>>
>>dat$wic <- factor(dat$wic)
>>dat$feedback <- factor(dat$feedback)
>>dat$info  <- factor(dat$info)
>>dat$pers  <- factor(dat$pers)
>>dat$imag  <- factor(dat$imag)
>>dat$meta  <- factor(dat$meta)
>>
>># set up predictor matrix for imputations
>>
>>predMatrix <- make.predictorMatrix(dat)
>>predMatrix[,"id"] <- 0 # don't use effect size id for imputing
>>predMatrix[,"author"] <- 0 # don't use author for imputing
>>predMatrix[,"vi"] <- 0 # don't use vi for imputing
>>predMatrix["id",] <- 0 # don't impute id (no NA's)
>>predMatrix["author",] <- 0 # don't impute author (no NA's)
>>predMatrix["yi",] <- 0 # don't impute yi (no NA's)
>>predMatrix["vi",] <- 0 # don't impute vi (no NA's)
>>predMatrix
>>
>># specify imputation method
>>
>>impMethod <- make.method(dat)
>>impMethod
>>
>># generate multiple imputations
>>
>>imp <- mice(dat, print=FALSE, m=20, predictorMatrix=predMatrix,
>method=impMethod,
>>seed=1234)
>>
>># fit model of interest to each of the 20 imputed datasets
>>
>>fit <- with(imp,
>>            rma.mv(yi = yi,
>>                   V = vi,
>>                   slab = author,
>>                   random = ~ 1 | author/id,
>>                   test = "t",
>>                   method = "REML",
>>                   mods = ~ length + wic + feedback + info + pers + imag +
>meta))
>>
>># pool and round results
>>
>>pool <- summary(pool(fit))
>>pool[-1] <- round(pool[-1], digits=4)
>>pool
>>
>># view output from first imputed dataset
>>
>>fit$analyses[1]
>>
>>From: Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
>>Sent: Saturday, 16 October 2021 12:18 AM
>>To: Thomas Swanton <thomas.swanton using sydney.edu.au>; r-sig-meta-analysis using r-
>>project.org <r-sig-meta-analysis using r-project.org>
>>Subject: RE: Extracting pooled rma.mv model summary statistics after multiple
>>imputation
>>
>>Hi Tom,
>>
>>Could you provide a minimal and fully reproducible example (see:
>https://protect-
>>au.mimecast.com/s/-pXHCnx1jni7Z8PPPu9Oavy?domain=stat.ethz.ch) that can be used
>>to discuss this? Right now, it's not entirely clear to me where exactly you are
>>stuck.
>>
>>Best,
>>Wolfgang
>>
>>>-----Original Message-----
>>>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>>>Behalf Of Thomas Swanton
>>>Sent: Friday, 15 October, 2021 2:02
>>>To: r-sig-meta-analysis using r-project.org
>>>Subject: [R-meta] Extracting pooled rma.mv model summary statistics after
>>>multiple imputation
>>>
>>>Dear colleagues,
>>>
>>>Is anyone able to help me understand how to extract pooled rma.mv model summary
>>>statistics after applying multiple imputation using mice please?
>>>
>>>I am conducting a three-level mixed effects multiple meta-regression using the
>>>rma.mv function in metafor, and have applied multiple imputation using mice
>>>following the instructions available here: https://www.metafor-
>>>project.org/doku.php/tips:multiple_imputation_with_mice_and_metafor
>>>
>>>I have the table of pooled model coefficients as in the worked example, but I
>>>haven't been able to work out how to extract pooled model summary statistics
>>>(e.g., tau-squared, Q statistic).
>>>
>>>Are you able to help me understand how to pool them across all the imputed
>>>datasets please?
>>>
>>>Many thanks,
>>>Tom
>>>
>>>Thomas Swanton, PhD Candidate
>>>School of Psychology, The University of Sydney, Australia
>>>thomas.swanton using sydney.edu.au
>>>
>>>Tom Swanton | PhD Candidate and Research
>>> Assistant
>>>
>>>The University of Sydney
>>>
>>>Faculty of Science, School of Psychology, Brain & Mind Centre, Gambling
>>Treatment
>>>& Research Clinic
>>>
>>>Rm 204, Level 2, 94 Mallett St, Camperdown | NSW | 2050
>>>
>>>+61 432 891 501
>>>
>>>thomas.swanton using sydney.edu.au | sydney.edu.au
>>>
>>>Twitter: @TomSwanton |
>>> ResearchGate: researchgate.net/profile/Thomas-Swanton
>>>
>>>Editorial Assistant, International Gambling Studies
>>>
>>>We acknowledge the tradition of custodianship and law of the Country on which
>>the
>>>University
>>> of Sydney campuses stand. We pay our respects to those who have cared and
>>>continue to care for Country.