[R-meta] GLMM vs. Inverse variance weighting for proportions (and comparison with stata)

[James Pustejovsky]

Hi Rachel,

I'm not familiar with metapreg so can't comment on it specifically. I'll
just offer that there are several different techniques for arriving at
parameter estimates in GLMMs. It could be that the differences that you're
seeing stem from using different estimation techniques. For example,
lme4::glmer() uses Laplace approximation by default, but you can turn on
more accurate approximations by setting the nAGQ parameter to something
bigger than 1. The meta::metaprop() function relies on metafor::rma.glmm(),
which defaults to using adaptive Gaussian quadrature with 7 quadrature
points. And brms does Hamiltonian Monte Carlo simulation to sample from the
posterior. From skimming through the article you linked, I could not tell
what estimation technique the metapreg package implements. It might be
something other than Laplace approximation or adaptive Gaussian
quadrature---maybe penalized quasi-likelihood?

Note that if you switch to calling metafor::rma.glmm() directly, you can
access some additional estimation options See the "Note" section in the
metafor documentation (
https://wviechtb.github.io/metafor/reference/rma.glmm.html#note-1).

James

On Fri, Dec 6, 2024 at 11:41 AM Rachael Burke via R-sig-meta-analysis <
r-sig-meta-analysis using r-project.org> wrote:

> Dear colleagues,
>
> I’m sorry if this question is partly (or mostly) about an entirely
> different package from ‘meta’.  But I am having a meta-analysis problem and
> I wondering if anyone could point me in the right direction.
>
> I am doing a meta-analysis of a single proportion, pooling studies (no
> comparisons).  I have used metaprop with a GLMM approach – all good, it
> makes sense to me.  I get very similar point estimate results to if I code
> up in lme4 or brms myself – all good.
>
> A colleague has looked in stata, and using defaults in metaprop user
> written package has got a VERY different answer (as in, 14% pooled estimate
> compared to 19% pooled estimate for our actual data).  Stata / metaprop
> uses inverse variance weighting and I can more-or-less recreate this
> estimate in R using meta by changing “method” to “inverse”.  Reasonably
> good, so far.
>
> I wanted to recreate the GLMM approach in stata to show the difference.  I
> used metapreg in stata, which has lovely documentation publication (
> https://archpublichealth.biomedcentral.com/articles/10.1186/s13690-023-01215-y).
> But that gives a very different answer to using a GLMM in meta in R.  I
> haven’t really used stata in a decade
>
> I think it’s probably not fair to ask this listserv to comment on someone
> else’s stata package.  And I know I probably just need to figure it out on
> my own whether GLMM vs. inverse variance approach is preferred.
>
> But I am nervous that I am getting such very different pooled estimates
> and I was wondering if any could help shed light on what I should be doing
> (particularly on the R vs. stata differences for a GLMM based approach)?
>
> Thank you for any pointers.
>
> Best wishes,
>
> Rachael
>
> An example:
> CODE AND RESULTS IN R
> library(tidyverse)
> library(meta)
>
> df <- tibble(author=c("alice","bob","chris","dawn","ewan"),
> npeople=c(400,30,3000,250,50), nevent=c(80,3,267,90,19)) # might need to
> put quotation marks back around authors, listserv doesn’t allow quotations
> marks?
>
> metaprop(data=df,
>          n=npeople,
>          event=nevents,
>          random=TRUE,
>          common=FALSE,
>          method = "GLMM")
>
> This gives: 0.2022 [0.1137; 0.3337]
>
> metaprop(data=df,
>          n=npeople,
>          event=nevent,
>          random=TRUE,
>          common=FALSE,
>          method = "Inverse",
>          sm="PFT")
>
> Gives: 0.2132 [0.1050; 0.3459] # a percentage point different!
>
> CODE AND RESULTS IN STATA
>
> # Need to recreate the toy data for four studies above
> # ssc install metaprop
> # ssc intall metapreg
>
> metaprop nevent npeople, random dp(3)
> Gives: 0.221       0.108        0.334   # similar to R meta with inverse
> variance method
>
> metapreg nevent npeople, model(random) studyid(name) dp(3) # this should
> use a GLMM approach
> Gives: 0.225     0.114          0.426n # similar to inverse variance AND
> DIFFERENT TO R META GLMM APPROACH
>
>
>
>
>
>
>
>
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-meta-analysis mailing list @ R-sig-meta-analysis using r-project.org
> To manage your subscription to this mailing list, go to:
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>

	[[alternative HTML version deleted]]