[R-meta] robust variance estimator in meta-analyses of rare events (proportions)

[Viechtbauer, Wolfgang (SP)]

A small qualification: With meta/metafor, you can do the mixed-effects logistic model approach using ML estimation. For a fully Bayesian approach, you have to check out other packages (unless you want to use JAGS/Bugs/Stan/etc directly).

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On Behalf Of Viechtbauer, Wolfgang (SP)
Sent: Thursday, 11 October, 2018 11:22
To: pier-alexandre.tardif.1 using ulaval.ca; r-sig-meta-analysis using r-project.org
Subject: Re: [R-meta] robust variance estimator in meta-analyses of rare events (proportions)

Just as another thought here:

For proportions with rare events, I would say these are sensible options:

- inverse variance methods but using a variance-stabilized outcome (i.e., the arcsine square root transformed proportion or the Freeman-Tukey double arcsine transformed proportion)

- mixed-effects logistic models (either estimated using ML or in a fully Bayesian manner)

- beta-binomial models

The first two you can easily do with meta/metafor. For the latter, you can use VGAM or aod3.

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On Behalf Of Röver, Christian
Sent: Tuesday, 09 October, 2018 13:33
To: pier-alexandre.tardif.1 using ulaval.ca; r-sig-meta-analysis using r-project.org
Subject: Re: [R-meta] robust variance estimator in meta-analyses of rare events (proportions)

Hi Pier-Alex,

sorry for the confusion -- I thought this was about ratios of
probabilities or odds. I am afraid in case of the proportions
themselves, the practical implementation is not quite as simple. On the
other hand, a Bayesian approach would still have the advantage that it
does not require continuity corrections (or other approximations),
while on the other hand it requires (a) spending some thought on
reasonable prior distributions and (b) an implementation probably in
BUGS/JAGS or Stan.

Cheers,

Christian

On Mon, 2018-10-01 at 15:33 +0000, Pier-Alexandre Tardif wrote:
> Hello Christian,
> 
> Thanks for your reply and suggestions. I just went over the two
> papers and the detailed instructions of the metaStan package and I
> have two quick questions:
> 
> Arguments required to conduct each stage of the meta-analysis are
> «ntrl, nctrl, rtrt, rctrl». However, I do not aim to estimate a
> 'treatment effect' but only a 'summary proportion'. How can I then
> adapt this for a meta-analysis of proportions (no comparison groups)?
> And do you think the «priors» you came up with in the aforementioned
> papers are reasonable choices to conduct this kind of meta-analysis?
> 
> Best,
> 
> Pier-Alex
> 
> -----Message d'origine-----
> De : Röver, Christian [mailto:christian.roever using med.uni-goettingen.de]
>  
> Envoyé : 29 septembre 2018 07:03
> À : Pier-Alexandre Tardif <pier-alexandre.tardif.1 using ulaval.ca>; 
> r-sig-meta-analysis using r-project.org
> Objet : Re: [R-meta] robust variance estimator in meta-analyses of
> rare events (proportions)
> 
> Hi Pier-Alex,
> 
> since exact Likelihood-based methods avoiding continuity corrections
> and approximations often run into numerical problems with zero event-
> counts (see e.g. https://arxiv.org/abs/1807.09037), another option
> may
> be to switch to a Bayesian approach. There is software available
> (also
> implementing "exact" binomial likelihoods instead of normal
> approximations), and priors distributions may also be readily
> motivated; see e.g. here: https://arxiv.org/abs/1809.04407.
> 
> Cheers,
> 
> Christian Röver
> 
> On Fri, 2018-09-28 at 13:48 +0000, Pier-Alexandre Tardif wrote:
> > Dear all,
> > 
> > I'm currently conducting a meta-analysis of proportions of rare
> > events (including studies with zero event). Based on previous
> > literature (see postscript below), it is generally suggested to
> > avoid
> > the following:
> > [1] continuity corrections;
> > [2] non-exact distributions;
> > [3] inverse variance methods to estimate the weights;
> > [4] fixed-effects models;
> > [5] not handling the correlation between the mean & variance.
> > 
> > Given the available packages in R (as far as I know), not all could
> > be avoided ([3] remains). The following code is twofold. The lines
> > preceded with �...� are recent additions made by Dr Pustejovksy
> > based
> > on this vignette: 
> > 
https://cran.r-project.org/web/packages/clubSandwich/vignettes/meta-analysis-with-CRVE.html
> > . Lines without �...� are my previous 'standard' meta-analysis of
> > proportions. Since there are a lot of steps following estimating
> > the
> > summary effect sizes + SE (e.g. assessing outliers, heterogeneity,
> > small-study effect, etc.), I would have like to be able to use
> > existing options to deal with these further steps. So here's my
> > question: how could I obtain my back transformed effect size +
> > confidence intervals after having used the robust variance
> > estimator
> > (coef_test)?
> > 
> > ies.da=escalc(xi=cases, ni=total, data=dat, measure="PFT",
> > add=0)  #Solutions [1] and [5] by using a double-arcsine
> > transformation without continuity correction
> > �...� ies.da$ES_id <- 1:nrow(ies.da) #Creates a unique ID for each
> > effect size
> > pes.da=rma(yi, vi, data=ies.da, method="REML", level=95) #Solutions
> > [4]
> > �...� coef_test(pes.da, vcov = "CR2", cluster = ies.da$ES_id)
> > #Sandwich variance estimator is robust to model misspecification
> > (solutions [2]).
> > pes=predict(pes.da, transf=transf.ipft.hm, targ=list(ni=dat$total))
> > #Backtransformation of the Freeman-Tukey double arcsine using the
> > harmonic mean
> > print(pes, digits=5)
> > 
> > Thanks a lot,
> > Pier-Alex
> > 
> > P.S.
> > Stijnen (2010) Random effects meta-analysis of event outcome in the
> > framework of the generalized linear mixed model with applications
> > in
> > sparse data
> > B�hning (2010) Meta-analysis of clinical trials with rare events
> > Trikalinos (2013) Simulation-based comparison of methods for meta-
> > analysis of proportions and rates
> > Kuss (2014) Statistical methods for meta-analysis including
> > information from studies without any events-add nothing to nothing
> > and succeed nevertheless
> > Ma (2016) Meta-analysis of proportions of rare events-a comparison
> > of
> > exact likelihood methods with robust variance estimation
> > 
> > Also in the Cochrane Handbook: �Methods that should be avoided with
> > the rare events are the inverse-variance methods (including the
> > DerSimonian and Laird random-effects method)�.