[R-meta] robust variance estimator in meta-analyses of rare events (proportions)

[Pier-Alexandre Tardif]

Hello again,

Thanks for the follow up. I have compared the two methods (inverse variance with double arcsine vs mixed-effects logistic) and I still have one question.

1) Summary proportions obtained with either model are really low and relatively similar:
1st model (inverse variance, double arcsine): 0.01306 [95% CI: 0.00561-0.02271]
2nd model (mixed-effects logistic model): 0.0167 [95% CI: 0.0099-0.0281])

In terms of 'presentation', would it make sense to rescale these proportions and how could we then interpret them?

Thanks,

Pier-Alex


-----Message d'origine-----
De : Viechtbauer, Wolfgang (SP) [mailto:wolfgang.viechtbauer using maastrichtuniversity.nl] 
Envoyé : 11 octobre 2018 05:36
À : Pier-Alexandre Tardif <pier-alexandre.tardif.1 using ulaval.ca>; r-sig-meta-analysis using r-project.org
Objet : RE: [R-meta] robust variance estimator in meta-analyses of rare events (proportions)

A small qualification: With meta/metafor, you can do the mixed-effects logistic model approach using ML estimation. For a fully Bayesian approach, you have to check out other packages (unless you want to use JAGS/Bugs/Stan/etc directly).

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On Behalf Of Viechtbauer, Wolfgang (SP)
Sent: Thursday, 11 October, 2018 11:22
To: pier-alexandre.tardif.1 using ulaval.ca; r-sig-meta-analysis using r-project.org
Subject: Re: [R-meta] robust variance estimator in meta-analyses of rare events (proportions)

Just as another thought here:

For proportions with rare events, I would say these are sensible options:

- inverse variance methods but using a variance-stabilized outcome (i.e., the arcsine square root transformed proportion or the Freeman-Tukey double arcsine transformed proportion)

- mixed-effects logistic models (either estimated using ML or in a fully Bayesian manner)

- beta-binomial models

The first two you can easily do with meta/metafor. For the latter, you can use VGAM or aod3.

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On Behalf Of Röver, Christian
Sent: Tuesday, 09 October, 2018 13:33
To: pier-alexandre.tardif.1 using ulaval.ca; r-sig-meta-analysis using r-project.org
Subject: Re: [R-meta] robust variance estimator in meta-analyses of rare events (proportions)

Hi Pier-Alex,

sorry for the confusion -- I thought this was about ratios of probabilities or odds. I am afraid in case of the proportions themselves, the practical implementation is not quite as simple. On the other hand, a Bayesian approach would still have the advantage that it does not require continuity corrections (or other approximations), while on the other hand it requires (a) spending some thought on reasonable prior distributions and (b) an implementation probably in BUGS/JAGS or Stan.

Cheers,

Christian

On Mon, 2018-10-01 at 15:33 +0000, Pier-Alexandre Tardif wrote:
> Hello Christian,
> 
> Thanks for your reply and suggestions. I just went over the two papers 
> and the detailed instructions of the metaStan package and I have two 
> quick questions:
> 
> Arguments required to conduct each stage of the meta-analysis are 
> «ntrl, nctrl, rtrt, rctrl». However, I do not aim to estimate a 
> 'treatment effect' but only a 'summary proportion'. How can I then 
> adapt this for a meta-analysis of proportions (no comparison groups)?
> And do you think the «priors» you came up with in the aforementioned 
> papers are reasonable choices to conduct this kind of meta-analysis?
> 
> Best,
> 
> Pier-Alex
> 
> -----Message d'origine-----
> De : Röver, Christian [mailto:christian.roever using med.uni-goettingen.de]
>  
> Envoyé : 29 septembre 2018 07:03
> À : Pier-Alexandre Tardif <pier-alexandre.tardif.1 using ulaval.ca>;
> r-sig-meta-analysis using r-project.org
> Objet : Re: [R-meta] robust variance estimator in meta-analyses of 
> rare events (proportions)
> 
> Hi Pier-Alex,
> 
> since exact Likelihood-based methods avoiding continuity corrections 
> and approximations often run into numerical problems with zero event- 
> counts (see e.g. https://arxiv.org/abs/1807.09037), another option may 
> be to switch to a Bayesian approach. There is software available (also 
> implementing "exact" binomial likelihoods instead of normal 
> approximations), and priors distributions may also be readily 
> motivated; see e.g. here: https://arxiv.org/abs/1809.04407.
> 
> Cheers,
> 
> Christian Röver
> 
> On Fri, 2018-09-28 at 13:48 +0000, Pier-Alexandre Tardif wrote:
> > Dear all,
> > 
> > I'm currently conducting a meta-analysis of proportions of rare 
> > events (including studies with zero event). Based on previous 
> > literature (see postscript below), it is generally suggested to 
> > avoid the following:
> > [1] continuity corrections;
> > [2] non-exact distributions;
> > [3] inverse variance methods to estimate the weights; [4] 
> > fixed-effects models; [5] not handling the correlation between the 
> > mean & variance.
> > 
> > Given the available packages in R (as far as I know), not all could 
> > be avoided ([3] remains). The following code is twofold. The lines 
> > preceded with  ...  are recent additions made by Dr Pustejovksy 
> > based on this vignette:
> > 
https://cran.r-project.org/web/packages/clubSandwich/vignettes/meta-analysis-with-CRVE.html
> > . Lines without  ...  are my previous 'standard' meta-analysis of 
> > proportions. Since there are a lot of steps following estimating the 
> > summary effect sizes + SE (e.g. assessing outliers, heterogeneity, 
> > small-study effect, etc.), I would have like to be able to use 
> > existing options to deal with these further steps. So here's my
> > question: how could I obtain my back transformed effect size + 
> > confidence intervals after having used the robust variance estimator 
> > (coef_test)?
> > 
> > ies.da=escalc(xi=cases, ni=total, data=dat, measure="PFT",
> > add=0)  #Solutions [1] and [5] by using a double-arcsine 
> > transformation without continuity correction  ...  ies.da$ES_id <- 
> > 1:nrow(ies.da) #Creates a unique ID for each effect size 
> > pes.da=rma(yi, vi, data=ies.da, method="REML", level=95) #Solutions 
> > [4]  ...  coef_test(pes.da, vcov = "CR2", cluster = ies.da$ES_id) 
> > #Sandwich variance estimator is robust to model misspecification 
> > (solutions [2]).
> > pes=predict(pes.da, transf=transf.ipft.hm, targ=list(ni=dat$total)) 
> > #Backtransformation of the Freeman-Tukey double arcsine using the 
> > harmonic mean print(pes, digits=5)
> > 
> > Thanks a lot,
> > Pier-Alex
> > 
> > P.S.
> > Stijnen (2010) Random effects meta-analysis of event outcome in the 
> > framework of the generalized linear mixed model with applications in 
> > sparse data B hning (2010) Meta-analysis of clinical trials with 
> > rare events Trikalinos (2013) Simulation-based comparison of methods 
> > for meta- analysis of proportions and rates Kuss (2014) Statistical 
> > methods for meta-analysis including information from studies without 
> > any events-add nothing to nothing and succeed nevertheless Ma (2016) 
> > Meta-analysis of proportions of rare events-a comparison of exact 
> > likelihood methods with robust variance estimation
> > 
> > Also in the Cochrane Handbook:  Methods that should be avoided with 
> > the rare events are the inverse-variance methods (including the 
> > DerSimonian and Laird random-effects method) .