[R-meta] rma.glmm and relative risk

[Viechtbauer, Wolfgang (SP)]

Hi David,

I personally never quite understood the "OR vs RR" debate. They estimate different things, so whenever somebody points out that an OR is a biased estimate of the RR, my reaction is: No sh*t, Sherlock.

But if your colleagues really want RRs, then at least you can get those now also with the binomial model. Not sure what the best strategy is when there are convergence warnings. They might be innocuous or might indicate real problems. This is ultimately a glmer() question though, since this is what rma.glmm() uses to fit these models. A search for "Model failed to converge with max|grad|" leads to lots of discussions around this warning message.

Best,
Wolfgang

-----Original Message-----
From: David Hajage [mailto:dhajage using gmail.com] 
Sent: Thursday, 05 September, 2019 10:12
To: Viechtbauer, Wolfgang (SP)
Cc: r-sig-meta-analysis using r-project.org
Subject: Re: [R-meta] rma.glmm and relative risk

Dear Wolfgang, 

Thank you very much for this (so complete) answer. 

My request is explained by:
- the desire to compare the results obtained with the one-step and two-step methods
- the need to provide RR to a collaborator who finds the OR misleading

I fully understand the problems of convergence. What would you recommend in this situation?
- provide RR after verifying that there is no convergence issues? would this be acceptable, or subject to questioning at the time of publication?
- explain the possible convergence issues and convince the investigator to "accept" OR estimations?
Thank you again, 
Best, 
David

On Wed, Sep 4, 2019 at 11:59 PM Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
Hi David,

measure="RR" is not (currently) available in rma.glmm() because that would require using a log instead of the logit link and you are (much?) more likely to run into convergence problems when doing logistic mixed-effects models with a log link.

Just for the fun of it, I just pushed an update to the 'devel' version of metafor on GitHub (which you can install as described here: https://wviechtb.github.io/metafor/#installation) that adds an (undocumented) 'family' argument to rma.glmm() that allows you to change the link function (and in principle even the family, but that would be really asking for trouble). You still have to use measure="OR" -- which is a bit of a misnomer when switching to a log link -- but I didn't want to make extensive changes to rma.glmm() right now.

Let's try an example (where the outcome is not rare, because for rare outcomes, you won't see much of a difference between RRs and ORs). I'll start with the analysis using ORs:

### copy data into 'dat'
dat <- dat.linde2005

### remove studies with no response information and study with no responses in either group
dat <- dat[-c(5,6,26),]

res <- rma(measure="OR", ai=ai, ci=ci, n1i=n1i, n2i=n2i, data=dat)
res
predict(res, transf=exp)

res <- rma.glmm(measure="OR", ai=ai, ci=ci, n1i=n1i, n2i=n2i, data=dat, model="UM.RS")
res
predict(res, transf=exp)

So, using a 'normal-normal' (two-step) model and using a logistic random-effects model gives very similar results here. Okay, now let's use a 'normal-normal' model with RRs:

res <- rma(measure="RR", ai=ai, ci=ci, n1i=n1i, n2i=n2i, data=dat)
res
predict(res, transf=exp)

And now you can do:

res <- rma.glmm(measure="OR", ai=ai, ci=ci, n1i=n1i, n2i=n2i, data=dat, model="UM.RS", family=binomial(link="log"))
res
predict(res, transf=exp)

Two things to note: The results are indeed close again to those from the normal-normal model. But you also get a convergence warning (it does converge, but with some troubles).

So, feel free to play around with that. I don't know if I will leave that option in (and I'll have to go through all functions that operate on 'rma.glmm' model objects to check that nothing else breaks as a result). Use at your own risk.

In principle, one could now even move to a risk difference scale, by using an identity link:

rma(measure="RD", ai=ai, ci=ci, n1i=n1i, n2i=n2i, data=dat)

But this is even more difficult to fit and fails spectacularly here:

rma.glmm(measure="OR", ai=ai, ci=ci, n1i=n1i, n2i=n2i, data=dat, model="UM.RS", family=binomial(link="identity"))

For our beloved BCG dataset, this does work, but I had to switch to the UM.FS model:

dat <- dat.bcg
rma(measure="RD", ai=tpos, bi=tneg, ci=cpos, di=cneg, data=dat)
rma.glmm(measure="OR", ai=tpos, bi=tneg, ci=cpos, di=cneg, data=dat, model="UM.FS", family=binomial(link="identity"))

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On Behalf Of David Hajage
Sent: Wednesday, 04 September, 2019 18:29
To: r-sig-meta-analysis using r-project.org
Subject: [R-meta] rma.glmm and relative risk

Dear all,

First, I would like to thank Mr Viechtbauer and all the contributors of the
metafor package: it is very useful and convenient piece of software.

I have a little question regarding the function ‘rma.glmm’. I have a binary
outcome, and applied the following one-step model:
> rma.glmm(measure = "OR",
+        ai = exp_event, bi = exp_noevent, ci = con_event, di = con_noevent,
+        data = ma, model="UM.RS")

I would prefer to express the results in terms of relative risk. This seems
possible when applying a two-step approach (because measure = “RR” is an
option of the function ‘escalc’), but it doesn’t seem possible with
rma.glmm.
Is there a reason for that? And is there a workaround within your package?
(I know that I can directly use glmer, but I would really prefer staying in
the workflow of metafor package).

Thank you very much for you help,

Best wishes,
-- 
David