[R-meta] Three-level meta-analysis of response ratios when there is more than one "control group"

[Viechtbauer, Wolfgang (SP)]

Hi Gabriele,

You should compute the covariances as described in Lajeunesse (2011). See eq (8). For these toy data, this would be:

dat <- read.csv("dat_GM.csv")

calc.v <- function(x) {
   v <- matrix(x$SD_C[1]^2 / (x$N_C[1] * x$X_C[1]^2), nrow=nrow(x), ncol=nrow(x))
   diag(v) <- x$vi
   v
}

V <- bldiag(lapply(split(dat, dat$commonControl_ID), calc.v))
V

V is the var-cov matrix of the sampling errors. When using rma.mv(), this is what you should give as the second argument.

You may also want to take a look at:

http://www.metafor-project.org/doku.php/analyses:gleser2009

So, in your case:

rma.mv(yi, V, random = ~ 1 | Study/commonControl_ID/effect_ID, data=dat)

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-
>project.org] On Behalf Of Gabriele Midolo
>Sent: Wednesday, 11 April, 2018 23:20
>To: James Pustejovsky
>Cc: r-sig-meta-analysis at r-project.org
>Subject: Re: [R-meta] Three-level meta-analysis of response ratios when
>there is more than one "control group"
>
>ATTACHMENT(S) REMOVED: dat_GM.csv
>
>Hi James (all),
>Thanks again for your exhaustive answer.
>Sorry if I insist, but I still did not understand why multi-level
>modelling via rma.mv is not a good option to deal with non-independence
>in my case (or maybe I missed something from your email?). I had a read
>to Lajeunesse (2011) and Lajeunesse (2016) [here, he describes a
>practical example in R on to build variance covariance matrix when you
>have multiple treatments and one control and use them with metafor].
>However, I saw e.g. in Van den Noortgate et al (2013) that such issue can
>be dealt with multiple-level meta-analytic approaches too (?), or at
>least that is my interpretation.
>
>So, based on Lajeunesse's example, I have simulated data that could
>potentially reflect the data of my meta-analysis (I took the liberty to
>attach these, hope it is not an issue with the rules of r-sig-meta-
>analysis mailing list).
>Here I have three studies that report treatment and control change (X_T
>and X_C, respectively). The yi and vi were calculated with the "ROM"
>option in escalc().
>If I had a case where yi of each study shares the same (only one)
>control, I would fit the model with following:
>
>"res1<-rma.mv(yi,vi,data=dat,random=~ 1 | Study/effect_ID)"
>However, I should account for the fact that I have multiple control group
>for each yi in each study, so wouldn't be correct to add a third level
>"commonControl_ID"? i.e.:
>
>"res2<-rma.mv(yi,vi,data=dat,random=~ 1 |
>Study/commonControl_ID/effect_ID)"
>Which should be identical to:
>
>"res3<-rma.mv(yi,vi,data=dat,random=list(~ 1 | effect_ID, ~ 1 |
>commonControl_ID, ~ 1 | Study))"
>(... at least, the model output based on my data example looks like they
>are). Or maybe all of what I have put above does not make sense and I am
>misunderstanding the usage of multi-level modeling?
>Thanks for your patience,
>Gabriele
>
>On 10 April 2018 at 18:38, James Pustejovsky <jepusto at gmail.com> wrote:
>Gabriele,
>
>I see what you mean. Your explanation suggests that there would be high
>variance in the outcome from study to study, due to examining different
>species or other aspects of the study's design. I don't think that this
>precludes using the raw means as effect sizes---it just means that there
>would be a large between-study variance component. I think this approach
>would still make it easier to model how the ES might depend on covariates
>that have variation within a given study (such as altitude level).
>
>The difficulty with using the ROM effect size is that when you compare
>multiple conditions to a single control condition, there is correlation
>in the effect size estimates. In the notation of your original email, you
>would have
>cov(yi1, yi2) != 0
>cov(yi1, yi3) != 0
>cov(yi2, yi3) != 0
>In fact, there would be very strong dependence because yi3 = yi2 - yi1.
>So if you use this approach, it would be critical to account for the
>sampling covariance between the effect sizes. Lajeunesse (2011) gives
>formulas for the covariances that you'd need.
>
>James
>
>Lajeunesse, M. J. (2011). On the meta-analysis of resposne ratios for
>studies with correlated and multi-design groups. Ecology, 92(11), 2049–
>2055.
>
>On Tue, Apr 10, 2018 at 10:22 AM, Gabriele Midolo
><gabriele.midolo at gmail.com> wrote:
>Dear James (all),
>Thanks for your ideas, very appreciated.
>I understand what you mean, but SLA can be calculated on different
>species which are likely to report very different magnitude depending on
>the species. Plus I think the ratios calculated with "ROM" should provide
>a standardized measure of how much a single species' SLA changes in
>response to altitudinal shift, that is what I think might be interesting
>to measure in the context of my study.
>I am not unkeen to use your option of course, I am just wondering if a
>three-level meta-analysis could deal with a type of independency
>described above, or it can work with data that share multiple treatment
>"doses" compated to a single control...
>Gabriele
>
>On 10 April 2018 at 16:21, James Pustejovsky <jepusto at gmail.com> wrote:
>Gabriele,
>
>If all of the studies use SLA as the outcome measure, would it make sense
>to model the mean levels directly (instead of calculating proportionate
>changes)? In other words, use the mean SLA level as the effect size. If
>it makes sense conceptually, I think this would be a good way to go
>because then the effect size estimates at each altitude level would be
>independent of each other, conditional on the true mean levels for that
>study, and a three-level meta-analysis model would be a natural fit.
>
>James
>
>On Sun, Apr 8, 2018 at 11:12 AM, Gabriele Midolo
><gabriele.midolo at gmail.com> wrote:
>Dear all,
>
>I have a question that is more methodological but somehow related to
>metafor.
>I want to conduct an (ecological) meta-analysis on specific leaf area
>(SLA) response to increased altitdue (i.e. elevation) in mountain
>ecosystems. Primary studies selected report the mean (+ SE and sample
>size) of SLA sampled at different altitudinal levels. The picture
>attached is an example of how row primary data are normally reported in
>the articles (modified, from Seguí et al 2018, fig.1c
>[https://doi.org/10.1007/s00035-017-0195-9]).
>The A, B and C (in red) values represents the mean values of SLA
>calculated at 1900, 2200 and 2350 m above the sea level (i.e. altitude)
>that should, in my opinion, be suitable for calculating log-transformed
>response ratios (RR) indicating how much SLA increases/decreases compared
>to a population of plants sampled to a lower altitiude. Thus, given the
>design of such studies, I propose that multiple RR (yi) must be
>calulcated within each study as follows:
>
>yi1= ln(B/A)
>yi2=ln(C/A)
>yi3=ln(C/B)
>...
>if a D value would have been reported by the authors, sampled to a higher
>altitdue than 2350 m, then I woul also calculate yi4=ln(D/A),
>yi5=ln(D/B), yi6=ln(D/C) for this study.
>
>This approach make sense to me because there is no "proper" control and
>treatment and you are not just interested to estimate SLA changes by
>comparing mean values reported at higher altitudes with only the one
>sampled at the lowest altitudinal level (yi1,yi2), but also between higer
>altitudinal levels (yi3). This is also supposed to allow to look in meta-
>regession how the altitudinal shift (so, the difference in altitudes e.g.
>300m for yi1) affect the effect size responses. So - and here finally
>comes my question - with rma.mv I should be able to safely account for
>non-independence by fitting a model with the "random =~1|Experiment/ID"
>structure (?). Is this type of data suitable for three-level mixed-effect
>meta-analysis? I used already this structure in a previous meta-analysis
>I conducted in the past, but back then I was working with multiple
>treatments compared to just one single control in each study.
>I see some similar meta-analysis in the past have used the r-to-z
>transformed effect size and focused on the correlation - in my case -
>between altitude and SLA, but not sure this is what I would like to
>investigate in the first place...
>
>Hope I was clear, and my apologies if I was messy.
>
>Thanks a lot for reading this
>Gabriele