[R-meta] Three-level meta-analysis of response ratios when there is more than one "control group"

[Gabriele Midolo]

Hi James (all),

Thanks again for your exhaustive answer.
Sorry if I insist, but I still did not understand why multi-level modelling
via rma.mv is not a good option to deal with non-independence in my case
(or maybe I missed something from your email?). I had a read to Lajeunesse
(2011) and Lajeunesse (2016) [here, he describes a practical example in R
on to build variance covariance matrix when you have multiple treatments
and one control and use them with metafor]. However, I saw e.g. in Van den
Noortgate et al (2013) that such issue can be dealt with multiple-level
meta-analytic approaches too (?), or at least that is my interpretation.

So, based on Lajeunesse's example, I have simulated data that could
potentially reflect the data of my meta-analysis (I took the liberty to
attach these, hope it is not an issue with the rules of r-sig-meta-analysis
mailing list).
Here I have three studies that report treatment and control change (X_T and
X_C, respectively). The yi and vi were calculated with the "ROM" option in
escalc().
If I had a case where yi of each study shares the same (only one) control,
I would fit the model with following:

"res1<-rma.mv(yi,vi,data=dat,random=~ 1 | Study/effect_ID)"

However, I should account for the fact that I have multiple control group
for each yi in each study, so wouldn't be correct to add a third level
"commonControl_ID"? i.e.:

"res2<-rma.mv(yi,vi,data=dat,random=~ 1 | Study/commonControl_ID/effect_ID)"

Which should be identical to:

"res3<-rma.mv(yi,vi,data=dat,random=list(~ 1 | effect_ID, ~ 1 |
commonControl_ID, ~ 1 | Study))"

(... at least, the model output based on my data example looks like they
are). Or maybe all of what I have put above does not make sense and I am
misunderstanding the usage of multi-level modeling?

Thanks for your patience,
Gabriele

On 10 April 2018 at 18:38, James Pustejovsky <jepusto at gmail.com> wrote:

> Gabriele,
>
> I see what you mean. Your explanation suggests that there would be high
> variance in the outcome from study to study, due to examining different
> species or other aspects of the study's design. I don't think that this
> precludes using the raw means as effect sizes---it just means that there
> would be a large between-study variance component. I think this approach
> would still make it easier to model how the ES might depend on covariates
> that have variation within a given study (such as altitude level).
>
> The difficulty with using the ROM effect size is that when you compare
> multiple conditions to a single control condition, there is correlation in
> the effect size estimates. In the notation of your original email, you
> would have
> cov(yi1, yi2) != 0
> cov(yi1, yi3) != 0
> cov(yi2, yi3) != 0
> In fact, there would be very strong dependence because yi3 = yi2 - yi1. So
> if you use this approach, it would be critical to account for the sampling
> covariance between the effect sizes. Lajeunesse (2011) gives formulas for
> the covariances that you'd need.
>
> James
>
> Lajeunesse, M. J. (2011). On the meta-analysis of resposne ratios for
> studies with correlated and multi-design groups. Ecology, 92(11), 2049–2055.
>
> On Tue, Apr 10, 2018 at 10:22 AM, Gabriele Midolo <
> gabriele.midolo at gmail.com> wrote:
>
>> Dear James (all),
>>
>> Thanks for your ideas, very appreciated.
>> I understand what you mean, but SLA can be calculated on different
>> species which are likely to report very different magnitude depending on
>> the species. Plus I think the ratios calculated with "ROM" should provide a
>> standardized measure of how much a single species' SLA changes in response
>> to altitudinal shift, that is what I think might be interesting to measure
>> in the context of my study.
>> I am not unkeen to use your option of course, I am just wondering if a
>> three-level meta-analysis could deal with a type of independency described
>> above, or it can work with data that share multiple treatment "doses"
>> compated to a single control...
>>
>> Gabriele
>>
>> On 10 April 2018 at 16:21, James Pustejovsky <jepusto at gmail.com> wrote:
>>
>>> Gabriele,
>>>
>>> If all of the studies use SLA as the outcome measure, would it make
>>> sense to model the mean levels directly (instead of calculating
>>> proportionate changes)? In other words, use the mean SLA level as the
>>> effect size. If it makes sense conceptually, I think this would be a good
>>> way to go because then the effect size estimates at each altitude level
>>> would be independent of each other, conditional on the true mean levels for
>>> that study, and a three-level meta-analysis model would be a natural fit.
>>>
>>> James
>>>
>>> On Sun, Apr 8, 2018 at 11:12 AM, Gabriele Midolo <
>>> gabriele.midolo at gmail.com> wrote:
>>>
>>>> Dear all,
>>>>
>>>> I have a question that is more methodological but somehow related to
>>>> metafor.
>>>> I want to conduct an (ecological) meta-analysis on specific leaf area
>>>> (SLA) response to increased altitdue (i.e. elevation) in mountain
>>>> ecosystems. Primary studies selected report the mean (+ SE and sample size)
>>>> of SLA sampled at different altitudinal levels. The picture attached is an
>>>> example of how row primary data are normally reported in the articles
>>>> (modified, from Seguí et al 2018, fig.1c [https://doi.org/10.1007/s0003
>>>> 5-017-0195-9]).
>>>> The A, B and C (in red) values represents the mean values of SLA
>>>> calculated at 1900, 2200 and 2350 m above the sea level (i.e. altitude)
>>>> that should, in my opinion, be suitable for calculating log-transformed
>>>> response ratios (RR) indicating how much SLA increases/decreases compared
>>>> to a population of plants sampled to a lower altitiude. Thus, given the
>>>> design of such studies, I propose that multiple RR (yi) must be calulcated
>>>> within each study as follows:
>>>>
>>>> yi1= ln(B/A)
>>>> yi2=ln(C/A)
>>>> yi3=ln(C/B)
>>>> ...
>>>> if a D value would have been reported by the authors, sampled to a
>>>> higher altitdue than 2350 m, then I woul also calculate yi4=ln(D/A),
>>>> yi5=ln(D/B), yi6=ln(D/C) for this study.
>>>>
>>>> This approach make sense to me because there is no "proper" control and
>>>> treatment and you are not just interested to estimate SLA changes by
>>>> comparing mean values reported at higher altitudes with only the one
>>>> sampled at the lowest altitudinal level (yi1,yi2), but also between higer
>>>> altitudinal levels (yi3). This is also supposed to allow to look in
>>>> meta-regession how the altitudinal shift (so, the difference in altitudes
>>>> e.g. 300m for yi1) affect the effect size responses. So - and here finally
>>>> comes my question - with rma.mv I should be able to safely account for
>>>> non-independence by fitting a model with the "random =~1|Experiment/ID"
>>>> structure (?). Is this type of data suitable for three-level mixed-effect
>>>> meta-analysis? I used already this structure in a previous meta-analysis I
>>>> conducted in the past, but back then I was working with multiple treatments
>>>> compared to just one single control in each study.
>>>> I see some similar meta-analysis in the past have used the r-to-z
>>>> transformed effect size and focused on the correlation - in my case -
>>>> between altitude and SLA, but not sure this is what I would like to
>>>> investigate in the first place...
>>>>
>>>> Hope I was clear, and my apologies if I was messy.
>>>>
>>>> Thanks a lot for reading this
>>>> Gabriele
>>>>
>>>>
>>>> ​
>>>>
>>>> _______________________________________________
>>>> R-sig-meta-analysis mailing list
>>>> R-sig-meta-analysis at r-project.org
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>>>
>>>>
>>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20180411/e4b617ba/attachment-0001.html>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: =?UTF-8?B?U2VndcOtX2V0YWxfMjAxOF9leGFtcGxlLlBORw==?=
Type: image/png
Size: 35051 bytes
Desc: not available
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20180411/e4b617ba/attachment-0001.png>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: dat_GM.csv
Type: application/vnd.ms-excel
Size: 1667 bytes
Desc: not available
URL: <https://stat.ethz.ch/pipermail/r-sig-meta-analysis/attachments/20180411/e4b617ba/attachment-0001.xlb>