<div dir="ltr"><div><div><div><div>Dear Wolfgang & James (all),<br><br></div>Thanks for the code Wolfgang, it works fine and it is indeed possible (at least in terms of coding and modeling) to fit the model accounting for "multiple controls" within each study (so, metafor can indeed handle this type of variance-covariance matrix, at least with my toy data).<br><br></div>I understand your concern James (and many thanks for raising these points by the way!). I was indeed thinking to use one "control" only too (e.g. the lowest mean values sampled at lower altitude) and I also think using raw means could also be interesting for my meta-analysis. However, I am not sure to which extent having dependent ES with the "ROM" approach could be an issue in my specific case, given that model is potentially accounting for these dependencies?... But I will be careful with this and give the priority to the "one-control" approach first. In sum, I wanted to have a meta-analysis of
ROM for each possible pair of altitudes because I think there is less information loss in this way (?). I think there is no ecological or methodological reason to treat one altitudinal level as a control and the other ones as treatments, since all these levels could be used as a base to compare plant traits response along an altitudinal gradient (or at least potentially?).<br><br></div>Cheers,<br></div>Gabriele<br></div><div class="gmail_extra"><br><div class="gmail_quote">On 12 April 2018 at 16:43, James Pustejovsky <span dir="ltr"><<a href="mailto:jepusto@gmail.com" target="_blank">jepusto@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Gabriele,<div><br></div><div>
<span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">If you are going to use the ROMs approach, </span>Wolfgang's point about using a block-diagonal covariance matrix is crucial. Accounting for the covariances between the ROM effect size estimates is necessary for the other parts of the model (especially the variance components) to make sense. </div><div><br></div><div>The Van den Noortgate et al. (2013) paper argues that it is not necessary to include these sample covariances if you instead include an additional random effect within the study. The premise is essentially that the additional random effect makes the model robust to mis-specification of the covariances. My concern with this argument is that, while it seems to be true for certain specific models, the robustness property might not hold generally, and short of doing a simulation study that mimics the conditions of your meta-analysis, there's not really any easy way to tell whether it holds. So better to go with the approach that includes the covariances--a properly specified model--so that the variance component estimates and standard errors and everything else will come out okay. </div><div><br></div><div>However, I think there still might be an issue with the ROM approach as you have described it. Specifically, if you calculate ROM for each possible pair of altitudes, then the set of ROM estimates will be perfectly dependent. Some of the ES estimates will be exact linear combinations of the other ES estimates. I'm not sure if metafor can even deal with the resulting variance-covariance matrices, and even if it can it seems like a tricky and complicated model to interpret. Two possible ways to simplify it would be: 1) pick an arbitrary "control" condition and compare all of the other conditions to it (e.g., always treat the highest altitude as the control, and all of the lower altitudes as treatment conditions) or 2) use raw means (or log-means) as effect sizes, as I suggested earlier, and which has the further benefit that the ES estimates are independent (so the van den Noortgate 3-level meta-analysis model would actually be perfectly well-specified here).</div><span class="HOEnZb"><font color="#888888"><div><br></div></font></span><div><span class="HOEnZb"><font color="#888888">James</font></span><div><div class="h5"><br><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Apr 11, 2018 at 4:20 PM, Gabriele Midolo <span dir="ltr"><<a href="mailto:gabriele.midolo@gmail.com" target="_blank">gabriele.midolo@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div>Hi James (all),<br><br></div>Thanks again for your exhaustive answer.<br></div><div>Sorry if I insist, but I still did not understand why multi-level modelling via <a href="http://rma.mv" target="_blank">rma.mv</a> is not a good option to deal with non-independence in my case (or maybe I missed something from your email?). I had a read to
Lajeunesse
(2011) and
Lajeunesse (2016) [here, he describes a practical example in R on to build variance covariance matrix when you have multiple treatments and one control and use them with metafor]. However, I saw e.g. in Van den Noortgate et al (2013) that such issue can be dealt with multiple-level meta-analytic approaches too (?), or at least that is my interpretation.<br></div><div><br>So, based on
Lajeunesse's example, I have simulated data that could potentially reflect the data of my meta-analysis (I took the liberty to attach these, hope it is not an issue with the rules of
r-sig-meta-analysis
mailing list).<br></div><div>Here I have three studies that report treatment and control change (X_T and X_C, respectively). The yi and vi were calculated with the "ROM" option in escalc().<br></div><div>If I had a case where yi of each study shares the same (only one) control, I would fit the model with following:<br>
<br>"res1<-<a href="http://rma.mv" target="_blank">rma.mv</a>(yi,vi,data=dat,r<wbr>andom=~ 1 | Study/effect_ID)"<br><br></div><div>However, I should account for the fact that I have multiple control group for each yi in each study, so wouldn't be correct to add a third level "commonControl_ID"? i.e.:<br></div><div><br>"res2<-<a href="http://rma.mv" target="_blank">rma.mv</a>(yi,vi,data=dat,r<wbr>andom=~ 1 | Study/commonControl_ID/effect_<wbr>ID)"<br><br></div><div>Which should be identical to:<br></div><div><br>"res3<-<a href="http://rma.mv" target="_blank">rma.mv</a>(yi,vi,data=dat,r<wbr>andom=list(~ 1 | effect_ID, ~ 1 | commonControl_ID, ~ 1 | Study))"<br><br></div><div>
(...
at least, the model output based on my data example looks like they are). Or maybe all of what I have put above does not make sense and I am misunderstanding the usage of multi-level modeling?<br><br></div><div>Thanks for your patience,<br></div><div>Gabriele<br></div></div><div class="m_-3183530218368206644HOEnZb"><div class="m_-3183530218368206644h5"><div class="gmail_extra"><br><div class="gmail_quote">On 10 April 2018 at 18:38, James Pustejovsky <span dir="ltr"><<a href="mailto:jepusto@gmail.com" target="_blank">jepusto@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Gabriele,<div><br></div><div>I see what you mean. Your explanation suggests that there would be high variance in the outcome from study to study, due to examining different species or other aspects of the study's design. I don't think that this precludes using the raw means as effect sizes---it just means that there would be a large between-study variance component. I think this approach would still make it easier to model how the ES might depend on covariates that have variation within a given study (such as altitude level).</div><div><br></div><div>The difficulty with using the ROM effect size is that when you compare multiple conditions to a single control condition, there is correlation in the effect size estimates. In the notation of your original email, you would have </div><div>cov(yi1, yi2) != 0</div><div>cov(yi1, yi3) != 0</div><div>cov(yi2, yi3) != 0</div><div>In fact, there would be very strong dependence because yi3 = yi2 - yi1. So if you use this approach, it would be critical to account for the sampling covariance between the effect sizes. Lajeunesse (2011) gives formulas for the covariances that you'd need.</div><div><br></div><div>James</div><div><br></div><div>Lajeunesse, M. J. (2011). On the meta-analysis of resposne ratios for studies with correlated and multi-design groups. Ecology, 92(11), 2049–2055.<br></div></div><div class="m_-3183530218368206644m_1789836861617316695HOEnZb"><div class="m_-3183530218368206644m_1789836861617316695h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Apr 10, 2018 at 10:22 AM, Gabriele Midolo <span dir="ltr"><<a href="mailto:gabriele.midolo@gmail.com" target="_blank">gabriele.midolo@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>
<div>Dear James (all),<br><br></div><span>Thanks for your ideas, very appreciated.<br>I
understand what you mean, but SLA can be calculated on different
species which are likely to report very different magnitude depending on
the species. Plus I think the ratios calculated with "ROM" should
provide a standardized measure of how much a single species' SLA changes
in response to altitudinal shift, that is what I think might be
interesting to measure in the context of my study.<br>I am not unkeen to
use your option of course, I am just wondering if a three-level
meta-analysis could deal with a type of independency described above, or
it can work with data that share multiple treatment "doses" compated to
a single control...
<br><br></span></div><span class="m_-3183530218368206644m_1789836861617316695m_-5423724694741904787HOEnZb"><font color="#888888">Gabriele<br></font></span></div><div class="gmail_extra"><br><div class="gmail_quote"><span>On 10 April 2018 at 16:21, James Pustejovsky <span dir="ltr"><<a href="mailto:jepusto@gmail.com" target="_blank">jepusto@gmail.com</a>></span> wrote:<br></span><div><div class="m_-3183530218368206644m_1789836861617316695m_-5423724694741904787h5"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Gabriele,<div><br></div><div>If all of the studies use SLA as the outcome measure, would it make sense to model the mean levels directly (instead of calculating proportionate changes)? In other words, use the mean SLA level as the effect size. If it makes sense conceptually, I think this would be a good way to go because then the effect size estimates at each altitude level would be independent of each other, conditional on the true mean levels for that study, and a three-level meta-analysis model would be a natural fit.</div><span class="m_-3183530218368206644m_1789836861617316695m_-5423724694741904787m_-6276785066066874975HOEnZb"><font color="#888888"><div><br></div><div>James</div></font></span></div><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="m_-3183530218368206644m_1789836861617316695m_-5423724694741904787m_-6276785066066874975h5">On Sun, Apr 8, 2018 at 11:12 AM, Gabriele Midolo <span dir="ltr"><<a href="mailto:gabriele.midolo@gmail.com" target="_blank">gabriele.midolo@gmail.com</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="m_-3183530218368206644m_1789836861617316695m_-5423724694741904787m_-6276785066066874975h5"><div dir="ltr">Dear all,<br><br>I have a question that is more methodological but somehow related to metafor. <br>I want to conduct an (ecological) meta-analysis on specific leaf area (SLA) response to increased altitdue (i.e. elevation) in mountain ecosystems. Primary studies selected report the mean (+ SE and sample size) of SLA sampled at different altitudinal levels. The picture attached is an example of how row primary data are normally reported in the articles (modified, from Seguí et al 2018, fig.1c [<a href="https://doi.org/10.1007/s00035-017-0195-9%5D" target="_blank">https://doi.org/10.1007/s0003<wbr>5-017-0195-9]</a>). <br>The A, B and C (in red) values represents the mean values of SLA calculated at 1900, 2200 and 2350 m above the sea level (i.e. altitude) that should, in my opinion, be suitable for calculating log-transformed response ratios (RR) indicating how much SLA increases/decreases compared to a population of plants sampled to a lower altitiude. Thus, given the design of such studies, I propose that multiple RR (yi) must be calulcated within each study as follows:<br><br>yi1= ln(B/A)<br>yi2=ln(C/A)<br>yi3=ln(C/B)<br>... <br>if a D value would have been reported by the authors, sampled to a higher altitdue than 2350 m, then I woul also calculate yi4=ln(D/A), yi5=ln(D/B), yi6=ln(D/C) for this study.<br><br>This approach make sense to me because there is no "proper" control and treatment and you are not just interested to estimate SLA changes by comparing mean values reported at higher altitudes with only the one sampled at the lowest altitudinal level (yi1,yi2), but also between higer altitudinal levels (yi3). This is also supposed to allow to look in meta-regession how the altitudinal shift (so, the difference in altitudes e.g. 300m for yi1) affect the effect size responses. So - and here finally comes my question - with <a href="http://rma.mv" target="_blank">rma.mv</a> I should be able to safely account for non-independence by fitting a model with the "random =~1|Experiment/ID" structure (?). Is this type of data suitable for three-level mixed-effect meta-analysis? I used already this structure in a previous meta-analysis I conducted in the past, but back then I was working with multiple treatments compared to just one single control in each study.<br>I see some similar meta-analysis in the past have used the r-to-z transformed effect size and focused on the correlation
- in my case -
between altitude and SLA, but not sure this is what I would like to investigate in the first place...<br><br>Hope I was clear, and my apologies if I was messy.<br><br>Thanks a lot for reading this<br>Gabriele<br><br><img src="cid:ii_jfqzmxqo0_162a5f6c7131ed52" width="440" height="437"><br><br></div>
<br></div></div><span>______________________________<wbr>_________________<br>
R-sig-meta-analysis mailing list<br>
<a href="mailto:R-sig-meta-analysis@r-project.org" target="_blank">R-sig-meta-analysis@r-project.<wbr>org</a><br>
<a href="https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis" rel="noreferrer" target="_blank">https://stat.ethz.ch/mailman/l<wbr>istinfo/r-sig-meta-analysis</a><br>
<br></span></blockquote></div><br></div>
</blockquote></div></div></div><br></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div></div></div></div></div>
</blockquote></div><br></div>