[R-meta] Comparing several predictors and responses & most appropriate model

[James Pustejovsky]

Wolfgang has a nice worked example of bivariate meta-analysis that
illustrates what I had in mind:
https://www.metafor-project.org/doku.php/analyses:vanhouwelingen2002
(see the sections on "Bivariate approach" and "Regression of true log
odds")
Examples of the more general multivariate meta-analysis case are also
relevant:
https://www.metafor-project.org/doku.php/analyses:berkey1998

Thanks for sharing the pre-print--it looks interesting and relevant.

Best,
James

On Tue, Jan 3, 2023 at 9:26 AM Alicia Foxx <afoxx using u.northwestern.edu> wrote:

> Hi James,
>
> Happy new year and thank you so much for your thoughtful response. We're
> working with plants but thought bird ecology might be a more tangible
> example. Let's say having a longer beak means you get more seeds and have
> healthier offspring - then birds with longer beaks can out-compete birds
> with shorter beaks and perhaps produce more fledglings (theory re:
> equalizing fitness/fitness differences).
>
> I appreciate your notes on the bivariate MA and think we should explore it
> further - we'd be grateful for any readings you could point us to as this
> doesn't seem like a popular method as others. I think this was our
> confusion with the "outcomes" aspect.
>
> Could you say a little bit more about your note re: " I would think one
> could investigate the question by using a bivariate meta-analysis where the
> bivariate ES would be (fitness ratio, beak length difference) and the
> predictor would be separate indicator variables for each variable (i.e., a
> dummy for fitness ratio and a dummy for beak length difference) and the
> model would include bivariate random effects with an unstructured
> covariance matrix."
>
> I also just came across a preprint (Mahon et al. 2021
> <https://www.biorxiv.org/content/10.1101/2021.07.21.453226v1>) that gets
> at the non-independence issue that may be helpful in our case.
>
>
> Thank you,
>
>
> Alicia
> ________________________________________
>
> *Dr. Alicia Foxx, MS, PhD*
>
> she/her/hers
>
> Research Scientist
>
> The Chicago Botanic Garden & New Roots for Restoration
> <http://www.newrootsforrestoration.org/>
>
> Adjunct Professor: Northwestern University
> Associate Editor: Ecological Solutions and Evidence
> <https://besjournals.onlinelibrary.wiley.com/journal/26888319>
>
> ResearchGate <https://www.researchgate.net/profile/Alicia-Foxx> & Google
> Scholar <https://scholar.google.com/citations?user=nlWrL0YAAAAJ&hl=en>
>
> E: afoxx using chicagobotanic.org
>
> _________________________________________
>
> On Thu, Dec 29, 2022 at 2:13 PM James Pustejovsky <jepusto using gmail.com>
> wrote:
>
>> Hi Alicia,
>>
>> This seems like a pretty challenging modeling problem, and (speaking
>> personally) I'm reluctant to offer guidance because I don't know the
>> scientific context of the problem that you're investigating. Could you say
>> more about the evolutionary or ecological question that you're aiming to
>> investigate with the synthesis? I follow the example you gave in which
>> you're trying to understand how beak length relates to species
>> fitness---but is there any relevant ecological theory to suggest *how*
>> differences in beak length alter fitness when in competition with another
>> species?
>>
>> So with that disclaimer, below are some comments/reactions to your
>> questions.
>>
>> Best,
>> James
>>
>> On Tue, Dec 27, 2022 at 10:35 AM Alicia Foxx <afoxx using u.northwestern.edu>
>> wrote:
>>
>>> Hello Everyone,
>>>
>>> My coauthor and I are working on a meta-analysis in which we want to know
>>> the relationship (direction and magnitude) between a variable (mean
>>> difference on the x-axis) and a log response ratio (on the y-axis). As a
>>> toy example, we want to know if differences in beak length between bird
>>> species predict fitness outcomes across studies. From each study, we
>>> collected differences in beak length between species as well as fitness
>>> outcomes of birds alone (i.e., not in competition) and in competition.
>>>
>> <snip>
>>
>>>    As you can see, each study contributes several log response ratios
>>>    (i.e., y effect sizes) and several raw mean differences (i.e.,
>>> predictor
>>>    effect sizes). We have several questions:
>>>       What is the best model to assess an overall relationship between
>>>       differences in beak length and fitness across studies?
>>>          It seems that a meta-regression would work for this question but
>>>          our concern is that this is an inappropriate model given that
>>> we’re
>>>          comparing one effect size (mean difference) with another (log
>>> response
>>>          ratio). We investigated bivariate meta-analyses, but the inputs
>>> appear to
>>>          both be outcomes.
>>>
>>>
>>
>> My first thought would be to use a bivariate meta-analysis, as you
>> considered. I'm not sure what you mean by "the inputs appear to both be
>> outcomes." Could you say more about what you see as the drawback of
>> bivariate meta-analysis?
>>
>> Ignoring the non-independence issue, I would think one could investigate
>> the question by using a bivariate meta-analysis where the bivariate ES
>> would be (fitness ratio, beak length difference) and the predictor would be
>> separate indicator variables for each variable (i.e., a dummy for fitness
>> ratio and a dummy for beak length difference) and the model would include
>> bivariate random effects with an unstructured covariance matrix. The
>> question of how fitness ratio relates to beak length difference could then
>> be answered by looking at the *covariance* between the random effects. One
>> could even compute the regression of fitness ratio on beak length
>> difference by calculating it directly from the random effects variance
>> components (i.e., the covariance divided by the variance of the beak length
>> random effects). There's surely some formula for the standard error of that
>> beta estimate, but one could also just do a bootstrap (resampling studies)
>> to get a standard error / confidence interval.
>>
>>          How do we account for non-independence at the level of comparison
>>>          (Species A vs. Species B, and species B vs. species A), and
>>> study. We’ve
>>>          thought about nested random effects or even mathematical
>>> corrections for
>>>          the non-independence.
>>>          -
>>>
>>
>> This seems pretty tricky, and is where I would be looking for relevant
>> evolutionary/ecological theory to guide how to approach the model. It seems
>> like you've really got a system with *three* variables instead of just two:
>> 1. Beak length differences (B vs A)
>> 2. Competitive fitness of species B when in competition with A (relative
>> to baseline fitness)
>> 3. Competitive fitness of species A when in competition with B (relative
>> to baseline fitness)
>> So perhaps there would be a way to do a multivariate regression of
>> (competitive fitness of B, competitive fitness of A) on beak length
>> difference? A tri-variate meta-analysis model, instead of the bivariate
>> model I sketched above?
>>
>> One thing I wonder about is whether variables (2) and (3) are really
>> different things. Is competitive fitness of species B when in competition
>> with A a distinct variable from competitive fitness of species A when in
>> competition with B? Or are those variables different ways of saying the
>> same thing? Or maybe it would make sense to look at a ratio-of-ratios, such
>> as
>>
>> [(competitive fitness of species B when in competition with A) /
>> (baseline fitness of B)] / [(competitive fitness of species A when in
>> competition with B) / (baseline fitness of A)]
>>
>> which is the same thing as
>>
>> [(competitive fitness of species B when in competition with A) /
>> (competitive fitness of species A when in competition with B)] / [(baseline
>> fitness of B) / (baseline fitness of A)]
>>
>> This ratio-of-ratios seems like it could be interpreted as a measure of
>> how the relative fitness of the two species changes when they are put into
>> competition compared to their relative fitness in the absence of
>> competition. Using the ratio-of-ratios would allow for use of the bivariate
>> model as described above.
>>
>

	[[alternative HTML version deleted]]