[R-meta] Comparing several predictors and responses & most appropriate model

[James Pustejovsky]

Hi Alicia,

This seems like a pretty challenging modeling problem, and (speaking
personally) I'm reluctant to offer guidance because I don't know the
scientific context of the problem that you're investigating. Could you say
more about the evolutionary or ecological question that you're aiming to
investigate with the synthesis? I follow the example you gave in which
you're trying to understand how beak length relates to species
fitness---but is there any relevant ecological theory to suggest *how*
differences in beak length alter fitness when in competition with another
species?

So with that disclaimer, below are some comments/reactions to your
questions.

Best,
James

On Tue, Dec 27, 2022 at 10:35 AM Alicia Foxx <afoxx using u.northwestern.edu>
wrote:

> Hello Everyone,
>
> My coauthor and I are working on a meta-analysis in which we want to know
> the relationship (direction and magnitude) between a variable (mean
> difference on the x-axis) and a log response ratio (on the y-axis). As a
> toy example, we want to know if differences in beak length between bird
> species predict fitness outcomes across studies. From each study, we
> collected differences in beak length between species as well as fitness
> outcomes of birds alone (i.e., not in competition) and in competition.
>
<snip>

>    As you can see, each study contributes several log response ratios
>    (i.e., y effect sizes) and several raw mean differences (i.e., predictor
>    effect sizes). We have several questions:
>       What is the best model to assess an overall relationship between
>       differences in beak length and fitness across studies?
>          It seems that a meta-regression would work for this question but
>          our concern is that this is an inappropriate model given that
> we’re
>          comparing one effect size (mean difference) with another (log
> response
>          ratio). We investigated bivariate meta-analyses, but the inputs
> appear to
>          both be outcomes.
>
>

My first thought would be to use a bivariate meta-analysis, as you
considered. I'm not sure what you mean by "the inputs appear to both be
outcomes." Could you say more about what you see as the drawback of
bivariate meta-analysis?

Ignoring the non-independence issue, I would think one could investigate
the question by using a bivariate meta-analysis where the bivariate ES
would be (fitness ratio, beak length difference) and the predictor would be
separate indicator variables for each variable (i.e., a dummy for fitness
ratio and a dummy for beak length difference) and the model would include
bivariate random effects with an unstructured covariance matrix. The
question of how fitness ratio relates to beak length difference could then
be answered by looking at the *covariance* between the random effects. One
could even compute the regression of fitness ratio on beak length
difference by calculating it directly from the random effects variance
components (i.e., the covariance divided by the variance of the beak length
random effects). There's surely some formula for the standard error of that
beta estimate, but one could also just do a bootstrap (resampling studies)
to get a standard error / confidence interval.

         How do we account for non-independence at the level of comparison
>          (Species A vs. Species B, and species B vs. species A), and
> study. We’ve
>          thought about nested random effects or even mathematical
> corrections for
>          the non-independence.
>          -
>

This seems pretty tricky, and is where I would be looking for relevant
evolutionary/ecological theory to guide how to approach the model. It seems
like you've really got a system with *three* variables instead of just two:
1. Beak length differences (B vs A)
2. Competitive fitness of species B when in competition with A (relative to
baseline fitness)
3. Competitive fitness of species A when in competition with B (relative to
baseline fitness)
So perhaps there would be a way to do a multivariate regression of
(competitive fitness of B, competitive fitness of A) on beak length
difference? A tri-variate meta-analysis model, instead of the bivariate
model I sketched above?

One thing I wonder about is whether variables (2) and (3) are really
different things. Is competitive fitness of species B when in competition
with A a distinct variable from competitive fitness of species A when in
competition with B? Or are those variables different ways of saying the
same thing? Or maybe it would make sense to look at a ratio-of-ratios, such
as

[(competitive fitness of species B when in competition with A) / (baseline
fitness of B)] / [(competitive fitness of species A when in competition
with B) / (baseline fitness of A)]

which is the same thing as

[(competitive fitness of species B when in competition with A) /
(competitive fitness of species A when in competition with B)] / [(baseline
fitness of B) / (baseline fitness of A)]

This ratio-of-ratios seems like it could be interpreted as a measure of how
the relative fitness of the two species changes when they are put into
competition compared to their relative fitness in the absence of
competition. Using the ratio-of-ratios would allow for use of the bivariate
model as described above.

	[[alternative HTML version deleted]]