[R-meta] Categorical mixed effect models and interpretation of results

[James Pustejovsky]

Here's a little bit of a different perspective on the need for species
random effects in the model. I would argue that the question of whether or
not to include species random effects in the model should be determined by
the goal of the synthesis.

On the one hand, if the goal is to draw generalizations across a population
of species, and if it is reasonable to treat the 12 observed species as a
sample from the population, then it would be appropriate to include random
effects for species (or more generally, to start modeling the species
effects using phylogenetic methods as Wolfgang suggested).

On the other hand, if the 12 observed species constitute the full relevant
population, or if inter-species variation is not of interest for the goals
of the synthesis, or if one is content to restrict generalization to the
observed species, then species random effects would not be appropriate.
(Instead, one could simply treat species as a categorical covariate.)



On Tue, Jun 19, 2018 at 8:55 AM Viechtbauer, Wolfgang (SP) <
wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:

> Hi Alex,
>
> Please keep the mailing list in cc.
>
> I would not say that the results are necessarily 'incorrect'. However,
> just adding random effects for study assumes that there is no heterogeneity
> in the outcomes for different experiments within studies, which is a big
> assumption to make. Also, not adding random effects for species assumes
> that outcomes for the same species are no more similar to each other than
> they are when outcomes are measured in different species. Again, that's a
> questionable assumption. In fact, outcomes in different species may be
> correlated to the degree that species resemble each other, so that gets us
> into phylogenetic meta-analyses. See, for example:
>
> Nakagawa, S., & Santos, E. S. A. (2012). Methodological issues and
> advances in biological meta-analysis. Evolutionary Ecology, 26(5),
> 1253-1274.
>
> Noble, D. W., Lagisz, M., O'Dea R, E., & Nakagawa, S. (2017).
> Non-independence and sensitivity analyses in ecological and evolutionary
> meta-analyses. Molecular Ecology, 26(9), 2410-2425.
>
> Ignoring such issues could lead to underestimation of the uncertainty in
> parameter estimates and hence inflated Type I error rates.
>
> Best,
> Wolfgang
>
> -----Original Message-----
> From: Alexander Sullivan (BIO - Student) [mailto:
> Alexander.J.Sullivan using uea.ac.uk]
> Sent: Tuesday, 19 June, 2018 12:32
> To: Viechtbauer, Wolfgang (SP)
> Subject: Re: Categorical mixed effect models and interpretation of results
>
> Dear Wolfgang,
>
> Thank you for your prompt reply. This does appear to improve my model.
>
> However, I have noticed in the meta-analytical literature especially
> within my research area, ocean acidification, that many authors simply use
> study id as their random effect. Does this mean their results are
> incorrect? or simply less precise than a more complicated model?
>
> Best,
>
> Alex
>
> MSc Ecology and Conservation student
> University of East Anglia, UK
> ________________________________________
> From: Viechtbauer, Wolfgang (SP) <
> wolfgang.viechtbauer using maastrichtuniversity.nl>
> Sent: 19 June 2018 09:31:52
> To: Alexander Sullivan (BIO - Student); r-sig-meta-analysis using r-project.org
> Subject: RE: Categorical mixed effect models and interpretation of results
>
> Hi Alex,
>
> Yes, in principle this is right. If 'Treatment' only has two levels, then
> the QM-test and the test of the treatment coefficient are identical, so you
> can also just look at the latter.
>
> However, I think your random effects structure is too simple given what
> you wrote. At the least, it should be something like:
>
> random = ~ 1 | Study_id / Exp_id
>
> where Exp_id is, as the name implies, the experiment id. See:
>
> http://www.metafor-project.org/doku.php/analyses:konstantopoulos2011
>
> and esp. the "A Common Mistake in the Three-Level Model" section.
>
> You might also want to consider adding random effects for species. For
> example:
>
> random = list(~ 1 | Study_id / Exp_id, ~ 1 | Species)
>
> would add species random effects (as a crossed random effect, not nested
> within study and/or experiment).
>
> Best,
> Wolfgang
>
> -----Original Message-----
> From: R-sig-meta-analysis [mailto:
> r-sig-meta-analysis-bounces using r-project.org] On Behalf Of Alexander
> Sullivan (BIO - Student)
> Sent: Tuesday, 19 June, 2018 10:58
> To: r-sig-meta-analysis using r-project.org
> Subject: [R-meta] Categorical mixed effect models and interpretation of
> results
>
> Dear all,
>
> In my meta-analysis I have [say] 100 experiments, from 30 studies, across
> 12 species. These experiments can be equally divided into two treatment
> levels [low and high]. If I want to determine if there is a significant
> difference in effect sizes between these two treatment levels would this be
> the right code:
>
> res <- rma.mv(yi, vi, method = "REML", data = mydata, mods =
> ~factor(Treatment), random = ~1|Study_id)
>
> ... and then from the output of this model if the Qm statistic is
> significant I could say there is a significant difference in effect sizes
> between the two treatments?
>
> Thank you for your time,
>
> Alex
>
> MSc Ecology and Conservation student
> University of East Anglia, UK
>
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis using r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>

	[[alternative HTML version deleted]]