[R-sig-eco] 2-way adonis (PERMANOVA) incl interaction - how to test for main effects?
Jari Oksanen
j@ri@ok@@nen @ending from oulu@fi
Thu Nov 1 19:29:45 CET 2018
Gian,
The marginal tests for one-term models had a problem in vegan 2.5-1 and vegan 2.5-2. This is fixed in vegan 2.5-3, so please upgrade. (It never occurred to me that people would use marginal tests for one-term models where the fitted model is compared against the empty model, and that case was not handled correctly: any other test — total or sequential — should yield identical results in this case and marginal tests make no sense).
I haven’t even considered implementing type-XXX tests in vegan::adonis. The function uses standard R stats functions which only allow sequential tests or tests of marginal terms (ignoring non-marginal main effects when there are interactions). However, if you are interested in interaction term A:B, your really should inspect the model A+B+A:B. It makes no sense in any sense to study model A:B without the main effects. That would give misleading results for A:B. The interpretation of ~A:B is based on standard R stats::model.matrix() in vegan, and these seem to expand a bit differently for factors A & B and continuous variables A & B. So beware, brother beware. Use stats::model.matrix() to see how they really unfold in your case.
Cheers, Giari
On 1 Nov 2018, at 20:10, Gian Maria Niccolò Benucci <gian.benucci using gmail.com<mailto:gian.benucci using gmail.com>> wrote:
Thank you very much Steve,
I really appreciate you answer. I do not want to drop the main effects out
form the model, I was just trying to understand how adonis works. I was
more confused on how to use the adonis function correctly rather than on
the theory that is behind it. I have an almost "fully crossed" design (that
was the plan, the fact is that I have to remove 2 samples because of the
poor sequencing results) with two factors "Growhouse" and "Stage" in this
case. I am of course interested in the effect of the main factors, but I
also wanted to see if there was a significant interaction and if my
developmental stage (i.e. mature or young) was influenced by where my
organism where growing (i.e. growhouse 1 or 2).
Iif I do as you said, Type I SS for ~ A + B + A*B depends on order so... it
is sequential SS(A) SS(B|A) SS(A*B|A B)
The interaction is not significant, so maybe worth in this case perform a
Type II test directly?
adonis(t(otu_fungi_out) ~ Stage + Growhouse + Stage : Growhouse,
data=metadata_fungi_out, permutations=9999)
Call:
adonis(formula = t(otu_fungi_out) ~ Stage + Growhouse +
Stage:Growhouse, data = metadata_fungi_out, permutations = 9999)
Permutation: free
Number of permutations: 9999
Terms added sequentially (first to last)
Df SumsOfSqs MeanSqs F.Model R2 Pr(>F)
Stage 1 0.4877 0.48769 2.7060 0.10408 0.0238 *
Growhouse 1 0.3765 0.37647 2.0889 0.08034 0.0561 .
Stage:Growhouse 1 0.2171 0.21708 1.2045 0.04633 0.2376
Residuals 20 3.6045 0.18023 0.76925
Total 23 4.6857 1.00000
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
adonis(t(otu_fungi_out) ~ Growhouse + Stage + Stage : Growhouse,
data=metadata_fungi_out, permutations=9999)
Call:
adonis(formula = t(otu_fungi_out) ~ Growhouse + Stage +
Stage:Growhouse, data = metadata_fungi_out, permutations = 9999)
Permutation: free
Number of permutations: 9999
Terms added sequentially (first to last)
Df SumsOfSqs MeanSqs F.Model R2 Pr(>F)
Growhouse 1 0.3765 0.37647 2.0889 0.08034 0.0563 .
Stage 1 0.4877 0.48769 2.7060 0.10408 0.0271 *
Growhouse:Stage 1 0.2171 0.21708 1.2045 0.04633 0.2364
Residuals 20 3.6045 0.18023 0.76925
Total 23 4.6857 1.00000
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Thank you,
Gian
On Thu, 1 Nov 2018 at 11:06, Steve Brewer <jbrewer using olemiss.edu<mailto:jbrewer using olemiss.edu>> wrote:
Gian,
I am bit confused by what your concern is. First, if the imbalance is not
that severe, the approach you take to analyzing a two-way permanova (type
I, type II, type III ss) is not going to matter that much. Indeed, if the
design were balanced, they would give you identical results. Second,
regardless of the lack of balance, for the models y ~ A + B + A:B and y ~
B + A + A:B, the test for the interaction will be the same. So, I don’t
understand why you would want to drop the main effects from the model,
effectively combining them with interaction. That doesn’t make any sense to
me. The problem is with the tests of the main effects.
My advice is to run both models (i.e., A first, then B first) using type I
ss. As mentioned, both models will give you the same interaction result. If
the interaction is all that you’re interested in, problem solved. Interpret
only the interaction and and ignore the main effects. If the interaction is
not significant and low, then interpret only the main effects, focusing
only on the second main effect in each of the differently-ordered models
(which are equivalent to Type II ss tests). And these results will tell you
pretty the same thing as type III tests if there is little or no
interaction. I would not worry about trying to estimate the main effects
while controlling for the interaction (Ellen’s question), which cannot be
done using type I or type II SS in 2-way permanova using adonis. But why
would you want to? The lack of a balanced design results in the main
effects and the interaction not being independent of one another. Forcing
that independence by using type III ss can only work by essentially
"throwing away" some of the information associated with the main effects,
possibly resulting in an overly conservative test. The lower the
interaction, however, the less is thrown away and the less it matters.
Steve
Stephen Brewer
jbrewer using olemiss.edu<mailto:jbrewer using olemiss.edu>
Professor
Department of Biology
PO Box 1848
University of Mississippi
University, Mississippi 38677-1848
Brewer web page - https://jstephenbrewer.wordpress.com<https://jstephenbrewer.wordpress.com/>
FAX - 662-915-5144<tel:662-915-5144> Phone - 662-202-5877<tel:662-202-5877>
On Oct 31, 2018, at 5:45 PM, Gian Maria Niccolò Benucci <
gian.benucci using gmail.com<mailto:gian.benucci using gmail.com>> wrote:
Thank you Jari,
So to test if there are significant interaction I should use
Stage:Growhouse
i.e. A:B. This will test the interaction and main effects that are marginal
and so removed. How matters then if I include by="margin" or not? The R2
are the same (please see below) but the p-value changes. I assume the
second way is most correct, is it?
*> adonis2(t(otu_fungi_out) ~ Stage : Growhouse, data=metadata_fungi_out,
permutations=9999)*
Permutation test for adonis under reduced model
Terms added sequentially (first to last)
Permutation: free
Number of permutations: 9999
adonis2(formula = t(otu_fungi_out) ~ Stage:Growhouse, data =
metadata_fungi_out, permutations = 9999)
Df SumOfSqs R2 F Pr(>F)
Stage:Growhouse 3 1.0812 0.23075 1.9998 0.0211 *
Residual 20 3.6045 0.76925
Total 23 4.6857 1.00000
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
*> adonis2(t(otu_fungi_out) ~ Stage : Growhouse, data=metadata_fungi_out,
by = "margin", permutations=9999)*
Permutation test for adonis under reduced model
Marginal effects of terms
Permutation: free
Number of permutations: 9999
adonis2(formula = t(otu_fungi_out) ~ Stage:Growhouse, data =
metadata_fungi_out, permutations = 9999, by = "margin")
Df SumOfSqs R2 F Pr(>F)
Stage:Growhouse 3 1.0812 0.23075 1.9998 0.006 **
Residual 20 3.6045 0.76925
Total 23 4.6857 1.00000
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Cheers,
Gian
On Tue, 30 Oct 2018 at 05:47, Jari Oksanen <jari.oksanen using oulu.fi<mailto:jari.oksanen using oulu.fi>> wrote:
Hello Gian,
These formulae expand into different models. Compare
model.matrix(~ Stage:Growhouse, data=metadata_fungi_out)
model.matrix(~ Stage*Growhouse, data=metadata_fungi_out)
The first model (Stage:Growhouse) will also contain (implicitly) main
effects and all these terms are marginal and can be removed, whereas the
latter Stage*Growhouse expands to explicit main effects and interaction
effects, and only the interaction effects are marginal and can be removed.
This is also reflected in the degrees of freedom in your anova table: In
the first case Stage:Growhouse has 3 df, and in the latter only 1 df (and
the main effects ignored had 2 df).
Ciao, Giari
On 29 Oct 2018, at 19:11, Gian Maria Niccolò Benucci <
gian.benucci using gmail.com<mailto:gian.benucci using gmail.com>> wrote:
Hello Jari,
It is a little bit confusing. If A*B unfolds in A+B+A:B then A:B is the
real interaction component.
So, which if the code below will test the variance for the interaction
alone?
adonis2(t(otu_fungi_out) ~ *Stage : Growhouse*, data=metadata_fungi_out,
by = "margin", permutations=9999)
Permutation test for adonis under reduced model
Marginal effects of terms
Permutation: free
Number of permutations: 9999
adonis2(formula = t(otu_fungi_out) ~ Stage:Growhouse, data =
metadata_fungi_out, permutations = 9999, by = "margin")
Df SumOfSqs R2 F Pr(>F)
Stage:Growhouse 3 1.0812 0.23075 1.9998 1e-04 ***
Residual 20 3.6045 0.76925
Total 23 4.6857 1.00000
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
adonis2(t(otu_fungi_out) ~ *Stage * Growhouse*, data=metadata_fungi_out,
by = "margin", permutations=9999)
Permutation test for adonis under reduced model
Marginal effects of terms
Permutation: free
Number of permutations: 9999
adonis2(formula = t(otu_fungi_out) ~ Stage * Growhouse, data =
metadata_fungi_out, permutations = 9999, by = "margin")
Df SumOfSqs R2 F Pr(>F)
Stage:Growhouse 1 0.2171 0.04633 1.2045 0.2443
Residual 20 3.6045 0.76925
Total 23 4.6857 1.00000
The results is clearly very different. Also, in a normal adonis call I
didn't have any significance for the interaction that I have instead if I
use A:B. So ~ A*B will not test for interactions at all?
*adonis*(t(otu_fungi_out) ~ Stage * Growhouse, data=metadata_fungi_out,
permutations=9999)
Call:
adonis(formula = t(otu_fungi_out) ~ Stage * Growhouse, data =
metadata_fungi_out, permutations = 9999)
Permutation: free
Number of permutations: 9999
Terms added sequentially (first to last)
Df SumsOfSqs MeanSqs F.Model R2 Pr(>F)
Stage 1 0.4877 0.48769 2.7060 0.10408 0.0247 *
Growhouse 1 0.3765 0.37647 2.0889 0.08034 0.0542 .
Stage:Growhouse 1 0.2171 0.21708 1.2045 0.04633 0.2507
Residuals 20 3.6045 0.18023 0.76925
Total 23 4.6857 1.00000
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Thank you!
Gian
On Tue, 16 Oct 2018 at 08:54, Jari Oksanen <jari.oksanen using oulu.fi<mailto:jari.oksanen using oulu.fi>> wrote:
On 16/10/18 11:23, Torsten Hauffe wrote:
"adonis2(speciesdataset~A*B, by="margin") but then only the effect of
the
interaction is tested."
This is not entirely correct.
adonis2(speciesdataset~A:B, by="margin") would test the interaction
alone.
~A*B unfolds to ~A+B+A:B
Well, it was correct: the only **marginal** effect in ~A+B+A:B is A:B (A
and B are not marginal), and by = "margin" will only analyse marginal
effects.
Cheers, Jari Oksanen
On Tue, 16 Oct 2018 at 11:51, Ellen Pape <ellen.pape using gmail.com<mailto:ellen.pape using gmail.com>> wrote:
Hi all,
I don't know whether this is the correct mailing group to address this
question:
I would like to perform a 2-way permanova analysis in R (using adonis
in
vegan). By default you are performing sequential tests (by="terms"),
so
when you have 2 or more factors, the order of these factors matter.
However, since I wanted to circumvent this, I chose for the option
by="margin" (adonis2(speciesdataset~A*B, by="margin")) but then only
the
effect of the interaction is tested. On the "help page" of anova. cca
it
says: "if you select by="margin" -> the current function only
evaluates
marginal terms. It will, for instance, ignore main effects that are
included in interaction terms."
My question now is: can I somehow get the main effects tested anyhow,
when
the interaction term is not significant?
Thanks,
Ellen
[[alternative HTML version deleted]]
_______________________________________________
R-sig-ecology mailing list
R-sig-ecology using r-project.org<mailto:R-sig-ecology using r-project.org>
https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
[[alternative HTML version deleted]]
_______________________________________________
R-sig-ecology mailing list
R-sig-ecology using r-project.org<mailto:R-sig-ecology using r-project.org>
https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
_______________________________________________
R-sig-ecology mailing list
R-sig-ecology using r-project.org<mailto:R-sig-ecology using r-project.org>
https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
[[alternative HTML version deleted]]
_______________________________________________
R-sig-ecology mailing list
R-sig-ecology using r-project.org<mailto:R-sig-ecology using r-project.org>
https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
*----- Do not print this email unless you really need to. Save paper and
protect the environment! -----*
[[alternative HTML version deleted]]
_______________________________________________
R-sig-ecology mailing list
R-sig-ecology using r-project.org<mailto:R-sig-ecology using r-project.org>
https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
[[alternative HTML version deleted]]
_______________________________________________
R-sig-ecology mailing list
R-sig-ecology using r-project.org<mailto:R-sig-ecology using r-project.org>
https://stat.ethz.ch/mailman/listinfo/r-sig-ecology
[[alternative HTML version deleted]]
More information about the R-sig-ecology
mailing list