[R-sig-ME] Fwd: Question about non-significant interactions
Ché Lucero
|ex@|ucero @end|ng |rom gm@||@com
Wed Jul 10 16:23:51 CEST 2019
Francesco,
To answer the question you originally asked as I understand it: the
interaction test would be telling you whether the difference you find
between tasks varies between groups. The result of the interaction test is
not information about whether differences between tasks (or differences
between groups) are legitimate. It's telling you if the result of comparing
the tasks varied depending on which group you're comparing the task results
in. The interaction test in telling you that you did not observe the
pairwise difference between your tasks varying between different groups of
subjects significantly.
HTH,
-Ché
On Wed, Jul 10, 2019 at 9:43 AM Fox, John <jfox using mcmaster.ca> wrote:
> Dear Francesco and Thierry,
>
> To elaborate slightly on Thierry's suggestion and also to address some
> other points:
>
> (1) You (Francesco) could use the linearHypothesis() function in the car
> package to test more specific hypotheses. It would probably be easier to
> formulate such hypotheses if you fit an equivalent cell-mean model,
> defining a factor with levels for the 9 = 3*3 combinations of levels of
> your two factors.
>
> (2) The type-III tests produced by Anova() still aren't sensible. Did you
> look at the material from ?Anova that I included in my previous response?
>
> (3) You say that Anova() uses "a form of shortcut to the traditional way
> of model-fitting/ reduction via the function anova() comparing models."
> That's not quite true in general, because what Anova() does depends on the
> model class and the type of test statistic selected. It's true that for a
> mixed-effects model, Anova() produces Wald tests rather than
> likelihood-ratio tests. Anova() doesn't have a "bglmerMod" method, and so I
> assume that the "merMod" method is inherited or that the default method is
> being used.
>
> Best,
> John
>
> -----------------------------
> John Fox, Professor Emeritus
> McMaster University
> Hamilton, Ontario
> Canada L8S 4M4
> web: socserv.mcmaster.ca/jfox
>
>
> ________________________________________
> From: R-sig-mixed-models [r-sig-mixed-models-bounces using r-project.org] on
> behalf of Thierry Onkelinx via R-sig-mixed-models [
> r-sig-mixed-models using r-project.org]
> Sent: July 10, 2019 2:51 AM
> To: Francesco Romano
> Cc: r-sig-mixed-models using r-project.org
> Subject: Re: [R-sig-ME] Fwd: Question about non-significant interactions
>
> Dear Francesco,
>
> To answer your question, you should convert your hypothesis in a set of
> linear contrasts and test those.
>
> Best regards,
>
> ir. Thierry Onkelinx
> Statisticus / Statistician
>
> Vlaamse Overheid / Government of Flanders
> INSTITUUT VOOR NATUUR- EN BOSONDERZOEK / RESEARCH INSTITUTE FOR NATURE AND
> FOREST
> Team Biometrie & Kwaliteitszorg / Team Biometrics & Quality Assurance
> thierry.onkelinx using inbo.be
> Havenlaan 88 bus 73, 1000 Brussel
> www.inbo.be
>
>
> ///////////////////////////////////////////////////////////////////////////////////////////
> To call in the statistician after the experiment is done may be no more
> than asking him to perform a post-mortem examination: he may be able to say
> what the experiment died of. ~ Sir Ronald Aylmer Fisher
> The plural of anecdote is not data. ~ Roger Brinner
> The combination of some data and an aching desire for an answer does not
> ensure that a reasonable answer can be extracted from a given body of data.
> ~ John Tukey
>
> ///////////////////////////////////////////////////////////////////////////////////////////
>
> <https://www.inbo.be>
>
>
> Op di 9 jul. 2019 om 23:25 schreef Francesco Romano <fbromano77 using gmail.com
> >:
>
> > ---------- Forwarded message ---------
> > From: Francesco Romano <fbromano77 using gmail.com>
> > Date: Tue, Jul 9, 2019 at 11:24 PM
> > Subject: Re: [R-sig-ME] Question about non-significant interactions
> > To: Fox, John <jfox using mcmaster.ca>
> >
> >
> > Dear John,
> >
> > Thanks for the reply. One of my research entails examining the
> relationship
> > between 3 groups of speakers, the 3 levels of the group categorical
> > variable previously mentioned, and two tasks. One prediction is that one
> > group will perform better than other groups on one test but not the
> other.
> >
> > I fit a maximal model using the bglmr function as shown previously, then
> > used car::Anova to determine main effects. My understanding from previous
> > interaction with you precisely here on r-sig-me is that the function
> works
> > as a form of shortcut to the traditional way of model-fitting/ reduction
> > via the function anova() comparing models, eliminating terms one at a
> time.
> >
> > I hope this is clearer now and yes, the question is more of a statistical
> > one than an R one, even though I suspect the mixed-effect aspect of the
> > regression may be relevant to answering it.
> >
> > Frank
> >
> > Tue, Jul 9, 2019 at 11:03 PM Fox, John <jfox using mcmaster.ca> wrote:
> >
> > > Dear Francesco,
> > >
> > > I didn't entirely follow your question and I expect that to answer it,
> it
> > > would be necessary to know more about what your research entails. As
> you
> > > imply, this seems to be more a statistics question than an R question.
> > It's
> > > also not clear to me what function you used to fit the mixed-effects
> > > logistic regression.
> > >
> > > But I did notice that you're apparently using Anova() for type-III
> tests
> > > with the default contr.treatment() coding for factors. The main-effect
> > > tests that result are not sensible. As it says in ?Anova:
> > >
> > > "Warning
> > > Be careful of type-III tests: For a traditional multifactor ANOVA model
> > > with interactions, for example, these tests will normally only be
> > sensible
> > > when using contrasts that, for different terms, are orthogonal in the
> > > row-basis of the model, such as those produced by contr.sum,
> contr.poly,
> > or
> > > contr.helmert, but not by the default contr.treatment. In a model that
> > > contains factors, numeric covariates, and interactions, main-effect
> tests
> > > for factors will be for differences over the origin. In contrast (pun
> > > intended), type-II tests are invariant with respect to (full-rank)
> > contrast
> > > coding. If you don't understand this issue, then you probably shouldn't
> > use
> > > Anova for type-III tests."
> > >
> > > I hope that this is of some help,
> > > John
> > > -----------------------------
> > > John Fox, Professor Emeritus
> > > McMaster University
> > > Hamilton, Ontario
> > > Canada L8S 4M4
> > > web: socserv.mcmaster.ca/jfox
> > >
> > >
> > > ________________________________________
> > > From: R-sig-mixed-models [r-sig-mixed-models-bounces using r-project.org] on
> > > behalf of Francesco Romano [fbromano77 using gmail.com]
> > > Sent: July 9, 2019 9:49 AM
> > > To: r-sig-mixed-models using r-project.org
> > > Subject: [R-sig-ME] Question about non-significant interactions
> > >
> > > Dear all,
> > >
> > >
> > > I have more of a theoretical than practical question for you. The
> model I
> > > am using has two IVs, group (3 levels) and task (2 levels), and a
> > > categorical DV (correct versus incorrect), hence logistic regression.
> > > Random effects for subjects and items, as well as slopes for group by
> > item
> > > and task by subject.
> > >
> > > I am interested in the effect of belonging any of three groups, the
> > levels
> > > of the group IV, in order to test some a priori predictions. The
> bayesian
> > > wrapper is to help the model converge.
> > >
> > > Here is the output:
> > >
> > > > summary(paper2analysis1)
> > > Cov prior : item ~ wishart(df = 5.5, scale = Inf, posterior.scale =
> cov,
> > > common.scale = TRUE)
> > > : Participant ~ wishart(df = 4.5, scale = Inf,
> > posterior.scale =
> > > cov, common.scale = TRUE)
> > > Prior dev : 6.9466
> > >
> > > Generalized linear mixed model fit by maximum likelihood (Laplace
> > > Approximation) ['bglmerMod']
> > > Family: binomial ( logit )
> > > Formula: correctness ~ task * group + (1 + task | Participant) + (1 +
> > > group | item)
> > > Data: data
> > > Control: glmerControl(optimizer = "bobyqa")
> > >
> > > AIC BIC logLik deviance df.resid
> > > 3857.8 3957.2 -1913.9 3827.8 5570
> > >
> > > Scaled residuals:
> > > Min 1Q Median 3Q Max
> > > -2.0196 -0.3744 -0.2312 -0.1368 6.9534
> > >
> > > Random effects:
> > > Groups Name Variance Std.Dev. Corr
> > > item (Intercept) 1.1266 1.0614
> > > groupL2 0.1311 0.3620 -0.12
> > > groupNS 0.2029 0.4504 -0.31 0.17
> > > Participant (Intercept) 0.7582 0.8708
> > > taskpriming 1.2163 1.1029 -0.77
> > > Number of obs: 5585, groups: item, 219; Participant, 46
> > >
> > > Fixed effects:
> > > Estimate Std. Error z value Pr(>|z|)
> > > (Intercept) -2.49187 0.28318 -8.800 < 2e-16 ***
> > > taskpriming 1.30911 0.37367 3.503 0.000459 ***
> > > groupL2 -0.04042 0.38322 -0.105 0.916005
> > > groupNS -1.01144 0.36607 -2.763 0.005727 **
> > > taskpriming:groupL2 0.04305 0.48693 0.088 0.929544
> > > taskpriming:groupNS -0.04942 0.46034 -0.107 0.914506
> > > ---
> > > Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
> > >
> > > Correlation of Fixed Effects:
> > > (Intr) tskprm gropL2 gropNS tsk:L2
> > > taskpriming -0.733
> > > groupL2 -0.660 0.482
> > > groupNS -0.693 0.507 0.509
> > > tskprmng:L2 0.499 -0.632 -0.755 -0.386
> > > tskprmng:NS 0.530 -0.676 -0.390 -0.750 0.508
> > >
> > > The model was then subjected to car::Anova for ANOVA type III analysis
> > with
> > > the following output:
> > >
> > > > car::Anova(paper2analysis1, type = "III")
> > > Analysis of Deviance Table (Type III Wald chisquare tests)
> > >
> > > Response: correctness
> > > Chisq Df Pr(>Chisq)
> > > (Intercept) 77.4344 1 < 2.2e-16 ***
> > > task 12.2737 1 0.0004594 ***
> > > group 9.9237 2 0.0070000 **
> > > task:group 0.0391 2 0.9806462
> > > ---
> > > Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
> > >
> > > I am not sure how to interpret the non-significant interaction in this
> > > case. Does this mean that, although simple effects exist at group level
> > > within one particular task or at task level within one particular
> group,
> > I
> > > lack sufficient power to conclude those effects are real? If I look at
> > the
> > > simple effects, I do indeed find such effects but am not sure how to
> > > interpret them against the lack of a main interaction. At a practical
> > > level, the interaction, rather than the main effects, is the most
> > important
> > > part of the analysis.
> > >
> > > Thank you in advance for any advice.
> > >
> > > Francesco
> > >
> > >
> > >
> > >
> > >
> > > Best,
> > >
> > > Frank
> > >
> > > [[alternative HTML version deleted]]
> > >
> > > _______________________________________________
> > > R-sig-mixed-models using r-project.org mailing list
> > > https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
> > >
> > --
> > Inviato da Gmail Mobile
> > --
> > Inviato da Gmail Mobile
> >
> > [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > R-sig-mixed-models using r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
> >
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-mixed-models using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>
> _______________________________________________
> R-sig-mixed-models using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>
--
PhD Student
Experience and Cognition Lab
Department of Human Development
Cornell University
Martha Van Rensselaer Hall
Ithaca, NY 14853
acl249 using cornell.edu
http://casasanto.com/chelucero/
[[alternative HTML version deleted]]
More information about the R-sig-mixed-models
mailing list