[R-sig-ME] subjects within groups and effects of group
Phillip Alday
phillip.alday at mpi.nl
Tue Jan 30 19:45:49 CET 2018
> However, I would like to know how effects of spectral power (Alpha)
> over Drive vary as a function of Condition.
I may have lost track of your experimental design by this point (I have
experiments of my own to keep track of :) ), but my point had more to do
with the symmetry of interactions -- it may be that the impact of
Condition is modulated by endogenous fluctuations in alpha power. Or
given that alpha power is related to certain aspects of attention, it
may be that alpha power interacts with / modulates the effect of
Condition simply because it serves as a decent proxy for aspects of
attention which are relevant. The same holds for a potential interaction
with Drive and especially with SeqNo (as you could expect attention to
vary somewhat over the course of an experiment).
In other words, because alpha correlates to some extent with attention
and level of attention of course impacts performance on cognitive tasks
(including operating complex machinery such as a car), I would expect to
find some interaction effects between alpha power and other aspects of
the experimental manipulations. This is more a experimental psychology
consideration and less-so a statistical one. On a related statistical
note, I wouldn't include any interactions with alpha power in the random
effects as I suspect that would unnecessarily overparameterize the model.
So I would probably start with a model like:
log(RT) ~ 1 + (PzAlpha + Condition + Drive + SeqNo)^3 + (1 + PzAlpha |
subject) + (1 | trial)
This includes main effects and two- and three-way interactions but
excludes four-way interactions, which are hard to interpret, hard to
compute and probably won't drastically improve your model fit anyway
(based on my experience with these types of data). I allow the effect of
alpha power to vary by subject, in case there are baseline differences
in alpha power that affect the correlation with attention, but I would
drop this effect immediately if the model doesn't converge. Condition is
between subjects, so allowing it to vary by subject doesn't make much
sense. Trial only gets a random intercept because it is just capturing
some notion of fixed repetition and I wouldn't expect the other effects
of the experimental manipulation to vary strongly by Trial.
I would then check the model fit (e.g. by plotting predictions vs.
observed data). If the fit is good enough, then I would try to make some
inferences based on it, even knowing the model isn't perfect -- after
all, "all models are wrong, but some models are useful". You could try
to add the four-way interaction back in or by-subject slopes for SeqNo,
etc. but I suspect you'll have trouble getting those models to converge
with only 1839 observations and they probably won't fit the data that
much better (based on my experience with this type of data).
For lmer() and car::Anova(), it doesn't really matter if your predictors
are between or within subjects / items /etc. Between-subjects
manipulations tend provide better estimates (Andrew Gelman frequently
brings this up on his blog), which in practical terms means that they
have better power, but that's about it.
For post-hoc tests, I would recommend the emmeans package (successor to
lsmeans). The documentation is rather extensive, including lots of notes
and examples on interactions.
This is slowly drifting away from statistical issues and more towards
neuro/psych issues -- if other people on the list feel like it's gone
too far away from mixed models, just let us know. :)
Best,
Phillip
On 26/01/18 23:37, P Greenwood wrote:
> Dear Dr. Alday
>
> Could you elaborate a bit on your answer to my question in decomposing
> the effect of condition. Condition was randomly assigned to two groups
> of participants. I did include both levels of Condition in my analysis
> (the output I sent originally). One approach might be to use the Anova
> command to perform a likelihood ratio test to compare a model that
> includes condition with a model that does not include condition.
> (Perhaps that is what you mean by “test the difference?”) However, I
> would like to know how effects of spectral power (Alpha) over Drive vary
> as a function of Condition.
>
> Thanks also for the information on sequence information.
>
> Thanks so much
>
> Pam
>
>
> P.M. Greenwood, Ph.D.
> /Associate Professor of Psychology/
> /Editorial Board, NeuroImage/
> /David King Hall 2052/
> /George Mason University/
> /MSN 3F5, 4400 University Drive/
> /Fairfax, VA 22030-4444/
>
> /Ph: /703 993-4268
> /fax/: 703 993-1359
> /email: Pgreenw1 at gmu.edu <mailto:Pgreenw1 at gmu.edu>/
> /http://psychology.gmu.edu/people/pgreenw1/
>
>> On Jan 25, 2018, at 5:44 PM, Alday, Phillip <Phillip.Alday at mpi.nl
>> <mailto:Phillip.Alday at mpi.nl>> wrote:
>>
>> Completely agree with Thierry here.
>>
>> In addition to the usual considerations about the bias-variance
>> tradeoff and partial pooling, you need to have things in one model if
>> you really want to compare them. The Difference Between “Significant”
>> and “Not Significant” is not Itself Statistically Significant (Gelman
>> and Stern 2012, doi:10.1198/000313006X152649
>> <https://secure-web.cisco.com/1K0nLxMcHeaEQ3zltL5zGKV6Wf3PuASMvvhRXl8z8qKf_35cqaaAeKSo2Q9Jln-azfrr34PbzOOlzZaZYuDWxL5arm_4R9mtGR3Sfsj9-ShTlNCxMGN06gFN5920BieQ1AbiUfjBLRNCvTRcAUTTNlwGGWzVpfqdGudEkiQ-lN89uB95el2DuEfyJW_E5dtTTKpWwSEYaJQc-1ZqKU74d4imV2ENHCLwror_8EZuYaZ51caefF2SHC0JYTTA_uKYgP_FECA8Q_-j4IjYDn_ZmjdvhekseOw9ixb4f2uGavmMS9iVdBAbeqT7xIO8l66l4yZaRUC7isfxVvlhJDIVb4-gRWFgjzG6uSmMzHBhJL44Y26l0SXjQcNYytqFIhCBmM8ZP-SmusBWVAQAymVxHbPAYWXgqpQif7ESbchFfnl3NrBmMQzazxgNQt-ymStEY0d-GMFAIpZfZxFmXh9Y570HQn5LungV8fQy2VbMa-yw/https%3A%2F%2Fdoi.org%2F10.1198%2F000313006X152649>),
>> so if you care about the significance of the difference, then you need
>> to actually test the difference!
>>
>> For your other question
>>
>>
>>> Trial refers to stimulus events. The stimuli are the same on each
>>> Trial, although the order of the Trials varies between Drives. But,
>>> yes, Trial is a sequence number for the repetition so that there
>>> could be some adaptation or change in response related to number of
>>> exposures. (Assuming that is what you meant). How would I include
>>> Trial as a continuous fixed effect?
>>
>> I would use slightly different names to make things clear. Separate
>> 'Trial' (a fixed series of stimuli) from SeqNo (the sequential
>> position of a given Trial within a Drive).
>>
>> Then your model looks something like this:
>>
>> lmer(RT ~ 1 + Condition*PzAlpha + Drive + SeqNo + (1 | subject) + (1 |
>> trial)
>>
>> I've left out any interactions there, but I suspect you'll at least
>> have an interaction with alpha and sequence number -- I imagine that
>> later trials (i.e. higher sequence numbers) will have worse RTs
>> (exhaustion effects) as will trials with higher alpha power and that
>> this two effects will enhance each other.
>>
>> Including sequence information in the model has received some
>> attention in the psycholinguistic as well as the broader psychology
>> literature as a way of controlling for adapt ion effects. GAMMs have
>> been proposed for such cases to allow for non linear adaptation
>> effects, but I wouldn't mess around with that until you feel much more
>> comfortable with the standard LMMs.
>>
>> And of course, if SeqNo doesn't improve model fit, you can simply omit
>> it for parsimony and easy of both interpretation and fitting.
>>
>> Phillip
>>
>>
>> On 25/01/18 17:33, Thierry Onkelinx wrote:
>>> Dear Pam,
>>>
>>> I'd probably combine both datasets in a single analysis.
>>>
>>> Best regards,
>>>
>>> ir. Thierry Onkelinx
>>> Statisticus / Statistician
>>>
>>> Vlaamse Overheid / Government of Flanders
>>> INSTITUUT VOOR NATUUR- EN BOSONDERZOEK / RESEARCH INSTITUTE FOR NATURE
>>> AND FOREST
>>> Team Biometrie & Kwaliteitszorg / Team Biometrics & Quality Assurance
>>> thierry.onkelinx at inbo.be
>>> Havenlaan 88 bus 73, 1000 Brussel
>>> www.inbo.be
>>>
>>> ///////////////////////////////////////////////////////////////////////////////////////////
>>> To call in the statistician after the experiment is done may be no
>>> more than asking him to perform a post-mortem examination: he may be
>>> able to say what the experiment died of. ~ Sir Ronald Aylmer Fisher
>>> The plural of anecdote is not data. ~ Roger Brinner
>>> The combination of some data and an aching desire for an answer does
>>> not ensure that a reasonable answer can be extracted from a given body
>>> of data. ~ John Tukey
>>> ///////////////////////////////////////////////////////////////////////////////////////////
>>>
>>>
>>>
>>>
>>> 2018-01-24 14:02 GMT+01:00 P Greenwood <pgreenw1 at gmu.edu>:
>>>> Dear Drs Alday and Onkelinx
>>>>
>>>> I wondered if you had thoughts on the best way to conduct followup analysis
>>>> of the between-subjects Condition to which people were randomly assigned.
>>>>
>>>> Pam Greenwood
>>>>
>>>> P.M. Greenwood, Ph.D.
>>>> Associate Professor of Psychology
>>>> Editorial Board, NeuroImage
>>>> David King Hall 2052
>>>> George Mason University
>>>> MSN 3F5, 4400 University Drive
>>>> Fairfax, VA 22030-4444
>>>>
>>>> Ph: 703 993-4268
>>>> fax: 703 993-1359
>>>> email: Pgreenw1 at gmu.edu
>>>> http://psychology.gmu.edu/people/pgreenw1
>>>>
>>>> On Jan 19, 2018, at 8:09 AM, P Greenwood <pgreenw1 at gmu.edu> wrote:
>>>>
>>>> Thanks to you both.
>>>>
>>>> Trial refers to stimulus events. The stimuli are the same on each Trial,
>>>> although the order of the Trials varies between Drives. But, yes, Trial is
>>>> a sequence number for the repetition so that there could be some adaptation
>>>> or change in response related to number of exposures. (Assuming that is
>>>> what you meant). How would I include Trial as a continuous fixed effect?
>>>>
>>>> If the effect of Condition were “significant.” how would one decompose that
>>>> to examine each group (Condition) separately?
>>>>
>>>> Regards
>>>>
>>>> Pam
>>>>
>>>>
>>>> P.M. Greenwood, Ph.D.
>>>> Associate Professor of Psychology
>>>> Editorial Board, NeuroImage
>>>> David King Hall 2052
>>>> George Mason University
>>>> MSN 3F5, 4400 University Drive
>>>> Fairfax, VA 22030-4444
>>>>
>>>> Ph: 703 993-4268
>>>> fax: 703 993-1359
>>>> email: Pgreenw1 at gmu.edu
>>>> http://psychology.gmu.edu/people/pgreenw1
>>>>
>>>> On Jan 19, 2018, at 5:58 AM, Phillip Alday <phillip.alday at mpi.nl> wrote:
>>>>
>>>> Dear Pam, (dear Thierry,)
>>>>
>>>> if I'm reading the description correctly, Pam is conceiving of Trial as
>>>> being an "Item"-type factor (crossed with subject). To rephrase
>>>> Thierry's comment a bit -- if Trial corresponds to an Item (concrete
>>>> stimulus realization sampled from the population of possible stimuli for
>>>> this manipulation) that is the same across subjects, then this is a good
>>>> way to model that. If Trial doesn't correspond to an invariant set of
>>>> items, but is rather just repetitions of the same task (perhaps with
>>>> some random variation that isn't identical across subjects), then
>>>> modeling Trial as a random effect doesn't really help much. However, if
>>>> Trial is just a sequence number for the repetition, it might make sense
>>>> to instead include Trial as a continuous fixed effect in order to model
>>>> adaptation effects.
>>>>
>>>> Best,
>>>> Phillip
>>>>
>>>> On 19/01/18 10:44, Thierry Onkelinx wrote:
>>>>
>>>> Dear Pam,
>>>>
>>>> You are handling condition and subject correctly.
>>>>
>>>> There might be a problem with trial. Does trial indicates dependent
>>>> replication of the study? Is there a common effect of trial X for all
>>>> subjects? Because that is what your current model assumes. In case the
>>>> trials are independent, then you don't need to include it in the
>>>> model.
>>>>
>>>> Note that Condition + PzAlpha + PzAlpha*Condition is verbose. You can
>>>> write it as PzAlpha*Condition.
>>>>
>>>> Best regards,
>>>>
>>>> ir. Thierry Onkelinx
>>>> Statisticus / Statistician
>>>>
>>>> Vlaamse Overheid / Government of Flanders
>>>> INSTITUUT VOOR NATUUR- EN BOSONDERZOEK / RESEARCH INSTITUTE FOR NATURE
>>>> AND FOREST
>>>> Team Biometrie & Kwaliteitszorg / Team Biometrics & Quality Assurance
>>>> thierry.onkelinx at inbo.be
>>>> Havenlaan 88 bus 73, 1000 Brussel
>>>> www.inbo.be
>>>>
>>>> ///////////////////////////////////////////////////////////////////////////////////////////
>>>> To call in the statistician after the experiment is done may be no
>>>> more than asking him to perform a post-mortem examination: he may be
>>>> able to say what the experiment died of. ~ Sir Ronald Aylmer Fisher
>>>> The plural of anecdote is not data. ~ Roger Brinner
>>>> The combination of some data and an aching desire for an answer does
>>>> not ensure that a reasonable answer can be extracted from a given body
>>>> of data. ~ John Tukey
>>>> ///////////////////////////////////////////////////////////////////////////////////////////
>>>>
>>>>
>>>>
>>>>
>>>> 2018-01-18 18:51 GMT+01:00 P Greenwood <pgreenw1 at gmu.edu>:
>>>>
>>>> Hello
>>>>
>>>> I wanted some advice about handling subjects within groups and effects of
>>>> group (randomly assigned). I want to predict reaction time (RT) as a
>>>> function of “Condition,” alpha band power (PzAlpha), and drive. People
>>>> (subjects) are randomly assigned to Condition, of which there are two. Each
>>>> person has data from 5 drives, and for each drive there are 10 trials.
>>>> There are 19 subjects in one group and 20 in the other.
>>>>
>>>> My question is this: Am I handling the “between subjects” factor of
>>>> Condition correctly? Also, am I treating subjects within group correctly?
>>>> I am pasting in some of my data. The output is below.
>>>>
>>>> Regards
>>>>
>>>> Pam Greenwood
>>>>
>>>> library(lme4)
>>>> library(lmerTest)
>>>> INFAST_Behavioral <- read.csv(“….
>>>> na.omit(INFAST_Behavioral)
>>>> INFAST_Behavioral$RT = scale(INFAST_Behavioral$RT, center = TRUE, scale =
>>>> TRUE)
>>>> INFAST_Behavioral$PzAlpha = scale(INFAST_Behavioral$PzAlpha, center = TRUE,
>>>> scale = TRUE)
>>>> sumModelInteraction <- lmer(RT ~ 1 + (Condition + PzAlpha + Drive +
>>>> PzAlpha*Condition) + (1 | subject) + (1 | trial), data = INFAST_Behavioral)
>>>> summary(sumModelInteraction)
>>>>
>>>> subject Condition Drive trial FzAlpha CzAlpha
>>>> PzAlpha FzTheta CzTheta PzTheta FzDelta CzDelta PzDelta RT ACC
>>>> 1 HumanLanguage 1 1 -1.41 -4.3585 -5.5431 6.1516
>>>> 1.5911 3.6247 22.38 18.181 13.812 1568.984857 1
>>>> 1 HumanLanguage 1 2 -7.8605 2.0156 4.7392 15.992
>>>> 12.122 6.9088 26.861 20.592 16.326 1721.359714 1
>>>> 1 HumanLanguage 1 3 -2.6982 -5.6067 -10.038 6.285
>>>> 5.5172 1.2894 13.565 12.981 11.63 1257.092571 1
>>>> 1 HumanLanguage 1 4 3.3975 4.8789 -1.3249 7.0177
>>>> 9.6703 6.1539 10.231 12.261 12.485 1559.461429 1
>>>> …(skipping to Subject 2)
>>>> 2 HumanLanguage 1 1 1.6791 2.8887
>>>> 0.28174 -11.387 -9.9352 3.5936 -1.5767 3.9401 6.7201 1302.328857
>>>> 1
>>>> 2 HumanLanguage 1 2 -13.284 -8.2603 -6.6124 -5.9373
>>>> -8.7551 0.10394 4.5621 10.204 12.261 969.0088571 1
>>>> 2 HumanLanguage 1 3 -0.048973 1.1329 0.67399
>>>> -2.1432 2.5077 -2.4641 9.4667 10.883 7.1396 721.3997143 1
>>>> 2 HumanLanguage 1 4 5.0779 6.8916 6.3892 -1.8682
>>>> 3.1637 7.9712 8.0994 10.883 10.975 707.1145714 1
>>>> 2 HumanLanguage 1 5 -7.0495 -2.782 3.1668 8.4332
>>>> 10.646 9.3726 -3.5937 -7.3769 5.4472 892.8214286 1
>>>> 2 HumanLanguage 1 6 -1.462 -8.1223 -6.5896 -10.895
>>>> -5.6311 0.39941 7.5473 12.783 14.698 611.8802857 1
>>>> 2 HumanLanguage 1 7 -2.6402 -5.1213 -3.7372 3.4542
>>>> 4.2234 -0.99898 1.4089 4.1976 0.56587 761.8742857 1
>>>> 2 HumanLanguage 1 8 3.4393 4.6302 1.5525 1.4604
>>>> 3.1716 3.1622 -2.3427 2.908 4.2259 680.9251429 1
>>>> 2 HumanLanguage 1 9 -0.81024 -0.21642
>>>> -2.3876 2.5839 4.7307 1.5441 3.3761 8.4485 12.02 769.0168571 1
>>>> 2 HumanLanguage 1 10 -6.4045 -4.4937 -2.2449 0.94456
>>>> 2.7048 0.65565 -1.9791 0.26436 1.8435 885.6788571 1
>>>>
>>>> Results:
>>>>
>>>> Linear mixed model fit by REML t-tests use Satterthwaite approximations to
>>>> degrees of freedom [
>>>> lmerMod]
>>>> Formula: RT ~ 1 + (Condition + PzAlpha + Drive + PzAlpha * Condition) +
>>>> (1 | subject) + (1 | trial)
>>>> Data: INFAST_Behavioral
>>>>
>>>> REML criterion at convergence: 3876.4
>>>>
>>>> Scaled residuals:
>>>> Min 1Q Median 3Q Max
>>>> -3.4308 -0.5227 -0.1194 0.3547 8.4095
>>>>
>>>> Random effects:
>>>> Groups Name Variance Std.Dev.
>>>> subject (Intercept) 0.580073 0.76163
>>>> trial (Intercept) 0.004778 0.06912
>>>> Residual 0.434918 0.65948
>>>> Number of obs: 1839, groups: subject, 39; trial, 10
>>>>
>>>> Fixed effects:
>>>> Estimate Std. Error
>>>> df t value Pr(>|t|)
>>>> (Intercept) -0.27054 0.17607 40.80000 -1.537
>>>> 0.13213
>>>> ConditionMachineLang 0.41644 0.24595 36.90000 1.693 0.09884 .
>>>> PzAlpha 0.01192 0.02411 1797.40000 0.494
>>>> 0.62117
>>>> Drive 0.02948 0.01083 1788.40000 2.722
>>>> 0.00655 **
>>>> ConditionMachineLanguage:PzAlpha -0.01998 0.03476 1803.10000 -0.575
>>>> 0.56560
>>>> ---
>>>> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>>>> P.M. Greenwood, Ph.D.
>>>> Associate Professor of Psychology
>>>> Editorial Board, NeuroImage
>>>> David King Hall 2052
>>>> George Mason University
>>>> MSN 3F5, 4400 University Drive
>>>> Fairfax, VA 22030-4444
>>>>
>>>> Ph: 703 993-4268
>>>> fax: 703 993-1359
>>>> email: Pgreenw1 at gmu.edu
>>>> http://psychology.gmu.edu/people/pgreenw1
>>>>
>>>>
>>>> [[alternative HTML version deleted]]
>>>>
>>>> _______________________________________________
>>>> R-sig-mixed-models at r-project.org mailing list
>>>> https://secure-web.cisco.com/1joP--B1_xWT50x7mVQZsmihBAOqrRroakmdBR5MIMiOARC9Fd1hE1yw-1sCnwGxdnwBghNSYF9p3BRdhCaL0o-KBtN4F3VSOMBOaVY1oHIj6WYPh_sr8e-zRksQa9F4ECA_XGS2Afp73TF2WsPYwPhZh0vxQMs_BOy--csyKCdriarDzsvIs-lPMcfSch2Ym2oCTwBXQ_YkEWQehip65TFdXUagB8dg1r-d1G0r841y-84pF20_DPQ6J18R57bYFsQp5D34gsd2C2Jf6WMFXlHUZLbGyg7TV0F8CEvUF6vNcNdjIJTUSz79up7bz_iYDKOezABzwahOdHoC45jHqvTmDYhjaqzCkPsymio7HL4h8UlFKyj0xzrIe3rEFvRQsMaNQdctVCMacfdjj-nJiV9Ab0-aVDdezvOlj8IwbPCwr9QwTBeZzUqOwf5ENxq1r/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-mixed-models
>>>>
>>>>
>>>> _______________________________________________
>>>> R-sig-mixed-models at r-project.org mailing list
>>>> https://secure-web.cisco.com/1joP--B1_xWT50x7mVQZsmihBAOqrRroakmdBR5MIMiOARC9Fd1hE1yw-1sCnwGxdnwBghNSYF9p3BRdhCaL0o-KBtN4F3VSOMBOaVY1oHIj6WYPh_sr8e-zRksQa9F4ECA_XGS2Afp73TF2WsPYwPhZh0vxQMs_BOy--csyKCdriarDzsvIs-lPMcfSch2Ym2oCTwBXQ_YkEWQehip65TFdXUagB8dg1r-d1G0r841y-84pF20_DPQ6J18R57bYFsQp5D34gsd2C2Jf6WMFXlHUZLbGyg7TV0F8CEvUF6vNcNdjIJTUSz79up7bz_iYDKOezABzwahOdHoC45jHqvTmDYhjaqzCkPsymio7HL4h8UlFKyj0xzrIe3rEFvRQsMaNQdctVCMacfdjj-nJiV9Ab0-aVDdezvOlj8IwbPCwr9QwTBeZzUqOwf5ENxq1r/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-mixed-models
>>>>
>>>>
>>>>
>>
>
More information about the R-sig-mixed-models
mailing list