[R-sig-ME] Help with determining effect sizes
Maarten Jung
M@@rten@Jung @end|ng |rom m@||box@tu-dre@den@de
Sat Oct 5 21:57:37 CEST 2019
Hi Francesco,
You already got the unstandardized effect sizes - these simply are the
estimated reaction time differences ("Contrasts Estimate (ms)" in your
table).
If you standardize these differences by dividing by a measure of variation,
e.g. an estimate of the standard deviation of your response variable, you
get so called standardized effect sizes. I guess the latter is what your
reviewer is looking for but (as also mentioned in [1]) one can argue that
this standardization has its own pitfalls.
Best,
Maarten
On Sat, 5 Oct 2019, 21:05 Francesco Romano <fbromano77 using gmail.com> wrote:
> Hi Maarten,
>
> Thank you so much for your suggestions. In the source that you
> recommended, a paper is mentioned that advises reporting unstandardised
> effect sizes but I’m not sure how to do that. Is it the same as the
> division you explained in your message? If so, how would I obtain the
> estimated means for the specific contrasts in my table?
>
> Francesco
>
> On Sat, Oct 5, 2019 at 12:13 PM Maarten Jung <
> Maarten.Jung using mailbox.tu-dresden.de> wrote:
>
>> Dear Francesco,
>>
>> I don't think there is a "standard" way to calculate effect sizes for
>> linear mixed models due to the way the variance is partitioned (see
>> e.g. [1]).
>> One way to compute something similar to Cohen's d would be to divide
>> the difference between the estimated means of two conditions by a
>> rough estimate of the standard deviation of the response variable
>> which you can get by
>> sd(predict(your_model_name))
>>
>> Best,
>> Maarten
>>
>> [1]
>> https://afex.singmann.science/forums/topic/compute-effect-sizes-for-mixed-objects#post-295
>>
>>
>> On Sat, Oct 5, 2019 at 10:01 AM Francesco Romano <fbromano77 using gmail.com>
>> wrote:
>> >
>> > Dear all,
>> >
>> > A journal has asked that I determine the effect sizes for a series of
>> > dummy-coded contrasts from the following ME model:
>> >
>> > RT ~ Group * Grammaticality + (1 + Grammaticality | Participant) +
>> > (1 + Group | item)
>> >
>> > Here RT is my continuous outcome variable measured in milliseconds,
>> Group
>> > is a factor with 3 levels (NS, L2, and HL), and Grammaticality a factor
>> > with 2 levels (gr and ungr). After relevelling —NOTE: I am deliberately
>> > omitting the call for each new relevelled model here— I obtained a
>> series
>> > of contrasts which are tabulated below (not sure you can view this
>> whole):
>> >
>> >
>> > Reference level
>> >
>> > Contrasts
>> >
>> > Estimate
>> >
>> > (ms)
>> >
>> > Effect size
>> >
>> > (Cohen’s *d*)
>> >
>> > SE
>> >
>> > df
>> >
>> > *t*
>> >
>> > *p*
>> >
>> > HL
>> >
>> > GR vs UNGR
>> >
>> > -213
>> >
>> >
>> >
>> > 89
>> >
>> > 72.13
>> >
>> > -2.399
>> >
>> > < .05*
>> >
>> > L2
>> >
>> > GR vs UNGR
>> >
>> > -408
>> >
>> >
>> >
>> > 90
>> >
>> > 74.18
>> >
>> > -4.513
>> >
>> > < .001***
>> >
>> > L1
>> >
>> > GR vs UNGR
>> >
>> > -111
>> >
>> >
>> >
>> > 73
>> >
>> > 70.02
>> >
>> > -1.520
>> >
>> > > .05
>> >
>> >
>> >
>> > HL > L2
>> >
>> > -25
>> >
>> >
>> >
>> > 191
>> >
>> > 43.48
>> >
>> > -.135
>> >
>> > > .05
>> >
>> > GR
>> >
>> > L1 > HL
>> >
>> > 400
>> >
>> >
>> >
>> > 175
>> >
>> > 43.81
>> >
>> > 2.286
>> >
>> > < .05*
>> >
>> >
>> >
>> > L1 > L2
>> >
>> > 374
>> >
>> >
>> >
>> > 179
>> >
>> > 43.59
>> >
>> > 2.092
>> >
>> > < .05*
>> >
>> >
>> >
>> > HL > L2
>> >
>> > -219
>> >
>> >
>> >
>> > 179
>> >
>> > 42.70
>> >
>> > -1.226
>> >
>> > > .05
>> >
>> > UNGR
>> >
>> > L1 > HL
>> >
>> > 298
>> >
>> >
>> >
>> > 164
>> >
>> > 43
>> >
>> > 1.817
>> >
>> > > .05
>> >
>> >
>> >
>> > L1> L2
>> >
>> > 77
>> >
>> >
>> >
>> > 166
>> >
>> > 42.03
>> >
>> > .469
>> >
>> > > .05
>> >
>> > How would I go about determining the Cohen's *d* for each of the
>> contrasts?
>> >
>> > The model call is:
>> >
>> > Linear mixed model fit by REML. t-tests use Satterthwaite's method
>> > ['lmerModLmerTest']
>> > Formula: RT ~ Group * Grammaticality + (1 + Grammaticality |
>> Participant) +
>> >
>> > (1 + Group | item)
>> > Data: RTanalysis
>> >
>> > REML criterion at convergence: 52800
>> >
>> > Scaled residuals:
>> > Min 1Q Median 3Q Max
>> > -2.1696 -0.6536 -0.1654 0.5060 5.0134
>> >
>> > Random effects:
>> > Groups Name Variance Std.Dev. Corr
>> > item (Intercept) 71442 267.29
>> > GroupL2 1144 33.82 0.80
>> > GroupNS 9951 99.76 -0.43 -0.88
>> > Participant (Intercept) 235216 484.99
>> > Grammaticalityungr 50740 225.25 -0.39
>> > Residual 378074 614.88
>> > Number of obs: 3342, groups: item, 144; Participant, 46
>> >
>> > Fixed effects:
>> > Estimate Std. Error df t value Pr(>|t|)
>> > (Intercept) 2801.98 136.70 48.85 20.498 <2e-16
>> ***
>> > GroupL2 -25.86 191.20 43.48 -0.135 0.8931
>> > GroupNS -400.63 175.22 43.81 -2.286 0.0271
>> *
>> > Grammaticalityungr -213.87 89.17 72.13 -2.399 0.0190
>> *
>> > GroupL2:Grammaticalityungr -194.57 107.25 42.55 -1.814 0.0767
>> .
>> > GroupNS:Grammaticalityungr 102.31 99.39 43.45 1.029 0.3090
>> > ---
>> > Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>> >
>> > Correlation of Fixed Effects:
>> > (Intr) GropL2 GropNS Grmmtc GrL2:G
>> > GroupL2 -0.672
>> > GroupNS -0.744 0.526
>> > Grmmtcltyng -0.404 0.222 0.260
>> > GrpL2:Grmmt 0.259 -0.391 -0.205 -0.589
>> > GrpNS:Grmmt 0.299 -0.202 -0.392 -0.702 0.540
>> > convergence code: 0
>> > Model failed to converge with max|grad| = 0.0477764 (tol = 0.002,
>> component
>> > 1)
>> >
>> > The distribution of the outcome is fairly normal and the overall mean,
>> > without considering the two fixed effects, is very close to the means of
>> > each of the three groups (without considering the effect of
>> Grammaticality)
>> > as well as the means of the two levels of grammaticality (without
>> > considering the effect of group).
>> >
>> > The package simR can simulate data to determine power, amongst other
>> things,
>> > but I am not sure how to do this for models with interactions such as
>> mine.
>> >
>> > Use of simR is recommended in Brysbaert and Stevens (2018)
>> > https://www.journalofcognition.org/articles/10.5334/joc.10/. Perhaps
>> there
>> > is a simpler way of extracting *d *from the stats I already know?
>> >
>> > Any help would be greatly appreciated,
>> >
>> > Francesco
>> >
>> > [[alternative HTML version deleted]]
>> >
>> > _______________________________________________
>> > R-sig-mixed-models using r-project.org mailing list
>> > https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>
> --
> Inviato da Gmail Mobile
>
[[alternative HTML version deleted]]
More information about the R-sig-mixed-models
mailing list