[R-sig-ME] p-values glmer in lme4

[Phillip Alday]

And to add to the mix: MixedModels.jl labels the the (mean/stderr)
column 'z' for LMM and provides p-values, which is an interesting
development from the Doug Bates' R FAQ entry on the lack of p-values in
lme4. (And the default estimation is once again ML and not REML.) None
of it terribly surprising based on DB's comments here and elsewhere over
the years, but an interesting course nonetheless.

Phillip

On 07/19/2017 11:18 PM, Ben Bolker wrote:
>
>    I agree that the label should go according to the inference used. The
> funny (??) thing is that in base lmer (not in the lmerTest extension
> package) *the summary doesn't print p-values at all*, so according to
> the rubric above we can't tell whether the (mean/stderr) column should
> be labeled 'z' or 't'. (We could label that column 'mean/std.err.', but
> then some users would ask "why doesn't the summary print either z or t
> values?" :-( )
>
>
> On 17-07-19 03:58 PM, Fox, John wrote:
>> Hi Ben,
>>
>> I'm glad that you chimed in on this question.
>>
>> Of course, what you say about (virtually) all the p-values being
>> approximations is correct. My own preference would be to use
>> "t-value" when you look up a p-value (approximate or not) for a Wald
>> statistic in a t-distribution and "z-value" when you look up (an
>> asymptotic approximation to) a p-value in the standard-normal
>> distribution. Frankly, however, this seems a bit like splitting
>> hairs, and so I think that what you do now is fine.
>>
>> Best, John
>
>>> -----Original Message----- From: R-sig-mixed-models
>>> [mailto:r-sig-mixed-models-bounces at r-project.org] On Behalf Of Ben
>>> Bolker Sent: Wednesday, July 19, 2017 3:29 PM To:
>>> r-sig-mixed-models at r-project.org Subject: Re: [R-sig-ME] p-values
>>> glmer in lme4
>>>
>>>
>>> This diagnosis sounds correct, and I agree that calling these
>>> numbers "z values" is probably the best way to make the reviewers
>>> happy.
>>>
>>> It opens an interesting terminological can of worms.  My initial
>>> reaction to John's post was "oh, I guess glmer should print 'z
>>> value' rather than 't value' even for fits using families with an
>>> estimated dispersion parameter". Then I thought "but if that's true
>>> shouldn't lmer also print 'z value' rather than 't value', since it
>>> provides essentially the same numbers?" Then I thought "if we
>>> switch lmer to printing 'z value' will everyone start asking 'why 
>>> does lmer provide z values rather than t values?"  Sigh.
>>>
>>> The point is that most of this, while unfairly confusing, is just
>>> convention.  "z values" and "t values" are the same thing - MLEs
>>> (or REML estimates) of the parameters divided by their estimated
>>> standard deviations. Of the common (G)LMM applications, the *only*
>>> case in which these values are actually known to follow a t
>>> distribution exactly is for linear mixed models (Gaussian 
>>> conditional distribution), in the classic case of a balanced,
>>> nested design (and, implied by John below, that the fit uses REML).
>>> Otherwise it becomes a question of which approximations you're
>>> happy with.
>>>
>>> And the sampling distributions of these values are never Normal
>>> (even in the perfect theoretical world where all model assumptions
>>> are true), except asymptotically.
>>>
>>>
>>> On 17-07-19 02:50 PM, Fox, John wrote:
>>>> Dear Leen Catrysse,
>>>>
>>>> I'm going to assume that you used the glmer() function in the
>>>> lme4 package
>>> to fit your gamma GLMM. I notice that the summary() for a gamma
>>> model fit by glmer() reports a "t value" for each fixed-effect
>>> coefficient  -- simply the Wald statistics given by the ratio of
>>> the estimated coefficient to its estimated asymptotic standard
>>> error -- followed by a "Pr(>|z|)".
>>>> I suspect that the Wald statistic is labelled as a "t value"
>>>> because the gamma
>>> GLMM has an estimated dispersion parameter, but because there are
>>> no degrees of freedom calculated for the estimated dispersion (as
>>> there could be, for example, for a LMM fit by REML), I think that
>>> it would probably be preferable to call the Wald statistic a "z
>>> value." In any event, the notation "Pr(>|z|)" suggests that the
>>> standard normal distribution is used to obtain a p- value.
>>>> So, to satisfy the reviewer, why not just call the Wald
>>>> statistics "z-values"
>>> rather than a "t-values"?
>>>> I hope this helps, John
>>>>
>>>> ----------------------------- John Fox, Professor Emeritus 
>>>> McMaster University Hamilton, Ontario, Canada Web:
>>>> socserv.mcmaster.ca/jfox
>>>>
>>>>
>>>>
>>>>> -----Original Message----- From: R-sig-mixed-models 
>>>>> [mailto:r-sig-mixed-models-bounces at r-project.org] On Behalf Of
>>>>> Catrysse Leen Sent: Wednesday, July 19, 2017 7:21 AM To:
>>>>> r-sig-mixed-models at r-project.org Subject: [R-sig-ME] p-values
>>>>> glmer in lme4
>>>>>
>>>>> Dear,
>>>>>
>>>>> I used GLMM to analyse eye-tracking data with the gamma
>>>>> distribution and the log link. P-values were automatically
>>>>> computed in the output (based on the asymptotic Wald tests). We
>>>>> received a comment of a reviewer that the output of our GLMM
>>>>> is inconsistent, as we report a t-value from the output and the
>>>>> p-value based on the asymptotic Wald tests. Does anyone has
>>>>> some feedback on how we can deal with this comment?
>>>>>
>>>>> Thanks in advance, Leen Catrysse
>>>>>
>>>>> [[alternative HTML version deleted]]
>>>>>
>>>>> _______________________________________________ 
>>>>> R-sig-mixed-models at r-project.org mailing list 
>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>>> _______________________________________________ 
>>>> R-sig-mixed-models at r-project.org mailing list 
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>>>>
>>> _______________________________________________ 
>>> R-sig-mixed-models at r-project.org mailing list 
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models