[R] glm questions --- saturated model

[David Firth]

On Tuesday, Mar 16, 2004, at 20:28 Europe/London, Paul Johnson wrote:

> I'm confused going back and forth between the textbooks and these 
> emails.  Please pardon me that I seem so pedantic.
>
> I am pretty certain that -2lnL(saturated) is not 0 by definition.

I'm pretty certain of that too.

> In the binomial model with groups of size=1, then the observed scores 
> will be {0,1} but the predicted mean will be some number in [0,1], and 
> so -2lnL will not be 0.  I'm reading, for example, Annette Dobson, An 
> Introduction to Generalized Linear Models, 2ed (2002  p. 77), where it 
> gives the formula one can use for -2lnL(saturated) in the binomial 
> model.
>
> For the Normal distribution, Dobson says
>
> -2lnL(saturated) = N log(2 pi sigma^2)
>
> She gives the saturated model -2lnL(saturated) for lots of 
> distributions, actually.
>
> I thought the point in the first note from Prof. Firth was that the 
> deviance is defined up to an additive constant because you can add or 
> subtract from lnL in the deviance formula
>
> D = -2[lnL(full) - lnL(subset)]
>
> and the deviance is unaffected.  But I don't think that means there is 
> a completely free quantity in lnL(saturated).

No, that's not what I said earlier.  Nor what I meant.

lnL(any model, including saturated) is defined only up to an additive 
constant.  Dobson's formulae are presumably correct, and would remain 
so if we added a constant.

For discrete distributions we can probably all agree to define 
likelihood as the probability of getting the observed data (ie to use 
counting measure on 0,1,2,... or whatever to define our "density") --- 
in which case the arbitrariness is resolved.  For continuous 
distributions we can't do that: the probability is zero for every set 
of data.  So we use density with respect to some measure, the choice of 
which measure leads to an arbitrary constant (as Peter said).  But the 
value of the constant doesn't matter -- and that really is the point.

I hope that's somehow clearer.

Best wishes,
David