[R] Higher log-likelihood in null vs. fitted model

[Duncan Murdoch]

On 12-05-31 8:53 AM, Andrew Miles wrote:
> Two related questions.
>
> First, I am fitting a model with a single predictor, and then a null model
> with only the intercept.  In theory, the fitted model should have a higher
> log-likelihood than the null model, but that does not happen.  See the
> output below.  My first question is, how can this happen?

I suspect you'll need to give sample data before anyone can really help 
with this.

>
>> m
>
> Call:  glm(formula = school ~ sv_conform, family = binomial, data = dat,
>      weights = weight)
>
> Coefficients:
> (Intercept)   sv_conform
>      -2.5430       0.2122
>
> Degrees of Freedom: 1488 Total (i.e. Null);  1487 Residual
> Null Deviance:    786.1
> Residual Deviance: 781.9 AIC: 764.4
>> null
>
> Call:  glm(formula = school ~ 1, family = binomial, data = dat, weights =
> weight)
>
> Coefficients:
> (Intercept)
>       -2.532
>
> Degrees of Freedom: 1488 Total (i.e. Null);  1488 Residual
> Null Deviance:    786.1
> Residual Deviance: 786.1 AIC: 761.9
>> logLik(m); logLik(null)
> 'log Lik.' -380.1908 (df=2)
> 'log Lik.' -379.9327 (df=1)
>>
>
> My second question grows out of the first.  I ran the same two model on the
> same data in Stata and got identical coefficients.  However, the
> log-likelihoods were different than the one's I got in R, and followed my
> expectations - that is, the null model has a lower log-likelihood than the
> fitted model.  See the Stata model comparison below.  So my question is,
> why do identical models fit in R and Stata have different log-likelihoods?

That's easier:  they use different base measures.  The likelihood is 
only defined up to a multiplicative constant, so the log likelihoods can 
have an arbitrary constant added to them and still be valid.  But I 
would have expected both models to use the same base measure, so the 
differences in log-likelihood should match.

Duncan Murdoch


> -----------------------------------------------------------------------------
>         Model |    Obs    ll(null)        ll(model)     df          AIC
>      BIC
> -------------+---------------------------------------------------------------
>          mod1 |   1489    -393.064   -390.9304     2    785.8608    796.4725
>          null |      1489    -393.064   -393.064      1     788.1279
>   793.4338
>
> Thanks in advance for any input or references.
>
> Andrew Miles
>
> 	[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.