[R] Different LLRs on multinomial logit models in R and SPSS

[David Winsemius]

On Jan 7, 2011, at 8:26 AM, sovo0815 at gmail.com wrote:

> On Thu, 6 Jan 2011, David Winsemius wrote:
>
>> On Jan 6, 2011, at 11:23 AM, Sören Vogel wrote:
>>
>>> Thanks for your replies. I am no mathematician or statistician by  
>>> far,
>>> however, it appears to me that the actual value of any of the two  
>>> LLs
>>> is indeed important when it comes to calculation of
>>> Pseudo-R-Squared-s. If Rnagel devides by (some transformation of)  
>>> the
>>> actiual value of llnull then any calculation of Rnagel should  
>>> differ.
>>> How come? Or is my function wrong? And if my function is right, how
>>> can I calculate a R-Squared independent from the software used?
>>
>> You have two models in that function, the null one with ".~ 1" and  
>> the origianl one and you are getting a ratio on the likelihood  
>> scale (which is a difference on the log-likelihood or deviance  
>> scale).
>
> If this is the case, calculating 'fit' indices for those models must  
> end up in different fit indices depending on software:
>
> n <- 143
> ll1 <- 135.02
> ll2 <- 129.8
> # Rcs
> (Rcs <- 1 - exp( (ll2 - ll1) / n ))
> # Rnagel
> Rcs / (1 - exp(-ll1/n))
> ll3 <- 204.2904
> ll4 <- 199.0659
> # Rcs
> (Rcs <- 1 - exp( (ll4 - ll3) / n ))
> # Rnagel
> Rcs / (1 - exp(-ll3/n))
>
> The Rcs' are equal, however, the Rnagel's are not. Of course, this  
> is no question, but I am rather confused. When publishing results I  
> am required to use fit indices and editors would complain that they  
> differ.

It is well known that editors are sometimes confused about statistics,  
and if an editor is insistent on publishing indices that are in fact  
arbitrary then that is a problem. I would hope that the editor were  
open to education. (And often there is a statistical associate editor  
who will be more likely to have a solid grounding and to whom one can  
appeal in situations of initial obstinancy.)  Perhaps you will be  
doing the world of science a favor by suggesting that said editor  
first review a first-year calculus text regarding the fact that  
indefinite integrals are only calculated up to a arbitrary constant  
and that one can only use the results in a practical setting by  
specifying the limits of integration. So it is with likelihoods. They  
are only meaningful when comparing two nested models. Sometimes the  
software obscures this fact, but it remains a statistical _fact_.

Whether you code is correct (and whether the Nagelkerke "R^2" remain  
invariant with respect to such transformations) I cannot say. (I  
suspect that it would be, but I have never liked the NagelR2 as a  
measure, and didn't really like R^2 as a measure in linear regression  
for that matter, either.) I focus on fitting functions to trends,  
examining predictions, and assessing confidence intervals for  
parameter estimates. The notion that model fit is well-summarized in a  
single number blinds one to other critical issues such as the  
linearity and monotonicity assumptions implicit in much of regression  
(mal-)practice.

So, if someone who is more enamored of (or even more knowledgeably  
scornful of)  the Nagelkerke R^2 measure wants to take over here, I  
will read what they say with interest and appreciation.

>
> Sören

David Winsemius, MD
West Hartford, CT