[R] Different LLRs on multinomial logit models in R and SPSS

[Sören Vogel]

Hello, thanks for all your replies, it was a helpful lesson for me 
(and hopefully for my colleagues, too). Bests, Sören

On 11-01-07 11:23, David Winsemius wrote:

> Date: Fri, 7 Jan 2011 11:23:04 -0500
> From: David Winsemius <dwinsemius at comcast.net>
> To: sovo0815 at gmail.com
> Cc: r-help at r-project.org
> Subject: Re: [R] Different LLRs on multinomial logit models in R and SPSS
> 
>
> On Jan 7, 2011, at 8:26 AM, sovo0815 at gmail.com wrote:
>
>> On Thu, 6 Jan 2011, David Winsemius wrote:
>> 
>>> On Jan 6, 2011, at 11:23 AM, Sören Vogel wrote:
>>> 
>>>> Thanks for your replies. I am no mathematician or statistician by far,
>>>> however, it appears to me that the actual value of any of the two LLs
>>>> is indeed important when it comes to calculation of
>>>> Pseudo-R-Squared-s. If Rnagel devides by (some transformation of) the
>>>> actiual value of llnull then any calculation of Rnagel should differ.
>>>> How come? Or is my function wrong? And if my function is right, how
>>>> can I calculate a R-Squared independent from the software used?
>>> 
>>> You have two models in that function, the null one with ".~ 1" and the 
>>> origianl one and you are getting a ratio on the likelihood scale (which is 
>>> a difference on the log-likelihood or deviance scale).
>> 
>> If this is the case, calculating 'fit' indices for those models must end up 
>> in different fit indices depending on software:
>> 
>> n <- 143
>> ll1 <- 135.02
>> ll2 <- 129.8
>> # Rcs
>> (Rcs <- 1 - exp( (ll2 - ll1) / n ))
>> # Rnagel
>> Rcs / (1 - exp(-ll1/n))
>> ll3 <- 204.2904
>> ll4 <- 199.0659
>> # Rcs
>> (Rcs <- 1 - exp( (ll4 - ll3) / n ))
>> # Rnagel
>> Rcs / (1 - exp(-ll3/n))
>> 
>> The Rcs' are equal, however, the Rnagel's are not. Of course, this is no 
>> question, but I am rather confused. When publishing results I am required 
>> to use fit indices and editors would complain that they differ.
>
> It is well known that editors are sometimes confused about statistics, and if 
> an editor is insistent on publishing indices that are in fact arbitrary then 
> that is a problem. I would hope that the editor were open to education. (And 
> often there is a statistical associate editor who will be more likely to have 
> a solid grounding and to whom one can appeal in situations of initial 
> obstinancy.)  Perhaps you will be doing the world of science a favor by 
> suggesting that said editor first review a first-year calculus text regarding 
> the fact that indefinite integrals are only calculated up to a arbitrary 
> constant and that one can only use the results in a practical setting by 
> specifying the limits of integration. So it is with likelihoods. They are 
> only meaningful when comparing two nested models. Sometimes the software 
> obscures this fact, but it remains a statistical _fact_.
>
> Whether you code is correct (and whether the Nagelkerke "R^2" remain 
> invariant with respect to such transformations) I cannot say. (I suspect that 
> it would be, but I have never liked the NagelR2 as a measure, and didn't 
> really like R^2 as a measure in linear regression for that matter, either.) I 
> focus on fitting functions to trends, examining predictions, and assessing 
> confidence intervals for parameter estimates. The notion that model fit is 
> well-summarized in a single number blinds one to other critical issues such 
> as the linearity and monotonicity assumptions implicit in much of regression 
> (mal-)practice.
>
> So, if someone who is more enamored of (or even more knowledgeably scornful 
> of)  the Nagelkerke R^2 measure wants to take over here, I will read what 
> they say with interest and appreciation.
>
>> 
>> Sören
>
> David Winsemius, MD
> West Hartford, CT
>

-- 
Sören Vogel, sovo0815 at gmail.com, http://sovo0815.wordpress.com/