[R] multinomial estimation output stat question - not R question

[Greg Snow]

Mark,

There are a couple of possible things that could be going on here:

In regular ANOVA cases you can have a situation where you have 3 groups, A, B, and C where A and C are significantly different from each other, but B lies between them in such a way that we cannot say that B is significantly different from A or C (variation is large enough that the mean of B could equal that of A or C).  Clearly B cannot equal both A and C if C and A are not the same, it is just a matter of lack of evidence.  B could be the same as A, or it could be the same as C, or it could be something different from either.

So in your case it could be that there is evidence to show a significant difference between positive and negative, but not enough to show how neutral compares to them.

You could try refitting your model with a different baseline to see if there is a significant difference between the new baseline and one of the other levels of the factor.

Another possibility is that in some cases of logistic regressions (and that could easily carry over to multinomial regressions) you get a large coefficient that is very meaningful, but due to the flattness of the likelihood in that region, the wald test overestimates the variance by quite a bit and results in non-significant conclusions.  Look at the size of the coefficient and the size of the standard error estimate, if both are large, then this could be the case and you should ignore the wald test and look more at other types of tests.

Hope this helps,

--
Gregory (Greg) L. Snow Ph.D.
Statistical Data Center
Intermountain Healthcare
greg.snow at imail.org
(801) 408-8111



> -----Original Message-----
> From: r-help-bounces at r-project.org
> [mailto:r-help-bounces at r-project.org] On Behalf Of
> markleeds at verizon.net
> Sent: Monday, September 01, 2008 6:31 PM
> To: r-help at r-project.org
> Subject: [R] multinomial estimation output stat question -
> not R question
>
> I am estimating a multinomial model with two quantitative
> predictors, X1 and X2, and 3 responses. The responses are
> called neutral, positive and negative with neutral being the
> baseline. There are actually many models being estimated
> because I estimate the model over time and also for various
> parameter sets but that's not important. When I estimate a
> model, since neutral is the baseline and there is no
> interaction term, I get back coefficients
>
> X1 negative
> X2 negative
>
> X1 positive
> X2 positive
>
> Usually the signs of the coefficients are what I would
> expect. Also, I've read about Anova so I think that I kind of
>  understand what that is doing. But, what I'm confused about
> is the following: In some of the models, I can get back wald
> statistics for X1 say, where both the X1 negative Wald stat
> and the X1 positive Wald stat are not significant.
> Yet, the pvalue from the Anova for the X1 variable overall is
> significant ? Is this possible ? I think I'm not
> understanding  the Anova output as well as I thought because,
> to me, that seems inconsistent ?
>
> I understand that the Wald statistics for the particular
> variables are kind of analogous to the t-stats in a regular
> regression in that they are a function of the decrease in
> deviance conditional on all the rest of the variables being
> in the model. The pvalue in the Anova table I thought was
> kind of doing the same thing except not differentiating
> between the factors and just calculating the decrease in
> deviance due to
> X1 overall without regard to the particular factor response ?
>
> If I'm right in my interpretation of the Anova output, then
> can that still happen ?
>
> If I'm wrong about my interpretation, and it can happen, can
> someone tell me where to look for an explanation on why that
> can happen and possibly explain where my interpretation is
> wrong ? I just want to understand my output as best as I can.
>
> If it can't happen, then it's puzzling because it is happening.
>
> Thanks for any insights, comments or references. The output
> is not easily reproducible or else I would reproduce it here.
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>