R-beta: Stars again

[Peter Dalgaard BSA]

Jim Lindsey <jlindsey at alpha.luc.ac.be> writes:

> The more I think about these stars, the more it bothers me. I do not
> think that they provide a useful graphical display.
> 1. As I already mentioned, in non-normal models, the standard
> deviation can be very misleading in judging significance. The stars
> should be put on a table of changes in deviance when each parameter in
> turn is removed from the model.

I've been trying to stay out of this, but your line of reasoning seems
to be based on a belief that deviances have better asymptotic
behaviour than the Wald test. I don't think that's necessarily true,
or at least: they too can be misleading. The correct thing would be to
improve the approximation of the p values as much as possible. There's
some interesting new theory out that seems very promising in that
respect.

The Wald-type statistics that one usually sees has the practical
advantage that it depends only on the fitted parameters, but of course
these days it's not really an obstacle to have to do a couple of extra
iterations (per parameter, mind you!)

> 2. Once we realize that, we see that the stars on parameters
> corresponding to a factor variable are meaningless. The contrasts are
> usually rather arbitrary (as Brian Ripley recently pointed out) and
> variables, not individual parameters, are usually to be compared. The stars
> should refer to a variable, not to a parameter.

Yes. Or at least, the joint significance of the dummy variables should
be available. Sometimes you *do* use a meaningful parametrisation of
contrasts, and the associated p-values might be informative.

> 3. Then, if we think of interactions, all stars for main effects are
> meaningless because they cannot be removed without destroying the
> hierarchy of the model.

True. Well almost. There are actually a couple of cases where it makes
sense. (y~C+F:C with F a factor and C continuous gives two lines with
common intercept, e.g.)

>   Apparently my position is evolving to where I believe that the
> option should be removed entirely, or implemented as a new function
> based on changes in deviance.

I'm not all that big a fan of them either, but as long as there is an
on/off switch... 

The other issues are more important and actually relate more to
p-values than to significance stars. 

Two things to note:

(1) The deviance information is actually available via anova.glm and
friends. 

(2) Given that we decide that it would be better to use e.g. a display
that adds joint test statistics for factors and interactions, how many
*other* packages will break, because they extract information from
summary() output? (Perhaps not really that many)
-- 
   O__  ---- Peter Dalgaard             Blegdamsvej 3  
  c/ /'_ --- Dept. of Biostatistics     2200 Cph. N   
 (*) \(*) -- University of Copenhagen   Denmark      Ph: (+45) 35327918
~~~~~~~~~~ - (p.dalgaard at biostat.ku.dk)             FAX: (+45) 35327907
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._