[R] effect size

[Liaw, Andy]

> From: David A. van Leeuwen
> 
> Prof Brian Ripley wrote:
> 
> >I think you want to call summary.lm on the aov object, but 
> this depends on 
> >what you mean by `effect size'.
> >
> I guess this is what we wanted.
> 
> >Given that, I wonder if you are used to standard terminology.
> >  
> >
> No, I am not, unfortunately.  We are doing lots of 
> statistical analyses, 
> using R because it is fab and such, but reviewers are looking 
> for SPSS 
> output using terminology that we can't find in the R bundle---but our 
> general impression is that R does things way more cleverer and better 
> than click-until-you-seed-red-signifficant-effect tools found 
> elsewhere. 

I believe there is `effect size' in DOE, e.g., for two-level, main effect
only designs, effect sizes are something like 0.5*(mean(high) - mean(low)).
This is obviously not what you meant.
 
> Reactions on r-help caused us to request for a better 
> specification of 
> the `effect size' that people wanted, and it turns out to be
> $$ SS(effect) / \Sum SS $$ (SS being sums-of-squares).  To 
> me, a simple 
> physicist, that sounds as `the fraction of explained variance' by the 
> factor.  Looking at the formulas in help(summary.lm) is seems that 
> summary.lm()$r.squared  is exactly what we want (for a one-way aov).
> 
> Is there a way to quickly tabulate the expression $$ SS(effect) / 
> (\Sum_{effects} SS(effect) + SS(residuals)) $$ ?
> 
> The numbers are practically there in the summary.aov() table. 
>  Only the 
> grand total SS needs to be calculated.

I guess you want a partition of R^2 by factors in the model.  The only
situations where I can see this being sensible are:

- One-way classification; i.e., only one factor.
- Completely balanced design.

For designs with more than one factor, the balanceness guarantees the
orthogonality among the sums of squares, so it makes sense to partition R^2
that way.  Short of that, there's no unique way to compute the SS (that's
where the infamous 4 types of SS come from), and I don't see a sensible way
to go about it.

Think about the analogous situation in multiple linear regression:  unless
all predictor variables are orthogonal, there's no sensible way to compute
proportion of variance explained by one variable, because that variable is
correlated with other variables, and it's contribution to R^2 depends on
what other variables are in the model.

Andy 

 
> >For example, R does have an effects() function, and that 
> might be what you 
> >want.
> >
> >  
> >
> I don't really understand the effects()---it must be related to 
> coefficients() but it obviously is different. There is 
> model.tables.aov() which is also enlightening, but I think it 
> is really 
> the $ r^2 $ that we were looking for (our reviewers calling this an $ 
> \eta^2 $---if that clarifies things).
> 
> Thanks,
> 
> ---david
> 
> >On Mon, 15 Mar 2004, David A. van Leeuwen wrote:
> >
> >  
> >
> >>Having searched google '[R] aov effect size' without any results I 
> >>wonder if I not completely miss something. 
> >>
> >>Is there any R function that calculates the effect size of 
> an AOV's main 
> >>effect or interaction effect?  It should be related to the 
> F's and the 
> >>degree of freedom of the error, but the multitude in 
> numbers in aov() 
> >>baffle me a bit.
> >>    
> >>
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://www.stat.math.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! 
> http://www.R-project.org/posting-guide.html
> 
> 


------------------------------------------------------------------------------
Notice:  This e-mail message, together with any attachments,...{{dropped}}