[R-sig-eco] ICC confidence intervals and power analysis for random effects in lmer?

[Tom A. Langen - tlangen]

> 2) A reviewer requested a power analysis of the ability to detect a significant random effect. Any tips on how to approach that?

The reviewer was requesting a post-hoc power analysis,  either an 'observed power' or 'detectable effect size' analysis. Hoenig and Heisey (2001) provide the definitive paper showing that the use of post hoc power analysis is fallacious, and that confidence intervals provide a more informative way to evaluate inferences about not rejecting the null statistical hypotheses. There are other good recent papers making the same point, but Hoenig and Heisey (2001) is definitive in my opinion. 

See: Hoenig, J. M., and D. M. Heisey. 2001. The abuse of power: The pervasive fallacy of power calculations for data analysis. Am. Stat. 55: 1-6.

Note that this only applies to post hoc power analysis. Use of power analysis as an aid to planning experiments is an essential tool for experimental design.

Tom Langen
 
Associate Professor 
Departments of Biology & Psychology 
Clarkson University 

Box 5805, Clarkson U., Potsdam NY 13699-5805 
Phone: 315 268 7933, Fax: 315 268 7118 
www.clarkson.edu/~tlangen   


-----Original Message-----
From: r-sig-ecology-bounces at r-project.org [mailto:r-sig-ecology-bounces at r-project.org] On Behalf Of Bradley Carlson
Sent: Wednesday, April 11, 2012 9:51 PM
To: r-sig-ecology
Subject: [R-sig-eco] ICC confidence intervals and power analysis for random effects in lmer?

I'm performing an analysis of behavioral variation among individual
tadpoles, using individual ID as a random effect and time as a continuous
fixed covariate in the lmer function in lmer4 package. I'm really
interested in making inferences about the random effect (i.e. the extent of
variation among individuals). I'd like to do two things that I can't seem
to find straightforward answers about and I'm hoping someone can help or
point me to a good resource.

1) The intraclass correlation coefficient is of particular interest to me,
as it describes the proportion of variation that occurs among individuals.
Ideally I'd like to report a confidence interval of the ICC but I can't
find any way to calculate one, other than a function in the psychometric
package that appears to only work when there are no covariates in the model
(random effect only).

2) A reviewer requested a power analysis of the ability to detect a
significant random effect. Any tips on how to approach that?

Thanks for any help,

Brad

-- 

Bradley Evan Carlson
PhD Candidate
Intercollege Graduate Degree Program in Ecology
The Pennsylvania State University
University Park, PA 16802

Email: carbrae at gmail.com
http://homes.bio.psu.edu/people/faculty/langkilde/index_files/carlson.htm
https://sites.google.com/site/bradleyecarlson/home

	[[alternative HTML version deleted]]

_______________________________________________
R-sig-ecology mailing list
R-sig-ecology at r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-ecology