[R-sig-eco] ICC confidence intervals and power analysis for random effects in lmer?

Bob O'Hara bohara at senckenberg.de
Fri Apr 13 08:23:35 CEST 2012


On 04/13/2012 04:17 AM, Bradley Carlson wrote:
> Thanks for the tips everyone - I'll look into MCMC sampling for the CI. As
> far as the power analysis goes, I'm somewhat familiar with the criticisms
> regarding power analysis. I think this reviewer was curious about it
> because there was a small sample size (small number of levels of the random
> effect) and the ICC point estimate was not so low as to be biologically
> insignificant. It would be nice to state in the paper how much larger of a
> sample size would have enabled us to detect an effect given the observed
> variation, as though this were a pilot study for planning a bigger
> experiment. I'll certainly bring up the suggested citations, but I would
> still be interested to know if there is a method available for performing a
> power analysis for an LRT of a random effect. Thanks again,
I'd use simulation: you can write a function to simulate data and fit 
the model to it, so you can run that for different sample sizes. It's 
not too fiddly to write, and you can leave the computer to run it over 
the weekend, so you can run reasonable sample sizes.

Bob

> Brad
>
> On Thu, Apr 12, 2012 at 12:18 PM, Brian Inouye<bdinouye at bio.fsu.edu>  wrote:
>
>> In addition to Bob O'Hara's suggestion, here is another citation you can
>> give the reviewer/editor, as to why retrospective power analyses are a
>> waste of time.
>>
>> Hoenig, J. M. and D. M. Heisey (2001). "The abuse of power: the pervasive
>> fallacy of power calculations for data analysis." American Statistician
>> 55(1): 19 - 24.
>>
>> Last year the ESA updated its author guidelines for reporting statistics,
>> and removed a suggestion to report power analyses that had been inserted in
>> the 1980s.
>>
>> -Brian Inouye
>> Florida State University
>> Chair, statistical ecology section of the ESA
>>
>>
>>
>> On 4/12/2012 6:00 AM, r-sig-ecology-request at r-**project.org<r-sig-ecology-request at r-project.org>wrote:
>>
>>> 2) A reviewer requested a power analysis of the ability to detect a
>>> significant random effect. Any tips on how to approach that?
>>>
>> Report the random effect and confidence intervals. Retrospective power
>> analyses are pretty pointless (e.g. see http://beheco.oxfordjournals.**
>> org/content/14/3/446.full<http://beheco.oxfordjournals.org/content/14/3/446.full>),
>> unless you're planning to repeat the experiment. Bob
>>
>> ______________________________**_________________
>> R-sig-ecology mailing list
>> R-sig-ecology at r-project.org
>> https://stat.ethz.ch/mailman/**listinfo/r-sig-ecology<https://stat.ethz.ch/mailman/listinfo/r-sig-ecology>
>>
>
>


-- 
Bob O'Hara

Biodiversity and Climate Research Centre
Senckenberganlage 25
D-60325 Frankfurt am Main,
Germany

Tel: +49 69 798 40226
Mobile: +49 1515 888 5440
WWW:   http://www.bik-f.de/root/index.php?page_id=219
Blog: http://blogs.nature.com/boboh
Journal of Negative Results - EEB: www.jnr-eeb.org



More information about the R-sig-ecology mailing list