[R-sig-ME] Power analysis for a LMM with unbalanced data and continuous covariates

[Steven McKinney]

Well said Jake,

The only proper answer to a reviewer's question about the power associated with these findings is
"We did not perform an a-priori power calculation so the power is unknown".

In this case, if you fail to reject the null hypothesis, you can make no conclusion that will have
a known error rate.  Failure to reject does not mean you can conclude that there is no
difference in plasticity across the range of population sizes.  The answer remains unknown.
A confidence interval provides useful information, and power calculations for future studies
could be undertaken with these data, after some discussion as to what size of plasticity
differences is scientifically or biologically meaningful.

Post-hoc power calculations are inappropriate, and if the reviewer insists on them, you
can educate the reviewer about the inappropriate nature of post-hoc power calculations.

Russel Lenth has very good articles discussing this issue.

 

Steven McKinney, Ph.D.

Statistician
Molecular Oncology and Breast Cancer Program
British Columbia Cancer Research Centre

email: smckinney +at+ bccrc +dot+ ca

tel: 604-675-8000 x7561

BCCRC
Molecular Oncology
675 West 10th Ave, Floor 4
Vancouver B.C.
V5Z 1L3
Canada

________________________________________
From: r-sig-mixed-models-bounces at r-project.org [r-sig-mixed-models-bounces at r-project.org] On Behalf Of Jake Westfall [jake987722 at hotmail.com]
Sent: June 4, 2014 9:52 AM
To: Jackie Wood; r-sig-mixed-models at r-project.org
Subject: Re: [R-sig-ME] Power analysis for a LMM with unbalanced data and continuous covariates

Hi Jackie,

Can I suggest a reframing of the problem? The reviewer asked for a post hoc power analysis, but it seems to me that what they are really interested in is knowing something about the range of plausible values (in colloquial terms) for the effect size in question. Your non-significant result suggests that 0 is a plausible value for the effect size, but it may be the case that various "large" values would also be plausible / consistent with your results. In other words, it sounds like they really just want to see some confidence intervals for the effect in question, and they suspect that these confidence intervals will be wide. confint() provides various methods of computing confidence intervals for the parameter estimates of an lmer model. Maybe this will satisfy the reviewer?

Jake

> From: jackiewood7 at gmail.com
> Date: Wed, 4 Jun 2014 12:06:41 -0400
> To: r-sig-mixed-models at r-project.org
> Subject: [R-sig-ME] Power analysis for a LMM with unbalanced data and continuous covariates
>
> Dear R users,
>
> I was wondering if anyone can offer some advice regarding how to conduct a
> power analysis (if it is possible) for a LMM with an unbalanced data set.
>
> Basically I'm conducting a common garden experiment investigating the
> strength of phenotypic plasticity (measured as the magnitude of the
> family-level slope between two temperatures) in relation to population size
> across a number of stream fish populations. We found no evidence that
> plasticity differs across the range of population sizes used in our
> experiment, however one of the reviewers wanted to know the power of our
> analysis to detect Type II error (i.e. failing to reject a false
> null-hypothesis). I realize that post-hoc power analyses are generally
> frowned upon, but here we are.
>
> For the experiment, I generated between 6 and 23 full-sib families for each
> of 8 differentially abundant fish populations. We were constrained in the
> number of families we could generate in certain populations since they were
> so small we could only ethically collect gametes from a small number of
> individuals which explains our unbalanced data.
>
> I have looked at the simulation example at http://rpubs.com/bbolker/11703
> and while what's done here does make sense to me, the example is for a
> balanced design (2 treatments, 30 individuals per treatment, 5 observations
> per individual). I do understand that the point of simulation (at least I
> think so) is to compare power across a range of different values of the
> experimental parameters of interest but I was wondering if it is possible
> to specify different numbers of families per population in a simulation or
> if I'm restricted to running simulations with only equal numbers of
> families per pop.
>
> Additionally, my model is currently set up as:
>
> glmm<- lmer(formula=fam.slope~N*egg.size+N*density+(1|stream)
>
> where N (population size), egg size, and density are continuous. In the
> example given by Ben Bolker he states that if you want to include
> continuous covariates you have to decide what the distribution is going to
> be but I was wondering how the code would differ from the discrete fixed
> effects used in the example? I've tried to look for this, but keep getting
> referred back to the original example.
>
> Any insight would be much appreciated!
>
> Cheers,
> Jackie
>
> --
> Jacquelyn L.A. Wood, PhD.
> Biology Department
> Concordia University
> 7141 Sherbrooke St. West
> Montreal, QC
> H4B 1R6
> Phone: (514) 293-7255
>
>       [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models

        [[alternative HTML version deleted]]

_______________________________________________
R-sig-mixed-models at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models