[R-sig-ME] how to know if random factors are significant?

[David Duffy]

On Thu, 3 Apr 2008, John Maindonald wrote:

> Debate over the use of results from twin studies to partition effects
> on measured IQ into environmental and genetic components illustrates
> the point.  The variance components are relevant only in the
> populations of parents who adopted one or other twin.  More to the
> present point, the Flynn effect by which there've been huge IQ
> increases between one generation and the next requires the invocation
> of some mixture of environmental and genetic effects that are outside
> the ken of both the twin studies data and the models used to analyze
> that data.  In biology, do not expect anything to be simple.  As I
> understand it, there've been a variety of attempts to explain the
> Flynn effect, but no clear consensus.
>

Well, this discussion is straying more into the topic of whether the study 
of individual differences is useful.  Even though the mechanism of cohort 
effects on IQ measures is unknown, a suitable observational design can 
still look at variances within generations and covariances between 
generations for various types of relative pair.  A just as difficult human 
phenotype is adult height, where all these same issues are acting (strong 
secular trends, very strong familial resemblance).

> The analyst ought to worry about implications of the with/without
> disputed random effect for power (or effective sample size, or ...) as
> well as for the p-value or CI limits.  The analyst who omits the
> disputed random effect has to worry both that the p-value might be
> unreasonably optimistic and the power curve unreasonably optimistic.
>

This same bugbear is brought up all the time where a particular fixed 
effect/covariate is "not statistically significant" in the present study, 
even though it is known to have effects in other studies.  It is generally 
recommended to include that covariate in one's models. I have never been 
particularly impressed by the examples (eg Breslow and Day) that purport 
to demonstrate problems with just dropping it out -- but it is one place 
where a Bayesian framework deals sensibly with the prior scientific 
information.

Cheers, David Duffy.
-- 
| David Duffy (MBBS PhD)                                         ,-_|\
| email: davidD at qimr.edu.au  ph: INT+61+7+3362-0217 fax: -0101  /     *
| Epidemiology Unit, Queensland Institute of Medical Research   \_,-._/
| 300 Herston Rd, Brisbane, Queensland 4029, Australia  GPG 4D0B994A v