[R-sig-ME] Negative Variance

[John Maindonald]

A variance components model that has a variance structure 

  block variance + plot (within block) variance + subplot (within plot) variance

makes sense only if blocks take out some part of the variation, i.e., variation
between plots within blocks is (in the absence of treatment effects) smaller
than variation between plots in different blocks.  Similarly for subplots
within/between plots.  

If on the contrary, there is more variation between between plots within blocks
than between plots in different blocks (this is likely to happen if there is a 
nutrient or fertility or moisture gradient within blocks), then a model that has
the form on the second line above will if allowed account for this by returning
a negative block component of variance estimate.  It does this in order to get
a plausible variance-covariance structure.

Of course, once a gradient has been identified, it can be accommodated in the
model.  This does not however undo all the malign effects of an unfortunate
experimental design.

John Maindonald             email: john.maindonald at anu.edu.au
phone : +61 2 (6125)3473    fax  : +61 2(6125)5549
Centre for Mathematics & Its Applications, Room 1194,
John Dedman Mathematical Sciences Building (Building 27)
Australian National University, Canberra ACT 0200.
http://www.maths.anu.edu.au/~johnm

On 05/06/2013, at 11:05 AM, Ben Bolker <bbolker at gmail.com> wrote:

> John Maindonald <john.maindonald at ...> writes:
> 
>> 
>> Negative variance estimates can be very useful in alerting that the
>> variance-covariance structure is not what one expects.  Or they may
>> allow the simplest way of specifying the overall variance-covariance
>> structure, short of specifying the variance-covariance structure in
>> some detail.
>> 
>> I was told of an experiment where the experimenters had chosen 
>> blocks to be at right angles to the river bank, accordingly maximising
>> between plot variance.  This came to light, in data analysed away
>> from the scene of the original trial, when the block variance was
>> estimated as negative -- a very useful diagnostic.  Certainly, one can 
>> check on such a possibility by specifying a suitable variance-covariance 
>> structure, but how many analysts will take that trouble?
> 
>  I don't quite get the geometry you're talking about, but
> I take the general point that diagnostics are good and that
> one wouldn't necessarily think to consider negative correlation.
> 
>> Or one has results from each of two eyes per person.  After allowing
>> for any systematic left/right difference, are two eyes from the same
>> individual more or less different than two eyes from different 
>> individuals?  I doubt that there is a general answer that applies to all
>> types of eye measurements.
> 
>  I find this one a little bit less convincing -- here it would
> seem to be perfectly natural to fit a model that allowed for
> positive or negative correlation.
> 
>  The fact remains that, whether or not it's a good idea,
> this is very hard to do in nlme/lme4 for
> structural reasons. Luckily people are suggesting alternative
> packages.  (Anyone who would like to edit 
> http://glmm.wikidot.com/pkg-comparison accordingly is welcome
> to do so ...)
> 
>  I don't think "how do I estimate negative variances?" has quite
> risen to the level of a FAQ yet, so I won't bother adding it
> to http://glmm.wikidot.com/faq (although again, if anyone wants
> to take the initiative to do so I wouldn't complain).
> 
>    Ben Bolker
> 
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models