[R-sig-ME] Is BLUP a good thing?

[Douglas Bates]

On Mon, Mar 28, 2011 at 7:07 PM, Dominick Samperi <djsamperi at gmail.com> wrote:
> I should add that I personally think the BLUP is a good thing!
>
> Why? Because it provides a beautiful link between two topics
> that should be of interest to readers of this list: hypothesis
> testing and smoothing spline fitting. For the smoothing spline
> connection see Terry Speed's comments that follow Robinson (1991).
>
> Indeed, in a smoothing spline context there is much less confusion
> about what is a fixed effect and what is a random effect. The fixed
> effects are the coefficients of the finite dimensional kernel of the
> smoothing operator, for example.
>
> Of course, the common denominator here is the fact that
> a penalized objective (leading to BLUP's) appears in both
> subject areas.

The connection has not escaped my notice.  For over a decade I had the
office next to Grace Wahba and the smoothing spline view kind of oozed
through the wall.

This is one of the reasons that I view the intermediate calculations
in linear and generalized linear mixed-models as penalized least
squares or penalized iteratively reweighted least squares.  Most books
and most software for fitting such models relies on a generalized
least squares approach but to me it is much more effective to view
this as a penalized problem.

> On Mon, Mar 28, 2011 at 12:21 AM, Dominick Samperi <djsamperi at gmail.com> wrote:
>> After reading the recent publications on the dangers of applying BLUP to
>> natural populations (Hadfield et al 2010, Morrissey et al 2010) I was
>> left wondering why it works at all. The latter paper claims that BLUP
>> has a long and successful history when applied to animal breeding,
>> but no examples showing its effectiveness were presented.
>>
>> The paper Hadfield et al 2010 makes the interesting point that
>> BLUP's are often used to estimate effects that are not explicitly
>> accounted for in the model. I think this zeros in precisely on the
>> problem. If the effect is not accounted for in the model, then
>> the model is being used metaphorically, making
>> a scientific analysis of cause/effect relationships very difficult
>> and open to differing interpretations.
>>
>> Some recent books on mixed models do not say a word about
>> BLUP, perhaps to avoid any discussion of the difficulties.
>>
>> So, twenty years after Robinson (1991) has a consensus
>> formed on the question of whether or not BLUP is indeed a good thing?
>>
>> Doug Bates makes the useful point that it really should
>> be called the Bayesian posterior mode, at least in the case of
>> a Gaussian prior, but even this insight does not really address
>> the question of how BLUP can be used effectively.
>>
>> It seems to me the "predicting" (or "estimating") a random effect
>> that is *assumed* to have a zero mean is a little like
>> estimating the intercept in a linear model for
>> which the intercept has been excluded. Similarly, how does one
>> estimate a random effect when the assumed noise is spherical?
>> Furthermore, note that the formula for the random effect BLUP
>> predicts exactly zero as the G-matrix goes to zero, something
>> that Gianola refers to as "paradoxical".
>>
>> I understand how an analysis of variance can help to determine
>> what factors are more important than others, but extracting
>> information from "white noise" is bound to leave much room
>> for differing interpretations. This is not to say that the conclusions
>> are necessarily wrong or ineffective, but they may not be effective
>> for the traditional reasons (p-values, etc.), and consensus may
>> play as large a role as science/statistics.
>>
>> Being a non-expert I hope that these comments are not considered
>> to be too basic or off-topic.
>>
>> Thanks,
>> Dominick
>>
>
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>