[R-sig-ME] John Nelder and Nelder-Lee HGLMs

Mon Nov 1 01:52:40 CET 2010

Hi Murray,

I reviewed the HGLM book for JRSSA a few years ago.  As part of the
review I implemented a simple comparison of the HGLM strategy within R
to compare with lme, and, not surprisingly, it worked just fine for
simple models.  Models with more complicated structure, for example,
random slopes, were not straightforward.  I can send the pdf and
sweave file to anyone who is interested.

I believe that Jim Lindsey has made some progress in implementing
these models in R, but I haven't checked recently.  From memory, it
was his 'repeated' package that had the essential code.

http://www.commanster.eu/rcode.html

I don't feel particularly strongly the same way that you do about the
increased space of models to search --- rather, it's pleasant to feel
that there is soemthing constructive that one could do if the qq plot
of the random effects is rather more skewed than is comfortable, also
it's good that a wider array of models can be accommodated in one
modelling approach when we do have reason to support them --- e.g. the
beta-binomial.

I do have some unease with the theory.  I suspect that the unease is
driven by some ad-hocceries that were deployed to make the theory make
sense.  As I recall, the authors had to invent an idea of a 'canonical
scale' to narrow the space of models down to those with nice
properties, but the required conditions are hard to meet.  An
alternative constraint is that the random effects must occur linearly
in the linear predictor.  If either of these constraints hold then the
HGLM does very nicely.  However, it's not obvious why they *should*
hold.  Nelder confirmed this impression in a pleasant email exchange
after my review was published.  So it seems to me that there is at
least one important piece of the theory still missing.

Cheers,

Andrew

On Mon, Nov 01, 2010 at 12:34:22PM +1300, Murray Jorgensen wrote:
> Roger Payne's obituary for John Nelder may be found at
> 
> http://www.vsni.co.uk/home-pages/john-nelder/
> 
> There can be no doubt that John Nelder has changed the face of modern 
> statistics with his work on linear models and generalized linear models 
> which form the core of the Genstat package and were central in the 
> development of S and R.
> 
> I want to draw the attention of this Sig to the following passage in the 
> obituary in which GLMMs are criticised.
> 
> 
> <quote>
> John’s other major activity at Imperial College was his collaboration 
> with Youngjo Lee to develop the theory of hierarchical generalized 
> linear models (HGLMs); see the papers by Lee & Nelder (1996, 2001, 2006) 
> and the book by Lee, Nelder & Pawitan (2006). The 1996 and 2006 papers 
> were presented as “read papers” at meetings of the Royal Statistical 
> Society; it is impressive to note that John was 81 years old when he and 
> Youngjo presented the 2006 paper. HGLMs aimed to provide satisfactory 
> methods of analysis for non-Normal data when there is more than one 
> source of random variation. John viewed generalized linear models as a 
> way of liberating statisticians from the “tyranny” of the Normal 
> distribution, and was a little bemused to see this same tyranny 
> reestablished in methods that were devised initially to extend 
> generalized linear models. These generalized linear mixed models (GLMMs) 
> catered for additional random variation by adding additional 
> Normally-distributed random effects into the linear model of the 
> generalized linear model. John and Youngjo’s new HGLMs extended the 
> methodology to include the beta-binomial, gamma and inverse-gamma 
> distributions, and showed that the conjugate HGLMs (namely binomial GLM 
> with additional beta-binomial random effects, or Poisson with gamma, or 
> gamma with inverse gamma) had attractive advantages in their 
> mathematical theory, computing algorithms and philosophical 
> interpretation. HGLMs can be fitted very efficiently by two interlinked 
> generalized linear models. So we have access to a familiar repertoire of 
> model checking techniques, and can base our choice of error 
> distributions on the data rather than on prejudice or software 
> limitations. Furthermore the analysis can still be carried out 
> interactively – always a very important consideration for John.
> </quote>
> 
> 
> I have some difficulties with the views of this paragraph and wish to 
> make some comments. Firstly HGLMs do allow added flexibility to the 
> modelling of non-normal data by allowing for non-normal distributions of 
> random effects. However unless there is knowledge about the about the 
> nature of the random effect distributions from the context of the 
> application this flexibility just adds problems by allowing a much 
> larger model space within which to choose and estimate a model.
> 
> Secondly Nelder and Lee do not use standard likelihood or Bayesian 
> methods to fit their HGLMs but instead develop another construction 
> called h-likelihood. It is a while since I tried to look at these but I 
> remember being reminded of the 'classification likelihood' approach to 
> finite mixture modelling where assignments of data to components were 
> treated as parameters to be estimated along with the component 
> parameters and mixing proportions. A number of papers have commented 
> that this is not a good idea, for example
> @ARTICLE{lr83,
>   author  = {Little, R. J. A. and Rubin, D. B.},
>   title   = {On jointly estimating parameters and
> missing data by maximizing the complete data likelihood},
>   journal = {Amer. Statist.},
>   volume  = {37},
>   number  = {},
>   pages   = {218-220},
>   year    = {1983}
> }
> 
> I wonder if members of this list can point me to discussions, critical 
> or supportive, of Lee and Nelder's models and methods. Of course I am 
> aware of the discussion of their JRSS paper so you needn't remind me of 
> that.
> 
> Also, is anyone aware if someone is planning to implement HGLMs, by any 
> estimation method, in R?
> 
> Best wishes,  Murray
> 
> 
> -- 
> Dr Murray Jorgensen      http://www.stats.waikato.ac.nz/Staff/maj.html
> Department of Statistics, University of Waikato, Hamilton, New Zealand
> Email: maj at waikato.ac.nz                                Fax 7 838 4155
> Phone  +64 7 838 4773 wk    Home +64 7 825 0441   Mobile 021 0200 8350
> 
> _______________________________________________
> R-sig-mixed-models at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models

-- 
Andrew Robinson  
Program Manager, ACERA 
Department of Mathematics and Statistics            Tel: +61-3-8344-6410
University of Melbourne, VIC 3010 Australia               (prefer email)
http://www.ms.unimelb.edu.au/~andrewpr              Fax: +61-3-8344-4599
http://www.acera.unimelb.edu.au/