[R-sig-ME] Fwd: same old question - lme4 and p-values

Tue Apr 8 03:13:30 CEST 2008

On Tue, 2008-04-08 at 10:25 +1000, John Maindonald wrote:
> Well, I may have been a bit carried away!
> 
> BUGS is though a bit different, surely. Estimation is done from
> the beginning in a Bayesian framework.  It had not occurred to
> me. till mcmcsamp() came along, that one could do use classical
> estimates, and then graft an MCMC calculation on the end to
> get posterior density estimates.  Purists may think this hybrid
> approach not quite kosher.

MCMC is just a technique that can be used to solve integrals by
simulation. There is nothing intrinsically Bayesian about it. It's just
that complicated integrals necessarily crop up in Bayesian problems. I
think that you are right in that mcmcsamp would upset purists. Why
prefer ML or REML parameter estimates, but then derive a Bayesian
posterior density? But if you get a posterior density, you can get the
posterior mean, median or mode from that, and use that as your estimate
if you want to be more Bayesian.

>  I'd expect that it would be problematic
> if a highly informative prior was used in the MCMC calculation
> (is that correct?).

Only problematic for frequentists and likelihoodists. :-)

> 
> Note however that a prior is chosen that makes the calculation
> relatively straightforward.
> 
> I presume this hybrid approach is a lot less expensive,
> computationally, than Bayesian MCMC estimation of parameters
> as well as posterior densities?
> 
> John Maindonald             email: john.maindonald at anu.edu.au
> phone : +61 2 (6125)3473    fax  : +61 2(6125)5549
> Centre for Mathematics & Its Applications, Room 1194,
> John Dedman Mathematical Sciences Building (Building 27)
> Australian National University, Canberra ACT 0200.
> 
> 
> On 8 Apr 2008, at 9:46 AM, Simon Blomberg wrote:
> 
> > On Mon, 2008-04-07 at 20:47 +1000, John Maindonald wrote:
> > [ snip ]
> >>
> >> Douglas's mcmcsamp() has advanced the state of the art
> >> for multi-level models, offering an approach that had not
> >> previously been readily available.  It is anyone's guess
> >> where it, and statistics and graphs that it makes readily
> >> possible, will in the course of time fit among styles of
> >> presentation that application area people find helpful.
> >
> > Well, it's been possible to easily implement multi-level models in  
> > BUGS
> > using MCMC for a long time. Would you agree that BUGS is readily
> > available? :-) Doug has made it more convenient for R users, but I'm  
> > not
> > sure it has necessarily advanced the state of the art. Maybe brought R
> > up to speed (but ahead of other software which tends to start with the
> > letter S).
> >
> > Simon.
> >
> >>
> >> John Maindonald             email: john.maindonald at anu.edu.au
> >> phone : +61 2 (6125)3473    fax  : +61 2(6125)5549
> >> Centre for Mathematics & Its Applications, Room 1194,
> >> John Dedman Mathematical Sciences Building (Building 27)
> >> Australian National University, Canberra ACT 0200.
> >>
> >>
> >> On 7 Apr 2008, at 12:05 PM, David Henderson wrote:
> >>
> >>> Hi John:
> >>>
> >>>> For all practical purposes, a CI is just the Bayesian credible
> >>>> interval that one gets with some suitable "non-informative prior".
> >>>> Why not then be specific about the prior, and go with the Bayesian
> >>>> credible interval?  (There is an issue whether such a prior can
> >>>> always be found.  Am right in judging this no practical  
> >>>> consequence?)
> >>>
> >>>
> >>> What?  Could you explain this a little more?  There is nothing
> >>> Bayesian about a classical (i.e. not Bayesian credible set or
> >>> highest posterior density, or whatever terminology you prefer) CI.
> >>> The interpretation is completely different, and the assumptions used
> >>> in deriving the interval are also different.  Even though the
> >>> interval created when using a noninformative prior is similar to a
> >>> classical CI, they are not the same entity.
> >>>
> >>> Now, while i agree with the arguments about p-values and their
> >>> validity, there is one aspect missing from this discussion.  When
> >>> creating a general use package like lme4, we are trying to create
> >>> software that enables statisticians and researchers to perform the
> >>> statistical analyses they need and interpret the results in ways
> >>> that HELP them get published.  While I admire Doug for "drawing a
> >>> line in the sand" in regard to the use of p-values in published
> >>> research, this is counter to HELPING the researcher publish their
> >>> results.  There has to be a better way to further your point in the
> >>> community than FORCING your point upon them.  Education of the next
> >>> generation of researchers and journal editors is admittedly slow,
> >>> but a much more community friendly way of getting your point used in
> >>> practice.
> >>>
> >>> Just my $0.02...
> >>>
> >>> Dave H
> >>> --
> >>> David Henderson, Ph.D.
> >>> Director of Community
> >>> REvolution Computing
> >>> 1100 Dexter Avenue North, Suite 250
> >>> 206-577-4778 x3203
> >>> DNADave at Revolution-Computing.Com
> >>> http://www.revolution-computing.com
> >>>
> >>
> >> _______________________________________________
> >> R-sig-mixed-models at r-project.org mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
> > -- 
> > Simon Blomberg, BSc (Hons), PhD, MAppStat.
> > Lecturer and Consultant Statistician
> > Faculty of Biological and Chemical Sciences
> > The University of Queensland
> > St. Lucia Queensland 4072
> > Australia
> > Room 320 Goddard Building (8)
> > T: +61 7 3365 2506
> > http://www.uq.edu.au/~uqsblomb
> > email: S.Blomberg1_at_uq.edu.au
> >
> > Policies:
> > 1.  I will NOT analyse your data for you.
> > 2.  Your deadline is your problem.
> >
> > The combination of some data and an aching desire for
> > an answer does not ensure that a reasonable answer can
> > be extracted from a given body of data. - John Tukey.
> >
> > _______________________________________________
> > R-sig-mixed-models at r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
-- 
Simon Blomberg, BSc (Hons), PhD, MAppStat. 
Lecturer and Consultant Statistician 
Faculty of Biological and Chemical Sciences 
The University of Queensland 
St. Lucia Queensland 4072 
Australia
Room 320 Goddard Building (8)
T: +61 7 3365 2506
http://www.uq.edu.au/~uqsblomb
email: S.Blomberg1_at_uq.edu.au

Policies:
1.  I will NOT analyse your data for you.
2.  Your deadline is your problem.

The combination of some data and an aching desire for 
an answer does not ensure that a reasonable answer can 
be extracted from a given body of data. - John Tukey.