[R-sig-ME] lmer and p-values (variable selection)
Liaw, Andy
andy_liaw at merck.com
Wed Mar 30 01:44:55 CEST 2011
From: John Maindonald
>
> Yes, the effect size is ultimately more important. But one needs
> to be somewhat sure that the effect is real, and that it is estimated
> appropriately. p-values can contribute to a story that gives some
> smaller or larger confidence that claimed effects are real. They
> are just one of several routes that contribute to this end. Opinions
> differ on whether, in any particular circumstance, they are the
> best route.
>
> The discussion that prompted these various comments related to
> a different use of p-values (and p-value 'alternatives'), one that is
> even more controversial. It related to the use of p-values in
> excluding or including model explanatory terms. Here, there are
> several related issues:
>
> 1) Inference for model parameters should take account of the
> process that has generated the model that is under consideration.
> This includes any omission of terms that are judged of no statistical
> consequence. The standard interpretations of p-values apply,
> strictly, only if there has been no elimination/selection of
> variables.
>
> 2) In models that have certain types of imbalance, parameter
> estimates can change markedly (even to changing sign), depending
> on what other terms are in the model.
>
> 3) Point 2 argues for choosing the model that is on
> scientific grounds
> most reasonable, and sticking with it. If model parameters are
> important to the subsequent discussion, be sure that their estimates
> condition on the 'correct' other set of model variables,
> i.e., that the
> other variables that are in the model are the ones that are required
> to allow this interpretation.
I'm afraid that all too often the reason models are chosen on
"statistical ground" is the lack of "scientific ground". Sort of
a catch 22, I guess... Even when "scientific ground" exists,
what exactly constitute one, and how do we know it's not
another rabbit (or ozone) hole?
Andy
> 4) One may however allow fine tuning that simplifies the model, while
> changing nothing of consequence (and it really is necessary to check
> that there are no changes of consequence). p-values may have a
> limited use in such fine tuning, but for that purpose the
> p=0.05 cutoff is
> not appropriate.
>
> John Maindonald email: john.maindonald at anu.edu.au
> phone : +61 2 (6125)3473 fax : +61 2(6125)5549
> Centre for Mathematics & Its Applications, Room 1194,
> John Dedman Mathematical Sciences Building (Building 27)
> Australian National University, Canberra ACT 0200.
> http://www.maths.anu.edu.au/~johnm
>
> On 29/03/2011, at 10:35 PM, Manuel Spínola wrote:
>
> > I am not a statistician, but what the p-value is telling me?
> >
> > Is not more important the effect size?
> >
> > Best,
> >
> > Manuel
> >
> > On 28/03/2011 04:40 p.m., Ben Bolker wrote:
> >>
> >> On 03/28/2011 06:15 PM, John Maindonald wrote:
> >>
> >>> Elimination of a term with a p-value greater than say
> 0.15 or 0.2 is
> >>> however likely to make little differences to estimates of
> other terms
> >>> in the model. Thus, it may be a reasonable way to proceed. For
> >>> this purpose, an anti-conservative (smaller than it should be)
> >>> p-value will usually serve the purpose.
> >> Note that naive likelihood ratio tests of random effects
> are likely to
> >> be conservative (in the simplest case, true p-values are twice the
> >> nominal value) because of boundary issues and those of
> fixed effects are
> >> probably anticonservative because of finite-size effects
> (see PB 2000
> >> for examples of both cases.)
> >>
> >>> John Maindonald email: john.maindonald at anu.edu.au
> >>> phone : +61 2 (6125)3473 fax : +61 2(6125)5549
> >>> Centre for Mathematics & Its Applications, Room 1194,
> >>> John Dedman Mathematical Sciences Building (Building 27)
> >>> Australian National University, Canberra ACT 0200.
> >>> http://www.maths.anu.edu.au/~johnm
> >>>
> >> Ben
> >>
> >> _______________________________________________
> >> R-sig-mixed-models at r-project.org mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
> >>
> >>
> >
> >
> > --
> > Manuel Spínola, Ph.D.
> > Instituto Internacional en Conservación y Manejo de Vida Silvestre
> > Universidad Nacional
> > Apartado 1350-3000
> > Heredia
> > COSTA RICA
> > mspinola at una.ac.cr
> > mspinola10 at gmail.com
> > Teléfono: (506) 2277-3598
> > Fax: (506) 2237-7036
> > Personal website: Lobito de río
> > Institutional website: ICOMVIS
>
>
> [[alternative HTML version deleted]]
>
>
Notice: This e-mail message, together with any attachme...{{dropped:11}}
More information about the R-sig-mixed-models
mailing list