[R] Re: HOWTO compare univariate binomial glm lrm models which are not nested
Frank E Harrell Jr
f.harrell at vanderbilt.edu
Sun Apr 17 17:13:45 CEST 2005
Jan Verbesselt wrote:
> Dear all,
>
> Thanks a lot for the input. I will take the considerations into account.
>
> Referring to;
> "2 or 3 completely pre-chosen models or you will invalidate inference and
> estimates if you use these comparisons to build a final model"
>
> The aim is not use the comparisons to build a final model but to select the
> explanatory variable which explains most of the variance or has the best
> predictive ability (p247 10.8 Harrell, 2001).
One procedure that will shed light on this is to bootstrap the ranks of
the chi-square statistics for competing variables. I think you will be
surprised how wide the confidence intervals for the ranks are. There is
an example in the Alzola & Harrell document although it is for partial
chi-squares for competing variables in a single model.
-FH
>
> I'm comparing variables, which are all related to the remotely sensed water
> content of vegetation, with binary fire occurrence data (1: fire / 0: no
> fire). The aim is to select the water related variable which has the best
> 'performance' (Referring to literature about logistic regression used for
> evaluation of fire danger indices).
>
> e.g. a lrm model is lrm(firedata~waterrelated.variable)
>
> Thanks a lot and best regards,
> Jan
>
> ***
> "
> In addition to Brian's comment, AIC may be of use. You can't really use
> c-index (ROC area) as it is not sensitive enough for comparing two
> models. But whatever you use, the bad news is that you can't use the
> results to compare more than 2 or 3 completely pre-chosen models or you
> will invalidate inference and estimates if you use these comparisons to
> build a final model.
>
> Frank
> "
> ***
>
>
>
--
Frank E Harrell Jr Professor and Chair School of Medicine
Department of Biostatistics Vanderbilt University
More information about the R-help
mailing list