[R] Re : Re : How to compare the effect of a variable across regression models?
Frank Harrell
f.harrell at vanderbilt.edu
Fri Aug 13 20:37:24 CEST 2010
Frank E Harrell Jr Professor and Chairman School of Medicine
Department of Biostatistics Vanderbilt University
On Fri, 13 Aug 2010, Biau David wrote:
> This is all very interesting indeed.
>
> so I appreciate that the effect of a variable will depend on the
> presence of other variables and that the effect of this variable has
> a statistical meaning only in a specific model. With the
> particularity of inconsistency if data arise from non normal
> distribution as for the Cox model.
It's worse than that. In the Cox PH model the degree of non
proportional hazards depends on which variables are adjusted for.
>
> I have two comments though; the first is statistical and the second more methodological:
>
> - first, either I am strict on the inconsistency phenomenon and, given the fact that it is very unlikely that I even come close to
> finding all relevant variables to a model, all models I may build are necessarily biased OR I have to be less strict. How to use tests
> (score, Wald, LR) to compare models when, at most, only one of them is unbiased?
>
> - second (relevance first depend on the answer to comment above I guess), I study the effect of age without adustment (model 1), after
> adjusting for presentation variables (model 2) and after adjusting for presentation and treatment variables (model 3). I certainly do not
> pretend that I am truly looking at the effect of age and I know that what I call age across the different models is not the same
> (otherwise I would not compare it or I would hope not to find any difference). I am merely looking the effect of something that is
> measured by 'age' and that emcompasses imbalances at presentation, treatment, and remaining unknown counfounders (first model),
> imbalances in treatment and remaining unknown counfounders (second model), and imbalances in the remaining confounders (third model). The
> remaining unknown confounders potentially include a true effect of age, if such a thing exists, and of other variables I don't know about
> (which in fact are still presentation and treatment variables since there are no other possibilities!).
>
> As everything fits in a cohesive interpretation: older patients presents with worse tumors and are undertreated (as shown by descriptive
> statistics) AND as this is shown by the modelisation of the effect by an erosion of the effect of 'age' from model 1 to 3, I was tempted,
> with all due precautions, to conclude that the increased risk for older patients was due in part because they present with worse tumors,
> in part because the are undertreated, and in part because of something else. Would that be overinterpreting the data? Should I even drop
> the idea of building and presenting these models?
I'm not sure. But often we learn by predicting a predictor from the
other predictors.
Frank
>
> Thanks again,
>
> David Biau.
>
> _________________________________________________________________________________________________________________________________________
> De : Bert Gunter <gunter.berton at gene.com>
> À : Biau David <djmbiau at yahoo.fr>
> Cc : Frank Harrell <f.harrell at vanderbilt.edu>; r help list <r-help at r-project.org>
> Envoyé le : Ven 13 août 2010, 18h 22min 58s
> Objet : Re: [R] Re : How to compare the effect of a variable across regression models?
>
> Just to amplify a bit on what Frank said...
>
> Except in special circumstances (othogonal designs, say), regression
> models are only "guaranteed" to produce useful predictions -- they may
> not tell you anything meaningful about the relative effects of the
> regressors because, as Frank said, that depends on both the study
> design AND the true effects. So, for example, typically if one
> removes a regressor from a model and refits, the values of the
> remaining coefficients will change.
>
> Moreover, it is difficult to understand even conceptually what "the
> effect of a variable" should mean if, as is typical, it interacts with
> other variables: the "effect" of the variable then depends on what the
> values of the other variables are.
>
> A very simple example of this that I used to use in teaching was the
> effect of barbituates and alcohol on sleepiness. In the absence of
> barbituates, the effect of a few drinks is to make you modestly
> sleepy; in the presence of barbituates, the effect of a few drinks is
> to make you modestly dead!
>
> The point is that we live in a multivariate interactive world. Terms
> like "the effect of a variable" derive from univariate thinking and
> need to be very carefully defined to make sense in such a world -- and
> then studies need to be carefully designed -- happenstance data are
> rarely sufficient -- to quantify them.
>
> .. Not what scientists like to hear, I think, but this is the reality.
>
> Further comments and criticisms welcome, of course.
>
> Cheers,
> Bert Gunter
> Genentech Nonclinical Statistics
>
> On Fri, Aug 13, 2010 at 6:59 AM, Biau David <djmbiau at yahoo.fr> wrote:
> > OK,
> >
> > thank you very much for the answer.I will look into that. Hopefully I'll find
> > smoething that will work out.
> >
> > Best,
> >
> > David Biau.
> >
> >
> >
> >
> > ________________________________
> > De : Frank Harrell <f.harrell at vanderbilt.edu>
> >
> > Cc : r help list <r-help at r-project.org>
> > Envoyé le : Ven 13 août 2010, 15h 50min 18s
> > Objet : Re: [R] How to compare the effect of a variable across regression
> > models?
> >
> >
> > David,
> >
> > In the Cox and many other regression models, the effect of a variable is
> > context-dependent. There is an identifiability problem in what you are doing,
> > as discussed by
> >
> > @ARTICLE{for95mod,
> > author = {Ford, Ian and Norrie, John and Ahmadi, Susan},
> > year = 1995,
> > title = {Model inconsistency, illustrated by the {Cox} proportional hazards
> > model},
> > journal = Stat in Med,
> > volume = 14,
> > pages = {735-746},
> > annote = {covariable adjustment; adjusted estimates; baseline imbalances;
> > RCT; model misspecification; model identification}
> > }
> >
> > One possible remedy, which may not work for your goals, is to embed all models
> > in a grand model that is used for inference.
> >
> > When coefficients ARE comparable in some sense, you can use the bootstrap to get
> > confidence bands for differences in regressor effects between models.
> >
> > Frank
> >
> > Frank E Harrell Jr Professor and Chairman School of Medicine
> > Department of Biostatistics Vanderbilt University
> >
> > On Fri, 13 Aug 2010, Biau David wrote:
> >
> >> Hello,
> >>
> >> I would like, if it is possible, to compare the effect of a variable across
> >> regression models. I have looked around but I haven't found anything. Maybe
> >> someone could help? Here is the problem:
> >>
> >> I am studying the effect of a variable (age) on an outcome (local recurrence:
> >> lr). I have built 3 models:
> >> - model 1: lr ~ age y = \beta_(a1).age
> >> - model 2: lr ~ age + presentation variables (X_p) y = \beta_(a2).age
> > +
> >> \BETA_(p2).X_p
> >> - model 3: lr ~ age + presentation variables + treatment variables( X_t)
> >> y = \beta_(a3).age + \BETA_(p3).X_(p) + \BETA_(t3).X_t
> >>
> >> Presentation variables include variables such as tumor grade, tumor size,
> >>etc...
> >> the physician cannot interfer with these variables.
> >> Treatment variables include variables such as chemotherapy, radiation,
> > surgical
> >> margins (a surrogate for adequate surgery).
> >>
> >> I have used cph for the models and restricted cubic splines (Design library)
> >>for
> >> age. I have noted that the effect of age decreases from model 1 to 3.
> >>
> >> I would like to compare the effect of age on the outcome across the different
> >> models. A test of \beta_(a1) = \beta_(a2) = \beta_(a3) and then two by two
> >> comparisons or a global trend test maybe? Is that possible?
> >>
> >> Thank you for your help,
> >>
> >>
> >> David Biau.
> >>
> >>
> >>
> >>
> >> [[alternative HTML version deleted]]
> >>
> >> ______________________________________________
> >> R-help at r-project.org mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-help
> >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> >> and provide commented, minimal, self-contained, reproducible code.
> >>
> >
> >
> >
> >
> > [[alternative HTML version deleted]]
> >
> >
> > ______________________________________________
> > R-help at r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
> >
> >
>
>
>
More information about the R-help
mailing list