[R] Re : How to compare the effect of a variable across regression models?

[Bert Gunter]

Just to amplify a bit on what Frank said...

Except in special circumstances (othogonal designs, say), regression
models are only "guaranteed" to produce useful predictions -- they may
not tell you anything meaningful about the relative effects of the
regressors because, as Frank said, that depends on both the study
design AND the true effects.  So, for example, typically if one
removes a regressor from a model and refits, the values of the
remaining coefficients will change.

Moreover, it is difficult to understand even conceptually what "the
effect of a variable" should mean if, as is typical, it interacts with
other variables: the "effect" of the variable then depends on what the
values of the other variables are.

A very simple example of this that I used to use in teaching was the
effect of barbituates and alcohol on sleepiness. In the absence of
barbituates, the effect of a few drinks is to make you modestly
sleepy; in the presence of barbituates, the effect of a few drinks is
to make you modestly dead!

The point is that we live in a multivariate interactive world. Terms
like "the effect of a variable" derive from univariate thinking and
need to be very carefully defined to make sense in such a world -- and
then studies need to be carefully designed -- happenstance data are
rarely sufficient -- to quantify them.

.. Not what scientists like to hear, I think, but this is the reality.

Further comments and criticisms welcome, of course.

Cheers,
Bert Gunter
Genentech Nonclinical Statistics

On Fri, Aug 13, 2010 at 6:59 AM, Biau David <djmbiau at yahoo.fr> wrote:
> OK,
>
> thank you very much for the answer.I will look into that. Hopefully I'll find
> smoething that will work out.
>
> Best,
>
>  David Biau.
>
>
>
>
> ________________________________
> De : Frank Harrell <f.harrell at vanderbilt.edu>
>
> Cc : r help list <r-help at r-project.org>
> Envoyé le : Ven 13 août 2010, 15h 50min 18s
> Objet : Re: [R] How to compare the effect of a variable across regression
> models?
>
>
> David,
>
> In the Cox and many other regression models, the effect of a variable is
> context-dependent.  There is an identifiability problem in what you are doing,
> as discussed by
>
> @ARTICLE{for95mod,
>  author = {Ford, Ian and Norrie, John and Ahmadi, Susan},
>  year = 1995,
>  title = {Model inconsistency, illustrated by the {Cox} proportional hazards
>          model},
>  journal = Stat in Med,
>  volume = 14,
>  pages = {735-746},
>  annote = {covariable adjustment; adjusted estimates; baseline imbalances;
>           RCT; model misspecification; model identification}
> }
>
> One possible remedy, which may not work for your goals, is to embed all models
> in a grand model that is used for inference.
>
> When coefficients ARE comparable in some sense, you can use the bootstrap to get
> confidence bands for differences in regressor effects between models.
>
> Frank
>
> Frank E Harrell Jr   Professor and Chairman        School of Medicine
>                     Department of Biostatistics   Vanderbilt University
>
> On Fri, 13 Aug 2010, Biau David wrote:
>
>> Hello,
>>
>> I would like, if it is possible, to compare the effect of a variable across
>> regression models. I have looked around but I haven't found anything. Maybe
>> someone could help? Here is the problem:
>>
>> I am studying the effect of a variable (age) on an outcome (local recurrence:
>> lr). I have built 3 models:
>> - model 1: lr ~ age      y = \beta_(a1).age
>> - model 2: lr ~ age +  presentation variables (X_p)        y = \beta_(a2).age
> +
>> \BETA_(p2).X_p
>> - model 3: lr ~ age + presentation variables + treatment variables( X_t)
>>       y = \beta_(a3).age  + \BETA_(p3).X_(p) + \BETA_(t3).X_t
>>
>> Presentation variables include variables such as tumor grade, tumor size,
>>etc...
>> the physician cannot interfer with these variables.
>> Treatment variables include variables such as chemotherapy, radiation,
> surgical
>> margins (a surrogate for adequate surgery).
>>
>> I have used cph for the models and restricted cubic splines (Design library)
>>for
>> age. I have noted that the effect of age decreases from model 1 to 3.
>>
>> I would like to compare the effect of age on the outcome across the different
>> models. A test of \beta_(a1) = \beta_(a2) = \beta_(a3) and then two by two
>> comparisons or a global trend test maybe? Is that possible?
>>
>> Thank you for your help,
>>
>>
>> David Biau.
>>
>>
>>
>>
>>     [[alternative HTML version deleted]]
>>
>> ______________________________________________
>> R-help at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
>
>
>
>        [[alternative HTML version deleted]]
>
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
>