This is all very interesting indeed.

so I appreciate that the effect of a variable will depend on the presence of 
other variables and that the effect of this variable has a statistical meaning 
only in a specific model. With the particularity of inconsistency if data arise 
from non normal distribution as for the Cox model.

I have two comments though; the first is statistical and the second more 
methodological:

- first, either I am strict on the inconsistency phenomenon and, given the fact 
that it is very unlikely that I even come close to finding all relevant 
variables to a model, all models I may build are necessarily biased OR I have to 
be less strict. How to use tests (score, Wald, LR) to compare models when, at 
most, only one of them is unbiased?

- second (relevance first depend on the answer to comment above I guess), I 
study the effect of age without adustment (model 1), after adjusting for 
presentation variables (model 2) and after adjusting for presentation and 
treatment variables  (model 3). I certainly do not pretend that I am truly 
looking at the effect of age and I know that what I call age across the 
different models is not the same (otherwise I would not compare it or I would 
hope not to find any difference). I am merely looking the effect of something 
that is measured by 'age' and that emcompasses imbalances at presentation, 
treatment, and remaining unknown counfounders (first model), imbalances in 
treatment and remaining unknown counfounders (second model), and imbalances in 
the remaining confounders (third model). The remaining unknown confounders 
potentially include a true effect of age, if such a thing exists, and of other 
variables I don't know about (which in fact are still presentation and treatment 
variables since there are no other possibilities!).

As everything fits in a cohesive interpretation: older patients presents with 
worse tumors and are undertreated (as shown by descriptive statistics) AND as 
this is shown by the modelisation of the effect by an erosion of the effect of 
'age' from model 1 to 3, I was tempted, with all due precautions, to conclude 
that the increased risk for older patients was due in part because they present 
with worse tumors, in part because the are undertreated, and in part because of 
something else. Would that be overinterpreting the data? Should I even drop the 
idea of building and presenting these models?

Thanks again,

 David Biau.




________________________________
De : Bert Gunter <gunter.berton@gene.com>

Cc : Frank Harrell <f.harrell@vanderbilt.edu>; r help list 
<r-help@r-project.org>
Envoyé le : Ven 13 août 2010, 18h 22min 58s
Objet : Re: [R] Re : How to compare the effect of a variable across regression 
models?

Just to amplify a bit on what Frank said...

Except in special circumstances (othogonal designs, say), regression
models are only "guaranteed" to produce useful predictions -- they may
not tell you anything meaningful about the relative effects of the
regressors because, as Frank said, that depends on both the study
design AND the true effects.  So, for example, typically if one
removes a regressor from a model and refits, the values of the
remaining coefficients will change.

Moreover, it is difficult to understand even conceptually what "the
effect of a variable" should mean if, as is typical, it interacts with
other variables: the "effect" of the variable then depends on what the
values of the other variables are.

A very simple example of this that I used to use in teaching was the
effect of barbituates and alcohol on sleepiness. In the absence of
barbituates, the effect of a few drinks is to make you modestly
sleepy; in the presence of barbituates, the effect of a few drinks is
[[elided Yahoo spam]]

The point is that we live in a multivariate interactive world. Terms
like "the effect of a variable" derive from univariate thinking and
need to be very carefully defined to make sense in such a world -- and
then studies need to be carefully designed -- happenstance data are
rarely sufficient -- to quantify them.

.. Not what scientists like to hear, I think, but this is the reality.

Further comments and criticisms welcome, of course.

Cheers,
Bert Gunter
Genentech Nonclinical Statistics


> OK,
>
> thank you very much for the answer.I will look into that. Hopefully I'll find
> smoething that will work out.
>
> Best,
>
>  David Biau.
>
>
>
>
> ________________________________
> De : Frank Harrell <f.harrell@vanderbilt.edu>
>
> Cc : r help list <r-help@r-project.org>
> Envoyé le : Ven 13 août 2010, 15h 50min 18s
> Objet : Re: [R] How to compare the effect of a variable across regression
> models?
>
>
> David,
>
> In the Cox and many other regression models, the effect of a variable is
> context-dependent.  There is an identifiability problem in what you are doing,
> as discussed by
>
> @ARTICLE{for95mod,
>  author = {Ford, Ian and Norrie, John and Ahmadi, Susan},
>  year = 1995,
>  title = {Model inconsistency, illustrated by the {Cox} proportional hazards
>          model},
>  journal = Stat in Med,
>  volume = 14,
>  pages = {735-746},
>  annote = {covariable adjustment; adjusted estimates; baseline imbalances;
>           RCT; model misspecification; model identification}
> }
>
> One possible remedy, which may not work for your goals, is to embed all models
> in a grand model that is used for inference.
>
> When coefficients ARE comparable in some sense, you can use the bootstrap to 
>get
> confidence bands for differences in regressor effects between models.
>
> Frank
>
> Frank E Harrell Jr   Professor and Chairman        School of Medicine
>                     Department of Biostatistics   Vanderbilt University
>
> On Fri, 13 Aug 2010, Biau David wrote:
>
>> Hello,
>>
>> I would like, if it is possible, to compare the effect of a variable across
>> regression models. I have looked around but I haven't found anything. Maybe
>> someone could help? Here is the problem:
>>
>> I am studying the effect of a variable (age) on an outcome (local recurrence:
>> lr). I have built 3 models:
>> - model 1: lr ~ age      y = \beta_(a1).age
>> - model 2: lr ~ age +  presentation variables (X_p)        y = \beta_(a2).age
> +
>> \BETA_(p2).X_p
>> - model 3: lr ~ age + presentation variables + treatment variables( X_t)
>>       y = \beta_(a3).age  + \BETA_(p3).X_(p) + \BETA_(t3).X_t
>>
>> Presentation variables include variables such as tumor grade, tumor size,
>>etc...
>> the physician cannot interfer with these variables.
>> Treatment variables include variables such as chemotherapy, radiation,
> surgical
>> margins (a surrogate for adequate surgery).
>>
>> I have used cph for the models and restricted cubic splines (Design library)
>>for
>> age. I have noted that the effect of age decreases from model 1 to 3.
>>
>> I would like to compare the effect of age on the outcome across the different
>> models. A test of \beta_(a1) = \beta_(a2) = \beta_(a3) and then two by two
>> comparisons or a global trend test maybe? Is that possible?
>>
>> Thank you for your help,
>>
>>
>> David Biau.
>>
>>
>>
>>
>>     [[alternative HTML version deleted]]
>>
>> ______________________________________________
>> R-help@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
>
>
>
>        [[alternative HTML version deleted]]
>
>
> ______________________________________________
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
>



      
	[[alternative HTML version deleted]]

