[R] Re : Re : How to compare the effect of a variable across regression models?
Bert Gunter
gunter.berton at gene.com
Fri Aug 13 20:49:19 CEST 2010
Comments inline below.
-- Bert
On Fri, Aug 13, 2010 at 11:04 AM, Biau David <djmbiau at yahoo.fr> wrote:
> This is all very interesting indeed.
>
> so I appreciate that the effect of a variable will depend on the presence of
> other variables and that the effect of this variable has a statistical
> meaning only in a specific model. With the particularity of inconsistency if
> data arise from non normal distribution as for the Cox model.
-- ?? No idea what you mean here -- do you?
>
> I have two comments though; the first is statistical and the second more
> methodological:
>
> - first, either I am strict on the inconsistency phenomenon and, given the
> fact that it is very unlikely that I even come close to finding all relevant
> variables to a model, all models I may build are necessarily biased OR I
> have to be less strict.
-- "All models are wrong, but some are useful." -- George E.P. Box
You realize, of course, that "The Standard Model" is also a biased
model... but a rather accurate one over a large range of real
conditions. (Alas, it does seem to miss about 95% of the observed mass
in the universe, but that's a detail...).
How to use tests (score, Wald, LR) to compare models
> when, at most, only one of them is unbiased?
-- Not at all?
>
> - second (relevance first depend on the answer to comment above I guess), I
> study the effect of age without adustment (model 1), after adjusting for
> presentation variables (model 2) and after adjusting for presentation and
> treatment variables (model 3). I certainly do not pretend that I am truly
> looking at the effect of age and I know that what I call age across the
> different models is not the same (otherwise I would not compare it or I
> would hope not to find any difference). I am merely looking the effect of
> something that is measured by 'age' and that emcompasses imbalances at
> presentation, treatment, and remaining unknown counfounders (first model),
> imbalances in treatment and remaining unknown counfounders (second model),
> and imbalances in the remaining confounders (third model). The remaining
> unknown confounders potentially include a true effect of age, if such a
> thing exists, and of other variables I don't know about (which in fact are
> still presentation and treatment variables since there are no other
> possibilities!).
>
> As everything fits in a cohesive interpretation: older patients presents
> with worse tumors and are undertreated (as shown by descriptive statistics)
> AND as this is shown by the modelisation of the effect by an erosion of the
> effect of 'age' from model 1 to 3, I was tempted, with all due precautions,
> to conclude that the increased risk for older patients was due in part
> because they present with worse tumors, in part because the are
> undertreated, and in part because of something else. Would that be
> overinterpreting the data? Should I even drop the idea of building and
> presenting these models?
-- No clue, but it sounds like, as is often the case, you should
consult your local statistician to help you gain some clarity.
>
> Thanks again,
>
> David Biau.
>
> ________________________________
> De : Bert Gunter <gunter.berton at gene.com>
> À : Biau David <djmbiau at yahoo.fr>
> Cc : Frank Harrell <f.harrell at vanderbilt.edu>; r help list
> <r-help at r-project.org>
> Envoyé le : Ven 13 août 2010, 18h 22min 58s
> Objet : Re: [R] Re : How to compare the effect of a variable across
> regression models?
>
> Just to amplify a bit on what Frank said...
>
> Except in special circumstances (othogonal designs, say), regression
> models are only "guaranteed" to produce useful predictions -- they may
> not tell you anything meaningful about the relative effects of the
> regressors because, as Frank said, that depends on both the study
> design AND the true effects. So, for example, typically if one
> removes a regressor from a model and refits, the values of the
> remaining coefficients will change.
>
> Moreover, it is difficult to understand even conceptually what "the
> effect of a variable" should mean if, as is typical, it interacts with
> other variables: the "effect" of the variable then depends on what the
> values of the other variables are.
>
> A very simple example of this that I used to use in teaching was the
> effect of barbituates and alcohol on sleepiness. In the absence of
> barbituates, the effect of a few drinks is to make you modestly
> sleepy; in the presence of barbituates, the effect of a few drinks is
> to make you modestly dead!
>
> The point is that we live in a multivariate interactive world. Terms
> like "the effect of a variable" derive from univariate thinking and
> need to be very carefully defined to make sense in such a world -- and
> then studies need to be carefully designed -- happenstance data are
> rarely sufficient -- to quantify them.
>
> .. Not what scientists like to hear, I think, but this is the reality.
>
> Further comments and criticisms welcome, of course.
>
> Cheers,
> Bert Gunter
> Genentech Nonclinical Statistics
>
> On Fri, Aug 13, 2010 at 6:59 AM, Biau David <djmbiau at yahoo.fr> wrote:
>> OK,
>>
>> thank you very much for the answer.I will look into that. Hopefully I'll
>> find
>> smoething that will work out.
>>
>> Best,
>>
>> David Biau.
>>
>>
>>
>>
>> ________________________________
>> De : Frank Harrell <f.harrell at vanderbilt.edu>
>>
>> Cc : r help list <r-help at r-project.org>
>> Envoyé le : Ven 13 août 2010, 15h 50min 18s
>> Objet : Re: [R] How to compare the effect of a variable across regression
>> models?
>>
>>
>> David,
>>
>> In the Cox and many other regression models, the effect of a variable is
>> context-dependent. There is an identifiability problem in what you are
>> doing,
>> as discussed by
>>
>> @ARTICLE{for95mod,
>> author = {Ford, Ian and Norrie, John and Ahmadi, Susan},
>> year = 1995,
>> title = {Model inconsistency, illustrated by the {Cox} proportional
>> hazards
>> model},
>> journal = Stat in Med,
>> volume = 14,
>> pages = {735-746},
>> annote = {covariable adjustment; adjusted estimates; baseline imbalances;
>> RCT; model misspecification; model identification}
>> }
>>
>> One possible remedy, which may not work for your goals, is to embed all
>> models
>> in a grand model that is used for inference.
>>
>> When coefficients ARE comparable in some sense, you can use the bootstrap
>> to get
>> confidence bands for differences in regressor effects between models.
>>
>> Frank
>>
>> Frank E Harrell Jr Professor and Chairman School of Medicine
>> Department of Biostatistics Vanderbilt University
>>
>> On Fri, 13 Aug 2010, Biau David wrote:
>>
>>> Hello,
>>>
>>> I would like, if it is possible, to compare the effect of a variable
>>> across
>>> regression models. I have looked around but I haven't found anything.
>>> Maybe
>>> someone could help? Here is the problem:
>>>
>>> I am studying the effect of a variable (age) on an outcome (local
>>> recurrence:
>>> lr). I have built 3 models:
>>> - model 1: lr ~ age y = \beta_(a1).age
>>> - model 2: lr ~ age + presentation variables (X_p) y =
>>> \beta_(a2).age
>> +
>>> \BETA_(p2).X_p
>>> - model 3: lr ~ age + presentation variables + treatment variables( X_t)
>>> y = \beta_(a3).age + \BETA_(p3).X_(p) + \BETA_(t3).X_t
>>>
>>> Presentation variables include variables such as tumor grade, tumor size,
>>>etc...
>>> the physician cannot interfer with these variables.
>>> Treatment variables include variables such as chemotherapy, radiation,
>> surgical
>>> margins (a surrogate for adequate surgery).
>>>
>>> I have used cph for the models and restricted cubic splines (Design
>>> library)
>>>for
>>> age. I have noted that the effect of age decreases from model 1 to 3.
>>>
>>> I would like to compare the effect of age on the outcome across the
>>> different
>>> models. A test of \beta_(a1) = \beta_(a2) = \beta_(a3) and then two by
>>> two
>>> comparisons or a global trend test maybe? Is that possible?
>>>
>>> Thank you for your help,
>>>
>>>
>>> David Biau.
>>>
>>>
>>>
>>>
>>> [[alternative HTML version deleted]]
>>>
>>> ______________________________________________
>>> R-help at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide
>>> http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>>
>>
>>
>>
>>
>> [[alternative HTML version deleted]]
>>
>>
>> ______________________________________________
>> R-help at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>>
>
>
--
Bert Gunter
Genentech Nonclinical Biostatistics
467-7374
http://devo.gene.com/groups/devo/depts/ncb/home.shtml
More information about the R-help
mailing list