Dear Suman,

Im not quite sure if I understand your question. I understand by
calibration, correcting or distorting model coefficients due to some
penalization, such as lasso or bootstraping. Thus, p values in this
scenarios does not make much sense to me. The issue is how much should you
penalize the model to prevent overfitting. Updating calibration assumes that
you have a estimation dataset and a validation dataset, if you decide that
the model performance is not similar in both, thus it is reasonable to go on
calibrating/updating the model.

But perhaps you are talking about comparing two different models already
calibrated? If it is so, then some may argue that it is possible to test
areas or partial areas under ROC curves or Brier scores of predicted values
through hypothesis tests. Although never seen it from logistic models and
does not make much sense to me either.

Kind regards,

Abraço forte e que a força esteja com você,

Dr. Pedro Emmanuel A. A. do Brasil
Instituto de Pesquisa Clínica Evandro Chagas
Fundação Oswaldo Cruz
Rio de Janeiro - Brasil
Av. Brasil 4365
Tel 55 21 3865-9648
email: pedro.brasil@ipec.fiocruz.br
email: emmanuel.brasil@gmail.com

---Apoio aos softwares livres
www.zotero.org - gerenciamento de referências bibliográficas.
www.broffice.org ou www.openoffice.org - textos, planilhas ou apresentações.
www.epidata.dk - entrada de dados.
www.r-project.org - análise de dados.
www.ubuntu.com - sistema operacional


2011/2/17 <r-sig-epi-request@r-project.org>

> Send R-sig-Epi mailing list submissions to
>        r-sig-epi@r-project.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>        https://stat.ethz.ch/mailman/listinfo/r-sig-epi
> or, via email, send a message with subject or body 'help' to
>        r-sig-epi-request@r-project.org
>
> You can reach the person managing the list at
>        r-sig-epi-owner@r-project.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of R-sig-Epi digest..."
>
>
> Today's Topics:
>
>   1. Compare calibration from two LR models (Suman Kundu)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Wed, 16 Feb 2011 06:54:26 -0800 (PST)
> From: Suman Kundu <suman_math@yahoo.com>
> To: r-sig-epi@r-project.org
> Subject: [R-sig-Epi] Compare calibration from two LR models
> Message-ID: <830070.74768.qm@web112509.mail.gq1.yahoo.com>
> Content-Type: text/plain
>
> Dear Friends,
>
> I have the following question on comparing calibration (using
> Hosmer-Lemeshow test) of two logistic regression models based on reported
> p-value.
>
> Can we use p-values for comparing calibration for two different logistic
> regression models? If one model has p value .2 and another model has .8, is
> it justified to conclude that second model is better calibrated compare to
> the first model ? [My intuition says NO, because p- value only measure of
> consistency between the observed data and the null hypothesis. And, here the
> null hypothesis is that the model is well calibrated. p-value does not tell
> about how strong the calibration is.]
>
> Thank you for your response.
>
> Best regards,
> Suman Kundu
> PhD student.
> Erasmus MC, Rotterdam
>
>
>
>
>        [[alternative HTML version deleted]]
>
>
>
> ------------------------------
>
> _______________________________________________
> R-sig-Epi mailing list
> R-sig-Epi@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-epi
>
>
> End of R-sig-Epi Digest, Vol 50, Issue 3
> ****************************************
>

	[[alternative HTML version deleted]]

