[Rd] Wrongly converging glm()

[Mark Leeds]

H Ravi: Yes, IIRC that's EXACTLY what was going on in my case. Based on
the codes from Rvmmin,  the objective functon wasn't changing much and I
think norm of the gradient was close to zero so it was difficult for me to
detect the issue. I found it when I
happenered to notice that the objective function seemed independent of one
of the parameters.






On Fri, Jul 21, 2017 at 4:36 PM, Ravi Varadhan <ravi.varadhan at jhu.edu>
wrote:

> “So, what I learned the hard way was  termination due to reasonable
> stopping  criteria DOES NOT NECESSARILY EQUAL OPTIMAL.”
>
>
>
> Yes, I agree, Mark.
>
>
>
> Let me add another observation.  In the “optimx” package, John Nash and I
> implemented a check for optimality conditions – first and second order KKT
> conditions.  This involves checking whether the gradient is sufficiently
> small and the Hessian is positive definite (for local minimum) at the final
> parameter values.  However, it can be quite time consuming to compute these
> quantities and in some problems checking KKT can take up more effort than
> finding the solution!  Furthermore, it is difficult to come up with good
> thresholds for determining “small” gradient and “positive definite”
> Hessian, since these can depend upon the scale of the objective function
> and the parameters.
>
>
>
> Ravi
>
>
>
> *From:* Mark Leeds [mailto:markleeds2 at gmail.com]
> *Sent:* Friday, July 21, 2017 3:09 PM
> *To:* Ravi Varadhan <ravi.varadhan at jhu.edu>
> *Cc:* Therneau, Terry M., Ph.D. <therneau at mayo.edu>; r-devel at r-project.org;
> jorismeys at gmail.com; westra.harmjan at outlook.com
>
> *Subject:* Re: [Rd] Wrongly converging glm()
>
>
>
> Hi Ravi: Well said. In John's Rvmmin package, he has codes for explaining
> the cause
>
> of the termination. The codes returned were fine. The problem was that
>
> the model I was using could have multiple solutions ( regardless of the
> data
> sent in ) so, even though the stopping criteria was reached, it turned out
> that one of the parameters ( there were two parameters ) could have really
> been anything and the same likelihood value would  be returned. So, what I
> learned the hard way was  termination due to reasonable stopping  criteria
> DOES NOT NECESSARILY EQUAL OPTIMAL. But I lived in the dark about this for
> a long time and only happened to notice it when playing around with the
> likelihood by fixing the offending parameter to various values and
> optimizing over the
> non-offending parameter. Thanks for eloquent explanation.
>
>
>         Mark
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Fri, Jul 21, 2017 at 9:22 AM, Ravi Varadhan <ravi.varadhan at jhu.edu>
> wrote:
>
> Please allow me to add my 3 cents.  Stopping an iterative optimization
> algorithm at an "appropriate" juncture is very tricky.  All one can say is
> that the algorithm terminated because it triggered a particular stopping
> criterion.  A good software will tell you why it stopped - i.e. the
> stopping criterion that was triggered.  It is extremely difficult to make a
> failsafe guarantee that the triggered stopping criterion is the correct one
> and that the answer obtained is trustworthy. It is up to the user to
> determine whether the answer makes sense.  In the case of maximizing a
> likelihood function, it is perfectly reasonable to stop when the algorithm
> has not made any progress in increasing the log likelihood.  In this case,
> the software should print out something like "algorithm terminated due to
> lack of improvement in log-likelihood."  Therefore, I don't see a need to
> issue any warning, but simply report the stopping criterion that was
> applied to terminate the algorithm.
>
> Best,
> Ravi
>
> -----Original Message-----
> From: R-devel [mailto:r-devel-bounces at r-project.org] On Behalf Of
> Therneau, Terry M., Ph.D.
> Sent: Friday, July 21, 2017 8:04 AM
> To: r-devel at r-project.org; Mark Leeds <markleeds2 at gmail.com>;
> jorismeys at gmail.com; westra.harmjan at outlook.com
> Subject: Re: [Rd] Wrongly converging glm()
>
> I'm chiming in late since I read the news in digest form, and I won't copy
> the entire conversation to date.
>
> The issue raised comes up quite often in Cox models, so often that the
> Therneau and Grambsch book has a section on the issue (3.5, p 58).  After a
> few initial iterations the offending coefficient will increase by a
> constant at each iteration while the log-likelihood approaches an asymptote
> (essentially once the other coefficients "settle down").
>
> The coxph routine tries to detect this case and print a warning, and this
> turns out to be very hard to do accurately.  I worked hard at tuning the
> threshold(s) for the message several years ago and finally gave up; I am
> guessing that the warning misses > 5% of the cases when the issue is true,
> and that 5% of the warnings that do print are incorrect.
> (And these estimates may be too optimistic.)   Highly correlated
> predictors tend to trip
> it up, e.g., the truncated power spline basis used by the rcs function in
> Hmisc.
>
> All in all, I am not completely sure whether the message does more harm
> than good.  I'd be quite reluctant to go down the same path again with the
> glm function.
>
> Terry Therneau
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
>
>

	[[alternative HTML version deleted]]