[R] fundamental guide to use of numerical optimizers?

[cberry at tajo.ucsd.edu]

Paul Johnson <pauljohn32 at gmail.com> writes:

> I was in a presentation of optimizations fitted with both MPlus and
> SAS yesterday.  In a batch of 1000 bootstrap samples, between 300 and
> 400 of the estimations did not converge.  The authors spoke as if this
> were the ordinary cost of doing business, and pointed to some
> publications in which the nonconvergence rate was as high or higher.
>
> I just don't believe that's right, and if some problem is posed so
> that the estimate is not obtained in such a large sample of
> applications, it either means the problem is badly asked or badly
> answered.  But I've got no traction unless I can actually do
> better....

A few years back there was a brouhaha in which a too lax convergence
criterion in the Splus gam() function resulted in wrong results. 

See

        http://www.ihapss.jhsph.edu/publications/Results/nmmaps_faq.htm

I think this was also reported in the lay press.

IIRC, at that time there was an assertion that gam() was buggy, but it
turned out that for the particular problem a more stringent tolerance
was needed than the default provided. The original report used results
that hadn't actually converged.

<rant> The trouble is there are many instances of monkey-see, monkey-do
data analysis. It seems that some authors do not really want to dig into
their data if the story it tells is not simple and firmly supported. And
not understanding why many bootstrap samples do not converge seems like
an instance of sweeping data-dirt under the rug.</rant>

The questions you ask below full under the rubric of 'numerical
analysis'. You might look here to start:

           http://en.wikipedia.org/wiki/Numerical_analysis

Chuck

>
> Perhaps I can use this opportunity to learn about R functions like
> optim, or perhaps maxLik.
>
>>From reading r-help, it seems to me there are some basic tips for
> optimization, such as:
>
> 1. It is wise to scale the data so that all columns have the same
> range before running an optimizer.
>
> 2. With estimates of variance parameters, don't try to estimate sigma
> directly, instead estimate log(sigma) because that puts the domain of
> the solution onto the real number line.
>
> 3 With estimates of proportions, estimate instead the logit, for the
> same reason.
>
> Are these mistaken generalizations?  Are there other tips that
> everybody ought to know?
>
> I understand this is a vague question, perhaps the answers are just in
> the folklore. But if somebody has written them out, I would be glad to
> know.

-- 
Charles C. Berry                            Dept of Family/Preventive Medicine
ccberry at ucsd edu			    UC San Diego
http://famprevmed.ucsd.edu/faculty/cberry/  La Jolla, San Diego 92093-0901