[R] Relatively Simple Maximization Using Optim Doesnt Optimize

[Duncan Murdoch]

On 12/03/2020 1:22 p.m., Abby Spurdle wrote:
> I'm sorry, Duncan.
> But I disagree.
> 
> This is not a "bug" in optim function, as such.
> (Or at least, there's nothing in this discussion to suggest that there's a bug).
> But rather a floating point arithmetic related problem.
> 
> The OP's function looks simple enough, at first glance.
> But it's not.
> 
> Plotting a numerical approximation of the derivative, makes the
> problem more apparent:

There is nothing in that plot to indicate that the result given by 
optim() should be accepted as optimal.  The numerical approximation to 
the derivative is 0.055851 everywhere in your graph, with numerical 
errors out in the 8th decimal place or later.  Clearly the max occurs 
somewhere to the right of that.  Yes, the 2nd derivative calculation 
will be terrible if R chooses a step size of 0.00001 when calculating 
it, but why would it do that, given that the 1st derivative is 3 orders 
of magnitude larger?


> ----------
> plot_derivative <- function (f, a = sol - offset, b = sol + offset,
> sol, offset=0.001, N=200)
> {   FIRST <- 1:(N - 2)
>      LAST <- 3:N
>      MP <- 2:(N - 1)
> 
>      x <- seq (a, b, length.out=N)
>      y <- f (x)
>      dy <- (y [LAST] - y [FIRST]) / (x [LAST] - x [FIRST])
> 
>      plot (x [MP], dy, type="l", xlab="x", ylab="dy/dx (approx)")
> }
> 
> optim.sol <- optim (1001, production1 ,method="CG", control = list
> (fnscale=-1) )$par
> plot_derivative (production1, sol=optim.sol)
> abline (v=optim.sol, lty=2, col="grey")
> ----------
> 
> So, I would say the optim function (including the CG method) is doing
> what it's supposed to do.
> 
> And collating/expanding on Nash's, Jeff's and Eric's comments:
> (1) An exact solution can be derived quickly, so using a numerical
> method is unnecessary, and inefficient.
> (2) Possible problems with the CG method are noted in the documentation.
> (3) Numerical approximations of the function's derivative need to be
> well-behaved for gradient-based numerical methods to work properly.
> 
> 
> On Fri, Mar 13, 2020 at 3:42 AM Duncan Murdoch <murdoch.duncan using gmail.com> wrote:
>>
>> It looks like a bug in the CG method.  The other methods in optim() all
>> work fine.  CG is documented to be a good choice in high dimensions; why
>> did you choose it for a 1 dim problem?
>>
>> Duncan Murdoch