[R] Bad optimization solution

[Sundar Dorai-Raj]

Paul Smith said the following on 5/7/2007 3:25 PM:
> On 5/7/07, Paul Smith <phhs80 at gmail.com> wrote:
>>> I think the problem is the starting point.  I do not remember the details
>>> of the BFGS method, but I am almost sure the (.5, .5) starting point is
>>> suspect, since the abs function is not differentiable at 0.  If you perturb
>>> the starting point even slightly you will have no problem.
>>>
>>>              "Paul Smith"
>>>              <phhs80 at gmail.com
>>>              >                                                          To
>>>              Sent by:                  R-help <r-help at stat.math.ethz.ch>
>>>              r-help-bounces at st                                          cc
>>>              at.math.ethz.ch
>>>                                                                    Subject
>>>                                        [R] Bad optimization solution
>>>              05/07/2007 04:30
>>>              PM
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> Dear All
>>>
>>> I am trying to perform the below optimization problem, but getting
>>> (0.5,0.5) as optimal solution, which is wrong; the correct solution
>>> should be (1,0) or (0,1).
>>>
>>> Am I doing something wrong? I am using R 2.5.0 on Fedora Core 6 (Linux).
>>>
>>> Thanks in advance,
>>>
>>> Paul
>>>
>>> ------------------------------------------------------
>>> myfunc <- function(x) {
>>>   x1 <- x[1]
>>>   x2 <- x[2]
>>>   abs(x1-x2)
>>> }
>>>
>>> optim(c(0.5,0.5),myfunc,lower=c(0,0),upper=c(1,1),method="L-BFGS-B",control=list(fnscale=-1))
>> Yes, with (0.2,0.9), a correct solution comes out. However, how can
>> one be sure in general that the solution obtained by optim is correct?
>> In ?optim says:
>>
>>      Method '"L-BFGS-B"' is that of Byrd _et. al._ (1995) which allows
>>      _box constraints_, that is each variable can be given a lower
>>      and/or upper bound. The initial value must satisfy the
>>      constraints. This uses a limited-memory modification of the BFGS
>>      quasi-Newton method. If non-trivial bounds are supplied, this
>>      method will be selected, with a warning.
>>
>> which only demands that "the initial value must satisfy the constraints".
> 
> Furthermore, X^2 is everywhere differentiable and notwithstanding the
> reported problem occurs with
> 
> myfunc <- function(x) {
>   x1 <- x[1]
>   x2 <- x[2]
>   (x1-x2)^2
> }
> 
> optim(c(0.2,0.2),myfunc,lower=c(0,0),upper=c(1,1),method="L-BFGS-B",control=list(fnscale=-1))
> 
> Paul
> 

Then perhaps supply the gradient:

mygrad <- function(x) {
   x1 <- x[1]
   x2 <- x[2]
   c(2, -2) * c(x1, x2)
}

optim(c(0.2,0.2),myfunc,mygrad,lower=c(0,0),upper=c(1,1),
       method="L-BFGS-B",control=list(fnscale=-1))

HTH,

--sundar