[R] Trouble with optim

[Damon Wischik]

> I thinks it has found a *local* minimum.  It is not a global optimizer.

It has not found a local minimum. The derivatives at its last guess are
these:

d/dprob: -9.4, at prob=.7, 
  function is pretty much linear for prob in [.2,1]
d/dmu: 0.0002, at mu=21, 
  function is pretty much linear for mu in [5,30]
  (Yes, a clear sign of a badly scaled problem!)
d/dtheta: -6900, at theta=.001, 
  function has a vertical asymptote tending to +Inf as theta->0
  (though to make the function finite, I bounded it to theta>=.001)  

It might believe it's found a local minimum in mu, but it oughtn't to
think it's found one in at least one of the other parameters. If it does
think it has found a local minimum, why does it think so?

I'm still puzzled by its behaviour at the last points it visits:

Eval fn at 0.7425713 21.12820 0.001 --- Val = 64.99
Eval fn at 0.7425713 21.12920 0.002 --- Val = 60.20449
Eval fn at 0.7425713 21.12920 0.001 --- Val = 64.99

it visits a good point, and then straight away after settles on a worse!

> You appear to have made no attempt to scale the problem as the help page 
> suggests you need to.  Please look into this, as from what little 
> information you give I think the initial step has overshot.

I did note that suggestion, both in the help page and in MASS (4th ed). In
this particular case, I can see how to scale things (I can even
provide derivatives). (Although the loglik function, as a function of
theta, has a vertical asymptote at theta=0, so I guess I would have to
reexpress the loglik function as a function of some transform of theta).

But this is part of a general model-fitting routine, with vectors for
prob, mu and theta according to factors, in which it is not at all clear
to me how to scale parameters. Does anyone have any general
recommendations?

Damon.


In the unlikely event that the following is of interest, here is a trace
of the values it visits

Eval fn at 0.7 10.3 8.5 --- Val = 42.70597
Eval fn at 0.701 10.3 8.5 --- Val = 42.70595
Eval fn at 0.699 10.3 8.5 --- Val = 42.70603
Eval fn at 0.7 10.301 8.5 --- Val = 42.70462
Eval fn at 0.7 10.299 8.5 --- Val = 42.70732
Eval fn at 0.7 10.3 8.501 --- Val = 42.70698
Eval fn at 0.7 10.3 8.499 --- Val = 42.70496
Eval fn at 0.723089 11.10079 7.901503 --- Val = 41.20297
Eval fn at 0.724089 11.10079 7.901503 --- Val = 41.20411
Eval fn at 0.722089 11.10079 7.901503 --- Val = 41.20188
Eval fn at 0.723089 11.10179 7.901503 --- Val = 41.20203
Eval fn at 0.723089 11.09979 7.901503 --- Val = 41.20392
Eval fn at 0.723089 11.10079 7.902503 --- Val = 41.20397
Eval fn at 0.723089 11.10079 7.900503 --- Val = 41.20197
Eval fn at 0.001 13.98670 5.534439 --- Val = 79.65909
Eval fn at 0.002 13.98670 5.534439 --- Val = 74.81006
Eval fn at 0.001 13.98670 5.534439 --- Val = 79.65909
Eval fn at 0.001 13.98770 5.534439 --- Val = 79.65899
Eval fn at 0.001 13.98570 5.534439 --- Val = 79.6592
Eval fn at 0.001 13.98670 5.535439 --- Val = 79.6602
Eval fn at 0.001 13.98670 5.533439 --- Val = 79.65797
Eval fn at 0.4611668 12.14759 7.042902 --- Val = 40.72491
Eval fn at 0.4621668 12.14759 7.042902 --- Val = 40.71531
Eval fn at 0.4601668 12.14759 7.042902 --- Val = 40.73455
Eval fn at 0.4611668 12.14859 7.042902 --- Val = 40.72437
Eval fn at 0.4611668 12.14659 7.042902 --- Val = 40.72545
Eval fn at 0.4611668 12.14759 7.043902 --- Val = 40.72593
Eval fn at 0.4611668 12.14759 7.041902 --- Val = 40.72389
Eval fn at 0.6522179 14.65926 5.138706 --- Val = 36.97182
Eval fn at 0.6532179 14.65926 5.138706 --- Val = 36.96971
Eval fn at 0.6512179 14.65926 5.138706 --- Val = 36.97397
Eval fn at 0.6522179 14.66026 5.138706 --- Val = 36.97182
Eval fn at 0.6522179 14.65826 5.138706 --- Val = 36.97183
Eval fn at 0.6522179 14.65926 5.139706 --- Val = 36.97298
Eval fn at 0.6522179 14.65926 5.137706 --- Val = 36.97066
Eval fn at 0.7531634 19.00619 1.708864 --- Val = 32.87589
Eval fn at 0.7541634 19.00619 1.708864 --- Val = 32.87811
Eval fn at 0.7521634 19.00619 1.708864 --- Val = 32.87372
Eval fn at 0.7531634 19.00719 1.708864 --- Val = 32.87603
Eval fn at 0.7531634 19.00519 1.708864 --- Val = 32.87575
Eval fn at 0.7531634 19.00619 1.709864 --- Val = 32.87732
Eval fn at 0.7531634 19.00619 1.707864 --- Val = 32.87446
Eval fn at 0.8034285 21.17072 0.001 --- Val = 64.44045
Eval fn at 0.8044285 21.17072 0.001 --- Val = 64.43177
Eval fn at 0.8024285 21.17072 0.001 --- Val = 64.44914
Eval fn at 0.8034285 21.17172 0.001 --- Val = 64.44045
Eval fn at 0.8034285 21.16972 0.001 --- Val = 64.44045
Eval fn at 0.8034285 21.17072 0.002 --- Val = 59.65652
Eval fn at 0.8034285 21.17072 0.001 --- Val = 64.44045
Eval fn at 0.7706081 19.75740 1.116143 --- Val = 32.23114
Eval fn at 0.7716081 19.75740 1.116143 --- Val = 32.23324
Eval fn at 0.7696081 19.75740 1.116143 --- Val = 32.2291
Eval fn at 0.7706081 19.75840 1.116143 --- Val = 32.23126
Eval fn at 0.7706081 19.75640 1.116143 --- Val = 32.23103
Eval fn at 0.7706081 19.75740 1.117143 --- Val = 32.23228
Eval fn at 0.7706081 19.75740 1.115143 --- Val = 32.23000
Eval fn at 0.7425713 21.12920 0.001 --- Val = 64.99
Eval fn at 0.7435713 21.12920 0.001 --- Val = 64.98061
Eval fn at 0.7415713 21.12920 0.001 --- Val = 64.99941
Eval fn at 0.7425713 21.13020 0.001 --- Val = 64.99
Eval fn at 0.7425713 21.12820 0.001 --- Val = 64.99
Eval fn at 0.7425713 21.12920 0.002 --- Val = 60.20449
Eval fn at 0.7425713 21.12920 0.001 --- Val = 64.99

And a trace of what optim is doing. I'm sure this can tell me what's going
wrong, but I don't understand the optimization method, so I can't
interpret this output.

N = 3, M = 5 machine precision = 2.22045e-016
L = 0.001 0.001 0.001 
X0 = 0.7 10.3 8.5 
U = 1 1.#INF 1.#INF 
At X0, 0 variables are exactly at the bounds
At iterate     0  f=       42.706  |proj g|=       1.3527
Iteration     0

---------------- CAUCHY entered-------------------

There are 2  breakpoints

Piece      1 f1, f2 at start point -2.8533e+000 2.8533e+000
Distance to the next break point =  7.6921e+000
Distance to the stationary point =  1.0000e+000

GCP found in this segment
Piece      1 f1, f2 at start point -2.8533e+000 2.8533e+000
Distance to the stationary point =  1.0000e+000
Cauchy X =  0.739001 11.6527 7.48904 

---------------- exit CAUCHY----------------------

3  variables are free at GCP on iteration 1
LINE SEARCH 0 times; norm of step = 1
X = 0.723089 11.1008 7.9015 
G = 1.11513 -0.944325 1.00394 
Iteration     1

---------------- CAUCHY entered-------------------

There are 2  breakpoints

Piece      1 f1, f2 at start point -3.1431e+000 7.9760e+000
Distance to the next break point =  6.4754e-001
Distance to the stationary point =  3.9407e-001

GCP found in this segment
Piece      1 f1, f2 at start point -3.1431e+000 7.9760e+000
Distance to the stationary point =  3.9407e-001
Cauchy X =  0.283647 11.4729 7.50588 

---------------- exit CAUCHY----------------------

3  variables are free at GCP on iteration 2
LINE SEARCH 1 times; norm of step = 1.37898
X = 0.461167 12.1476 7.0429 
G = -9.62019 -0.542 1.022 
Iteration     2

---------------- CAUCHY entered-------------------

There are 2  breakpoints

Piece      1 f1, f2 at start point -9.3886e+001 4.1691e+003
Distance to the next break point =  5.6011e-002
Distance to the stationary point =  2.2519e-002

GCP found in this segment
Piece      1 f1, f2 at start point -9.3886e+001 4.1691e+003
Distance to the stationary point =  2.2519e-002
Cauchy X =  0.677808 12.1598 7.01989 

---------------- exit CAUCHY----------------------

3  variables are free at GCP on iteration 3
LINE SEARCH 0 times; norm of step = 3.15767
X = 0.652218 14.6593 5.13871 
G = -2.13069 -0.00539645 1.15734 
Iteration     3

---------------- CAUCHY entered-------------------

There are 2  breakpoints

Piece      1 f1, f2 at start point -5.8793e+000 2.4656e+002
Distance to the next break point =  1.6322e-001
Distance to the stationary point =  2.3846e-002

GCP found in this segment
Piece      1 f1, f2 at start point -5.8793e+000 2.4656e+002
Distance to the stationary point =  2.3846e-002
Cauchy X =  0.703026 14.6594 5.11111 

---------------- exit CAUCHY----------------------

3  variables are free at GCP on iteration 4
LINE SEARCH 2 times; norm of step = 6.49509
X = 0.770608 19.7574 1.11614 
G = 2.07073 0.113724 1.1404 
Iteration     4

---------------- CAUCHY entered-------------------

There are 3  breakpoints

Piece      1 f1, f2 at start point -5.6014e+000 2.7530e+002
Distance to the next break point =  3.7166e-001
Distance to the stationary point =  2.0346e-002

GCP found in this segment
Piece      1 f1, f2 at start point -5.6014e+000 2.7530e+002
Distance to the stationary point =  2.0346e-002
Cauchy X =  0.728477 19.7551 1.09294 

---------------- exit CAUCHY----------------------

3  variables are free at GCP on iteration 5
LINE SEARCH 0 times; norm of step = 1.7681
X = 0.742571 21.1292 0.001 
G = -9.39676 0.000205735 -4785.51 

iterations 5
function evaluations 9
segments explored during Cauchy searches 5
BFGS updates skipped 0
active bounds at final generalized Cauchy point 0
norm of the final projected gradient 4785.51
final function value 64.99

X = 0.742571 21.1292 0.001 
F = 64.99
final  value 64.990005 
converged