[R] Dealing with -Inf in a maximisation problem.

[Rolf Turner]

I am trying to deal with a maximisation problem in which it is possible 
for the objective function to (quite legitimately) return the value 
-Inf, which causes the numerical optimisers that I have tried to fall over.

The -Inf values arise from expressions of the form "a * log(b)", with b 
= 0.  Under the *starting* values of the parameters, a must equal equal 
0 whenever b = 0, so we can legitimately say that a * log(b) = 0 in 
these circumstances.  However as the maximisation algorithm searches 
over parameters it is possible for b to take the value 0 for values of
a that are strictly positive.  (The values of "a" do not change during
this search, although they *do* change between "successive searches".)

Clearly if one is *maximising* the objective then -Inf is not a value of
particular interest, and we should be able to "move away".  But the 
optimising function just stops.

It is also clear that "moving away" is not a simple task; you can't 
estimate a gradient or Hessian at a point where the function value is -Inf.

Can anyone suggest a way out of this dilemma, perhaps an optimiser that 
is equipped to cope with -Inf values in some sneaky way?

Various ad hoc kludges spring to mind, but they all seem to be fraught 
with peril.

I have tried changing the value returned by the objective function from
"v" to exp(v) --- which maps -Inf to 0, which is nice and finite. 
However this seemed to flatten out the objective surface too much, and 
the search stalled at the 0 value, which is the antithesis of optimal.

The problem arises in a context of applying the EM algorithm where the 
M-step cannot be carried out explicitly, whence numerical optimisation.
I can give more detail if anyone thinks that it could be relevant.

I would appreciate advice from younger and wiser heads! :-)

cheers,

Rolf Turner

-- 
Technical Editor ANZJS
Department of Statistics
University of Auckland
Phone: +64-9-373-7599 ext. 88276