[R] any more direct-search optimization method in R

[Ben Bolker]

    Mathematica may well have good optimization routines; I know MATLAB
does (e.g. the optimization toolbox, 
http://www.mathworks.com/products/optimization/?BB=1 , has a 
general-constraints nonlinear optimizer) -- I also think
more people develop optimization code in MATLAB because of its use in
engineering circles.  (The DIRECT algorithm is implemented in
TOMLAB, a MATLAB add-on package from a third party.)
(S+NuOPT is also available from Insightful,
although it doesn't look like it does global stochastic stuff;
our friends at Otter Research have ADMB, although I don't
know how well it handles noisy objective functions.)

    Drawbacks: (1) switching platforms is a pain, (2) most of
these alternatives are
expensive [although if you have the money it may well be
worth it], (3) Mathematica is notoriously bad about documenting
its methods and giving references to the peer-reviewed literature.
If I were regularly doing really difficult optimization
problems I might switch to MATLAB+add-ons.

   I would like to see more optimization choices implemented in
R, but haven't gotten around to doing so myself yet.  For hard
optimization problems, however, it is nearly always true that
you have to know something about your problem and tune methods
accordingly (see Spencer Graves' message in this thread).

   Ben

-------- Original Message --------
Subject: Re: [R] any more direct-search optimization method in R
Date: Tue, 28 Feb 2006 09:55:15 -0800 (PST)
From: Greg Tarpinian <sasprog474474 at yahoo.com>
To: Ben Bolker <bolker at ufl.edu>

This may not be the most helpful advice, but Mathematica is a wonderful
platform having many more built-in optimization routines than R.  I have
found the simulated annealing facility to be robust and much easier to
use than R's own facility.  This may be a case where switching to another
platform would be the easiest solution.

Kind regards,

     Greg



--- Ben Bolker <bolker at ufl.edu> wrote:

> Ingmar Visser <I.Visser <at> uva.nl> writes:
> 
> > 
> > If you have only boundary constraints on parameters you can use method
> > L-BFGS in optim.
> > Hth, ingmar
> > 
> > > From: Weijie Cai <wcai11 <at> hotmail.com>
> 
> > > 
> > > I am dealing with a noisy function (gradient,hessian not available) with
> > > simple boundary constraints (x_i>0). I've tried constrOptim() using
> nelder
> > > mead to minimize it but it is way too slow and the returned results are
> not
> > > satisfying. simulated annealing is so hard to tune and it always crashes
> R
> > > program in my case. I wonder if there are any packages or functions can
> do
> > > direct search optimization?
> > > 
> 
>    Noisy functions are really challenging to optimize; (1) there is no
> "best" method (despite all the papers doing comparisons of stochastic
> global optimizers on various sets of test functions); (2) the fancier
> methods are hard to program [and existing implementations tend have more
> restricted licenses]; (3) they tend to be slow (thousands of function
> evaluations).   Packages on CRAN that *might* be helpful are
> genalg, DEoptim.
>    A typical "poor man's" approach to boundary constraints is to
> add a quadratic penalty (perhaps not even trying to evaluate the
> objective function -- e.g. substituting the value at the closest
> boundary point) for parameters outside the constraints into
> the objective function.
>    With more information (number of parameters, time to compute a
> single function evaluation, kind of noise) we might be able to help
> more.
>   
>   Ben Bolker
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
> 


__________________________________________________




-- 
620B Bartram Hall                            bolker at zoo.ufl.edu
Zoology Department, University of Florida    http://www.zoo.ufl.edu/bolker
Box 118525                                   (ph)  352-392-5697
Gainesville, FL 32611-8525                   (fax) 352-392-3704