[R] any more direct-search optimization method in R
Jasjeet Singh Sekhon
sekhon at berkeley.edu
Wed Mar 1 12:33:30 CET 2006
> Given that information, I think a genetic algorithm
> should probably do well with your problem.
You may want to try the rgenoud package (R-GENetic Optimization Using
Derivatives) which is on CRAN. For more information see:
http://sekhon.berkeley.edu/rgenoud/
It works well for these kinds of problems and the package is
relatively flexible. For time consuming problems it can be run in
parallel if you have multiple cores/cpus or machines.
Cheers,
JS.
=======================================
Jasjeet S. Sekhon
Associate Professor
Survey Research Center
UC Berkeley
http://sekhon.berkeley.edu/
V: 510-642-9974 F: 617-507-5524
=======================================
> Given that information, I think a genetic algorithm
> should probably do well with your problem. Standard
> derivative-based optimizers are going to get frustrated
> and give up. I can believe that Nelder-Mead could
> get confused as well, though I'm not sure that it will.
>
> 'genopt' from S Poetry does have box constraints for
> the parameters. I'm not sure what other genetic algorithms
> that are in R are like.
>
> Patrick Burns
> patrick at burns-stat.com
> +44 (0)20 8525 0696
> http://www.burns-stat.com
> (home of S Poetry and "A Guide for the Unwilling S User")
>
> Weijie Cai wrote:
>
> >Hi All,
> >Thanks for all your replies especially for Graves suggestions. You are right
> >I should give more information about my function. So my responds to your
> >questions are:
> >1. 2. the function itself is not continuous/smooth. The evaluation at each
> >point is a random number with a non-constant variance. When it approaches
> >the global minimum, the variance is very small. There is some kind of
> >structure from the surface plot of my function but its form is intractable,
> >unfortunately.
> >
> >3. 4. each evaluation of my function is not slow. The returned results by
> >constrOptim() are just not quite close to true global minimum (error can be
> >as large as 0.2). Of course I can ignore the message of nonconvergence, the
> >precision is really not satisfying. Every time nelder-mead will use up 300
> >default iterations when doing optimization. I guess the essential reason is
> >the randomness of function surface.
> >
> >5. Yes I am sure there is a global minimum. I did a lengthy computation at
> >rough grids and global minimum is very close to true minimum.
> >
> >6. Do you mean I start from a "minimum" found by grid searching? That's what
> >I did. I never tried using smooth functions to approximate my function
> >though.
> >
> >WC
> >
> >
> >
> >
> >>From: Spencer Graves <spencer.graves at pdf.com>
> >>To: Ingmar Visser <I.Visser at uva.nl>
> >>CC: Weijie Cai <wcai11 at hotmail.com>, r-help at stat.math.ethz.ch
> >>Subject: Re: [R] any more direct-search optimization method in R
> >>Date: Tue, 28 Feb 2006 09:33:35 -0800
> >>
> >>WC:
> >>
> >> What do you mean by "noisy" in this context?
> >>
> >> 1. You say, "gradient, hessian not available". Is it continuous with
> >>perhaps discontinuities in the first derivative?
> >>
> >> 2. Or is it something you can compute only to, say, 5 significant
> >>digits, and some numerical optimizers get lost trying to estimate
> >>derivatives from so fine a grid that the gradient and hessian are mostly
> >>noise?
> >>
> >> 3. Also, why do you think "constrOptim" is too slow? Does it call your
> >>function too many times or does your function take too long to compute each
> >>time it's called?
> >>
> >> 4. What's not satisfactory about the results of "constrOptim"?
> >>
> >> 5. Do you know if only one it has only one local minimum in the region,
> >>or might it have more?
> >>
> >> 6. Regardless of the answers to the above, have you considered using
> >>"expand.grid" to get starting values and narrow the search (with possibly
> >>system.time or proc.time to find out how much time is required for each
> >>function evaluation)? I haven't tried this, but I would think it would be
> >>possible to fit a spline (either exactly or a smoothing spline) to a set of
> >>points, then optimize the spline.
> >>
> >> hope this helps.
> >> spencer graves
> >>
> >>Ingmar Visser wrote:
> >>
> >>
> >>
> >>>If you have only boundary constraints on parameters you can use method
> >>>L-BFGS in optim.
> >>>Hth, ingmar
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>>From: Weijie Cai <wcai11 at hotmail.com>
> >>>>Date: Tue, 28 Feb 2006 11:48:32 -0500
> >>>>To: <r-help at stat.math.ethz.ch>
> >>>>Subject: [R] any more direct-search optimization method in R
> >>>>
> >>>>Hello list,
> >>>>
> >>>>I am dealing with a noisy function (gradient,hessian not available) with
> >>>>simple boundary constraints (x_i>0). I've tried constrOptim() using
> >>>>nelder
> >>>>mead to minimize it but it is way too slow and the returned results are
> >>>>not
> >>>>satisfying. simulated annealing is so hard to tune and it always crashes
> >>>>R
> >>>>program in my case. I wonder if there are any packages or functions can
> >>>>do
> >>>>direct search optimization?
> >>>>
> >>>>A rough search in literature shows multidirectional search and DIRECT
> >>>>algorithm may help. Is there any other satisfying algorithm?
> >>>>
> >>>>Thanks,
> >>>>WC
> >>>>
> >>>>______________________________________________
> >>>>R-help at stat.math.ethz.ch mailing list
> >>>>https://stat.ethz.ch/mailman/listinfo/r-help
> >>>>PLEASE do read the posting guide!
> >>>>http://www.R-project.org/posting-guide.html
> >>>>
> >>>>
> >>>______________________________________________
> >>>R-help at stat.math.ethz.ch mailing list
> >>>https://stat.ethz.ch/mailman/listinfo/r-help
> >>>PLEASE do read the posting guide!
> >>>http://www.R-project.org/posting-guide.html
> >>>
> >>>
> >
> >______________________________________________
> >R-help at stat.math.ethz.ch mailing list
> >https://stat.ethz.ch/mailman/listinfo/r-help
> >PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
> >
> >
> >
> >
> >
>
>
More information about the R-help
mailing list