[R] Optim() and Instability

[Gabor Grothendieck]

Tyipcally the parameters being optimized should be the same order of
magnitude or else you can expect numerical problems.  That is what the
fnscale control parameter is for.

On Sat, Nov 14, 2015 at 10:15 AM, Lorenzo Isella
<lorenzo.isella at gmail.com> wrote:
> Dear All,
> I am using optim() for a relatively simple task: a linear model where
> instead of minimizing the sum of the squared errors, I minimize the sum
> of the squared relative errors.
> However, I notice that the default algorithm is very sensitive to the
> choice of the initial fit parameters, whereas I get much more stable
> (and therefore better?) results with the BFGS algorithm.
> I would like to have some feedback on this (perhaps I made a mistake
> somewhere).
> I provide a small self-contained example.
> You can download a tiny data set from the link
>
> https://www.dropbox.com/s/tmbj3os4ev3d4y8/data-instability.csv?dl=0
>
> whereas I paste the script I am using at the end of the email.
> Any feedback is really appreciated.
> Many thanks
>
> Lorenzo
>
> ################################################################
>
> min.perc_error <- function(data, par) {
>              with(data, sum(((par[1]*x1 + par[2]*x2+par[3]*x3 -
>              y)/y)^2))
>                            }
>
> par_ini1 <- c(.3,.1, 1e-3)
>
> par_ini2 <- c(1,1, 1)
>
>
> data <- read.csv("data-instability.csv")
>
> mm_def1 <-optim(par = par_ini1
>                    , min.perc_error, data = data)
>
> mm_bfgs1 <-optim(par = par_ini1
>                    , min.perc_error, data = data, method="BFGS")
>
> print("fit parameters with the default algorithms and the first seed
> ")
> print(mm_def1$par)
>
> print("fit parameters with the BFGS algorithms and the first seed  ")
> print(mm_bfgs1$par)
>
>
>
> mm_def2 <-optim(par = par_ini2
>                    , min.perc_error, data = data)
>
> mm_bfgs2 <-optim(par = par_ini2
>                    , min.perc_error, data = data, method="BFGS")
>
>
>
>
> print("fit parameters with the default algorithms and the second seed
> ")
> print(mm_def2$par)
>
> print("fit parameters with the BFGS algorithms and the second seed  ")
> print(mm_bfgs2$par)
>
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.



-- 
Statistics & Software Consulting
GKX Group, GKX Associates Inc.
tel: 1-877-GKX-GROUP
email: ggrothendieck at gmail.com