[R] Can I monitor the iterative/convergence process whileusing Optim or MaxLik?
William Dunlap
wdunlap at tibco.com
Wed Sep 15 00:15:26 CEST 2010
I think the default method for optim never evaluates
the gradient function: VALUE and X are both NULL in
your example. Also o$counts shows NA for the gradient
entry. Use method="BGFS":
> o <- optim(c(-1.2, 1), fr, tfr <- trackFn(grr), method="BFGS")
> with(environment(tfr), str(VALUE))
num [1:43, 1:2] -215.6 136.42 56.44 10.27 9.35 ...
- attr(*, "dimnames")=List of 2
..$ : chr [1:43] "val" "val" "val" "val" ...
..$ : NULL
Bill Dunlap
Spotfire, TIBCO Software
wdunlap tibco.com
> -----Original Message-----
> From: Ravi Varadhan [mailto:rvaradhan at jhmi.edu]
> Sent: Tuesday, September 14, 2010 3:01 PM
> To: William Dunlap
> Cc: Greg Snow; r-help at r-project.org
> Subject: Re: RE: [R] Can I monitor the iterative/convergence
> process whileusing Optim or MaxLik?
>
> Bill,
>
> I am not able to get this to work for tracking inside the
> gradient function:
>
> trackFn <- function (fn) {
> # return function like fn that stashes its
> # inputs and outputs in local datasets.
> X <- NULL
> VALUE <- NULL
> force(fn)
> function(x) {
> X <<- rbind(X, x) # stash arguments
> val <- fn(x)
> VALUE <<- rbind(VALUE, val) # stash value
> val
> }
> }
>
> fr <- function(x) { ## Rosenbrock Banana function
> x1 <- x[1]
> x2 <- x[2]
> 100 * (x2 - x1 * x1)^2 + (1 - x1)^2
> }
>
> grr <- function(x) { ## Gradient of 'fr'
> x1 <- x[1]
> x2 <- x[2]
> c(-400 * x1 * (x2 - x1 * x1) - 2 * (1 - x1), 200 * (x2 - x1 * x1))
> }
>
> o <- optim(c(-1.2, 1), fr, tfr <- trackFn(grr))
> #with(environment(tfr), matplot(X, VALUE, type="l"))
> with(environment(tfr), plot(X, type="l"))
>
> What is wrong here?
>
> Ravi.
>
> ____________________________________________________________________
>
> Ravi Varadhan, Ph.D.
> Assistant Professor,
> Division of Geriatric Medicine and Gerontology
> School of Medicine
> Johns Hopkins University
>
> Ph. (410) 502-2619
> email: rvaradhan at jhmi.edu
>
>
> ----- Original Message -----
> From: William Dunlap <wdunlap at tibco.com>
> Date: Tuesday, September 14, 2010 5:38 pm
> Subject: RE: [R] Can I monitor the iterative/convergence
> process while using Optim or MaxLik?
> To: Ravi Varadhan <rvaradhan at jhmi.edu>, Greg Snow
> <greg.snow at imail.org>
> Cc: r-help at r-project.org
>
>
> > You can also save the objective (or gradient) function
> > arguments and values in datasets so you can later plot
> > or print them. E.g., the following lets you avoid hand
> > editing the objective function to do this. If you wanted
> > to track the gradient, replace th c(VALUE,val) with
> > rbind(VALUE,val). (Either would be ok with the objective.)
> >
> > trackFn <- function (fn) {
> > # return function like fn that stashes its
> > # inputs and outputs in local datasets.
> > X <- NULL
> > VALUE <- NULL
> > force(fn)
> > function(x) {
> > X <<- rbind(X, x) # stash arguments
> > val <- fn(x)
> > VALUE <<- c(VALUE, val) # stash value
> > val
> > }
> > }
> >
> > Typical usage would be
> >
> > > fr <- function(x) { ## Rosenbrock Banana function
> > + x1 <- x[1]
> > + x2 <- x[2]
> > + 100 * (x2 - x1 * x1)^2 + (1 - x1)^2
> > + }
> > > o <- optim(c(-1.2, 1), tfr <- trackFn(fr))
> > > with(environment(tfr), matplot(X, VALUE, type="l"))
> > > with(environment(tfr), plot(X, type="l"))
> >
> > Bill Dunlap
> > Spotfire, TIBCO Software
> > wdunlap tibco.com
> >
> > > -----Original Message-----
> > > From: r-help-bounces at r-project.org
> > > [ On Behalf Of Ravi Varadhan
> > > Sent: Tuesday, September 14, 2010 2:19 PM
> > > To: Greg Snow
> > > Cc: r-help at r-project.org
> > > Subject: Re: [R] Can I monitor the iterative/convergence
> > > process while using Optim or MaxLik?
> > >
> > > This is generally not the best way to do it for the following
> > > reasons.
> > >
> > > The objective function may be evaluated multiple times within
> > > a single iteration, for various reasons including (i)
> > > gradient evaluation when analytic gradient is not specified,
> > > and (ii) during line-search stretgy for steplength
> > > determination. Therefore, embedding a `cat' or `print'
> > > statement inside the objective function may generate a lot of
> > > noise. You can run your suggested `optim' example to
> verify this!
> > >
> > > A better approach is to embed the `cat' statement inside the
> > > gradient function, in addition to using the `trace' and
> > > `REPORT' arguments, which are integers in `optim'. Here is
> > > the `optim' example that you suggested:
> > >
> > > fr <- function(x) { ## Rosenbrock Banana function
> > > x1 <- x[1]
> > > x2 <- x[2]
> > > 100 * (x2 - x1 * x1)^2 + (1 - x1)^2
> > > }
> > >
> > > grr.print <- function(x) { ## Gradient of 'fr'
> > > cat("pars: ", x, '\n')
> > > flush.console()
> > > x1 <- x[1]
> > > x2 <- x[2]
> > > c(-400 * x1 * (x2 - x1 * x1) - 2 * (1 - x1), 200 * (x2 -
> x1 * x1))
> >
> > > }
> > >
> > > > optim(c(-1.2,1), fr, grr.print, method = "BFGS",
> > > control=list(trace=1, REPORT=1))
> > >
> > > This prints out both parameter values and the function value
> > > at each iteration.
> > >
> > > This approach cannot, however, be used in `optim' when
> > > analytic gradient is not available (or for derivative free
> > > methods, e.g. Nelder-Mead).
> > >
> > >
> > > Hope this helps,
> > > Ravi.
> > >
> > >
> ____________________________________________________________________
> > >
> > > Ravi Varadhan, Ph.D.
> > > Assistant Professor,
> > > Division of Geriatric Medicine and Gerontology
> > > School of Medicine
> > > Johns Hopkins University
> > >
> > > Ph. (410) 502-2619
> > > email: rvaradhan at jhmi.edu
> > >
> > >
> > > ----- Original Message -----
> > > From: Greg Snow <Greg.Snow at imail.org>
> > > Date: Tuesday, September 14, 2010 4:27 pm
> > > Subject: Re: [R] Can I monitor the iterative/convergence
> > > process while using Optim or MaxLik?
> > > To: Sally Luo <shali623 at gmail.com>, "r-help at r-project.org"
> > > <r-help at r-project.org>
> > >
> > >
> > > > The simple way to do this is just to rewrite your
> function to be
> >
> > > > optimized to print out the current values that it was called
> > with,
> > > > this way you will see where it is and what it is doing.
> > > Look at the
> > > > cat function for how to print the values, also look at the
> > > > flush.console function.
> > > >
> > > > For example you could take the 1st example on the optim
> > > help page and
> > > > insert the following 2 lines as the 1st 2 lines of the
> f function:
> > > >
> > > > cat(x, '\n')
> > > > flush.console()
> > > >
> > > > and then at each iteration you can see the values tried.
> > > >
> > > > --
> > > > Gregory (Greg) L. Snow Ph.D.
> > > > Statistical Data Center
> > > > Intermountain Healthcare
> > > > greg.snow at imail.org
> > > > 801.408.8111
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: r-help-bounces at r-project.org [
> > > > > project.org] On Behalf Of Sally Luo
> > > > > Sent: Tuesday, September 14, 2010 1:50 PM
> > > > > To: r-help at r-project.org
> > > > > Subject: [R] Can I monitor the iterative/convergence
> > > process while
> > > > > using Optim or MaxLik?
> > > > >
> > > > > Hi R-helpers,
> > > > >
> > > > > Is it possible that I have the estimates from each
> > > step/iteration shown
> > > > > on
> > > > > the computer screen in order to monitor the process
> > > while I am using
> > > > > Optim
> > > > > or MaxLik?
> > > > >
> > > > > Thanks for your help.
> > > > >
> > > > > Maomao
> > > > >
> > > > > [[alternative HTML version deleted]]
> > > > >
> > > > > ______________________________________________
> > > > > R-help at r-project.org mailing list
> > > > >
> > > > > PLEASE do read the posting guide
> > > > > guide.html
> > > > > and provide commented, minimal, self-contained,
> > > reproducible code.
> > > >
> > > > ______________________________________________
> > > > R-help at r-project.org mailing list
> > > >
> > > > PLEASE do read the posting guide
> > > > and provide commented, minimal, self-contained,
> reproducible code.
> > >
> > > ______________________________________________
> > > R-help at r-project.org mailing list
> > >
> > > PLEASE do read the posting guide
> > >
> > > and provide commented, minimal, self-contained,
> reproducible code.
> > >
>
More information about the R-help
mailing list