[R] BFGS versus L-BFGS-B

[Prof. John C Nash]

There are considerable differences between the algorithms. And BFGS is an unfortunate
nomenclature, since there are so many variants that are VERY different. It was called
"variable metric" in my book from which the code was derived, and that code was from Roger
Fletcher's Fortran VM code based on his 1970 paper. L-BFGS-B is a later and more
complicated algorithm with some pretty nice properties. The code is much larger.

Re: "less memory" -- this will depend on the number of parameters, but to my knowledge
there are no good benchmark studies of memory and performance. Perhaps someone wants to
propose one for Google Summer of Code (see
http://rwiki.sciviews.org/doku.php?id=developers:projects:gsoc2011
).

The optimx package can call Rvmmin which has box constraints (also Rcgmin that is intended
for very low memory). Also several other methods with box constraints, including L-BFGS-B.
Worth a try if you are seeking a method for multiple "production" runs. Unfortunately, we
seem to have some CRAN check errors on Solaris and some old releases -- platforms I do not
have -- so it may be a few days or more until we sort out the issues, which seem to be
related to alignment of the underlying packages for which optimx is a wrapper.

Use of transformation can be very effective. But again, I don't think there are good
studies on whether use of box constraints or transformations is "better" and when. Another
project, which I have made some tentative beginings to carry out. Collaborations welcome.

Best,

JN


On 02/25/2011 06:00 AM, r-help-request at r-project.org wrote:
> Message: 86
> Date: Fri, 25 Feb 2011 00:11:59 -0500
> From: Brian Tsai <btsai00 at gmail.com>
> To: r-help at r-project.org
> Subject: [R] BFGS versus L-BFGS-B
> Message-ID:
> 	<AANLkTimSzVkJbUhV-bBR1easPX9oOTJxQCUjGUjR5Lo6 at mail.gmail.com>
> Content-Type: text/plain
> 
> Hi all,
> 
> I'm trying to figure out the effective differences between BFGS and L-BFGS-B
> are, besides the obvious that L-BFGS-B should be using a lot less memory,
> and the user can provide box constraints.
> 
> 1) Why would you ever want to use BFGS, if L-BFGS-B does the same thing but
> use less memory?
> 
> 2) If i'm optimizing with respect to a variable x that must be non-negative,
> a common approach is to do a change of variables x = exp(y), and optimize
> unconstrained with respect to y.  Is optimization using box constraints on
> x, likely to produce as good a result as unconstrained optimization on y?
> 
> - Brian.
> 
> 	[[alternative HTML version deleted]]
> 
> 
>