[R] optimization with Sparse matrices

[Ben Bolker]

Dube, Jean-Pierre <Jean-Pierre.Dube <at> chicagobooth.edu> writes:

> 
> To whom it may concern,
> 
> I am trying to maximize a log-likelihood function using optim. 
> This is a simple problem with only 18
> parameters.  To conserve memory, I am using sparse matrices 
> (SLAM) for some of the data matrices used in the
> computation of the likelihood.  However, optim appears to
> convert the sparse matrix back to regular data
> format.  This causes me to run out of memory as R tries 
> to create an 8GB matrix.  In short, it does not look as
> though "optim" is compatible with sparse matrices. 
> Does anyone have a suggestion for how I can maximize a
> function in R using sparse matrices for some of the data inputs?
> 

  I wouldn't guess that optim() itself would do any conversion
of the data values (are these passed as additional parameters
or used as global variables?)  The only thing optim() does with
parameters in '...' is pass them along to the objective
function as follows:

    fn1 <- function(par) fn(par, ...)

and I can't see a reason that would convert them in any way.
It seems more likely to me (?) that there is some step in 
your objective function where you do an operation that accidentally
converts the sparse back to non-sparse matrices ...

  Can you construct a minimal example that shows this happening?