[R-sig-Geo] 2 problems with errorsarlm(method="SparseM")

[Roger Bivand]

On Wed, 1 Feb 2006, Martin Reismann wrote:

> (1) When running a large SAR (weight matrix 12544 * 12544) with errorsarlm
> and SparseM-method I ran into this error message:
>  
> > Error in .local(x, ...) : Increase nnzlmax
> > Error in det(chol((I - lambda * csrw), tmpmax = tmpmax)) :
> >         unable to find the argument 'x' in selecting a method for
> function 'det' 
>  
> The listw-object stems from nb2listw and is style="W".
> It happens on both Linux (32 bit, 4GB RAM) and Solaris (64 bit, 8GB RAM)

When in doubt, immediately after the error exit from an R function, type

> traceback()

which will list out the function calls back to the function in error. I 
think that the problem is that because of the density of your spatial 
weights matrix, chol() for sparse matrices is running out of working 
space, and that nnzlmax now needs setting (Danlin Yu reported a similar 
problem with tmpmax over a year ago).

Could you repeat the problem and then run and report traceback()? Could
you also report sum(card()) for your list of neighbours? I think your
matrix is quite dense, and knowing how dense it is will help work out how
big nnzlmax needs to be.

>  
> (2) Being able to use mat2listw instead of nb2listw would be preferable
> because mat2listw allows matrix elements different from 0 and 1.
>  

Why can't you use the glist= argument to nb2listw()?

Also note that method="SparseM" should only work with weights that are 
either symmetric or similar to symmetric - I'll make this clearer on the 
help page, and block style="M" for that method.

This means that symmetric general weights, like inverse distance, may be 
possible, but that both asymmetric neighbours and asymmetric general 
weights for symmetric neighbours are banned for method="SparseM" - 
obviously because chol() for sparse matrices only works when they are 
symmetric (row standardised symmetric neighbours are asymmetric weights, 
but the determinant is the same for the similar-to-symmetric version.

I guess the problem below will go away when I block the entrance.

Possibly the only weights matrix that could be tolerated is a fully 
symmetric neighbours matrix with symmetric general weights. There is 
different code in spautolm() doing much the same as errorsarlm(), but I'm 
not sure that both problems will not re-occur there until they are fixed. 
Getting nnzlmax right is possible, but chol() is just not going to work if 
the weights are neither symmetric nor similar-to-symmetric.

Roger

PS. Access to your data would speed up finding a solution, then I can run 
traceback() myself.

> Unfortunately errorsarlm(..., method="SparseM") on a listw-object made by
> mat2listw crashed in an early stage.
>  

PPS. In R, "crash" is reserved for R itself terminating and exiting with
data loss. Here, it just reported something wrong and returned to the
prompt, so the function did its best, with no "crash" as such.

> Error message:
> > Error in validObject(.Object) : invalid class "matrix.csr" object:
> > invalid object for slot "ja" in class "matrix.csr": 
> > got class "numeric", should be or extend class "integer"
> 
> It might be because mat2listw creates listw-objects of weights style="M"
> (underlying style unknown).
> Is there a way to transform a listw-object created by mat2listw into a
> "clean" one with proper weights style set?
>  
>  
> I appreciate your help!
>  
> Martin
> 
> 
> 

-- 
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand at nhh.no