[R-sig-Geo] Fitting SAR models in spdep (was: spatial 2SLS)

[Roger Bivand]

On Mon, 13 Dec 2004, Danlin Yu wrote:

> Dear Mihai Nica:
> 
>     I think it is a memory issue, too. I ran into the same problem when
> using around 3000 observations. Although I used memory.size() to increase
> the memroy allocation size, spdep took too long to run.
> 

I think you mean the two SAR fitting functions, lagsarlm() and 
errorsarlm()? The default method is "eigen", which does need plenty of 
memory with large n (because it both solves the eigenproblem for an nxn 
matrix, and later inverts an nxn matrix). There has always been a "sparse" 
method using code from Netlib called sparse from UCB, which works on some 
platforms, but not others, and has an unresolved memory bug. 

I'm pleased to report that Roger Koenker has been kind enough to add a
det() for symmetric (and similar to symmetric) sparse matrices to the
SparseM package. From spdep_0.3-5, spdep depends on SparseM, and
errorsarlm() is now pretty dependable for large n and fast. lagsarlm() is
dependable, but its fitting speed is slowed by fitting models for LR tests
dropping each of the X variables in turn, as well as the specified model,
so that it fits k models, not just 1. The function calls for the lag
models are cheaper than the error models, though.

The restriction to symmetric (and similar to symmetric) sparse matrices is
important to bear in mind. With symmetric neighbour lists (contiguity,
graphe-based and distance bands), and even symmetric inverse distances,
and all weighting styles this is OK (because "W" and "S" style are similar
to symmetric, and the others are symmetric), but absolutely not for
k-nearest-neighbours or for asymmetric general weights (say migration
tables).

I would be very interested in hearing whether this approach is helpful, 
and how this scales.

Roger

>     The problem, I think, might be due to the fact that spdep's spatial
> autoregressive models use the exact method (correct me if I am wrong),
> while GeoDa uses an approximatioin method (see the paper "Fast maximum
> likelihood estimation of very large spatial autoregressive models: a
> characteristic polynomial approach", in Computational Statistics & Data
> Analysis 35 (2001) 301-319). If the method could be incorporated in R, it
> will be great.
> 
>     Danlin
> 
> On Mon, 13 Dec 2004, Mihai Nica wrote:
> 
> > Greetings:
> > 
> > I would like to take advantage of this thread and ask if there is a way to
> > "enhance" the code in spdep. I have a dataset with 3030 observations and two
> > independent variables. On the same old computer it runs great in GeoDa, but
> > fails in R. I assume it is a memory issue, but having two different ways to
> > check my results would be great.
> > 
> > Thanks,
> > 
> > Mihai Nica
> > Jackson State University
> > 155 B Parkhurst Dr.
> > Jackson, MS 39202
> > 601 914 0361
> > 
> > ----- Original Message ----- 
> > From: "Luc Anselin" <anselin at uiuc.edu>
> > To: "Darla Munroe" <munroe.9 at osu.edu>
> > Cc: <r-sig-geo at stat.math.ethz.ch>
> > Sent: Monday, December 13, 2004 8:36 AM
> > Subject: Re: [R-sig-Geo] spatial 2SLS
> > 
> > 
> > > I have some functions that implement this. We used them in last summer's
> > > ICPSR spatial regression course. They are not (yet) very user friendly
> > > and
> > > not up to official R standards yet, but they definitely work.
> > > The functions include:
> > > - standard 2SLS (or IV estimation)
> > > - spatial 2SLS (using WX as instruments for Wy)
> > > - heteroskedastic robust spatial 2SLS (White adjusted variances)
> > > Anyone who wants them and doesn't need a lot of hand holding can e-mail
> > > me directly.
> > > L.
> > >
> > >
> > > On Dec 13, 2004, at 8:21 AM, Darla Munroe wrote:
> > >
> > > > Is the spatial 2SLS developed by Kelejian available in spdep?  If not,
> > > > is it
> > > > available anywhere else at this time?  (besides in SpaceStat).
> > > >
> > > >
> > > >
> > > > Thanks,
> > > >
> > > > Darla Munroe
> > > >
> > > >
> > > > [[alternative HTML version deleted]]
> > > >
> > > > _______________________________________________
> > > > R-sig-Geo mailing list
> > > > R-sig-Geo at stat.math.ethz.ch
> > > > https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> > > >
> > >
> > > _______________________________________________
> > > R-sig-Geo mailing list
> > > R-sig-Geo at stat.math.ethz.ch
> > > https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> > >
> > 
> > _______________________________________________
> > R-sig-Geo mailing list
> > R-sig-Geo at stat.math.ethz.ch
> > https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> > 
> 
>     Sincerely,
>     Danlin Yu
> 
> ----------------------------------
> Lecturer, Ph.D. Candidate
> Department of Geography
> University of Wisconsin, Milwaukee
> Tel: (414)229-3943
> Fax: (414)229-3981
> Email: danlinyu at uwm.edu
> 
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo at stat.math.ethz.ch
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> 

-- 
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Breiviksveien 40, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 93 93
e-mail: Roger.Bivand at nhh.no