[R-sig-Geo] Fitting SAR models in spdep (was: spatial 2SLS)

Danlin Yu danlinyu at csd.uwm.edu
Mon Dec 13 19:03:22 CET 2004


Dear Professor Roger Bivand:

    Yes, I was talking about the two SAR fitting functions. I just did a
very quick run on a dataset with 3437 observations (those are point data,
and I generate the listw object using dnearneigh() - not sure whether the
result is symmetric or asymmetric). However, I tried both "sparse" and
"SparseM" method on errorsarlm(), both failed to go through, the errors
given as follow:

    For "sparse" method:
    "Error in spwdet(sparseweights, rho = rho, debug = debug) : 
        Suspicious allocation of memory in sparse functions
         ...bailing out! Save workspace and quit R!"

    For "SparseM" method:
    "Error in .local(x, ...) : Increase tmpmax
     Error in det(chol(x)) : Unable to find the argument "x" in selecting
     a method for function "det" "

    By the way, I am still using R 2.0.0 and when I loaded the libraries,
I got warnings. I will keep trying and keep you posted as I progress.
Thanks.


On Mon, 13 Dec 2004, Roger Bivand wrote:

> On Mon, 13 Dec 2004, Danlin Yu wrote:
> 
> > Dear Mihai Nica:
> > 
> >     I think it is a memory issue, too. I ran into the same problem when
> > using around 3000 observations. Although I used memory.size() to increase
> > the memroy allocation size, spdep took too long to run.
> > 
> 
> I think you mean the two SAR fitting functions, lagsarlm() and 
> errorsarlm()? The default method is "eigen", which does need plenty of 
> memory with large n (because it both solves the eigenproblem for an nxn 
> matrix, and later inverts an nxn matrix). There has always been a "sparse" 
> method using code from Netlib called sparse from UCB, which works on some 
> platforms, but not others, and has an unresolved memory bug. 
> 
> I'm pleased to report that Roger Koenker has been kind enough to add a
> det() for symmetric (and similar to symmetric) sparse matrices to the
> SparseM package. From spdep_0.3-5, spdep depends on SparseM, and
> errorsarlm() is now pretty dependable for large n and fast. lagsarlm() is
> dependable, but its fitting speed is slowed by fitting models for LR tests
> dropping each of the X variables in turn, as well as the specified model,
> so that it fits k models, not just 1. The function calls for the lag
> models are cheaper than the error models, though.
> 
> The restriction to symmetric (and similar to symmetric) sparse matrices is
> important to bear in mind. With symmetric neighbour lists (contiguity,
> graphe-based and distance bands), and even symmetric inverse distances,
> and all weighting styles this is OK (because "W" and "S" style are similar
> to symmetric, and the others are symmetric), but absolutely not for
> k-nearest-neighbours or for asymmetric general weights (say migration
> tables).
> 
> I would be very interested in hearing whether this approach is helpful, 
> and how this scales.
> 
> Roger
> 
> >     The problem, I think, might be due to the fact that spdep's spatial
> > autoregressive models use the exact method (correct me if I am wrong),
> > while GeoDa uses an approximatioin method (see the paper "Fast maximum
> > likelihood estimation of very large spatial autoregressive models: a
> > characteristic polynomial approach", in Computational Statistics & Data
> > Analysis 35 (2001) 301-319). If the method could be incorporated in R, it
> > will be great.
> > 
> >     Danlin
> > 
> > On Mon, 13 Dec 2004, Mihai Nica wrote:
> > 
> > > Greetings:
> > > 
> > > I would like to take advantage of this thread and ask if there is a way to
> > > "enhance" the code in spdep. I have a dataset with 3030 observations and two
> > > independent variables. On the same old computer it runs great in GeoDa, but
> > > fails in R. I assume it is a memory issue, but having two different ways to
> > > check my results would be great.
> > > 
> > > Thanks,
> > > 
> > > Mihai Nica
> > > Jackson State University
> > > 155 B Parkhurst Dr.
> > > Jackson, MS 39202
> > > 601 914 0361
> > > 
> > > ----- Original Message ----- 
> > > From: "Luc Anselin" <anselin at uiuc.edu>
> > > To: "Darla Munroe" <munroe.9 at osu.edu>
> > > Cc: <r-sig-geo at stat.math.ethz.ch>
> > > Sent: Monday, December 13, 2004 8:36 AM
> > > Subject: Re: [R-sig-Geo] spatial 2SLS
> > > 
> > > 
> > > > I have some functions that implement this. We used them in last summer's
> > > > ICPSR spatial regression course. They are not (yet) very user friendly
> > > > and
> > > > not up to official R standards yet, but they definitely work.
> > > > The functions include:
> > > > - standard 2SLS (or IV estimation)
> > > > - spatial 2SLS (using WX as instruments for Wy)
> > > > - heteroskedastic robust spatial 2SLS (White adjusted variances)
> > > > Anyone who wants them and doesn't need a lot of hand holding can e-mail
> > > > me directly.
> > > > L.
> > > >
> > > >
> > > > On Dec 13, 2004, at 8:21 AM, Darla Munroe wrote:
> > > >
> > > > > Is the spatial 2SLS developed by Kelejian available in spdep?  If not,
> > > > > is it
> > > > > available anywhere else at this time?  (besides in SpaceStat).
> > > > >
> > > > >
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Darla Munroe
> > > > >
> > > > >
> > > > > [[alternative HTML version deleted]]
> > > > >
> > > > > _______________________________________________
> > > > > R-sig-Geo mailing list
> > > > > R-sig-Geo at stat.math.ethz.ch
> > > > > https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> > > > >
> > > >
> > > > _______________________________________________
> > > > R-sig-Geo mailing list
> > > > R-sig-Geo at stat.math.ethz.ch
> > > > https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> > > >
> > > 
> > > _______________________________________________
> > > R-sig-Geo mailing list
> > > R-sig-Geo at stat.math.ethz.ch
> > > https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> > > 
> > 
> >     Sincerely,
> >     Danlin Yu
> > 
> > ----------------------------------
> > Lecturer, Ph.D. Candidate
> > Department of Geography
> > University of Wisconsin, Milwaukee
> > Tel: (414)229-3943
> > Fax: (414)229-3981
> > Email: danlinyu at uwm.edu
> > 
> > _______________________________________________
> > R-sig-Geo mailing list
> > R-sig-Geo at stat.math.ethz.ch
> > https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> > 
> 
> -- 
> Roger Bivand
> Economic Geography Section, Department of Economics, Norwegian School of
> Economics and Business Administration, Breiviksveien 40, N-5045 Bergen,
> Norway. voice: +47 55 95 93 55; fax +47 55 95 93 93
> e-mail: Roger.Bivand at nhh.no
> 
> 

    Sincerely,
    Danlin Yu

----------------------------------
Lecturer, Ph.D. Candidate
Department of Geography
University of Wisconsin, Milwaukee
Tel: (414)229-3943
Fax: (414)229-3981
Email: danlinyu at uwm.edu




More information about the R-sig-Geo mailing list