[R-sig-Geo] Spatial Lag and Error Model

[Roger Bivand]

On Fri, 25 Mar 2011, Janmaat, John wrote:

> Hello again,
>
> Thanks for the suggestion of updating the packages.  That dealt with one
> issue.  For the rest, here is some of the info you asked for.  For
> errorsarlm and lagsarlm I get a memory allocation problem.
>
>> sessionInfo()
> R version 2.12.1 (2010-12-16)
> Platform: i386-pc-mingw32/i386 (32-bit)
>
> locale:
> [1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United
> States.1252    LC_MONETARY=English_United States.1252
> [4] LC_NUMERIC=C                           LC_TIME=English_United
> States.1252
>
> attached base packages:
> [1] stats     graphics  grDevices utils     datasets  methods   base
>
>
> other attached packages:
> [1] RANN_2.1.2         pgirmess_1.5.1     spdep_0.5-29
> coda_0.14-2        deldir_0.0-13      nlme_3.1-98        MASS_7.3-11
> Matrix_0.999375-48
> [9] boot_1.2-43        spgwr_0.6-10       maptools_0.8-4
> lattice_0.19-17    foreign_0.8-42     sp_0.9-79
>
> loaded via a namespace (and not attached):
> [1] grid_2.12.1  tools_2.12.1
>
> Here are the error messages when commands run for full sample:
>
>> wd.GMerrorsarSum <- GMerrorsar(frmSum,data=wd2,weightings)
>> wd.GMerrorsarWin <- GMerrorsar(frmWin,data=wd2,weightings)
>> wd.errorsarSum <- errorsarlm(frmSum,data=wd2,weightings)
> Error: cannot allocate vector of size 922.5 Mb

See ?errorrsarlm, method= argument. You should not expect to be able to 
use dense matrix methods with as many observations. For k nearest 
neighbours, I suggest LU for an exact result, perhaps MC for an 
approximation.

>> wd.errorsarWin <- errorsarlm(frmWin,data=wd2,weightings)
> Error: cannot allocate vector of size 922.5 Mb

Ditto.

>> wd.MESSlagsarSum <- lagmess(frmSum,data=wd2,weightings)
>> wd.MESSlagsarWin <- lagmess(frmWin,data=wd2,weightings)
>> wd.lagsarSum <- lagsarlm(frmSum,data=wd2,weightings)
> Error: cannot allocate vector of size 922.5 Mb

Ditto - ?lagsarlm

>> wd.lagsarWin <- lagsarlm(frmWin,data=wd2,weightings)
> Error: cannot allocate vector of size 922.5 Mb

Ditto.

You may also see singularities because your RHS variables appear to be 
close to colinear.

>> wd.GMerrorsarSum <- gstsls(frmSum, data=wd2,weightings)
>> wd.GMerrorsarWin <- gstsls(frmWin, data=wd2,weightings)
>
> If I run the same set of commands on a subset of the data, then the
> command output is:
>
>> wd.GMerrorsarSum <- GMerrorsar(frmSum,data=wd2,weightings)
>> wd.GMerrorsarWin <- GMerrorsar(frmWin,data=wd2,weightings)
>> wd.errorsarSum <- errorsarlm(frmSum,data=wd2,weightings)
> Error in solve.default(asyvar, tol = tol.solve) :
>  system is computationally singular: reciprocal condition number =
> 8.6909e-27
>> wd.errorsarWin <- errorsarlm(frmWin,data=wd2,weightings)
> Error in solve.default(asyvar, tol = tol.solve) :
>  system is computationally singular: reciprocal condition number =
> 8.34845e-27

Both of these come when inverting the asymmetric covariance matrix of the 
variables, read about why this happens on the function help pages - it is 
described in details, and may also concern the scaling of your variables.

>> wd.MESSlagsarSum <- lagmess(frmSum,data=wd2,weightings)
>> wd.MESSlagsarWin <- lagmess(frmWin,data=wd2,weightings)
>> wd.lagsarSum <- lagsarlm(frmSum,data=wd2,weightings)
> Error in solve.default(inf, tol = tol.solve) :
>  system is computationally singular: reciprocal condition number =
> 7.82721e-27
>> wd.lagsarWin <- lagsarlm(frmWin,data=wd2,weightings)
> Error in solve.default(inf, tol = tol.solve) :
>  system is computationally singular: reciprocal condition number =
> 7.81985e-27

Ditto.

>> wd.GMerrorsarSum <- gstsls(frmSum, data=wd2,weightings)
>> wd.GMerrorsarWin <- gstsls(frmWin, data=wd2,weightings)
>>
>
> Suggestions?
>

Read the function help pages fully before you use the functions?

Roger

>
>
> -----Original Message-----
> From: Roger Bivand [mailto:Roger.Bivand at nhh.no]
> Sent: Friday, March 25, 2011 4:12 AM
> To: Janmaat, John
> Cc: r-sig-geo at r-project.org
> Subject: Re: [R-sig-Geo] Spatial Lag and Error Model
>
> On Thu, 24 Mar 2011, Janmaat, John wrote:
>
>>
>>
>> -----Original Message-----
>> From: Janmaat, John
>> Sent: Thursday, March 24, 2011 3:40 PM
>> To: 'Roger.Bivand at nhh.no'
>> Subject: RE: [R-sig-Geo] Spatial Lag and Error Model
>>
>> Hello Roger,
>>
>> Thanks for your reply.
>>
>> A code snippet:
>>
>> nl <- knearneigh(cbind(wd2$Long,wd2$Lat),k=4)
>> nb <- knn2nb(nl)
>> weightings <- nb2listw(nb)
>> wd.GMerrorsarSum <- gstsls(frmWin, data=wd2,weightings)
>>
>> The error message:
>>
>> Error in solve.default(QQ, Qye) :
>>  system is computationally singular: reciprocal condition number =
>> 4.20784e-28
>>
>> There are 10,996 observations.
>
> You did not provide sessionInfo() output. If you update spdep, you'll
> find
> that this issue (of poorly conditioned crossproduct matrices in the
> stsls
> step) was addressed in December 2010, and the current release should
> work.
>
> The code and error messages from errorsarlm() and lagsarlm() will
> probably
> also show other misunderstandings, but that can be addressed if you post
>
> them.
>
> Hope this helps,
>
> Roger
>
>>
>> The dataset has been processed to remove any overlapping points (no
> zero
>> distances) and to remove points that do not have at least one
> neighbour
>> within 100m.  Overlapping points existed as accounts would change when
> a
>> resident moved, leading to multiple observations for a single lot.
> This
>> is what I originally thought could be the issue, as there are then
> zero
>> distances.  It has also had all observations with NA removed.  The
> model
>> represented in frmWin is solved fine by lm(), with no variables
> dropped.
>>
>> I'm not sure how to check for near linear dependence in WX though, so
> I
>> would appreciate it if you could direct me there.
>>
>> I am using k nearest neighbours, as opposed to rook or queen as my
> data
>> has lat and long for the lot centroid, as opposed to a polygon for
> each
>> lot.
>>
>> There are some natural boundaries within the data, such that it can be
>> divided into subsets where within each subset the neighbour list for
> the
>> observations in the subset is no different from that generated for the
>> whole dataset.  I have also done an analysis separately for such
>> subsets, with the same result.
>>
>> Thanks in advance for any suggestions.
>>
>> John.
>>
>> -----Original Message-----
>> From: Roger Bivand [mailto:Roger.Bivand at nhh.no]
>> Sent: Thursday, March 24, 2011 1:27 AM
>> To: Janmaat, John
>> Cc: r-sig-geo at r-project.org
>> Subject: Re: [R-sig-Geo] Spatial Lag and Error Model
>>
>> On Wed, 23 Mar 2011, Janmaat, John wrote:
>>
>>> Hello,
>>>
>>>
>>>
>>> I have a city water use dataset with almost 12000 observations.  I am
>>> using spdep.  I am using K nearest neighbours (4) to build the
>> neighbour
>>> list.  I am trying to estimate a model to predict household water
> use,
>>> controlling for a set of independent variables like lot size, house
>>> size, etc.  The spatial model diagnostics, lm.LMtests, suggests that
>>> there is both a spatial error and a spatial lag in the data.
> However,
>>> on the lm estimations (lagsarlm,errorsarlm), report that the system
> is
>>> singular.  I can estimate the model using GMerrorsar and lagmess.
>>> However, gstsls exits with the same error as the lm estimations.
>>>
>>> Is there a function I am missing to estimate a model with both
> spatial
>>> autocorrelation and a spatial lagged dependent variable that won't
>> have
>>> the singularity problem?
>>>
>>
>> Well, what we are missing are the verbatim error messages and function
>> calls. It may well be that your diagnosis of the problem is not
> precise
>> enough, especially as the code used depends on your choice of input
>> arguments. Does lm() of the same model report any unfitted
> coefficients
>> (are there near-linear dependencies present in the X variables, or
>> between
>> X and WX)?
>>
>> Roger
>>
>>>
>>>
>>> Thanks,
>>>
>>>
>>>
>>> John.
>>>
>>>
>>>
>>> --------------------
>>>
>>> Dr. John Janmaat
>>>
>>> Department of Economics
>>>
>>> I.K. Barber School of Arts and Sciences
>>>
>>> University of British Columbia - Okanagan Campus
>>>
>>> 3333 University Way, Kelowna, BC
>>>
>>> V1V 1V7
>>>
>>> Tel: (250)807-8021
>>>
>>> WWW: http://people.ok.ubc.ca/jjanmaat/
>>>
>>>
>>>
>>>
>>> 	[[alternative HTML version deleted]]
>>>
>>> _______________________________________________
>>> R-sig-Geo mailing list
>>> R-sig-Geo at r-project.org
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>
>>
>>
>
>

-- 
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand at nhh.no