[R-sig-Geo] Question spdep package - Moran's I

[Roger Bivand]

On Mon, 17 Aug 2009, Michael Haenlein wrote:

> Dear Roger,
>
> Thanks again for your help!
>
> Based on your recommendations, I switched to errorsarlm.
> Again, method="eigen" resulted in the error message "Error: cannot allocate
> vector of size 340.5 Mb" - but I expected that.
>
> I then tried method="Matrix" and got another error message:
> "Non-symmetric neighbours list
> Error in as_dsTMatrix_listw(listw) : not a symmetric matrix"
>
> Finally I tried method="spam" and got a series of error messages:
> "Error in stop("insufficient space for sparse matrix addition") : no
> function to return from, jumping to top level
> In addition: Warning message: In optimize(sar.error.f.sp, interval =
> interval, maximum = TRUE,  :   NA/Inf replaced by maximum positive value"
>
> Although I got a whole bunch of these messages my model converged to a
> solution, but it does not feel right as Lambda is essentially 1 and I don't
> get a log likelihood:

The message from the Matrix method was correct, spam ought not to have 
tried to continue, I guess it is failing and lambda is ending up at the 
end of its interval.

The spatial weights must be symmetric or similar to symmetric for spatial 
regression fitting. The Jacobian is found by a sparse Cholesky 
decomposition, which means that W must be symmetric. listw2U() should get 
you there, I should have spotted the problem before. You may also need to 
play with the interval= argument.

Let me know how it goes.

Roger

>
> Call: errorsarlm(formula = y ~ Gender, data = Michael_dat, listw =
> Network_01, method = "spam")
> Type: error
>
> Coefficients:
>     lambda (Intercept)     GenderM    GenderND     GenderU
> 0.99899996  3.86318643  0.11191533 -0.08194235 -0.55121204
>
> Log likelihood: NA
>
> I really feel bad about asking you again.
> I will try to make my network symmetrical and hope that this will solve the
> error in the method="Matrix" option.
> Where the other errors in method="spam" are concerned, I honestly don't have
> a clue where the issue might be.
> If you could point me to any direction where I might have made a mistake,
> I'd appreciate it a lot.
>
> Thanks again very, very much for your help,
>
> Best,
>
> Michael
>
>
>
> -----Original Message-----
> From: Roger Bivand [mailto:Roger.Bivand at nhh.no]
> Sent: Montag, 17. August 2009 16:20
> To: Michael Haenlein
> Cc: Haenlein, Michael; r-sig-geo at stat.math.ethz.ch
> Subject: RE: [R-sig-Geo] Question spdep package - Moran's I
>
> On Mon, 17 Aug 2009, Michael Haenlein wrote:
>
>> Dear Roger,
>>
>> Thanks very much for your reply!
>>
>> I think what the reviewer is referring to is a lagsarlm() model --
>> thanks very much for bringing this functionality to my attention!
>>
>> I just tried to fit such a model but I get an error message: "Error:
>> Cannot allocate vector of size 340.5 MB".
>>
>> I assume this is because my network is very large:
>> Number of regions: 6681
>> Number of nonzero links: 39770
>> Percentage nonzero weights: 0.08909896 Average number of links:
>> 5.952702 Non-symmetric neighbours list
>>
>
> ?larsarlm and see method="Matrix". lagsarlm() is not the same model as
> errorsarlm(). It will also run slowly in the sparse matrix version; I
> suggest using errorsarlm() which does include the spatial autocorrelation
> of the error in the model, rather than the spatial autocorrelation of the
> dependent variable, so is better aligned with what lm.morantest() was
> testing. Both will give likelihood ratio tests of the significance of the
> spatial coefficient, but lagsarlm() only gives LR tests of the
> coefficients on the x, while with errorsarlm(), you get more-or-less
> regular t-tests (they are z-tests because the coefficient standard errors
> are the asymptotic ones).
>
> Unless you have a substantive hypothesis for how y_i affects y_j
> irrespective of the x variables, errorsalm() may be a safer fallback,
> because it is only saying that there is some residual autocorrelation in
> the aspatial model, and that is what gets modelled.
>
> Best wishes,
>
> Roger
>
>>
>> I'm working on Lenovo Thinkpad (X300) with Windows Vista Business, SP 2
> and
>> 2.00 GB RAM.
>> Would you know whether there is any chance to run this model without
>> upgrading my RAM?
>> My current memory.limit() in R is 1535 which seems as far as I can get
> with
>> 2.00 GB RAM.
>> Otherwise I might need to find someone with a more powerful machine than
>> mine.
>>
>> Thanks very much again for having pointed me towards lagsarlm()!
>>
>> Best,
>>
>> Michael
>>
>>
>>
>> -----Original Message-----
>> From: Roger Bivand [mailto:Roger.Bivand at nhh.no]
>> Sent: Montag, 17. August 2009 14:30
>> To: Haenlein, Michael
>> Cc: r-sig-geo at stat.math.ethz.ch
>> Subject: Re: [R-sig-Geo] Question spdep package - Moran's I
>>
>> On Mon, 17 Aug 2009, Haenlein, Michael wrote:
>>
>>
>> If the referee is refering to the social networks literature, you will
> have
>> to find out what "one step" means there, if anything.
>> It is not a term used in connection with Moran's I for spatial data.
>>
>> If you believe that the referee's "one step" might be met by fitting a
> model
>> including both the x variables and the dependence in either y or the
>> residuals of the regression, look at lagsarlm() and errorsarlm(). Maybe
>> check against the Ward/Gleditsch "Spatial regression models" Sage volume
> if
>> need be.
>>
>> When working across disciplines, nothing constitutes absolute authority in
>> this way. Testing residuals ought to be OK, but see Schabenberger & Gotway
>> for caveats (especially about concluding that autocorrelation is present
>> when the real problem is model misspecification).
>>
>> Hope this helps,
>>
>> Roger
>>
>>
>
>

-- 
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand at nhh.no