[R-sig-Geo] Question spdep package - Moran's I

[Roger Bivand]

On Tue, 18 Aug 2009, Michael Haenlein wrote:

> Dear Roger,
>
> As you suggested, I transformed my network using listw2U(...) prior to
> running the errorsarlm(...) model and this time it worked fine!
> I did the analysis a couple of times using different definitions of
> proximity and independent variables and the results make sense overall:
>
> Lambda: 0.84064 LR test value: 588.13 p-value: < 2.22e-16
> Lambda: 0.074582 LR test value: 1075.5 p-value: < 2.22e-16
> Lambda: 0.21073 LR test value: 133.26 p-value: < 2.22e-16
>
> Lambda: 0.83978 LR test value: 586.29 p-value: < 2.22e-16
> Lambda: 0.074612 LR test value: 1074.4 p-value: < 2.22e-16
> Lambda: 0.20948 LR test value: 131.68 p-value: < 2.22e-16
>
> Yet, (unfortunately) another problem came up in the meantime. I tried to
> apply the same procedure to test for autocorrelation in another network I'm
> working with which is substantially larger than the first one:
>
> Number of regions: 19668
> Number of nonzero links: 51598
> Percentage nonzero weights: 0.01333867
> Average number of links: 2.623449
> Non-symmetric neighbours list
>
> Again, I applied listw2U(...) and this worked fine resulting in:
>
> Number of regions: 19668
> Number of nonzero links: 32934
> Percentage nonzero weights: 0.008513813
> Average number of links: 1.674497
>
> But when I tried to use errorsarlm(...) in the same way as before I get the
> error message:
> "Error in as_dsTMatrix_listw(listw) : ncard2 incorrectly given"
>
> As I said, the approach I took is exactly the same. Both networks are
> sampled out of a much larger network, it's just that the second network uses
> a different sampling approach and has more regions. So I assume that the
> error message is due to the larger network size.
>
> Is there any chance to work around this problem?
> Should I probably use "spam" instead of "Matrix"?
> Or is errorsarlm(...) simply just not suited to analyze networks of such
> size?

Dear Michael,

There is another implementation of errorsarlm(), called spautolm(). Could 
you try that? "Matrix" is prefered to "spam", and ongoing development is 
with the authors of the now recommended Matrix package. In spautolm(), 
some tricks that they put together are used. If you still run into 
problems, I'd be interested in trying to run the analyses with your data 
and weights, to see what may need fixing, as handling larger data sets is 
not fully settled at all, very much still work in progress.

We know how to do it in theory, and the results are OK, but there are 
wrinkles that need navigating. When the number of observations gets very 
large, there is also the issue of ensuring homogeneity (so spautolm() has 
a weights= argument; it was written to match the development of spatial 
regression in Waller & Gotway (2004)).

Best wishes,

Roger

>
> Thanks very much for your help,
>
> Best,
>
> Michael
>
>
>
> -----Original Message-----
> From: Roger Bivand [mailto:Roger.Bivand at nhh.no]
> Sent: Montag, 17. August 2009 18:27
> To: Michael Haenlein
> Cc: r-sig-geo at stat.math.ethz.ch
> Subject: RE: [R-sig-Geo] Question spdep package - Moran's I
>
> The message from the Matrix method was correct, spam ought not to have tried
> to continue, I guess it is failing and lambda is ending up at the end of its
> interval.
>
> The spatial weights must be symmetric or similar to symmetric for spatial
> regression fitting. The Jacobian is found by a sparse Cholesky
> decomposition, which means that W must be symmetric. listw2U() should get
> you there, I should have spotted the problem before. You may also need to
> play with the interval= argument.
>
> Let me know how it goes.
>
> Roger
>
>

-- 
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand at nhh.no