[R-sig-Geo] inverse distance weights matrix

Danlin Yu yud at mail.montclair.edu
Tue Apr 11 16:28:18 CEST 2006


Terry:

According to my experience, if you set the tol.solve to a smaller value, i.e., smaller than the reported one at e-019, errorsarlm will work through.

Cheers,

___________________________________________
Danlin Yu, Ph.D.
Assistant Professor
Department of Earth & Environmental Studies
Montclair State University
Montclair, NJ, 07043
Tel: 973-655-4313
Fax: 973-655-4072
email: yud at mail.montclair.edu

----- Original Message -----
From: "Griffin, Terry W" <twgriffi at purdue.edu>
Date: Tuesday, April 11, 2006 9:56 am
Subject: Re: [R-sig-Geo] inverse distance weights matrix

> Thanks Roger,
> 
> I've been working on creating inverse distance matrices; and have been
> successful in this first step.  However, as you suggested, the
> connectedness of these matrices will cause procedures to be slow, even
> prohibitive with my computer (P4 3Gz 1GB RAM) (ran out of RAM).
> 
> I've tried a simpler first order queen weights matrix to see if I 
> couldget the errorsarlm model to run; but come to a different 
> problem and an
> error message:
> 
> source("spat.cot.R")
> Error in solve.default(asyvar, tol = tol.solve) : 
>        system is computationally singular: reciprocal condition 
> number= 1.16119e-019
> > traceback()
> 6: solve.default(asyvar, tol = tol.solve)
> 5: solve(asyvar, tol = tol.solve)
> 4: errorsarlm(YLD02 ~ CLAY + DIST + PEG + ROT + SUN, data = 
> cotton, 
>       q1)
> 3: eval.with.vis(expr, envir, enclos)
> 2: eval.with.vis(ei, envir)
> 1: source("spat.cot.R")
> 
> where "spat.cot.R" contains:
> spatcot.err<-errorsarlm(YLD02~CLAY+DIST+PEG+ROT+SUN,data=cotton, q1)
> 
> Do I need to set my tol to a different value or am I having other
> issues?
> 
> Thank you,
> 
> Terry
>  
> 
> 
> 
> -----Original Message-----
> From: Roger Bivand [Roger.Bivand at nhh.no] 
> Sent: Saturday, March 18, 2006 3:07 PM
> To: Griffin, Terry W
> Cc: r-sig-geo at stat.math.ethz.ch
> Subject: RE: [R-sig-Geo] inverse distance weights matrix
> 
> On Sat, 18 Mar 2006, Griffin, Terry W wrote:
> 
> > Thank you Roger,
> > 
> > I've posted the original shape file for the dataset on the 
> website at:
> > http://web.ics.purdue.edu/~twgriffi/R_matrix.htm
> > 
> > One shape file is projected while the other is not; however both 
> setsof
> > *.dbf are identical and contain both decimal degrees and 
> Northing and
> > Easting in meters.  
> > 
> > I chose a <=75 meter distance threshold and a power=1.  
> > 
> > The final inverse distance matrix was row standardized so it is not
> > symmetric. The weight characteristics you list below are nearly
> > identical to those that SpaceStat provides.  
> 
> OK - it looks like this at the moment:
> 
> library(rgdal)
> cotton_proj <- readOGR(".", "cotton_proj") # for shapefiles in the
> current
>                	# working directory, first argument
> plot(cotton_proj, cex=0.2, axes=TRUE)
> library(spdep)
> d75 <- dnearneigh(coordinates(cotton_proj), 0, 75)
> dlist <- nbdists(d75, coordinates(cotton_proj))
> idlist <- lapply(dlist, function(x) 1/x)
> w75 <- nb2listw(d75, glist=idlist, style="W")
> 
> Characteristics of d75:
> 
> > d75
> Neighbour list object:
> Number of regions: 2451 
> Number of nonzero links: 1182994 
> Percentage nonzero weights: 19.69228 
> Average number of links: 482.6577 
> 
> But:
> 
> > inMAT <- matrix(scan("cottonID75wm", sep=","), nrow=2451, 
> ncol=2451, 
> + byrow=TRUE)
> > range(rowSums(inMAT))
> [1] 0.8159028 1.9115231
> > d75MAT <- nb2mat(d75, glist=idlist, style="W")
> > range(rowSums(d75MAT))
> [1] 1 1
> > w75 <- nb2listw(d75, glist=idlist, style="W")
> > range(sapply(w75$weights, sum))
> [1] 1 1
> 
> So something in the construction of your matrix wasn't securing 
> row 
> standardisation. The values do differ, this is the upper left corner:
> 
> > d75MAT[1:5,1:5]
>        [,1]       [,2]       [,3]       [,4]       [,5]
> 1 0.00000000 0.06735710 0.03227068 0.02182278 0.01660474
> 2 0.05975093 0.00000000 0.05493976 0.02863058 0.01954588
> 3 0.02615210 0.05019078 0.00000000 0.05461937 0.02771729
> 4 0.01639324 0.02424508 0.05062943 0.00000000 0.05216365
> 5 0.01172946 0.01556467 0.02416008 0.04905229 0.00000000
> > inMAT[1:5,1:5]
>         [,1]     [,2]     [,3]     [,4]     [,5]
> [1,] 0.000000 0.131420 0.061844 0.041726 0.031848
> [2,] 0.101440 0.000000 0.090156 0.047180 0.032441
> [3,] 0.040305 0.076122 0.000000 0.083545 0.042787
> [4,] 0.023708 0.034730 0.072836 0.000000 0.076431
> [5,] 0.016296 0.021506 0.033593 0.068830 0.000000
> > rowSums(inMAT[1:5,])
> [1] 1.911523 1.658689 1.533153 1.440908 1.379051
> > rowSums(d75MAT[1:5,])
> 1 2 3 4 5 
> 1 1 1 1 1 
> 
> They are not the same matrices, although the differences are very 
> small- 
> here I've used the shapefile points, which are stored as binary 
> floating
> point, rather than using the DBF values, which are a possibly 
> rounded 
> character representation, and that might be enough to cause the 
> difference. I think you should be able to generate w75, and that it
> should 
> do what you need, but I'd be grateful for feedback on how you get on.
> 
> Best wishes,
> 
> Roger
> 
> PS: with numbers of neighbours between 177 and 678 (both in your 
> matrix 
> and in d75), even allowing for the dampening effect of inverse 
> distance,
> most analyses will be slow. 75m means that many sites are related to
> many 
> in your case. Doing:
> 
> symbols(408725, 3660800, circles=75, inches=FALSE, fg="red", add=TRUE)
> 
> on top of the earlier plot shows your chosen neighbourhood for an 
> arbitrary point.
> 
> > 
> > I really appreciate this.  Thank you,
> > 
> > Terry
> > 
> > 
> > 
> > -----Original Message-----
> > From: Roger Bivand [Roger.Bivand at nhh.no] 
> > Sent: Friday, March 17, 2006 5:50 PM
> > To: Griffin, Terry W
> > Cc: r-sig-geo at stat.math.ethz.ch
> > Subject: RE: [R-sig-Geo] inverse distance weights matrix
> > 
> > Terry:
> > 
> > http://web.ics.purdue.edu/~twgriffi/mat4rf
> > 
> > and 
> > 
> > http://web.ics.purdue.edu/~twgriffi/mat4rs
> > 
> > throw access errors.
> > 
> > Do you have the coordinates and your specifications for the 
> inverse 
> > weighted neighbours (distance threshold, power if any)? 
> > 
> > inMAT <- matrix(scan("cottonID75wm", sep=","), nrow=2451, 
> ncol=2451, 
> >   byrow=TRUE)
> > 
> > does read the matrix - the weights aren't symmetric, are they?
> > 
> > From the weights matrix I get:
> > 
> > inMATlw <- mat2listw(inMAT)
> > 
> > print(is.symmetric.nb(inMATlw$neighbours))
> > [1] TRUE
> > 
> > print(inMATlw$neighbours)
> > Neighbour list object:
> > Number of regions: 2451 
> > Number of nonzero links: 1182982 
> > Percentage nonzero weights: 19.69208 
> > Average number of links: 482.6528 
> > 
> > If you could post the coordinates and your specifications, I 
> could see
> > how 
> > to reproduce the weights within R if you like.
> > 
> > Roger
> > 
> > 
> > On Fri, 17 Mar 2006, Griffin, Terry W wrote:
> > 
> > > 
> > > The MATLAB routines are the Spatial Econometrics Toolbox 
> format.  I
> > > tried the read.dat2listw() in R; the code and error message 
> for a
> > small
> > > matrix example is below; this uses a full matrix from MATLAB
> > (realizing
> > > that "read.dat2listw" is intended for sparse matrices). 
> > > 
> > > > mattry<-read.dat2listw("mat4rf")
> > > Error in "[.default"(sn, , 1) : incorrect number of dimensions
> > > > traceback()
> > > 5: NextMethod("[")
> > > 4: "[.factor"(sn, , 1)
> > > 3: sn[, 1]
> > > 2: unique(sn[, 1])
> > > 1: read.dat2listw("mat4rf")
> > > 
> > > 
> > > I'm not able to use dlmwrite in MATLAB to export the sparse matrix
> so
> > I
> > > tried to import the full matrix into R (code is above).  The 
> MATLAB> code
> > > I'm using came from this listserve archive.  The MATLAB code and
> error
> > > message are below.  I'm able to use dlmwrite to create the MATLAB
> full
> > > matrix (used above in the R code).
> > > 
> > > load testmat.GWT; A = (spconvert(testmat)); %have to change 
> name in
> > two
> > > locations on this line
> > > n=size(A); nobs=n(1);
> > >  % Normalize matrix (adapted form LeSage 1999)
> > > wt = sparse(A);
> > >  [i1,j1,s1]=find(wt);
> > > rsum=sum(wt);
> > > for i=1:nobs;
> > >             ind=find(i1==i);
> > > s1(ind,1)=s1(ind,1)/rsum(1,i);
> > > end;
> > > [m,n] = size(wt);
> > > W = sparse(i1,j1,s1,m,n);
> > > S = sparse(W);
> > > dlmwrite('mat4rst', S); %give the new sparse weight matrix a
> permanent
> > > name for R 
> > > ??? Error using ==> sprintf
> > > Function is not defined for sparse inputs.
> > > 
> > > Error in ==> dlmwrite at 172
> > >         str = sprintf(format,m(i,:));
> > > 
> > > If anyone wants to give it a try, I've put the small example
> matrices
> > > and the "real" matrix up on my website at:
> > > http://web.ics.purdue.edu/~twgriffi/R_matrix.htm
> > > 
> > > My objective is to use R for my spatial data analysis using 
> inverse> > distance weights matrices.  I'm struggling with the 
> inverse distance
> > > matrices.  I hope to either create these inverse distance matrices
> in
> > > SpaceStat then import into R or create the matrices in R.  
> > > 
> > > Thank you,
> > > 
> > > Terry
> > > 
> > > 
> > > 
> > > 
> > > -----Original Message-----
> > > From: Roger Bivand [Roger.Bivand at nhh.no] 
> > > Sent: Friday, March 17, 2006 2:43 PM
> > > To: Griffin, Terry W
> > > Cc: r-sig-geo at stat.math.ethz.ch
> > > Subject: Re: [R-sig-Geo] inverse distance weights matrix
> > > 
> > > On Thu, 16 Mar 2006, Griffin, Terry W wrote:
> > > 
> > > > Greetings,
> > > > 
> > > >  
> > > > 
> > > > I'm migrating from MATLAB to R and am having trouble converting
> > > inverse
> > > > distance weights matrix into a form usable by R.  I've 
> created the
> > > > matrix in SpaceStat (in *.FMT format) and converted to *.GWT
> format
> > > and
> > > > also have it in MATLAB *.m format.
> > > > 
> > > 
> > > I wonder whether your Matlab data are in the Spatial Econometrics
> > > Toolbox 
> > > format - if they are, you could try the read.dat2listw() 
> function 
> > > mentioned on the read.gwt2nb function help page.
> > > >  
> > > > 
> > > > I've tried using the read.gwt2nb in R, but without success.  
> There> are
> > > > nearly 2,451 observations with an average of 482 linkages.  
> About20
> > %
> > > > have nonzero weights.  Any suggestions are appreciated.
> > > 
> > > Without details of what you mean by "without success", it's 
> hard to
> > > offer 
> > > any advice. Could you create a small example of the problems 
> you are
> 
> > > seeing, and repost with copies of the commands you give and any
> error 
> > > messages? 
> > > 
> > > If you are seeing errors ("Error in ..." messages from R), please
> also
> > > type traceback() at the prompt and report the output. You 
> could also
> > put
> > > the uncooperative GWT file on a website for others to try to see
> what
> > is
> > > going on.
> > > 
> > > > 
> > > >  
> > > > 
> > > > Thank you,
> > > > 
> > > > Terry
> > > > 
> > > >  
> > > > 
> > > >  
> > > > 
> > > > Terry W. Griffin
> > > > 
> > > > Graduate Research Assistant
> > > > 
> > > > Agricultural Economics
> > > > 
> > > > Purdue University
> > > > 
> > > > 403 W State St
> > > > 
> > > > West Lafayette, IN 47907
> > > > 
> > > > 765-494-4257
> > > > 
> > > > http://web.ics.purdue.edu/~twgriffi/
> > > > <http://web.ics.purdue.edu/~twgriffi/> 
> > > > 
> > > >  
> > > > 
> > > >  
> > > > 
> > > > 
> > > > 	[[alternative HTML version deleted]]
> > > > 
> > > > _______________________________________________
> > > > R-sig-Geo mailing list
> > > > R-sig-Geo at stat.math.ethz.ch
> > > > https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> > > > 
> > > 
> > > 
> > 
> > 
> 
> -- 
> Roger Bivand
> Economic Geography Section, Department of Economics, Norwegian 
> School of
> Economics and Business Administration, Helleveien 30, N-5045 Bergen,
> Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
> e-mail: Roger.Bivand at nhh.no
> 
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo at stat.math.ethz.ch
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>




More information about the R-sig-Geo mailing list