Dear Aaron,

I am really  a tool user and not a tool maker (actually an ecologist 
doing some biostatistics)... so, I take the liberty of sending a copy of 
this e-mail to the r-help list where capable computer persons and true 
statisticians may provide more relevant information and also to Paulo 
Ribeiro and Peter Diggle, the authors of geoR..

I really feel that your huge matrix cannot be handled in R that easy, 
and I get the same kind of error as you:

 > m=matrix(1,ncol=18227, nrow=18227)
Error: cannot allocate vector of size 2595496 Kb
In addition: Warning messages:
1: Reached total allocation of 511Mb: see help(memory.size)
2: Reached total allocation of 511Mb: see help(memory.size)

However, if you want to compute a distance matrix, have a look to the 
function:

?dist

and try it... You will not have to create the distance matrix yourself 
from the coordinates file (but you may meet meory problem anyway). Still 
more straigthfully, if you intend to use interpolation methods such as 
kriging, you don't need to manage the distance matrix building by 
yourself. See:

library(geoR)
library(help=geoR)
?variog
?variofit
?likfit?

etc...

If you want further use geostatics (eg via geoR or gstat), you will 
anyway have to manage with memory limits (not due to R). On my computer 
(portable HP compaq nx7000) I can hardly manage with more than 2000 
observations using geoR (far from your 18227 observations). I had to 
krige on a dataset with 9000 observations recently and has been led to 
subsample randomly 2000 values. You can however try and  increase the 
memory allocated to R on your computer. The size limit is hardware 
dependent, eg:

?memory.size
?memory.limit
/ memory.limit(size=500000000) /

Another way may be to perform local kriging (eg kriging within the 
dataset on a fixed radius), but to my knowldege this cannot be done with 
geoR (unfortunately). The library gstat offers this option, but has much 
more limited possibilities than geoR considering other issues (variogram 
analysis, etc...).

library(gstat)
?krige

see argument 'maxdist'

Hope this can help,

Kind regards


Patrick Giraudoux
-- 

Department of Environmental Biology
EA3184 usc INRA
University of Franche-Comte
25030 Besancon Cedex
(France)

tel. +33 381 665 745
fax +33 381 665 797
http://lbe.univ-fcomte.fr


Aaron Swoboda a écrit :

> Dear Sir:
>
>
> I ran across your post to the R-help archive from February 9it is 
> attached below since it was nearly two years ago!). I am beginning to 
> learn R and am interested in analyzing some of my data in a spatial 
> context. I am having a problem that seems similar to the problem you 
> encountered, trying to work with a large matrix in Windows XP). I am 
> wondering if you could help steer me in the direction that helped you 
> solve your problem. I am trying to construct a distance matrix 
> containing the all of the distances between my 18,000 observations. 
> Trying to make a matrix that large gets an error message...
>
> m=matrix(1,ncol=18227, nrow=18227)  
>
> Error: cannot allocate vector of size 2595496 Kb
> In addition: Warning messages:
> 1: Reached total allocation of 1023Mb: see help(memory.size)
>
> Thank you for any help you may be able to send my way.
>
> ~Aaron Swoboda
>
> Below is your posting to the R-help list...
>
>
> Dear all,
>
> I a read with great interest the e-mails related to Arnav Sheth about 
> memory limitation when computing a distance matrix. I suspect
> that I will also meet some memory limitation using GeoR. I am 
> currently running GeoR on a geodata object including 2686 geographical
> coordinates.
>
> krige.conv() can handle it (it takes 10-15 mn of computing) but 
> requests an increased memory.
>
> /> memory.limit(size=500000000) /
>
> When the computing is completed, the computer speed is considerably 
> slowed down for any application. It is thus most necessary to
> shut it down and restart. I will probably have to handle a set of 
> 5000-6000 coordinates in once in the next few months. I wonder if
> it will go through it on my plateform (Windows XP and compaq nx7000). 
> If not, will the limitation due to R or to Windows? Does an
> alternate solution exist?
>
> Thanks for any hint,
>
> Patrick Giraudoux
>
>-- 
>    Dr. Aaron M. Swoboda
>    3208 Posvar Hall
>    Graduate School of Public and International Affairs
>    University of Pittsburgh
>    Pittsburgh, PA 15260
>
>    e-mail: aswoboda@gspia.pitt.edu
>    Office: 412-648-7604
>    Fax:    412-648-2605
>  
>

	[[alternative HTML version deleted]]


