[R-sig-Geo] memory problems with GWR tests

[Roger Bivand]

On Mon, 19 May 2008, Michal Gallay wrote:

> Dear R-Experts,
>
> there is an error message regarding allocation of memory coming up after 
> attempting to run GWR with hatmatrix=TRUE (there is no problem when this is 
> set to FALSE).
>
> I would like to test GWR outputs using e.g. LMZ.F2GWR.test(x)or 
> BFC99.gwr.test(x).
>
>>  resid.Gauss <- gwr(regress.formula, data=resid, gweight=gwr.Gauss,
>>  bandwidth=10, hatmatrix=TRUE))
>
> Error: cannot allocate vector of size 505.4 Mb
>
>
> Why it cannot allocate 505 MB if there is 2 GB RAM on my laptop?
> Running R from WinXP command prompt:
> Rgui --max-mem-size=1800Mb
> The memory problem persists.
>
> I read the instructions regarding the memory.limit(), Memory(), as well as 
> the R help.I could decimate the number of points involved in GWR but that 
> would be a test on a different scale, wouldn't it? The datasets comprise 8 
> 000 - 1 000 000 points. A Monte Carlo simultation would require too much 
> time.
>
> I would like to know if there would be any way around the memory problem to 
> test a GWR object.

Absolutely no way round at all, the hat matrix must be n x n (and you need 
several copies typically). In addition, the tests only have meaning if the 
assumptions they require are met, which under non-stationarity they 
typically are not. So if you need to test, choose several subsets of 
observations, and see if the test results correspond.

Hope this helps,

Roger

>
> I much appreciate any advice.
>
> Michal
>
> PS: There is some other information about the session:
>
> My seesion info:
>
> sessionInfo()
> R version 2.6.2 (2008-02-08) i386-pc-mingw32 
> locale: 
> LC_COLLATE=Slovak_Slovakia.1250;LC_CTYPE=Slovak_Slovakia.1250;LC_MONETARY=Slovak_Slovakia.1250;LC_NUMERIC=C;LC_TIME=Slovak_Slovakia.1250
>
> attached base packages:
> [1] stats     graphics  grDevices utils     datasets  methods   base 
> other attached packages: [1] lattice_0.17-6 spgwr_0.5-1 maptools_0.6-5 
> foreign_0.8-23 sp_0.9-23
>
> loaded via a namespace (and not attached):
> [1] grid_2.6.2
>>  gc()
>        used (Mb) gc trigger  (Mb)  max used  (Mb)
> Ncells 269045  7.2     531268  14.2    531268  14.2
> Vcells 413865  3.2   84325970 643.4 105241855 803.0
>>  memory.profile()
>       NULL      symbol    pairlist     closure environment     promise
>          1        6876      143458        3669         825        5271
>   language     special     builtin        char     logical     integer
>     37538         151        1464       16489        4324        5293 double
>     complex   character         ...         any        list
>       2446           1       36010           0           0        3262 
> expression    bytecode externalptr     weakref         raw
>          1           0         665         177           1
>> 
>
>

-- 
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand at nhh.no