[R-sig-Geo] ggwr and memory problems

[Roger Bivand]

On Mon, 21 Jan 2008, Luca Moiana wrote:

>
>
>
>> Date: Mon, 21 Jan 2008 14:38:18 +0100
>> From: Roger.Bivand at nhh.no
>> To: luca_moiana at hotmail.com
>> CC: r-sig-geo at stat.math.ethz.ch
>> Subject: Re: [R-sig-Geo] ggwr and memory problems
>>
>> On Mon, 21 Jan 2008, Luca Moiana wrote:
>>
>>> Dear List,
>>>
>>> Here is my problem:
>>>
>>> I wanna run a ggwr on a 9000 records Spatial Points Data Frame using R
>>> on a Windows Machine (Dual processor, 4 GB RAM).
>>
>> Have you tuned Windows memory use as discussed in the R for Windows FAQ -
>> section 2.9? The binaries are 32-bit, and need to be told how much memory
>> to use when trying to carry out memory intensive work.
>
> We tried this but didn't change anything.

OK. It may run on Linux, because the memory allocation there accepts many 
small free patches but Windows wants a single free chunk the size of the 
request.

>
>
>>
>>>
>>> When I try to calculate bandwidth using:
>>>
>>> Sdati14400test.sel
>>> <- ggwr.sel(E14400 ~ V211 + V213 + V240 + V313 + V321 + V322 + V331511 +
>>> LnMPI25l.max + B:A, family = poisson(link = log), data = Sdati14400test,
>>> coords=Sdati14400test.coords, adapt = FALSE, gweight = gwr.gauss, verbose =
>>> TRUE, longlat = FALSE)
>>>
>>> I get a memory allocation error saying that the software is not able to
>>> allocate a 749 Mb memory.
>>>
>>> Any suggestion??
>>
>> It isn't strictly necessary to use all the observations to find the
>> bandwidth - take a couple of 5% samples and see if the results differ
>> much.
>
> I didn't know that and I would try, but then I'll have memory problems when I try to run ggwr??
> Is there a command to obtain a random 5% sample??
>

Try subsetting the data= argument object: df[o,] with the output of o <- 
sample(). Remember to say set.seed(whatever) to be able to repeat if need 
be.

>
>>
>>>
>>> I can also switch and use the same machine with a 64bit Ubuntu SO.
>>>
>>
>> You can try that, but consider dividing the fit.points up into chunks, and
>> running several R processes when actually fitting the ggwr model. The data
>> points stay the same, but fit subsets of the fit.points in separate
>> processes.
>
> I don't have fit.points cause I'm working on the entire Lombardy Region 
> (Northern Italy) and I'd like to compare the model from ggwr with glm 
> models a colleague obtained from a regular glm.

If no fit.points are given, the data points are copied across as fit 
points internally. You are free to subset the data.points into many 
fit.points, and concatenate the output objects afterwards. This should 
remove the difficulty.

Roger

>
> MANY THANKS
>
>
>>
>> ggwr() has not (yet) been adapted for using a cluster, but gwr() has and a
>> snow socket cluster will run happily on Linux there, and since it is run
>> within the function, it concatenates the results before returning. If this
>> would be useful of ggwr(), consider taking a look at the code.
>>
>> Roger
>>
>>>
>>> THANK A LOT
>>>
>>>
>>>
>>> Luca Moiana
>>>
>>>
>>> _________________________________________________________________
>>>
>>>
>>> 	[[alternative HTML version deleted]]
>>>
>>> _______________________________________________
>>> R-sig-Geo mailing list
>>> R-sig-Geo at stat.math.ethz.ch
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>
>>
>> --
>> Roger Bivand
>> Economic Geography Section, Department of Economics, Norwegian School of
>> Economics and Business Administration, Helleveien 30, N-5045 Bergen,
>> Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
>> e-mail: Roger.Bivand at nhh.no
>>
>
> _________________________________________________________________
> Express yourself instantly with MSN Messenger! Download today it's FREE!
> http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

-- 
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand at nhh.no