[R-sig-Geo] Problem with spatial error model - discontinuous polygons - memory problem

[Roger Bivand]

On Fri, 11 Jan 2008, Debarchana Ghosh wrote:

> I tried the following but I still can't get to increase the memory.
>
> I'm working on 64-bit windows machine with 16 GB RAM. However I can't get to
> use the RAM properly.
>
>> test<-errorsarlm(SALE_VALUE~., data=modnetwork, adrweights,
> zero.policy=TRUE)
> Error: cannot allocate vector of size 2.1 Gb
>> help(memory.size)
>> memory.size(max = FALSE)
> [1] 168.1048
>> memory.size(max = TRUE)
> [1] 204.4375
>> memory.limit(size = NA)
> [1] 3583.875
>> memory.limit(size = 4095.00)
> NULL
>
> How can I increase the memory size in R to utilize the 16 GB RAM.

(16GB is wasted on Windows IMHO)

Not on Windows - there are tips on the R for Windows FAQ for running the 
32-bit R binary with maximum memomy for a 32-bit platform. There are no 
gcc Windows 64-bit compilers available yet, so there is no 64-bit Windows 
binary. 64-bit service is available on OSX (experimental but including 
x86_64 at http://r.research.att.com/) and Unix/Linux (which are what the 
developeRs mostly use).

The main difficulty seems to be that the Win32 API (and ABI) was quite 
well known, while reconstructing what M$ has done with the Win64 API and 
ABI is still taking time. The MinGW developers are working on it, but 
because few applications actually need it (there are better alternatives), 
progress is slow, and Win32 binaries run on Win64 on x86_64.

In any case, finding the eigenvalues of an nxn matrix (and operating on 
such matrices) is not really necessary, since sparse matrix routes are 
provided. See the method= argument documented in ?errorsarlm - "spam" is 
somewhat faster than "Matrix" in preliminary trials.

Roger

>
> Thanks for all your help.
>
> Debs.
>
>
> On Jan 11, 2008 2:05 AM, Roger Bivand <Roger.Bivand at nhh.no> wrote:
>
>> On Thu, 10 Jan 2008, Debarchana Ghosh wrote:
>>
>>> Hi,
>>>
>>> I'm trying to run a spatial error model using the spdep package. I'm
>> working
>>> on a windows machine with 2GB ram.
>>>
>>> baseshpPIN<-readShapePoly("tricounty_2005.shp", IDvar="PIN",
>> verbose=TRUE)
>>>
>>> ## The shape has lots of discontinuous polygons
>>>
>>> basenb<-poly2nb(baseshpPIN, row.names=PIN)
>>>
>>> adrweights<-nb2listw(basenb, style="W", zero.policy=TRUE)
>>>
>>> modnetwork.errormod<-errorsarlm(SALE_VALUE~., data=modnetwork,
>> adrweights)
>>> Error in errorsarlm(SALE_VALUE ~ ., data = modnetwork, adrweights) :
>>>        NAs in lagged dependent variable
>>> In addition: Warning message:
>>> NAs in lagged values in: lag.listw(listw, y, zero.policy = zero.policy)
>>>
>>> When I tried to Moran's I test, it gave similar errors. How can I solve
>> this
>>> problem? Is there any wrong with the listw object (adrweights)?
>>
>> Add the zero.policy=TRUE argument to each subsequent command too.
>>
>> This is in ?errorsarlm:
>>
>> "zero.policy: if TRUE assign zero to the lagged value of zones without
>>           neighbours, if FALSE (default) assign NA - causing
>>           'errorsarlm()' to terminate with an error"
>>
>> Roger
>>
>>>
>>> I'll appreciate any help.
>>>
>>> Thanks,
>>> Debs.
>>>
>>>
>>>
>>>
>>
>> --
>> Roger Bivand
>> Economic Geography Section, Department of Economics, Norwegian School of
>> Economics and Business Administration, Helleveien 30, N-5045 Bergen,
>> Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
>> e-mail: Roger.Bivand at nhh.no
>>
>>
>
>
>

-- 
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand at nhh.no