[R-SIG-Mac] memory allocation problems

[Antonio P. Ramos]

Thanks for the comments.

What I'm doing is very simple: I'm running an one dimensional item
response model, similar to the ones use in psychology and educational
testing data via Markov Chain Monte Carlo Methods.


model_m12<- ideal(rollcall_m2,  maxiter = 500 000 000, thin = 1000,
burnin = 5000,
		   store.item = TRUE,  normalize=T,
priors=list(xp=1e-12,xpv=1e-12,bp=1e-12,bpv=1e-12), verbose=T)

# my data matrix is provided by the rollcall object, but it has just
155*17 dimensions: rollcall_m2

# the number of interactions is maxiter/thin= 500,000

# store.item=true is the main source of the problem: it's store the
discrimination
# parameter, which consumer a large amount of memory. Unfortunately, I
need this information.


So, if R can access up to 3.5 Gb how can I fix the problem. I'm sure
lot's of mac users will be also interested in increasing its memory
allocation capabilities in R.

Cheers,

Antonio.





On Mon, Jun 30, 2008 at 11:38 AM, Kasper Daniel Hansen
<khansen at stat.berkeley.edu> wrote:
> Thanks for the clarification. How did you get that output?
>
> Kasper
>
> On Jun 30, 2008, at 10:23 AM, Simon Urbanek wrote:
>
>>
>> On Jun 30, 2008, at 1:04 PM, Kasper Daniel Hansen wrote:
>>
>>> Like Sean is aying, you most likely are using _way_ more memory than 1.2
>>> GB.
>>>
>>> However, if you a re running 32bit R (which is the case if you use the
>>> CRAN binary) R can only access 2GB,
>>
>> That's not true, 32-bit process can use up to about 3.5GB of RAM:
>>
>> Virtual Memory Map of process 2849 (R)
>> Output report format:  2.2  -- 32-bit process
>> [...]
>> ReadOnly portion of Libraries: Total=72.9M resident=36.6M(50%)
>> swapped_out_or_unallocated=36.3M(50%)
>> Writable regions: Total=3.4G written=3.4G(100%) resident=3.4G(99%)
>> swapped_out=3352K(0%) unallocated=19.3M(1%)
>>
>> so it should make no real difference for Antonio (unless he doesn't mind
>> waiting while the machine swaps). Nonetheless using 64-bit R is fine as
>> well, especially on Leopard - albeit that doesn't fix incorrect use of
>> memory by users :).
>>
>> Cheers,
>> S
>>
>>
>>> so you can squeeze a little more out of your machine by switching to a
>>> 64bit version of R. You can check what version you have by typing
>>> R> .Machine
>>> and look for sizeof.pointer - if it is 4 you are using 32bit, if it is 8
>>> you are using 64 bit.
>>>
>>> If you want the 64 bit version of R you can download a binary from
>>> Simon's page: r.research.att.com , but you need to also get the preview
>>> build of GCC 4.2 which is available from Apple's developer site (although
>>> hard to find these days).
>>>
>>> Kasper
>>>
>>> On Jun 30, 2008, at 3:23 AM, Sean Davis wrote:
>>>
>>>> On Sun, Jun 29, 2008 at 6:35 AM, Antonio P. Ramos
>>>> <ramos.grad.student at gmail.com> wrote:
>>>>>
>>>>> Hi everybody,
>>>>>
>>>>> I have a memory allocation problem while using  R in my macbook pro,
>>>>> which runs the latest leopard. I'm trying to run a monte carlo
>>>>> simulation with 500,000 interactions, but the machine failed:
>>>>>
>>>>>
>>>>> Starting MCMC Iterations...
>>>>> Error: cannot allocate vector of size 1.2 Gb
>>>>> R(176,0xa0640fa0) malloc: *** mmap(size=1239990272) failed (error
>>>>> code=12)
>>>>> *** error: can't allocate region
>>>>> *** set a breakpoint in malloc_error_break to debug
>>>>> R(176,0xa0640fa0) malloc: *** mmap(size=1239990272) failed (error
>>>>> code=12)
>>>>> *** error: can't allocate region
>>>>> *** set a breakpoint in malloc_error_break to debug
>>>>>
>>>>>
>>>>> Since my machine has 4 Gb of memory, and since I'm not running nothing
>>>>> in addition to the simulation, I found it strange. This is my machine:
>>>>>
>>>>> Model Identifier:     MacBookPro3,1
>>>>> Processor Name:       Intel Core 2 Duo
>>>>> Processor Speed:      2.4 GHz
>>>>> Memory:       4 GB
>>>>>
>>>>>
>>>>> Unfortunately, I could figure it out how to solve it. Any help?
>>>>
>>>> The error message above means that R failed to allocate a vector of
>>>> size 1.2Gb.  That doesn't mean that R was using only 1.2 Gb, but that
>>>> it was trying to allocate a new block of memory of that size in
>>>> addition to the memory that was already in use.  The system on the Mac
>>>> uses a fair amount of memory; R was probably using memory as well.  In
>>>> short, you probably need more memory or be more clever about how you
>>>> are using the memory you have.  Without more details about what you
>>>> are doing, it is difficult to know how to change the latter.
>>>>
>>>> Sean
>>>>
>>>> _______________________________________________
>>>> R-SIG-Mac mailing list
>>>> R-SIG-Mac at stat.math.ethz.ch
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>>>
>>> _______________________________________________
>>> R-SIG-Mac mailing list
>>> R-SIG-Mac at stat.math.ethz.ch
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>>>
>>>
>
>