[Rd] R process killed when allocating too large matrix (Mac OS X)

[Simon Urbanek]

On May 4, 2016, at 9:49 PM, Hervé Pagès <hpages at fredhutch.org> wrote:

> Hi,
> 
> Interesting "feature" in 10.11.4. I wonder if the process is killed
> before or after malloc() returns. If before, it seems very blunt:
> "You're asking too much and I don't like it so I kill you now".
> If after it doesn't look much better: "You're asking a lot and I
> don't like it but I give it to you anyway. I'll kill you quickly
> later".
> 
> Why wouldn't the kernel just refuse to give that memory instead
> (i.e. malloc() returns NULL).
> 
> Just curious...
> 

You can google it, but this seems to be standard practice in modern OSes (including OS X and Linux) where the kernel is not actually allocating the physical memory when the application asks for it so malloc() will succeed and give only a pointer to a virtual space where it will be mapping physical memory into as needed (aka opportunistic allocation). However, when the OS figures out later that it is overcommitted, it has no good way to signal the app that it can't use what it was promised, so it will use SIGKILL. The rationale is that in most cases the kernel can scramble to fulfill the request so it gives it more flexibility in managing the memory (it can spread pages to RAM, swap, shared ,...), but when it cannot recover it has no choice but to kill the process.

I don't have the debugger open anymore, but if I recall correctly in my case the kill happened in copyVector() -> duplicate(), so it was not necessarily on the initial allocation but rather further down the line.

Cheers,
Simon



> H.
> 
> On 05/04/2016 06:21 PM, Simon Urbanek wrote:
>> 
>> On May 4, 2016, at 9:00 PM, Marius Hofert <marius.hofert at uwaterloo.ca> wrote:
>> 
>>> Hi Simon,
>>> 
>>> thanks for your quick reply.
>>> 
>>> 1) ... so you can reproduce this?
>> 
>> Yes, I can on 10.11.4.
>> 
>> 
>>> 2) Do you know a way how this can be 'foreseen'? We allocate larger
>>> matrices in the copula package depending on the user's input
>>> dimension. It would be good to tell her/him "Your dimension is quite
>>> large. Be aware of killers in your neighborhood"... before the killer
>>> attacks.
>>> 
>> 
>> Not directly, because the system is happy to accommodate R's request for more memory but then it will strike out of the blue. Since that decision is made by the kernel and it doesn't expose its thinking about how it feels, it's hard to tell. I couldn't find any API on OS X akin to didReceiveMemoryWarning in iOS, unfortunately.
>> 
>> You could guess - e.g. by checking the total system memory
>> 
>> int mib [] = { CTL_HW, HW_MEMSIZE };
>> int64_t value = 0;
>> size_t length = sizeof(value);
>> sysctl(mib, 2, &value, &length, NULL, 0);
>> 
>> and comparing it to the sizes involved. However, even that is not foolproof, because it all depends on the other processes' memory usage, swap space size etc. There are slightly more involved ways to query the VM system as well, but I'm not sure I'd want to go so deep into the weeds, especially since this becomes quickly highly OS-specific.
>> 
>> Cheers,
>> Simon
>> 
>> 
>> 
>>> Thanks & cheers,
>>> Marius
>>> 
>>> 
>>> On Wed, May 4, 2016 at 8:54 PM, Simon Urbanek
>>> <simon.urbanek at r-project.org> wrote:
>>>> 
>>>> On May 4, 2016, at 6:14 PM, Marius Hofert <marius.hofert at uwaterloo.ca> wrote:
>>>> 
>>>>>> Can you elaborate on "leads to R being killed"? You should tell to the killer not to do it again :).
>>>>> 
>>>>> Hi Simon!
>>>>> 
>>>>> Sure, but who do you tell it if you don't know the killer?
>>>>> This is all the killer left me with, the 'crime scene' if you like :-)
>>>>> 
>>>>>> m <- matrix(0, 90000, 100000)
>>>>> Killed: 9
>>>>> 
>>>>> My colleague Wayne Oldford also tried it on his Mac machine and
>>>>> apparently the killer went further down the hallway to his office
>>>>> now... so scary. Here is Wayne's sessionInfo():
>>>>> 
>>>> 
>>>> Yes, indeed, scary - since it means someone is killing R which means there is not much R itself can do about it. In fact from the syslog I see
>>>> 
>>>> May  4 20:48:11 ginaz kernel[0]: low swap: killing pid 56256 (R)
>>>> 
>>>> so it's the kernel's own defense mechanism. The bad thing is that R cannot do anything about it - the kernel just decides to snipe processes it thinks are dangerous to the health of the system, and it does so without a warning.
>>>> 
>>>> Cheers,
>>>> Simon
>>>> 
>>> 
>> 
>> ______________________________________________
>> R-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>> 
> 
> -- 
> Hervé Pagès
> 
> Program in Computational Biology
> Division of Public Health Sciences
> Fred Hutchinson Cancer Research Center
> 1100 Fairview Ave. N, M1-B514
> P.O. Box 19024
> Seattle, WA 98109-1024
> 
> E-mail: hpages at fredhutch.org
> Phone:  (206) 667-5791
> Fax:    (206) 667-1319
>