[Rd] R/C++/memory leaks
Luke Tierney
luke at stat.uiowa.edu
Tue Feb 27 19:32:18 CET 2007
On Tue, 27 Feb 2007, Ernest Turro wrote:
>
> On 27 Feb 2007, at 17:45, Luke Tierney wrote:
>
>> On Tue, 27 Feb 2007, Ernest Turro wrote:
>>
>>> Hi Ross,
>>>
>>> On 26 Feb 2007, at 22:34, Ross Boylan wrote:
>>>
>>>> On Mon, 2007-02-26 at 16:08 +0000, Ernest Turro wrote:
>>>>> Thanks for your comments Ross. A couple more comments/queries below:
>>>>>
>>>>> On 26 Feb 2007, at 06:43, Ross Boylan wrote:
>>>>>
>>>>>> [details snipped]
>>>>>>
>>>>>> The use of the R api can be confined to a wrapper function. But
>>>>>> I can
>>>>>> think of no reason that a change to the alternate approach I
>>>>>> outlined
>>>>>> would solve the apparent leaking you describe.
>>>>>>
>>>>>
>>>>> I'm not sure I see how a wrapper function using the R API would
>>>>> suffice. Example:
>>>> It doesn't sound as if it would suffice. I was responding to your
>>>> original remark that
>>>>
>>>>> Since this is a standalone C++ program too, I'd rather use the R API
>>>>> as little as possible... But I will look at your solution if I find
>>>>> it is really necessary.. Thanks
>>>>
>>>> I thought that was expressing a concern about using the alternate
>>>> approach I outlined because it would use the R API. If you need to
>>>> use
>>>> that API for other reasons, you're still stuck with it :)
>>>>>
>>>>> During heavy computation in the C++ function I need to allow
>>>>> interrupts from R. This means that R_CheckUserInterrupt needs to be
>>>>> called during the computation. Therefore, use of the R API can't be
>>>>> confined to just the wrapper function.
>>>>>
>>>>> In fact, I'm worried that some of the libraries I'm using are failing
>>>>> to release memory after interrupt and that that is the problem. I
>>>>> can't see what I could do about that... E.g.
>>>>>
>>>>> #include <valarray>
>>>>>
>>>>> valarray<double> foo; // I don't know 100% that the foo object hasn't
>>>>> allocated some memory. if the program is interrupted it wouldn't be
>>>>> released....
>>>> That's certainly possible, but you seem to be overlooking the
>>>> possibility that all the code is releasing memory appropriately,
>>>> but the
>>>> process's memory footprint isn't going down correspondingly. In my
>>>> experience that's fairly typical behavior.
>>>>
>>>
>>> OK, but does this still explain why the footprint keeps increasing
>>> indefinitely when i do run, interrupt, run, interrupt, run,
>>> interrupt......?
>>>
>>>
>>>> In that case, depending on your point of view, you either don't have a
>>>> problem or you have a hard problem. If you really want the memory
>>>> released back to the system, it's a hard problem. If you don't
>>>> care, as
>>>> long as you have no leaks, all's well.
>>>>
>>>>>
>>>>> I find it's very unfortunate that R_CheckUserInterrupt doesn't return
>>>>> a value. If it did (e.g. if it returned true if an interrupt has
>>>>> occurred), I could just branch off somewhere, clean up properly and
>>>>> return to R.
>>>>>
>>>>> Any ideas on how this could be achieved?
>>>> I can't tell from the info page what function gets called in R if
>>>> there
>>>> is an interrupt, but it sounds as you could do the following hack:
>>>> The R interrupt handler gets a function that calls a C function of
>>>> your
>>>> devising. The C function sets a flag meaning "interrupt requested".
>>>> Then in your main code, you periodically call R_CheckUserInterrupt.
>>>> When it returns you check the flag; if it's set, you cleanup and exit.
>>>> Ross
>>>>
>>>
>>> If this is feasible, it's by far the best solution.
>>>
>>> in error.c:
>>>
>>> void R_CheckUserInterrupt(void)
>>> {
>>> R_CheckStack();
>>> /* This is the point where GUI systems need to do enough event
>>> processing to determine whether there is a user interrupt event
>>> pending. Need to be careful not to do too much event
>>> processing though: if event handlers written in R are allowed
>>> to run at this point then we end up with concurrent R
>>> evaluations and that can cause problems until we have proper
>>> concurrency support. LT */
>>> #if ( defined(HAVE_AQUA) || defined(Win32) )
>>> R_ProcessEvents();
>>> #else
>>> if (R_interrupts_pending)
>>> onintr();
>>> #endif /* Win32 */
>>> }
>>>
>>> Leaving aside the HAVE_AQUA and Win32 cases, I would like to write a
>>> new function:
>>
>> Unfortunately we can't leave those aside. If standard unix where
>> interrupts arrive as signals is all you care about then you can just
>> save, replace and restore the R SIGINT handler around your code with
>> one that sets a flag of your own. Things are not so simple on GUI
>> systems where detecting a user interrupt action requires event
>> processing, which might result in errors and non-local exits in
>> response to those.
>>
>> There is an internal mechanism for registering C level on.exit
>> routines but this is not in a form that can be made public as it would
>> tie down implementation decisions too much. It is principle possible
>> to build something around R_ToplevelExec, but that is not at this
>> point part of the public API and so is subject to change. We might
>> consider providing a variant of R_CheckInterrupts that either just
>> checks or that executes cleanup code sometime after 2.5 is released.
>>
>
> Yes, I think it would be great if you added a variant of
> R_CheckUserInterrupt() that actually returned something (e.g. a boolean) to
> the R API.
>
> So in your view, is the best solution to pass a memory cleanup C function to
> on.exit() before the call? What about memory allocated in some of the
> external libraries that I use? E.g. as I mentioned above:
Depends on the cleanup actions you have to take.
Best,
luke
>
>>>>>
>>>>> #include <valarray>
>>>>> // ...
>>>>> valarray<double> foo; // I don't know 100% that the foo object hasn't
>>>>> allocated some memory. if the program is interrupted it wouldn't be
>>>>> released....
>
>
> Thanks for your help,
>
> Ernest
>
>
>> Best,
>>
>> luke
>>
>>>
>>> int R_CheckInterruptsPending(void)
>>> {
>>> R_CheckStack();
>>> return R_interrupts_pending;
>>> }
>>>
>>> and then in my C++ code:
>>>
>>> if(R_checkInterruptsPending) {
>>> // clean up
>>> // ...
>>> R_CheckInterruptsPending();
>>> }
>>>
>>> R_CheckStack() is declared in R_ext/Utils.h but the variable
>>> R_interrupts_pending isn't, so how could I access it? In other words,
>>> how can I extend error.c .....
>>>
>>>
>>> Thanks,
>>>
>>> E
>>>
>>> ______________________________________________
>>> R-devel at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>
>>
>> --
>> Luke Tierney
>> Chair, Statistics and Actuarial Science
>> Ralph E. Wareham Professor of Mathematical Sciences
>> University of Iowa Phone: 319-335-3386
>> Department of Statistics and Fax: 319-335-3017
>> Actuarial Science
>> 241 Schaeffer Hall email: luke at stat.uiowa.edu
>> Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu
>
--
Luke Tierney
Chair, Statistics and Actuarial Science
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa Phone: 319-335-3386
Department of Statistics and Fax: 319-335-3017
Actuarial Science
241 Schaeffer Hall email: luke at stat.uiowa.edu
Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu
More information about the R-devel
mailing list