[Rd] R/C++/memory leaks

[Luke Tierney]

On Tue, 27 Feb 2007, Ernest Turro wrote:

> Hi Ross,
>
> On 26 Feb 2007, at 22:34, Ross Boylan wrote:
>
>> On Mon, 2007-02-26 at 16:08 +0000, Ernest Turro wrote:
>>> Thanks for your comments Ross. A couple more comments/queries below:
>>>
>>> On 26 Feb 2007, at 06:43, Ross Boylan wrote:
>>>
>>>> [details snipped]
>>>>
>>>> The use of the R api can be confined to a wrapper function.  But
>>>> I can
>>>> think of no reason that a change to the alternate approach I
>>>> outlined
>>>> would solve the apparent leaking you describe.
>>>>
>>>
>>> I'm not sure I see how a wrapper function using the R API would
>>> suffice. Example:
>> It doesn't sound as if it would suffice.  I was responding to your
>> original remark that
>>
>>> Since this is a standalone C++ program too, I'd rather use the R API
>>> as little as possible... But I will look at your solution if I find
>>> it is really necessary.. Thanks
>>
>> I thought that was expressing a concern about using the alternate
>> approach I outlined because it would use the R API.  If you need to
>> use
>> that API for other reasons, you're still stuck with it :)
>>>
>>> During heavy computation in the C++ function I need to allow
>>> interrupts from R. This means that R_CheckUserInterrupt needs to be
>>> called during the computation. Therefore, use of the R API can't be
>>> confined to just the wrapper function.
>>>
>>> In fact, I'm worried that some of the libraries I'm using are failing
>>> to release memory after interrupt and that that is the problem. I
>>> can't see what I could do about that... E.g.
>>>
>>> #include <valarray>
>>>
>>> valarray<double> foo; // I don't know 100% that the foo object hasn't
>>> allocated some memory. if the program is interrupted it wouldn't be
>>> released....
>> That's certainly possible, but you seem to be overlooking the
>> possibility that all the code is releasing memory appropriately,
>> but the
>> process's memory footprint isn't going down correspondingly.  In my
>> experience that's fairly typical behavior.
>>
>
> OK, but does this still explain why the footprint keeps increasing
> indefinitely when i do run, interrupt, run, interrupt, run,
> interrupt......?
>
>
>> In that case, depending on your point of view, you either don't have a
>> problem or you have a hard problem.  If you really want the memory
>> released back to the system, it's a hard problem.  If you don't
>> care, as
>> long as you have no leaks, all's well.
>>
>>>
>>> I find it's very unfortunate that R_CheckUserInterrupt doesn't return
>>> a value. If it did (e.g. if it returned true if an interrupt has
>>> occurred), I could just branch off somewhere, clean up properly and
>>> return to R.
>>>
>>> Any ideas on how this could be achieved?
>> I can't tell from the info page what function gets called in R if
>> there
>> is an interrupt, but it sounds as you could do the following hack:
>> The R interrupt handler gets a function that calls a C function of
>> your
>> devising.  The C function sets a flag meaning "interrupt requested".
>> Then in your main code, you periodically call R_CheckUserInterrupt.
>> When it returns you check the flag; if it's set, you cleanup and exit.
>> Ross
>>
>
> If this is feasible, it's by far the best solution.
>
> in error.c:
>
> void R_CheckUserInterrupt(void)
> {
>     R_CheckStack();
>     /* This is the point where GUI systems need to do enough event
>        processing to determine whether there is a user interrupt event
>        pending.  Need to be careful not to do too much event
>        processing though: if event handlers written in R are allowed
>        to run at this point then we end up with concurrent R
>        evaluations and that can cause problems until we have proper
>        concurrency support. LT */
> #if  ( defined(HAVE_AQUA) || defined(Win32) )
>     R_ProcessEvents();
> #else
>     if (R_interrupts_pending)
>   onintr();
> #endif /* Win32 */
> }
>
> Leaving aside the HAVE_AQUA and Win32 cases, I would like to write a
> new function:

Unfortunately we can't leave those aside.  If standard unix where
interrupts arrive as signals is all you care about then you can just
save, replace and restore the R SIGINT handler around your code with
one that sets a flag of your own.  Things are not so simple on GUI
systems where detecting a user interrupt action requires event
processing, which might result in errors and non-local exits in
response to those.

There is an internal mechanism for registering C level on.exit
routines but this is not in a form that can be made public as it would
tie down implementation decisions too much.  It is principle possible
to build something around R_ToplevelExec, but that is not at this
point part of the public API and so is subject to change.  We might
consider providing a variant of R_CheckInterrupts that either just
checks or that executes cleanup code sometime after 2.5 is released.

Best,

luke

>
> int R_CheckInterruptsPending(void)
> {
>     R_CheckStack();
>     return R_interrupts_pending;
> }
>
> and then in my C++ code:
>
> if(R_checkInterruptsPending) {
> 	// clean up
> 	// ...
> 	R_CheckInterruptsPending();
> }
>
> R_CheckStack() is declared in R_ext/Utils.h but the variable
> R_interrupts_pending isn't, so how could I access it? In other words,
> how can I extend error.c .....
>
>
> Thanks,
>
> E
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

-- 
Luke Tierney
Chair, Statistics and Actuarial Science
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa                  Phone:             319-335-3386
Department of Statistics and        Fax:               319-335-3017
    Actuarial Science
241 Schaeffer Hall                  email:      luke at stat.uiowa.edu
Iowa City, IA 52242                 WWW:  http://www.stat.uiowa.edu