[Rd] External pointers and an apparent memory leak

[Matt Shotwell]

James, 

Works for me!

In a file, 'test.c', I have:

void h5R_allocate_finalizer(SEXP eptr) {
    char* vector = (char*) R_ExternalPtrAddr(eptr);
    free(vector);
    printf("memory freed at address %lu\n", (long unsigned) vector);
    R_ClearExternalPtr(eptr);
}

SEXP h5R_allocate(SEXP size) {
    int i = INTEGER(size)[0];
    char* vector = (char*) malloc(i*sizeof(char));
    printf("memory allocated at address %lu\n", (long unsigned) vector);
    SEXP e_ptr = R_MakeExternalPtr(vector, R_NilValue, R_NilValue);
    R_RegisterCFinalizerEx(e_ptr, h5R_allocate_finalizer, TRUE);
    return e_ptr;
}

I generated the shared library with R CMD SHLIB. Then, at the R prompt:

R> dyn.load('test.so')
R> system(paste('ps -eo pid,vsz | grep', Sys.getpid()))
10961  77160
R> v <- .Call('h5R_allocate', as.integer(2^24))
memory allocated at address 140127935406096
R> system(paste('ps -eo pid,vsz | grep', Sys.getpid()))
10961  93548
R> rm(v); gc()
memory freed at address 140127935406096
         used (Mb) gc trigger (Mb) max used (Mb)
Ncells 132769  7.1     350000 18.7   350000 18.7
Vcells 129312  1.0     786432  6.0   555216  4.3
R> system(paste('ps -eo pid,vsz | grep', Sys.getpid()))
10961  77160

It appears that the external pointer is behaving as expected. However, I
did find that gc() may not immediately collect the removed object 'v'.
Sometimes it took more than one call. In any case, the memory was freed
when R exits.

Best,
Matt

On Wed, 2011-09-14 at 21:21 +0000, James Bullard wrote:
> I'm using external pointers and seemingly leaking memory. My determination of a memory leak is that the R process continually creeps up in memory as seen by top while the usage as reported by gc() stays flat. I have isolated the C code:
> 
> void h5R_allocate_finalizer(SEXP eptr) {
>     Rprintf("Calling the finalizer\n");
>     void* vector = R_ExternalPtrAddr(eptr);
>     free(vector);
>     R_ClearExternalPtr(eptr);
> }
> 
> SEXP h5R_allocate(SEXP size) {
>     int i = INTEGER(size)[0];
>     char* vector = (char*) malloc(i*sizeof(char));
>     SEXP e_ptr = R_MakeExternalPtr(vector, R_NilValue, R_NilValue);
>     R_RegisterCFinalizerEx(e_ptr, h5R_allocate_finalizer, TRUE);
>     return e_ptr;
> }
> 
> 
> If I run an R program like this:
> 
> v <- replicate(100000, {
>   .Call("h5R_allocate", as.integer(1000000))
> })
> rm(v)
> gc()
> 
> Then you can see the problem (top reports that R still has a bunch of memory, but R doesn't think it does). I have tried using valgrind and it says I have memory left on the table at the end lest you think it is because top. Also, I have tried Free/Calloc as well and this doesn't make a difference. Finally, I see this in both R-2-12 (patched) and R-2-13 - I think it is more an understanding issue on my part.
> 
> thanks much in advance, to me it really resembles the connection.c code, but what am I missing?
> 
> thanks, jim
> 
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel