[Rd] .Call and to reclaim the memory by allocVector

[Prof Brian Ripley]

Please do not post to multiple lists! I've removed R-help.

You have not told us your OS ('linux', perhaps but what CPU), nor how you 
know 'the memory was still not reclaimed back to the operating system'. 
But that is how many OSes work: their malloc maintains a pool of memory 
pages, and free() does not return the memory to the OS kernel, just to the 
process' pool.  It depends on what you meant by 'the operating system'.

Why does this bother you?  150Mb of virtual memory is nothing these days.


On Thu, 23 Aug 2007, Yongchao Ge wrote:

> Hi,
>
> I am not sure if this is a bug and I apologize if it is something I
> didn't read carefully in the R extension manual. My initial search on the
> R help and R devel list archive didn't find useful information.

Exactly this topic was thrashed to death under the misleading title of 
'Suspected memory leak' earlier this month in a thread that started on 
R-help and moved to R-devel. See e.g.

https://stat.ethz.ch/pipermail/r-devel/2007-August/046669.html

from the author of the R memory allocator.


> I am using .Call (as written in the R extension manual) for the C code
> and have found that the .Call didn't release the memory claimed by
> allocVector. Even after applying gc() function and removing the R object
> created by the .Call function, the memory was still not reclaimed back to
> the operating system.
>
> Here is an example. It was modified from the convolve2 example from the R
> extension manual. Now I am computing the crossproduct of a and b, which
> returns a vector of size length(a)*length(b).
>
> The C code is at the end of this message with the modification commented.
> The R code is here
> ----------------------------
> dyn.load("crossprod2.so")
> cp <- function(a, b) .Call("crossprod2", a, b)
> gctorture()
> a<-1:10000
> b<-1:1000
> gc() #i
>
> c<-cp(a,b)
> rm(c)
> gc() #ii
> --------------
>
> When I run the above code in a fresh start R (version 2.5.0)
> the gc() inforamation is below. I report the last column ("max
> used (Mb)" ) here, which agrees the linux command "ps aux". Apparently
> even after I removing the object "c", we still have un-reclaimed 70M bytes
> of memory, which is approximately the memory size for the object "c".
>
> If I run the command "c<-cp(a,b)" for three or four times and then remove the
> object "c" and apply gc() function, the unclaimed memory can reach 150M
> bytes. I tried gc(reset=TRUE), and it doesn't seem to make difference.
>
> Can someone suggest what caused this problem and what the solution will
> be?  When you reply the email, please cc to me as I am not on the help
> list.
>
> Thanks,
>
> Yongchao
>
> ------------------------------------------------
>> dyn.load("crossprod2.so")
>> cp <- function(a, b) .Call("crossprod2", a, b)
>> gctorture()
>> a<-1:10000
>> b<-1:1000
>
>
>> gc() #i
>          used (Mb) gc trigger (Mb) max used (Mb)
> Ncells 173527  4.7     467875 12.5   350000  9.4
> Vcells 108850  0.9     786432  6.0   398019  3.1
>>
>> c<-cp(a,b)
>> rm(c)
>> gc() #ii
>          used (Mb) gc trigger (Mb) max used (Mb)
> Ncells 233998  6.3     467875 12.5   350000  9.4
> Vcells 108866  0.9   12089861 92.3 10119856 77.3
>>
> -----------------------------------------------
>
>
>
>
>
>
> --------------------------------------------
> #include "R.h"
> #include "Rinternals.h"
> #include "Rdefines.h"
> SEXP crossprod2(SEXP a, SEXP b);
> //modified from convolve2 in the R extension
> //R CMD SHLIB crossprod2.c
>
> #include <R.h>
> #include <Rinternals.h>
> SEXP crossprod2(SEXP a, SEXP b)
> {
>      R_len_t i, j, na, nb, nab;
>      double *xa, *xb, *xab;
>      SEXP ab;
>
>      PROTECT(a = coerceVector(a, REALSXP));
>      PROTECT(b = coerceVector(b, REALSXP));
>      na = length(a); nb = length(b);
>
>      //nab = na + nb - 1;
>      nab=na*nb;// we are doing the cross product
>      PROTECT(ab = allocVector(REALSXP, nab));
>      xa = REAL(a); xb = REAL(b);
>      xab = REAL(ab);
>      for(i = 0; i < nab; i++) xab[i] = 0.0;
>      for(i = 0; i < na; i++)
> 	  for(j = 0; j < nb; j++) //xab[i + j] += xa[i] * xb[j];
> 	       xab[i*nb + j] += xa[i] * xb[j];//we are computing crossproduct
>      UNPROTECT(3);
>      return(ab);
> }
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595