[R] dyn.load, dyn.unload -memory management

[Thomas Lumley]

On Fri, 28 Jul 2006, johan Faux wrote:

> hello everybody,
>
> I have some code which looks like:
>
> dyn.load("lpSolve.so")
> res <- lp(some.parameters)
>
> and everything runs fine.
>
> What lp() does, it's just calling a C function which is in "lpSolve.so"
>
> If I call lp() a large number of times:
>
> for(1 in 1:5000){
> gc(verbose=TRUE)
> res <- lp(some.parameters)
> }
>
> then gc() is showing that the memory used by R process remain almost constant all the time. But the system memory used is going up _very fast_ and the above code never succeed because the memory used rich the limit.
>> From this I'm drawing the conclusion that there is some memory leak on the C code called by lp() which gc() cannot report since this is a C process.
>
> Does all this makes any sense or I'm wrong?


gc() does track some C memory usage, but not memory obtained directly 
from the C implementation by malloc().  The symptoms are consistent with 
the C code allocating via malloc()/calloc() and then not free()ing the 
memory.


>
> The next thing I tried was
>
> for(1 in 1:5000){
> gc(verbose=TRUE)
> dyn.load("lpSolve.so")
> res <- lp(some.parameters)
> dyn.unload("lpSolve.so")
> }
>
> But the problem is still there. The memory used is going up really fast. 
> Now my other question is: When I call dyn.unload() doesnt it terminate 
> the previous C process?  Next time in the loop, a new C process starts, 
> and I will not have any problem with memory.

No, this won't help at all.  dyn.unload() may or may not do anything on 
your system, but it certainly won't affect the way the C implementation 
handles memory.  The only ways to return memory allocated by malloc() are 
to do a free() for every allocated chunk or to quit R.

> P.S. lpSolve.so above is from package "lpSolve". I thought that getting 
> "lpSolve.so" and doing the above trick will solve my problem, but I have 
> been surely wrong.
>

Indeed.

You need to either report this to the lpsolve package maintainers (with a 
reproducible example) and hope that they will fix it or look at the source 
code to find out which allocations are not being freed and fix them.


 	-thomas