[Rd] Memory management issues

[Duncan Murdoch]

On 05/07/2009 10:54 AM, Yuri D'Elia wrote:
> Hi everybody,
> 
> I have been interfacing some C++ library code into an R package but
> ran into optimization issues specific to memory management that require
> some insight into the GC.
> 
> One of the C++ libraries returns simple vectors of integers, doubles and
> complex which are allocated and managed from the library itself. I
> cannot know the length of the array beforehand, so I cannot
> pre-allocate that memory through the GC.
> 
> Right now I'm allocating via allocVector and copying all the data in it.
> However, this requires twice the amount of space (and time), and we're
> running out of memory when doing concurrent analysis.
> 
> What I'd would like to do is:
> 
> - "patch" the SEXP returned to R so that DATAPTR() points directly to
>   the required address. 

The normal way to do what you want is to use an "external pointer".  R 
assumes that memory management for those is handled completely 
externally.  External pointers can have finalizers, so when you no 
longer have a need for the object, you can ask the external library to 
release it.

> 
> - create a normal LISTSXP in the package, which holds a reference
>   to all these objects, so that GC never takes place.

The list would hold the external pointers, which act like references.
> 
> - turn these objects read-only, or, at least, ensure that they are
>   never free()d or remalloc()ed. overwriting the contents is not a
>   critical issue.

That won't happen.

I wouldn't try to trick the memory manager into thinking that it 
allocated these things; that will likely just lead to problems.

Duncan Murdoch

> 
> Would that approach work?
> Are there any alternative approaches?
> Any specific advice about turning these objects read-only?
> 
> Thanks in advance.
> 
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel