[Rd] A memory management question

[Luke Tierney]

On Mon, 5 Sep 2005, dhinds at sonic.net wrote:

> Luke Tierney <luke at stat.uiowa.edu> wrote:
>
>> This is not supported by the memory manager.  Using SETLENGTH to
>> change the length would confuse the garbage collector--we should
>> probably remove SETLENGTH from the headers.
>
>> The memory manager does over-allocate small vectors by rounding up to
>> convenient sizes, and the real size could be computed, but this is not
>> true for large allocations--these correspond to malloc calls for the
>> requested size--and in any case the memory manager relies on LENGTH
>> giving the correct amount (maybe not heavily but this could change).
>
>> A GC does not move objects.
>
>> Using R level vectors for the purpose you describe is in any case
>> tricky since it is hard to reliably prevent copying. You are better
>> off using something like an external pointer into an R-allocated
>> object that is only accessible through the external pointer.  Then you
>> can manage the filled length yourself.
>
> Ok... since GC does not move objects, and large vectors are allocated
> using a regular malloc, and malloc/free manages space independent of
> the LENGTH information, it seems that SETLENGTH would be "safe" if it
> was possible to guarantee for an interval of time that this particular
> value would not be moved or released due to any user activity?
>
> What if I create the full-length vector, make it visible using
> defineVar(), then protect the vector by creating a reference with
> R_MakeExternalPtr(), and R_PreserveObject() this reference?  Then
> shouldn't the vector be left alone until I release the reference?
> And I could then play with SETLENGTH() on that vector safely, so long
> as I restore it before releasing the reference, and so long as I only
> perform operations that modify the vector in-place?

It might or might not work now but is not guaranteed to do so reliably
in the future.  Seeing the risks of leaving SETLENGTH exposed, it is
very likely that SETLENGTH will be removed from the sources after the
2.2.0 release.

If you provide your own methods to read and write the external pointer
then you don' need this; this is safer than relying on undocumented
behavior of [ and [<- in any case.  You also then don't need to use
R_PreserveObject unless you really need to use it from the C level
outside of a context where an R reference exists.

luke



> i.e., should something like this work:
>
> static SEXP ptr;
>
> do_init()
> {
>    SEXP s = PROTECT(allocVector(RAWSXP, 1000));
>    defineVar("mystuff", s, R_BaseEnv);
>    ptr = R_MakeExternalPtr(RAW(s), R_NilValue, s);
>    R_PreserveObject(ptr);
>    SETLENGTH(s, 0);
>    UNPROTECT(1);
> }
>
> do_extend()
> {
>    SEXP s = R_ExternalPtrProtected(ptr);
>    memcpy(RAW(s)+LENGTH(s), "xxxx", 4);
>    SETLENGTH(s, LENGTH(s)+4);
> }
>
> do_finish()
> {
>    SEXP s = R_ExternalPtrProtected(ptr);
>    SETLENGTH(s, 1000);
>    R_ReleaseObject(ptr);
> }
>
> i.e., if the user tries to modify "mystuff", they'll end up with a
> copy, but the value pointed to by ptr will hang around (no longer
> accessible by the user) until do_finish() is called?
>
> -- Dave
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

-- 
Luke Tierney
Chair, Statistics and Actuarial Science
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa                  Phone:             319-335-3386
Department of Statistics and        Fax:               319-335-3017
    Actuarial Science
241 Schaeffer Hall                  email:      luke at stat.uiowa.edu
Iowa City, IA 52242                 WWW:  http://www.stat.uiowa.edu