[Rd] A memory management question

[dhinds@sonic.net]

Luke Tierney <luke at stat.uiowa.edu> wrote:

> This is not supported by the memory manager.  Using SETLENGTH to
> change the length would confuse the garbage collector--we should
> probably remove SETLENGTH from the headers.

> The memory manager does over-allocate small vectors by rounding up to
> convenient sizes, and the real size could be computed, but this is not
> true for large allocations--these correspond to malloc calls for the
> requested size--and in any case the memory manager relies on LENGTH
> giving the correct amount (maybe not heavily but this could change).

> A GC does not move objects.

> Using R level vectors for the purpose you describe is in any case
> tricky since it is hard to reliably prevent copying. You are better
> off using something like an external pointer into an R-allocated
> object that is only accessible through the external pointer.  Then you
> can manage the filled length yourself.

Ok... since GC does not move objects, and large vectors are allocated
using a regular malloc, and malloc/free manages space independent of
the LENGTH information, it seems that SETLENGTH would be "safe" if it
was possible to guarantee for an interval of time that this particular
value would not be moved or released due to any user activity?

What if I create the full-length vector, make it visible using
defineVar(), then protect the vector by creating a reference with
R_MakeExternalPtr(), and R_PreserveObject() this reference?  Then
shouldn't the vector be left alone until I release the reference?
And I could then play with SETLENGTH() on that vector safely, so long
as I restore it before releasing the reference, and so long as I only
perform operations that modify the vector in-place?

i.e., should something like this work:

static SEXP ptr;

do_init()
{
    SEXP s = PROTECT(allocVector(RAWSXP, 1000));
    defineVar("mystuff", s, R_BaseEnv);
    ptr = R_MakeExternalPtr(RAW(s), R_NilValue, s);
    R_PreserveObject(ptr);
    SETLENGTH(s, 0);
    UNPROTECT(1);
}

do_extend()
{
    SEXP s = R_ExternalPtrProtected(ptr);
    memcpy(RAW(s)+LENGTH(s), "xxxx", 4);
    SETLENGTH(s, LENGTH(s)+4);
}

do_finish()
{
    SEXP s = R_ExternalPtrProtected(ptr);
    SETLENGTH(s, 1000);
    R_ReleaseObject(ptr);
}

i.e., if the user tries to modify "mystuff", they'll end up with a
copy, but the value pointed to by ptr will hang around (no longer
accessible by the user) until do_finish() is called?

-- Dave