[Rd] [External] Data pointer caching vs ALTREP

[iuke-tier@ey m@iii@g oii uiow@@edu]

Thanks for the report and the careful analysis. This looks like a bug
in the wrapper implementation. I'll look more carefully and patch
R-devel/R-patched next week when I am back from useR. For caching it
should be sufficient for now to cache the writable pointer.

Best,

luke

On Mon, 6 Jul 2026, Ivan Krylov via R-devel wrote:

> Hello R-devel,
>
> Here's a simplification of the problem that currently plagues lme4:
>
> x <- local({
>  # y[[1]] is large enough to have its own mmap() allocation
>  y <- list(rep(1, 1e7))
>  # y[[1]] is also large enough (>=64 elements) so structure() will
>  # create a wrapper instead of duplicating
>  structure(y[[1]], foo = 'bar')
>  # y[[1]] now has a reference count of >=2
> })
> # now x has the only reference to the original vector
>
> writeLines('
> #include <Rinternals.h>
>
> static double run_arbitrary_R_code(SEXP x) {
>  const double * xd = REAL(x); // oops, forgot REAL_RO
>  // since the underlying vector is MAYBE_SHARED, the
>  // wrapper in x duplicated the underlying vector and
>  // returned a different data pointer
>  Rprintf("REAL(x) = %p\\n", xd);
>
>  double sum = 0;
>  for (R_xlen_t i = 0; i < XLENGTH(x); ++i) sum += xd[i];
>
>  R_gc();
>  // the original vector is now freed, memory unmapped
>
>  return sum;
> }
>
> SEXP foo(SEXP x) {
>  // cache a read-only pointer to the data of x
>  const double *xd = REAL_RO(x);
>  Rprintf("REAL_RO(x) = %p\\n", xd);
>
>  double ret1 = run_arbitrary_R_code(x);
>  // xd is now invalid
>
>  double ret2 = 0;
>  for (R_xlen_t i = 0; i < XLENGTH(x); ++i) ret2 += xd[i]; // <-- crash
>  return ScalarLogical(ret1 == ret2);
> }
> ', 'foo.c')
> stopifnot(tools::Rcmd('SHLIB foo.c') == 0)
> dyn.load(paste0('foo', .Platform$dynlib.ext))
> .Call('foo', x)
> # REAL_RO(x) = 0x7f11527b4040
> # REAL(x) = 0x7f114db68040
> #
> #  *** caught segfault ***
> # address 0x7f11527b4040, cause 'memory not mapped'
>
> In the original lme4 code the data pointers are cached inside Eigen
> objects courtesy of RcppEigen (shared with the R session, ensuring that
> the R objects underlying the data pointers stay protected), and the role
> of run_arbitrary_R_code() is played by a %*% operation (which uses
> REAL() to read a vector).
>
> Is there a nice way to solve this problem of a protected object's data
> pointer suddenly becoming invalid? (Where is the best place to solve
> it? lme4, Rcpp, R itself?) Would it be safe for lme4 to cache writable
> data pointers (e.g. REAL() instead of DATAPTR_RO()), assuming that the
> former will stay the same as long as the object exists? Or does it need
> an extensive rewrite to always refresh cached data pointers?
>
>

-- 
Luke Tierney
Professor Emeritus
Department of Statistics and
    Actuarial Science
University of Iowa
241 Schaeffer Hall                  email:   luke-tierney using uiowa.edu
Iowa City, IA 52242                 WWW:  http://stat.uiowa.edu