[Rd] segfault / crash when asking for large memory via strrep()

[Martin Maechler]

We've had this more general topic on R-help,  and also in R-devel recently.
There's one case here where I get the feeling R never gets into
swapping but more directly aborts possibly from a bug we can
more easily fix.

Today I've been working (successfully! - not yet committed) at
fixing  str() for very large strings.

In this process, I've found that

   pc <- function(.) paste(., collapse=".1.2.3.4.5.")
   p  <- function(.) strrep(pc(.), 64L)
   p(p(p(p(LETTERS))))

produces a (memory related) segmentation fault (aka "crash")
very reproducibly and relatively quickly
both on my Linux (Fedora 22) desktop and on our Windows server.

 *** caught segfault ***
address 0x7fc52dc89000, cause 'memory not mapped'

Traceback:
 1: strrep(pc(.), 64L)
 2: p(p(p(p(LETTERS))))
 3: system.time(L2 <- p(p(p(p(LETTERS)))))

In the debugger, the symptoms point to the possibility of a
bug just in the C parts of strrep() :


Program received signal SIGSEGV, Segmentation fault.
0x00007ffff54d6223 in __strcpy_sse2_unaligned () from /usr/lib64/libc.so.6
Missing separate debuginfos, use: dnf debuginfo-install bzip2-libs-1.0.6-14.fc22.x86_64 libgcc-5.3.1-6.fc22.x86_64 libgfortran-5.3.1-6.fc22.x86_64 libgomp-5.3.1-6.fc22.x86_64 libicu-54.1-4.fc22.x86_64 libquadmath-5.3.1-6.fc22.x86_64 libstdc++-5.3.1-6.fc22.x86_64 ncurses-libs-5.9-18.20150214.fc22.x86_64 pcre-8.38-4.fc22.x86_64 readline-6.3-5.fc22.x86_64 xz-libs-5.2.0-2.fc22.x86_64 zlib-1.2.8-7.fc22.x86_64
(gdb) bt
#0  0x00007ffff54d6223 in __strcpy_sse2_unaligned () from /usr/lib64/libc.so.6
#1  0x0000000000457def in do_strrep (call=<optimized out>, op=<optimized out>, args=<optimized out>, 
    env=<optimized out>) at ../../../R/src/main/character.c:1658
#2  0x00000000004d6844 in bcEval (body=body at entry=0xd66840, rho=rho at entry=0x45253b8, 
    useCache=useCache at entry=TRUE) at ../../../R/src/main/eval.c:5648
#3  0x00000000004dd240 in Rf_eval (e=0xd66840, rho=0x45253b8) at ../../../R/src/main/eval.c:616
#4  0x00000000004dedaf in Rf_applyClosure (call=call at entry=0x45250a8, op=op at entry=0xd668e8, 
    arglist=0x45251f8, rho=rho at entry=0x4525000, suppliedvars=0xa57188)
    at ../../../R/src/main/eval.c:1134
#5  0x00000000004dd3b1 in Rf_eval (e=0x45250a8, rho=0x4525000) at ../../../R/src/main/eval.c:732
#6  0x00000000004dedaf in Rf_applyClosure (call=call at entry=0x4525718, op=op at entry=0x4524d28, 
    arglist=0x4524f90, rho=rho at entry=0xa8ea30, suppliedvars=0xa57188)
    at ../../../R/src/main/eval.c:1134
#7  0x00000000004dd3b1 in Rf_eval (e=0x4525718, rho=0xa8ea30) at ../../../R/src/main/eval.c:732
#8  0x00000000004e0cde in do_set (call=0x4525670, op=0xa61358, args=<optimized out>, rho=0xa8ea30)
    at ../../../R/src/main/eval.c:2196