[Rd] model.matrix memory problem (PR#13838)

[Olaf Mersmann]

Hi,

Excerpts from Torsten.Hothorn's message of Thu Jul 16 17:20:10 +0200 2009:
> `model.matrix' might kill R with a segfault (on a illposed problem, but anyway):
> 
> mydf <- as.data.frame(sapply(1:40, function(i) gl(2, 100)))
> f <- as.formula(paste("~ - 1 + ", paste(names(mydf), collapse = ":"), sep = ""))
> X <- model.matrix(f, data = mydf)
> 
>   *** caught segfault ***
> address 0x18, cause 'memory not mapped'
> Segmentation fault

I've taken a look at this. The problem lies in lines 1784 - 1798 of
src/main/model.c. What happens is that 'k' overflows (signed
int). That means k is 0 after the loop an nc is set to 0. That means
the allocated model matrix 'x' is too small which results in the
observed segfault. 

I can provide a patch which checks for overflow and throws an error if
that is the desired behaviour.

Greetings,
Olaf Mersmann