[R] loess crash

[Liaw, Andy]

FWIW...

Running the example (with x5 added) on Linux and R-1.5.1 gave segmentation
fault.  Running it on a copy of R-1.5.0 compiled w/o optimization gives:

> result1 <- loess(y~x1+x2+x3+x4+x5,data2)
Warning messages: 
1: k>d2MAX in ehg136.  Need to recompile with increased dimensions. 
2: pseudoinverse used at 1.0256 1.017 1.0161 1.0371 1.036 
3: neighborhood radius 0 
4: reciprocal condition number  0 
5: There are other near singularities as well. 0 
> result1
Call:
loess(formula = y ~ x1 + x2 + x3 + x4 + x5, data = data2)

Number of Observations: 500 
Equivalent Number of Parameters: NaN 
Residual Standard Error: NaN 

Andy


> -----Original Message-----
> From: John Deke [mailto:jdeke2 at comcast.net]
> Sent: Monday, September 16, 2002 7:36 AM
> To: r-help at stat.math.ethz.ch
> Subject: Re: [R] loess crash
> 
> 
> Here's a simple example that yields the crash:
> 
> library(modreg)
> data1 <- array(runif(500*5),c(500,5))
> colnames(data1) <- c("x1","x2","x3","x4","x5")
> y <- 
> 3+2*data1[,"x1"]+15*data1[,"x2"]+13*data1[,"x3"]-8*data1[,"x4"
> ]+14*data1[,"x5"]+rnorm(500)
> data2 <- cbind(y,data1)
> data2 <- as.data.frame(data2)
> result1 <- loess(y~x1+x2+x3+x4,data2)
> 
> To get the crash, I just add x5--
> 
> result1 <- loess(y~x1+x2+x3+x4+x5,data2)
> 
> And bammo -- I'm dead. It doesn't even pause -- Rgui crashes, 
> and I mean 
> really crashes -- the program is terminated, I get the little Windows 
> dialogue saying that a log file is being generated -- the 
> whole dramatic 
> death scene.
> 
> I know its a computationally intensive thing, but the one 
> that doesn't 
> crash (with four explanatory variables) runs almost 
> instantly. Its hard to 
> see how adding a fifth could be so catastrophic. But I am 
> somewhat new to 
> this particular methodology....
> 
> John
> 
> At 03:38 AM 9/16/2002, Peter Dalgaard BSA wrote:
> >John Deke <jdeke2 at comcast.net> writes:
> >
> > > Hmm... if I reduce the number of observations to just 
> 500, I still get
> > > the error.
> > >
> > > I don't think its an issue of colinearity, because I've 
> tried several
> > > different combinations of variables, all of which work 
> just fine in an
> > > OLS or logistic regression.
> > >
> > > I'm probably doing something stupid, but I'm not seeing it...
> > >
> > > At 02:00 PM 9/15/2002, John Deke wrote:
> > > >Hi,
> > > >
> > > > I have a data frame with 6563 observations. I can run a 
> regression
> > > > with loess using four explanatory variables. If I add a fifth, R
> > > > crashes. There are no missings in the data, and if I run a
> > > > regression with any four of the five explanatory variables, it
> > > > works. Its only when I go from four to five that it crashes.
> >
> >Hmm... I wouldn't try loess with more than one or two descriptors. I
> >mean, it's a smoothing method and representing a smooth function of
> >many variables can be computationally demanding.
> >
> >The Fortran source code for loess is one of the more 
> obfuscated pieces
> >of R, but I can see that some structures inside of it are of fixed
> >size, which might explain it (BTW: Does R really crash, or just say
> >memory exhausted?).
> >
> >Do you have a simple example that reproduces the crash (using random
> >numbers, e.g.)?
> >
> >--
> >    O__  ---- Peter Dalgaard             Blegdamsvej 3
> >   c/ /'_ --- Dept. of Biostatistics     2200 Cph. N
> >  (*) \(*) -- University of Copenhagen   Denmark      Ph: 
> (+45) 35327918
> >~~~~~~~~~~ - (p.dalgaard at biostat.ku.dk)             FAX: 
> (+45) 35327907
> >-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
> .-.-.-.-.-.-.-.-.-
> >r-help mailing list -- Read 
> http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
> >Send "info", "help", or "[un]subscribe"
> >(in the "body", not the subject !)  To: 
> r-help-request at stat.math.ethz.ch
> >_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
> ._._._._._._._._._ 
> >
> 
> -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.
> -.-.-.-.-.-.-.-.-
> r-help mailing list -- Read 
> http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
> Send "info", "help", or "[un]subscribe"
> (in the "body", not the subject !)  To: 
> r-help-request at stat.math.ethz.ch
> _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._.
> _._._._._._._._._
> 

------------------------------------------------------------------------------
Notice: This e-mail message, together with any attachments, contains information of Merck & Co., Inc. (Whitehouse Station, New Jersey, USA) that may be confidential, proprietary copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named on this message.  If you are not the intended recipient, and have received this message in error, please immediately return this by e-mail and then delete it.

==============================================================================

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._