[R] Function completely locks up my computer if the input is too big

[Milan Bouchet-Valat]

Le mardi 05 mars 2013 à 15:19 -0800, Benjamin Caldwell a écrit :
> Hi all,
> 
> Thanks for the suggestions. Updating the function as below to break the
> problem into chunks seemed to do the trick - perhaps there is a relatively
> small limit to the size of a vector that R can work with?
On the contrary, that's because the limit supported by R is too high for
your computer's memory that the OS tries to allocate too much memory and
swaps to death. Admittedly, an OS should be smart enough not to
completely freeze in the process, but that's just how it is...

(If R did not support such a long vector, you would just get a "nice"
error message, that's all.)


Regards

> Best
> 
> rotate <- function(x,y,tilt,threshold){
> 
> df.main<-data.frame(x,y)
> 
>  if(length(x)> threshold){
> l <- round(length(x)/ threshold, 0)
> dfchunk <- split(df.main, factor(sort(rank(row.names(df.main))%%l)))
>  n<-length(summary(dfchunk)[,1])
> xy<-vector("list", n)
> for (i in 1:n){
> wk.df <- dfchunk[[i]]
> x <- wk.df$x
> y <- wk.df$y
> d2 <- x^2+y^2
> rotate.dis<-sqrt(d2)
> or.rad <- atan(x/y)
> or.deg <- Rad2Deg(or.rad)
> 
>  or.deg[is.na(or.deg)] <- 0
>  tilt.in <- tilt + or.deg
> xy[[i]]<-data.frame(Pol2Car(distance=rotate.dis, deg=tilt.in))
> }
>  xy<-do.call("rbind", xy[1:n])
>   } else {
>  d2 <- x^2+y^2
> rotate.dis<-sqrt(d2)
> or.rad <- atan(x/y)
> or.deg <- Rad2Deg(or.rad)
> 
> n <- length(or.deg)
> for(i in 1:n){
> if(is.na(or.deg[i])==TRUE) {or.deg[i] <- 0}
> }
>  tilt.in <- tilt + or.deg
> 
> xy<-data.frame(Pol2Car (distance=rotate.dis, deg=tilt.in))
> }
> 
> xy
> }
> 
> *Ben Caldwell*
> 
> Graduate Fellow
> University of California, Berkeley
> 130 Mulford Hall #3114
> Berkeley, CA 94720
> Office 223 Mulford Hall
> (510)859-3358
> 
> 
> On Tue, Mar 5, 2013 at 1:44 PM, Peter Alspach <
> Peter.Alspach at plantandfood.co.nz> wrote:
> 
> > Tena koe Benjamin
> >
> > I haven't looked at you code in detail, but in general ifelse is slow and
> > can generally be avoided.  For example,
> >
> > ben <- 1:10^7
> > system.time(BEN <- ifelse(ben<10, NA, -ben))
> >    user  system elapsed
> >    1.31    0.24    1.56
> > system.time({BEN1 <- -ben; BEN1[BEN1> -10] <- NA})
> >    user  system elapsed
> >    0.17    0.03    0.20
> > all.equal(BEN, BEN1)
> > [1] TRUE
> >
> > HTH ...
> >
> > Peter Alspach
> >
> > -----Original Message-----
> > From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org]
> > On Behalf Of Benjamin Caldwell
> > Sent: Wednesday, 6 March 2013 10:18 a.m.
> > To: r-help
> > Subject: [R] Function completely locks up my computer if the input is too
> > big
> >
> > Dear r-help,
> >
> >
> > Somewhere in my innocuous function to rotate an object in Cartesian space
> > I've created a monster that completely locks up my computer (requires a
> > hard reset every time). I don't know if this is useful description to
> > anyone - the mouse still responds, but not the keyboard and not windows
> > explorer.
> >
> > The script only does this when the input matrix is large, and so my
> > initial runs didn't catch it as I used a smaller matrix to speed up the
> > test runs.
> > When I tried an input matrix with a number of dimensions in the same order
> > of magnitude as the data I want to read in, R and my computer choked. This
> > was a surprise for me, as I've always been able to break execution in the
> > past or do other tasks. So i tried it again, and still no dice.
> >
> > Now I need the function to work as subsequent functions/functionality are
> > dependent, and I can't see anything on the face of it that would likely
> > cause the issue.
> >
> > Any insight on why this happens in general or specifically in my case are
> > appreciated. Running R 15.2, Platform: x86_64-w64-mingw32/x64 (64-bit) on a
> > windows 7 machine with 4 mb RAM. In the meantime I suppose I'll write a
> > loop to do this function piece-wise for larger data and see if that helps.
> >
> > Script is attached and appended below.
> >
> > Thanks
> >
> > Ben Caldwell
> >
> >
> >
> > #compass to polar coordinates
> >
> > compass2polar <- function(x) {-x+90}
> >
> >
> >
> > #degrees (polar) to radians
> >
> > Deg2Rad <- function(x) {(x*pi)/180}
> >
> >
> >
> > # radians to degrees
> >
> > Rad2Deg <- function (rad) (rad/pi)*180
> >
> >
> >
> > # polar to cartesian coordinates - assumes degrees those from a compass.
> > output is a list, x & y of equal length
> >
> > Pol2Car <- function(distance,deg) {
> >
> >
> > rad <- Deg2Rad(compass2polar(deg))
> >
> > rad <- rep(rad, length(distance))
> >
> >
> > x <- ifelse(is.na(distance), NA, distance * cos(rad))
> >
> > y <- ifelse(is.na(distance), NA, distance * sin(rad))
> >
> >
> > x<-round(x,2)
> >
> > y<-round(y,2)
> >
> >
> > cartes<- list(x,y)
> >
> > name<-c('x','y')
> >
> > names(cartes)<-name
> >
> > cartes
> >
> > }
> >
> >
> >
> >
> >
> > #rotate an object, with assumed origin at 0,0, in any number of degrees
> >
> > rotate <- function(x,y,tilt){ 8
> >
> >
> > d2 <- x^2+y^2
> >
> > rotate.dis<-sqrt(d2)
> >
> > or.rad <- atan(x/y)
> >
> > or.deg <- Rad2Deg(or.rad)
> >
> >
> > n <- length(or.deg)
> >
> > for(i in 1:n){
> >
> > if(is.na(or.deg[i])==TRUE) {or.deg[i] <- 0}
> >
> > }
> >
> > # browser()
> >
> > tilt.in <- tilt + or.deg
> >
> >
> > xy<-Pol2Car (distance=rotate.dis, deg=tilt.in)
> >
> >  # if(abs(tilt) >= 0) {
> >
> >  # shift.frame <- cbind(xy$x, xy$y)
> >
> > # shift.frame.val <- shift.frame[shift.frame[,2]==min(shift.frame[,2]),]
> >
> > # shift.x<- shift.frame.val[1] * -1
> >
> > # shift.y<- shift.frame.val[2] * -1
> >
> > # x<-xy$x + shift.x
> >
> > # y<-xy$y + shift.y
> >
> > # }
> >
> > # name <- c('x', 'y')
> >
> > # xy<-list(x,y)
> >
> > # names(xy)<-name
> >
> >  xy
> >
> > }
> >
> >
> > x <- seq(0,5, .5)
> >
> > y <- seq(0,5, .5)
> >
> > z <- seq(0,5, .5)
> >
> > dquad<-expand.grid(x,y,z)
> >
> > name<-c("y","x","z")
> >
> > names(dquad)<-name
> >
> >
> > plot(dquad$x, dquad$y, xlim=c(-25,25), ylim=c(-25,25))
> >
> >
> > #this works fine
> >
> > rotated<-rotate(dquad$x, dquad$y, 45)
> >
> >
> >
> > points(rotated$x, rotated$y, col='green')
> >
> >
> > # profiling of both time and memory
> >
> > Rprof("myFunction.out", memory.profiling=T)
> >
> > y <- myFunction(x)
> >
> > Rprof(NULL)
> >
> > summaryRprof("myFunction.out", memory="both")
> >
> >
> >
> > #############
> >
> > x <- seq(0,5, .1)
> >
> > y <- seq(0,5, .1)
> >
> > z <- seq(0,5, .1)
> >
> > dquad<-expand.grid(x,y,z)
> >
> > name<-c("y","x","z")
> >
> > names(dquad)<-name
> >
> > # running the below locks up my machine (commented out to avoid accidental
> > run)
> >
> > # rotated<-rotate(dquad$x, dquad$y, 45)
> >
> > The contents of this e-mail are confidential and may be subject to legal
> > privilege.
> >  If you are not the intended recipient you must not use, disseminate,
> > distribute or
> >  reproduce all or any part of this e-mail or attachments.  If you have
> > received this
> >  e-mail in error, please notify the sender and delete all material
> > pertaining to this
> >  e-mail.  Any opinion or views expressed in this e-mail are those of the
> > individual
> >  sender and may not represent those of The New Zealand Institute for Plant
> > and
> >  Food Research Limited.
> >
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.