[R] speeding up a loop

[David Winsemius]

On Oct 17, 2013, at 2:56 PM, Ye Lin wrote:

> Hey R professionals,
> 
> I have a large dataset and I want to run a loop on it basically creating a
> new column which gathers information from another reference table.
> 
> When I run the code, R just freezes and even does not response after 30min
> which is really unusual. I tried sapply as well but does not improve at
> all.
> 
> I am running R 3.0.2 on Windows 7.  I checked the system, when I run the
> code, my CPU usage is about 25%-30% that is taxing my desktop.

A guess: It's not your CPU use ... it's your RAM use. You've probably exhausted your RAM and your system has paged out to virutla memory
> 
> Here is my code:
> 
> #df1 is the data set I want to add a new column#
> #b is the reference tabel#
> 
> for (i in (1:nrow(df1))) {
>  begin=which(b$Time2==df1$start[i] & b$Date==df1$Date[i])
>  date=unlist(strsplit(as.character(dff$end[i])," "))[1]
>   end=ifelse(date=="2013-10-17",
>   which(b$Time2==df1$end[i] & b$Date==df1$Date[i]),
>   which(b$Time2==df1$end[i]-3600*24 & b$Date==as.Date(df1$Date[i])+1))
>    df1$new[i] <- sum(b[begin:end,]$Power)
> }
> 

I get: 
Error in strsplit(as.character(dff$end[i]), " ") : object 'dff' not found

If I change the dff to df1, I get: 
Error in begin:end : argument of length 0

-- 
David.
> And here is a mimic sample of df1 & b:
> 
> df1 <- structure(list(Date = structure(c(1369699200, 1369699200,
> 1369699200,
> 1369699200, 1369699200), tzone = "UTC", class = c("POSIXct",
> "POSIXt")), start = structure(c(1381991205, 1381990247, 1382010454,
> 1382007281, 1381992288), tzone = "UTC", class = c("POSIXct",
> "POSIXt")), end = structure(c(1381992405, 1381993727, 1382010694,
> 1382007461, 1381992468), tzone = "UTC", class = c("POSIXct",
> "POSIXt"))), .Names = c("Date", "start", "end"), row.names = c(NA,
> -5L), class = "data.frame")
> 
> 
> b <- structure(list(Date = structure(c(1369699200, 1369699200, 1369699200,
> 1369699200, 1369699200, 1369699200, 1369699200, 1369699200, 1369699200,
> 1369699200, 1369699200, 1369699200, 1369699200, 1369699200, 1369699200,
> 1369699200, 1369699200, 1369699200, 1369699200, 1369699200, 1369699200,
> 1369699200, 1369699200, 1369699200, 1369699200, 1369699200, 1369699200,
> 1369699200, 1369699200, 1369699200, 1369699200, 1369699200, 1369699200,
> 1369699200, 1369699200, 1369699200, 1369699200, 1369699200, 1369699200,
> 1369699200, 1369699200, 1369699200, 1369699200, 1369699200, 1369699200,
> 1369699200, 1369699200, 1369699200, 1369699200, 1369699200), tzone = "UTC",
> class = c("POSIXct",
> "POSIXt")), Time2 = structure(c(1381989634, 1381989694, 1381989754,
> 1381989814, 1381989874, 1381989934, 1381989994, 1381990054, 1381990114,
> 1381990174, 1381990234, 1381990294, 1381990354, 1381990414, 1381990474,
> 1381990534, 1381990594, 1381990654, 1381990714, 1381990774, 1381990834,
> 1381990894, 1381990954, 1381991014, 1381991074, 1381991134, 1381991194,
> 1381991254, 1381991314, 1381991374, 1381991434, 1381991494, 1381991554,
> 1381991614, 1381991674, 1381991734, 1381991794, 1381991854, 1381991914,
> 1381991974, 1381992034, 1381992094, 1381992154, 1381992214, 1381992274,
> 1381992334, 1381992394, 1381992454, 1381992514, 1381992574), tzone = "UTC",
> class = c("POSIXct",
> "POSIXt")), Power = c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12,
> 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28,
> 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44,
> 45, 46, 47, 48, 49, 50)), .Names = c("Date", "Time2", "Power"
> ), row.names = c(NA, -50L), class = "data.frame")
> 
> Thanks for your help!
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

David Winsemius
Alameda, CA, USA