[R] apply vs. foreach vs. foreach with doSMP (multi cores)

[Tao Shi]

Hi David and list,

I'm a little puzzled to see these results below.  Since, "apply" is basically a for loop, I was expecting "foreach" uses about same amount of time as apply, whereas foreach after registering 2-cores runs much faster.  However, the results show apply is the fastest.  

Also could you please explain the error message (i.e. Error in ipcTaskSetEnvironment(taskq, envir) : ...) in the second run?

The results were recorded on REvolution R 3.2 and I observed the same on regular R2.11.0.

Many thanks!

...Tao



> library(doSMP)
Loading required package: foreach
Loading required package: iterators
Loading required package: codetools
foreach: simple, scalable parallel programming from REvolution Computing
Use REvolution R for scalability, fault tolerance and more.
http://www.revolution-computing.com
Loading required package: revoIPC
> 
> m <- matrix(rnorm(100000), 10000, 10)
> 
> system.time(tmp <- t(apply(m, 1, function(x) x/mean(x))))
   user  system elapsed 
   0.21    0.00    0.21 
> system.time(tmp1 <- foreach(i=1:nrow(m), .combine=rbind) %dopar% (m[i,] / mean(m[i,])))
   user  system elapsed 
   5.50    0.00    5.53 
Warning message:
executing %dopar% sequentially: no parallel backend registered 
> 
>
> w <- startWorkers(2)
Warning messages:
1: In startWorkers(2) : there is an existing doSMP session using doSMP1
2: In startWorkers(2) : there is an existing doSMP session using doSMP2
> registerDoSMP(w)
> system.time(tmp1 <- foreach(i=1:nrow(m), .combine=rbind) %dopar% (m[i,] / mean(m[i,])))
   user  system elapsed 
   6.02    0.03    7.84 
> stopWorkers(w)
> 
> ## second run
> system.time(tmp <- t(apply(m, 1, function(x) x/mean(x))))
   user  system elapsed 
   0.22    0.02    0.23 
> system.time(tmp1 <- foreach(i=1:nrow(m), .combine=rbind) %dopar% (m[i,] / mean(m[i,])))
Error in ipcTaskSetEnvironment(taskq, envir) : 
  The task queue has been freed.
Timing stopped at: 0.03 0.02 0.04 
> 
> 
> w <- startWorkers(2)
Warning messages:
1: In startWorkers(2) : there is an existing doSMP session using doSMP1
2: In startWorkers(2) : there is an existing doSMP session using doSMP2
> registerDoSMP(w)
> system.time(tmp1 <- foreach(i=1:nrow(m), .combine=rbind) %dopar% (m[i,] / mean(m[i,])))
   user  system elapsed 
   6.11    0.01    7.62 
> stopWorkers(w)
> dim(m)
[1] 10000    10
> sessionInfo()
R version 2.10.1 (2009-12-14) 
i386-pc-intel32 

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252    LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                           LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] doSMP_1.0-1     revoIPC_1.0-2   foreach_1.3.0   codetools_0.2-2 iterators_1.0.3 Revobase_3.2.0 
> 

 		 	   		  
_________________________________________________________________
The New Busy is not the old busy. Search, chat and e-mail from your inbox.

N:WL:en-US:WM_HMP:042010_3