[R-sig-hpc] Problems parallelizing glmnet

[Patrik Waldmann]

y<-rnorm(1000)
x<-matrix(rnorm(1000*10000),ncol=10000)
dimx<-dim(x)

library(doParallel)
library(foreach)
cl <- makeCluster(8, methods=FALSE)
registerDoParallel(cl)
print(system.time(
pval <- foreach (i =1:dimx[2], .combine=c) %dopar% {
mod <- lm(y ~ x[,i])
summary(mod)$coefficients[2,4]
}
))

  user  system elapsed 
 12.28    2.75  231.93 

stopCluster(cl)

library(parallel)
cl <- makeCluster(8, methods=FALSE)
print(system.time(
pval <- unlist(mclapply(1:dimx[2], function(i) summary(lm(y ~ x[,i]))$coefficients[2,4]))
))

  user  system elapsed 
 21.80    1.33   25.78 

stopCluster(cl)

>>> "Brian G. Peterson" <brian at braverock.com> 09/06/12 20:03 PM >>>
On 09/06/2012 12:15 PM, Patrik Waldmann wrote:
> I would like to avoid foreach since we showed earlier that it is VERY slow.

A search of the list archives doesn't provide any backup that you have 
'showed' anything.

The only other post by you that I can see on the list shows the total 
elapsed time decreasing with the use of foreach, even though it was, 
indeed, a trivial function evaluation.

-- 
Brian

_______________________________________________
R-sig-hpc mailing list
R-sig-hpc at r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-hpc