In this case, each iteration of the function is very quick:
> system.time(summary(lm(y ~ x[,1]))$coefficients[2,4])
   user  system elapsed
   0.01    0.00    0.02

And you are doing 10,000 iterations, so overhead matters a lot.  In the
glmnet problem, each iteration of the function is very slow, and you are
doing 8 iterations, so overhead doesn't matter at all.

Finally, I suspect that using the doMC foreach backend will improve things
considerably, but I can't currently test that.

On Thu, Sep 6, 2012 at 4:16 PM, Patrik Waldmann
<patrik.waldmann@boku.ac.at>wrote:

> y<-rnorm(1000)
> x<-matrix(rnorm(1000*10000),ncol=10000)
> dimx<-dim(x)
>
> library(doParallel)
> library(foreach)
> cl <- makeCluster(8, methods=FALSE)
> registerDoParallel(cl)
> print(system.time(
> pval <- foreach (i =1:dimx[2], .combine=c) %dopar% {
> mod <- lm(y ~ x[,i])
> summary(mod)$coefficients[2,4]
> }
> ))
>
>   user  system elapsed
>  12.28    2.75  231.93
>
> stopCluster(cl)
>
> library(parallel)
> cl <- makeCluster(8, methods=FALSE)
> print(system.time(
> pval <- unlist(mclapply(1:dimx[2], function(i) summary(lm(y ~
> x[,i]))$coefficients[2,4]))
> ))
>
>   user  system elapsed
>  21.80    1.33   25.78
>
> stopCluster(cl)
>
> >>> "Brian G. Peterson" <brian@braverock.com> 09/06/12 20:03 PM >>>
> On 09/06/2012 12:15 PM, Patrik Waldmann wrote:
> > I would like to avoid foreach since we showed earlier that it is VERY
> slow.
>
> A search of the list archives doesn't provide any backup that you have
> 'showed' anything.
>
> The only other post by you that I can see on the list shows the total
> elapsed time decreasing with the use of foreach, even though it was,
> indeed, a trivial function evaluation.
>
> --
> Brian
>
> _______________________________________________
> R-sig-hpc mailing list
> R-sig-hpc@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
>
> _______________________________________________
> R-sig-hpc mailing list
> R-sig-hpc@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
>

	[[alternative HTML version deleted]]

