In your specific example, foreach will not be appreciably slower than
clusterApply, as the cv.glmnet function takes a long time to compute
relative to the computational overhead of using foreach.  I'm too lazy to
wait for the glmnet models to run on a 10,000 column matrix, so I
benchmarked some different functions on a 10 column matrix.  In my tests,
the speeds of clusterApply and foreach were almost exactly the same: both 3
times faster than a sequential approach.  You can add more columns to the x
matrix and add more replications to the benchmark experiment if you wish to
further explore this issue, but using foreach to solve your problem is fine:

require(glmnet)
require(caret)
require(foreach)
require(doParallel)
require(rbenchmark)

cores<-detectCores()
cl <- makeCluster(cores, methods=FALSE)
registerDoParallel(cl)

tuneAlpha_Seq <- function(x, y, alphas){
  lapply(alphas, function(alpha) {
    stopifnot(require(glmnet))
    cv.glmnet(x, y, alpha=alpha)
  })
}

tuneAlpha_Par<- function(x, y, alphas){
  clusterExport(cl, c("x","y"))
  clusterApply(cl, alphas, function(alpha) {
    stopifnot(require(glmnet))
    cv.glmnet(x, y, alpha=alpha)
  })
}

tuneAlpha_Foreach_Seq <- function(x, y, alphas){
  foreach(alpha=alphas) %do% {
    stopifnot(require(glmnet))
    cv.glmnet(x, y, alpha=alpha)
  }
}

tuneAlpha_Foreach_Par <- function(x, y, alphas){
  foreach(alpha=alphas) %dopar% {
    stopifnot(require(glmnet))
    cv.glmnet(x, y, alpha=alpha)
  }
}

x <- matrix(rnorm(2000*100),ncol=100)
y <- matrix(rnorm(2000),ncol=1)
alphas <- seq(0, 1, length.out= cores)

benchmark(replications=10,
          tuneAlpha_Seq(x, y, alphas),
          tuneAlpha_Par(x, y, alphas),
          tuneAlpha_Foreach_Seq(x, y, alphas),
          tuneAlpha_Foreach_Par(x, y, alphas),
          columns=c('test', 'elapsed', 'replications', 'relative'),
          order='relative')

stopCluster(cl)

                                 test elapsed replications relative
tuneAlpha_Par(x, y, alphas)                 7.47       10   1.000
tuneAlpha_Foreach_Par(x, y, alphas)    7.83       10    1.048
tuneAlpha_Foreach_Seq(x, y, alphas)   23.13      10    3.096
tuneAlpha_Seq(x, y, alphas)                 24.26      10    3.248

On Thu, Sep 6, 2012 at 1:45 PM, Patrik Waldmann
<patrik.waldmann@boku.ac.at>wrote:

> Have a look at 'Parallel linear model', foreach was ten times slower (at
> that specific example). I don't know the reason.
>
> Patrik
>
> >>> Zachary Mayer <zach.mayer@gmail.com> 09/06/12 19:20 PM >>>
> I was not aware of this unfortunate limitation.  Was it specific to a
> particular back-end, or just foreach in general?
>
> On Thu, Sep 6, 2012 at 1:15 PM, Patrik Waldmann
> <patrik.waldmann@boku.ac.at>wrote:
>
> > I would like to avoid foreach since we showed earlier that it is VERY
> slow.
> >
> > Patrik
> >
> > >>> Zachary Mayer <zach.mayer@gmail.com> 09/06/12 18:09 PM >>>
> > You could also use the foreach package, if you do not wish to use caret,
> > e.g.:
> >
> > tuneAlpha <- function(..., alphas){
> >   stopifnot(require(foreach))
> >  #Run Models
> > modelList <- foreach(alpha=alphas) %dopar% {
> >   stopifnot(require(glmnet))
> > cv.glmnet(..., alpha=alpha)
> > }
> >  #Choose best model
> > errors <- unlist(lapply(modelList, function(x) min(sqrt(x$cvm))))
> > return(modelList[[which.min(errors)]])
> > }
> >
> > x <- matrix(rnorm(2000*100),ncol=100)
> > y <- matrix(rnorm(2000),ncol=1)
> > model <- tuneAlpha(x, y, alphas=c(0,1), family="gaussian", nfolds=10,
> > type.measure="mse")
> >
> > You should probably pass an explicit "foldid" parameter as well, so each
> > model uses the same cross-validation folds.
> >
> > On Thu, Sep 6, 2012 at 11:58 AM, Zachary Mayer <zach.mayer@gmail.com>
> > wrote:
> >
> > > Hasn't the caret package already solved this problem?
> > >
> > > You can pass the tuneGrid parameter to specify your custom alpha and
> > > lambda sequence, an the trainControl parameter to specify what kind of
> > > cross-validation you wish to use.
> > >
> > > Caret uses foreach, so you can register a parallel backend of your
> > choice.
> > >
> > > Sent from my iPhone
> > >
> > > On Sep 6, 2012, at 11:56 AM, Patrik Waldmann <
> patrik.waldmann@boku.ac.at
> > >
> > > wrote:
> > >
> > > > I want to run the cv.glmnet function with the same data (y and x)
> with
> > > different values on the alpha parameter determined by the number of
> > cores,
> > > but the result is absurd. What is wrong in the code below?
> > > >
> > > > Patrik Waldmann
> > > >
> > > > x <- matrix(rnorm(2000*10000),ncol=10000)
> > > > y <- matrix(rnorm(2000),ncol=1)
> > > >
> > > > library(parallel)
> > > > cvglmnet <- function(...) {
> > > > library(glmnet)
> > > > cv.glmnet(x,y,alpha=alphasplit)
> > > > }
> > > > system.time(cores<-detectCores())
> > > > system.time(cl <- makeCluster(cores, methods=FALSE))
> > > > alpha<-seq(0, 1,by=1/(cores-1))
> > > > alphasplit<-clusterSplit(cl,alpha)
> > > > system.time(clusterExport(cl, c("x","y","cvglmnet","alphasplit")))
> > > > system.time(outbrlist<-clusterEvalQ(cl, cvglmnet(x,y,alphasplit)))
> > > > system.time(totoutbr<-do.call(cbind,outbrlist))
> > > > stopCluster(cl)
> > > >
> > > >    [[alternative HTML version deleted]]
> > > >
> > > > _______________________________________________
> > > > R-sig-hpc mailing list
> > > > R-sig-hpc@r-project.org
> > > > https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
> > >
> >
> >
>
>

	[[alternative HTML version deleted]]

