[R-sig-hpc] foreach package - Parallel Apply (Section 5.2 in Vignettes foreach Manual)

[Stephen Weston]

It's worth noting that the iter method for matrices also allows you to iterate
over multiple columns at a time.  But instead of specifying the number
of blocks, you have to specify the number of columns in each block.

Here's an example:

> library(iterators)
> x <- matrix(rnorm(16), 4, 4)
> it <- iter(x, chunksize=2)
> print(as.list(it))
[[1]]
           [,1]       [,2]
[1,]  0.1811217  1.7126129
[2,]  0.1415969  0.6980985
[3,] -1.3921071 -0.9700261
[4,] -1.2338734 -0.5530707

[[2]]
           [,1]        [,2]
[1,] -1.1162171  1.09920729
[2,]  0.9893546 -0.37677605
[3,] -0.5508948 -0.02057267
[4,] -0.3817569 -1.53422901

The other method is more convenient when you want to
split up the work evenly between all of the parallel workers,
however.

- Steve


On Wed, Dec 9, 2009 at 5:38 PM, Dirk Eddelbuettel <edd at debian.org> wrote:
>
> On 10 December 2009 at 09:12, Debabrata Midya wrote:
> | Dear hpc users,
> |
> | Thanks in advance.
> |
> | I am using R 2.10.0 on Windows XP, Intel(R) Core(TM) 2 Duo CPU E8400 @3.00GHz 2.99GHz, 1.96 of RAM.
> |
> | I need you assistance to solve the problem below:
> |
> | foreach package - Parallel Apply (Section 5.2 in Vignettes foreach Manual)
> |
> | "Now we're only sending any given column of the matrix to one parallel execution worker. But it
> | would be even more efficient if we sent the matrix in bigger chunks. To do that, we use a function
> | called iblkcol that returns an iterator that will return multiple columns of the original matrix.
> | That means that the R expression will need to execute the user's function once for every column in
> | its submatrix".
> |
> | The function below is from the above Vigenette:
> |
> | applyKernel <- function(newX, FUN, d2, d.call, dn.call = NULL, ...) {
> |  foreach(x = iblkcol(newX, 3), .combine = "c", .packages = "foreach") %dopar%
> |  {
> |     foreach(i = 1:ncol(x)) %do% FUN(array(x[, i], d.call, dn.call), ...)
> |  }
> | }
> |
> | > applyKernel(matrix(1:16, 4), mean, 4, 4)
> |
> | Error in eval(expr, envir, enclos) : could not find function "iblkcol"
> |
> | How can I get this function?
>
> I am not sure how familiar you are with vignettes but the key is that they
> contain all the code.  So here it is, straight from the package sources. The
> <<...>> and @ lines are the Sweave markers, R code is in-between.
>
> <<ex13.iter, results=hide, echo=FALSE>>=
> iblkcol <- function(a, chunks) {
>  n <- ncol(a)
>  i <- 1
>
>  nextElem <- function() {
>    if (chunks <= 0 || n <= 0) stop('StopIteration')
>    m <- ceiling(n / chunks)
>    r <- seq(i, length=m)
>    i <<- i + m
>    n <<- n - m
>    chunks <<- chunks - 1
>    a[,r, drop=FALSE]
>  }
>
>  structure(list(nextElem=nextElem), class=c('iblkcol', 'iter'))
> }
> nextElem.iblkcol <- function(obj) obj$nextElem()
> @
>
>
> Use the sources for R and its packages. They are a much-underused treasure.
>
> Dirk
>
> |
> | Once again, thank you very much for the time you have given.
> |
> |
> | I am looking forward for your reply.
> |
> | Regards,
> |
> | Deb
> |
> |       [[alternative HTML version deleted]]
> |
> | _______________________________________________
> | R-sig-hpc mailing list
> | R-sig-hpc at r-project.org
> | https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
>
> --
> Three out of two people have difficulties with fractions.
>
> _______________________________________________
> R-sig-hpc mailing list
> R-sig-hpc at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
>