[R-sig-hpc] foreach package - Parallel Apply (Section 5.2 in Vignettes foreach Manual)
Stephen Weston
stephen.b.weston at gmail.com
Thu Dec 10 00:07:33 CET 2009
It's worth noting that the iter method for matrices also allows you to iterate
over multiple columns at a time. But instead of specifying the number
of blocks, you have to specify the number of columns in each block.
Here's an example:
> library(iterators)
> x <- matrix(rnorm(16), 4, 4)
> it <- iter(x, chunksize=2)
> print(as.list(it))
[[1]]
[,1] [,2]
[1,] 0.1811217 1.7126129
[2,] 0.1415969 0.6980985
[3,] -1.3921071 -0.9700261
[4,] -1.2338734 -0.5530707
[[2]]
[,1] [,2]
[1,] -1.1162171 1.09920729
[2,] 0.9893546 -0.37677605
[3,] -0.5508948 -0.02057267
[4,] -0.3817569 -1.53422901
The other method is more convenient when you want to
split up the work evenly between all of the parallel workers,
however.
- Steve
On Wed, Dec 9, 2009 at 5:38 PM, Dirk Eddelbuettel <edd at debian.org> wrote:
>
> On 10 December 2009 at 09:12, Debabrata Midya wrote:
> | Dear hpc users,
> |
> | Thanks in advance.
> |
> | I am using R 2.10.0 on Windows XP, Intel(R) Core(TM) 2 Duo CPU E8400 @3.00GHz 2.99GHz, 1.96 of RAM.
> |
> | I need you assistance to solve the problem below:
> |
> | foreach package - Parallel Apply (Section 5.2 in Vignettes foreach Manual)
> |
> | "Now we're only sending any given column of the matrix to one parallel execution worker. But it
> | would be even more efficient if we sent the matrix in bigger chunks. To do that, we use a function
> | called iblkcol that returns an iterator that will return multiple columns of the original matrix.
> | That means that the R expression will need to execute the user's function once for every column in
> | its submatrix".
> |
> | The function below is from the above Vigenette:
> |
> | applyKernel <- function(newX, FUN, d2, d.call, dn.call = NULL, ...) {
> | foreach(x = iblkcol(newX, 3), .combine = "c", .packages = "foreach") %dopar%
> | {
> | foreach(i = 1:ncol(x)) %do% FUN(array(x[, i], d.call, dn.call), ...)
> | }
> | }
> |
> | > applyKernel(matrix(1:16, 4), mean, 4, 4)
> |
> | Error in eval(expr, envir, enclos) : could not find function "iblkcol"
> |
> | How can I get this function?
>
> I am not sure how familiar you are with vignettes but the key is that they
> contain all the code. So here it is, straight from the package sources. The
> <<...>> and @ lines are the Sweave markers, R code is in-between.
>
> <<ex13.iter, results=hide, echo=FALSE>>=
> iblkcol <- function(a, chunks) {
> n <- ncol(a)
> i <- 1
>
> nextElem <- function() {
> if (chunks <= 0 || n <= 0) stop('StopIteration')
> m <- ceiling(n / chunks)
> r <- seq(i, length=m)
> i <<- i + m
> n <<- n - m
> chunks <<- chunks - 1
> a[,r, drop=FALSE]
> }
>
> structure(list(nextElem=nextElem), class=c('iblkcol', 'iter'))
> }
> nextElem.iblkcol <- function(obj) obj$nextElem()
> @
>
>
> Use the sources for R and its packages. They are a much-underused treasure.
>
> Dirk
>
> |
> | Once again, thank you very much for the time you have given.
> |
> |
> | I am looking forward for your reply.
> |
> | Regards,
> |
> | Deb
> |
> | [[alternative HTML version deleted]]
> |
> | _______________________________________________
> | R-sig-hpc mailing list
> | R-sig-hpc at r-project.org
> | https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
>
> --
> Three out of two people have difficulties with fractions.
>
> _______________________________________________
> R-sig-hpc mailing list
> R-sig-hpc at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
>
More information about the R-sig-hpc
mailing list