[R-sig-hpc] Using snow on a looping structure

[Martin Morgan]

Hi Gang --

"Gang Chen" <gangchen6 at gmail.com> writes:

> I'm a newbie running parallel computing, so, sorry for this simple question.
>
> My original code without parallel computing is like this:
>
> runAna <- function(myData, Model, ...) {
>       myStat <- wFun(myData, Model, ...)   # myStat: a vector with a
> length of nStat
>       return(myStat)
> }
>
> rStat <- array(0, dim=c(dimx, dimy, dimz, nStat))
> for (i in 1:dimx)
> for (j in 1:dimy)
> for (k in 1:dimz)
>      rStat[i, j, k,] <- runAna(rData[i, j, k,], Model, ...)   # each
> analysis is on the 4th dimension, and returns nStat numbers which are
> stored in the 4th dimension of rStat

I think what you want is along the lines of

> a <- array(1:(2*3*4*5), c(2,3,4,5))
> b <- apply(a, c(1,2,3), range)

and then as you guessed

> library(snow)
> cl <- makeCluster(nNodes, type="SOCK")
> d <- parApply(cl, a, c(1,2,3), range)
> identical(b, d)
[1] TRUE

so for your example, I'd guess

runStat <- parApply(cl, rData, c(1,2,3), runAna, Model=Model)

This is not quite what you want -- the 'result' dimension is the first
rather than last

> dim(a)
[1] 2 3 4 5
> dim(b)
[1] 2 2 3 4

array-munging is not a speciality of mine, but a simple work-around is
to reorder the dimensions of the original array, so you're applying
to, and writing in, the slice indexed by the first entry

> a <- array(1:(5*2*3*4), c(5,2,3,4))
> b <- apply(a, c(2,3,4), range)
> dim(a)
[1] 5 2 3 4
> dim(b)
[1] 2 2 3 4
> d <- parApply(cl, a, c(2,3,4), range)

Hope that helps,

Martin

> I'm trying to run the above analysis using snow on a machine with two
> processors, but could not figure out how to correctly set it up:
>
> nNodes <- 2
> library(snow)
> cl <- makeCluster(nNodes, type = "SOCK")	
>
> I thought I would use parApply, but how should I combine the looping
> with parApply? Or no looping at all with something like parApply(cl,
> rData, c(1,2,3), ...)?
>
> Thanks in advance,
> Gang
>
> _______________________________________________
> R-sig-hpc mailing list
> R-sig-hpc at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc

-- 
Martin Morgan
Computational Biology / Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N.
PO Box 19024 Seattle, WA 98109

Location: Arnold Building M2 B169
Phone: (206) 667-2793