[R] Parallel processing question ...

[Steve Lianoglou]

Hi David,

On Wed, Feb 9, 2011 at 10:11 AM, Robinson, David G <drobin at sandia.gov> wrote:
> Steve,
> Thanks for taking the time to look at the question. my apologies for the
> confusing post. In an attempt to keep the post short, I seem to have
> confused the issue.
>
> The variable of interest in each iteration is the vector lambda  and the
> goal is to collect all the lambda vectors and characterize the statistics of
> lambda over the course of the simulation (this is just a simply gibbs
> sampler) . In the series processing world I simply use cbind to accumulate
> the lambda vectors into an array called lambdas (as performed in  commented
> out  commands).
>
> What I am trying to do now is use a combination of foreach/dopar to do the
> same type of accumulation.
>
> I am not trying to capture any other variables from the loop except lambda.
> As you suggested, I have tried removing the .combine argument and simply
> collect the resulting list. Unfortunately, the lambda vectors don’t appear
> in the resulting list.

Ok, so let's take a look at your code again (without the commented block):

lambdas<- foreach(j=1:(2*burn_in), .combine=cbind) %dopar% {
   N <- update_N(min_sets, min_sets_indexes, lambda)
   lambda <- rgamma(K, shape=a+N, rate=bT)
   lambda
   if (j%%100==0) { print(j); print(lambda); print(N)}
}

Assuming you have all the vars defined properly. Note that your first
line in the loop passes a variable `lambda` into your update_N
function, then you reassign lambda to the result of your rgamma call.

I guess these two things are different, so why not name one of them
something else just in case (I've renamed it to LAMBDA below).

The last line of the block in your loop is an if-statement, which
essentially stops you from returning your lambda vector. Try:

lambdas<- foreach(j=1:(2*burn_in), .combine=cbind) %dopar% {
   N <- update_N(min_sets, min_sets_indexes, LAMBDA)
   lambda <- rgamma(K, shape=a+N, rate=bT)
   if (j%%100==0) { print(j); print(lambda); print(N)}
   lambda
}

-- 
Steve Lianoglou
Graduate Student: Computational Systems Biology
 | Memorial Sloan-Kettering Cancer Center
 | Weill Medical College of Cornell University
Contact Info: http://cbio.mskcc.org/~lianos/contact