Steve,
 
Thank you very much for your quick response.
 
I like the approach you have provided to me.
 
Just few clarifications please.
 
1: 
 
# Create an iterator that will return "numWorkers" values that sum to itermax
    it <- idiv(itermax, chunks=getDoParWorkers())
I think itermax is divided into "numWorkers" and a chunk of iterations will be allocated to each worker. Am I correct?
 
2:
 
# Pick a starting solution based on randomness
      cursol <- pickRandomSolution()
Does each worker receive random solution [that means initial random solutions are different from one worker to another]? Am I correct?
 
3:

# Pick the best solution that the cluster workers find
    bestsol <- foreach(i=it, .combine='bestSolution') %dopar% {
      lookForBestSolution(i)
    }
At the end, .combine='bestSolution' provides best solution as per the defined function bestSolution. That means this is a solution combined from different workers? Am I correct?
 
Once again, thank you very much for the time you have given.
Regards,

Deb

>>> Stephen Weston <stephen.b.weston@gmail.com> 11/12/2009 1:26 am >>>
I was trying to say that it is possible to parallelize iterative solutions
in some cases.  For instance, if you're starting from a solution that is
based on randomness, and iteratively improving it, you could
parallelize it in this fashion:

    library(doSNOW)

    # Create, initialize, and register a snow socket cluster
    cl <- makeSOCKcluster(2)
    clusterSetupRNG(cl)
    registerDoSNOW(cl)

    # Define a simple search function
    lookForBestSolution <- function(itermax) {
      # Pick a starting solution based on randomness
      cursol <- pickRandomSolution()

      # Repeatedly try to improve the solution
      for (i in seq(length=itermax)) {
        x <- tryToImproveSolution(cursol)
        cursol <- bestSolution(x, cursol)
      }

      # Return the best solution found
      cursol
    }

    # Define some very silly example functions for purposes of illustration
    pickRandomSolution <- function() rnorm(1)
    tryToImproveSolution <- function(x) x - rnorm(1)
    bestSolution <- function(...) min(...)

    # Define the total number of times you're willing to improve your solution
    itermax <- 500

    # Create an iterator that will return "numWorkers" values that sum
to itermax
    it <- idiv(itermax, chunks=getDoParWorkers())

    # Pick the best solution that the cluster workers find
    bestsol <- foreach(i=it, .combine='bestSolution') %dopar% {
      lookForBestSolution(i)
    }
    print(bestsol)

    # Shutdown the socket cluster
    stopCluster(cl)


Note that the call to clusterSetupRNG is essential for programs
of this type.  Otherwise, you could have each worker compute
exactly the same result, and you don't gain anything from parallelism.

- Steve


On Wed, Dec 9, 2009 at 10:39 PM, Debabrata Midya
<Debabrata.Midya@services.nsw.gov.au> wrote:
> Steve, Martin, Tim,
>
> Thanks to all of you to give some time and ideas on it.
>
> Your are correct Steve, "The "real" problem is bigger" at least in number of
> dimensions (in my example code d = 3).
>
> The code I have provided is an example one to start with. Although, the
> concept to the real problem will not differ much compared to the example
> code, I have described the whole problem below in A and B.
>
> How can I make a parallel version of it?
>
> A. I have described a variable called cursol in my example code. In my final
> code, I like to have cursol as a matrix of order m x (d+1). Where m = number
> of possible solutions (as for example m= 100). d is the number of variables.
> In my example code d= 3 (trainX= c(x[1], x[2] and x[3])). The d+1 -th
> variable is the value of the func at trainX.
>
> At the end of j = 1 (iteration number 1), the best possible solution
> (minimum value) of the problem is: cursol[1, ]; the second best solution is
> cursol[2, ] and so on. [Note: I have ordered the matrix cursol as per
> cursol[, d+1] from lowest value to highest]
>
> B. At the start of j = 2 (the iteration number 2), trainX = cursol[, 1:d].
> And the iteration goes like A.
>
> That means the whole problem is:
>
> func <- function() {
>
>    # Some calculations
>
>    value # Value of the function at some trainX
> }
>
> main <- function(args) {
>
>      # Some initializations if needed
>
>      # Some calculations if needed
>
>     # trainX: Some value / calculated value
>
>      for i in 1:itermax)
>        {
>
>            # All calculations including the call to the external function
> func
>
>            cursol <- ----
>
>            trainX <- cursol + runif() # Using random number
>        }
> }
>
> solution <- main(args)
>
> Once again, thank you very much for the time you have given.
> Regards,
> Deb
>
>>>> Stephen Weston <stephen.b.weston@gmail.com> 10/12/2009 1:13 pm >>>
> I was thinking that you'd have one task per worker, and that each
> task would take a smaller number of iterations and return their
> best result.  The final result would be the best result of all.
> Of course, you'd probably have to do more total iterations in order
> to get as good a result as in the sequential version, but I think
> you'd still get some benefit from running in parallel.
>
> I haven't looked over the code too carefully, so maybe I'm
> misreading it.  I was also assuming that the "real" problem
> was bigger, since this version looks like it would run very quickly
> sequentially, and so wouldn't be worth doing in parallel.
>
> Also, checkval should be vectorized using pmax and pmin.
>
> - Steve
>
>
> On Wed, Dec 9, 2009 at 8:22 PM, Martin Morgan <mtmorgan@fhcrc.org> wrote:
>> Hi Deb, Steve,
>>
>> Stephen Weston wrote:
>>> Using doSNOW with a socket cluster is probably your best bet.
>>>
>>> - Steve
>>>
>>>
>>> On Wed, Dec 9, 2009 at 6:52 PM, Debabrata Midya
>>> <Debabrata.Midya@services.nsw.gov.au> wrote:
>>>> Tim,
>>>>
>>>> Thanks for your reply.
>>>>
>>>> One clarification please.
>>>>
>>>> I am using R 2.10.0 on Windows XP, Intel(R) Core(TM) 2 Duo CPU E8400
>>>> @3.00GHz 2.99GHz, 1.96 of RAM.
>>>> I have checked that Windows binary of doMC is not available. Any advice.
>>>>
>>>> Once again, thank you very much for the time you have given.
>>>>
>>>> Regards,
>>>>
>>>> Deb
>>>>
>>>>>>> Tim Triche <tim.triche@gmail.com> 10/12/2009 3:54 am >>>
>>>> library(foreach)
>>>> library(doMC) # or doWhatever, take your pick
>>>> registerDoMC(4)
>>>>
>>>> results <- foreach( j=1:itermax ) %dopar% {
>>>>
>>>> # stuff
>>>> if( converged ) break
>>>>
>>>> }
>>
>> Unfortunately, the iterations are not independent; it's instructive to
>> try to get something sensible out of
>>
>>  res0 <- 0
>>  tmp <- foreach (j = 1:10) %dopar% { res0 <- res0 + j }
>>  res0
>>
>> versus what you might expect
>>
>>  res1 <- 0
>>  for (j in 1:10) { res1 <- res1 + j }
>>  res1
>>
>> and to understand how Deb's problem maps to this case.
>>
>> Martin
>>
>>
>>>>
>>>> The details are up to you. I don't remember how best to break out of a
>>>> foreach loop, try help('foreach').
>>>> Have fun. It's pretty straightforward once you get used to foreach()
>>>> behavior, less so up to that point.
>>>> But it's a really useful tool to have in your arsenal for the problem
>>>> you describe. Try to loop over the biggest chunks of processing you can get
>>>> away with, as fork()ing processes has some consequences.
>>>>
>>>>
>>>>
>>>> On Tue, Dec 8, 2009 at 8:02 PM, Debabrata Midya
>>>> <Debabrata.Midya@services.nsw.gov.au> wrote:
>>>>
>>>>
>>>>
>>>> Dear hpc users,
>>>>
>>>> Thanks in advance.
>>>>
>>>> I am using R 2.10.0 on Windows XP, Intel(R) Core(TM) 2 Duo CPU E8400
>>>> @3.00GHz 2.99GHz, 1.96 of RAM.
>>>>
>>>> I like to have a parallel version of the code below. This is an example
>>>> code only.
>>>>
>>>> A: cursol contains values of x[1], x[2], x[3] and minimum value from
>>>> func using x's.
>>>>
>>>> ###########
>>>> # Code starts
>>>> ###########
>>>>
>>>> func <- function(x)
>>>> {
>>>> 10 * x[1] - 11 * x[2] + 12 * x[3]
>>>> }
>>>>
>>>> main <- function(xl, xu)
>>>> {
>>>> checkval <- function(x, xl, xu)
>>>> {
>>>> i <- 1
>>>> while (i <= 3)
>>>> {
>>>> if (x[i] < xl[i]) x[i] <- xl[i]
>>>> if (x[i] > xu[i]) x[i] <- xu[i]
>>>> i <- i + 1
>>>> }
>>>> x
>>>> }
>>>>
>>>> itermax <- 50
>>>>
>>>> j <- 1
>>>> while (j <= itermax)
>>>> {
>>>> # trainX is a matrix of 3 x 3
>>>> trainX <- matrix(rep(xl, 3), 3, 3, byrow=TRUE) + matrix(runif(9, 0, 1),
>>>> 3, 3)
>>>> trainX
>>>> trainX <- t(apply(trainX, 1, checkval, xl, xu))
>>>> trainX
>>>> sold <- apply(trainX, 1, func)
>>>> osold <- cbind(1:3, sold)
>>>> osold <- osold[order(osold[, 2]), ]
>>>> osold
>>>>
>>>> if (j > 1)
>>>> {
>>>> if (osold[1, 2] < cursol[1, 4])
>>>> {
>>>> cursol <- matrix(c(trainX[osold[1,1], ], osold[1,2]), 1, 4)
>>>> }
>>>> else
>>>> {
>>>> cursol <- cursol
>>>> }
>>>> }
>>>>
>>>> if (j <= 1)
>>>> {
>>>> cursol <- matrix(c(trainX[osold[1,1], ], osold[1,2]), 1, 4)
>>>> }
>>>>
>>>> cursol
>>>> j <- j + 1
>>>> }
>>>>
>>>> cursol
>>>> }
>>>>
>>>> xl <- c(1, 2, 3)
>>>> xu <- c(2, 3, 4)
>>>>
>>>> cursol <- main(xl, xu)
>>>> cursol
>>>>
>>>>
>>>> ###########
>>>> # Code ends
>>>> ###########
>>>>
>>>> Once again, thank you very much for the time you have given.
>>>>
>>>> I am looking forward for your reply.
>>>>
>>>> Regards,
>>>>
>>>> Deb
>>>>
>>>>
>>>> [[alternative HTML version deleted]]
>>>>
>>>> _______________________________________________
>>>> R-sig-hpc mailing list
>>>> R-sig-hpc@r-project.org 
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc 
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Absolute certainty is the privilege of mathematicians and madmen.
>>>> It is, for scientists, an unobtainable ideal.
>>>> --Cassius J. Keyser
>>>>
>>>>        [[alternative HTML version deleted]]
>>>>
>>>> _______________________________________________
>>>> R-sig-hpc mailing list
>>>> R-sig-hpc@r-project.org 
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc 
>>>>
>>>
>>> _______________________________________________
>>> R-sig-hpc mailing list
>>> R-sig-hpc@r-project.org 
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc 
>>
>>
>> --
>> Martin Morgan
>> Computational Biology / Fred Hutchinson Cancer Research Center
>> 1100 Fairview Ave. N.
>> PO Box 19024 Seattle, WA 98109
>>
>> Location: Arnold Building M1 B861
>> Phone: (206) 667-2793
>>
>

	[[alternative HTML version deleted]]

