[R] Parallel computing with the snow package: external file I/O possible?

[Martin Morgan]

"Waichler, Scott R" <Scott.Waichler at pnl.gov> writes:

> genoud()) should be independent.  However, in the code below, the
> directory created by each node has the same random number in its name.

This is probably because 'random' numbers are generated starting from
a 'seed', the seed is determined (by default) from the system time,
and the system time on the two nodes is identical. The same seed is
being used, hence the same random number sequence.

See http://www.stat.uiowa.edu/~luke/R/cluster/cluster.html

Martin

> I was expecting the contents of fun() or fn() to be independent from all
> other executions of the same function.  What am I missing here?
>
> #  Begin code
> library(snow)
> setDefaultClusterOptions(outfile="/tmp/cluster1")
> setDefaultClusterOptions(master="moab")
> cl <- makeCluster(c("moab", "escalante"), type="SOCK")
>
> # Define base pathname for output from my.test()
> base.dir <- "~"
>
> # Define a function that is called by clusterCall()
> my.test <- function(base.dir) {
>   this.host <- as.character(system("hostname", intern=T))
>   this.rnd <- sample(1:1e6, 1)
>   test.file <- paste(sep="", base.dir, this.host, "_", this.rnd)
>   file.create(test.file)
> }  # end my.test()
>
> clusterCall(cl, my.test, base.dir)
> stopCluster(cl)
> #  End code 
>
> For example, the files "moab_65835" and "escalante_65835" are created.
>
> Regards,
> Scott Waichler
> Pacific Northwest National Laboratory
> scott.waichler at pnl.gov