[R-sig-hpc] doMPI: Issues for RNG and finalizing
Frank Weber
|r@nk@weber @end|ng |rom tu-dortmund@de
Tue Feb 4 10:46:49 CET 2020
Hi everyone,
I would like to start using the package doMPI, but I am having trouble
setting a seed for random number generation (RNG) in doMPI. If I run the
following code:
### Start R code
library(doMPI)
cl_obj <- startMPIcluster(
defaultopts = list(seed = 27510L)
)
registerDoMPI(cl_obj)
n_slaves <- clusterSize(cl_obj)
unif_res <- foreach(
icount(n_slaves),
.combine = "c"
) %dopar% {
runif(1)
}
print(unif_res)
closeCluster(cl_obj)
mpi.quit()
### End R code
then I get the following error message:
### Start error message
Error in nextRNGSubStream(chunkseed) : invalid value of 'seed'
Calls: %dopar% ... <Anonymous> -> master -> submitTaskChunk ->
nextRNGSubStream
### End error message
However, if I use
### Start R code
library(doMPI)
cl_obj <- startMPIcluster()
registerDoMPI(cl_obj)
n_slaves <- clusterSize(cl_obj)
unif_res <- foreach(
icount(n_slaves),
.combine = "c",
.options.mpi = list(seed = 27510L)
) %dopar% {
runif(1)
}
print(unif_res)
closeCluster(cl_obj)
mpi.quit()
### End R code
then this error does not occur anymore and I get the desired random
numbers as a result. Thus, my question is why the first approach does not
work. From the documentation (?doMPI::`doMPI-package` and
?doMPI::startMPIcluster), I thought that it should be working this way.
Note that there is also the function setRngDoMPI(), but I understood the
doMPI documentation in the way that:
- setRngDoMPI() does guarantee different RNG streams for the workers,
but the results are not reproducible
- option "seed" (in argument "defaultopts" of startMPIcluster() as well
as in argument ".options.mpi" of foreach()) guarantees different RNG
streams for the workers and the results are reproducible.
Since I want to have reproducible results, I chose option "seed" instead
of setRngDoMPI(). But correct me if I did not understood the documentation
correctly.
Another issue I am having is when finalizing the MPI cluster: The code
### Start R code
closeCluster(cl_obj)
mpi.quit()
### End R code
does not work for me as the R code after closeCluster(cl_obj) is not
executed anymore. Thus, the MPI cluster continues to run until the
time-out of the job. Using
### Start R code
mpi.exit()
closeCluster(cl_obj)
### End R code
does not work neither as the following error is thrown:
### Start error message
Error: package Rmpi is required by doMPI so will not be detached
### End error message
Help for both issues (but primarily the first one) would be very appreciated.
Thanks and best regards,
Frank Weber
More information about the R-sig-hpc
mailing list