[R-sig-hpc] doMPI: Issues for RNG and finalizing

[Frank Weber]

Hi everyone,

I would like to start using the package doMPI, but I am having trouble
setting a seed for random number generation (RNG) in doMPI. If I run the
following code:

### Start R code
library(doMPI)

cl_obj <- startMPIcluster(
  defaultopts = list(seed = 27510L)
)
registerDoMPI(cl_obj)
n_slaves <- clusterSize(cl_obj)

unif_res <- foreach(
  icount(n_slaves),
  .combine = "c"
) %dopar% {
  runif(1)
}
print(unif_res)

closeCluster(cl_obj)
mpi.quit()
### End R code

then I get the following error message:

### Start error message
Error in nextRNGSubStream(chunkseed) : invalid value of 'seed'
Calls: %dopar% ... <Anonymous> -> master -> submitTaskChunk ->
nextRNGSubStream
### End error message

However, if I use

### Start R code
library(doMPI)

cl_obj <- startMPIcluster()
registerDoMPI(cl_obj)
n_slaves <- clusterSize(cl_obj)

unif_res <- foreach(
  icount(n_slaves),
  .combine = "c",
  .options.mpi = list(seed = 27510L)
) %dopar% {
  runif(1)
}
print(unif_res)

closeCluster(cl_obj)
mpi.quit()
### End R code

then this error does not occur anymore and I get the desired random
numbers as a result. Thus, my question is why the first approach does not
work. From the documentation (?doMPI::`doMPI-package` and
?doMPI::startMPIcluster), I thought that it should be working this way.
Note that there is also the function setRngDoMPI(), but I understood the
doMPI documentation in the way that:

  - setRngDoMPI() does guarantee different RNG streams for the workers,
but the results are not reproducible
  - option "seed" (in argument "defaultopts" of startMPIcluster() as well
as in argument ".options.mpi" of foreach()) guarantees different RNG
streams for the workers and the results are reproducible.

Since I want to have reproducible results, I chose option "seed" instead
of setRngDoMPI(). But correct me if I did not understood the documentation
correctly.

Another issue I am having is when finalizing the MPI cluster: The code

### Start R code
closeCluster(cl_obj)
mpi.quit()
### End R code

does not work for me as the R code after closeCluster(cl_obj) is not
executed anymore. Thus, the MPI cluster continues to run until the
time-out of the job. Using

### Start R code
mpi.exit()
closeCluster(cl_obj)
### End R code

does not work neither as the following error is thrown:

### Start error message
Error: package ‘Rmpi’ is required by ‘doMPI’ so will not be detached
### End error message

Help for both issues (but primarily the first one) would be very appreciated.

Thanks and best regards,
Frank Weber