[R-sig-hpc] R parallel package <-> stopCluster method needed for mpi cluster

[Paul Johnson]

hi, everybody in r-sig-hpc land.  hopefully this year is bringing you
good news.

I'm trying to digest the fact that R now has a parallel package.  I'm
pretty sure I've run into something that is on the TODO list. Briefly,
the stopCluster method from R parallel doesn't lead back to the
stopCluster methods from snow to shut down  an MPI cluster.

Now, the long version:

We have an OpenMPI based cluster, I've been working up examples that
use Rmpi, snow, snowft, and now R parallel.  Here's my parallel
effort.

http://winstat.quant.ku.edu/svn/hpcexample/trunk/Ex65-R-parallel

Its just a submission script and an example cluster usage.

I'm stuck on (at least) one detail.  If you examine

http://winstat.quant.ku.edu/svn/hpcexample/trunk/Ex65-R-parallel/parallel-hello.R

you see the cluster is created like so

library(parallel)
## Causes the snow package's MPI support to be invoked,
cl <- makeCluster(10, type="MPI")

That actually seems to be fine. the parallel makeCluster method refers
the work to snow::makeCluster. Works fine.

After that, how to close the thing down?   When I run

stopCluster(cl)

processing crashes, and the error output has this:

Loading required package: Rmpi
Error in UseMethod("stopCluster") :
  no applicable method for 'stopCluster' applied to an object of class
"c('spawnedMPIcluster', 'MPIcluster', 'cluster')"
Calls: <Anonymous>
Execution halted
--------------------------------------------------------------------------
orterun has exited due to process rank 0 with PID 28684 on
node compute-2-11.local exiting without calling "finalize". This may
have caused other processes in the application to be
terminated by signals sent by orterun (as reported here).
--------------------------------------------------------------------------



I *think* I need to call the stopCluster method from snow.  It seems
like this should have worked:

snow::stopCluster(cl)

or this:

snow::stopCluster.MPIcluster(cl)

but it gives the same error.

But if I load the snow package, all is good.

library(snow)
stopCluster(cl)
mpi.quit()

The error log has a warning, but otherwise it appears OK.

Attaching package: 'snow'

The following object(s) are masked from 'package:parallel':

    clusterApply, clusterApplyLB, clusterCall, clusterEvalQ,
    clusterExport, clusterMap, clusterSplit, makeCluster, parApply,
    parCapply, parLapply, parRapply, parSapply, splitIndices,
    stopCluster

What do you think.  This is a known issue?  Should I bother somebody
about it? I mean somebody besides  you, of course.

-- 
Paul E. Johnson
Professor, Political Science
1541 Lilac Lane, Room 504
University of Kansas