[R-sig-hpc] digesting "parallel" package in R-2.14. how about you?

[Paul Johnson]

Have you been testing out R's new parallel package? I'm getting
started and wonder about your experience.  The help page is very brief
because most of the functions are not exported, so I'm finding it a
bit difficult to use.

Our cluster runs RMPI over OpenMPI.  Initially, I thought that would
be a bad problem because parallel is intended for "sock" and "fork"
clusters.  That seemed bad,  but parallel's makeCluster help page says
it will call snow to make the cluster.  So I think that's OK for me,
if I were using a snow cluster.  The parallel vignette also says its
functions will work with clusters created by snow, so that's hopeful.

In the past, I've blundered into conflicts between OpenMPI and the
other frameworks that are drawn in by some packages.  OpenMPI doesn't
play well with libraries that want to spawn thread, for example.
OpenMPI is not compatible with OpenMP and the multicore package, but
the parallel package uses those things. I worry they will slide in
where I don't want them.

Here's an example of an error I got today, I don't know yet what
causes it, but I'm suspicious :)

--------------------------------------------------------------------------
An MPI process has executed an operation involving a call to the
"fork()" system call to create a child process.  Open MPI is currently
operating in a condition that could result in memory corruption or
other system errors; your MPI job may hang, crash, or produce silent
data corruption.  The use of fork() (or system() or other calls that
create child processes) is strongly discouraged.

The process that invoked fork was:

  Local host:          compute-2-11.local (PID 8290)
  MPI_COMM_WORLD rank: 0

If you are *absolutely sure* that your application will successfully
and correctly survive a call to fork(), you may disable this warning
by setting the mpi_warn_on_fork MCA parameter to 0.
--------------------------------------------------------------------------



-- 
Paul E. Johnson
Professor, Political Science
1541 Lilac Lane, Room 504
University of Kansas