[R-sig-hpc] cores vs. slots vs. slaves
Dirk Eddelbuettel
edd at debian.org
Fri Oct 7 00:37:29 CEST 2011
On 6 October 2011 at 15:07, Jeff Howbert wrote:
| We are seeing some odd behavior in the allocation of R tasks to MPI slots
| under OpenMPI. All of the following takes place on a parallel compute
| system we built and run on Amazon Web Services. Hardware is two AWS cluster
| compute nodes (cc1.4xlarge), each with 8 cores. We designate one node as
| the "head" node, and the other as the "client" node.
|
|
|
| Scenario A: We request, via the hostfile, 8 slots on the head node and 8
| slots on the client node. We then spawn an R cluster using RMPI, with 1
| master and 15 slaves. The master (rank 0) and 8 slaves (rank 1-8), i.e. a
| total of 9 R tasks, appear on the head node, and only 7 slaves (rank 9-15)
| appear on the client node.
|
|
|
| Scenario B: We request, via the hostfile, 7 slots on the head node and 8
| slots on the client node. We then spawn an R cluster using RMPI, with 1
| master and 15 slaves (same as Scenario A). The master (rank 0) and 7 slaves
| (rank 1-7), i.e. a total of 8 R tasks, appear on the head node, and 8 slaves
| (rank 8-15) appear on the client node.
|
|
|
| We have run several parallel jobs under both scenarios, and Scenario A is
| always considerably slower (c. 40%) than Scenario B, suggesting that in
| Scenario A there really are more slots (9?) than available cores (8) on the
| head node.
|
|
|
| Does anyone have insight into the source of this behavior? Is it OpenMPI,
| RMPI, or some interaction between the two? Thanks.
I'd bet it is the Open MPI layer. Rmpi, after all, just requests into MPI
via the MPI. Can you cook up a simple C client program which you can test
the same way?
Dirk
| Jeff Howbert
|
| Brian Pratt
|
| Insilicos LLC
|
|
|
|
| [[alternative HTML version deleted]]
|
| _______________________________________________
| R-sig-hpc mailing list
| R-sig-hpc at r-project.org
| https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
--
New Rcpp master class for R and C++ integration is scheduled for
San Francisco (Oct 8), more details / reg.info available at
http://www.revolutionanalytics.com/products/training/public/rcpp-master-class.php
More information about the R-sig-hpc
mailing list