[Rd] R with ATLAS avoids Linux cpu affinity

[Dirk Eddelbuettel]

Chris,

On 6 December 2010 at 15:43, Chris Jewell wrote:
| Hi all,
| 
| I have a problem with cpu affinity in my R-2.11.1 installation compiled against ATLAS running on a Linux (Ubuntu 10.04) cluster under GridEngine.  I wish to use Grid Engine's core binding feature to bind user processes into the number of cores they request on the cluster, thus preventing badly behaved multi-threaded libraries from consuming more cores than requested.  An example of this is R compiled against multithreaded ATLAS, which needs to be bound into a single core if a user submits a 1 core job.  Grid Engine achieves this through the sched_setaffinity system call under Linux 2.6.  For most applications (including if I write a test C program that uses ATLAS BLAS), this works well, and prevents threads from 'leaking' outside the cpu set they are assigned.  However, R appears to be able to avoid the core binding.  This is *very* strange as I was under the impression that any child processes or threads inherit the cpu affinity of the parent.
| 
| Does anyone have experience of this and could offer a comment or solution? 

Atlas will _always_ use all the cores 'compiled in'.  

Ubuntu's current package just uses one (as it is not a multithreaded
build). This will change with future package as per the Debian / Ubuntu
package maintainer.  If you built Atlas locally, you may be use all cores
(depending on how you built).

There is simply no way to 'scale up or down' dynamically with Atlas -- but
both MKL and GotoBLAS2 can do this.  See my package / paper on BLAS and GPU
benchmarking (cf http://dirk.eddelbuettel.com/blog/code/gcbd/ for two posts
and links) for details.

AFAICT R should not alter CPU affinity.  So if I were you I'd swap the BLAS
implementation and try again.  As you are on Ubuntu, you can try the MKL that
comes with the (now a littler older) Revolution R in Ubuntu 9.10; otherwise I
can highly recommend the GotoBLAS2 helper package listed in my paper for a
local GotoBLAS2 built.   The script may be out of sync with the fairly recent
license change of GotoBLAS2 (to the more liberal BSD license permitting
redistribution).  With some luck we will GotoBLAS2 deb packages in future
Debian and Ubuntu releases.

Hth,  Dirk

-- 
Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com