[R-sig-hpc] Debian/Ubuntu + threaded BLAS/ATLAS (solved)

[Simon Urbanek]

I thought I'll post this here since I could not find an answer anywhere I searched: how to setup R with threaded BLAS on Debian/Ubuntu.

First, neither Debian nor Ubuntu come with optimized ATLAS binaries, simply because by definition they need to be optimized for a particular machine. The good news: it is easy to build:

apt-get source atlas
cd atlas-3.*
# less debian/README.Debian # read the docs - it's short
# sudo apt-get build-dep atlas # <- you may need this if you don't have all tools installed
fakeroot debian/rules custom

That will give you (after a long while) a bunch of libatlas*.deb files that you can easily install with sudo dpkg -i 

Now if you simply configure R --with-blas --with-lapack it will work, and you'll get a nice and fast BLAS, but unfortunately it will be single-threaded:

> m=matrix(rnorm(4e6),2e3)
> system.time(tcrossprod(m))
   user  system elapsed 
  0.684   0.012   0.697 

The problem is that all those nice libraries that are handled by update-alternatives are just the single-threaded part of ATLAS. If you want to use the parallelized part, you'll need to add the libpt* libraries by hand. This worked for me:

--with-blas='/usr/lib/atlas-base/libptcblas.a /usr/lib/atlas-base/libptf77blas.a /usr/lib/atlas-base/libatlas.a -lpthread -lm' --with-lapack

I opted for using the static version of ATLAS, but you can equally well use the shared one.

> m=matrix(rnorm(4e6),2e3)
> system.time(tcrossprod(m))
   user  system elapsed 
  1.548   0.068   0.133 

Note that by default it will use all cores. You can add "-t N" to the atlas configure flags in debian/rules if you want to change that (e.g. if you want to combine it with explicit parallelization).

I'm simply sharing this in the hope that it will be helpful - please no flames ;).

Cheers,
Simon