[R-sig-hpc] Debian/Ubuntu + threaded BLAS/ATLAS (solved)

Simon Urbanek simon.urbanek at r-project.org
Thu Mar 15 05:07:47 CET 2012


On Mar 14, 2012, at 11:52 PM, Dirk Eddelbuettel wrote:

> 
> On 14 March 2012 at 23:09, Simon Urbanek wrote:
> | I thought I'll post this here since I could not find an answer anywhere I searched: how to setup R with threaded BLAS on Debian/Ubuntu.
> | 
> | First, neither Debian nor Ubuntu come with optimized ATLAS binaries, simply because by definition they need to be optimized for a particular machine. The good news: it is easy to build:
> | 
> | apt-get source atlas
> 
> You could also go to newer Atlas sources at this point.
> 
> | cd atlas-3.*
> | # less debian/README.Debian # read the docs - it's short
> | # sudo apt-get build-dep atlas # <- you may need this if you don't have all tools installed
> | fakeroot debian/rules custom
> 
> "Yes but."  
> 
> Someone should write an atlas-helper package that deals with this and ...
> 
> | That will give you (after a long while) a bunch of libatlas*.deb files that you can easily install with sudo dpkg -i 
> | 
> | Now if you simply configure R --with-blas --with-lapack it will work, and you'll get a nice and fast BLAS, but unfortunately it will be single-threaded:
> | 
> | > m=matrix(rnorm(4e6),2e3)
> | > system.time(tcrossprod(m))
> |    user  system elapsed 
> |   0.684   0.012   0.697 
> | 
> | The problem is that all those nice libraries that are handled by update-alternatives are just the single-threaded part of ATLAS. If you want to use the parallelized part, you'll need to add the libpt* libraries by hand. This worked for me:
> 
> ... also this.  Because if you do not deal well with update-alternatives, you
> loose the plug-and-play of installing and installing BLAS providing packages.
> 

Believe me, I searched the net like a madman because I really didn't want to give this up. Half the reason I posted this here is to not forget it when I update R again ;). I really don't want to compile things manually on Debian - it defeats the point and makes upgrades a hell.

I think the prober solution would be to simply include the shared libpt* version as well under the alternatives control - more precisely package libptatlas like we used to long time ago, so you could use it instead of libatlas. You could then switch blas to it. It is actually trivial to do, that's why I was searching so hard because I thought someone must have done it (also I found one thread that was hinting at this being present in some most recent packaging but I did not find it ...). I can easily hack up the building part to build libptatlas.so, but I don't know enough about how the alternatives system registers things...


> For the gcbd package/vignette/paper I hacked together a plugin-package,
> locally built and hence optmised, but following the Atlas packaging:
> 
> edd at max:~$ dpkg -f /var/spool/atlas39/libatlas39_3.9.25-1_amd64.deb 
> dpkg -f /var/spool/atlas39/libatlas39_3.9.25-1_amd64.deb 
> Package: libatlas39
> Source: atlas39
> Version: 3.9.25-1
> Architecture: amd64
> Maintainer: Dirk Eddelbuettel <edd at debian.org>
> Installed-Size: 22656
> Depends: libc6 (>= 2.4), libgcc1 (>= 1:4.1.1), libgfortran3 (>= 4.3)
> Suggests: libblas3gf, liblapack3gf
> Provides: libatlas.so.3gf, libblas.so.3gf, liblapack.so.3gf
> Section: libs
> Priority: optional
> Homepage: http://math-atlas.sourceforge.net/
> Description: Automatically Tuned Linear Algebra Software, version 3.9.25
> ATLAS is an approach for the automatic generation and optimization of
> numerical software. Currently ATLAS supplies optimized versions for the
> complete set of linear algebra kernels known as the Basic Linear Algebra
> Subroutines (BLAS), and a subset of the linear algebra routines in the
> LAPACK library.
> .
> The libraries in this package are built without any processor extension
> instructions, and should run on all processors of this general
> architecture, albeit less than optimally.
> .
> This package is a local package meant for run-time, built in a somewhat
> non-standard way off the Atlas 3.9.25 sources.
> edd at max:~$ 
> 
> Someone with interest and a little time should get together with Sylvestre
> and see ifwe could turn this into a helper package, or an experimental
> package, or ...
> 
> | 
> | --with-blas='/usr/lib/atlas-base/libptcblas.a /usr/lib/atlas-base/libptf77blas.a /usr/lib/atlas-base/libatlas.a -lpthread -lm' --with-lapack
> | 
> | I opted for using the static version of ATLAS, but you can equally well use the shared one.
> | 
> | > m=matrix(rnorm(4e6),2e3)
> | > system.time(tcrossprod(m))
> |    user  system elapsed 
> |   1.548   0.068   0.133 
> | 
> | Note that by default it will use all cores. You can add "-t N" to the atlas configure flags in debian/rules if you want to change that (e.g. if you want to combine it with explicit parallelization).
> | 
> | I'm simply sharing this in the hope that it will be helpful - please no flames ;).
> 
> Very useful.  It all helps -- Claudia's post was very good, your's here is
> too and I am just trying to move the cart further along.
> 
> Dirk
> 
> | Cheers,
> | Simon
> | 
> | _______________________________________________
> | R-sig-hpc mailing list
> | R-sig-hpc at r-project.org
> | https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
> 
> -- 
> "Outside of a dog, a book is a man's best friend. Inside of a dog, it is too
> dark to read." -- Groucho Marx
> 
> 



More information about the R-sig-hpc mailing list