[R-sig-Debian] Special BLAS and explcit parallel code

[Dirk Eddelbuettel]

On 16 March 2011 at 00:32, mat wrote:
| Dirk, thanks a lot for answering to these (maybe trivial) questions!! 
| Very nice! Answers below
| 
| Le 15. 03. 11 18:57, Dirk Eddelbuettel a écrit :
| > On 15 March 2011 at 18:27, Matthieu Stigler wrote:
| > | Hi
| > |
| > | I just read the thread on gotoBLAS, as well as the excellent vignette of
| > | gcbd. I still have some confusion and would like to ask very basic
| > | questions, hope I am not taking too much of your time.
| > |
| > | The point that retained my attention was the question of using implicit
| > | (multi-threaded blas) versus explicit (parallel code) optimisation. As I
| > | understood, the ideal would be to use a multi-threaded BLAS for simple
| > | code, and restrict it to use one core when parallel R code is used? Do
| >
| > Right, as fine-grained parallelism (via BLAS) can clash with coarse-grained
| > parallelism (via, say, multicore) where you could end up with 'overbooked'
| > cpus.  Being able to tell the BLAS implementation to _not_ use parallel code
| > is a nice feature ... which Atlas for example does not have.
| >
| > | you agree? I could not find the presentation of R Bivand on this... Any
| > | other references on this question?
| > |
| > | Secondly, I am a little bit confused about how this is done on Ubuntu.
| > | First of all.... I am not sure of which BLAS is actually being used... a
| > | dpkg -l indicated me that both liblas (-dev and 3gf) and libatlas
| > | (3gf-base) are installed... Doing:
| > |
| > | $ ls -n /usr/lib/R/lib/
| > | total 2528
| > | -rw-r--r-- 1 0 0 2583076 2011-02-26 04:21 libR.so
| > |
| > | I don't see which one is used actually... how can I figure out?
| >
| > Use the 'ldd' command, not the 'ls' command, and run
| >
| >      $ ldd /usr/lib/R/lib/libR.so
| on one machine:
| libblas.so.3gf => /usr/lib/atlas/libblas.so.3gf
| >      $ ldd /usr/lib/R/modules/lapack.so
| liblapack.so.3gf => /usr/lib/atlas/liblapack.so.3gf (0x00007fcc50272000)
| 
| so I guess this mean I am already using the atlas implementation of 
| BLAS? Good!

Looks like you do.

| > That should show you e.g. your Atlas libblas and liblapack, if you have those
| > packages installed.
| >
| > Also note that /usr/bin/R sets more LD_LIBRARY_PATH arguments meaning so it
| > can potentially see more locations than the system default for ld.so
| > reflected in the ldd output you just saw..  That is the trick which was used
| > by the MKL package so that R saw those libraries but other programs did
| > not... (as per the wishes of Intel who gave REvo permission to distribute MKL
| > just for R on Ubuntu).
| >
| > Makes sense?
| definitely! (although still need to meditate on the story of locations 
| for ld.so)
| 
| Now the question is, once I install say gotBLAS through this script you 
| recommend, is it easy to switch from one BLAS to another, as you did in 
| your investigation, or does it require some tricky methods?

The idea is to just use the package managers. By having a proper .deb package
based on Goto (using the gotoblas2-helper) you can drop Goto in and out as
you see fit.

But do take a look at the benchmark results in the gcbd vignette -- the
differences are not that big.  Unless you really really need it, it may not
be worth the sysadmin'ing hazzles.

Dirk

-- 
Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com