[R-sig-Debian] Special BLAS and explcit parallel code

[Matthieu Stigler]

Le 16/03/2011 01:48, Dirk Eddelbuettel a écrit :
> On 16 March 2011 at 00:32, mat wrote:
> | Dirk, thanks a lot for answering to these (maybe trivial) questions!!
> | Very nice! Answers below
> |
> | Le 15. 03. 11 18:57, Dirk Eddelbuettel a écrit :
> |>  On 15 March 2011 at 18:27, Matthieu Stigler wrote:
> |>  | Hi
> |>  |
> |>  | I just read the thread on gotoBLAS, as well as the excellent vignette of
> |>  | gcbd. I still have some confusion and would like to ask very basic
> |>  | questions, hope I am not taking too much of your time.
> |>  |
> |>  | The point that retained my attention was the question of using implicit
> |>  | (multi-threaded blas) versus explicit (parallel code) optimisation. As I
> |>  | understood, the ideal would be to use a multi-threaded BLAS for simple
> |>  | code, and restrict it to use one core when parallel R code is used? Do
> |>
> |>  Right, as fine-grained parallelism (via BLAS) can clash with coarse-grained
> |>  parallelism (via, say, multicore) where you could end up with 'overbooked'
> |>  cpus.  Being able to tell the BLAS implementation to _not_ use parallel code
> |>  is a nice feature ... which Atlas for example does not have.
> |>
> |>  | you agree? I could not find the presentation of R Bivand on this... Any
> |>  | other references on this question?
> |>  |
> |>  | Secondly, I am a little bit confused about how this is done on Ubuntu.
> |>  | First of all.... I am not sure of which BLAS is actually being used... a
> |>  | dpkg -l indicated me that both liblas (-dev and 3gf) and libatlas
> |>  | (3gf-base) are installed... Doing:
> |>  |
> |>  | $ ls -n /usr/lib/R/lib/
> |>  | total 2528
> |>  | -rw-r--r-- 1 0 0 2583076 2011-02-26 04:21 libR.so
> |>  |
> |>  | I don't see which one is used actually... how can I figure out?
> |>
> |>  Use the 'ldd' command, not the 'ls' command, and run
> |>
> |>       $ ldd /usr/lib/R/lib/libR.so
> | on one machine:
> | libblas.so.3gf =>  /usr/lib/atlas/libblas.so.3gf
> |>       $ ldd /usr/lib/R/modules/lapack.so
> | liblapack.so.3gf =>  /usr/lib/atlas/liblapack.so.3gf (0x00007fcc50272000)
> |
> | so I guess this mean I am already using the atlas implementation of
> | BLAS? Good!
>
> Looks like you do.
this is what I suspected: on one other machine (very freshly installed 
Ubunu 10.10 with R from CRAN miror), seems I have the standard blas:
$ ldd /usr/lib/R/lib/libR.so
libblas.so.3gf => /usr/lib/libblas.so.3gf (0x00007f3bc03e5000)
$  ldd /usr/lib/R/modules/lapack.so
liblapack.so.3gf => /usr/lib/liblapack.so.3gf (0x00007f2d7b14b000)
libblas.so.3gf => /usr/lib/libblas.so.3gf (0x00007f2d7aecc000)

am I right this is the standard blas (as you are refering in your 
paper?). If yes, how one does do to link R to atlas (don't know how this 
happened on other machine :-))... will simply installing libatlas-dev do 
the trick?

Thanks!!
> |>  That should show you e.g. your Atlas libblas and liblapack, if you have those
> |>  packages installed.
> |>
> |>  Also note that /usr/bin/R sets more LD_LIBRARY_PATH arguments meaning so it
> |>  can potentially see more locations than the system default for ld.so
> |>  reflected in the ldd output you just saw..  That is the trick which was used
> |>  by the MKL package so that R saw those libraries but other programs did
> |>  not... (as per the wishes of Intel who gave REvo permission to distribute MKL
> |>  just for R on Ubuntu).
> |>
> |>  Makes sense?
> | definitely! (although still need to meditate on the story of locations
> | for ld.so)
> |
> | Now the question is, once I install say gotBLAS through this script you
> | recommend, is it easy to switch from one BLAS to another, as you did in
> | your investigation, or does it require some tricky methods?
>
> The idea is to just use the package managers. By having a proper .deb package
> based on Goto (using the gotoblas2-helper) you can drop Goto in and out as
> you see fit.
>
> But do take a look at the benchmark results in the gcbd vignette -- the
> differences are not that big.  Unless you really really need it, it may not
> be worth the sysadmin'ing hazzles.
>
> Dirk
>