[Rd] Control multi-threading in standard matrix product

[Tomas Kalibera]

Hi Ghislain,

I think you might be comparing two versions of R with different BLAS 
implementations, one that is single threaded (is your 3.3.2 used with 
reference blas?) and one that is multi threaded (3.4.1 with openblas). 
Could you check with "perf"? E.g. run your benchmark with "perf record" 
in both cases and you should see the names of the hot BLAS functions and 
this should reveal the BLAS implementation (look for dgemm).

In Ubuntu, if you install R from the package system, whenever you run it 
it will use the BLAS currently installed via the package system. However 
if you build R from source on Ubuntu, by default, it will use the 
reference BLAS which is distributed with R. Section "Linear algebra" of 
"R Installation and Administration" has details on how to build R with 
different BLAS/LAPACK implementations.

Sadly there is no standard way to specify the number of BLAS worker 
threads. RhpcBLASctl has specific code for several existing 
implementations, but R itself does not attempt to control BLAS multi 
threading in any way. It is expected the user/system administrator will 
configure their BLAS implementation of choice to use the number of 
threads they need. A similar problem exists in other internally 
multi-threaded third-party libraries, used by packages - R cannot 
control how many threads they run.

Best
Tomas

On 08/21/2017 02:55 PM, Ghislain Durif wrote:
> Dear R Core Team,
>
> I wish to report what can be viewed as a bug or at least a strange
> behavior in R-3.4.1. I ask my question here (as recommended on
> https://www.r-project.org/bugs.html) since I am not member of the R's
> Bugzilla.
>
> When running 'R --vanilla' from the command line, the standard matrix
> product is by default based on BLAS and multi-threaded on all cores
> available on the machine, c.f. following examples:
>
> n=10000
> p=1000
> q=5000
> A = matrix(runif(n*p),nrow=n, ncol=p)
> B = matrix(runif(p*q),nrow=p, ncol=q)
> C = A %*% B # multi-threaded matrix product
>
>
> However, the default behavior to use all available cores can be an
> issue, especially on shared computing resources or when the matrix
> product is used in parallelized section of codes (for instance with
> 'mclapply' from the 'parallel' package). For instance, the default
> matrix product is single-threaded in R-3.3.2 (I ran a test on my
> machine), this new features will deeply affect the behavior of existing
> R packages that use other multi-threading solutions.
>
> Thanks to this stackoverflow question
> (https://stackoverflow.com/questions/45794290/in-r-how-to-control-multi-threading-in-blas-parallel-matrix-product),
> I now know that it is possible to control the number of BLAS threads
> thanks to the package 'RhpcBLASctl'. However, being able to control the
> number of threads should maybe not require to use an additional package.
>
> In addition, the doc 'matmult' does not mention this point, it points to
> the 'options' doc page and especially the 'matprod' section, in which
> the multi-threading is not discussed.
>
>
> Here is the results of the 'sessionInfo()' function on my machine for
> R-3.4.1:
> R version 3.4.1 (2017-06-30)
> Platform: x86_64-pc-linux-gnu (64-bit)
> Running under: Ubuntu 16.04.3 LTS
>
> Matrix products: default
> BLAS: /usr/lib/openblas-base/libblas.so.3
> LAPACK: /usr/lib/libopenblasp-r0.2.18.so
>
> locale:
>    [1] LC_CTYPE=fr_FR.utf8       LC_NUMERIC=C
>    [3] LC_TIME=fr_FR.utf8        LC_COLLATE=fr_FR.utf8
>    [5] LC_MONETARY=fr_FR.utf8    LC_MESSAGES=fr_FR.utf8
>    [7] LC_PAPER=fr_FR.utf8       LC_NAME=C
>    [9] LC_ADDRESS=C              LC_TELEPHONE=C
> [11] LC_MEASUREMENT=fr_FR.utf8 LC_IDENTIFICATION=C
>
> attached base packages:
> [1] stats     graphics  grDevices utils     datasets  methods base
>
> loaded via a namespace (and not attached):
> [1] compiler_3.4.1
>
>
>
> and for R-3.3.2:
> R version 3.3.2 (2016-10-31)
> Platform: x86_64-pc-linux-gnu (64-bit)
> Running under: Ubuntu 16.04.3 LTS
>
> locale:
>    [1] LC_CTYPE=fr_FR.utf8       LC_NUMERIC=C
>    [3] LC_TIME=fr_FR.utf8        LC_COLLATE=fr_FR.utf8
>    [5] LC_MONETARY=fr_FR.utf8    LC_MESSAGES=fr_FR.utf8
>    [7] LC_PAPER=fr_FR.utf8       LC_NAME=C
>    [9] LC_ADDRESS=C              LC_TELEPHONE=C
> [11] LC_MEASUREMENT=fr_FR.utf8 LC_IDENTIFICATION=C
>
> attached base packages:
> [1] stats     graphics  grDevices utils     datasets  methods base
>
>
> Thanks in advance,
> Best regards
> ||
>