[R-sig-hpc] Disabling implicit parallelism of multithreaded BLAS from an R package

[Dirk Eddelbuettel]

Scott, 

On 13 August 2014 at 20:08, Scott Ritchie wrote:
| Hi Claudia,
| 
| The system I tested it on is running Ubuntu 12.04, so it should
| theoretically work.

I keep forgetting you run an outdated OS version.  As I recall, the OpenBLAS
in that release is older / was lagging.  This may work better on newer ones.
Worst case, Consider rebuilding the OpenBLAS package from source on your box to create a
local backp. (Been there, done that, need to document the process ...)

Do you have a small / simple testcase?

| I'd also like a way of disabling implicit parallelism if R has been
| compiled with Goto BLAS and MKL, since I assume those users will have the
| same problem.

You may need to change their APIs...  MKL, as I recall, listen to the same
(Intel-given) variable.  GotoBLAS is a dead project which was succeeded by
OpenBLAS, so changing Goto may be impossible.

Thanks also to Claudia for that reminder. I had forgotten this Linux-only
aspect. 

Cheers, Dirk


 
| On 13 August 2014 18:21, beleites,claudia <claudia.beleites at ipht-jena.de>
| wrote:
| 
| > Hi Scott
| >
| > You may need to contact the developers of openblas about
| > openblas_set_num_threads. 2yrs ago when Simon was writing that wrapper it
| > was only implemented for Linux. All other is just had a non- working dummy.
| >
| > Hth
| >
| > Claudia
| >
| > --
| > Claudia Beleites, Chemist
| > Spectroscopy/Imaging
| > Leibniz Institute of Photonic Technology
| > Albert-Einstein-Str. 9
| > 07745 Jena
| > Germany
| >
| > email: claudia.beleites at ipht-jena.de
| > phone: +49 3641 206-133
| > fax:   +49 2641 206-399
| >
| >
| > ________________________________________
| > Von: r-sig-hpc-bounces at r-project.org [r-sig-hpc-bounces at r-project.org]"
| > im Auftrag von "Scott Ritchie [sritchie73 at gmail.com]
| > Gesendet: Mittwoch, 13. August 2014 06:40
| > An: r-sig-hpc at r-project.org
| > Betreff: [R-sig-hpc] Disabling implicit parallelism of multithreaded BLAS
| >       from an R package
| >
| > Hi all,
| >
| > I'm developing a package in which I explicitly handle parallelism, and in
| > each parallel R session, running some SVD calculations through Rcpp +
| > Armadillo.
| >
| > On one of my test machines, R 3.1.0 has been compiled against OpenBLAS, so
| > the calls to SVD are implicitly parallel.
| >
| > I can disable this by exporting the appropriate environment variable before
| > calling R (OPENBLAS_NUM_THREADS=1), but I would ideally like to abstract
| > that detail away from the user (as some of my intended target audience may
| > be biologists without a strong computational background, but nevertheless
| > operating on a cluster someone else has set up).
| >
| > I've tried a couple of things to disable the implicit parallelism without
| > any success:
| >  - Sys.setenv doesn't have any effect, even on the new R sessions spawned
| > through `foreach` with the `doMC` parallel backend (this surprises me, is
| > my mental model of parallelism in R correct?)
| >  - exporting the environment variables using `cstdlib::putenv` in the C++
| > code directly before the call to `svd`.
| >  - The solution I found through the r-sig-hpc archives:
| > https://stat.ethz.ch/pipermail/r-sig-hpc/2012-July/001432.html. Directly
| > calling openblas_set_num_threads(1); in the C++ code has no effect.
| >
| > I've also tried explicitly disabling BLAS in the hope that Armadillo will
| > call the LAPACK routines directly (although I may have done this
| > incorrectly) by doing the following:
| >  - removing the $(BLAS_LIBS) from the PKG_LIBS variable in the
| > Makevars/Makevars.win files
| >  - Setting #define ARMA_DONT_USE_BLAS and #define ARMA_DONT_USE_WRAPPER
| > before including the <RcppArmadillo.h> header file in my source code.
| >
| > Ideally I'm looking for a solution that allows me to set the maximum number
| > of threads to 1 for all the multithreaded BLAS libraries in my package to
| > prevent conflicts between the implicit and explicit parallelism. Is this
| > possible?
| >
| > Regards,
| >
| > Scott Ritchie
| >
| >         [[alternative HTML version deleted]]
| >
| > _______________________________________________
| > R-sig-hpc mailing list
| > R-sig-hpc at r-project.org
| > https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
| 
| 	[[alternative HTML version deleted]]
| 
| _______________________________________________
| R-sig-hpc mailing list
| R-sig-hpc at r-project.org
| https://stat.ethz.ch/mailman/listinfo/r-sig-hpc

-- 
http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org