[R-sig-hpc] Disabling implicit parallelism of multithreaded BLAS from an R package
Dirk Eddelbuettel
edd at debian.org
Wed Aug 13 13:33:47 CEST 2014
Scott,
On 13 August 2014 at 20:08, Scott Ritchie wrote:
| Hi Claudia,
|
| The system I tested it on is running Ubuntu 12.04, so it should
| theoretically work.
I keep forgetting you run an outdated OS version. As I recall, the OpenBLAS
in that release is older / was lagging. This may work better on newer ones.
Worst case, Consider rebuilding the OpenBLAS package from source on your box to create a
local backp. (Been there, done that, need to document the process ...)
Do you have a small / simple testcase?
| I'd also like a way of disabling implicit parallelism if R has been
| compiled with Goto BLAS and MKL, since I assume those users will have the
| same problem.
You may need to change their APIs... MKL, as I recall, listen to the same
(Intel-given) variable. GotoBLAS is a dead project which was succeeded by
OpenBLAS, so changing Goto may be impossible.
Thanks also to Claudia for that reminder. I had forgotten this Linux-only
aspect.
Cheers, Dirk
| On 13 August 2014 18:21, beleites,claudia <claudia.beleites at ipht-jena.de>
| wrote:
|
| > Hi Scott
| >
| > You may need to contact the developers of openblas about
| > openblas_set_num_threads. 2yrs ago when Simon was writing that wrapper it
| > was only implemented for Linux. All other is just had a non- working dummy.
| >
| > Hth
| >
| > Claudia
| >
| > --
| > Claudia Beleites, Chemist
| > Spectroscopy/Imaging
| > Leibniz Institute of Photonic Technology
| > Albert-Einstein-Str. 9
| > 07745 Jena
| > Germany
| >
| > email: claudia.beleites at ipht-jena.de
| > phone: +49 3641 206-133
| > fax: +49 2641 206-399
| >
| >
| > ________________________________________
| > Von: r-sig-hpc-bounces at r-project.org [r-sig-hpc-bounces at r-project.org]"
| > im Auftrag von "Scott Ritchie [sritchie73 at gmail.com]
| > Gesendet: Mittwoch, 13. August 2014 06:40
| > An: r-sig-hpc at r-project.org
| > Betreff: [R-sig-hpc] Disabling implicit parallelism of multithreaded BLAS
| > from an R package
| >
| > Hi all,
| >
| > I'm developing a package in which I explicitly handle parallelism, and in
| > each parallel R session, running some SVD calculations through Rcpp +
| > Armadillo.
| >
| > On one of my test machines, R 3.1.0 has been compiled against OpenBLAS, so
| > the calls to SVD are implicitly parallel.
| >
| > I can disable this by exporting the appropriate environment variable before
| > calling R (OPENBLAS_NUM_THREADS=1), but I would ideally like to abstract
| > that detail away from the user (as some of my intended target audience may
| > be biologists without a strong computational background, but nevertheless
| > operating on a cluster someone else has set up).
| >
| > I've tried a couple of things to disable the implicit parallelism without
| > any success:
| > - Sys.setenv doesn't have any effect, even on the new R sessions spawned
| > through `foreach` with the `doMC` parallel backend (this surprises me, is
| > my mental model of parallelism in R correct?)
| > - exporting the environment variables using `cstdlib::putenv` in the C++
| > code directly before the call to `svd`.
| > - The solution I found through the r-sig-hpc archives:
| > https://stat.ethz.ch/pipermail/r-sig-hpc/2012-July/001432.html. Directly
| > calling openblas_set_num_threads(1); in the C++ code has no effect.
| >
| > I've also tried explicitly disabling BLAS in the hope that Armadillo will
| > call the LAPACK routines directly (although I may have done this
| > incorrectly) by doing the following:
| > - removing the $(BLAS_LIBS) from the PKG_LIBS variable in the
| > Makevars/Makevars.win files
| > - Setting #define ARMA_DONT_USE_BLAS and #define ARMA_DONT_USE_WRAPPER
| > before including the <RcppArmadillo.h> header file in my source code.
| >
| > Ideally I'm looking for a solution that allows me to set the maximum number
| > of threads to 1 for all the multithreaded BLAS libraries in my package to
| > prevent conflicts between the implicit and explicit parallelism. Is this
| > possible?
| >
| > Regards,
| >
| > Scott Ritchie
| >
| > [[alternative HTML version deleted]]
| >
| > _______________________________________________
| > R-sig-hpc mailing list
| > R-sig-hpc at r-project.org
| > https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
|
| [[alternative HTML version deleted]]
|
| _______________________________________________
| R-sig-hpc mailing list
| R-sig-hpc at r-project.org
| https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
--
http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
More information about the R-sig-hpc
mailing list