[R-sig-hpc] Disabling implicit parallelism of multithreaded BLAS from an R package

[Scott Ritchie]

Hi again Dirk,

If its just an issue with the particular setup I'm using, I'm happy to
stick `export OPENBLAS_NUM_THREADS=1` in my bash profile.

Good to know about Goto and MKL. I'd just seen them mentioned in the gcbd
vignette you suggested I read, so assumed they would have the same issue. I
had originally hope to just set the relevant environment variables for all
of them (even if they weren't relevant to the user).

I'll construct a minimal example to use as a test case and send it to the
mailing list.

Cheers,

Scott


On 13 August 2014 21:33, Dirk Eddelbuettel <edd at debian.org> wrote:

>
> Scott,
>
> On 13 August 2014 at 20:08, Scott Ritchie wrote:
> | Hi Claudia,
> |
> | The system I tested it on is running Ubuntu 12.04, so it should
> | theoretically work.
>
> I keep forgetting you run an outdated OS version.  As I recall, the
> OpenBLAS
> in that release is older / was lagging.  This may work better on newer
> ones.
> Worst case, Consider rebuilding the OpenBLAS package from source on your
> box to create a
> local backp. (Been there, done that, need to document the process ...)
>
> Do you have a small / simple testcase?
>
> | I'd also like a way of disabling implicit parallelism if R has been
> | compiled with Goto BLAS and MKL, since I assume those users will have the
> | same problem.
>
> You may need to change their APIs...  MKL, as I recall, listen to the same
> (Intel-given) variable.  GotoBLAS is a dead project which was succeeded by
> OpenBLAS, so changing Goto may be impossible.
>
> Thanks also to Claudia for that reminder. I had forgotten this Linux-only
> aspect.
>
> Cheers, Dirk
>
>
>
> | On 13 August 2014 18:21, beleites,claudia <claudia.beleites at ipht-jena.de
> >
> | wrote:
> |
> | > Hi Scott
> | >
> | > You may need to contact the developers of openblas about
> | > openblas_set_num_threads. 2yrs ago when Simon was writing that wrapper
> it
> | > was only implemented for Linux. All other is just had a non- working
> dummy.
> | >
> | > Hth
> | >
> | > Claudia
> | >
> | > --
> | > Claudia Beleites, Chemist
> | > Spectroscopy/Imaging
> | > Leibniz Institute of Photonic Technology
> | > Albert-Einstein-Str. 9
> | > 07745 Jena
> | > Germany
> | >
> | > email: claudia.beleites at ipht-jena.de
> | > phone: +49 3641 206-133
> | > fax:   +49 2641 206-399
> | >
> | >
> | > ________________________________________
> | > Von: r-sig-hpc-bounces at r-project.org [r-sig-hpc-bounces at r-project.org
> ]"
> | > im Auftrag von "Scott Ritchie [sritchie73 at gmail.com]
> | > Gesendet: Mittwoch, 13. August 2014 06:40
> | > An: r-sig-hpc at r-project.org
> | > Betreff: [R-sig-hpc] Disabling implicit parallelism of multithreaded
> BLAS
> | >       from an R package
> | >
> | > Hi all,
> | >
> | > I'm developing a package in which I explicitly handle parallelism, and
> in
> | > each parallel R session, running some SVD calculations through Rcpp +
> | > Armadillo.
> | >
> | > On one of my test machines, R 3.1.0 has been compiled against
> OpenBLAS, so
> | > the calls to SVD are implicitly parallel.
> | >
> | > I can disable this by exporting the appropriate environment variable
> before
> | > calling R (OPENBLAS_NUM_THREADS=1), but I would ideally like to
> abstract
> | > that detail away from the user (as some of my intended target audience
> may
> | > be biologists without a strong computational background, but
> nevertheless
> | > operating on a cluster someone else has set up).
> | >
> | > I've tried a couple of things to disable the implicit parallelism
> without
> | > any success:
> | >  - Sys.setenv doesn't have any effect, even on the new R sessions
> spawned
> | > through `foreach` with the `doMC` parallel backend (this surprises me,
> is
> | > my mental model of parallelism in R correct?)
> | >  - exporting the environment variables using `cstdlib::putenv` in the
> C++
> | > code directly before the call to `svd`.
> | >  - The solution I found through the r-sig-hpc archives:
> | > https://stat.ethz.ch/pipermail/r-sig-hpc/2012-July/001432.html.
> Directly
> | > calling openblas_set_num_threads(1); in the C++ code has no effect.
> | >
> | > I've also tried explicitly disabling BLAS in the hope that Armadillo
> will
> | > call the LAPACK routines directly (although I may have done this
> | > incorrectly) by doing the following:
> | >  - removing the $(BLAS_LIBS) from the PKG_LIBS variable in the
> | > Makevars/Makevars.win files
> | >  - Setting #define ARMA_DONT_USE_BLAS and #define ARMA_DONT_USE_WRAPPER
> | > before including the <RcppArmadillo.h> header file in my source code.
> | >
> | > Ideally I'm looking for a solution that allows me to set the maximum
> number
> | > of threads to 1 for all the multithreaded BLAS libraries in my package
> to
> | > prevent conflicts between the implicit and explicit parallelism. Is
> this
> | > possible?
> | >
> | > Regards,
> | >
> | > Scott Ritchie
> | >
> | >         [[alternative HTML version deleted]]
> | >
> | > _______________________________________________
> | > R-sig-hpc mailing list
> | > R-sig-hpc at r-project.org
> | > https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
> |
> |       [[alternative HTML version deleted]]
> |
> | _______________________________________________
> | R-sig-hpc mailing list
> | R-sig-hpc at r-project.org
> | https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
>
> --
> http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
>

	[[alternative HTML version deleted]]