[R-sig-Debian] R, OpenBLAS and OMP_NUM_THREADS

[Gordon Ball]

On 04/08/16 06:39, Ei-ji Nakama wrote:
> Hi,
> I was be half asleep in the heat ...
> 
> 2016-08-04 0:04 GMT+09:00 Dirk Eddelbuettel <edd at debian.org>:
>>
>> On 3 August 2016 at 16:45, Gordon Ball wrote:
>> | On 02/08/16 03:10, Ei-ji Nakama wrote:
>> | > Hi,
>> | >
>> | > Create /etc/profile.d/openblas.sh.
>> | > Write the following during in this file.
>> | > OPENBLAS_NUM_THREADS = 1
>> | > export OPENBLAS_NUM_THREADS
>> | >
>> | > OPENBLAS_NUM_THREADS environment variable does not affect the OMP_NUM_THREADS.
>>
>> Thumbs up to using /etc/profile.d/ -- very nice hook.
>>
>> | Unfortunately neither this nor anything else I've tried today appears to
>> | set the variable for sessions started through RStudio server (which may
>> | or may not be an appropriate issue here).
>> |
>> | It appears that the rstudio server spawns sessions with a new minimal
>> | environment (rstudio::core::system::launchChildProcess) and no option to
>> | inject or inherit variables. (Various methods of controlling the session
>> | environment are documented in the pro/paid version manual [1], but are
>> | not implemented in the public codebase - enterprise features, presumably).
>> |
>> | Hence approaches like setting the environment in the
>> | rstudio-server.service systemd unit, or creating a /etc/pam.d/rstudio
>> | service profile including pam_env.so (to load the setting from
>> | /etc/environment) don't work. It would probably be possible to work
>> | round this by creating a small binary wrapper for the rsession binary
>> | which sets the environment, but it would make a mess of the packaging.
>> |
>> | So I've gone with an `/etc/R/Rprofile.site` containing
>> |
>> | local({
>> |     if (require("RhpcBLASctl", quietly=TRUE)) blas_set_num_threads(1)
>> | })
>> |
>> | which does mean people get this library loaded in all their sessions but
>> | that doesn't seem to cause any particular trouble (yet).
> 
> In this case, hook of the / etc / R / Rprofile.site would be best
> 
> local({
>   if(require("RhpcBLASctl", quietly=TRUE)){
>   if(Sys.getenv("OPENBLAS_NUM_THREADS")=="" &
>      Sys.getenv("OMP_NUM_THREADS")=="")
>         blas_set_num_threads(1)
>   }
> })
> 
> $ OMP_NUM_THREADS=4 R -q -e 'library(RhpcBLASctl);blas_get_num_procs()'
>> library(RhpcBLASctl);blas_get_num_procs()
> [1] 4
> 
> maybe confusion would be minimal.

Yes, that looks a bit better, allowing it to be overridden by
environment variables.

The other thing that occurred to me was whether this setting would be
inherited by child R processes (otherwise you'd get a massive thread
explosion when running something intentionally parallel), but it looks
like at least the builtin `parallel` library does retain the BLAS
settings of the parent (presumably since it uses `fork()`).

It is maybe still worth setting `Sys.setenv(OPENBLAS_NUM_THREADS=1)`
along with the `blas_set_num_threads(1)` call in case there are any
multicore libraries for which this doesn't apply.


(Also, it hadn't occurred to me earlier but I realise you're the author
of RhpcBLASctl - appreciate it)


Gordon

> 
>> I was just working on something that needed environment variables (for
>> automating tests to a database backend) and populating
>>
>>    /etc/R/Renviron.site
>>
>> worked for me. Otherwise explicit code in Rprofile.site is of course good, as
>> is conditioning.  I do that too for some use cases.
> 
> I often forget to have set it...orz
> 
>> Dirk
>>
>> --
>> http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
> 
> 
>