[R-pkg-devel] Setting OpenMP threads (globally) for an R package

[Simon Urbanek]

Evan,

honestly, I think your request may be a red herring. Threads typically don't cause memory explosion, because OpenMP threads don't allocate new memory, but uncontrolled forking does. There are many things that are not allowed inside mclapply so that's where I would look. It may be better to look at the root cause first, but for that we would need more details on what you are doing.

Cheers,
Simon


> On Mar 18, 2022, at 2:51 AM, Evan Biederstedt <evan.biederstedt using gmail.com> wrote:
> 
> Hi R-package-devel
> 
> I'm developing an R package which uses `parallel::mclapply` and several
> other library dependencies which possibly rely upon OpenMP. Unfortunately,
> some functions explode the amount of memory used.
> 
> I've noticed that if I set `export OMP_NUM_THREADS=1` before starting R,
> the memory is far more manageable.
> 
> My question is, if there a way for me to achieve this behavior within the R
> package itself?
> 
> My initial try was to use `R/zzz.R` and an `.onLoad()` function to load
> these global variables upon loading the library.
> 
> ```
> .onLoad <- function(libname, pkgname){
>  Sys.setenv(OMP_NUM_THREADS=1)
> }
> ```
> 
> But this doesn't work. The memory still explodes. In fact, I'm worried that
> this cannot be done within an R package itself, as R has already started,
> e.g.  https://stackoverflow.com/a/27320691/5269850
> 
> Is there a recommended approach for this problem when writing R packages?
> 
> Package here: https://github.com/kharchenkolab/numbat
> 
> Related question on SO:
> https://stackoverflow.com/questions/71507979/set-openmp-threads-for-all-dependencies-in-r-package
> 
> Any help appreciated. Thanks, Evan
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-package-devel using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>