[R-pkg-devel] Setting OpenMP threads (globally) for an R package

[Ben Bolker]

    There is some code for managing OpenMP threading in the glmmTMB 
package, if you search its github repo for "openmp" ...

On 3/17/22 7:23 PM, Simon Urbanek wrote:
> Evan,
> 
> honestly, I think your request may be a red herring. Threads typically don't cause memory explosion, because OpenMP threads don't allocate new memory, but uncontrolled forking does. There are many things that are not allowed inside mclapply so that's where I would look. It may be better to look at the root cause first, but for that we would need more details on what you are doing.
> 
> Cheers,
> Simon
> 
> 
>> On Mar 18, 2022, at 2:51 AM, Evan Biederstedt <evan.biederstedt using gmail.com> wrote:
>>
>> Hi R-package-devel
>>
>> I'm developing an R package which uses `parallel::mclapply` and several
>> other library dependencies which possibly rely upon OpenMP. Unfortunately,
>> some functions explode the amount of memory used.
>>
>> I've noticed that if I set `export OMP_NUM_THREADS=1` before starting R,
>> the memory is far more manageable.
>>
>> My question is, if there a way for me to achieve this behavior within the R
>> package itself?
>>
>> My initial try was to use `R/zzz.R` and an `.onLoad()` function to load
>> these global variables upon loading the library.
>>
>> ```
>> .onLoad <- function(libname, pkgname){
>>   Sys.setenv(OMP_NUM_THREADS=1)
>> }
>> ```
>>
>> But this doesn't work. The memory still explodes. In fact, I'm worried that
>> this cannot be done within an R package itself, as R has already started,
>> e.g.  https://stackoverflow.com/a/27320691/5269850
>>
>> Is there a recommended approach for this problem when writing R packages?
>>
>> Package here: https://github.com/kharchenkolab/numbat
>>
>> Related question on SO:
>> https://stackoverflow.com/questions/71507979/set-openmp-threads-for-all-dependencies-in-r-package
>>
>> Any help appreciated. Thanks, Evan
>>
>> 	[[alternative HTML version deleted]]
>>
>> ______________________________________________
>> R-package-devel using r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>>
> 
> ______________________________________________
> R-package-devel using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel

-- 
Dr. Benjamin Bolker
Professor, Mathematics & Statistics and Biology, McMaster University
Director, School of Computational Science and Engineering
(Acting) Graduate chair, Mathematics & Statistics