[R-pkg-devel] OpenMP and CRAN checks

[Rodrigo Tobar Carrizo]

Hi list,

We are having an issue with submitting a new version of the imager package to CRAN, and would like to understand how we are getting a particular NOTE. The package uses OpenMP for parallelisation of tasks. While we have a routine to set the number of threads that OpenMP should use (it internally issues a `omp_set_num_threads(threads)` call), we don't invoke it ourselves -- we let the users do it. None of our OpenMP pragmas set an explicit number of threads either. In other words, by default we use the system as-is.

In my environment (8 cores, 2 threads per core) I can see this working as expected. If I set the OMP_NUM_THREADS or OMP_THREAD_LIMIT environment variables to a given number before starting R, I can see that parallelisation stays within the given boundary. For example:

=================== examples============================
# No explicit limit, usually means "use all available cores"
$> Rscript -e 'library(foreach); library(imager); temp = foreach(1:100)%do%{as.cimg(matrix(rnorm(400^2),400,400))}; times = system.time(foreach(1:10)%do%{parmed(temp)}); print(times["user.self"]/times["elapsed"])' |& tail -n 1
15.75708

# 1 thread via OMP_THREAD_LIMIT
$> OMP_THREAD_LIMIT=1 Rscript -e 'library(foreach); library(imager); temp = foreach(1:100)%do%{as.cimg(matrix(rnorm(400^2),400,400))}; times = system.time(foreach(1:10)%do%{parmed(temp)}); print(times["user.self"]/times["elapsed"])' |& tail -n 1
0.994263

# 2 threads via OMP_NUM_THREADS
$> OMP_NUM_THREADS=2 Rscript -e 'library(foreach); library(imager); temp = foreach(1:100)%do%{as.cimg(matrix(rnorm(400^2),400,400))}; times = system.time(foreach(1:10)%do%{parmed(temp)}); print(times["user.self"]/times["elapsed"])' |& tail -n 1
1.994128
=================== examples============================

It is also our understanding (from [1], and more detailed in [2]) that the CRAN machines set OMP_THREAD_LIMIT=2 when running package checks to avoid package examples and vignettes from using more than 2 threads each, and thus be able to share the infrastructure as expected.

However, upon submission we received the following NOTE:

* checking examples ... [24s/21s] NOTE
Examples with CPU time > 2.5 times elapsed time
             user system elapsed ratio
cannyEdges 1.095  0.052   0.385 2.979

What I cannot explain is how, if OMP_THREAD_LIMIT is set, the example runs using more than 2 threads, therefor triggering such a NOTE. Isn't the OpenMP runtime supposed to limit the number of threads if this limit is set, hence limiting our parallelisation? I wouldn't single out this particular example: since OpenMP is used across the codebase it might be a question of good/bad luck which example triggers the warning. The code also doesn't spawn any child processes, all CPU utilisation stems from the main R process.

I do appreciate that we might have a future, different problem, where OMP_THREAD_LIMIT is specified but the code could still *try* to use more threads than the given limit (hence triggering a warning in some OpenMP platforms). That's the issue presented in [2], which they went and fixed -- but the limit should be obeyed by the OpenMP runtime in the first place, which is what puzzles me.

Any hints/help/guidance would be appreciated.

Regards,

Rodrigo

[1] https://rdrr.io/cran/data.table/man/openmp-utils.html
[2] https://github.com/Rdatatable/data.table/issues/3300