[R-pkg-devel] OpenMP and CRAN checks

[Dirk Eddelbuettel]

Hi Rodrigo,

This came up recently again on social media where I illustrated how the
tiledb package deals with it. So a quick recap:

First off, let's make the goals clear.

We want to _simultaneously_
 - abide by CRAN Policy rules and cap ourselves to two cores there
 - do not impose any limits on our users: ALL cores ALL the time

The solution we implemented a while is to use a function _that is an opt-in_
which looks at the standard OpenMP variable as well as at R's own Ncpus:

    limitTileDBCores <- function(ncores, verbose=FALSE) {
      if (missing(ncores)) {
        ## start with a simple fallback: 'Ncpus' (if set) or else 2
        ncores <- getOption("Ncpus", 2L)
        ## also consider OMP_THREAD_LIMIT (cf Writing R Extensions), gets NA if envvar unset
        ompcores <- as.integer(Sys.getenv("OMP_THREAD_LIMIT"))
        ## and then keep the smaller
        ncores <- min(na.omit(c(ncores, ompcores)))
      }
      stopifnot(`The 'ncores' argument must be numeric or character` = is.numeric(ncores) || is.character(ncores))
      ## for brevity omitted here how ncores propagates to TileDB library -- creates `cfg`
      if (verbose) message("Limiting TileDB to ",ncores," cores. See ?limitTileDBCores.")
      invisible(cfg)
    }

The key is that we reflect the smaller of Ncpus and OMP_THREAD_LIMIT along
with a fall-back of two in case nothing is set.

That function is then called (and feeds into the library config) at the
beginning of each
 - help file example
 - unit test file
 - vignette

As example (from a help file) is

    \dontshow{ctx <- tiledb_ctx(limitTileDBCores())}

(where ctx a context object controlling, inter alia, the thread pool).

By throttling it anywhere CRAN executes code, and using the prescribed
maximum of two core, we satisfy goal one of not getting thrown off CRAN. By
making it an _explicit_ opt-in we satisfy our goal of never slowing down our
users who (presumably) do not opt in. And those who have, say, Ncpus set (as
I do to spread R's own package installations over all my cores) get the
maximum performance for examples, tests, and vignettes too as they opted in.

"Works for us" as they say.

Hope this helps,  Dirk

-- 
dirk.eddelbuettel.com | @eddelbuettel | edd using debian.org