[R-pkg-devel] Unexpected multi-core CPU usage in package tests

[Jon Clayden]

On Wed, 28 Feb 2024 at 17:36, Ivan Krylov <ikrylov using disroot.org> wrote:
>
> В Tue, 27 Feb 2024 11:14:19 +0000
> Jon Clayden <jon.clayden using gmail.com> пишет:
>
> > My testing route is to install the packages within the
> > 'rocker/r-devel' Docker container, which is Debian-based, then use
> > 'time' to evaluate CPU usage. Note that, even though 'RNifti' does not
> > use OpenMP, setting OMP_NUM_THREADS changes its CPU usage
>
> I think that's because rocker/r-devel uses parallel OpenBLAS:
>
> $ podman run --rm -it docker.io/rocker/r-devel \
>  R -q -s -e 'sessionInfo()' | grep -A1 BLAS
> BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
> LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.24.so;  LAPACK version 3.11.0
>
> The incoming CRAN check machine either sets the BLAS parallellism to 1
> or uses a non-parallel BLAS. With rocker/r-devel, you can run R with
> the environment variable OPENBLAS_NUM_THREADS set to 1. It's been
> effective in the past to run R -d gdb and set a breakpoint on
> pthread_create before launching the test. (In theory, it may be
> required to set a breakpoint on every system call that may be used to
> create threads, including various variations of clone(), subject to
> variations between operating systems, but pthread_create has been
> enough for me so far.)
>
> With OPENBLAS_NUM_THREADS=1, I'm only seeing OpenMP threads created by
> the mmand package during tests for your package tractor.base, and the
> latest commit (that temporary disables testing of mmand) doesn't hit
> the breakpoint or raise any NOTEs at all.

Very many thanks for testing this, Ivan. I did come across the
breakpoint suggestion when searching on this, and tried it, but I ran
into some problems with gdb within the container. Your experiments
confirm that my yardstick was just misleading me; I will add support
for explicitly setting the thread count within mmand, which should
give me a longer-term solution.

Thanks again,
Jon