[R-pkg-devel] Too many cores used in examples (not caused by data.table)

[Jan van der Laan]

You are not the only one; I did the same with some of my examples.

Would it be an option to ask for a default R-option, 'max.ncores', that 
specifies the maximum number of cores a process is allowed to use? CRAN 
could then require that that examples, tests and vignettes respect this 
option. That way there would be one uniform option to specify the 
maximum number of cores processes could use. That would also make it 
easier for system administrators to set default values for this (use the 
entire system; or use one code by default on a shared system).

Of course, we package maintainers could do this without involvement of 
R-code or CRAN. We only need to agree on a name and a default value for 
when the option is missing (0 = use all cores; 1 or 2; or ncores-1 ...).

Jan


On 24-10-2023 13:03, Greg Hunt wrote:
> In my case recently, after an hour or so’s messing about I disabled some
> tests and example executions to get rid of the offending times. I doubt
> that i am the only one to do that.
> 
> On Tue, 24 Oct 2023 at 9:38 pm, Helske, Jouni <jouni.helske using jyu.fi> wrote:
> 
>> Thanks for the help, I now tried resubmitting with
>> Sys.setenv("OMP_THREAD_LIMIT" = 2) at the top of the exchange example, but
>> I still get the same note:
>>
>> Examples with CPU time > 2.5 times elapsed time
>>            user system elapsed ratio
>> exchange 1.196   0.04   0.159 7.774
>>
>> Not sure what to try next.
>>
>> Best,
>> Jouni
>> ________________________________
>> From: Ivan Krylov <krylov.r00t using gmail.com>
>> Sent: Friday, October 20, 2023 16:54
>> To: Helske, Jouni <jouni.helske using jyu.fi>
>> Cc: r-package-devel using r-project.org <r-package-devel using r-project.org>
>> Subject: Re: [R-pkg-devel] Too many cores used in examples (not caused by
>> data.table)
>>
>> В Thu, 19 Oct 2023 05:57:54 +0000
>> "Helske, Jouni" <jouni.helske using jyu.fi> пишет:
>>
>>> But I just realised that bssm uses Armadillo via RcppArmadillo, which
>>> uses OpenMP by default for some elementwise operations. So, I wonder
>>> if that could be the culprit?
>>
>> I wasn't able to reproduce the NOTE either, despite manually setting
>> the environment variable
>> _R_CHECK_EXAMPLE_TIMING_CPU_TO_ELAPSED_THRESHOLD=2 before running R CMD
>> check, but I think I can see the code using OpenMP. Here's what I did:
>>
>> 0. Temporarily lower the system protections against capturing
>> performance traces of potentially sensitive parts:
>>
>> echo -1 | sudo tee /proc/sys/kernel/perf_event_paranoid
>>
>> (Set it back to 3 after you're done.)
>>
>> 1. Run the following command with the development version of the
>> package installed:
>>
>> env OPENBLAS_NUM_THREADS=1 \
>>   perf record --call-graph drawf,4096 \
>>   R -e 'library(bssm); system.time(replicate(100, example(exchange)))'
>>
>> OPENBLAS_NUM_THREADS=1 will prevent OpenBLAS from spawning worker
>> threads if you have it installed. (A different BLAS may need different
>> environment variables.)
>>
>> 2. Run `perf report` and browse collected call stack information.
>>
>> The call stacks are hard to navigate, but I think they are not pointing
>> towards Armadillo. At least, setting ARMA_OPENMP_THREADS=1 doesn't
>> help, but setting OMP_THREAD_LIMIT=1 does.
>>
>> --
>> Best regards,
>> Ivan
>>
>>          [[alternative HTML version deleted]]
>>
>> ______________________________________________
>> R-package-devel using r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>>
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-package-devel using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel