[R-pkg-devel] Examples are too long in computation for CRAN

[Uwe Ligges]

On 13.08.2023 08:14, Ivan Krylov wrote:
> В Sat, 12 Aug 2023 22:49:01 -0700
> Michael Topper <miketopper123 using gmail.com> пишет:
> 
>> It appears that some of my examples/tests are taking too
>> long to run for CRAN's standards.
> 
> I don't think they are running too long; I think they are too parallel.
> The elapsed time is below 1s, but the "user" time (CPU time spent in
> the process) is 7 to 13 times that. This suggests that your code
> resulted in starting more threads than CRAN allows (up to 2 if you have
> to test parallellism). Are you using OpenMP? data.table? makeCluster()?
> It's simplest to always to default to a parallelism factor of 2 in
> examples an tests, because determining the right number is a hard
> problem. (What if the computer is busy doing something else? What if
> the BLAS is already parallel enough?)
> 
>> Moreover, is there any insight as to why this would happen on the
>> third update of the package rather than on the first or second?
> 
> The rule has always depended on the particular system running the
> checks (five seconds on my 12-year-old ThinkPad or on my ultraportative
> with an Intel Atom that had snails in its ancestry?). Maybe some
> dependency of your package has updated and started creating threads
> where it previously didn't.
> 


Good points, not only for examples and tests, but also for defaults.

On shared resources (such as clusters) users may not expect the 
parallelization you use and then overallocate the resources.

Example: 20 cores available to the user who runs makeCluster() for 
paallelization, but the underlying code does multihreading on 20 cores. 
Then we end up in 20*20 threads on the machine slowing down the machine 
and processes of other uers.
Hence, defaults should also not be more than 2. Simply allow the user to 
ask for more.

Best,
Uwe Ligges