[R-pkg-devel] Checking the number of cores used

[Shu Fai Cheung]

Thanks for the suggestion. I will use rhub and a virtual machine to check again.

I read some previous posts and I thought I need to check the times to
see if there is any unintended usage of parallel processing, e.g., CPU
time > 2 x elapsed time. May I ask a few questions on this part?

This is from the "-Ex.Rout" file:

> base::cat("Time elapsed: ", proc.time() - base::get("ptime", pos = 'CheckExEnv'),"\n")
Time elapsed:  14.94 0.27 15.26 NA NA

First, is this something I can ignore, unless the CPU time is
substantially larger than elapsed time? This is the total time but the
parallel process may be triggered in only one of the many examples.

Second, are the files "-Ex.timings", showing per-example timing, only
available on some platforms? I could get it locally in Windows, and
can find it in Winbuilder output. However, I could not find them in
the rhub platforms I tried, not even in the Windows platform. I
suppose adding "--as-cran" will also add "--timings"?

Third, the "testthat.Rout" file only shows the total time:

> proc.time()
   user  system elapsed
 13.530   0.291  29.492

I believe the user time is not useful as we can use two processes in
testthat. How can we detect the use of more than two cores in the
tests?

Last, how can we detect the use of more than two cores in vignettes? I
checked and couldn't find similar timing information on vignettes.

Sorry for asking so many questions. I would like to have a reliable
way to detect hidden use of parallel processing such that I can
prevent the problem from happening. I have some ideas which package I
imported is causing the problem but I have used it before without
problem. Therefore, I would like to see if I overlooked anything.

Regards,
Shu Fai

On Tue, Sep 19, 2023 at 5:02 PM Uwe Ligges
<ligges using statistik.tu-dortmund.de> wrote:
>
>
>
> On 18.09.2023 16:10, Shu Fai Cheung wrote:
> > Hi All,
> >
> > I know we should not use more than 2 cores in tests, vignettes, etc. I
> > encountered and solved this issue before. However, I still committed
> > this mistake in a new package and would like find out where the cause
> > is.
> >
> > I have a package that already has parallel processing disabled by
> > default and I did not enable parallel processing in the examples and
> > tests (except for one test, which is always skipped by skip()).
> > However, I was told that somewhere in the package more than 2 cores
> > are used.
> >
> > I checked several times and even added a temporary 'stop()` to "trap"
> > parallel processing but still could not find where the source of the
> > problem is.
> >
> > I checked the timing in the log in R CMD check results from winbuilder
> > but everything seems OK. The user time and elapsed time are similar
> > for all the examples.
> >
> > Is there any quick way to check where things go wrong regarding the
> > number of cores? It is not easy to find the source of the problems
> > when there are many examples and tests.
>
> If it is OK on winbuilder but not on Linux, then likely something makes
> use of multithreading.
>
> Best,
> Uwe Ligges
>
>
>
> > Regards,
> > Shu Fai
> >
> > ______________________________________________
> > R-package-devel using r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-package-devel