[R-pkg-devel] Re-building vignettes had CPU time 9.2 times elapsed time

[Avraham Adler]

To be fair, data.table defaults to using 1/2 the available cores; they do not take the entire machine by default. 

Avi

Sent from my iPhone

> On Aug 25, 2023, at 6:46 PM, Duncan Murdoch <murdoch.duncan using gmail.com> wrote:
> 
> On 25/08/2023 6:13 p.m., Toby Hocking wrote:
>> Thanks Dirk. I agree.
>> data.table is not in a situation to update very soon, so the easiest
>> solution for the R community would be for CRAN to set OMP_THREAD_LIMIT
>> to 2 on the Windows and Debian machines doing this test.
>> Otherwise the 1400+ packages with hard dependencies on data.table will
>> each have to implement custom logic to limit threads to 2.
> 
> That doesn't follow.  data.table could update soon even if that wasn't their intention:  just include bug fixes and set the default OMP_THREAD_LIMIT to 2 in data.table.
> 
> The real problem is that there are two stubborn groups opposing each other:  the data.table developers and the CRAN maintainers.  The former think users should by default dedicate their whole machine to data.table.  The latter think users should opt in to do that.
> 
> Duncan Murdoch
> 
>> Toby
>>> On Fri, Aug 25, 2023 at 6:46 AM Dirk Eddelbuettel <edd using debian.org> wrote:
>>> 
>>> 
>>>> On 24 August 2023 at 07:42, Fred Viole wrote:
>>> | Hi, I am receiving a NOTE upon submission regarding the re-building of
>>> | vignettes for CPU time for the Debian check.
>>> |
>>> | I am unable to find any documented instances or solutions to this issue.
>>> | The vignettes currently build in 1m 54.3s locally and in 56s on the Win
>>> | check.
>>> |
>>> | https://win-builder.r-project.org/incoming_pretest/NNS_10.1_20230824_132459/Debian/00check.log
>>> 
>>> Please see, inter alia, the long running thread
>>> 
>>>    "Trouble with long-running tests on CRAN debian server"
>>> 
>>> started earlier this week (!!) on this list covering exactly this issue.
>>> 
>>> We can only hope CRAN comes to understand our point that _it_ should set a
>>> clearly-identifable variable (the OpenMP thread count would do) so that
>>> package data.table can this for its several hundred users.
>>> 
>>> As things currently stand, CRAN expects several hundred packages (such as
>>> your, guessing there this comes from data.table which I do not know for sure
>>> but you do import it) to make the change which is pretty close to the text
>>> book definition of madness.
>>> 
>>> Also see https://github.com/Rdatatable/data.table/issues/5658 with by now 24
>>> comments.  It is on the same issue.
>>> 
>>> Uwe, Kurt: Please please please set OMP_THREAD_LIMIT to 2 on the Windows and
>>> Debian machines doing this test.
>>> 
>>> Dirk
>>> 
>>> --
>>> dirk.eddelbuettel.com | @eddelbuettel | edd using debian.org
>>> 
>>> ______________________________________________
>>> R-package-devel using r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>> ______________________________________________
>> R-package-devel using r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-package-devel
> 
> ______________________________________________
> R-package-devel using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel