[R-pkg-devel] Run-times of examples in vignettes

[Dirk Eddelbuettel]

On 27 October 2020 at 12:21, Helmut Schütz wrote:
| is there somewhere an official statement about the maximum run-times of 
| examples in vignettes?

Seven minutes is excessive. I have (long) gone by the rule of "about one
minute" each for tests and examples. Rcpp is slightly above [1], especially
on Windows. I also tone down tests when on CRAN (using a scheme devised
almost a decade ago and described a few times here and in other places) where
to the package, having an extra fourth digit in the microrelease such as the
.4 currently in 1.0.5.4 signals 'full tests' whereas a three-digit release
number (such as 1.0.5 on CRAN signals reduced tests).  Works for me and on
all CI instances, requires no user input and no magic environment variable.

Now, we have some (partial) empirics on this as my reverse depends checks for Rcpp time
the runs (and the success/failure) in a small SQLite db (all on GH) [2]. I
also explicitly skip some packages taking too much time. I should probably
automate looking at the times and updating the list of skipped packages
automatically...

Dirk

[1] https://cloud.r-project.org/web/checks/check_results_Rcpp.html

[2] This is from an machine that is loaded by the reverse depends running six
    tests in parallel over four cores. This would be quicker on a nicer machine
    but ... this what we have, and what we very much appreciate having access
    to. It is also 'total time' for 'R CMD check'.
    

    sqlite> select package, runtime/60 as timeInMins from results order by timeInMins desc limit 10;
    package     timeInMins      
    ----------  ----------------
    cbq         54.2629543542862
    rstanarm    51.9673245469729
    OpenMx      44.4183359424273
    survHE      40.4267136971156
    pcFactorSt  36.404546391964 
    metaBMA     35.6222194115321
    trialr      35.2238613526026
    emIRT       34.6994982639949
    bmgarch     34.3939555843671
    bsem        33.4894991954168
    sqlite>

    So the first four or five clearly are candidates for skipping.


-- 
https://dirk.eddelbuettel.com | @eddelbuettel | edd using debian.org