[R-pkg-devel] CRAN pretest: Overall checktime 14 min > 10 min

[Uwe Ligges]

On 21.05.2018 18:04, Dirk Eddelbuettel wrote:
> 
> On 21 May 2018 at 17:46, Uwe Ligges wrote:
> | In addition to what Dirk said, I just added this experimental test for
> | CRAN incoming checks few days ago and it should not reject but lead to
> | manual inspection, this will be fixed on CRAN side shortly.
> |
> | Nevertheless: The idea is that we have timeouts for checking a package
> | and we want to be alerted of future timeout problems in advance when a
> | package is submitted.
> 
> Which is totally understandable given the limits we have on (human, chiefly)
> resources here -- but it is also worth reminding ourselves from time to time
> that this of course reduces actual test coverage ... which is generally not
> we want with these tests.  This whole topic needs some more discussion.


A very difficult topic, while some authors do not supply any sensible 
checks nor evben examples and we have to force them to add even simplest 
code for some runtime checks, there are other who try too much. We are 
frequently bitten by cases where maintainers try to increase code 
coverage in the tests, but actually do test features of other packages 
(such as warning or error messages another package or R generates). One 
of the issues to be discussed would be that sensible testing is 
required, but not everythuing that leads to a larger coverage percentage 
makes sense ...
Another frequent thing we see is thast people run MCMC simulations in 
the tests with thousands of iterations that are actually not needed for 
testing.

Best,
Uwe




> On 21 May 2018 at 18:41, Chris Njuguna wrote:
> | Thank you for this! It makes a lot of sense. I will adopt this version
> | naming and testing scheme.
> 
> Nice. I didn't give to much detail but if glance eg at
>    https://github.com/RcppCore/Rcpp/blob/master/tests/doRUnit.R#L45-L52
> for the detection (including an environment variable override) and eg at
>    https://github.com/RcppCore/Rcpp/blob/master/inst/unitTests/runit.DataFrame.R#L21-L23
> for one use case you should be able to get started. Ping me off-list with any
> questions.
> 
> Dirk
>