[R-pkg-devel] CRAN packages suggesting other packages but not using them conditionally

[Ben Bolker]

On 12/12/20 5:53 PM, Duncan Murdoch wrote:
> On 12/12/2020 4:41 p.m., Ben Bolker wrote:
>>
>>
>> On 12/12/20 4:08 PM, Spencer Graves wrote:
>>> Hi, Ben et al.:
>>>
>>>
>>> On 2020-12-12 13:43, Ben Bolker wrote:
>>>>     Apologies if I'm telling you something you already know:
>>>>
>>>>     By default, fda::CRAN() uses the presence of environment variables
>>>> matched by the regexp "^_R_" as a heuristic to decide whether it's
>>>> being running on CRAN.
>>>>
>>>>     testthat::skip_on_cran()  calls testthat::on_cran() to look for an
>>>> environment variable called NOT_CRAN equal to "true". The
>>>> devtools::check() machinery automatically sets this variable.
>>>
>>>
>>>   > testthat::on_cran
>>> Error: 'on_cran' is not an exported object from 'namespace:testthat'
>>
>>        on_cran() is intended to be used via testthat::skip_on_cran()
>> (which is exported, unlike on_cran()).
>>>
>>>
>>>         Besides, on my Mac, I get:
>>>
>>>
>>>   > testthat:::on_cran()
>>> [1] TRUE
>>>
>>>
>>>         My Mac is NOT CRAN, and I don't want that function to return 
>>> TRUE
>>> on my computer unless I explicitly run "R CMD check --on-cran".
>>
>>     The assumption of testthat is that it's going to be deployed via
>> devtools::check(), which automatically sets the environment variable
>> NOT_CRAN equal to 'true'. For testing on your machine, you could use
>>
>> Sys.setenv(NOT_CRAN="true"); testthat:::on_cran()
>>
>> or you could put
>>
>> export NOT_CRAN=true
>>
>> in the shell/in your testing pipeline.
>>
>>
>>>
>>>
>>>>     So: fda::CRAN() depends on breakable assumptions, defaults to FALSE
>>>> in an empty environment.  skip_on_cran() defaults to TRUE in an empty
>>>> environment (but defaults to FALSE in a devtools::check() environment).
>>>
>>>         If future changes break fda::CRAN, I will have to deal with 
>>> it then.
>>>
>>>
>>>         I'd be happier if the CRAN maintainers would develop a procedure
>>> to make it easier for package maintainers do two things:
>>>
>>>
>>>               * Include tests in their package that run longer than the
>>> time limit permitted on CRAN.
>>>
>>>
>>>               * Give error messages that the package maintainer wants to
>>> see but that should be suppressed on CRAN or when the user decides to
>>> run "R CMD check --as-cran".
>>
>>    I agree that this would be nice.
>>
>>     A simple mechanism would be to set an official/sanctioned/stable
>> environment variable such as _R_ON_CRAN in all CRAN testing pipelines.
> 
> What's wrong with users setting NOT_CRAN on all non-CRAN testing pipelines?

   This is where we started. Nothing's wrong with it, but setting 
_R_CRAN=true on CRAN testing pipelines and providing an on_cran() 
function in the utils package would also seem almost trivially easy for 
R-core/CRAN maintainers, and would simplify the process for R package 
developers who are less familiar with shell/scripting/etc. (although I 
admit that (wanting_to_skip_tests && not_familiar_with_env_vars && 
not_working_in_devtoolsverse) could be a small intersection ...)

> Most people want the same tests in both places.  Those who like writing 
> lots of time consuming tests are the ones who shouldn't mind a small 
> step to control them.

   True. It doesn't take that much to exceed 10 minutes on the CRAN 
windows pipeline any more, though.  I have 56 separate test files in 
lme4; on the Windows pipeline it takes 3 seconds just to *load* the lme4 
package, and every file gets tested on ix386 and x86_64, so I would use 
up about 6 minutes of my 10-minute checking budget before I even got 
started ...

    (Yes, I know I could combine the files so that I have to load the 
package less often during the testing phase, or possibly eliminate the 
library() calls - I don't remember whether test files have to run in a 
standalone R session, although it certainly seems like best practice).

> 
> Duncan Murdoch
> 
>