[R-pkg-devel] CRAN packages suggesting other packages but not using them conditionally

[Spencer Graves]

On 2020-12-12 19:50, Duncan Murdoch wrote:
> On 12/12/2020 6:01 p.m., Ben Bolker wrote:
>>
>>
>> On 12/12/20 5:50 PM, Duncan Murdoch wrote:
>>> On 12/12/2020 4:08 p.m., Spencer Graves wrote:
>>>> Hi, Ben et al.:
>>>>
>>>>
>>>> On 2020-12-12 13:43, Ben Bolker wrote:
>>>>>      Apologies if I'm telling you something you already know:
>>>>>
>>>>>      By default, fda::CRAN() uses the presence of environment 
>>>>> variables
>>>>> matched by the regexp "^_R_" as a heuristic to decide whether it's 
>>>>> being
>>>>> running on CRAN.
>>>>>
>>>>>      testthat::skip_on_cran()  calls testthat::on_cran() to look 
>>>>> for an
>>>>> environment variable called NOT_CRAN equal to "true". The
>>>>> devtools::check() machinery automatically sets this variable.
>>>>
>>>>
>>>>    > testthat::on_cran
>>>> Error: 'on_cran' is not an exported object from 'namespace:testthat'
>>>>
>>>>
>>>>        Besides, on my Mac, I get:
>>>>
>>>>
>>>>    > testthat:::on_cran()
>>>> [1] TRUE
>>>>
>>>>
>>>>        My Mac is NOT CRAN, and I don't want that function to return
>>>> TRUE on
>>>> my computer unless I explicitly run "R CMD check --on-cran".
>>>>
>>>>
>>>>>      So: fda::CRAN() depends on breakable assumptions, defaults to 
>>>>> FALSE
>>>>> in an empty environment.  skip_on_cran() defaults to TRUE in an empty
>>>>> environment (but defaults to FALSE in a devtools::check() 
>>>>> environment).
>>>>
>>>>        If future changes break fda::CRAN, I will have to deal with it
>>>> then.
>>>>
>>>>
>>>>        I'd be happier if the CRAN maintainers would develop a 
>>>> procedure to
>>>> make it easier for package maintainers do two things:
>>>>
>>>>
>>>>              * Include tests in their package that run longer than the
>>>> time
>>>> limit permitted on CRAN.
>>>
>>> That's very easy now.  Just put them in a "slowtests" directory, and
>>> tell R CMD check to use that.  How could it be easier?
>>
>>      How would you do that?  In "R CMD check --help" I see that one can
>> use --test-dir= to specify the test directory, but I don't see a way to
>> specify _additional_ test directories; short of setting a tests/
>> directory with CRAN-specific tests and a slowtests/ directory with
>> *both* CRAN-specific and CRAN-excluded tests (thus duplicating files,
>> which seems clunky), I don't see how to do this within a standard R CMD
>> check framework (without testing a CRAN-indicating environment variable,
>> which gets us back where we started ...)  Or would you run R CMD check
>> twice, once without and once with --test-dir=slowtests ?
> 
> What I would do is have the slowtests run the regular tests.  So if I 
> want both, I run slowtests.  If I want just the fast ones, I don't 
> specify.  I can't think why I wouldn't want to run the slow ones without 
> the fast ones, but if I did, it's not too hard to figure out a scheme 
> that runs fast by default, slow when requested, and both if you request 
> that instead.
> 
>>
>>     There doesn't seem to be very much in "Writing R Extensions" about
>> testing - a little bit in
>> https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Package-subdirectories 
>>
>>
>>     What am I missing?
>>
>>    Just to clarify, the ideal would be to be able to designate a separate
>> set of tests that were *not* run on CRAN, and to be able to run them in
>> the same "R CMD check" pass as the CRAN-specific tests. 
> 
> Yes, do that as described above.
> 
> There are a
>> bunch of ways to achieve this, but I think Spencer is saying (and I
>> agree) that it would be nice if it were there an official mechanism that
>> made this easier (and it seems pretty easy if the CRAN maintainers were
>> agreeable to the idea ...
> 
> There is such a mechanism, and I've just described it (and not for the 
> first time; it's also described in WRE).  I think the problem is that 
> you and Spencer are looking for something that's more complicated.  It 
> doesn't need to be complicated.


	  I want to put all the tests of a particular function in the 
"\examples" section.  If some things are too pedantic to show to a user, 
I can put them in "\dontshow".  If they run too long for CRAN, I wrap 
them in "if(!fda::CRAN()){...}", as I previously noted.


	  Putting slow tests in a "slowtest" directory to me violates a 
sensible rule of documentation, because it makes it harder to think 
about how complete a test suite is.


	  I probably should not broaden this discussion to include "\dontrun", 
but I will:  I think any example in "\dontrun" should be made to work 
and wrapped in something like "if(!fda::CRAN()){...}" if you don't want 
it to be run on CRAN, where you don't care if it breaks or not.  I've 
read too many books with examples that didn't work!  The "fda" package 
has 76 reverse dependencies.  I think most of those are attributable to 
the quality of the fundamental ideas, but I'd like to think that some of 
them are because I insisted in included decent unit tests in the 
"\examples" -- AND because I insisted on make sure all but a couple of 
the examples in the book actually worked!


	  Thanks very much to everyone who has contributed to this thread.  I 
don't think we've reached a consensus, but we've had a good discussion 
and may eventually help improve package documentation and testing 
practices in the future.


	  Spencer


> Duncan Murdoch
> 
> ______________________________________________
> R-package-devel using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel