[Bioc-devel] R cmd check time limits for BioConductor

[Laurent Gautier]

I think that you points are relevant, and the value of regression
tests is well understood... however
they require changes in "R CMD check" (and that's something for r-devel).

[1] The trouble with suggesting "R CMD check --no-tests" as a default,
is that package
maintainers with tests running quickly may feel unhappy (as they prefer having
the tests ran automatically without having to worry about what switch is where).

[2] The proposed change to the DESCRIPTION file (Test: run/don't run)
can be emulated,
for example by adding a variable DO_TEST at the top of the test script
and having a "if (DO_TEST)"
branching.

[3] The notion of short or long is very relative, I guess. In the
early days of bioconductor,
when the number of packages was smaller, packages maintainer were only
scolded when
the check was reaching half an hour or more.

Just a thought,


Laurent






2008/6/10 Kevin R. Coombes <krcoombes at mdacc.tmc.edu>:
> Hi,
>
> I have considered that possibility, but am not yet convinced that it is the
> best approach. I will, of course, do something like that if I cannot
> persuade this list that an alternative approach might be better. The basic
> argument is:
>
> * Complex algorithms can be better maintained if they are accompanied by
> regression testing.
> * "R CMD check" provides an automated method to run regression tests, with a
> defined directory structure for storing those tests.
> * Changing the directory location in the source makes running the regression
> tests more awkward and thus less likely to occur on a regular basis.
> * The "--no-tests" argument already provides a mechanism for preventing the
> tests from being run.
>
> What appears to be missing is either a mechanism to designate the tests as
> optional or to indicate a preference for not running some or all of them. I
> can think of three ways to accomplish my goals in this matter:
>
> [1] Make "--no-tests" the default way to run "R CMD check" at BioConductor.
> (Of course, this is unlikely to be the optimal solution since it merely
> avoids the question.)
> [2] Add a field to the DESCRIPTION file that tells "R CMD check" whether or
> not to run the tests. Something like
>        Tests: run
> or
>        Tests: dontrun
> [3] Add an optional special file in the tests directory that indicates the
> complexity/length of the tests that would allow "R CMD check" to decide
> whether or not to run them. Perhaps something like
>
> ###################
> # COMPLEXITY file
>
> test1.R: long
> test2.R: short
> ...
> ###################
>
> Of course, options [2] or [3] require changes to "R CMD check" (for which I
> should eventually move this discussion to the R-devel list), but I am really
> only interested in convincing BioConductor that (possibly complex)
> regression tests are a good thing, and should be encouraged by adopting
> something like [1].
>
> Best,
>        Kevin
>
> Laurent Gautier wrote:
>>
>> 2008/6/10 Kevin R. Coombes <krcoombes at mdacc.tmc.edu>:
>>>
>>> Hi,
>>>
>>> The BioConductor package guidelines say that a package should take less
>>> than
>>> five minutes to run "R CMD check". I have a package that is almost ready
>>> to
>>> submit; however, it currently includes nontrivial regression testing in
>>> the
>>> "tests" subdirectory. With the tests, the time for "R CMD check" could be
>>> significantly longer than five minutes. Without the tests, the package
>>> easily fits within the time limit.
>>>
>>> [1] I know that I can run "R CMD check --no-tests [PKG]" to prevent the
>>> tests from running when I check the code myself. Is there any way for a
>>> package submitted to BioConductor to indicate that the tests should be
>>> skipped?
>>>
>>> [2] Alternatively, is there an easy way to include the tests so that I
>>> can
>>> run them whenever I want to make sure I haven't broken the code (too
>>> badly
>>> ...), but not force everyone else to run them when checking the rest of
>>> the
>>> structure of the code and documentation?
>>
>> You could consider having them in your package, in a directory
>> inst/tests/ for example
>> (so the tests are still available from an installed package).
>>
>>> Thanks in advance,
>>>   Kevin
>>>
>>> _______________________________________________
>>> Bioc-devel at stat.math.ethz.ch mailing list
>>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>>>
>