[Rd] R CMD check recursive copy of tests/

[Seth Falcon]

"Henrik Bengtsson" <hb at stat.berkeley.edu> writes:
>> intentional I'd say: I did not implement it, but it seems much
>> more logical to keep the previous rule: All *.R files in
>> ./tests/ are run <period>
>> Subdirectories can be useful for organization, notably storing
>> test data.  I don't think it's a good idea to use so very many test files
>> that you need subdirectories, unless maybe you are thinking
>> about unit tests; and then, see below.
>
> Examples of subdirectories (some overlapping) are:
>
> units/ - tests of minimal code modules
> integration/ - tests of integrating the above units
> system/ - "real-world" scenarios/use cases
>
> requirements/ - every requirement should have at least on test.
> bugs/ - every bug fix should come with a new test.
> regression/ - every update should have a regression test to validate
> backward compatibility etc.
>
> robustness/ - Testing the robustness of estimators against outliers as
> well as extreme parameter settings.
> validation/ - validation of numeric results compared with alternative
> implementations or summaries.
>
> benchmarking/ - actually more measuring time, but can involve
> validation that a method is faster than an alternative.
> crossplatform/ - validate correctness across platforms.
> torture/ - pushing the limits.

Those all seem like reasonable examples, but the fact that R CMD check
doesn't recurse really isn't a problem.  You can have a driver script
at the top-level that runs as many of the tests in subdirs as you
want.  And this is really a good thing since as you mentioned later in
your response, some tests take a long time to run and probably are
best not automatically run during R CMD check.

+ seth

-- 
Seth Falcon | Computational Biology | Fred Hutchinson Cancer Research Center
BioC: http://bioconductor.org/
Blog: http://userprimary.net/user/