[Rd] Running package tests and not stop on first fail

[Martin Maechler]

>>>>> Jan Gorecki <J.Gorecki at wit.edu.pl>
>>>>>     on Fri, 4 Nov 2016 11:20:37 +0000 writes:

    > Martin, I submitted very simple patch on
    > https://bugs.r-project.org/bugzilla/show_bug.cgi?id=17176

    > Herve, While I like your idea, I prefer to keep my patch
    > simple, it is now exactly what Martin mentions. I think it
    > is a good start that can eventually be extended later for
    > what you are asking.

I tend to agree; this seems indeed much easier than I
anticipated.  Thank you, Jan!

I'm testing a version which uses the logical variable
'stop_on_error' rather than 'no_stop_on_error' (because
!no_stop_on_error is hard to mentally parse quickly).

My proposed name  '--no-stop-on-error'  was a quick shot; if
somebody has a more concise or better "English style" wording
(which is somewhat compatible with all the other options you see
from 'R CMD check --help'),
please speak up.

Martin

    > Regards, Jan

    > On 3 November 2016 at 17:25, Hervé Pagès
    > <hpages at fredhutch.org> wrote:
    >> 
    >> Hi Martin, Jan,
    >> 
    >> On 11/03/2016 03:45 AM, Martin Maechler wrote:
    >>>>>>>> 
    >>>>>>>> Jan Gorecki <J.Gorecki at wit.edu.pl> on Tue, 1 Nov
    >>>>>>>> 2016 22:51:28 +0000 writes:
    >>> 
    >>> 
    >>> > Hello community/devs, Is there an option to run
    >>> package > tests during R CMD check and not stop on first
    >>> error? I > know that testing frameworks (testhat and
    >>> others) can do > that but asking about just R and base
    >>> packages. Currently > when package check runs test
    >>> scripts in ./tests directory > it will stop after first
    >>> fail.  Do you think it could be > optionally available
    >>> to continue to run tests after > failures?  Regards, Jan
    >>> Gorecki
    >>> 
    >>> I agree that this would be a useful option sometimes.
    >>> 
    >>> So I would be supportive to get such an option, say,
    >>> 
    >>> R CMD check --no-stop-on-error <pkg>
    >> 
    >> 
    >> A couple of years ago the behavior of 'R CMD check' was
    >> changed to continue checking (e.g. the examples) after
    >> many types of errors, and to output a summary count of
    >> errors at the end if any have occurred.  So
    >> --no-stop-on-error could easily be interpreted as an
    >> option that controls this behavior (and would also
    >> suggest that the default has been reverted back to what
    >> it was prior to R 3.2.0), rather than an option that
    >> specifically controls what should happen while running
    >> the tests.
    >> 
    >> Cheers, H.
    >> 
    >>> 
    >>> into R if someone provided (relatively small) patches to
    >>> the R sources (i.e. subversion repos at
    >>> https://svn.r-project.org/R/trunk/ ).  The relevant
    >>> source code should basically all be in
    >>> src/library/tools/R/testing.R
    >>> 
    >>> Note that this may be complicated, also because
    >>> "parallel" checking is available in parts, via the
    >>> TEST_MC_CORES environment variable ((which is currently
    >>> only quickly documented in the 'R Administration ..'
    >>> manual))
    >>> 
    >>> 
    >>> Martin Maechler ETH Zurich
    >>> 
    >>> ______________________________________________
    >>> R-devel at r-project.org mailing list
    >>> https://stat.ethz.ch/mailman/listinfo/r-devel
    >>> 
    >> 
    >> --
    >> Hervé Pagès
    >> 
    >> Program in Computational Biology Division of Public
    >> Health Sciences Fred Hutchinson Cancer Research Center
    >> 1100 Fairview Ave. N, M1-B514 P.O. Box 19024 Seattle, WA
    >> 98109-1024
    >> 
    >> E-mail: hpages at fredhutch.org Phone: (206) 667-5791 Fax:
    >> (206) 667-1319