[R-pkg-devel] tryCatch() doesn't capture errors due to && or || with R CMD check --as-cran

[Martin Maechler]

>>>>> Lionel Henry 
>>>>>     on Thu, 1 Sep 2022 18:47:36 +0200 writes:

    > Hi Martin,
    > I'm not sure I understand what you mean regarding suppressed warnings
    > in testthat.

    > - All warnings are reported to users by `devtools::test()`, with
    > accompanying backtraces. They are not hidden.

    > - R CMD check doesn't report warnings emitted from files in `tests/`.

but you can easily grep them in the tests/*.Rout files.
There are none of these in the testthat case,
and it's own summary only *counts* the number of warnings.

At least for me just now in a development version of lme4 (github),
running with increased (non-default) testlevel
	export LME4_TEST_LEVEL=2

I get useful *.Rout files (with only one warning, easily greppable)
for the non-testthat tests whereas  for the testthat ones I only get

------------------------------------------------------------------------
[ FAIL 3 | WARN 1 | SKIP 1 | PASS 829 ]

══ Skipped tests ═══════════════════════════════════════════════════════════════
• On CRAN (1)

══ Failed tests ════════════════════════════════════════════════════════════════
── Failure (test-methods.R:878:9): parallel influence ──────────────────────────
`i1` not equal to `i2`.
Component "vcov[-case]": Component "1": Attributes: < Length mismatch: comparison on first 1 components >
    .............................
    .............................
Component "vcov[-case]": Component "9": Attributes: < Length mismatch: comparison on first 1 components >
...
── Failure (test-summary.R:33:3): lmer ─────────────────────────────────────────
tfun(cc1) not equal to c(...).
Lengths differ: 5 is not 10
── Failure (test-summary.R:46:3): lmer ─────────────────────────────────────────
tfun(cc2) not equal to c(...).
Lengths differ: 5 is not 10

[ FAIL 3 | WARN 1 | SKIP 1 | PASS 829 ]
Error: Test failures
Execution halted
------------------------------------------------------------------------

Telling me (twice) that there was  1 warning, but
I don't know of any way to *see* that warning
(after everything is run, and I have  lme4.Rcheck/  to look
into).


    > If you need some sort of warning to surface as errors, warnings, or
    > notes in R CMD check, it should be possible to come up with
    > something.

Yes, that would be important for some of us.

IIRC, It does not help to just add

  options(warn = 2)

at the beginning of the main tests/*.R  file, or inside the
tests/testthat/*.R files , even though that would be standard R
behavior,  no ?

    > This could take the form of special classes of conditions
    > that R CMD check would catch and emit corresponding diagnostics
    > for. We would change testthat to make sure all such conditions are
    > properly emitted during checks.

If they are emitted as in R from warning() or stop() -- from R or C,
that would be a big step forward for us (R core) who'd like to
see effects of changing base R, and for all package maintainers considering
changing their API or just deprecating something, and using  .Deprecated()
{or equivalent} for that.

Thank you, Lionel!

Martin