[R-pkg-devel] Test fails on M1 Mac on CRAN, but not on macOS builder

[Simon Urbanek]

Florian,

looking at the notes for 2.1-4 it says the tolerance has the wrong sign, i.e. you're adding it to the value on both sides of the interval (instead of subtracting for the lower bound). In your latest version the tolerances get added everywhere so that makes even less sense to me, but then I don't know what you actually intended to be completely honest. All I say, simply make sure you get the logic for the tolerance intervals right.

Cheers,
Simon


> On 19/05/2023, at 9:49 PM, Pein, Florian <f.pein using lancaster.ac.uk> wrote:
> 
> Dear everyone,
> my R package stepR (https://cran.r-project.org/web/packages/stepR/) fails the CRAN package checks on M1 Mac, but the error does not occur on the macOS builder (https://mac.r-project.org/macbuilder/submit.html). So, I am unable to reproduce the error and hence unable to fix it (starring at the code did not help either).
> 
> The relevant part is
> 
> * checking tests ...
>  Running ĄĨtestthat.RĄĶ [35s/35s]
> [36s/36s] ERROR
> Running the tests in ĄĨtests/testthat.RĄĶ failed.
> 
>> test_check("stepR")
>  [ FAIL 1 | WARN 0 | SKIP 23 | PASS 22741 ]
> 
>  ųųųų Failed tests ųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųųų
>  ĒwĒw Failure ('test-critVal.R:2463:3'): family 'hsmuce' works ĒwĒwĒwĒwĒwĒwĒwĒwĒwĒwĒwĒwĒwĒwĒwĒwĒwĒwĒwĒw
>  compare <= as.integer(ncol(compareStat) * testalpha + tolerance) is not TRUE
> 
>  `actual`:   FALSE
>  `expected`: TRUE
>  sqrt
>  Backtrace:
>      Ēg
>   1. Ē|ĒwstepR (local) testVector(...) at test-critVal.R:2463:2
>   2.   Ē|Ēwtestthat::expect_true(...) at test-critVal.R:50:2
> 
>  [ FAIL 1 | WARN 0 | SKIP 23 | PASS 22741 ]
>  Error: Test failures
>  Execution halted
> 
> 
> Has anyone an idea how to tackle this problem?
> 
> The test code is long (a full version is available on CRAN). The following is the code part that I think is relevant (once again I cannot reproduce the error, so I am also unable to give a minimal reproducible example, I can only guess one):
> 
> library(stepR)
> library(testthat)
> 
> testn <- 1024L
> teststat <- monteCarloSimulation(n = 1024L, r = 100L, family = "hsmuce") # essentially a matrix with values generated by rnorm()
> testalpha <- 0.0567
> tolerance <- 1e-12
> 
> ret <- critVal(n = 1024L, penalty = "sqrt", output = "vector", family = "hsmuce", alpha = testalpha, stat = teststat)
> 
> statVec <- as.numeric(teststat)
> tol <- min(min(diff(sort(statVec[is.finite(statVec)]))) / 2, 1e-12) # different to the CRAN version to be robust to two values very close to each other
> rejected <- matrix(FALSE, ncol(teststat), nrow(teststat))
> compare <- integer(ncol(teststat))
> 
> for (i in 1:nrow(teststat)) {
>  rejected[, i] <- teststat[i, ] > ret[i] + tol
> }
> 
> for (i in 1:ncol(teststat)) {
>  compare[i] <- max(rejected[i, ])
> }
> compare <- sum(compare)
> expect_true(compare <= as.integer(ncol(teststat) * testalpha + tolerance), info = "sqrt")
> 
> # version with an additional tolerance (suggested when the test failed on CRAN, but it does not help either)
> # both sides are small intgers, so it should not be needed
> expect_true(as.integer(compare + tolerance) <= as.integer(ncol(teststat) * testalpha + tolerance) + 0.5, info = "sqrt")
> 
> 
> I am not sure how to approach this problem, so any suggestions are very much welcomed.
> Many thanks and best wishes,
> Florian Pein
> (Lancaster university)
> 
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-package-devel using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel