[Rd] fix potential integer overflow in mantelhaen.test ?
Ben Bolker
bbolker at gmail.com
Wed Jan 24 18:45:17 CET 2018
Someone who was running into an integer overflow warning (and
downstream error) with mantelhaen.test() on a large data set asked about
it on StackOverflow:
https://stackoverflow.com/questions/48422398/na-nan-inf-in-foreign-function-error-with-mantelhaen-test/48428596#48428596
Coercing ntot to double (i.e. ntot -> as.numeric(ntot)) on lines 283
and 285 here:
https://github.com/wch/r-source/blob/af7f52f70101960861e5d995d3a4bec010bc89e6/src/library/stats/R/mantelhaen.test.R#L283
seems to fix the problem, fairly harmlessly (*maybe* there are some
edge cases where the integer and floating-point computations give
different answers??? these values are going to get sent to qr.solve() a
few lines later in any case ...)
If people think this is worthwhile I could submit a bug report.
cheers
Ben Bolker
More information about the R-devel
mailing list