[Rd] Question about quantile fuzz and GPL license

[GILLIBERT, Andre]

On 9/14/21 9:22 AM, Abel AOUN wrote:
> However I don't get why epsilon is multiplied by 4 instead of simply using epsilon.
> Is there someone who can explain this 4 ?

.Machine$double.eps is the "precision" of floating point values for values close to 1.0 (between 0.5 and 2.0).

Using fuzz = .Machine$double.eps would have no effect if nppm is greater than or equal to 2.
Using fuzz = 4 * .Machine$double.eps can fix rounding errors for nppm < 8; for greater nppm, it has no effect.

Indeed:
2 + .Machine$double.eps == 2
8+ 4*.Machine$double.eps == 8

Since nppm is approximatively equal to the quantile multiplied by the sample size, it can be much greater than 2 or 8.

Maybe the rounding errors are only problematic for small nppm; or only that case is taken in account.

Moreover, if rounding errors are cumulative, they can be much greater than the precision of the floating point value. I do not know how this constant was chosen and what the use-cases were.

--
Sincerely
Andre GILLIBERT


	[[alternative HTML version deleted]]