On July 11, 2012 04:51:50 Christian Maeder wrote:
Is it not enough to store floats into memory just before equality tests (or add rounding in the instance definitions of Float and Double in Eq and Ord)?
You have to be 100% consistent in how you do every operations in all cases otherwise different levels of rounding errors will creep into the results. It isn't too hard to imagine a floating point expression getting inlined somewhere, and the compiler generating code to evalulate it all in registers. Intermediate operations will then be done to 80 bit precision. Elsewhere, it doesn't get inlined and the compiler generates code to store intermediate results in memory. Intermediate operations will then be done to 32 bit precision. Different results will occur on the rounding boundaires. Always storing and reloading after every operations will give you consistent results. I guess the other option is to disable inlining etc. or somehow ensure they are applied consistently in all use cases of an expression. Cheers! -Tyson