What is the precission with which floating point operations are performed ? The following example shows an incorrect answer in the second case ? How can this problem be addressed i.e. the apparent precission seems to be a sevre limitation to any scientific computation ? Prelude> 1000.11 - 0.1 1000.01 Prelude> 10000.11 - 0.1 10000.0 _________________________________________________________________ Browse styles for all ages, from the latest looks to cozy weekend wear at MSN Shopping. And check out the beauty products! http://shopping.msn.com
On Mon, Dec 15, 2003 at 03:47:22PM +1100, mike V wrote:
What is the precission with which floating point operations are performed ? The following example shows an incorrect answer in the second case ? How can this problem be addressed i.e. the apparent precission seems to be a sevre limitation to any scientific computation ?
Prelude> 1000.11 - 0.1 1000.01 Prelude> 10000.11 - 0.1 10000.0
Hugs used to represent both Float and Double as float, but the Nov 2003 release uses float and double respectively (on most architectures).
mike V writes: | What is the precission with which floating point operations are performed ? | The following example shows an incorrect answer in the second case ? | How can this problem be addressed i.e. the apparent precission seems to be a | sevre limitation to any scientific computation ? | | Prelude> 1000.11 - 0.1 | 1000.01 | Prelude> 10000.11 - 0.1 | 10000.0 It seems to be a formatting thing, because: Prelude> 10000.11 - 0.1 - 10000 0.0107422 Also beware that some builds of Hugs give you single precision floats for the Double type. That's probably why I only got 7 decimal places just then, instead of 15. You can get proper double precision floats by doing your own Hugs build with the appropriate flag set. - Tom
participants (3)
-
mike V -
Ross Paterson -
Tom Pledger