Numerics (was: Re: Trouble with asinh)

The numerics in Haskell have not been carefully vetted, for a variety of reasons. Not just under MS Windows, even under Linux. Here's an example: atan has drastic loss of precision in the imaginary direction around zero in the complex domain. $ ghci GHCi, version 8.8.4: https://www.haskell.org/ghc/ :? for help Prelude> :m + Data.Complex Prelude Data.Complex> tan (1e-20 :+ 0) 1.0e-20 :+ 0.0 Prelude Data.Complex> atan (1e-20 :+ 0) 1.0e-20 :+ (-0.0) Prelude Data.Complex> tan (0 :+ 1e-20) 0.0 :+ 1.0e-20 Prelude Data.Complex> atan (0 :+ 1e-20) 0.0 :+ (-0.0) Although there have been amazing efforts to use fancy PLT methods to improve the numerics of programs using source-to-source transformations and such, the boring janitorial work of checking and fixing numeric issues in the standard library doesn't seem to attract people. To be fair, it isn't my cup of tea either...

Hey Barak, is Common Lisp the only extant language to take those issues
seriously or are there other examples or better ones?
(i bought the common lisp book a year or so ago because its one of the few
references i could that talk about language/ library design for numerics /
branch cuts etc)
please edumacate me :)
-Carter
On Fri, Sep 17, 2021 at 12:18 PM Barak A. Pearlmutter
The numerics in Haskell have not been carefully vetted, for a variety of reasons. Not just under MS Windows, even under Linux. Here's an example: atan has drastic loss of precision in the imaginary direction around zero in the complex domain.
$ ghci GHCi, version 8.8.4: https://www.haskell.org/ghc/ :? for help Prelude> :m + Data.Complex
Prelude Data.Complex> tan (1e-20 :+ 0) 1.0e-20 :+ 0.0
Prelude Data.Complex> atan (1e-20 :+ 0) 1.0e-20 :+ (-0.0)
Prelude Data.Complex> tan (0 :+ 1e-20) 0.0 :+ 1.0e-20
Prelude Data.Complex> atan (0 :+ 1e-20) 0.0 :+ (-0.0)
Although there have been amazing efforts to use fancy PLT methods to improve the numerics of programs using source-to-source transformations and such, the boring janitorial work of checking and fixing numeric issues in the standard library doesn't seem to attract people. To be fair, it isn't my cup of tea either... _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

Le 17/09/2021 à 19:16, Carter Schonwald a écrit :
Hey Barak, is Common Lisp the only extant language to take those issues seriously or are there other examples or better ones?
rr, ii = 1.0e-20+0j, 1.0e-20j
// tan / atan troubles cited by Barak. Common Lisp?? But even the ugly Python reacts better to such examples: *>>> from numpy import * * *>>> tan(rr),arctan(rr) ((1e-20+0j), (1e-20+0j))
tan(ii),arctan(ii) (1e-20j, 1e-20j)*
*==* This works also with cmath (with arctan as atan). ** Jerzy Karczmarczuk -- L'absence de virus dans ce courrier électronique a été vérifiée par le logiciel antivirus Avast. https://www.avast.com/antivirus

I suspect that most implementations of Common Lisp just call the C standard library catan(3) etc, which are well tuned. $ clisp Welcome to GNU CLISP 2.49.92 (2018-02-18) http://clisp.org/ [1]> (atan #c(0 1d-40)) #C(0 1.0d-40) In this particular case, the problem is that the Haskell Data.Complex code has its own implementation of atan, which uses a log(1 + x) in calculating the imaginary part. A foreign function call to the appropriate libm routine would robustly address this, but that would be difficult because it's trying to be generic over RealFloat a => Complex a, instead of special casing Complex Float / Complex Double. Anyway, the Standard Prelude code for this is naïve: it should call log1p, at the very least—which it actually goes to the trouble of defining correctly, but not exporting.

Hey Barak,
These are really good points. They seem worth submitting to the GHC issue
tracker. This would be a first step on the way to improve the current (not
ideal, as you say) state.
—
Best, Artem
On Fri, Sep 17, 2021 at 5:06 PM Barak A. Pearlmutter
I suspect that most implementations of Common Lisp just call the C standard library catan(3) etc, which are well tuned.
$ clisp Welcome to GNU CLISP 2.49.92 (2018-02-18) http://clisp.org/ [1]> (atan #c(0 1d-40)) #C(0 1.0d-40)
In this particular case, the problem is that the Haskell Data.Complex code has its own implementation of atan, which uses a log(1 + x) in calculating the imaginary part. A foreign function call to the appropriate libm routine would robustly address this, but that would be difficult because it's trying to be generic over RealFloat a => Complex a, instead of special casing Complex Float / Complex Double. Anyway, the Standard Prelude code for this is naïve: it should call log1p, at the very least—which it actually goes to the trouble of defining correctly, but not exporting. _______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

Hi all – I now have fixes for all the issues I’m aware of. However, it’s quite possible I’ve made a mistake somewhere (either in the new code or the testing), so if anyone would like to help review either please let me know. Group 1 issues (real numbers in Windows, where the defects are in mingw-w64: I’ve raised issue #20424https://gitlab.haskell.org/ghc/ghc/-/issues/20424 for this. The fixes (commits 66ba5f32https://github.com/mingw-w64/mingw-w64/commit/66ba5f3221c786de24f5fc4b9c0236... and 021dffb8ahttps://github.com/mingw-w64/mingw-w64/commit/021dffb8a482eb9d1b39569cd1ea42...) have been made in mingw-w64 and are getting integrated into Haskell soon. Group 2 issues (complex numbers, where the defects are in Complex.hs). I’ve raised issue #20425https://gitlab.haskell.org/ghc/ghc/-/issues/20425 for this and have the code fixes herehttps://gitlab.haskell.org/davjam/ghc/-/blob/ComplexBranchCuts/libraries/bas.... The new code changes many of the functions (I’ve given examples in the issue) and adds a few. I’ve also put the Haddock output herehttps://davjam.github.io/HaskellNumericsTestsFixes/TrigDiags/Data-Complex.ht.... (It now defines and gives an explanation of the branch cuts). I’ve also put some diagrams herehttps://davjam.github.io/HaskellNumericsTestsFixes/TrigDiags/Curr.html illustrating some of the problems. I’ve done about as much testing as I can think of, using the code herehttps://github.com/davjam/HaskellNumericsTestsFixes/blob/main/ComplexTests.h.... Ideally I’d bulk-test against a reliable independent source, but can’t find one. AFAICT WolframAlphahttps://www.wolframalpha.com/input/?i=sin%28-0.0%29, Excel, gnumeric, CLISP don’t support negative zeros. Pythonhttps://www.python.org/ seems to, but cmath has incorrect branch cuts (cmath.sqrt(-4-0j) gives 2j). Matlabhttps://www.advanpix.com/2016/04/28/branch-cuts-and-signed-zeros-in-matlab/ also seems deficient in a number of areas. (Hmmm: maybe no one cares about these working correctly??) Sorry about the delay in sending this, David.

Hi David,
I have one suggestion for the haddocks. I made a solid effort to wade into
the explanation of branch cuts and negative zeros, but I never managed to
figure out why I should care. ;) In other words: tl;dr.
Would it be possible to write a pithy few words right at the beginning as
an introduction that motivates the topic? If, as you say, few other
languages take the subject into account, it's very likely that few
programmers take it into account either.
In general, as a math-conscious member of the community, I'm happy to see
this kind of work being accomplished, so thanks!
-Bryan
On Fri, 22 Oct 2021, 17.48 David James,
Hi all – I now have fixes for all the issues I’m aware of. However, it’s quite possible I’ve made a mistake somewhere (either in the new code or the testing), so if anyone would like to help review either please let me know.
*Group 1 issues* (real numbers in Windows, where the defects are in mingw-w64: I’ve raised issue #20424 https://gitlab.haskell.org/ghc/ghc/-/issues/20424 for this. The fixes (commits 66ba5f32 https://github.com/mingw-w64/mingw-w64/commit/66ba5f3221c786de24f5fc4b9c0236... and 021dffb8a https://github.com/mingw-w64/mingw-w64/commit/021dffb8a482eb9d1b39569cd1ea42...) have been made in mingw-w64 and are getting integrated into Haskell soon.
*Group 2 issues* (complex numbers, where the defects are in Complex.hs). I’ve raised issue #20425 https://gitlab.haskell.org/ghc/ghc/-/issues/20425 for this and have the code fixes here https://gitlab.haskell.org/davjam/ghc/-/blob/ComplexBranchCuts/libraries/bas.... The new code changes many of the functions (I’ve given examples in the issue) and adds a few. I’ve also put the Haddock output here https://davjam.github.io/HaskellNumericsTestsFixes/TrigDiags/Data-Complex.ht.... (It now defines and gives an explanation of the branch cuts). I’ve also put some diagrams here https://davjam.github.io/HaskellNumericsTestsFixes/TrigDiags/Curr.html illustrating some of the problems.
I’ve done about as much testing as I can think of, using the code here https://github.com/davjam/HaskellNumericsTestsFixes/blob/main/ComplexTests.h.... Ideally I’d bulk-test against a reliable independent source, but can’t find one. AFAICT WolframAlpha https://www.wolframalpha.com/input/?i=sin%28-0.0%29, Excel, gnumeric, CLISP don’t support negative zeros. Python https://www.python.org/ seems to, but cmath has incorrect branch cuts (cmath.sqrt(-4-0j) gives 2j). Matlab https://www.advanpix.com/2016/04/28/branch-cuts-and-signed-zeros-in-matlab/ also seems deficient in a number of areas. (Hmmm: maybe no one cares about these working correctly??)
Sorry about the delay in sending this, David.
_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

Hi – thanks for your suggestion, though it’s a bit of a challenge! Do you know of an example where the existing haddocks have something similar?
The branch cuts and behaviour of negative zeros are just part of the spec for what the functions do. Hence I would expect that anyone who reads the Data.Complex page should want to know about them. (Else it’s a bit like saying “I want to use sqrt but have no interest in knowing that it will only return a positive value”).
Here’s my best attempt at pithy so far:
Data.Complex
Complex numbers that comply with the LIA standards with regard to signed zeros and branch cuts.
Branch Cuts & Principle Values
The “inverse” complex functions (such as sqrt, log, asin, which are mathematically multivalued) return only a single principal value within a defined range. In general, inverse(fn z) == z only when z is within the defined range. The inverse functions are continuous throughout the complex plane except for discontinuities at certain lines on the axes called “branch cuts”.
The ranges and branch cuts comply with the LIA standards and are detailed below. In particular, two (==) points on a branch cut will map to different points on the range boundary if they have zeros of different signs. In some cases this allows apparently identical expressions to be computationally equivalent, for example sqrt(z/(z-1)) * sqrt(1/(z-1)) and sqrt z / (z-1), although detailed analysis is required to determine the behaviour and equivalence of expressions in general.
Note that currently in Haskell:
f1 z = sqrt(z/(z-1)) * sqrt(1/(z-1))
f1 ((-4) :+ 0) = 0.0 :+ 0.4
f2 z = sqrt z / (z-1)
f2 ((-4) :+ 0) = 0.0 :+ (-0.4)
Negative zeros (which GHC supports) provide a mechanism to address this, but only if sqrt makes correct use of it for points on the branch (which it currently does not – hence my proposed fixes, which also address other issues such as overflow, etc).
[If you’re also asking about -0.0 in non-complex functions, here’s an example:
f x = atan(1/x)
Mathematically, f x is undefined at x=0. But, in Haskell, we get f 0 = 1.57.. and f(-0) = -1.57. This mirrors the maths f(x) -> pi/2 as x -> 0 from above, and f(x) -> -pi/2 as x -> 0 from below. (Whether this is what’s required is for the programmer/analyst to determine, depending on the problem being solved).]
Sorry, a not very pithy response 😊
Regards, David.
From: Bryan Richtermailto:b@chreekat.net
Sent: 23 October 2021 07:06
To: David Jamesmailto:dj112358@outlook.com
Cc: Haskell Cafemailto:haskell-cafe@haskell.org
Subject: Re: [Haskell-cafe] Numerics (was: Re: Trouble with asinh)
Hi David,
I have one suggestion for the haddocks. I made a solid effort to wade into the explanation of branch cuts and negative zeros, but I never managed to figure out why I should care. ;) In other words: tl;dr.
Would it be possible to write a pithy few words right at the beginning as an introduction that motivates the topic? If, as you say, few other languages take the subject into account, it's very likely that few programmers take it into account either.
In general, as a math-conscious member of the community, I'm happy to see this kind of work being accomplished, so thanks!
-Bryan
On Fri, 22 Oct 2021, 17.48 David James,
participants (6)
-
Artem Pelenitsyn
-
Barak A. Pearlmutter
-
Bryan Richter
-
Carter Schonwald
-
David James
-
Jerzy Karczmarczuk