Joachim Breitner
Hi,
currently, perf.haskell.org has stopped because recent commits fail to build with:
expected stdout not matched by reality --- fft2.stdout7 2018-11-28 12:00:24.126015465 -0500 +++ /tmp/runtest6392.1 2018-11-28 12:59:59.329423367 -0500 @@ -1,3 +1,3 @@ -result1 = 2.59635799135966e-12 -result2 = 2.59635799135966e-12 -result3 = 4.8279900966008427e-8 +result1 = 2.649891777856828e-12 +result2 = 2.649891777856828e-12 +result3 = 4.830900479646516e-8 make[2]: *** [../../mk/target.mk:102: runtests] Error 1
do others observe this as well? This is worrying, because there has been no change to nofib since it last worked.
Can we add a check of the functional correctness of the nofib programs to the CI somehow?
Yikes, this is indeed somewhat worrying. I see the same result as you for result1 and result2 but a slightly different number for result3. Looking at this particular test it appears to be quite numerically unstable. I count over ten distinct sets of .stdout files in spectral/fft2, some of which match what the results I observe quite closely. Overall we should indeed make sure these get tested in CI. Do you know how long ago this change occurred? Cheers, - Ben