[GHC] #15062: num009 is incredibly platform-sensitive
#15062: num009 is incredibly platform-sensitive -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.6.1 Component: Test Suite | Version: 8.2.2 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Test Case: | Blocked By: Blocking: | Related Tickets: Differential Rev(s): | Wiki Page: -------------------------------------+------------------------------------- The functions tested by `num009` are certainly worthwhile to test, but we really need to find a better way to test them. Currently the test seemingly fails in more places than it passes: * Fails on Darwin (#2370) * Fails on POWER8 (#13634) * Fails on Win32 when in the `ghci` way (no ticket) * Fails under i386 on CircleCI (e.g. https://circleci.com/gh/ghc/ghc/3666) Perhaps we should instead test the output against the same evaluations performed by a C program. This would eliminate spurious failures due to platform or C library differences. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15062> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#15062: num009 is incredibly platform-sensitive -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.6.1 Component: Test Suite | Version: 8.2.2 Resolution: | Keywords: ci-breakage Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * keywords: => ci-breakage -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15062#comment:1> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#15062: num009 is incredibly platform-sensitive -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.6.1 Component: Test Suite | Version: 8.2.2 Resolution: | Keywords: ci-breakage Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by osa1): Also fails here on i386 chroot environment.
Perhaps we should instead test the output against the same evaluations performed by a C program. This would eliminate spurious failures due to platform or C library differences.
Could you elaborate on this? This doesn't look like a spurious failure to me. I think either `sin`, `cos` etc. or C FFI when passing floats is broken on i386. Sounds pretty serious to me. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15062#comment:2> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#15062: num009 is incredibly platform-sensitive -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.6.1 Component: Test Suite | Version: 8.2.2 Resolution: | Keywords: ci-breakage Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Nothing is broken; all of the variations I've seen are at numerical precision. This is just expected variation from platform-to-platform. The transcendental operations tested by `num009` are typically implemented in terms of the platform's supported floating point operations by `libc`. `libc`s vary in their implementation strategies for these operations. Moreover, the extra precision of x87 (which we still use by default on x86) has an impact. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15062#comment:3> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#15062: num009 is incredibly platform-sensitive -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.6.1 Component: Test Suite | Version: 8.2.2 Resolution: | Keywords: ci-breakage Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by bgamari): Here is the i386 output for posterity, {{{ Actual stdout output differs from expected: --- ../../libraries/base/tests/Numeric/num009.run/num009.stdout.normalised 2018-04-19 18:22:43.131191836 +0000 +++ ../../libraries/base/tests/Numeric/num009.run/num009.run.stdout.normalised 2018-04-19 18:22:43.131191836 +0000 @@ -1 +1,81 @@ +uh oh! sind 3.141592653589793 +1.2246467991473532e-16 +1.2246063538223773e-16 +(4967757600021511,-105) +(4967593534291968,-105) +uh oh! sind 1.0e10 +-0.4875060250875107 +-0.48750602507627 +(-8782127811699939,-54) +(-8782127811497445,-54) +uh oh! sind 1.0e20 +-0.6452512852657808 +-0.7469218912594929 +(-5811906895766608,-53) +(-6727674302302237,-53) +uh oh! sinf 1.0e20 +0.6565767 +-0.7710884 +(11015529,-24) +(-12936717,-24) +uh oh! cosd 1.5707963267948966 +6.123233995736766e-17 +6.123031769111886e-17 +(4967757600021511,-106) +(4967593534291968,-106) +uh oh! cosd 1.0e10 +0.873119622676856 +0.8731196226831323 +(7864362414674714,-53) +(7864362414731245,-53) +uh oh! cosd 1.0e20 +0.7639704044417283 +-0.6649117899070088 +(6881233657531709,-53) +(-5988992978518909,-53) +uh oh! cosf 1.0e20 +0.7542593 +0.63672805 +(12654371,-24) +(10682524,-24) +uh oh! tand 3.141592653589793 +-1.2246467991473532e-16 +-1.2246063538223773e-16 +(-4967757600021511,-105) +(-4967593534291968,-105) +uh oh! tand 1.5707963267948966 +1.633123935319537e16 +1.6331778728383844e16 +(8165619676597685,1) +(8165889364191922,1) +uh oh! tand 1.0e10 +-0.5583496378112418 +-0.5583496377943541 +(-5029166441578320,-53) +(-5029166441426209,-53) +uh oh! tand 1.0e20 +-0.8446024630198843 +1.123339821307656 +(-7607502675465108,-53) +(5059072800651599,-52) +uh oh! tanf 1.5707964 +-2.2877334e7 +-2.2877332e7 +(-11438667,1) +(-11438666,1) +uh oh! tanf 1.0471976 +1.732051 +1.7320509 +(14529497,-23) +(14529496,-23) +uh oh! tanf 1.0e10 +-0.55834967 +-0.5583496 +(-9367553,-24) +(-9367552,-24) +uh oh! tanf 1.0e20 +0.870492 +-1.2110169 +(14604432,-24) +(-10158746,-23) Done *** unexpected failure for num009(normal) }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15062#comment:4> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#15062: num009 is incredibly platform-sensitive -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.6.1 Component: Test Suite | Version: 8.2.2 Resolution: | Keywords: ci-breakage Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Comment (by Ben Gamari <ben@…>): In [changeset:"5f5d0c9d43bbab922582f437c4a1a3f06ff3fd0e/ghc" 5f5d0c9d/ghc]: {{{ #!CommitTicketReference repository="ghc" revision="5f5d0c9d43bbab922582f437c4a1a3f06ff3fd0e" Mark test broken on powerpc64[le] Test num009 fails different results. #15062 lists more issues on other platforms. Test T14894 fails because DWARF support is not implemented in the PowerPC native code backend. T5435_v_asm_b fails because the runtime linker is not implemented for PowerPC 64-bit systems. Test Plan: validate Reviewers: bgamari, hvr, erikd, simonmar Reviewed By: bgamari Subscribers: rwbarton, thomie, carter GHC Trac Issues: #13634, #11261, #11259, #15062 Differential Revision: https://phabricator.haskell.org/D4825 }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15062#comment:5> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#15062: num009 is incredibly platform-sensitive -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: new Priority: high | Milestone: 8.6.1 Component: Test Suite | Version: 8.2.2 Resolution: | Keywords: ci-breakage Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #2370 | Differential Rev(s): Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * related: => #2370 Comment: See also #2370. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15062#comment:6> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#15062: num009 is incredibly platform-sensitive -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: patch Priority: high | Milestone: 8.6.1 Component: Test Suite | Version: 8.2.2 Resolution: | Keywords: ci-breakage Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #2370 | Differential Rev(s): Phab:D4854 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: new => patch * differential: => Phab:D4854 Comment: I am going to disable this test on 32-bit platforms in the interest of getting 32-bit CircleCI green. See Phab:D4854. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15062#comment:7> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#15062: num009 is incredibly platform-sensitive -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: patch Priority: high | Milestone: 8.6.1 Component: Test Suite | Version: 8.2.2 Resolution: | Keywords: ci-breakage Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #2370 | Differential Rev(s): Phab:D4854 Wiki Page: | -------------------------------------+------------------------------------- Comment (by Ben Gamari <ben@…>): In [changeset:"1ab2dcb04d96e72b4e8e921efd19f8e70d19f8ba/ghc" 1ab2dcb0/ghc]: {{{ #!CommitTicketReference repository="ghc" revision="1ab2dcb04d96e72b4e8e921efd19f8e70d19f8ba" testsuite: Mark num009 as broken due to #15062 Test Plan: Validate Reviewers: hvr Subscribers: rwbarton, thomie, carter GHC Trac Issues: #15062 Differential Revision: https://phabricator.haskell.org/D4854 }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15062#comment:8> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#15062: num009 is incredibly platform-sensitive -------------------------------------+------------------------------------- Reporter: bgamari | Owner: (none) Type: bug | Status: closed Priority: high | Milestone: 8.6.1 Component: Test Suite | Version: 8.2.2 Resolution: fixed | Keywords: ci-breakage Operating System: Unknown/Multiple | Architecture: | Unknown/Multiple Type of failure: None/Unknown | Test Case: Blocked By: | Blocking: Related Tickets: #2370 | Differential Rev(s): Phab:D4854 Wiki Page: | -------------------------------------+------------------------------------- Changes (by bgamari): * status: patch => closed * resolution: => fixed -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/15062#comment:9> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
participants (1)
-
GHC