[GHC] #10152: Testsuite diffs can be misleading when normalisers are in use
#10152: Testsuite diffs can be misleading when normalisers are in use -------------------------------------+------------------------------------- Reporter: ezyang | Owner: Type: task | Status: new Priority: low | Milestone: Component: Test Suite | Version: 7.11 Keywords: | Operating System: Unknown/Multiple Architecture: | Type of failure: None/Unknown Unknown/Multiple | Blocked By: Test Case: | Related Tickets: Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- When output fails to match, GHC will output the diff based on the original outputs. However, this is misleading: some "differences" will actually have been normalised by the whitespace and more substantive normalisers we apply. So you really have to compare the normalised output to see what the /real/ difference is. Unfortunately, the real normalised output is missing newlines, so it's not really suitable for diffing. So it's not really clear what an improved algorithm would be. This seems to not actually be a problem in practice, since SOP for messages which need heavy normalizing is that when the new message looks good, it's just 'make accept'ed. Can be a little confusing when you're developing normalisers. -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10152> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#10152: Testsuite diffs can be misleading when normalisers are in use -------------------------------------+------------------------------------- Reporter: ezyang | Owner: Type: task | Status: new Priority: low | Milestone: Component: Test Suite | Version: 7.11 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: None/Unknown | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by Thomas Miedema <thomasmiedema@…>): In [changeset:"5ddd90415f307cac72d75d86da58e552b168ee30/ghc"]: {{{ #!CommitTicketReference repository="ghc" revision="5ddd90415f307cac72d75d86da58e552b168ee30" Testsuite: diff non-whitespace normalised output (#10152) On a test failure, we show a diff between the expected and the actual output. The method of how we do this has changed a couple of times: * In 2007: 9951189ccf90b709436fa55ee49eeef031f79f4e "On failure, diff the normalised test outputs" * In 2011: 3019b1e409c129ef7af63e6a7408fb36ec44444b "When the output files differ, present the diffs between the *actual* output, not the normalised output. The latter may have newlines removed, making the diff unreadable." * In 2015 (now): do something in between. - Do apply the normalisers again, to make the diff smaller (only showing the actual problem). - But don't apply normalise_whitespace, as it indeed makes the diff unreadable. Differential Revision: https://phabricator.haskell.org/D984 }}} -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10152#comment:1> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
#10152: Testsuite diffs can be misleading when normalisers are in use -------------------------------------+------------------------------------- Reporter: ezyang | Owner: Type: task | Status: closed Priority: low | Milestone: 7.12.1 Component: Test Suite | Version: 7.11 Resolution: fixed | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: None/Unknown | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by thomie): * status: new => closed * resolution: => fixed * milestone: => 7.12.1 -- Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/10152#comment:2> GHC <http://www.haskell.org/ghc/> The Glasgow Haskell Compiler
participants (1)
-
GHC