Simon Jakobi pushed to branch wip/sjakobi/T16720 at Glasgow Haskell Compiler / GHC

Commits:

• 0cc5a523
  by Simon Jakobi at 2026-07-30T16:40:43+02:00

  testsuite: Note that dropping redundant output also affects the JUnit report
  
  The mismatching stream is dropped from the TestResult at construction, so
  the JUnit XML now carries only the diff where it previously also had the
  raw stream. Record in Note [Redundant output in test results] that this
  is intended, and switch the Note to the usual comment style.
  
  Assisted-by: Claude Fable 5
  

• 997fbe2e
  by Simon Jakobi at 2026-07-30T16:45:27+02:00

  testsuite: Don't crash on non-UTF-8 test output in failure results
  
  read_stdout, read_stderr_for and read_comp_stderr decoded with strict
  UTF-8, so a test emitting invalid UTF-8 (binary output, or a crash
  truncating a multi-byte character) raised UnicodeDecodeError and was
  reported as a framework failure. Newly reachable from the bad-exit-code
  branch of simple_run, which now attaches the captured output to the
  result even under -v0 and for expect-fail ways.
  
  Decode with errors='replace', like read_no_crs and safe_print.
  
  Assisted-by: Claude Fable 5
  

1 changed file:

• testsuite/driver/testlib.py

Changes:

 def read_stdout( name: TestName ) -> str:

     path = in_testdir(name, 'run.stdout')

     if path.exists():

-        return path.read_text(encoding='UTF-8')

+        return path.read_text(encoding='UTF-8', errors='replace')

     else:

         return ''

 def read_comp_stderr( name: TestName ) -> str:

     path = in_testdir(name, 'comp.stderr')

     if path.exists():

-        return path.read_text(encoding='UTF-8')

+        return path.read_text(encoding='UTF-8', errors='replace')

     else:

         return ''

 def read_stderr_for( phase: str, name: TestName ) -> str:

     path = in_testdir(name, phase + '.stderr')

     if path.exists():

-        return path.read_text(encoding='UTF-8')

+        return path.read_text(encoding='UTF-8', errors='replace')

     else:

         return ''

 # would drown out the summary.

 MAX_SUMMARY_OUTPUT_TESTS = 20

-"""

-Note [Redundant output in test results]

-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

-A failing test result carries up to three pieces of output: `diff`, `stdout`

-and `stderr`. For an output mismatch these overlap: the diff's `+` lines are

-the very stream that mismatched, normalised. Reporting both would print the

-same text twice, so the mismatching stream is dropped at the call sites in

-favour of the diff, which additionally shows what was expected. The *other*

-stream is kept: on a stdout mismatch, stderr is independent context.

-

-The drop is conditional on there being a diff at all: compare_outputs only

-runs `diff` when config.verbose >= 1, so under -v0 the stream is the only

-output there is.

-"""

+# Note [Redundant output in test results]

+# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

+# A failing test result carries up to three pieces of output: `diff`, `stdout`

+# and `stderr`. For an output mismatch these overlap: the diff's `+` lines are

+# the very stream that mismatched, normalised. Reporting both would print the

+# same text twice, so the mismatching stream is dropped at the call sites in

+# favour of the diff, which additionally shows what was expected. The *other*

+# stream is kept: on a stdout mismatch, stderr is independent context.

+#

+# The drop is conditional on there being a diff at all: compare_outputs only

+# runs `diff` when config.verbose >= 1, so under -v0 the stream is the only

+# output there is.

+#

+# Note that, since the drop happens at result construction, it also affects the

+# JUnit report (junit.py).

 def strip_diff_header(diff: Optional[str]) -> Optional[str]:

     # Drop diff(1)'s ---/+++ lines: they name normalised files in the test