[Git][ghc/ghc][wip/romes/step-out-11] 57 commits: Specialise: Improve specialisation by refactoring interestingDict

30 Jul 2025

      Rodrigo Mesquita pushed to branch wip/romes/step-out-11 at Glasgow Haskell Compiler / GHC

Commits:
f707bab4 by Andreas Klebinger at 2025-07-12T14:56:16+01:00
Specialise: Improve specialisation by refactoring interestingDict

This MR addresses #26051, which concerns missed type-class specialisation.
The main payload of the MR is to completely refactor the key function
      `interestingDict` in GHC.Core.Opt.Specialise

The main change is that we now also look at the structure of the
dictionary we consider specializing on, rather than only the type.

See the big `Note [Interesting dictionary arguments]`

- - - - -
ca7a9d42 by Simon Peyton Jones at 2025-07-12T14:56:16+01:00
Treat tuple dictionaries uniformly; don't unbox them

See `Note [Do not unbox class dictionaries]` in DmdAnal.hs,
sep (DNB1).

This MR reverses the plan in #23398, which suggested a special case to
unbox tuple dictionaries in worker/wrapper.  But:

- This was the cause of a pile of complexity in the specialiser (#26158)

- Even with that complexity, specialision was still bad, very bad
  See https://gitlab.haskell.org/ghc/ghc/-/issues/19747#note_626297

And it's entirely unnecessary!  Specialision works fine without
unboxing tuple dictionaries.

- - - - -
be7296c9 by Andreas Klebinger at 2025-07-12T14:56:16+01:00
Remove complex special case from the type-class specialiser

There was a pretty tricky special case in Specialise which is no
longer necessary.
   * Historical Note [Floating dictionaries out of cases]
   * #26158
   * #19747 https://gitlab.haskell.org/ghc/ghc/-/issues/19747#note_626297

This MR removes it.  Hooray.

- - - - -
4acf3a86 by Ben Gamari at 2025-07-15T05:46:32-04:00
configure: bump version to 9.15

- - - - -
45efaf71 by Teo Camarasu at 2025-07-15T05:47:13-04:00
rts/nonmovingGC: remove n_free

We remove the nonmovingHeap.n_free variable.

We wanted this to track the length of nonmovingHeap.free.
But this isn't possible to do atomically.

When this isn't accurate we can get a segfault by going past the end of
the list.

Instead, we just count the length of the list when we grab it in
nonmovingPruneFreeSegment.

Resolves #26186

- - - - -
c635f164 by Ben Gamari at 2025-07-15T14:05:54-04:00
configure: Drop probing of ld.gold

As noted in #25716, `gold` has been dropped from binutils-2.44.

Fixes #25716.

Metric Increase:
    size_hello_artifact_gzip
    size_hello_unicode_gzip
    ghc_prim_so

- - - - -
637bb538 by Ben Gamari at 2025-07-15T14:05:55-04:00
testsuite/recomp015: Ignore stderr

This is necessary since ld.bfd complains
that we don't have a .note.GNU-stack section,
potentially resulting in an executable stack.
- - - - -
d3cd4ec8 by Wen Kokke at 2025-07-15T14:06:39-04:00
Fix documentation for heap profile ID

- - - - -
73082769 by Ben Gamari at 2025-07-15T16:56:38-04:00
Bump win32-tarballs to v0.9
- - - - -
3b63b254 by Ben Gamari at 2025-07-15T16:56:39-04:00
rts/LoadArchive: Handle null terminated string tables

As of `llvm-ar` now emits filename tables terminated with null
characters instead of the usual POSIX `/\n` sequence.

Fixes #26150.

- - - - -
195f6527 by Tamar Christina at 2025-07-15T16:56:39-04:00
rts: rename label so name doesn't conflict with param

- - - - -
63373b95 by Tamar Christina at 2025-07-15T16:56:39-04:00
rts: Handle API set symbol versioning conflicts

- - - - -
48e9aa3e by Tamar Christina at 2025-07-15T16:56:39-04:00
rts: Mark API set symbols as HIDDEN and correct symbol type

- - - - -
959e827a by Tamar Christina at 2025-07-15T16:56:39-04:00
rts: Implement WEAK EXTERNAL undef redirection by target symbol name

- - - - -
65f19293 by Ben Gamari at 2025-07-15T16:56:39-04:00
rts/LoadArchive: Handle string table entries terminated with /

llvm-ar appears to terminate string table entries with `/\n` [1]. This
matters in the case of thin archives, since the filename is used. In the
past this worked since `llvm-ar` would produce archives with "small"
filenames when possible. However, now it appears to always use the
string table.

[1] https://github.com/llvm/llvm-project/blob/bfb686bb5ba503e9386dc899e1ebbe2488...

- - - - -
9cbb3ef5 by Ben Gamari at 2025-07-15T16:56:39-04:00
testsuite: Mark T12497 as fixed

Thanks to the LLVM toolchain update.

Closes #22694.

- - - - -
2854407e by Ben Gamari at 2025-07-15T16:56:39-04:00
testsuite: Accept new output of T11223_link_order_a_b_2_fail on Windows

The archive member number changed due to the fact that llvm-ar now uses a
string table.

- - - - -
28439593 by Ben Gamari at 2025-07-15T16:56:39-04:00
rts/linker/PEi386: Implement IMAGE_REL_AMD64_SECREL

This appears to now be used by libc++ as distributed by msys2.

- - - - -
2b053755 by Tamar Christina at 2025-07-15T16:56:39-04:00
rts: Cleanup merge resolution residue in lookupSymbolInDLL_PEi386 and make safe without dependent

- - - - -
e8acd2e7 by Wen Kokke at 2025-07-16T08:37:04-04:00
Remove the `profile_id` parameter from various RTS functions.

Various RTS functions took a `profile_id` parameter, intended to be used to
distinguish parallel heap profile breakdowns (e.g., `-hT` and `-hi`). However,
this feature was never implemented and the `profile_id` parameter was set to 0
throughout the RTS. This commit removes the parameter but leaves the hardcoded
profile ID in the functions that emit the encoded eventlog events as to not
change the protocol.

The affected functions are `traceHeapProfBegin`, `postHeapProfBegin`,
`traceHeapProfSampleString`, `postHeapProfSampleString`,
`traceHeapProfSampleCostCentre`, and `postHeapProfSampleCostCentre`.

- - - - -
76d392a2 by Wen Kokke at 2025-07-16T08:37:04-04:00
Make `traceHeapProfBegin` an init event.

- - - - -
bbaa44a7 by Peng Fan at 2025-07-16T16:50:42-04:00
NCG/LA64: Support finer-grained DBAR hints

For LA664 and newer uarchs, they have made finer granularity hints
available:

    Bit4: ordering or completion (0: completion, 1: ordering)
    Bit3: barrier for previous read (0: true, 1: false)
    Bit2: barrier for previous write (0: true, 1: false)
    Bit1: barrier for succeeding read (0: true, 1: false)
    Bit0: barrier for succeeding write (0: true, 1: false)

And not affect the existing models because other hints are treated
as 'dbar 0' there.

- - - - -
7da86e16 by Andreas Klebinger at 2025-07-16T16:51:25-04:00
Disable -fprof-late-overloaded-calls for join points.

Currently GHC considers cost centres as destructive to
join contexts. Or in other words this is not considered valid:

    join f x = ...
    in
              ... -> scc<tick> jmp

This makes the functionality of `-fprof-late-overloaded-calls` not feasible
for join points in general. We used to try to work around this by putting the
ticks on the rhs of the join point rather than around the jump. However beyond
the loss of accuracy this was broken for recursive join points as we ended up
with something like:

    rec-join f x = scc<tick> ... jmp f x

Which similarly is not valid as the tick once again destroys the tail call.
One might think we could limit ourselves to non-recursive tail calls and do
something clever like:

  join f x = scc<tick> ...
  in ... jmp f x

And sometimes this works! But sometimes the full rhs would look something like:

  join g x = ....
  join f x = scc<tick> ... -> jmp g x

Which, would again no longer be valid. I believe in the long run we can make
cost centre ticks non-destructive to join points. Or we could keep track of
where we are/are not allowed to insert a cost centre. But in the short term I will
simply disable the annotation of join calls under this flag.

- - - - -
7ee22fd5 by ARATA Mizuki at 2025-07-17T06:05:30-04:00
x86 NCG: Better lowering for shuffleFloatX4# and shuffleDoubleX2#

The new implementation

* make use of specialized instructions like (V)UNPCK{L,H}{PS,PD}, and
* do not require -mavx.

Close #26096

Co-authored-by: sheaf <sam.derbyshire@gmail.com>

- - - - -
c6cd2da1 by Jappie Klooster at 2025-07-17T06:06:20-04:00
Update interact docs to explain about buffering

We need to tell the user to set to the
appropriate buffer format.
Otherwise, this function may get randomly stuck,
or just behave confusingly.

issue: https://gitlab.haskell.org/ghc/ghc/-/issues/26131

NB, I'm running this with cabal *NOT* ghci. ghci messes with buffering anyway.

```haskell
interaction :: String -> String
interaction "jappie" = "hi"
interaction "jakob" = "hello"
interaction x = "unkown input: " <> x

main :: IO ()
main = interact  interaction
```
so in my input (prefixed by `>`) I get:

```
...
jappie
unkown input: jappie

we confirmed later this was due to lack of \n matching.
Anyway movnig on to more unexpected stuff:

```haskell
main :: IO ()
main = do
  interact  (concatMap interaction . lines)

get's stuck forever.
actually `^D` (ctrl+d) unstucks it and runs all input as expected.
for example you can get:
```
...
sdfkds
fakdsf
unkown input: sdfkdsunkown input: fakdsf

This program works!
```haskell
interaction :: String -> String
interaction "jappie" = "hi \n"
interaction "jakob" = "hello \n"
interaction x = "unkown input: " &lt;> x &lt;> "\n"

main :: IO ()
main = do
  interact  (concatMap interaction . lines)

the reason is that linebuffering is set for both in and output by default.
so lines eats the input lines, and all the \n postfixes make sure the buffer
is put out.

- - - - -
9fa590a6 by Zubin Duggal at 2025-07-17T06:07:03-04:00
fetch_gitlab: Ensure we copy users_guide.pdf and Haddock.pdf to the release docs directory

Fixes #24093

- - - - -
cc650b4b by Andrew Lelechenko at 2025-07-17T12:30:24-04:00
Add Data.List.NonEmpty.mapMaybe

As per https://github.com/haskell/core-libraries-committee/issues/337

- - - - -
360fa82c by Duncan Coutts at 2025-07-17T12:31:14-04:00
base: Deprecate GHC.Weak.Finalize.runFinalizerBatch

https://github.com/haskell/core-libraries-committee/issues/342

- - - - -
f4e8466c by Alan Zimmerman at 2025-07-17T12:31:55-04:00
EPA: Update exact printing based on GHC 9.14 tests

As a result of migrating the GHC ghc-9.14 branch tests to
ghc-exactprint in
https://github.com/alanz/ghc-exactprint/tree/ghc-9.14, a couple of
discrepancies were picked up

- The opening paren for a DefaultDecl was printed in the wrong place
- The import declaration level specifiers were not printed.

This commit adds those fixes, and some tests for them.
The tests brought to light that the ImportDecl ppr instance had not
been updated for level specifiers, so it updates that too.

- - - - -
8b731e3c by Matthew Pickering at 2025-07-21T13:36:43-04:00
level imports: Fix infinite loop with cyclic module imports

I didn't anticipate that downsweep would run before we checked for
cyclic imports. Therefore we need to use the reachability function which
handles cyclic graphs.

Fixes #26087

- - - - -
d751a9f1 by Pierre Thierry at 2025-07-21T13:37:28-04:00
Fix documentation about deriving from generics

- - - - -
f8d9d016 by Andrew Lelechenko at 2025-07-22T21:13:28-04:00
Fix issues with toRational for types capable to represent infinite and not-a-number values

This commit fixes all of the following pitfalls:
...
toRational (read "Infinity" :: Double)
179769313486231590772930519078902473361797697894230657273430081157732675805500963132708477322407536021120113879871393357658789768814416622492847430639474124377767893424865485276302219601246094119453082952085005768838150682342462881473913110540827237163350510684586298239947245938479716304835356329624224137216 % 1
toRational (read "NaN" :: Double)
269653970229347386159395778618353710042696546841345985910145121736599013708251444699062715983611304031680170819807090036488184653221624933739271145959211186566651840137298227914453329401869141179179624428127508653257226023513694322210869665811240855745025766026879447359920868907719574457253034494436336205824 % 1
...
realToFrac (read "NaN" :: Double) -- With -O0
Infinity
realToFrac (read "NaN" :: Double) -- With -O1
NaN
...
realToFrac (read "NaN" :: Double) :: CDouble
Infinity
realToFrac (read "NaN" :: CDouble) :: Double
Infinity
Implements https://github.com/haskell/core-libraries-committee/issues/338

- - - - -
5dabc718 by Zubin Duggal at 2025-07-22T21:14:10-04:00
haddock: Don't warn about missing link destinations for derived names.

Fixes #26114

- - - - -
9c3a0937 by Matthew Pickering at 2025-07-22T21:14:52-04:00
template haskell: use a precise condition when implicitly lifting

Implicit lifting corrects a level error by replacing references to `x`
with `$(lift x)`, therefore you can use a level `n` binding at level `n
+ 1`, if it can be lifted.

Therefore, we now have a precise check that the use level is 1 more than
the bind level.

Before this bug was not observable as you only had 0 and 1 contexts but
it is easily evident when using explicit level imports.

Fixes #26088

- - - - -
5144b22f by Andreas Klebinger at 2025-07-22T21:15:34-04:00
Add since tag and more docs for do-clever-arg-eta-expansion

Fixes #26113

- - - - -
c865623b by Andreas Klebinger at 2025-07-22T21:15:34-04:00
Add since tag for -fexpose-overloaded-unfoldings

Fixes #26112

- - - - -
49a44ab7 by Simon Hengel at 2025-07-23T17:59:55+07:00
Refactor GHC.Driver.Errors.printMessages

- - - - -
84711c39 by Simon Hengel at 2025-07-23T18:27:34+07:00
Respect `-fdiagnostics-as-json` for error messages from pre-processors

(fixes #25480)

- - - - -
d046b5ab by Simon Hengel at 2025-07-24T06:12:05-04:00
Include the rendered message in -fdiagnostics-as-json output

This implements #26173.

- - - - -
d2b89603 by Ben Gamari at 2025-07-24T06:12:47-04:00
rts/Interpreter: Factor out ctoi tuple info tables into data

Instead of a massive case let's put this into data which we can reuse
elsewhere.

- - - - -
4bc78496 by Sebastian Graf at 2025-07-24T16:19:34-04:00
CprAnal: Detect recursive newtypes (#25944)

While `cprTransformDataConWork` handles recursive data con workers, it
did not detect the case when a newtype is responsible for the recursion.
This is now detected in the `Cast` case of `cprAnal`.

The same reproducer made it clear that `isRecDataCon` lacked congruent
handling for `AppTy` and `CastTy`, now fixed.

Furthermore, the new repro case T25944 triggered this bug via an
infinite loop in `cprFix`, caused by the infelicity in `isRecDataCon`.
While it should be much less likely to trigger such an infinite loop now
that `isRecDataCon` has been fixed, I made sure to abort the loop after
10 iterations and emitting a warning instead.

Fixes #25944.

- - - - -
0a583689 by Sylvain Henry at 2025-07-24T16:20:26-04:00
STM: don't create a transaction in the rhs of catchRetry# (#26028)

We don't need to create a transaction for the rhs of (catchRetry#)
because contrary to the lhs we don't need to abort it on retry. Moreover
it is particularly harmful if we have code such as (#26028):

  let cN = readTVar vN >> retry
      tree = c1 `orElse` (c2 `orElse` (c3 `orElse` ...))
  atomically tree

Because it will stack transactions for the rhss and the read-sets of all
the transactions will be iteratively merged in O(n^2) after the
execution of the most nested retry.

- - - - -
a49eca26 by Simon Peyton Jones at 2025-07-25T09:49:58+01:00
Renaming around predicate types

.. we were (as it turned out) abstracting over
type-class selectors in SPECIALISATION rules!

Wibble isEqPred

- - - - -
f80375dd by Simon Peyton Jones at 2025-07-25T09:49:58+01:00
Refactor of Specialise.hs

This patch just tidies up `specHeader` a bit, removing one
of its many results, and adding some comments.

No change in behaviour.

Also add a few more `HasDebugCallStack` contexts.

- - - - -
1bd12371 by Simon Peyton Jones at 2025-07-25T09:49:58+01:00
Improve treatment of SPECIALISE pragmas -- again!

This MR does another major refactor of the way that SPECIALISE
pragmas work, to fix #26115, #26116, #26117.

* We now /always/ solve forall-constraints in an all-or-nothing way.
  See Note [Solving a Wanted forall-constraint] in GHC.Tc.Solver.Solve

  This means we might have unsolved quantified constraints, which need
  to be reported.  See `inert_insts` in `getUnsolvedInerts`.

* I refactored the short-cut solver for type classes to work by
  recursively calling the solver rather than by having a little baby
  solver that kept being not clever enough.
  See Note [Shortcut solving] in GHC.Tc.Solver.Dict

* I totally rewrote the desugaring of SPECIALISE pragmas, again.
  The new story is in Note [Desugaring new-form SPECIALISE pragmas]
  in GHC.HsToCore.Binds

  Both old-form and new-form SPECIALISE pragmas now route through the same
  function `dsSpec_help`.   The tricky function `decomposeRuleLhs` is now used only
  for user-written RULES, not for SPECIALISE pragmas.

* I improved `solveOneFromTheOther` to account for rewriter sets. Previously
  it would solve a non-rewritten dict from a rewritten one.  For equalities
  we were already dealing with this, in

Some incidental refactoring

* A small refactor: `ebv_tcvs` in `EvBindsBar` now has a list of coercions, rather
  than a set of tyvars.  We just delay taking the free vars.

* GHC.Core.FVs.exprFVs now returns /all/ free vars.
  Use `exprLocalFVs` for Local vars.
  Reason: I wanted another variant for /evidence/ variables.

* Ues `EvId` in preference to `EvVar`.  (Evidence variables are always Ids.)
  Rename `isEvVar` to `isEvId`.

* I moved `inert_safehask` out of `InertCans` and into `InertSet` where it
  more properly belongs.

Compiler-perf changes:
  * There was a palpable bug (#26117) which this MR fixes in
    newWantedEvVar, which bypassed all the subtle overlapping-Given
    and shortcutting logic.  (See the new `newWantedEvVar`.)  Fixing this
    but leads to extra dictionary bindings; they are optimised away quickly
    but they made CoOpt_Read allocate 3.6% more.

  * Hpapily T15164 improves.

  * The net compiler-allocation change is 0.0%

Metric Decrease:
    T15164
Metric Increase:
    CoOpt_Read
    T12425

- - - - -
953fd8f1 by Simon Peyton Jones at 2025-07-25T09:49:58+01:00
Solve forall-constraints immediately, or not at all

This MR refactors the constraint solver to solve forall-constraints immediately,
rather than emitting an implication constraint to be solved later.

The most immediate motivation was that when solving quantified constraints
in SPECIALISE pragmas, we really really don't want to leave behind half-
solved implications.  Also it's in tune with the approach of the new
short-cut solver, which recursively invokes the solver.

It /also/ saves quite a bit of plumbing; e.g

  - The `wl_implics` field of `WorkList` is gone,

  - The types of `solveSimpleWanteds` and friends are simplified.

  - An EvFun contains binding, rather than an EvBindsVar ref-cell that
    will in the future contain bindings.  That makes `evVarsOfTerm`
    simpler.  Much nicer.

It also improves error messages a bit.

All described in Note [Solving a Wanted forall-constraint] in
GHC.Tc.Solver.Solve.

One tiresome point: in the tricky case of `inferConstraintsCoerceBased`
we make a forall-constraint.  This we /do/ want to partially solve, so
we can infer a suitable context. (I'd be quite happy to force the user to
write a context, bt I don't want to change behavior.)  So we want to generate
an /implication/ constraint in `emitPredSpecConstraints` rather than a
/forall-constraint/ as we were doing before.  Discussed in (WFA3) of
the above Note.

Incidental refactoring

* `GHC.Tc.Deriv.Infer.inferConstraints` was consulting the state monad for
  the DerivEnv that the caller had just consulted.  Nicer to pass it as an
  argument I think, so I have done that.  No change in behaviour.

- - - - -
6921ab42 by Simon Peyton Jones at 2025-07-25T09:49:58+01:00
Remove duplicated code in Ast.hs for evTermFreeVars

This is just a tidy up.

- - - - -
1165f587 by Simon Peyton Jones at 2025-07-25T09:49:58+01:00
Small tc-tracing changes only

- - - - -
0776ffe0 by Simon Hengel at 2025-07-26T04:54:20-04:00
Respect `-fdiagnostics-as-json` for core diagnostics (see #24113)

- - - - -
cc1116e0 by Andrew Lelechenko at 2025-07-26T04:55:01-04:00
docs: add since pragma to Data.List.NonEmpty.mapMaybe

- - - - -
9620cae0 by Rodrigo Mesquita at 2025-07-30T11:11:39+01:00
debugger: Uniquely identify breakpoints by internal id

Since b85b11994e0130ff2401dd4bbdf52330e0bcf776 (support inlining
breakpoints), a breakpoint has been identified at runtime by *two* pairs
of <module,index>.

- The first, aka a 'BreakpointId', uniquely identifies a breakpoint in
  the source of a module by using the Tick index. A Tick index can index
  into ModBreaks.modBreaks_xxx to fetch source-level information about
  where that tick originated.

    - When a user specifies e.g. a line breakpoint using :break, we'll reverse
      engineer what a Tick index for that line
    - We update the `BreakArray` of that module (got from the
      LoaderState) at that tick index to `breakOn`.
    - A BCO we can stop at is headed by a BRK_FUN instruction. This
      instruction stores in an operand the `tick index` it is associated
      to. We look it up in the associated `BreakArray` (also an operand)
      and check wheter it was set to `breakOn`.

- The second, aka the `ibi_info_mod` + `ibi_info_ix` of the
  `InternalBreakpointId`, uniquely index into the `imodBreaks_breakInfo`
  -- the information we gathered during code generation about the
  existing breakpoint *ocurrences*.

  - Note that with optimisation there may be many occurrences of the
    same source-tick-breakpoint across different modules. The
    `ibi_info_ix` is unique per occurrence, but the `bi_tick_ix` may be
    shared. See Note [Breakpoint identifiers] about this.

  - Note that besides the tick ids, info ids are also stored in
    `BRK_FUN` so the break handler can refer to the associated
    `CgBreakInfo`.

In light of that, the driving changes come from the desire to have the
info_id uniquely identify the breakpoint at runtime, and the source tick
id being derived from it:

- An InternalBreakpointId should uniquely identify a breakpoint just
  from the code-generation identifiers of `ibi_info_ix` and `ibi_info_mod`.
  So we drop `ibi_tick_mod` and `ibi_tick_ix`.
- A BRK_FUN instruction need only record the "internal breakpoint id",
  not the tick-level id.
  So we drop the tick mod and tick index operands.
- A BreakArray should be indexed by InternalBreakpointId rather than
  BreakpointId

That means we need to do some more work when setting a breakpoint.
Specifically, we need to figure out the internal ids (occurrences of a
breakpoint) from the source-level BreakpointId we want to set the
breakpoint at (recall :break refers to breaks at the source level).

Besides this change being an improvement to the handling of breakpoints
(it's clearer to have a single unique identifier than two competing
ones), it unlocks the possibility of generating "internal" breakpoints
during Cg (needed for #26042).
It should also be easier to introduce multi-threaded-aware `BreakArrays`
following this change (needed for #26064).

Se also the new Note [ModBreaks vs InternalModBreaks]

- - - - -
586e7ea0 by Rodrigo Mesquita at 2025-07-30T11:11:39+01:00
cleanup: Move dehydrateCgBreakInfo to Stg2Bc

This no longer has anything to do with Core.

- - - - -
09f39496 by Rodrigo Mesquita at 2025-07-30T11:11:39+01:00
rts/Disassembler: Fix spacing of BRK_FUN

- - - - -
519b71e2 by Rodrigo Mesquita at 2025-07-30T11:11:40+01:00
debugger: Fix bciPtr in Step-out

We need to use `BCO_NEXT` to move bciPtr to ix=1, because ix=0 points to
the instruction itself!

I do not understand how this didn't crash before.

- - - - -
b8ed860a by Rodrigo Mesquita at 2025-07-30T11:11:40+01:00
debugger: Allow BRK_FUNs to head case continuation BCOs

When we start executing a BCO, we may want to yield to the scheduler:
this may be triggered by a heap/stack check, context switch, or a
breakpoint. To yield, we need to put the stack in a state such that
when execution is resumed we are back to where we yielded from.

Previously, a BKR_FUN could only head a function BCO because we only
knew how to construct a valid stack for yielding from one -- simply add
`apply_interp_info` + the BCO to resume executing. This is valid because
the stack at the start of run_BCO is headed by that BCO's arguments.

However, in case continuation BCOs (as per Note [Case continuation BCOs]),
we couldn't easily reconstruct a valid stack that could be resumed
because we dropped too soon the stack frames regarding the value
returned (stg_ret) and received (stg_ctoi) by that continuation.
This is especially tricky because of the variable type and size return
frames (e.g. pointer ret_p/ctoi_R1p vs a tuple ret_t/ctoi_t2).

The trick to being able to yield from a BRK_FUN at the start of a case
cont BCO is to stop removing the ret frame headers eagerly and instead
keep them until the BCO starts executing. The new layout at the start of
a case cont. BCO is described by the new Note [Stack layout when entering run_BCO].

Now, we keep the ret_* and ctoi_* frames when entering run_BCO.
A BRK_FUN is then executed if found, and the stack is yielded as-is with
the preserved ret and ctoi frames.
Then, a case cont BCO's instructions always SLIDE off the headers of the
ret and ctoi frames, in StgToByteCode.doCase, turning a stack like

   |     ....      |
   +---------------+
   |     fv2       |
   +---------------+
   |     fv1       |
   +---------------+
   |     BCO       |
   +---------------+
   | stg_ctoi_ret_ |
   +---------------+
   |    retval     |
   +---------------+
   | stg_ret_..... |
   +---------------+

into

   |     ....      |
   +---------------+
   |     fv2       |
   +---------------+
   |     fv1       |
   +---------------+
   |    retval     |
   +---------------+

for the remainder of the BCO.

Moreover, this more uniform approach of keeping the ret and ctoi frames
means we need less ad-hoc logic concerning the variable size of
ret_tuple vs ret_p/np frames in the code generator and interpreter:
Always keep the return to cont. stack intact at the start of run_BCO,
and the statically generated instructions will take care of adjusting
it.

Unlocks BRK_FUNs at the start of case cont. BCOs which will enable a
better user-facing step-out (#26042) which is free of the bugs the
current BRK_ALTS implementation suffers from (namely, using BRK_FUN
rather than BRK_ALTS in a case cont. means we'll never accidentally end
up in a breakpoint "deeper" than the continuation, because we stop at
the case cont itself rather than on the first breakpoint we evaluate
after it).

- - - - -
bed90fb6 by Rodrigo Mesquita at 2025-07-30T11:11:40+01:00
BRK_FUN with InternalBreakLocs for code-generation time breakpoints

At the start of a case continuation BCO, place a BRK_FUN.
This BRK_FUN uses the new "internal breakpoint location" -- allowing us
to come up with a valid source location for this breakpoint that is not associated with a source-level tick.

For case continuation BCOs, we use the last tick seen before it as the
source location. The reasoning is described in Note [Debugger: Stepout internal break locs].

Note how T26042c, which was broken because it displayed the incorrect
behavior of the previous step out when we'd end up at a deeper level
than the one from which we initiated step-out, is now fixed.

As of this commit, BRK_ALTS is now dead code and is thus dropped.

Note [Debugger: Stepout internal break locs]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Step-out tells the interpreter to run until the current function
returns to where it was called from, and stop there.

This is achieved by enabling the BRK_FUN found on the first RET_BCO
frame on the stack (See [Note Debugger: Step-out]).

Case continuation BCOs (which select an alternative branch) must
therefore be headed by a BRK_FUN. An example:

    f x = case g x of <--- end up here
        1 -> ...
        2 -> ...

    g y = ... <--- step out from here

- `g` will return a value to the case continuation BCO in `f`
- The case continuation BCO will receive the value returned from g
- Match on it and push the alternative continuation for that branch
- And then enter that alternative.

If we step-out of `g`, the first RET_BCO on the stack is the case
continuation of `f` -- execution should stop at its start, before
selecting an alternative. (One might ask, "why not enable the breakpoint
in the alternative instead?", because the alternative continuation is
only pushed to the stack *after* it is selected by the case cont. BCO)

However, the case cont. BCO is not associated with any source-level
tick, it is merely the glue code which selects alternatives which do
have source level ticks. Therefore, we have to come up at code
generation time with a breakpoint location ('InternalBreakLoc') to
display to the user when it is stopped there.

Our solution is to use the last tick seen just before reaching the case
continuation. This is robust because a case continuation will thus
always have a relevant breakpoint location:

    - The source location will be the last source-relevant expression
      executed before the continuation is pushed

    - So the source location will point to the thing you've just stepped
      out of

    - Doing :step-local from there will put you on the selected
      alternative (which at the source level may also be the e.g. next
      line in a do-block)

Examples, using angle brackets (<<...>>) to denote the breakpoint span:

    f x = case <<g x>> {- step in here -} of
        1 -> ...
        2 -> ...>

    g y = <<...>> <--- step out from here

    ...

    f x = <<case g x of <--- end up here, whole case highlighted
        1 -> ...
        2 -> ...>>

    doing :step-local ...

    f x = case g x of
        1 -> <<...>> <--- stop in the alternative
        2 -> ...

A second example based on T26042d2, where the source is a do-block IO
action, optimised to a chain of `case expressions`.

    main = do
      putStrLn "hello1"
      <<f>> <--- step-in here
      putStrLn "hello3"
      putStrLn "hello4"

    f = do
      <<putStrLn "hello2.1">> <--- step-out from here
      putStrLn "hello2.2"

    ...

    main = do
      putStrLn "hello1"
      <<f>> <--- end up here again, the previously executed expression
      putStrLn "hello3"
      putStrLn "hello4"

    doing step/step-local ...

    main = do
      putStrLn "hello1"
      f
      <<putStrLn "hello3">> <--- straight to the next line
      putStrLn "hello4"

Finishes #26042

- - - - -
78f06878 by Rodrigo Mesquita at 2025-07-30T11:11:40+01:00
bytecode: Don't PUSH_L 0; SLIDE 1 1

While looking through bytecode I noticed a quite common unfortunate
pattern:

...
PUSH_L 0
SLIDE 1 1

We do this often by generically constructing a tail call from a function
atom that may be somewhere arbitrary on the stack.
However, for the special case that the function can be found directly on
top of the stack, as part of the arguments, it's plain redundant to push
then slide it.

In this commit we add a small optimisation to the generation of
tailcalls in bytecode. Simply: lookahead for the function in the stack.
If it is the first thing on the stack and it is part of the arguments
which would be dropped as we entered the tail call, then don't push then
slide it.

In a simple example (T26042b), this already produced a drastic
improvement in generated code (left is old, right is with this patch):

```diff
3c3
< 2025-07-29 10:14:02.081277 UTC
---
...
2025-07-29 10:50:36.560949 UTC
160,161c160
<                                            PUSH_L   0
<                                            SLIDE    1 2

...
SLIDE    1 1
164,165d162
<                                       PUSH_L   0
<                                       SLIDE    1 1
175,176c172
<                             PUSH_L   0
<                             SLIDE    1 2
---
...
SLIDE    1 1
179,180d174
<                        PUSH_L   0
<                        SLIDE    1 1
206,207d199
<                        PUSH_L   0
<                        SLIDE    1 1
210,211d201
<                   PUSH_L   0
<                   SLIDE    1 1
214,215d203
<              PUSH_L   0
<              SLIDE    1 1
218,219d205
<         PUSH_L   0
<         SLIDE    1 1
222,223d207
<    PUSH_L   0
<    SLIDE    1 1
...
600,601c566
<                                                 PUSH_L   0
<                                                 SLIDE    1 2
---
...
SLIDE    1 1
604,605d568
<                                            PUSH_L   0
<                                            SLIDE    1 1
632,633d594
<                                            PUSH_L   0
<                                            SLIDE    1 1
636,637d596
<                                       PUSH_L   0
<                                       SLIDE    1 1
640,641d598
<                                  PUSH_L   0
<                                  SLIDE    1 1
644,645d600
<                             PUSH_L   0
<                             SLIDE    1 1
648,649d602
<                        PUSH_L   0
<                        SLIDE    1 1
652,653d604
<                   PUSH_L   0
<                   SLIDE    1 1
656,657d606
<              PUSH_L   0
<              SLIDE    1 1
660,661d608
<         PUSH_L   0
<         SLIDE    1 1
664,665d610
<    PUSH_L   0
<    SLIDE    1 1
```

I also compiled lib:Cabal to bytecode and counted the number of bytecode
lines with `find dist-newstyle -name "*.dump-BCOs" -exec wc {} +`:

    with unoptimized core:
    1190689 lines (before) - 1172891 lines (now)
    = 17798 less redundant instructions (-1.5%)

    with optimized core:
    1924818 lines (before) - 1864836 lines (now)
    = 59982 less redundant instructions (-3.1%)

- - - - -

217 changed files:

- .gitlab/rel_eng/fetch-gitlab-artifacts/fetch_gitlab.py
- compiler/GHC/ByteCode/Asm.hs
- compiler/GHC/ByteCode/Breakpoints.hs
- compiler/GHC/ByteCode/Instr.hs
- compiler/GHC/ByteCode/Linker.hs
- compiler/GHC/ByteCode/Types.hs
- compiler/GHC/CmmToAsm/LA64/CodeGen.hs
- compiler/GHC/CmmToAsm/LA64/Instr.hs
- compiler/GHC/CmmToAsm/LA64/Ppr.hs
- compiler/GHC/CmmToAsm/X86/CodeGen.hs
- compiler/GHC/CmmToAsm/X86/Instr.hs
- compiler/GHC/CmmToAsm/X86/Ppr.hs
- compiler/GHC/Core.hs
- compiler/GHC/Core/FVs.hs
- compiler/GHC/Core/LateCC/OverloadedCalls.hs
- compiler/GHC/Core/Make.hs
- compiler/GHC/Core/Opt/Arity.hs
- compiler/GHC/Core/Opt/CprAnal.hs
- compiler/GHC/Core/Opt/DmdAnal.hs
- compiler/GHC/Core/Opt/Monad.hs
- compiler/GHC/Core/Opt/SpecConstr.hs
- compiler/GHC/Core/Opt/Specialise.hs
- compiler/GHC/Core/Opt/WorkWrap/Utils.hs
- compiler/GHC/Core/Predicate.hs
- compiler/GHC/Core/Rules.hs
- compiler/GHC/Core/Subst.hs
- compiler/GHC/Core/TyCo/Rep.hs
- compiler/GHC/Core/Unfold/Make.hs
- compiler/GHC/Core/Utils.hs
- compiler/GHC/CoreToIface.hs
- compiler/GHC/Driver/Errors.hs
- compiler/GHC/Hs/ImpExp.hs
- compiler/GHC/HsToCore/Binds.hs
- compiler/GHC/HsToCore/Breakpoints.hs
- compiler/GHC/HsToCore/Errors/Ppr.hs
- compiler/GHC/HsToCore/Pmc/Solver/Types.hs
- compiler/GHC/Iface/Ext/Ast.hs
- compiler/GHC/Linker/Loader.hs
- compiler/GHC/Parser.y
- compiler/GHC/Rename/Splice.hs
- compiler/GHC/Runtime/Debugger/Breakpoints.hs
- compiler/GHC/Runtime/Eval.hs
- compiler/GHC/Runtime/Interpreter.hs
- compiler/GHC/StgToByteCode.hs
- compiler/GHC/SysTools/Process.hs
- compiler/GHC/Tc/Deriv.hs
- compiler/GHC/Tc/Deriv/Infer.hs
- compiler/GHC/Tc/Deriv/Utils.hs
- compiler/GHC/Tc/Errors.hs
- compiler/GHC/Tc/Errors/Ppr.hs
- compiler/GHC/Tc/Gen/App.hs
- compiler/GHC/Tc/Gen/Expr.hs
- compiler/GHC/Tc/Gen/Pat.hs
- compiler/GHC/Tc/Gen/Sig.hs
- compiler/GHC/Tc/Solver.hs
- compiler/GHC/Tc/Solver/Default.hs
- compiler/GHC/Tc/Solver/Dict.hs
- compiler/GHC/Tc/Solver/Equality.hs
- compiler/GHC/Tc/Solver/InertSet.hs
- compiler/GHC/Tc/Solver/Monad.hs
- compiler/GHC/Tc/Solver/Rewrite.hs
- compiler/GHC/Tc/Solver/Solve.hs
- + compiler/GHC/Tc/Solver/Solve.hs-boot
- compiler/GHC/Tc/Solver/Types.hs
- compiler/GHC/Tc/TyCl/PatSyn.hs
- compiler/GHC/Tc/Types/Constraint.hs
- compiler/GHC/Tc/Types/Evidence.hs
- compiler/GHC/Tc/Types/Origin.hs
- compiler/GHC/Tc/Utils/Monad.hs
- compiler/GHC/Tc/Utils/TcMType.hs
- compiler/GHC/Tc/Utils/TcType.hs
- compiler/GHC/Tc/Utils/Unify.hs
- compiler/GHC/Tc/Validity.hs
- compiler/GHC/Tc/Zonk/Type.hs
- compiler/GHC/Types/Error.hs
- compiler/GHC/Types/Var.hs
- compiler/GHC/Unit/Module/Graph.hs
- compiler/GHC/Utils/Logger.hs
- configure.ac
- docs/users_guide/9.14.1-notes.rst
- + docs/users_guide/diagnostics-as-json-schema-1_2.json
- docs/users_guide/eventlog-formats.rst
- docs/users_guide/profiling.rst
- docs/users_guide/using-optimisation.rst
- docs/users_guide/using.rst
- ghc/GHCi/UI.hs
- ghc/GHCi/UI/Monad.hs
- libraries/base/changelog.md
- libraries/base/src/Data/List/NonEmpty.hs
- libraries/base/src/GHC/Generics.hs
- libraries/base/src/GHC/Weak/Finalize.hs
- libraries/ghc-internal/src/GHC/Internal/Float.hs
- libraries/ghc-internal/src/GHC/Internal/Real.hs
- libraries/ghc-internal/src/GHC/Internal/System/IO.hs
- libraries/ghci/GHCi/Debugger.hs
- libraries/ghci/GHCi/Message.hs
- libraries/ghci/GHCi/Run.hs
- m4/find_ld.m4
- mk/get-win32-tarballs.py
- rts/Disassembler.c
- rts/Exception.cmm
- rts/Interpreter.c
- rts/PrimOps.cmm
- rts/ProfHeap.c
- rts/Profiling.c
- rts/RaiseAsync.c
- rts/RetainerSet.c
- rts/STM.c
- rts/Trace.c
- rts/Trace.h
- rts/eventlog/EventLog.c
- rts/eventlog/EventLog.h
- rts/include/rts/Bytecodes.h
- rts/linker/LoadArchive.c
- rts/linker/PEi386.c
- rts/sm/NonMoving.c
- rts/sm/NonMoving.h
- rts/sm/NonMovingAllocate.c
- rts/sm/Sanity.c
- testsuite/tests/count-deps/CountDepsAst.stdout
- testsuite/tests/count-deps/CountDepsParser.stdout
- + testsuite/tests/cpranal/sigs/T25944.hs
- + testsuite/tests/cpranal/sigs/T25944.stderr
- testsuite/tests/cpranal/sigs/all.T
- testsuite/tests/deriving/should_compile/T20815.hs
- testsuite/tests/deriving/should_fail/T12768.stderr
- testsuite/tests/deriving/should_fail/T1496.stderr
- testsuite/tests/deriving/should_fail/T5498.stderr
- testsuite/tests/deriving/should_fail/T7148.stderr
- testsuite/tests/deriving/should_fail/T7148a.stderr
- testsuite/tests/dmdanal/should_compile/T23398.hs
- testsuite/tests/dmdanal/should_compile/T23398.stderr
- testsuite/tests/driver/json.stderr
- testsuite/tests/driver/json_warn.stderr
- testsuite/tests/driver/recomp015/all.T
- testsuite/tests/ghci.debugger/scripts/T26042b.stdout
- testsuite/tests/ghci.debugger/scripts/T26042c.script
- testsuite/tests/ghci.debugger/scripts/T26042c.stdout
- + testsuite/tests/ghci.debugger/scripts/T26042d2.hs
- + testsuite/tests/ghci.debugger/scripts/T26042d2.script
- + testsuite/tests/ghci.debugger/scripts/T26042d2.stdout
- testsuite/tests/ghci.debugger/scripts/T26042e.stdout
- testsuite/tests/ghci.debugger/scripts/T26042f2.stdout
- testsuite/tests/ghci.debugger/scripts/T26042g.stdout
- testsuite/tests/ghci.debugger/scripts/all.T
- testsuite/tests/haddock/haddock_testsuite/Makefile
- + testsuite/tests/haddock/haddock_testsuite/T26114.hs
- + testsuite/tests/haddock/haddock_testsuite/T26114.stdout
- testsuite/tests/haddock/haddock_testsuite/all.T
- testsuite/tests/hiefile/should_run/HieQueries.stdout
- testsuite/tests/impredicative/T17332.stderr
- testsuite/tests/interface-stability/base-exports.stdout
- testsuite/tests/interface-stability/base-exports.stdout-javascript-unknown-ghcjs
- testsuite/tests/interface-stability/base-exports.stdout-mingw32
- testsuite/tests/interface-stability/base-exports.stdout-ws-32
- + testsuite/tests/lib/stm/T26028.hs
- + testsuite/tests/lib/stm/T26028.stdout
- + testsuite/tests/lib/stm/all.T
- testsuite/tests/numeric/should_run/T9810.stdout
- + testsuite/tests/perf/should_run/SpecTyFamRun.hs
- + testsuite/tests/perf/should_run/SpecTyFamRun.stdout
- + testsuite/tests/perf/should_run/SpecTyFam_Import.hs
- testsuite/tests/perf/should_run/all.T
- testsuite/tests/printer/Makefile
- + testsuite/tests/printer/TestLevelImports.hs
- + testsuite/tests/printer/TestNamedDefaults.hs
- testsuite/tests/printer/all.T
- testsuite/tests/quantified-constraints/T15290a.stderr
- testsuite/tests/quantified-constraints/T19690.stderr
- testsuite/tests/quantified-constraints/T19921.stderr
- testsuite/tests/quantified-constraints/T21006.stderr
- testsuite/tests/roles/should_fail/RolesIArray.stderr
- testsuite/tests/rts/all.T
- testsuite/tests/rts/linker/T11223/T11223_link_order_a_b_2_fail.stderr-ws-32-mingw32
- testsuite/tests/rts/linker/T11223/T11223_link_order_a_b_2_fail.stderr-ws-64-mingw32
- testsuite/tests/simd/should_run/all.T
- + testsuite/tests/simd/should_run/doublex2_shuffle.hs
- + testsuite/tests/simd/should_run/doublex2_shuffle.stdout
- + testsuite/tests/simd/should_run/doublex2_shuffle_baseline.hs
- + testsuite/tests/simd/should_run/doublex2_shuffle_baseline.stdout
- + testsuite/tests/simd/should_run/floatx4_shuffle.hs
- + testsuite/tests/simd/should_run/floatx4_shuffle.stdout
- + testsuite/tests/simd/should_run/floatx4_shuffle_baseline.hs
- + testsuite/tests/simd/should_run/floatx4_shuffle_baseline.stdout
- + testsuite/tests/simplCore/should_compile/T26051.hs
- + testsuite/tests/simplCore/should_compile/T26051.stderr
- + testsuite/tests/simplCore/should_compile/T26051_Import.hs
- + testsuite/tests/simplCore/should_compile/T26115.hs
- + testsuite/tests/simplCore/should_compile/T26115.stderr
- + testsuite/tests/simplCore/should_compile/T26116.hs
- + testsuite/tests/simplCore/should_compile/T26116.stderr
- + testsuite/tests/simplCore/should_compile/T26117.hs
- + testsuite/tests/simplCore/should_compile/T26117.stderr
- testsuite/tests/simplCore/should_compile/all.T
- + testsuite/tests/splice-imports/T26087.stderr
- + testsuite/tests/splice-imports/T26087A.hs
- + testsuite/tests/splice-imports/T26087B.hs
- + testsuite/tests/splice-imports/T26088.stderr
- + testsuite/tests/splice-imports/T26088A.hs
- + testsuite/tests/splice-imports/T26088B.hs
- testsuite/tests/splice-imports/all.T
- testsuite/tests/typecheck/should_compile/T12427a.stderr
- testsuite/tests/typecheck/should_compile/T23171.hs
- testsuite/tests/typecheck/should_compile/TcSpecPragmas.stderr
- testsuite/tests/typecheck/should_fail/T14605.hs
- testsuite/tests/typecheck/should_fail/T14605.stderr
- testsuite/tests/typecheck/should_fail/T15801.stderr
- testsuite/tests/typecheck/should_fail/T18640a.stderr
- testsuite/tests/typecheck/should_fail/T18640b.stderr
- testsuite/tests/typecheck/should_fail/T19627.stderr
- testsuite/tests/typecheck/should_fail/T21530b.stderr
- testsuite/tests/typecheck/should_fail/T22912.stderr
- testsuite/tests/typecheck/should_fail/tcfail174.stderr
- utils/check-exact/ExactPrint.hs
- utils/ghc-toolchain/src/GHC/Toolchain/Tools/Link.hs
- utils/haddock/haddock-api/haddock-api.cabal
- utils/haddock/haddock-api/src/Haddock/Interface/Rename.hs

The diff was not included because it is too large.

View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/8084c422f331ab5bbd3617fbce6e706...

-- 
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/8084c422f331ab5bbd3617fbce6e706...
You're receiving this email because of your account on gitlab.haskell.org.

Rodrigo Mesquita (＠alt-romes)

tags

participants (1)