Alan Zimmerman pushed to branch wip/az/exactprint-annotation-rationalisation at Glasgow Haskell Compiler / GHC Commits: c130188d by Rodrigo Mesquita at 2026-08-15T06:29:53-04:00 loopImports: Don't dup ms_uid in summary imports We were writing the ms_unitid of the mod summary with every single import of that module That complicated the code (as though the UnitId in that list could ever be something else) and also allocates unnecessarily per every mod import. Very slight allocation decrease measured locally in a few tests: (MultiComponentModulesRecomp: -0.06%; MultiComponentModulesRecomp100: -0.05%) Purely a clean up. - - - - - c71166a8 by Rodrigo Mesquita at 2026-08-15T06:29:53-04:00 downsweep: make control flow simpler and cache correct This refactor extracts the control flow of downsweep into a single function `dfsBuild`, which takes care of iteratively expanding and traversing all nodes of the in-construction module graph necessary to build a full `ModuleGraph`. There are three levels of caching going on, all of which are necessary to make sure we don't do repeated work (notably, NEVER summarise the same module twice). 1. `dfsBuild` accumulates the final module graph and never revisits the same node of the module graph. Cache is keyed by the final `ModuleGraph`s `NodeKey`s. 2. For Module A in home-unit u1, each import in the list of imports needs to be *found* (call to `findImportedModuleWithIsBoot`): at this point, we only have the `ModuleName` of the import, not the `Module`. This *finding* is somewhat expensive, so we cache it as well (`ImportsCache`). The cache key is the home-unit to which the module belongs~[1], the import package qualifier, and the ModuleName. [1] Different home-units will have different package flags, which means potentially different `Module` resolution for the same `ModuleName`. 3. The most expensive operation we want to avoid is summarising a `Module` into a `ModSummary`, which notably involves parsing the module header from scratch. The third cache, in essence, maps a `Module` to its `ModSummary` (named `ModSummaryCache`). This cache upholds the invariant: we NEVER summarise the same module twice. In practice, the cache key is the Module's UnitId and the Source path; the reason is we need to distinguish between `.hs` and `.hs-boot` files, as their summaries will differ. Note that (2) can't guarantee this alone: Two ModuleName imports in separate units can (and likely do) map to the same `Module`. Note that the previous implementation failed to achieve the no-duplicate-work summarisation invariant, and we ended up doing a quadratic amount of processing in scenarios like test `MultiComponentModules100`. See also Note [Downsweep Control Flow and Caching] Fixes #27461 Perf changes: MultiComponentModules(normal) ghc/alloc 2,097,389,264 1,992,186,736 -5.0% GOOD MultiComponentModules100(normal) ghc/alloc 24,310,173,770 21,293,867,360 -12.4% GOOD MultiComponentModulesRecomp(normal) ghc/alloc 602,761,394 498,543,984 -17.3% GOOD MultiComponentModulesRecomp100(normal) ghc/alloc 11,885,968,240 8,895,404,864 -25.2% GOOD ------------------------- Metric Decrease: MultiComponentModules MultiComponentModules100 MultiComponentModulesRecomp MultiComponentModulesRecomp100 ------------------------- - - - - - ebc4047b by Rodrigo Mesquita at 2026-08-15T06:29:53-04:00 implicitRequirementsShallow can never reach HoleUnit findImportedModule will never return `HoleUnit` for a `ModuleName` (a `HoleUnit` can only be found as a signature instantiation, never as a directly *imported* thing) Therefore, we can drop `[ModuleName]` returned by `implicitRequirementsShallow`, which makes many things dead code. Namely, the call to `implicitRequirementsShallow` from GHC.Driver.Downsweep which was a performance bottleneck (for doing lots of duplicate work in findImportedModule) is now entirely gone. Fixes #27053 In an MR with this patch and the downsweep refactor (previous commit), CI says: MultiComponentModules(normal) ghc/alloc 2,097,396,728 1,943,662,304 -7.3% GOOD MultiComponentModules100(normal) ghc/alloc 24,310,182,136 17,227,574,440 -29.1% GOOD MultiComponentModulesRecomp(normal) ghc/alloc 602,769,518 449,973,656 -25.3% GOOD MultiComponentModulesRecomp100(normal) ghc/alloc 11,885,976,408 4,828,894,160 -59.4% GOOD ------------------------- Metric Decrease: MultiComponentModules MultiComponentModules100 MultiComponentModulesRecomp MultiComponentModulesRecomp100 ------------------------- - - - - - 85a6ab01 by Rodrigo Mesquita at 2026-08-15T06:29:53-04:00 downsweep: Cache negative results When traversing a module graph structure, a uniquely identified node should always expand to the same thing. I don't see how visiting the same node which failed to be expanded a first time would ever successfully expand the second time we try to expand it (eg. when coming from a different edge to it -- it is still the same node!). The node expansion is local, based just based on the node itself, not on the path to get there. Therefore, this patch removes the weird behavior and commentary of `dfsBuild` wrt to `Nothing` not being cached and being potentially expanded a second time around to something different, which was misleading and, ultimately, incorrect. Now, we have a `MGRes`, which is more explicit about a node being Skipped just being a node that is ignored whenever it is found (and that skip is cached) -- and we may want to do this due to failures or due to just trying nodes which might not work on purpose, like hs-boots. We uniformly cache positive and negative results and remove the assumption that there might be an ordering in which the same node visited at a later time might be expanded differently. This makes it possible to traverse the module nodes in parallel without a change in behavior, since there's no longer a hidden ordering requirement. - - - - - 56747c3f by Rodrigo Mesquita at 2026-08-15T06:29:53-04:00 Organize and clean-up GHC.Driver.Downsweep Simply some cosmetic changes, moving definitions around to structure the module better into its relevant sections (In go (ns ++ ss), it's not a problem to use ++ because it's a good producer and we won't have to append fully before processing the next item in go) - - - - - 12f64118 by Wolfgang Jeltsch at 2026-08-15T06:31:12-04:00 Add support for textual output of bytecode file content - - - - - a737df91 by Brandon Chinn at 2026-08-15T12:40:25-04:00 Add law in qualified strings doc - - - - - e3188581 by Zubin Duggal at 2026-08-15T12:41:06-04:00 DmdAnal: Fix maxDmdType We need to eta expand the smaller DmdType using defaultArgDmd, like in lubDmdType. Introduce zipDmdType as a common combinator to implement both maxDmdType and lubDmdType uniformly. fixes #27626 - - - - - ca9b0b22 by mangoiv at 2026-08-15T12:41:47-04:00 hadrian: set the executable bit for hie-bios.bat - - - - - 1aac7095 by sheaf at 2026-08-16T04:37:16-04:00 Avoid wasteful allocations in mkTyConAppCo The idiom "traverse isReflCo_maybe" followed by "map fst" used in 'GHC.Core.Coercion.mkTyConAppCo' was allocating a lot of waste. This commit uses 'GHC.Data.Unboxed.traverseMaybeUB' to avoid all these unnecessary intermediate allocations. In a quick microbenchmark for 'mkTyConAppCo', this change resulted in: - refl case (all argument coercions are reflexive): - -60% runtime - -80% allocations - non-refl case: - from 0% to -12% runtime (depending on which argument is non-refl) - from 0% to -70% allocations ( -- '' -- ) Fixes #27648 ------------------------- Metric Decrease: FamAppCachePerf SimplCastPerf T12425 T15703 T26426 T3064 T9872a T9872b T9872b_defer T9872c T9872d T5321Fun T9020 T9630 TcPlugin_RewritePerf Metric Increase: LinkableUsage02 ------------------------- - - - - - 3915e982 by Alan Zimmerman at 2026-08-16T04:37:58-04:00 EPA: Remove al_trailing from AnnList It was not being used - - - - - fed942ac by Andreas Klebinger at 2026-08-17T12:11:57-04:00 testsuite: Use sigkill in process009. SIGHUB might have been blocked by a (grand)*parent of the test. In such cases the test would fail as the python process would simply exist gracefully instead of committing to a premature end. By using SIGKILL we can rely on the signal not being blocked, fixing #27578 in the process. - - - - - 574c875f by Andreas Klebinger at 2026-08-17T12:12:39-04:00 Simplify comparison in DFM.hs Fixes #27669 - - - - - d8f1a2a3 by Alan Zimmerman at 2026-08-17T12:13:18-04:00 TTG: Add extension points to BooleanFormula They are currently unused, but will be used for exact print annotations next, allowing us to get rid of LocatedBF / SrcSpanAnnBF - - - - - 9a82d258 by Alan Zimmerman at 2026-08-17T19:25:52+01:00 EPA: Remove LocatedBC / SrcSpanBF The custom annotations are now in the BooleanFormula TTG extension points, so LBooleanFormula can now use the standard LocatedA. - - - - - c58f6867 by Alan Zimmerman at 2026-08-17T19:26:59+01:00 EPA: remove unused addTrailingAnnToL. Squash appropriately - - - - - 0feb735d by Alan Zimmerman at 2026-08-17T19:26:59+01:00 EPS: Remove NoEpTok/NoEpUniTok, using an unhelpful SrcSpan instead Also introduce helper functions noEpTok and noEpUniTok to serve as simple replacements in code inserting an token annotation without location information. - - - - - d3af13a3 by Alan Zimmerman at 2026-08-17T19:26:59+01:00 EPA: Some haddock processing tweaks - - - - - 0feaf34d by Alan Zimmerman at 2026-08-17T19:26:59+01:00 Some haddock exactprint tests - - - - - 243430f8 by Alan Zimmerman at 2026-08-17T19:26:59+01:00 EPA: When adding comments honour trailing anns - - - - - d410e63c by Alan Zimmerman at 2026-08-17T19:26:59+01:00 EPA: Uses Parsers.parseModule for exactprint tests This is the advertised way to parse for use for exact printing in the ghc-exactprint library, make sure we test using it. - - - - - ea7a19f3 by Alan Zimmerman at 2026-08-17T19:26:59+01:00 EPA Fix HsCmdDo exact print with comments TODO: add test based on proc-do-complex-four-out.hs - - - - - 3ed851b0 by Alan Zimmerman at 2026-08-17T19:26:59+01:00 EPA: Add comments about remaining Anno SrcSpan instances - - - - - 3aa090d6 by Alan Zimmerman at 2026-08-17T19:26:59+01:00 EPA: Plan for Fixing AnnList Layout Properly - - - - - 74849f3e by Alan Zimmerman at 2026-08-17T19:26:59+01:00 EPA: Remove ListBanana / ListParens from AnnListBrackets - - - - - 19b5c3d2 by Alan Zimmerman at 2026-08-17T19:26:59+01:00 EPA: Tidy up mkHsDoPv, take AnnList directly And some extra stuff by accident. The main goal is to slim down AnnListBrackets to just have braces or none. This makes it clear that it serves only for lists which may have layout, and opens the route to use EpLayout instead of AnnListBrackets - - - - - 835ebf03 by Alan Zimmerman at 2026-08-17T19:26:59+01:00 EPA: Add specific layout field to AnnList - - - - - 94b8264a by Alan Zimmerman at 2026-08-17T19:26:59+01:00 EPA: Use EpaLocation in EpVirtualBraces reword: EPA Use vocurly as basis for AnnListLayout For the existing cases that populate AnnList. This also shows up AnnList usages that can never capture layout. - - - - - ef1b285b by Alan Zimmerman at 2026-08-17T19:26:59+01:00 EPA Use vocurly as basis for AnnListLayout For the existing cases that populate AnnList. This also shows up AnnList usages that can never capture layout. - - - - - 336af5b0 by Alan Zimmerman at 2026-08-17T19:26:59+01:00 EPA: AnnList clean up patch for empty where clause - - - - - 959ab6e3 by Alan Zimmerman at 2026-08-17T19:26:59+01:00 EPA: Add TTG fields to FamilyInfo This commit is a simple add, with unused extension fields. It lays the groundwork for using them for exact print annotations - - - - - 808b9851 by Alan Zimmerman at 2026-08-17T19:26:59+01:00 WIP annclassdecl - - - - - 28589c84 by Alan Zimmerman at 2026-08-17T19:26:59+01:00 EPA: Update AnnClsInstDecl to contain AnnList It was already separately tracking the '{', '}' and leading ';' values. This aligns it with layout-introducing parser productions ('vocurly'), so the layout processing can be handled uniformly - - - - - b0b3106c by Alan Zimmerman at 2026-08-17T19:26:59+01:00 EPA: Tidy up markAnnListA' It takes an action which can modify the AnnList, but this is never used. So remove it. - - - - - acd033f2 by Alan Zimmerman at 2026-08-17T19:26:59+01:00 EPA: Use AnnList for GADT declarations - - - - - e2509e78 by Alan Zimmerman at 2026-08-17T19:26:59+01:00 EPA: Use AnnList in AnnClassDecl - - - - - 8444d44e by Alan Zimmerman at 2026-08-17T19:37:00+01:00 EPA: Make local binds located Use LHsLocalBinds instead of HsLocalBinds This reverses an earlier change, and brings processing of local binds into alignement with the exact printing principle that the LocatedA contents provide the location and extent of the enclosed item, together with any comments or context-specific trailing items like commas or semi colons. The internal TTG extension points only carry exact print annotations related to printing the item within its bounds. This change brings back the problem of EmptyLocalBinds, which by definition cannot have a location. These get a noSrcSpan location, which is ignored in the exact printing process. - - - - - 8f2ad624 by Alan Zimmerman at 2026-08-17T19:37:00+01:00 EPA: Use AnnList in HsMultiIf It introduces layout, capture it - - - - - 2618238b by Alan Zimmerman at 2026-08-17T19:37:00+01:00 EPA: Use AnnList in DecBrl - - - - - a0b0d98d by Alan Zimmerman at 2026-08-17T21:39:38+01:00 EPA: Replace AnnListLayout / AnnListBrackets with EpLayout - - - - - 51af2cd1 by Alan Zimmerman at 2026-08-17T23:27:08+01:00 WIP: setLayoutBoth inside markAnnListA Current problem is it is a global one-shot, and MG has one too Check: is it needed? if so, honouring EpLayout may be needed - - - - - b2876e77 by Alan Zimmerman at 2026-08-17T23:27:08+01:00 Update AnnList plan. do not put into MR - - - - - c7ff8d0a by Alan Zimmerman at 2026-08-17T23:27:08+01:00 EPA: First pass implementation of HsList, for ClassDecls Just as a straight list replacement to start with, no payload. This shows the scope and invasiveness of the initial change - - - - - 39b59224 by Alan Zimmerman at 2026-08-17T23:27:08+01:00 EPA: HsList attempt WIP - - - - - dd2560bf by Alan Zimmerman at 2026-08-17T23:27:08+01:00 Enable ppr test for Haddock1. It currently fails - - - - - 91918304 by Alan Zimmerman at 2026-08-17T23:27:08+01:00 WIP on removing NoEpAnn. Likely abandon - - - - - 0b77f7c3 by Alan Zimmerman at 2026-08-17T23:27:08+01:00 EPA: Add an overview doc for exact printing - - - - - 57acfce1 by Simon Peyton Jones at 2026-08-17T23:27:08+01:00 Added an intro section - - - - - 153 changed files: - + ANNLIST-LAYOUT-PLAN.md - + ExactPrint.md - + changelog.d/27626 - + changelog.d/downsweep-refactor - + changelog.d/show-byte-code - compiler/GHC/ByteCode/Serialize.hs - + compiler/GHC/ByteCode/Show.hs - compiler/GHC/Core/Class.hs - compiler/GHC/Core/Coercion.hs - compiler/GHC/CoreToIface.hs - compiler/GHC/Data/BooleanFormula.hs - compiler/GHC/Data/Unboxed.hs - compiler/GHC/Data/Word64Map/Internal.hs - compiler/GHC/Data/Word64Map/Lazy.hs - compiler/GHC/Data/Word64Map/Strict.hs - compiler/GHC/Data/Word64Map/Strict/Internal.hs - compiler/GHC/Driver/Backpack.hs - compiler/GHC/Driver/Downsweep.hs - compiler/GHC/Driver/Env.hs - compiler/GHC/Driver/Main/Interactive.hs - compiler/GHC/Hs.hs - compiler/GHC/Hs/Binds.hs - compiler/GHC/Hs/Decls.hs - compiler/GHC/Hs/Doc.hs - compiler/GHC/Hs/DocString.hs - compiler/GHC/Hs/Dump.hs - compiler/GHC/Hs/Expr.hs - compiler/GHC/Hs/Extension/Pass.hs - compiler/GHC/Hs/ImpExp.hs - compiler/GHC/Hs/Instances.hs - compiler/GHC/Hs/Pat.hs - compiler/GHC/Hs/Stats.hs - compiler/GHC/Hs/Utils.hs - compiler/GHC/HsToCore/Arrows.hs - compiler/GHC/HsToCore/Expr.hs - compiler/GHC/HsToCore/Expr.hs-boot - compiler/GHC/HsToCore/Match.hs - compiler/GHC/HsToCore/Pmc/Desugar.hs - compiler/GHC/HsToCore/Quote.hs - compiler/GHC/HsToCore/Ticks.hs - compiler/GHC/Iface/Ext/Ast.hs - compiler/GHC/Iface/Syntax.hs - compiler/GHC/IfaceToCore.hs - compiler/GHC/Parser.y - compiler/GHC/Parser/Annotation.hs - compiler/GHC/Parser/PostProcess.hs - compiler/GHC/Parser/PostProcess/Haddock.hs - compiler/GHC/Rename/Bind.hs - compiler/GHC/Rename/Expr.hs - compiler/GHC/Rename/Module.hs - compiler/GHC/Rename/Splice.hs - compiler/GHC/Rename/Utils.hs - compiler/GHC/Runtime/Eval.hs - compiler/GHC/Tc/Deriv/Functor.hs - compiler/GHC/Tc/Deriv/Generate.hs - compiler/GHC/Tc/Gen/Arrow.hs - compiler/GHC/Tc/Gen/Expr.hs - compiler/GHC/Tc/Gen/Match.hs - compiler/GHC/Tc/Module.hs - compiler/GHC/Tc/TyCl.hs - compiler/GHC/Tc/TyCl/Class.hs - compiler/GHC/Tc/TyCl/PatSyn.hs - compiler/GHC/Tc/Types/Origin.hs - compiler/GHC/Tc/Utils/Backpack.hs - compiler/GHC/Tc/Zonk/Type.hs - compiler/GHC/ThToHs.hs - compiler/GHC/Types/Demand.hs - compiler/GHC/Types/Unique/DFM.hs - compiler/GHC/Unit/Env.hs - compiler/Language/Haskell/Syntax.hs - compiler/Language/Haskell/Syntax/Basic.hs - compiler/Language/Haskell/Syntax/BooleanFormula.hs - compiler/Language/Haskell/Syntax/Decls.hs - compiler/Language/Haskell/Syntax/Expr.hs - compiler/Language/Haskell/Syntax/Extension.hs - compiler/ghc.cabal.in - docs/users_guide/exts/qualified_strings.rst - docs/users_guide/using.rst - ghc/GHC/Driver/Session/Mode.hs - ghc/GHCi/UI.hs - ghc/Main.hs - hadrian/hie-bios.bat - testsuite/tests/count-deps/CountDepsParser.stdout - + testsuite/tests/dmdanal/should_run/M2.hs - + testsuite/tests/dmdanal/should_run/T27626.hs - + testsuite/tests/dmdanal/should_run/T27626.stdout - testsuite/tests/dmdanal/should_run/all.T - + testsuite/tests/driver/T27461/Main1.hs - + testsuite/tests/driver/T27461/Main2.hs - + testsuite/tests/driver/T27461/Makefile - + testsuite/tests/driver/T27461/T27461a.stderr - + testsuite/tests/driver/T27461/T27461b.script - + testsuite/tests/driver/T27461/T27461b.stderr - + testsuite/tests/driver/T27461/T27461b.stdout - + testsuite/tests/driver/T27461/all.T - + testsuite/tests/driver/T27461/src/Bar.hs - testsuite/tests/ghc-api/T25121_status.stdout - testsuite/tests/ghc-api/exactprint/T22919.stderr - testsuite/tests/ghc-api/exactprint/Test20239.stderr - testsuite/tests/ghc-api/exactprint/ZeroWidthSemi.stderr - testsuite/tests/ghc-api/fixed-nodes/FixedNodes.hs - testsuite/tests/ghc-api/fixed-nodes/InterfaceModuleGraph.hs - testsuite/tests/ghc-api/fixed-nodes/ModuleGraphInvariants.hs - testsuite/tests/haddock/should_compile_flag_haddock/T17544.stderr - testsuite/tests/haddock/should_compile_flag_haddock/T17544_kw.stderr - testsuite/tests/haddock/should_compile_flag_haddock/T24221.stderr - testsuite/tests/module/mod185.stderr - testsuite/tests/parser/should_compile/DumpParsedAst.stderr - testsuite/tests/parser/should_compile/DumpParsedAstComments.stderr - testsuite/tests/parser/should_compile/DumpRenamedAst.stderr - testsuite/tests/parser/should_compile/DumpSemis.stderr - testsuite/tests/parser/should_compile/DumpTypecheckedAst.stderr - testsuite/tests/parser/should_compile/KindSigs.stderr - testsuite/tests/parser/should_compile/T14189.stderr - testsuite/tests/parser/should_compile/T15279.stderr - testsuite/tests/parser/should_compile/T15323.stderr - testsuite/tests/parser/should_compile/T20452.stderr - testsuite/tests/parser/should_compile/T20718.stderr - testsuite/tests/parser/should_compile/T20718b.stderr - testsuite/tests/parser/should_compile/T20846.stderr - testsuite/tests/parser/should_compile/T23315/T23315.stderr - testsuite/tests/perf/compiler/hard_hole_fits.stderr - testsuite/tests/printer/AnnotationNoListTuplePuns.stdout - + testsuite/tests/printer/Haddock1.hs - testsuite/tests/printer/Makefile - testsuite/tests/printer/T18791.stderr - testsuite/tests/printer/Test20297.stdout - testsuite/tests/printer/Test24533.stdout - testsuite/tests/printer/all.T - testsuite/tests/process/process009.hs - testsuite/tests/process/process009.stdout - + testsuite/tests/show-bytecode/Example.hs - + testsuite/tests/show-bytecode/Makefile - + testsuite/tests/show-bytecode/all.T - + testsuite/tests/show-bytecode/normalize - + testsuite/tests/show-bytecode/show-bytecode-breakpoints.stdout - + testsuite/tests/show-bytecode/show-bytecode-breakpoints.stdout-javascript-unknown-ghcjs - + testsuite/tests/show-bytecode/show-bytecode-hpc.stdout - + testsuite/tests/show-bytecode/show-bytecode-vanilla.stdout - + testsuite/tests/show-bytecode/show-bytecode-vanilla.stdout-javascript-unknown-ghcjs - testsuite/tests/splice-imports/SI35.hs - utils/check-exact/ExactPrint.hs - utils/check-exact/Main.hs - utils/check-exact/Parsers.hs - utils/check-exact/Transform.hs - utils/check-exact/Utils.hs - utils/check-ppr/Main.hs - utils/haddock/haddock-api/src/Haddock/Backends/Hoogle.hs - utils/haddock/haddock-api/src/Haddock/Backends/LaTeX.hs - utils/haddock/haddock-api/src/Haddock/Backends/Xhtml/Decl.hs - utils/haddock/haddock-api/src/Haddock/Convert.hs - utils/haddock/haddock-api/src/Haddock/Interface/Rename.hs - utils/haddock/haddock-api/src/Haddock/Types.hs The diff was not included because it is too large. View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/c5fc327eca1bc6a7853ed4f2d20e0c3... -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/c5fc327eca1bc6a7853ed4f2d20e0c3... You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help