-
5d0ab71a
by fendor at 2026-07-27T17:31:05-04:00
Introduce global unit database cache
As a first step for better sharing of `UnitInfo` across `UnitEnv`,
we introduce a new datatype called `ExternalUnitDatabases`.
It primarily serves as an in-memory representation of *all*
`UnitDatabase`s across `UnitEnv`. This means, if multiple `HomeUnitEnv`s
depend on the same database, one way or another, we make sure that we
don't parse from disk every time.
Instead, we store the in-memory representation in `ExternalUnitDatabases`.
`ExternalUnitDatabaseCache` is the equivalent of `ExternalUnitState` in
the `UnitEnv`. It is a mutable variable wrapping `ExternalUnitDatabases`.
The mutable `ExternalUnitDatabaseCache` is used in `initUnits` to make
sure we don't parse the same unit database multiple times.
Almost by accident, we change the semantics of `initUnits` to honour
modifications to `packageDBFlags`.
The inability to change `packageDBFlags` while also reusing the already
parsed `UnitDatabase`s was reported in #26423 as a bug.
Hence, we think this behaviour change is warranted and acceptable,
especially since it comes with a breaking change to the `initUnits` API.
Add regression test for #26423
Closes #26423
-
6cce494a
by fendor at 2026-07-27T17:31:05-04:00
Introduce UnitIndex for global external unit caching
`UnitInfo`s have been observed to cause a lot of memory usage in #27500.
Especially with multiple home units, as the same (external) units are
processed from scratch, even though most of the time we end up with
exactly the same `UnitInfo`.
We introduce a `UnitEnv` global cache that allows us to store external
unit information that is used across all `HomeUnitEnv`s.
The most important change in this commit is the introduction of the `UnitIndex`.
It stores a global mapping of `UnitId` -> `UnitInfo`, and `initUnits`
always uses the cached `UnitInfo` entry to populate each
`HomeUnitEnv`'s `UnitState`.
This allows us to ensure the following property:
> Each `UnitInfo` should be alive exactly once in GHC.
All `UnitState`s should reference 'UnitInfo's stored in the 'UnitIndex'.
This ensured by calling 'initUnits' with the 'UnitIndex'.
In addition, the `ExternalUnitDatabases` may also hold a reference
to each on-disk representation of `UnitInfo`.
This means, we impose an hard upper bound on the number of `UnitInfo`s
alive in the GHC session:
> The number of alive `UnitInfo`s closure objects must be the
> sum of all loaded unit database times two.
We add performance regression tests that make sure the number of live
`UnitInfo` cannot exceed this threshold.
Closes #27500
-------------------------
Metric Decrease:
MultiComponentModules
MultiComponentModulesRecomp
MultiComponentModulesRecomp100
mhu-perf
LinkableUsage02
-------------------------
These metrics increases are especially notable, as we are not even
sharing anything big but merely the global package database with 50
entries.
It shows how careful sharing of `UnitInfo` can improve memory usage.
We expect this to be much more notable when the whole cabal package
database is shared across multiple home units.
`LinkableUsage02` metric decreases on unreg and i386 platform, only.
---
Technical details
To share the `UnitInfo`s correctly, it is important that we extract
the `WireMap` into the `UnitIndex`. At the moment of writing, `WireMap`
must be globally the same for all `HomeUnitEnv`s.
This is important, as we could otherwise not cache the "fully-resolved"
`UnitInfo`, as we don't change the `UnitId` or `unitAbiHash` when
resolving wired-in units. Thus, there could be ambiguities, when the
`WireMap` is not the same for all `UnitState`s across the `UnitEnv`.
We consider a `UnitInfo` fully-resolved, if wired-in units have been
updated, the `UnitInfo` has been validated and variables in the unit
config, such as `${pkgroot}` have been resolved.
Updating the wired-in units requires the `WireMap` to be globally the
same.
-
f8e3bee9
by Zubin Duggal at 2026-07-27T17:31:49-04:00
testsuite: skip runtime stats tests on debugged compilers
Debugged flavours build the boot libraries without optimisation, so the
runtime numbers do not match the baselines.
-
1e326770
by Zubin Duggal at 2026-07-27T17:31:50-04:00
testsuite: mark #20706 tests fragile rather than broken
Whether the static linux linker issues manifest depends on the host
toolchain.
-
c0b13cbe
by Zubin Duggal at 2026-07-27T17:31:50-04:00
testsuite: exclude libnuma from mostly-static
It needs static system libraries (libnuma.a) that many platforms do not
ship.
Fixes #26914
-
bee1913d
by Alan Zimmerman at 2026-07-28T16:42:29-04:00
EPA: ClsInstDecl with decls as [LHsDecl GhcPs] in GhcPs
Similar to 4fdfe75731e01dad7d7fa474c2703d0d3965afb1, this commit
changes the as-parsed representation of class instance declarations to
[LHsDecl GhcPs], and only separates them by type from the renamer onward.
This also allows us to remove all the AnnSortKey machinery for exact
printing, as it is now no longer needed.
-
72c55eee
by Cheng Shao at 2026-07-28T16:43:11-04:00
hadrian: implement and use writeFileAtomic to fix race condition
This patch implements `writeFileAtomic` in hadrian and change all
invocations of shake non-atomic `writeFile'` to use `writeFileAtomic`,
to avoid multiple hadrian concurrent invocations overwriting the same
in-tree generated file not in the build root directory. Fixes #27536.
Additional notes:
- `writeFileChanged`/`writeFileChangedBS` cannot be made atomic since
it involves reading the file's older version, so their uses are left
alone. It doesn't affect #27536 given their outputs are contained in
the build root directory.
- It's possible to shrink this patch by only making writes outside the
build root directory atomic. But I think it's not worth the effort
for fine grained distinction here, and atomic writes within the
build root directory should also improve robustness of a hadrian
build.
- In the longer term we do want to make a ghc build only generate
files within the build root directory, though that's a lot of work
and outside the scope of this particular bugfix.
Co-authored-by: Codex <codex@openai.com>
-
46d4f963
by Sylvain Henry at 2026-07-29T06:38:40-04:00
RTS: correctly mark slop bytes when shrinking large arrays (#19048)
Correctly mark slop bytes even when profiling is off so that heap census
doesn't traverse garbage-collected closures.
-
4762a8bf
by Simon Jakobi at 2026-07-29T06:39:23-04:00
Add -XLazyFieldAnnotations (GHC proposal 752)
Unbundle the prefix `~` lazy field annotation syntax from StrictData. The
new LazyFieldAnnotations extension controls whether `~` is accepted on
constructor fields. StrictData (and Strict, transitively) imply the new
extension.
See https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0752-lazy-field-annotations.rst.
Closes #24455.
Assisted-by: Claude Opus 4.8
-
0b6dcc84
by Simon Jakobi at 2026-07-29T06:40:04-04:00
testsuite: Relax T24471 residency tolerance
T24471 peak residency fluctuates enough on i386 to cause spurious
failures. Use the standard residency tolerance while retaining the
existing allocation threshold.
See https://gitlab.haskell.org/ghc/ghc/-/work_items/24471#note_682303.
Assisted-by: gpt-5.6-sol via Codex CLI
-
90e95b34
by Cheng Shao at 2026-07-29T06:40:45-04:00
compiler: fix missing top-level procedure labels in cmm dumps
This patch fixes missing top-level procedure labels in some
intermediate Cmm pass dumps. Fixes #27553.
Co-authored-by: Codex <codex@openai.com>
-
360a5946
by sheaf at 2026-07-29T06:41:35-04:00
Add some type-family-heavy performance tests
FamAppCachePerf stress-tests the performance of lookups in the
type family application cache.
T27336 is a minimisation extracted from the reported reproducer.
SimplCastPerf is a measure of coercion growth due to the simplifier
calling mkTransCo without re-optimising the result.
-
3ec9e2b9
by Mike Pilgrem at 2026-07-31T08:21:33-04:00
GHC Guide: Improve docs on response files
-
e5b2a1f7
by sheaf at 2026-07-31T08:22:23-04:00
Disable Core Lint for TcPlugin_RewritePerf
This is a compiler performance test, but the test source hard-coded
-dcore-lint, defeating the measurement.
-------------------------
Metric Decrease:
TcPlugin_RewritePerf
-------------------------
-
85b10c00
by Alan Zimmerman at 2026-07-31T22:09:47+01:00
EPA: Remove LocatedP from OverlapMode
We have
type LocatedP = GenLocated SrcSpanAnnP
type SrcSpanAnnP = EpAnn AnnPragma
As the first step in removing this in favour of LocatedA which only
captures location, comments and trailing annotations, we remove it
from OverlapMode
We do this by moving the AnnPragma into the TTG extension point
instead.
-
c9a34a00
by Viktor Dukhovni at 2026-08-02T04:34:17-04:00
Fix note typo
-
4f2a21f7
by Andreas Klebinger at 2026-08-02T22:46:46-04:00
Apply oneShot Monad trick to STG LintM
-
21e4b89d
by Andreas Klebinger at 2026-08-02T22:46:46-04:00
stgLint: Use a single reader env for read only arguments.
-
d415f38a
by Alan Zimmerman at 2026-08-02T22:47:27-04:00
EPA: Remove LocatedP from CType
The next step of removing use of LocatedP by moving
the AnnPragma for CType into its TTG extension point
instead.
-
e5f9f3eb
by Alan Zimmerman at 2026-08-03T18:26:18+01:00
EPA: Remove LocatedP, last use in WarningTxt
The last step of removing LocatedP, by moving the AnnPragma for
WarningTxt into its TTG extension point instead.
This also allows us to remove LocatedP and SrcSpanAnnP
-
05a324bf
by Alan Zimmerman at 2026-08-03T21:13:02+01:00
EPA: Replace AnnPragma with individual types
We introduced AnnPragma as a common type for all pragma usages wrapped
in LocatedP / SrcSpanAnnP. Now that those are gone, and the AnnPragma
moved into the TTG points for the given items, we can ensure that each
carries only the annotations it needs.
So we remove AnnPragma, and in its place bring in
AnnCType
AnnWarningTxt
AnnOverlap
AnnAnnDecl
AnnPragSCC
-
d1324892
by Alan Zimmerman at 2026-08-03T21:17:18+01:00
EPA: Remove LocatedE from WarningCategory
-
0c65c751
by Alan Zimmerman at 2026-08-03T21:17:18+01:00
EPA: Remove LocateE from XCImport and XCExport
-
a4cff4ac
by Alan Zimmerman at 2026-08-03T21:17:18+01:00
EPA: Remove LocatedE from HsRecFields dot
-
819d8fd9
by Alan Zimmerman at 2026-08-03T21:17:18+01:00
EPA: Remove LocatedE completely, last usage for pats
-
b4b1871b
by Alan Zimmerman at 2026-08-03T21:17:18+01:00
EPA: Remove AnnList (EpToken "where") usages
This is moving toward removing the parameter from AnnList completely
-
e0912045
by Alan Zimmerman at 2026-08-03T21:17:18+01:00
EPA remove AnnList (EpToken "rec") usages
-
ee44383e
by Alan Zimmerman at 2026-08-03T21:17:18+01:00
EPA: Remove last parameterised AnnList usage (EpaLocation)
Also remove the parameter
-
291eb095
by Alan Zimmerman at 2026-08-03T21:17:18+01:00
TTG: Add extension points to BooleanFormula
They are currently unused, but will be used for exact print annotations next
-
a3b25808
by Alan Zimmerman at 2026-08-03T21:22:16+01:00
EPA: Remove LocatedBC / SrcSpanBF
-
3748c627
by Alan Zimmerman at 2026-08-03T21:22:16+01:00
EPA: remove unused addTrailingAnnToL. Squash appropriately
-
86fe1085
by Alan Zimmerman at 2026-08-03T21:22:16+01:00
EPS: Remove NoEpTok/NoEpUniTok, using an unhelpful SrcSpan instead
Also introduce helper functions noEpTok and noEpUniTok to serve
as simple replacements in code inserting an token annotation without
location information.
-
46d192a7
by Alan Zimmerman at 2026-08-03T21:22:16+01:00
EPA: Some haddock processing tweaks
-
8c523fa1
by Alan Zimmerman at 2026-08-03T21:22:16+01:00
Some haddock exactprint tests
-
48bc4217
by Alan Zimmerman at 2026-08-03T21:22:16+01:00
EPA: When adding comments honour trailing anns
-
9a28180c
by Alan Zimmerman at 2026-08-03T21:22:16+01:00
EPA: Uses Parsers.parseModule for exactprint tests
This is the advertised way to parse for use for exact printing in the
ghc-exactprint library, make sure we test using it.
-
a4e41fb9
by Alan Zimmerman at 2026-08-03T21:22:16+01:00
EPA Fix HsCmdDo exact print with comments
TODO: add test based on proc-do-complex-four-out.hs
-
4a2e28d2
by Alan Zimmerman at 2026-08-03T22:40:03+01:00
EPA: Add comments about remaining Anno SrcSpan instances
-
c44b4a0c
by Alan Zimmerman at 2026-08-03T22:40:03+01:00
EPA: First pass implementation of HsList, for ClassDecls
Just as a straight list replacement to start with, no payload.
This shows the scope and invasiveness of the initial change
-
e6ee9704
by Alan Zimmerman at 2026-08-03T22:40:35+01:00
WIP
-
331c5b27
by Alan Zimmerman at 2026-08-03T22:40:43+01:00
Enable ppr test for Haddock1. It currently fails
-
6d3c2475
by Alan Zimmerman at 2026-08-03T22:40:43+01:00
WIP on removing NoEpAnn. Likely abandon
-
7f8e094c
by Alan Zimmerman at 2026-08-03T22:40:43+01:00
EPA: Add an overview doc for exact printing
-
3b067e16
by Simon Peyton Jones at 2026-08-03T22:40:43+01:00
Added an intro section