-
5d0ab71a
by fendor at 2026-07-27T17:31:05-04:00
Introduce global unit database cache
As a first step for better sharing of `UnitInfo` across `UnitEnv`,
we introduce a new datatype called `ExternalUnitDatabases`.
It primarily serves as an in-memory representation of *all*
`UnitDatabase`s across `UnitEnv`. This means, if multiple `HomeUnitEnv`s
depend on the same database, one way or another, we make sure that we
don't parse from disk every time.
Instead, we store the in-memory representation in `ExternalUnitDatabases`.
`ExternalUnitDatabaseCache` is the equivalent of `ExternalUnitState` in
the `UnitEnv`. It is a mutable variable wrapping `ExternalUnitDatabases`.
The mutable `ExternalUnitDatabaseCache` is used in `initUnits` to make
sure we don't parse the same unit database multiple times.
Almost by accident, we change the semantics of `initUnits` to honour
modifications to `packageDBFlags`.
The inability to change `packageDBFlags` while also reusing the already
parsed `UnitDatabase`s was reported in #26423 as a bug.
Hence, we think this behaviour change is warranted and acceptable,
especially since it comes with a breaking change to the `initUnits` API.
Add regression test for #26423
Closes #26423
-
6cce494a
by fendor at 2026-07-27T17:31:05-04:00
Introduce UnitIndex for global external unit caching
`UnitInfo`s have been observed to cause a lot of memory usage in #27500.
Especially with multiple home units, as the same (external) units are
processed from scratch, even though most of the time we end up with
exactly the same `UnitInfo`.
We introduce a `UnitEnv` global cache that allows us to store external
unit information that is used across all `HomeUnitEnv`s.
The most important change in this commit is the introduction of the `UnitIndex`.
It stores a global mapping of `UnitId` -> `UnitInfo`, and `initUnits`
always uses the cached `UnitInfo` entry to populate each
`HomeUnitEnv`'s `UnitState`.
This allows us to ensure the following property:
> Each `UnitInfo` should be alive exactly once in GHC.
All `UnitState`s should reference 'UnitInfo's stored in the 'UnitIndex'.
This ensured by calling 'initUnits' with the 'UnitIndex'.
In addition, the `ExternalUnitDatabases` may also hold a reference
to each on-disk representation of `UnitInfo`.
This means, we impose an hard upper bound on the number of `UnitInfo`s
alive in the GHC session:
> The number of alive `UnitInfo`s closure objects must be the
> sum of all loaded unit database times two.
We add performance regression tests that make sure the number of live
`UnitInfo` cannot exceed this threshold.
Closes #27500
-------------------------
Metric Decrease:
MultiComponentModules
MultiComponentModulesRecomp
MultiComponentModulesRecomp100
mhu-perf
LinkableUsage02
-------------------------
These metrics increases are especially notable, as we are not even
sharing anything big but merely the global package database with 50
entries.
It shows how careful sharing of `UnitInfo` can improve memory usage.
We expect this to be much more notable when the whole cabal package
database is shared across multiple home units.
`LinkableUsage02` metric decreases on unreg and i386 platform, only.
---
Technical details
To share the `UnitInfo`s correctly, it is important that we extract
the `WireMap` into the `UnitIndex`. At the moment of writing, `WireMap`
must be globally the same for all `HomeUnitEnv`s.
This is important, as we could otherwise not cache the "fully-resolved"
`UnitInfo`, as we don't change the `UnitId` or `unitAbiHash` when
resolving wired-in units. Thus, there could be ambiguities, when the
`WireMap` is not the same for all `UnitState`s across the `UnitEnv`.
We consider a `UnitInfo` fully-resolved, if wired-in units have been
updated, the `UnitInfo` has been validated and variables in the unit
config, such as `${pkgroot}` have been resolved.
Updating the wired-in units requires the `WireMap` to be globally the
same.
-
f8e3bee9
by Zubin Duggal at 2026-07-27T17:31:49-04:00
testsuite: skip runtime stats tests on debugged compilers
Debugged flavours build the boot libraries without optimisation, so the
runtime numbers do not match the baselines.
-
1e326770
by Zubin Duggal at 2026-07-27T17:31:50-04:00
testsuite: mark #20706 tests fragile rather than broken
Whether the static linux linker issues manifest depends on the host
toolchain.
-
c0b13cbe
by Zubin Duggal at 2026-07-27T17:31:50-04:00
testsuite: exclude libnuma from mostly-static
It needs static system libraries (libnuma.a) that many platforms do not
ship.
Fixes #26914
-
bee1913d
by Alan Zimmerman at 2026-07-28T16:42:29-04:00
EPA: ClsInstDecl with decls as [LHsDecl GhcPs] in GhcPs
Similar to 4fdfe75731e01dad7d7fa474c2703d0d3965afb1, this commit
changes the as-parsed representation of class instance declarations to
[LHsDecl GhcPs], and only separates them by type from the renamer onward.
This also allows us to remove all the AnnSortKey machinery for exact
printing, as it is now no longer needed.
-
72c55eee
by Cheng Shao at 2026-07-28T16:43:11-04:00
hadrian: implement and use writeFileAtomic to fix race condition
This patch implements `writeFileAtomic` in hadrian and change all
invocations of shake non-atomic `writeFile'` to use `writeFileAtomic`,
to avoid multiple hadrian concurrent invocations overwriting the same
in-tree generated file not in the build root directory. Fixes #27536.
Additional notes:
- `writeFileChanged`/`writeFileChangedBS` cannot be made atomic since
it involves reading the file's older version, so their uses are left
alone. It doesn't affect #27536 given their outputs are contained in
the build root directory.
- It's possible to shrink this patch by only making writes outside the
build root directory atomic. But I think it's not worth the effort
for fine grained distinction here, and atomic writes within the
build root directory should also improve robustness of a hadrian
build.
- In the longer term we do want to make a ghc build only generate
files within the build root directory, though that's a lot of work
and outside the scope of this particular bugfix.
Co-authored-by: Codex <codex@openai.com>
-
46d4f963
by Sylvain Henry at 2026-07-29T06:38:40-04:00
RTS: correctly mark slop bytes when shrinking large arrays (#19048)
Correctly mark slop bytes even when profiling is off so that heap census
doesn't traverse garbage-collected closures.
-
4762a8bf
by Simon Jakobi at 2026-07-29T06:39:23-04:00
Add -XLazyFieldAnnotations (GHC proposal 752)
Unbundle the prefix `~` lazy field annotation syntax from StrictData. The
new LazyFieldAnnotations extension controls whether `~` is accepted on
constructor fields. StrictData (and Strict, transitively) imply the new
extension.
See https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0752-lazy-field-annotations.rst.
Closes #24455.
Assisted-by: Claude Opus 4.8
-
0b6dcc84
by Simon Jakobi at 2026-07-29T06:40:04-04:00
testsuite: Relax T24471 residency tolerance
T24471 peak residency fluctuates enough on i386 to cause spurious
failures. Use the standard residency tolerance while retaining the
existing allocation threshold.
See https://gitlab.haskell.org/ghc/ghc/-/work_items/24471#note_682303.
Assisted-by: gpt-5.6-sol via Codex CLI
-
90e95b34
by Cheng Shao at 2026-07-29T06:40:45-04:00
compiler: fix missing top-level procedure labels in cmm dumps
This patch fixes missing top-level procedure labels in some
intermediate Cmm pass dumps. Fixes #27553.
Co-authored-by: Codex <codex@openai.com>
-
360a5946
by sheaf at 2026-07-29T06:41:35-04:00
Add some type-family-heavy performance tests
FamAppCachePerf stress-tests the performance of lookups in the
type family application cache.
T27336 is a minimisation extracted from the reported reproducer.
SimplCastPerf is a measure of coercion growth due to the simplifier
calling mkTransCo without re-optimising the result.
-
7d79915f
by Sven Tennie at 2026-07-30T08:29:23+02:00
hadrian: Add stage3 cross-bindist target with separate output folder
Add a 'binary-dist-stage3' target that packages target executables
(produced by the stage2 cross-compiler) into a separate
'bindist-stage3/' folder, distinct from the stage2 cross-compiler
bindist in 'bindist/'.
Key changes:
- BindistConfig: add 'bindistFolder' field so each config knows its
parent output folder. crossBindist uses 'bindist', targetBindist
uses 'bindist-stage3'. implicitBindistConfig now considers
finalStage to pick targetBindist for stage3 builds.
- BinaryDist:
- Drive configure/Makefile/install-file rules for both folders via
forM_ instead of a single 'bindist' path; per-stage distrib
directory avoids autoreconf races.
- binary-dist-stage3 phony outputs to 'bindist-stage3'.
- generateSettings is passed the compiler stage explicitly; ghc-pkg
recache uses the executable_stage builder.
- pkgToWrappers honors the supplied stage for program naming.
- Generate: per-stage configure.ac templates (templateRuleForStages)
with correct Build/Host platform interpolation for stage3 (target =
host = build). generateSettings takes the compiler stage as a
parameter rather than inferring it as predStage.
- CabalReinstall: use executable_stage from implicitBindistConfig
instead of hard-coded Stage2 for wrapper generation.
This allows a single cross-compile to produce both a cross-compiler
bindist (x86_64 host) and a target-architecture bindist (e.g. RISC-V)
that can be installed and run natively on the target.
-
4b478a5f
by Sven Tennie at 2026-07-30T08:29:23+02:00
ci: Combine cross stage2 and stage3 bindists in one job
Rename crossConfig to stage2CrossConfig for clarity and turn the
nightly x86_64-linux-deb13-riscv-cross job into a combined stage3
job (crossStage = Just 3). With CROSS_STAGE=3, build_hadrian invokes
both the 'binary-dist' and 'binary-dist-stage3' targets, then renames
the resulting tarballs via BIN_DIST_NAME and BIN_DIST_NAME_STAGE3.
The combined job produces two artifacts: the cross-compiler bindist
(stage2, runs on x86_64) and the target bindist (stage3, native RISC-V
executables) under their historical names, so downstream consumers
need no changes.
gen_ci.hs:
- stage2CrossConfig replaces crossConfig.
- binDistNameStage3 generates the backward-compatible stage3 artifact
name for jobs with crossStage == Just 3.
- jobVariables advertises BIN_DIST_NAME_STAGE3 for stage3 jobs.
- stage3Artifacts adds the stage3 tarball to artifactPaths.
- cross_jobs: riscv job now sets crossStage = Just 3.
ci.sh:
- CROSS_STAGE=3 builds both binary-dist and binary-dist-stage3.
- After building, rename the stage3 tarball from
_build/bindist-stage3/ to $BIN_DIST_NAME_STAGE3.tar.xz.
- cross_prefix is empty for CROSS_STAGE=3 (target GHC runs natively
on the target, no host-target distinction at install time).
jobs.yaml: regenerated. The riscv job keeps its existing name, now
with CROSS_STAGE=3 and the extra stage3 artifact + variable.
-
6267f048
by Sven Tennie at 2026-07-30T08:30:21+02:00
Fix: Use library stage for some settings
-
58298583
by Sven Tennie at 2026-07-30T08:30:39+02:00
CI: Use final cross stage wording more consistently
-
6ce80f83
by Sven Tennie at 2026-07-30T08:30:45+02:00
Fix typo
-
f6a690a1
by Sven Tennie at 2026-07-30T08:30:50+02:00
Fix wrapper - use executable stage
-
4f7e178d
by Sven Tennie at 2026-07-30T08:30:56+02:00
Cleanup
-
68514b9b
by Sven Tennie at 2026-07-30T08:44:14+02:00
Add changelog