Zubin pushed to branch wip/9.14.2-rc2 at Glasgow Haskell Compiler / GHC Commits: 494d5744 by fendor at 2026-09-03T23:24:45+05:30 Introduce UnitIndex for global external unit caching `UnitInfo`s have been observed to cause a lot of memory usage in #27500. Especially with multiple home units, as the same (external) units are processed from scratch, even though most of the time we end up with exactly the same `UnitInfo`. We introduce a `UnitEnv` global cache that allows us to store external unit information that is used across all `HomeUnitEnv`s. The most important change in this commit is the introduction of the `UnitIndex`. It stores a global mapping of `UnitId` -> `UnitInfo`, and `initUnits` always uses the cached `UnitInfo` entry to populate each `HomeUnitEnv`'s `UnitState`. This allows us to ensure the following property:

Each `UnitInfo` should be alive exactly once in GHC.

All `UnitState`s should reference 'UnitInfo's stored in the 'UnitIndex'. This ensured by calling 'initUnits' with the 'UnitIndex'. In addition, the `ExternalUnitDatabases` may also hold a reference to each on-disk representation of `UnitInfo`. This means, we impose an hard upper bound on the number of `UnitInfo`s alive in the GHC session:

The number of alive `UnitInfo`s closure objects must be the sum of all loaded unit database times two.

We add performance regression tests that make sure the number of live `UnitInfo` cannot exceed this threshold. Closes #27500 ------------------------- Metric Decrease: MultiComponentModules MultiComponentModulesRecomp MultiComponentModulesRecomp100 mhu-perf LinkableUsage02 ------------------------- These metrics increases are especially notable, as we are not even sharing anything big but merely the global package database with 50 entries. It shows how careful sharing of `UnitInfo` can improve memory usage. We expect this to be much more notable when the whole cabal package database is shared across multiple home units. `LinkableUsage02` metric decreases on unreg and i386 platform, only. --- Technical details To share the `UnitInfo`s correctly, it is important that we extract the `WireMap` into the `UnitIndex`. At the moment of writing, `WireMap` must be globally the same for all `HomeUnitEnv`s. This is important, as we could otherwise not cache the "fully-resolved" `UnitInfo`, as we don't change the `UnitId` or `unitAbiHash` when resolving wired-in units. Thus, there could be ambiguities, when the `WireMap` is not the same for all `UnitState`s across the `UnitEnv`. We consider a `UnitInfo` fully-resolved, if wired-in units have been updated, the `UnitInfo` has been validated and variables in the unit config, such as `${pkgroot}` have been resolved. Updating the wired-in units requires the `WireMap` to be globally the same. (cherry picked from commit 6cce494a7ec43953c7a949e0d8ce712abfe73da7) - - - - - 81ff8fcf by fendor at 2026-09-03T23:24:45+05:30 Reuse the UnitIndexCache after initialising multiple home units (cherry picked from commit 024c4d04a98f7e27b3dcb93f9b038f9b6b5a1ad9) - - - - - 36a525b2 by Andreas Klebinger at 2026-09-03T23:24:45+05:30 Specialise: Stop looping on recursive dictionaries in interestingDict interestingDict now doesn't look through loopbreaker unfoldings. Doing so would cause infinite loops on certain dictionaries. Fixes #27705. (cherry picked from commit 578bd18509f0d2aeb004231a197f7f3898f86a2a) - - - - - 10c01bb7 by Zubin Duggal at 2026-09-03T23:24:45+05:30 hadrian: constrain splitmix to keep building on Debian 9 - - - - - 2c0d4914 by Zubin Duggal at 2026-09-03T23:24:45+05:30 Bump file-io submodule to 0.1.6 - - - - - 8f2e7775 by Zubin Duggal at 2026-09-03T23:24:45+05:30 Bump haskeline submodule to 0.8.5.0 - - - - - 4c7174d7 by Zubin Duggal at 2026-09-03T23:24:45+05:30 Bump os-string submodule to 2.0.10 - - - - - 319278b9 by Zubin Duggal at 2026-09-03T23:24:45+05:30 Bump directory submodule to 1.3.11.0 - - - - - 65e02ee2 by Zubin Duggal at 2026-09-03T23:24:45+05:30 Prepare 9.14.2-rc2 - - - - - 62 changed files: - − changelog.d/27626 - − changelog.d/27627 - − changelog.d/27703 - − changelog.d/27704 - − changelog.d/27717 - − changelog.d/T26423 - − changelog.d/T26716 - − changelog.d/T27202 - − changelog.d/T27308 - − changelog.d/T27440 - − changelog.d/T27455 - − changelog.d/T27583 - − changelog.d/T27586 - − changelog.d/T27639 - − changelog.d/T27657 - − changelog.d/arm_ncg_fixes_T27430 - − changelog.d/llvm-23 - compiler/GHC.hs - compiler/GHC/Core/Opt/Specialise.hs - compiler/GHC/Driver/Backpack.hs - compiler/GHC/Driver/Env.hs - compiler/GHC/Driver/Main.hs - compiler/GHC/Driver/Session/Units.hs - compiler/GHC/Types/Unique.hs - compiler/GHC/Unit/Env.hs - compiler/GHC/Unit/External/Database.hs - + compiler/GHC/Unit/External/Index.hs - + compiler/GHC/Unit/External/ModuleOrigin.hs - + compiler/GHC/Unit/External/Providers.hs - + compiler/GHC/Unit/External/Query.hs - + compiler/GHC/Unit/External/Substitution.hs - + compiler/GHC/Unit/External/Validate.hs - + compiler/GHC/Unit/External/Visibility.hs - + compiler/GHC/Unit/External/Wired.hs - compiler/GHC/Unit/Info.hs - compiler/GHC/Unit/State.hs - compiler/GHC/Unit/Types.hs - compiler/ghc.cabal.in - docs/users_guide/9.14.2-notes.rst - ghc/GHCi/UI.hs - hadrian/cabal.project - hadrian/src/Rules/Generate.hs - libraries/base/changelog.md - libraries/directory - libraries/file-io - libraries/haskeline - libraries/os-string - testsuite/tests/count-deps/CountDepsAst.stdout - testsuite/tests/count-deps/CountDepsParser.stdout - + testsuite/tests/driver/TUnitInfo/Foo.hs - + testsuite/tests/driver/TUnitInfo/Makefile - + testsuite/tests/driver/TUnitInfo/all.T - + testsuite/tests/driver/TUnitInfo/genMhu.sh - + testsuite/tests/driver/TUnitInfo/generic-unit-info-space-mhu.stdout - + testsuite/tests/driver/TUnitInfo/generic-unit-info-space-single.stdout - + testsuite/tests/driver/TUnitInfo/generic-unit-info-space.hs - + testsuite/tests/driver/TUnitInfo/generic-unit-info-space.stdout - + testsuite/tests/simplCore/should_run/T27705.hs - + testsuite/tests/simplCore/should_run/T27705.stdout - + testsuite/tests/simplCore/should_run/T27705_Inst.hs - testsuite/tests/simplCore/should_run/all.T - utils/haddock/haddock-api/src/Haddock.hs The diff was not included because it is too large. View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/08f58e965838674e2a8260145b50e1e... -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/08f58e965838674e2a8260145b50e1e... You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help