Simon Peyton Jones pushed to branch wip/case-bndr-unf at Glasgow Haskell Compiler / GHC Commits: 953e4c7f by sheaf at 2026-10-26T18:19:10-04:00 Rework LogQueue to be a proper queue Now 'LogQueue' uses a 'TQueue' with a separate 'TVar' to indicate whether the log queue is finished logging, instead of the previous weird data representation using a reverse-order '[Maybe Msg]' with 'Nothing' denoting the end of the queue. - - - - - ec29c5fa by Rodrigo Mesquita at 2026-10-26T18:19:10-04:00 Refactor GHC.Driver.MakeAction Pull out of 'runParPipelines' the logic for creating a 'MakeEnv' ready to be used by multiple threads, as that will be useful for downsweep as well (which doesn't fit the 'runPipelines' flow), rather than being just for upsweep. The code is moved and re-structured to match the export list, simplify the sequentiality checks previously both in 'runPipelines' and 'runAllPipelines', which were weirdly similar and confusing; into the two part step where we need these checks: (1) to construct the MakeEnv, (2) to run the MakeActions in parallel. These two steps are separate and used to be too mixed up. Some additional little simplifications or clean ups here and there. - - - - - 3fb22073 by Rodrigo Mesquita at 2026-10-26T18:19:11-04:00 Refactor GHC.Driver.Downsweep in preparation for parallel downsweep Pure refactor to improve the code to facilitate implementing parallel downsweep in the next commit. This commit puts a MakeEnv into the DownsweepEnv, gives the fields proper names and uses RecordWildcards to simplify, rather than passing around all diagnostic wrappers, driver-message-things and using 10s of positional fields. No behavior changes here! - - - - - 52f385a7 by sheaf at 2026-10-26T18:19:11-04:00 Driver: enableCodeGenForTH in load', not downsweep enableCodeGenForTH modifies the module graph for planning, creating fresh temporary paths. So it should only be run when we are about to execute a build plan (e.g. in load'), so that consumers that want to know facts about the module graph don't go looking for files that don't exist yet (in particular 'haddock --no-compilation'). - - - - - 82402d14 by sheaf at 2026-10-26T18:19:11-04:00 2-phase Cache/Search Finder monad This commit restructures the finder abstraction by introducing the 'FinderM' monad, which splits module lookup operations into two phases: - a cache-only phase, performing no filesystem access, - from the first cache miss onwards, a search action which may access the filesystem. This allows consumers to distinguish between quick cached results versus more expensive filesystem search operations. - - - - - 5c302b40 by sheaf at 2026-10-26T18:19:11-04:00 Separate Home/External finder caches The caches for home modules and modules from external units have different lifetimes, needing to be invalidated at different points. This commit moves the cache for modules from external units into the 'UnitState' type, which keeps the cache correct whenever package flags or package databases change (setSessionDynFlags, :set -package ..., Backpack's addUnit). This also makes it easier to clear the cache of home modules, as they are now stored separately (obviating the 'isUnitEnvInstalledModule' check we used to perform). - - - - - 605fbb96 by sheaf at 2026-10-26T18:19:11-04:00 Add known home modules to the finder cache The finder looks in the file system for the on-disk source that corresponds to a Haskell module name. There are two kinds of modules where we need to bypass this search: - a file passed directly in the command line, which may not be in any search path and whose module name may differ from the file name; - Backpack signatures and modules, which do not exist as files at all. This commit handles these two situations by adding an immutable 'KnownHomeModules' in the finder cache, which is always consulted first before doing any search. This removes the ad-hoc imperative logic which relied on mutating the finder cache at careful points. 'setModuleGraph' now also takes the known home modules, so that the module graph and the known home modules are always set consistently. See Note [Known home modules] in GHC.Unit.Finder.Types. - - - - - 82718f44 by sheaf at 2026-10-26T18:19:11-04:00 Make 'UnresolvedImport' into a suitable cache key The goal of this commit is to make 'UnresolvedImport' suitable for use as a cache key. Specifically, during downsweep we want to be able to cache module resolution queries. We want to do this using some kind of map, e.g. (roughly speaking): Map UnresolvedImport ResolvedImport For this to work, 'UnresolvedImport' must contain **precisely** the input data to each resolution computation. In other words: 1. If something changes that materially affects resolution, it must be stored in the cache key. 2. If changing some input does not materially affect the outcome, then it must not be stored in the cache key. To satisfy (1), we change 'UnitNode' to also keep track of the home unit in whose unit state the unit's dependencies were looked up. For (2), 'ImportResolution' now stores a 'ModuleLookupScope' instead of an 'UnresolvedImportOrigin'. Plugin imports are expressed with a new 'LookupPlugin' scope. The source location of an import, which is immaterial to the resolution, is stored out of band outside 'UnresolvedImport', in 'ModSummary'. - - - - - d4635c7c by sheaf at 2026-10-26T18:19:11-04:00 Add GHC.Data.Dependent This commit adds a minimal implementation of dependent sums ('Some', 'DSum') and dependent maps ('DMap'). The implementation is not entirely performance optimal, as there are a few extraneous allocations compared to the full-blown implementation in the 'dependent-map' package. For our use case (caching downsweep computations), this does not matter much. See GHC.Data.Dependent. - - - - - 6759533e by sheaf at 2026-10-26T18:19:11-04:00 Driver: structured concurrent worker abstraction This commit introduces a structured concurrency framework in the style of the 'ki' library: a collection of threads within a scope. We implement two kinds of concurrent workers on top of this framework: - Independent workers cannot wait for one another at all. The only scheduling operation is to wait for quiescence. - Coordinating workers declare an STM readiness condition (waiting on other workers to complete) which gates their start. See Note [Deterministic concurrent workers] in GHC.Driver.Concurrency. This commit ports upsweep to this new framework, with downsweep being left as subsequent work. Further changes along the way: - Refactoring of how concurrency is acquired to avoid the footgun of trying to use a no-op 'AbstractSem' as a lock in the serial case. - The "re-run with -j1" logic for semaphore opening errors no longer triggers on late semaphore failures (part-way through a lengthy computation). - Logger threads are properly cleaned up on exception, with each concurrent worker's log queue and local TmpFs properly bracketed. - The 'GhcMessage -> AnyGhcDiagnostic' and 'Maybe Messager' arguments of 'depanalE', 'depanalPartial' and 'downsweep', which were all dead in practice, have been dropped. - Concurrent logging has been completely overhauled. Each worker has its own log queue. Workers whose logs are printed as we go are spawned in the order in which their logs must be printed, now using a TQueue instead of the old complicated LogQueueQueue mechanism. 'logThread' is replaced by 'withLogPrinter', which runs the thread that prints the log queues within a concurrent worker scope. These changes allow us to fix issues with the exception handling: we now always flush remaining log queues before exiting (even when the action throws an exception), and exceptions that arise while printing are properly rethrown. This fixes #28009. - - - - - 38a99006 by sheaf at 2026-10-26T18:19:11-04:00 Rule-based deterministic concurrent downsweep This commit rewrites downsweep as a single query-answering rule (see 'DownsweepRule') that can be executed by concurrent worker threads. The design allows every expensive operation (preprocessing files with CPP, parsing headers, reading interfaces) to be performed concurrently according to the -j<N>/-jsem flags. See Note [Rules-based downsweep] in GHC.Driver.Downsweep. The rules are run by "GHC.Driver.Concurrency.runRules" in a demand-driven way: each worker can demand other workers to run, but never wait on another worker. Every piece of work is done at most once. Fixes #27514 Memory usage can increase due to doing more work concurrently: ------------------------- Metric Increase: MultiComponentModulesRecomp MultiComponentModulesRecomp100 mhu-perf ------------------------- - - - - - 9fa7b7be by sheaf at 2026-10-26T18:19:11-04:00 Acquire/release semaphore tokens more readily This commit reworks GHC.Driver.MakeSem to use the scoped worker abstraction of GHC.Utils.Concurrent.Scope. This allows us to get rid of a lot of the tricky logic in GHC.Driver.MakeSem involving manual exception handling, thread spawning and lifetime management, etc. The main user-visible change is that token acquisition is no longer rate limited (due to a bug, there previously was a 1s rate limit on acquisition of semaphore tokens), with the release debounce period shortened from 1s to 10ms. Other changes: - With -jsem, the number of capabilities is now capped at the number of processors, bringing it in line with the -jN behaviour. - Pending jobs whose wait for a token is interrupted by an exception are now withdrawn from the jobserver, instead of acquiring a token that it hung onto without doing useful work. Fixes #27763 - - - - - c83c6733 by Alan Zimmerman at 2026-10-26T18:19:55-04:00 EPA: Update AnnClsInstDecl to contain AnnList It was already separately tracking the '{', '}' and leading ';' values. This aligns it with layout-introducing parser productions ('vocurly'), so the layout processing can be handled uniformly - - - - - a9760899 by Simon Peyton Jones at 2026-10-27T11:28:02+00:00 Make pushCast pure, and move it to Simplify.Utils pushCast and pushCastMCo were in the SimplM monad, but no longer do anything monadic. (Historically, pushing a coercion into an argument used to simplify that argument; but now the coercion is recorded in the sc_cast field of ApplyToVal.) This patch makes them pure functions, and moves them, together with addCastMCo, optOutCoercion and Note [The sc_cast field of ApplyToVal], from GHC.Core.Opt.Simplify.Iteration to GHC.Core.Opt.Simplify.Utils, alongside the SimplCont type they manipulate. No change in behaviour. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> - - - - - d9c710ce by Simon Peyton Jones at 2026-10-27T11:28:02+00:00 Lambda and case binders have vanilla IdInfo (#27965) Lambda binders, case binders, and case-alternative binders should never have RULES, a (non-default) inline pragma, or a stable unfolding. See new Note [Core binder IdInfo invariants] in GHC.Core, which lays out the invariants. This patch * Makes Core Lint check the invariant, using the new predicate GHC.Types.Id.lamBndrInfoOK. * Fixes the desugarer, which violated it. For a strict binding let !x = rhs in body we generate let x = rhs in case x of x { DEFAULT -> body } re-using x (shadowed) as the case binder; see Note [Desugar Strict binds]. But the case binder inherited x's IdInfo, including, say, a NOINLINE pragma (e.g. `res1` in GHC.Internal.Control.Monad.ST.Lazy.Imp). Now GHC.HsToCore.Utils.seqVar gives the case binder vanilla IdInfo. This was not just a matter of taste. For example h fm s = let {-# NOINLINE res1 #-} !res1 = fm s !(f, s') = res1 in (f+1, s') The NOINLINE case binder becomes a NOINLINE alias `let res3 = res2`, which is never inlined, and survives (as a silly join point) into the final code. * Fixes the Simplifier, which could violate it when duplicating a StrictBind continuation, which makes a join point let x = <> in b ==> join j x = b in j <> If x has such IdInfo, we now use a fresh binder for the join point let x = <> in b ==> join j x' = let x = x' in b in j <> See (DSB1) in Note [Duplicating StrictBind]. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> - - - - - e990bd9e by Simon Peyton Jones at 2026-10-27T11:28:02+00:00 Add varIsHNF, and use it in move_transit_vars `varIsHNF v` is the same as `exprIsHNF (Var v)`, but avoids building the Var only to take it apart. The (Var v) case of exprIsHNFlike now uses the shared varIsHNFlike, so the two cannot drift apart. In GHC.Core.Opt.WorkWrap.Utils.move_transit_vars we had the guard isUnliftedType var_ty || exprIsHNF (Var var) but exprIsHNF is already True of variables of (definitely) unlifted type, so the isUnliftedType test was redundant. Now it is just (varIsHNF var). No change in behaviour. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> - - - - - a7e4e77a by Simon Peyton Jones at 2026-10-27T11:28:02+00:00 Use SimplClo for the argument of ApplyToVal This patch renames SimplSR to SimplClo (a "closure"), and uses it not only as the range of the Simplifier's substitution, but also as the argument of an ApplyToVal continuation. ApplyToVal used to have three fields sc_env :: StaticEnv sc_arg :: CoreExpr sc_cast :: MOutCoercion but now has just sc_dup :: DupFlag sc_arg :: SimplClo An un-simplified argument is (ContEx env arg mco), and an already-simplified one is (DoneEx arg _). See Note [The argument of ApplyToVal] in GHC.Core.Opt.Simplify.Utils. The main payoff is that, since the substitution maps an Id to a SimplClo, beta-reduction can substitute the argument for the lambda binder without further ado. Accordingly, preInlineUnconditionally is split into * preInlineLetUnconditionally, for let-bindings * preInlineBetaUnconditionally, for beta-redexes, which takes a SimplClo The latter omits some checks that cannot fire for lambda binders. Other notes * The new (ContEx env arg mco) form makes it easy to push a cast into an argument (mkCloCastMCo), whether or not it has been simplified. See Note [The MOutCoercion field of ContEx]. * `preInlineUnconditionally` gets split into two variants, one for let-bindings and one for beta-reduction (which takes a `SimplClo`). There is a little bit of duplication here, but not much. * Fixes #27962. The simplifier was previously dropping a cast when dealing with runRW#. The new `SimplClo` makes that bug much harder to allow. * Fixes #27963. contOutArgs, which supplies the arguments for Plan (BEFORE) rule matching, ignored the sc_cast field of ApplyToVal. So in (f |> co) n ==> f (n |> co1) -- co1 in sc_cast a rule for `f` saw `f n` rather than `f (n |> co1)`, and failed to match (the rule matcher checks the types of template variables). If f's unfolding is active, f may then be inlined, and the rule is lost for good. Now contOutArgs uses simplCloExpr, which includes the pending cast. Almost no change in behaviour, except for the #27963 bug-fix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> - - - - - 89 changed files: - + changelog.d/T27763 - + changelog.d/T27962 - + changelog.d/T27963 - + changelog.d/parallel-downsweep - compiler/GHC.hs - compiler/GHC/Builtin.hs - compiler/GHC/Core.hs - compiler/GHC/Core/Lint.hs - compiler/GHC/Core/Opt/Simplify/Env.hs - compiler/GHC/Core/Opt/Simplify/Iteration.hs - compiler/GHC/Core/Opt/Simplify/Utils.hs - compiler/GHC/Core/Opt/SpecConstr.hs - compiler/GHC/Core/Opt/WorkWrap/Utils.hs - compiler/GHC/Core/Ppr.hs - compiler/GHC/Core/Utils.hs - + compiler/GHC/Data/Dependent.hs - compiler/GHC/Driver/Backpack.hs - + compiler/GHC/Driver/Concurrency.hs - + compiler/GHC/Driver/Config/Concurrency.hs - compiler/GHC/Driver/Downsweep.hs - compiler/GHC/Driver/Env.hs - compiler/GHC/Driver/Errors/Ppr.hs - compiler/GHC/Driver/Errors/Types.hs - compiler/GHC/Driver/Main/Passes.hs - compiler/GHC/Driver/Make.hs - compiler/GHC/Driver/MakeAction.hs - compiler/GHC/Driver/MakeFile.hs - compiler/GHC/Driver/MakeSem.hs - compiler/GHC/Driver/Pipeline/Execute.hs - compiler/GHC/Driver/Pipeline/LogQueue.hs - compiler/GHC/Hs/Decls.hs - compiler/GHC/Hs/Dump.hs - compiler/GHC/HsToCore/Utils.hs - compiler/GHC/Iface/Load.hs - compiler/GHC/Iface/Recomp.hs - compiler/GHC/Linker/Deps.hs - compiler/GHC/Parser.y - compiler/GHC/Parser/Header.hs - compiler/GHC/Runtime/Interpreter/JS.hs - compiler/GHC/Runtime/Loader.hs - compiler/GHC/StgToJS/Linker/Linker.hs - compiler/GHC/Tc/Gen/Splice.hs - compiler/GHC/Tc/Module.hs - compiler/GHC/Tc/Plugin.hs - compiler/GHC/Tc/Utils/Backpack.hs - compiler/GHC/Types/Error/Codes.hs - compiler/GHC/Types/Id.hs - compiler/GHC/Types/UnresolvedImport.hs - compiler/GHC/Unit/Finder.hs - + compiler/GHC/Unit/Finder/Cache.hs - compiler/GHC/Unit/Finder/Types.hs - compiler/GHC/Unit/Module/Graph.hs - compiler/GHC/Unit/Module/ModSummary.hs - compiler/GHC/Unit/Module/WholeCoreBindings.hs - compiler/GHC/Unit/State.hs - compiler/GHC/Unit/Types.hs - + compiler/GHC/Utils/Concurrent/Scope.hs - compiler/GHC/Utils/TmpFs.hs - compiler/ghc.cabal.in - ghc/GHCi/UI.hs - ghc/Main.hs - linters/lint-codes/LintCodes/Static.hs - testsuite/tests/count-deps/CountDepsParser.stdout - testsuite/tests/diagnostic-codes/codes.stdout - testsuite/tests/driver/Makefile - testsuite/tests/driver/T27461/Makefile - + testsuite/tests/driver/T27461/T27461c.stderr - testsuite/tests/driver/T27461/all.T - + testsuite/tests/driver/T27461/src/Wrong.hs - + testsuite/tests/driver/T28009.hs - testsuite/tests/driver/all.T - testsuite/tests/ghc-api/downsweep/IncrementalDownsweep.hs - testsuite/tests/ghc-api/downsweep/OldModLocation.hs - testsuite/tests/ghc-api/downsweep/PartialDownsweep.hs - testsuite/tests/ghc-api/fixed-nodes/FixedNodes.hs - testsuite/tests/ghc-api/fixed-nodes/InterfaceModuleGraph.hs - testsuite/tests/ghc-api/fixed-nodes/ModuleGraphInvariants.hs - testsuite/tests/haddock/should_compile_flag_haddock/T17544.stderr - testsuite/tests/plugins/defaulting-plugin/DefaultLifted.hs - testsuite/tests/printer/Test24533.stdout - + testsuite/tests/simplCore/should_compile/T27962.hs - + testsuite/tests/simplCore/should_compile/T27963.hs - + testsuite/tests/simplCore/should_compile/T27963.stderr - + testsuite/tests/simplCore/should_compile/T27965.hs - testsuite/tests/simplCore/should_compile/all.T - testsuite/tests/splice-imports/SI35.hs - utils/check-exact/ExactPrint.hs - utils/check-ppr/Main.hs - utils/haddock/haddock-api/src/Haddock/Interface.hs The diff was not included because it is too large. View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/7eda54bcf1541470b2465aa2af14bc5... -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/7eda54bcf1541470b2465aa2af14bc5... You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help