- ghc-commits - Haskell.org

[Git][ghc/ghc][master] Introduce a cache of home module name providers
by Marge Bot (＠marge-bot) 13 May '26

13 May '26

Marge Bot pushed to branch master at Glasgow Haskell Compiler / GHC Commits: 5fab2238 by Wolfgang Jeltsch at 2026-05-12T21:24:27+03:00 Introduce a cache of home module name providers This contribution introduces to the module graph a cache that maps home module names to sets of units providing them and changes the finder to use that cache. This is a performance optimization, especially for multi-home-unit builds. The particular changes are as follows: * In `GHC.Unit.Module.Graph`, `ModuleGraph` is extended with a new field `mg_home_module_name_providers_map`, exposed as `mgHomeModuleNameProvidersMap`. This is a cache that assigns to each home module name the set of IDs of home units that define it. Operations that construct module graphs are updated such that this cache stays synchronized. * In `GHC.Unit.Finder`, `findImportedModule` is changed to pull `mgHomeModuleNameProvidersMap` from `hsc_mod_graph` and pass it to `findImportedModuleNoHsc`, which now does not search home units in arbitrary order but prioritizes those units that the cache mentions as potential providers of the requested module. In addition, this contribution adds variants of the two multi-component compiler performance tests that use 100 units instead of 20, because with just 20 units the benefits from caching of home module name providers are still negligible. The following table shows the total time needed for running both multi-component tests before and after this contribution and with different numbers of units: | # of units | Before | After | |-----------:|-------:|------:| | 20 | 0:12 | 0:12 | | 100 | 0:47 | 0:42 | | 200 | 3:05 | 2:08 | Note that there seems to be a general overhead of 12 seconds that is not attributable to the actual tests, so that the real running times should be 12 seconds smaller than shown above. Resolves #27055. Metric Decrease: MultiComponentModules MultiComponentModulesRecomp Co-authored-by: Matthew Pickering <matthewtpickering(a)gmail.com> Co-authored-by: Fendor <fendor(a)posteo.de> - - - - - 6 changed files: - + changelog.d/more-efficient-home-unit-imports-finding - compiler/GHC/Unit/Finder.hs - compiler/GHC/Unit/Module/Graph.hs - testsuite/tests/perf/compiler/Makefile - testsuite/tests/perf/compiler/all.T - testsuite/tests/perf/compiler/genMultiComp.py Changes: ===================================== changelog.d/more-efficient-home-unit-imports-finding ===================================== @@ -0,0 +1,15 @@ +section: compiler +synopsis: Introduce a cache of home module name providers +issues: #27055 +mrs: !15888 +description: { + This contribution optimizes the algorithm for finding out which home + unit provides the module that a certain import declaration refers + to. The previous approach has been to simply search all home units + in no particular order. This change introduces a cache that allows + for efficiently determining those complete home units that provide a + certain module name and changes the module-finding algorithm such + that it searches these units before the other home units. This leads + to significant performance improvements in situations where there + are lots of home units. +} ===================================== compiler/GHC/Unit/Finder.hs ===================================== @@ -44,6 +44,11 @@ import GHC.Data.OsPath import GHC.Unit.Env import GHC.Unit.Types import GHC.Unit.Module +import GHC.Unit.Module.Graph + ( + HomeModuleNameProvidersMap, + mgHomeModuleNameProvidersMap + ) import GHC.Unit.Home import GHC.Unit.Home.Graph (UnitEnvGraph) import qualified GHC.Unit.Home.Graph as HUG @@ -72,7 +77,8 @@ import GHC.Driver.Config.Finder import GHC.Types.Unique.Set import qualified Data.List as L(sort) import Data.List.NonEmpty ( NonEmpty (..) ) -import qualified Data.Set as Set (toList) +import Data.Set (Set) +import qualified Data.Set as Set (empty, intersection, difference, null, toList) import qualified System.Directory as SD import qualified System.OsPath as OsPath import qualified Data.List.NonEmpty as NE @@ -177,12 +183,13 @@ getDirHash dir = do findImportedModule :: HscEnv -> ModuleName -> PkgQual -> IO FindResult findImportedModule hsc_env mod pkg_qual = - let fc = hsc_FC hsc_env - mhome_unit = hsc_home_unit_maybe hsc_env - dflags = hsc_dflags hsc_env - fopts = initFinderOpts dflags + let fc = hsc_FC hsc_env + mb_home_unit = hsc_home_unit_maybe hsc_env + dflags = hsc_dflags hsc_env + fopts = initFinderOpts dflags in do - findImportedModuleNoHsc fc fopts (hsc_unit_env hsc_env) mhome_unit mod pkg_qual + let home_module_name_providers_map = mgHomeModuleNameProvidersMap (hsc_mod_graph hsc_env) + findImportedModuleNoHsc fc fopts (hsc_unit_env hsc_env) home_module_name_providers_map mb_home_unit mod pkg_qual findImportedModuleWithIsBoot :: HscEnv -> ModuleName -> IsBootInterface -> PkgQual -> IO FindResult findImportedModuleWithIsBoot hsc_env mod is_boot pkg_qual = do @@ -195,55 +202,126 @@ findImportedModuleNoHsc :: FinderCache -> FinderOpts -> UnitEnv + -> HomeModuleNameProvidersMap -> Maybe HomeUnit -> ModuleName -> PkgQual -> IO FindResult -findImportedModuleNoHsc fc fopts ue mhome_unit mod_name mb_pkg = +findImportedModuleNoHsc fc fopts ue home_module_name_providers_map mb_home_unit mod_name mb_pkg = case mb_pkg of NoPkgQual -> unqual_import - ThisPkg uid | (homeUnitId <$> mhome_unit) == Just uid -> home_import + ThisPkg uid | (homeUnitId <$> mb_home_unit) == Just uid -> home_import | Just os <- lookup uid other_fopts -> home_pkg_import (uid, os) - | otherwise -> pprPanic "findImportModule" (ppr mod_name $$ ppr mb_pkg $$ ppr (homeUnitId <$> mhome_unit) $$ ppr uid $$ ppr (map fst all_opts)) + | otherwise -> pprPanic "findImportModule" (ppr mod_name $$ ppr mb_pkg $$ ppr (homeUnitId <$> mb_home_unit) $$ ppr uid $$ ppr (map fst all_opts)) OtherPkg _ -> pkg_import where - all_opts = case mhome_unit of - Nothing -> other_fopts - Just home_unit -> (homeUnitId home_unit, fopts) : other_fopts + mb_home_unit_id :: Maybe UnitId + mb_home_unit_id = homeUnitId <$> mb_home_unit - home_import = case mhome_unit of - Just home_unit -> findHomeModule fc fopts home_unit mod_name - Nothing -> pure $ NoPackage (panic "findImportedModule: no home-unit") + all_opts :: [(UnitId, FinderOpts)] + all_opts = case mb_home_unit_id of + Nothing -> other_fopts + Just home_unit_id -> (home_unit_id, fopts) : other_fopts + home_import :: IO FindResult + home_import = case mb_home_unit of + Just home_unit -> findHomeModule fc fopts home_unit mod_name + Nothing -> pure $ + NoPackage (panic "findImportedModule: no home-unit") + home_pkg_import :: (UnitId, FinderOpts) -> IO FindResult home_pkg_import (uid, opts) - -- If the module is reexported, then look for it as if it was from the perspective - -- of that package which reexports it. - | Just real_mod_name <- lookupUniqMap (finder_reexportedModules opts) mod_name = - findImportedModuleNoHsc fc opts ue (Just $ DefiniteHomeUnit uid Nothing) real_mod_name NoPkgQual - | elementOfUniqSet mod_name (finder_hiddenModules opts) = - return (mkHomeHidden uid) - | otherwise = - findHomePackageModule fc opts uid mod_name - - -- Do not be smart and change this to `foldr orIfNotFound home_import hs` as - -- that is not the same!! home_import is first because we need to look within ourselves - -- first before looking at the packages in order. - any_home_import = foldr1 orIfNotFound (home_import:| map home_pkg_import other_fopts) - - pkg_import = findExposedPackageModule fc fopts units mod_name mb_pkg - - unqual_import = any_home_import - `orIfNotFound` - findExposedPackageModule fc fopts units mod_name NoPkgQual - - units = case mhome_unit of - Nothing -> ue_homeUnitState ue - Just home_unit -> HUG.homeUnitEnv_units $ ue_findHomeUnitEnv (homeUnitId home_unit) ue - hpt_deps :: [UnitId] - hpt_deps = Set.toList (homeUnitDepends units) - other_fopts = map (\uid -> (uid, initFinderOpts (homeUnitEnv_dflags (ue_findHomeUnitEnv uid ue)))) hpt_deps + -- If the module is reexported, then look for it as if it was from the + -- perspective of that package which reexports it. + | Just real_mod_name + <- lookupUniqMap (finder_reexportedModules opts) mod_name + = findImportedModuleNoHsc fc opts ue home_module_name_providers_map + (Just $ DefiniteHomeUnit uid Nothing) + real_mod_name + NoPkgQual + | elementOfUniqSet mod_name (finder_hiddenModules opts) + = return (mkHomeHidden uid) + | otherwise + = findHomePackageModule fc opts uid mod_name + + any_home_import :: IO FindResult + any_home_import = foldr1 orIfNotFound $ + home_import :| map home_pkg_import other_fopts + -- Do not try to be smart and change this to `foldr orIfNotFound home_import + -- (map home_pkg_import other_fopts)`, as that would not be the same. + -- `home_import` is first because we need to first look within the current + -- unit before looking at the other units in order. + + pkg_import :: IO FindResult + pkg_import = findExposedPackageModule fc fopts unit_state mod_name mb_pkg + + unqual_import :: IO FindResult + unqual_import + = any_home_import + `orIfNotFound` + findExposedPackageModule fc fopts unit_state mod_name NoPkgQual + + unit_state :: UnitState + unit_state = case mb_home_unit_id of + Nothing -> ue_homeUnitState ue + Just home_unit_id -> HUG.homeUnitEnv_units $ + ue_findHomeUnitEnv home_unit_id ue + + home_unit_deps :: Set UnitId + home_unit_deps = homeUnitDepends unit_state + + ranked_home_unit_deps :: [UnitId] + ranked_home_unit_deps = rankedHomeUnitDeps home_module_name_providers_map + mod_name + home_unit_deps + + other_fopts :: [(UnitId, FinderOpts)] + other_fopts + = [ + (uid, opts) | + uid <- ranked_home_unit_deps, + let opts = initFinderOpts $ + homeUnitEnv_dflags (ue_findHomeUnitEnv uid ue) + ] + +-- | Yields the unit IDs from the given set as a list with those that refer to +-- providers of the given home module name coming first. This is to prioritize +-- such providers during module finding. +rankedHomeUnitDeps :: HomeModuleNameProvidersMap + -> ModuleName + -> Set UnitId + -> [UnitId] +rankedHomeUnitDeps _ _ home_unit_deps | Set.null home_unit_deps + = [] +-- The special handling of the situation where the dependency set is empty does +-- not change the result, but it avoids triggering evaluation of the module +-- graph. This is particularly important in one-shot mode, where the module +-- graph is not needed. Computing it nevertheless would result in a, possibly +-- dramatic, increase of memory usage. Worse, GHC would erroneously look for the +-- sources of modules, which would, for example, cause test `boot1` to fail with +-- the following error message: +-- +-- B.hs:3:1: error: [GHC-87110] +-- Could not find module ‘A’. +-- Use -v to see a list of the files searched for. +-- | +-- 3 | import {-# source #-} A +-- | ^^^^^^^^^^^^^^^^^^^^^^^ +rankedHomeUnitDeps home_module_name_providers_map mod_name home_unit_deps + = Set.toList cached_deps ++ Set.toList uncached_deps + where + + cached_providers :: Set UnitId + cached_providers = lookupWithDefaultUniqMap home_module_name_providers_map + Set.empty + mod_name + + cached_deps :: Set UnitId + cached_deps = Set.intersection home_unit_deps cached_providers + + uncached_deps :: Set UnitId + uncached_deps = Set.difference home_unit_deps cached_providers -- | Locate a plugin module requested by the user, for a compiler -- plugin. This consults the same set of exposed packages as @@ -261,15 +339,15 @@ findPluginModule :: HscEnv -> ModuleName -> IO FindResult findPluginModule hsc_env mod_name = do let fc = hsc_FC hsc_env let units = hsc_units hsc_env - let mhome_unit = hsc_home_unit_maybe hsc_env - findPluginModuleNoHsc fc (initFinderOpts (hsc_dflags hsc_env)) units mhome_unit mod_name + let mb_home_unit = hsc_home_unit_maybe hsc_env + findPluginModuleNoHsc fc (initFinderOpts (hsc_dflags hsc_env)) units mb_home_unit mod_name -- | A version of findExactModule which takes the exact parts of the HscEnv it needs -- directly. findExactModuleNoHsc :: FinderCache -> FinderOpts -> UnitEnvGraph FinderOpts -> UnitState -> Maybe HomeUnit -> InstalledModule -> IsBootInterface -> IO InstalledFindResult -findExactModuleNoHsc fc fopts other_fopts unit_state mhome_unit mod is_boot = do - res <- case mhome_unit of +findExactModuleNoHsc fc fopts other_fopts unit_state mb_home_unit mod is_boot = do + res <- case mb_home_unit of Just home_unit | isHomeInstalledModule home_unit mod -> findInstalledHomeModule fc fopts (homeUnitId home_unit) (moduleName mod) ===================================== compiler/GHC/Unit/Module/Graph.hs ===================================== @@ -67,6 +67,8 @@ module GHC.Unit.Module.Graph , mgLookupModule , mgLookupModuleName , mgHasHoles + , HomeModuleNameProvidersMap + , mgHomeModuleNameProvidersMap , showModMsg -- ** Reachability queries @@ -156,10 +158,12 @@ import GHC.Unit.Module.ModIface import GHC.Utils.Misc ( partitionWith ) import System.FilePath +import Data.Set (Set) +import qualified Data.Set as Set +import Data.Map (Map) import qualified Data.Map as Map import GHC.Types.Unique.DSet -import qualified Data.Set as Set -import Data.Set (Set) +import GHC.Types.Unique.Map (UniqMap, emptyUniqMap, listToUniqMap_C) import GHC.Unit.Module import GHC.Unit.Module.ModNodeKey import GHC.Unit.Module.Stage @@ -202,14 +206,32 @@ data ModuleGraph = ModuleGraph -- Cached computation, whether any of the ModuleGraphNode are isHoleModule, -- This is only used for a hack in GHC.Iface.Load to do with backpack, please -- remove this at the earliest opportunity. + , mg_home_module_name_providers_map :: HomeModuleNameProvidersMap + -- ^ For each module name, which home units provide it. } +type HomeModuleNameProvidersMap = UniqMap ModuleName (Set UnitId) + +mkHomeModuleNameProvidersMap :: [ModuleGraphNode] -> HomeModuleNameProvidersMap +mkHomeModuleNameProvidersMap nodes + = listToUniqMap_C Set.union $ + [ + (moduleName, Set.singleton unitID) | + ModuleNode _ moduleNodeInfo <- nodes, + let moduleName = moduleNodeInfoModuleName moduleNodeInfo, + let unitID = moduleNodeInfoUnitId moduleNodeInfo + ] + +mgHomeModuleNameProvidersMap :: ModuleGraph -> HomeModuleNameProvidersMap +mgHomeModuleNameProvidersMap = mg_home_module_name_providers_map + -- | Why do we ever need to construct empty graphs? Is it because of one shot mode? emptyMG :: ModuleGraph emptyMG = ModuleGraph [] (graphReachability emptyGraph, const Nothing) (graphReachability emptyGraph, const Nothing) (graphReachability emptyGraph, const Nothing) False + emptyUniqMap -- | Construct a module graph. This function should be the only entry point for -- building a 'ModuleGraph', since it is supposed to be built once and never modified. @@ -308,7 +330,7 @@ checkModuleGraph ModuleGraph{..} = where duplicate_errs = rights (Map.elems node_types) - node_types :: Map.Map NodeKey (Either ModuleNodeType ModuleGraphInvariantError) + node_types :: Map NodeKey (Either ModuleNodeType ModuleGraphInvariantError) node_types = Map.fromListWithKey go [ (mkNodeKey n, Left (moduleNodeType n)) | n <- mg_mss ] where -- Multiple nodes with the same key are not allowed. @@ -319,7 +341,7 @@ checkModuleGraph ModuleGraph{..} = -- | Check that all dependencies in the graph are present in the node_types map. -- This is a helper function used by checkModuleGraph. -checkAllDependenciesInGraph :: Map.Map NodeKey (Either ModuleNodeType ModuleGraphInvariantError) +checkAllDependenciesInGraph :: Map NodeKey (Either ModuleNodeType ModuleGraphInvariantError) -> ModuleGraphNode -> Maybe ModuleGraphInvariantError checkAllDependenciesInGraph node_types node = @@ -334,7 +356,7 @@ checkAllDependenciesInGraph node_types node = -- | Check if for the fixed module node invariant: -- -- Fixed nodes can only depend on other fixed nodes. -checkFixedModuleInvariant :: Map.Map NodeKey (Either ModuleNodeType ModuleGraphInvariantError) +checkFixedModuleInvariant :: Map NodeKey (Either ModuleNodeType ModuleGraphInvariantError) -> ModuleGraphNode -> Maybe ModuleGraphInvariantError checkFixedModuleInvariant node_types node = case node of @@ -484,13 +506,17 @@ isEmptyMG = null . mg_mss -- To preserve invariants, 'f' can't change the isBoot status. mapMG :: (ModSummary -> ModSummary) -> ModuleGraph -> ModuleGraph mapMG f mg@ModuleGraph{..} = mg - { mg_mss = flip fmap mg_mss $ \case - InstantiationNode uid iuid -> InstantiationNode uid iuid - LinkNode uid nks -> LinkNode uid nks - ModuleNode deps (ModuleNodeFixed key loc) -> ModuleNode deps (ModuleNodeFixed key loc) - ModuleNode deps (ModuleNodeCompile ms) -> ModuleNode deps (ModuleNodeCompile (f ms)) - UnitNode deps uid -> UnitNode deps uid + { mg_mss = new_mss + , mg_home_module_name_providers_map = mkHomeModuleNameProvidersMap new_mss } + where + new_mss = + flip fmap mg_mss $ \case + InstantiationNode uid iuid -> InstantiationNode uid iuid + LinkNode uid nks -> LinkNode uid nks + ModuleNode deps (ModuleNodeFixed key loc) -> ModuleNode deps (ModuleNodeFixed key loc) + ModuleNode deps (ModuleNodeCompile ms) -> ModuleNode deps (ModuleNodeCompile (f ms)) + UnitNode deps uid -> UnitNode deps uid -- | Map a function 'f' over all the 'ModSummaries', in 'IO'. -- To preserve invariants, 'f' can't change the isBoot status. @@ -856,7 +882,7 @@ moduleNodeInfoBootString mn@(ModuleNodeFixed {}) = -- described in the export list haddocks. -------------------------------------------------------------------------------- -newtype NodeMap a = NodeMap { unNodeMap :: Map.Map NodeKey a } +newtype NodeMap a = NodeMap { unNodeMap :: Map NodeKey a } deriving (Functor, Traversable, Foldable) -- | Transitive dependencies, including SOURCE edges @@ -932,7 +958,7 @@ moduleGraphNodesZero summaries = lookup_key :: ZeroScopeKey -> Maybe Int lookup_key = fmap zeroSummaryNodeKey . lookup_node - node_map :: Map.Map ZeroScopeKey ZeroSummaryNode + node_map :: Map ZeroScopeKey ZeroSummaryNode node_map = Map.fromList [ (s, node) | node <- nodes @@ -1031,7 +1057,7 @@ moduleGraphNodesStages summaries = lookup_key :: (NodeKey, ModuleStage) -> Maybe Int lookup_key = fmap stageSummaryNodeKey . lookup_node - node_map :: Map.Map (NodeKey, ModuleStage) StageSummaryNode + node_map :: Map (NodeKey, ModuleStage) StageSummaryNode node_map = Map.fromList [ (s, node) | node <- nodes @@ -1049,10 +1075,13 @@ moduleGraphNodesStages summaries = extendMG :: ModuleGraph -> ModuleGraphNode -> ModuleGraph extendMG ModuleGraph{..} node = ModuleGraph - { mg_mss = node : mg_mss - , mg_graph = mkTransDeps (node : mg_mss) - , mg_loop_graph = mkTransLoopDeps (node : mg_mss) - , mg_zero_graph = mkTransZeroDeps (node : mg_mss) + { mg_mss = new_mss + , mg_graph = mkTransDeps new_mss + , mg_loop_graph = mkTransLoopDeps new_mss + , mg_zero_graph = mkTransZeroDeps new_mss , mg_has_holes = mg_has_holes || maybe False isHsigFile (moduleNodeInfoHscSource =<< mgNodeIsModule node) + , mg_home_module_name_providers_map = mkHomeModuleNameProvidersMap new_mss } + where + new_mss = node : mg_mss ===================================== testsuite/tests/perf/compiler/Makefile ===================================== @@ -31,7 +31,11 @@ MultiModulesDefsWithCore: ./genMultiLayerModulesCore MultiComponentModulesRecomp: - '$(PYTHON)' genMultiComp.py + '$(PYTHON)' genMultiComp.py 20 20 + TEST_HC='$(TEST_HC)' TEST_HC_OPTS='$(TEST_HC_OPTS)' ./run + +MultiComponentModulesRecomp100: + '$(PYTHON)' genMultiComp.py 100 20 TEST_HC='$(TEST_HC)' TEST_HC_OPTS='$(TEST_HC_OPTS)' ./run MultiLayerModulesTH_Make_Prep: ===================================== testsuite/tests/perf/compiler/all.T ===================================== @@ -507,13 +507,31 @@ test('MultiComponentModulesRecomp', test('MultiComponentModules', [ collect_compiler_runtime(2), - pre_cmd('$PYTHON ./genMultiComp.py'), + pre_cmd('$PYTHON ./genMultiComp.py 20 20'), extra_files(['genMultiComp.py']), compile_timeout_multiplier(5) ], multiunit_compile, [['unitp%d' % n for n in range(20)], '-fno-code -fwrite-interface -v0']) +test('MultiComponentModulesRecomp100', + [ collect_compiler_runtime(2), + pre_cmd('$MAKE -s --no-print-directory MultiComponentModulesRecomp100'), + extra_files(['genMultiComp.py']), + compile_timeout_multiplier(5) + ], + multiunit_compile, + [['unitp%d' % n for n in range(100)], '-fno-code -fwrite-interface -v0']) + +test('MultiComponentModules100', + [ collect_compiler_runtime(2), + pre_cmd('$PYTHON ./genMultiComp.py 100 20'), + extra_files(['genMultiComp.py']), + compile_timeout_multiplier(5) + ], + multiunit_compile, + [['unitp%d' % n for n in range(100)], '-fno-code -fwrite-interface -v0']) + test('ManyConstructors', [ collect_compiler_stats('bytes allocated',2), pre_cmd('./genManyConstructors'), ===================================== testsuite/tests/perf/compiler/genMultiComp.py ===================================== @@ -7,11 +7,12 @@ # * A number of modules names Mod_<pid>_<mid>, each module imports all the top # modules beneath it, and all the modules in the current unit beneath it. +import sys import os import stat -modules_per = 20 -packages = 20 +packages = int(sys.argv[1]) +modules_per = int(sys.argv[2]) total = modules_per * packages def unit_dir(p): View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/5fab22387a66657e5e19f1ba490f09a… -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/5fab22387a66657e5e19f1ba490f09a… You're receiving this email because of your account on gitlab.haskell.org.

1 0

[Git][ghc/ghc] Pushed new branch wip/jeltsch/module-graph-reuse-in-downsweep
by Wolfgang Jeltsch (＠jeltsch) 13 May '26

13 May '26

Wolfgang Jeltsch pushed new branch wip/jeltsch/module-graph-reuse-in-downsweep at Glasgow Haskell Compiler / GHC -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/tree/wip/jeltsch/module-graph-reuse-in… You're receiving this email because of your account on gitlab.haskell.org.

1 0

[Git][ghc/ghc] Pushed new branch wip/mangoiv/ghc-9.12-bp
by Magnus (＠MangoIV) 13 May '26

13 May '26

Magnus pushed new branch wip/mangoiv/ghc-9.12-bp at Glasgow Haskell Compiler / GHC -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/tree/wip/mangoiv/ghc-9.12-bp You're receiving this email because of your account on gitlab.haskell.org.

1 0

[Git][ghc/ghc][wip/spj-reinstallable-base2] 2 commits: my mistakes
by Rodrigo Mesquita (＠alt-romes) 13 May '26

13 May '26

Rodrigo Mesquita pushed to branch wip/spj-reinstallable-base2 at Glasgow Haskell Compiler / GHC Commits: d39a5f3f by Rodrigo Mesquita at 2026-05-13T18:21:40+01:00 my mistakes - - - - - 8a4236f2 by Rodrigo Mesquita at 2026-05-13T18:43:02+01:00 keep fixing. 40% left going through all KnownKey names in GHC.Builtin.KnownKey - - - - - 2 changed files: - compiler/GHC/Builtin.hs - compiler/GHC/Builtin/KnownKeys.hs Changes: ===================================== compiler/GHC/Builtin.hs ===================================== @@ -453,8 +453,7 @@ standardClassKeys = derivableClassKeys ++ numericClassKeys ++ checkKnownKeys - [ randomClassKey, randomGenClassKey - , functorClassKey + [ functorClassKey , monadClassKey, monadPlusClassKey, monadFailClassKey , semigroupClassKey, monoidClassKey , isStringClassKey ===================================== compiler/GHC/Builtin/KnownKeys.hs ===================================== @@ -143,6 +143,8 @@ knownKeyTable , (mkTcOcc "Typeable", typeableClassKey) , (mkTcOcc "Functor", functorClassKey) , (mkTcOcc "Lift", liftClassKey) + , (mkTcOcc "MonadPlus", monadPlusClassKey) + , (mkTcOcc "MonadFail", monadPlusClassKey) -- Numeric classes , (mkTcOcc "Num", numClassKey) @@ -159,13 +161,23 @@ knownKeyTable , (mkVarOcc "toRational", toRationalClassOpKey) , (mkVarOcc "realToFrac", realToFracIdKey) + -- Int and Word + , (mkTcOcc "Int8", int8TyConKey) + , (mkTcOcc "Int16", int16TyConKey) + , (mkTcOcc "Int32", int32TyConKey) + , (mkTcOcc "Int64", int64TyConKey) + , (mkTcOcc "Word16", word16TyConKey) + , (mkTcOcc "Word32", word32TyConKey) + , (mkTcOcc "Word64", word64TyConKey) + -- FFI things , (mkTcOcc "ConstPtr", constPtrTyConKey) + , (mkTcOcc "Ptr", ptrTyConKey) + , (mkTcOcc "FunPtr", funPtrTyConKey) -- Class Monad, MonadFix, MonadZip , (mkTcOcc "Monad", monadClassKey) , (thenMClassOpOcc, thenMClassOpKey) - , (bindMClassOpOcc, bindMClassOpKey) , (returnMClassOpOcc, returnMClassOpKey) -- Class Applicative @@ -200,7 +212,9 @@ knownKeyTable , (mkTcOcc "Generic1", gen1ClassKey) -- Static pointers - , (mkVarOcc "makeStatic", makeStaticKey) + , (mkVarOcc "makeStatic", makeStaticKey) + , (mkDataOcc "StaticPtr", staticPtrDataConKey) + , (mkDataOcc "StaticPtrInfo", staticPtrDataConKey) -- Stable pointers , (mkTcOcc "StablePtr", stablePtrTyConKey) @@ -228,6 +242,11 @@ knownKeyTable , (mkDataOcc ":$$:", typeErrorVAppendDataConKey) , (mkDataOcc "ShowType", typeErrorShowTypeDataConKey) + -- Known lits + , (mkTcOcc "KnownNat", knownNatClassKey) + , (mkTcOcc "KnownSymbol", knownSymbolClassKey) + , (mkTcOcc "KnownChar", knownCharClassKey) + -- Unsafe coercion proofs , (mkVarOcc "unsafeCoerce#", unsafeCoercePrimIdKey) @@ -241,6 +260,10 @@ knownKeyTable -- JS primitives , (mkVarOcc "unsafeUnpackJSStringUtf8##", unsafeUnpackJSStringUtf8ShShKey) + , (mkTcOcc "JSVal", jsvalTyConKey) + + -- Type rep + , (mkTcOcc "TyCon", trTyConTyConKey) -- Known-key names that have BuiltinRules in ConstantFold , (mkVarOcc "unpackFoldrCString#", unpackCStringFoldrIdKey) @@ -432,14 +455,11 @@ withDictClassKey = mkPreludeClassUnique 21 dataToTagClassKey :: KnownKey dataToTagClassKey = mkPreludeClassUnique 23 - monadFailClassKey :: KnownKey monadFailClassKey = mkPreludeClassUnique 29 -monadPlusClassKey, randomClassKey, randomGenClassKey :: KnownKey -monadPlusClassKey = mkPreludeClassUnique 30 -randomClassKey = mkPreludeClassUnique 31 -randomGenClassKey = mkPreludeClassUnique 32 +monadPlusClassKey :: KnownKey +monadPlusClassKey = mkPreludeClassUnique 30 isStringClassKey :: KnownKey isStringClassKey = mkPreludeClassUnique 33 @@ -790,9 +810,6 @@ vecCountDataConKeys = map mkPreludeDataConUnique [90..95] vecElemDataConKeys :: [KnownKey] vecElemDataConKeys = map mkPreludeDataConUnique [96..105] --- Typeable things - - -- Unsafe equality unsafeReflDataConKey :: KnownKey unsafeReflDataConKey = mkPreludeDataConUnique 116 @@ -887,7 +904,6 @@ lazyIdKey = mkPreludeMiscIdUnique 104 oneShotKey = mkPreludeMiscIdUnique 106 runRWKey = mkPreludeMiscIdUnique 107 - nospecIdKey :: KnownKey nospecIdKey = mkPreludeMiscIdUnique 109 @@ -916,19 +932,12 @@ coerceKey = mkPreludeMiscIdUnique 135 -- Just a placeholder for unbound variables produced by the renamer: unboundKey :: KnownKey -unboundKey = mkPreludeMiscIdUnique 136 - - +unboundKey = mkPreludeMiscIdUnique 136 -bindMClassOpKey, thenMClassOpKey, returnMClassOpKey :: KnownKey -bindMClassOpKey = mkPreludeMiscIdUnique 154 +thenMClassOpKey, returnMClassOpKey :: KnownKey thenMClassOpKey = mkPreludeMiscIdUnique 155 -- (>>) returnMClassOpKey = mkPreludeMiscIdUnique 157 --- MonadFail operations -failMClassOpKey :: KnownKey -failMClassOpKey = mkPreludeMiscIdUnique 159 - -- Conversion functions fromIntegralIdKey, realToFracIdKey, toIntegerClassOpKey, toRationalClassOpKey :: KnownKey fromIntegralIdKey = mkPreludeMiscIdUnique 190 @@ -958,7 +967,6 @@ sappendClassOpKey = mkPreludeMiscIdUnique 554 mappendClassOpKey :: KnownKey mappendClassOpKey = mkPreludeMiscIdUnique 556 - makeStaticKey :: KnownKey makeStaticKey = mkPreludeMiscIdUnique 561 @@ -968,9 +976,8 @@ unsafeEqualityProofIdKey = mkPreludeMiscIdUnique 570 unsafeCoercePrimIdKey = mkPreludeMiscIdUnique 571 -- HasField class ops -getFieldClassOpKey, setFieldClassOpKey :: KnownKey +getFieldClassOpKey :: KnownKey getFieldClassOpKey = mkPreludeMiscIdUnique 572 -setFieldClassOpKey = mkPreludeMiscIdUnique 573 -- "Unsatisfiable" constraints unsatisfiableIdKey :: KnownKey View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/baf37fe7c53b2b0e1b08495d065933… -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/baf37fe7c53b2b0e1b08495d065933… You're receiving this email because of your account on gitlab.haskell.org.

1 0

[Git][ghc/ghc][wip/spj-reinstallable-base2] 2 commits: orphans missing from essentials
by Rodrigo Mesquita (＠alt-romes) 13 May '26

13 May '26

Rodrigo Mesquita pushed to branch wip/spj-reinstallable-base2 at Glasgow Haskell Compiler / GHC Commits: 9a1fb983 by Rodrigo Mesquita at 2026-05-13T15:45:40+01:00 orphans missing from essentials - - - - - baf37fe7 by Rodrigo Mesquita at 2026-05-13T17:25:45+01:00 Everything in ghc-internal which needs to construct callstack needs imports - - - - - 14 changed files: - libraries/base/src/GHC/Essentials.hs - libraries/ghc-internal/src/GHC/Internal/Bits.hs - libraries/ghc-internal/src/GHC/Internal/Char.hs - libraries/ghc-internal/src/GHC/Internal/Control/Monad/Fail.hs - libraries/ghc-internal/src/GHC/Internal/Data/Maybe.hs - libraries/ghc-internal/src/GHC/Internal/Float.hs - libraries/ghc-internal/src/GHC/Internal/IO.hs-boot - libraries/ghc-internal/src/GHC/Internal/List.hs - libraries/ghc-internal/src/GHC/Internal/Records.hs - libraries/ghc-internal/src/GHC/Internal/Stack.hs-boot - libraries/ghc-internal/src/GHC/Internal/Stack/Constants.hsc - libraries/ghc-internal/src/GHC/Internal/Stack/ConstantsProf.hsc - libraries/ghc-internal/src/GHC/Internal/Text/ParserCombinators/ReadPrec.hs - libraries/ghc-internal/src/GHC/Internal/Unsafe/Coerce.hs Changes: ===================================== libraries/base/src/GHC/Essentials.hs ===================================== @@ -24,7 +24,8 @@ module GHC.Essentials -- Misc , (.), (&&), not, foldrList, build, map - , seq# + , seq#, ($), assert, considerAccessible + , augment, otherwise, -- Applicative , Applicative, pure, mzip, (<*>), (*>), liftA2 ===================================== libraries/ghc-internal/src/GHC/Internal/Bits.hs ===================================== @@ -73,6 +73,7 @@ import GHC.Internal.Prim ( (-#), (==#), (>=#) ) import GHC.Internal.Real +import GHC.Internal.Stack.Types infixl 8 `shift`, `rotate`, `shiftL`, `shiftR`, `rotateL`, `rotateR` infixl 7 .&. ===================================== libraries/ghc-internal/src/GHC/Internal/Char.hs ===================================== @@ -11,6 +11,7 @@ module GHC.Internal.Char ) where import GHC.Internal.Base +import GHC.Internal.Stack.Types import GHC.Internal.Err (error) import GHC.Internal.Show import GHC.Internal.Prim (chr#, int2Word#, leWord#, Int#, Char#) ===================================== libraries/ghc-internal/src/GHC/Internal/Control/Monad/Fail.hs ===================================== @@ -21,7 +21,7 @@ import GHC.Internal.Base import {-# SOURCE #-} GHC.Internal.IO (throwIO) import {-# SOURCE #-} GHC.Internal.IO.Exception (userError) import GHC.Internal.Maybe (Maybe(..)) -import GHC.Internal.Stack.Types (HasCallStack) +import GHC.Internal.Stack.Types -- | When a value is bound in @do@-notation, the pattern on the left -- hand side of @<-@ might not match. In this case, this class ===================================== libraries/ghc-internal/src/GHC/Internal/Data/Maybe.hs ===================================== @@ -34,7 +34,7 @@ module GHC.Internal.Data.Maybe import GHC.Internal.Base import GHC.Internal.Err (error) import GHC.Internal.Maybe (Maybe(..)) -import GHC.Internal.Stack.Types (HasCallStack) +import GHC.Internal.Stack.Types -- $setup -- Allow the use of some Prelude functions in doctests. ===================================== libraries/ghc-internal/src/GHC/Internal/Float.hs ===================================== @@ -203,6 +203,7 @@ import GHC.Internal.Arr import GHC.Internal.Float.RealFracMethods import GHC.Internal.Float.ConversionUtils import GHC.Internal.Bignum.BigNat +import GHC.Internal.Stack.Types as Rebindable #if WORD_SIZE_IN_BITS == 64 import GHC.Internal.Prim ( ===================================== libraries/ghc-internal/src/GHC/Internal/IO.hs-boot ===================================== @@ -1,9 +1,10 @@ {-# LANGUAGE Unsafe #-} {-# LANGUAGE NoImplicitPrelude #-} +{-# OPTIONS_GHC -fdefines-known-key-names #-} module GHC.Internal.IO where -import GHC.Internal.Stack.Types (HasCallStack) +import GHC.Internal.Stack.Types import GHC.Internal.Types import {-# SOURCE #-} GHC.Internal.Exception.Type (Exception, SomeException) ===================================== libraries/ghc-internal/src/GHC/Internal/List.hs ===================================== @@ -49,7 +49,7 @@ import GHC.Internal.Err (error) import GHC.Internal.Num (Num(..)) import GHC.Internal.Bignum.Integer (Integer) import GHC.Internal.Prim (seq) -import GHC.Internal.Stack.Types (HasCallStack) +import GHC.Internal.Stack.Types infixl 9 !?, !! infix 4 `elem`, `notElem` ===================================== libraries/ghc-internal/src/GHC/Internal/Records.hs ===================================== @@ -32,6 +32,7 @@ module GHC.Internal.Records import GHC.Internal.Base import GHC.Internal.Err( error ) +import GHC.Internal.Stack.Types -- | Constraint representing the fact that the field @x@ belongs to -- the record type @r@ and has field type @a@. This will be solved ===================================== libraries/ghc-internal/src/GHC/Internal/Stack.hs-boot ===================================== @@ -1,9 +1,9 @@ -{-# LANGUAGE NoImplicitPrelude, RankNTypes #-} +{-# LANGUAGE NoImplicitPrelude, RankNTypes, MagicHash #-} module GHC.Internal.Stack where -import GHC.Internal.Base (String) -import GHC.Internal.Stack.Types (HasCallStack, CallStack, SrcLoc) +import GHC.Internal.Base +import GHC.Internal.Stack.Types prettyCallStackLines :: CallStack -> [String] prettyCallStack :: CallStack -> String ===================================== libraries/ghc-internal/src/GHC/Internal/Stack/Constants.hsc ===================================== @@ -9,6 +9,7 @@ import GHC.Internal.Err (error) import GHC.Internal.Num import GHC.Internal.Show import GHC.Internal.Real +import GHC.Internal.Stack.Types #include "Rts.h" #undef BLOCK_SIZE ===================================== libraries/ghc-internal/src/GHC/Internal/Stack/ConstantsProf.hsc ===================================== @@ -8,6 +8,7 @@ import GHC.Internal.Enum import GHC.Internal.Err (error) import GHC.Internal.Num import GHC.Internal.Show +import GHC.Internal.Stack.Types import GHC.Internal.Real -- This file is a copy of GHC.Internal.Stack.Constants, but compiled with PROFILING ===================================== libraries/ghc-internal/src/GHC/Internal/Text/ParserCombinators/ReadPrec.hs ===================================== @@ -64,6 +64,7 @@ import qualified GHC.Internal.Text.ParserCombinators.ReadP as ReadP import GHC.Internal.Base import GHC.Internal.Num( Num(..) ) import GHC.Internal.Control.Monad.Fail +import GHC.Internal.Stack.Types as Rebindable -- --------------------------------------------------------------------------- -- The readPrec type ===================================== libraries/ghc-internal/src/GHC/Internal/Unsafe/Coerce.hs ===================================== @@ -25,6 +25,7 @@ module GHC.Internal.Unsafe.Coerce import GHC.Internal.Base import GHC.Internal.Arr (amap) -- For amap/unsafeCoerce rule import GHC.Internal.Err (error) +import GHC.Internal.Stack.Types as Rebindable {- Note [Implementing unsafeCoerce] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/a0f4834ce0c1fa6b1b7c75776b78cd… -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/a0f4834ce0c1fa6b1b7c75776b78cd… You're receiving this email because of your account on gitlab.haskell.org.

1 0

[Git][ghc/ghc][wip/T26989] 11 commits: Move the `Text.Read` implementation into `base`
by Simon Peyton Jones (＠simonpj) 13 May '26

13 May '26

Simon Peyton Jones pushed to branch wip/T26989 at Glasgow Haskell Compiler / GHC Commits: 44cf9cd7 by Wolfgang Jeltsch at 2026-05-12T09:48:18-04:00 Move the `Text.Read` implementation into `base` - - - - - 4ac3f7d6 by Vladislav Zavialov at 2026-05-12T09:49:03-04:00 EPA: Use AnnParen for tuples and sums Summary of changes * Do not use AnnParen in XListTy, replace it with EpToken "[" and "]" * Specialise AnnParen to tuple/sums by dropping the AnnParensSquare and keeping only AnnParens and AnnParensHash * Use AnnParen in XExplicitTuple * Use AnnParen in XExplicitTupleTy * Use AnnParen in XTuplePat * Use AnnParen in XExplicitSum (via AnnExplicitSum) * Use AnnParen in XSumPat (via EpAnnSumPat) This is a refactoring with no user-facing changes. - - - - - 1bdcddec by Duncan Coutts at 2026-05-12T09:49:48-04:00 Add minimal dlltool support to ghc-toolchain The dlltool is a tool that can create dll import libraries from .def files. These .def files list the exported symbols of dlls. Its somewhat like gnu linker scripts, but more limited. We will need dlltool to build the rts and ghc-internal libraries as DLLs on Windows. The rts and ghc-internal libraries have a recursive dependency on each other. Import libraries can be used to resolve recursive dependencies between dlls. We will use an import library for the rts when linking the ghc-internal library. - - - - - f7fc3770 by Duncan Coutts at 2026-05-12T09:49:48-04:00 Add minimal dlltool support into ./configure Find dlltool, and hopefully support finding it within the bundled llvm toolchain on windows. - - - - - e4e22bfb by Duncan Coutts at 2026-05-12T09:49:48-04:00 Update the default host and target files for dlltool support - - - - - 5666c8f9 by Duncan Coutts at 2026-05-12T09:49:48-04:00 Add dlltool as a hadrian builder Optional except on windows. - - - - - 5e14fe3f by Duncan Coutts at 2026-05-12T09:49:48-04:00 Update and generate libHSghc-internal.def from .def.in file The only symbol that the rts imports from the ghc-internal package now is init_ghc_hs_iface. So the rts only needs an import lib that defines that one symbol. Also, remove the libHSghc-prim.def because it is redundant. The rts no longer imports anything from ghc-prim. Keep libHSffi.def for now. We may yet need it once it is clear how libffi is going to be built/used for ghc. - - - - - 3d91e4a6 by Duncan Coutts at 2026-05-12T09:49:48-04:00 Add rule to build libHSghc-internal.dll.a and link into the rts On windows only, with dynamic linking. This is needed because on windows, all symbols in dlls must be resolved. No dangling symbols allowed. References to external symbols must be explicit. We resolve this with an import library. We create an import library for ghc-internal, a .dll.a file. This is a static archive containing .o files that define the symbols we need, and crucially have ".idata" sections that specifies the symbols the dll imports and from where. Note that we do not install this libHSghc-internal.dll.a, and it does not need to list all the symbols exported by that package. We create a special purpose import lib and only use it when linking the rts dll, so it only has to list the symbols that the rts uses from ghc-internal (which is exactly one symbol: init_ghc_hs_iface). - - - - - c8dae539 by Alice Rixte at 2026-05-12T09:50:52-04:00 Script for downloading and copying `base-exports` file - - - - - 396fb53e by Simon Peyton Jones at 2026-05-13T15:27:16+01:00 Do not use mkCast during typechecking This commit fixes #27219. The problem was that the typechecker was using `mkCast`, whose assertion checks legitimately fail when applied to types that contain unification variables. - - - - - aebd95a8 by Simon Peyton Jones at 2026-05-13T15:27:16+01:00 Major refactor of the Simplifier The main payload of this patch is to refactor the Simplifer to avoid repeated simplification when using Plan (AFTER) for rule rewrites. The need for this was shown up by #26989. See Note [Avoid repeated simplification] in GHC.Core.Opt.Simplify.Iteration. Related refactoring: * Refactor the two fields `sc_dup` and `sc_env` in `ApplyToVal` into one, `sc_env`. Reason: the envt is irrelevant in the "simplified" case, so the data type describes the possiblitiies much more accurately now. * Some refactoring in `knownCon` to split off `wrapDataConFloats`. * Refactor `lookupRule` and its auxiliary functions to return `RuleMatch`, a new data type. See Note [data RuleMatch] in GHC.Core. Ditto for BuiltinRule. This RuleMatch returns fragments of the target in rm_args and rm_floats, leaving `rm_rhs` to be the stuff from the RULE itself. Doing this has routine consequences in GHC.Core.Opt.ConstantFold. Many changes there but all routine. * When doing occurrence analysis on RULEs, make the occ-info on the rule binders relate just to the RHS, not the LHS. See (OUR1) in Note Note [OccInfo in unfoldings and rules] This means that Lint must not complain about the fact that the patterns in the RULE mentions binders that are marked dead. See Note [Dead occurrences] in GHC.Core.Lint. I changed the Core pretty-printer so that it didn't suppress dead binders, else I can't see those binders in RULEs. That led to quite a lot of testsuite wibbles. * Refactor FloatBinds, so that it is used both by `exprIsConApp_mabye` and by `lookupRule` * Move the definition of FloatBinds out of GHc.Core.Make, into GHC.Core. * Add FloatTick as an extra constructor. * Refactor `lookupRule` to use `FloatBinds` instead of `BindWrapper`. This refactor just shares more code. (Rename GHC.Core.Opt.FloatOut.FloatBinds to FloatLets, to avoid gratuitious name clash with GHC.Core.FloatBinds.) Corecion optimisation * In simpleOpt, when composing coercions, call new function `optTransCo`. This is much lighter weight than full blown coercion optimisation. * Make `GHC.Core.Opt.Arity.pushCoValArg` and `pushCoTyArg` return the coercionLKind of the coercion. This saves recomputing that coercionLKind at the key call sites in GHC.Core.Opt.Simplify.Iteration.pushCast. * Rename `addCoerce` in GHC.Core.Simplify.Iteration to become `pushCast`. * In the `ApplyToVal` case of `pushCast` we had a very unsavoury call to `simplArg`. I eliminated it by adding a field `sc_cast` to `ApplyToVal` that records any pending casts. Much nicer now. See Note [The sc_cast field of ApplyToVal]. * Don't optimise coercions if the type-substitution is empty. See Note [Optimising coercions] in GHC.Core.Opt.Simplify.Iteration. The fix for #26838 is dramatic. For the test in perf/compiler/T26839 we have Compiler allocs: Before: 7,363M After: 688M Compile time goes down generally. Here are compiler-alloc changes over 0.5%: CoOpt_Read(normal) 729,184,920 -0.7% CoOpt_Singletons(normal) 666,916,960 -4.6% GOOD LargeRecord(normal) 1,227,056,876 +1.1% T12227(normal) 256,827,604 -4.6% GOOD T12425(optasm) 76,879,410 -0.8% T12545(normal) 787,826,918 -10.8% GOOD T12707(normal) 775,186,464 -0.9% T13253(normal) 318,599,596 -0.8% T14766(normal) 685,857,320 -1.0% T15304(normal) 1,123,333,422 -2.2% T15630(normal) 123,142,330 -2.6% T15630a(normal) 123,092,100 -2.6% T15703(normal) 299,751,682 -2.9% GOOD T17516(normal) 964,072,280 +1.0% T18223(normal) 367,016,820 -6.2% GOOD T18730(optasm) 130,643,770 -3.3% GOOD T20261(normal) 535,608,584 -0.7% T21839c(normal) 340,340,436 -0.9% T24984(normal) 85,568,392 -1.9% T3064(normal) 174,631,992 -1.2% T3294(normal) 1,215,886,432 -0.7% T5030(normal) 141,449,704 -17.2% GOOD T5321Fun(normal) 258,484,744 -1.9% T8095(normal) 770,532,232 -2.7% T9630(normal) 858,423,408 -14.5% GOOD T9872c(normal) 1,591,709,448 +0.7% info_table_map_perf(normal) 19,700,614,458 -1.3% geo. mean -0.7% minimum -17.2% maximum +1.1% Metric Decrease: CoOpt_Singletons T12227 T12545 T12707 T15703 T18223 T18730 T21839c T5030 T9630 - - - - - 99 changed files: - + changelog.d/ghc-api-epa-parens - compiler/GHC/Core.hs - compiler/GHC/Core/Coercion.hs - compiler/GHC/Core/Coercion/Opt.hs - compiler/GHC/Core/Lint.hs - compiler/GHC/Core/Make.hs - compiler/GHC/Core/Opt/Arity.hs - compiler/GHC/Core/Opt/ConstantFold.hs - compiler/GHC/Core/Opt/FloatIn.hs - compiler/GHC/Core/Opt/FloatOut.hs - compiler/GHC/Core/Opt/OccurAnal.hs - compiler/GHC/Core/Opt/Simplify/Env.hs - compiler/GHC/Core/Opt/Simplify/Iteration.hs - compiler/GHC/Core/Opt/Simplify/Utils.hs - compiler/GHC/Core/Opt/SpecConstr.hs - compiler/GHC/Core/Opt/Specialise.hs - compiler/GHC/Core/Ppr.hs - compiler/GHC/Core/Rules.hs - compiler/GHC/Core/SimpleOpt.hs - compiler/GHC/Core/TyCo/Subst.hs - compiler/GHC/CoreToStg/Prep.hs - compiler/GHC/Data/List/SetOps.hs - compiler/GHC/Driver/Config/Core/Lint.hs - compiler/GHC/Hs/Dump.hs - compiler/GHC/Hs/Expr.hs - compiler/GHC/Hs/Pat.hs - compiler/GHC/Hs/Type.hs - compiler/GHC/HsToCore/Pmc/Solver.hs - compiler/GHC/IfaceToCore.hs - compiler/GHC/Parser.y - compiler/GHC/Parser/Annotation.hs - compiler/GHC/Parser/PostProcess.hs - compiler/GHC/Tc/Types/Evidence.hs - compiler/GHC/Types/Id/Make.hs - configure.ac - distrib/configure.ac.in - hadrian/cfg/default.host.target.in - hadrian/cfg/default.target.in - hadrian/src/Builder.hs - hadrian/src/Rules/Generate.hs - hadrian/src/Rules/Library.hs - hadrian/src/Rules/Rts.hs - libraries/base/src/Data/Functor/Classes.hs - libraries/base/src/Data/Functor/Compose.hs - libraries/base/src/Prelude.hs - libraries/base/src/Text/Read.hs - libraries/ghc-internal/ghc-internal.cabal.in - libraries/ghc-internal/src/GHC/Internal/IO/Encoding.hs - − libraries/ghc-internal/src/GHC/Internal/Text/Read.hs - m4/find_llvm_prog.m4 - m4/fp_setup_windows_toolchain.m4 - m4/ghc_toolchain.m4 - m4/prep_target_file.m4 - rts/.gitignore - + rts/win32/libHSghc-internal.def.in - testsuite/tests/codeGen/should_compile/T25177.stderr - testsuite/tests/deSugar/should_compile/T13208.stdout - testsuite/tests/ghc-api/T25121_status.stdout - + testsuite/tests/interface-stability/.gitignore - testsuite/tests/interface-stability/README.mkd - + testsuite/tests/interface-stability/download-base-exports.sh - testsuite/tests/linters/notes.stdout - testsuite/tests/numeric/should_compile/T15547.stderr - testsuite/tests/numeric/should_compile/T20347.stderr - testsuite/tests/numeric/should_compile/T20374.stderr - testsuite/tests/numeric/should_compile/T20376.stderr - testsuite/tests/parser/should_compile/DumpParsedAst.stderr - testsuite/tests/parser/should_compile/DumpRenamedAst.stderr - testsuite/tests/parser/should_compile/KindSigs.stderr - testsuite/tests/parser/should_compile/T20452.stderr - + testsuite/tests/perf/compiler/T26989.hs - + testsuite/tests/perf/compiler/T26989a.hs - testsuite/tests/perf/compiler/all.T - testsuite/tests/printer/T18052a.stderr - testsuite/tests/simplCore/should_compile/DsSpecPragmas.stderr - testsuite/tests/simplCore/should_compile/RewriteHigherOrderPatterns.stderr - testsuite/tests/simplCore/should_compile/T15205.stderr - testsuite/tests/simplCore/should_compile/T18668.stderr - testsuite/tests/simplCore/should_compile/T19246.stderr - testsuite/tests/simplCore/should_compile/T19599.stderr - testsuite/tests/simplCore/should_compile/T19599a.stderr - testsuite/tests/simplCore/should_compile/T21917.stderr - testsuite/tests/simplCore/should_compile/T23074.stderr - testsuite/tests/simplCore/should_compile/T24359a.stderr - testsuite/tests/simplCore/should_compile/T25160.stderr - testsuite/tests/simplCore/should_compile/T25718c.stderr-ws-32 - testsuite/tests/simplCore/should_compile/T25718c.stderr-ws-64 - testsuite/tests/simplCore/should_compile/T26051.stderr - testsuite/tests/simplCore/should_compile/T26116.stderr - testsuite/tests/simplCore/should_compile/T8331.stderr - testsuite/tests/simplCore/should_compile/T8848a.stderr - testsuite/tests/simplCore/should_compile/spec004.stderr - testsuite/tests/th/T24111.stdout - testsuite/tests/typecheck/should_compile/T13032.stderr - testsuite/tests/typecheck/should_compile/subsumption_sort_hole_fits.stderr - testsuite/tests/typecheck/should_fail/T21130.stderr - utils/check-exact/ExactPrint.hs - utils/ghc-toolchain/exe/Main.hs - utils/ghc-toolchain/src/GHC/Toolchain/Target.hs The diff was not included because it is too large. View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/710a4f21b994c26970ff7b817124fc… -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/710a4f21b994c26970ff7b817124fc… You're receiving this email because of your account on gitlab.haskell.org.

1 0

[Git][ghc/ghc][wip/spj-reinstallable-base2] Fix CallStack and ExceptionContext are TC
by Rodrigo Mesquita (＠alt-romes) 13 May '26

13 May '26

Rodrigo Mesquita pushed to branch wip/spj-reinstallable-base2 at Glasgow Haskell Compiler / GHC Commits: a0f4834c by Rodrigo Mesquita at 2026-05-13T15:08:24+01:00 Fix CallStack and ExceptionContext are TC - - - - - 1 changed file: - compiler/GHC/Builtin/KnownKeys.hs Changes: ===================================== compiler/GHC/Builtin/KnownKeys.hs ===================================== @@ -216,10 +216,10 @@ knownKeyTable , (mkTcOcc "IP", ipClassKey) -- Callstacks - , (mkVarOcc "CallStack", callStackTyConKey) + , (mkTcOcc "CallStack", callStackTyConKey) -- Exception context - , (mkVarOcc "ExceptionContext", exceptionContextTyConKey) + , (mkTcOcc "ExceptionContext", exceptionContextTyConKey) -- Custom type errors , (mkTcOcc "TypeError", errorMessageTypeErrorFamKey) View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/a0f4834ce0c1fa6b1b7c75776b78cdf… -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/a0f4834ce0c1fa6b1b7c75776b78cdf… You're receiving this email because of your account on gitlab.haskell.org.

1 0

[Git][ghc/ghc][wip/semaphore-v2] Update to semaphore-compat 2.0.0 using v2 of the protocol
by Zubin (＠wz1000) 13 May '26

13 May '26

Zubin pushed to branch wip/semaphore-v2 at Glasgow Haskell Compiler / GHC Commits: 9d4a5f5e by Zubin Duggal at 2026-05-13T18:17:42+05:30 Update to semaphore-compat 2.0.0 using v2 of the protocol On Linux and other POSIX platforms, GHC's -jsem jobserver client now speaks v2 of the semaphore-compat protocol, which uses Unix domain sockets in place of POSIX named semaphores. This avoids the libc-ABI issues that affected the old implementation. Windows is unaffected and continues to use the v1 protocol (Win32 named semaphores); its reported protocol version remains v1. When GHC receives a -jsem name whose protocol version it does not support, it emits a -Wsemaphore-version-mismatch warning and falls back to -j<N> rather than crashing. ghc --info exposes the supported version in a new "Semaphore version" entry so cabal-install can detect a mismatch before invoking GHC. Users on a cabal-install that predates the v2 update will continue to build successfully on Linux/POSIX, but will lose the cross-process -jsem coordination and fall back to -j<N> per GHC invocation. Users must upgrade to a cabal-install that supports protocol v2 to recover full parallelism. Also fix a leak in cleanupSem (#27253): cleanupSem used to snapshot heldTokens and release them before killing the loop, while the loop's in-flight acquire/release children could still be mutating it. Cleanup now runs inside the loop's own exit handler, after draining the active child via a new activeChild TVar, so the snapshot has no concurrent mutator. See also: - GHC proposal amendment: https://github.com/ghc-proposals/ghc-proposals/pull/673 - cabal-install patch: https://github.com/haskell/cabal/pull/11628 - semaphore-compat MR: https://gitlab.haskell.org/ghc/semaphore-compat/-/merge_requests/8 Bump semaphore-compat submodule to 2.0.0 Fixes #25087 and #27253 - - - - - 19 changed files: - + changelog.d/jobserver-leak-fix - + changelog.d/semaphore-v2 - compiler/GHC/Driver/Errors/Ppr.hs - compiler/GHC/Driver/Errors/Types.hs - compiler/GHC/Driver/Flags.hs - compiler/GHC/Driver/MakeAction.hs - compiler/GHC/Driver/MakeSem.hs - compiler/GHC/Driver/Session.hs - compiler/GHC/Types/Error/Codes.hs - docs/users_guide/using-warnings.rst - docs/users_guide/using.rst - hadrian/cabal.project - hadrian/hadrian.cabal - hadrian/src/Flavour.hs - hadrian/src/Rules/Generate.hs - hadrian/src/Settings/Packages.hs - hadrian/stack.yaml - libraries/semaphore-compat - testsuite/tests/diagnostic-codes/codes.stdout Changes: ===================================== changelog.d/jobserver-leak-fix ===================================== @@ -0,0 +1,11 @@ +section: compiler +issues: #27253 +mrs: !15729 +synopsis: + Fix a token leak in the ``-jsem`` jobserver shutdown path +description: + ``cleanupSem`` used to snapshot ``heldTokens`` and release them + before killing the loop, while the loop's in-flight acquire/release + children could still be mutating it. Cleanup now runs inside the + loop's own exit handler, after draining the active child via a new + ``activeChild`` TVar, so the snapshot has no concurrent mutator. ===================================== changelog.d/semaphore-v2 ===================================== @@ -0,0 +1,30 @@ +section: compiler +issues: #25087 +mrs: !15729 +synopsis: + Update to semaphore-compat 2.0.0 (``-jsem`` protocol v2) +description: + On Linux and other POSIX platforms, GHC's ``-jsem`` jobserver client + now speaks v2 of the semaphore-compat protocol, which uses Unix + domain sockets in place of POSIX named semaphores. This avoids the + libc-ABI issues that affected the old implementation. Windows is + unaffected and continues to use the v1 protocol (Win32 named + semaphores); its reported protocol version remains v1. + + When GHC receives a ``-jsem`` name whose protocol version it does not + support, it now emits a ``-Wsemaphore-version-mismatch`` warning and + falls back to ``-j<N>`` rather than crashing. ``ghc --info`` exposes the + supported version in a new ``"Semaphore version"`` entry so + cabal-install can detect a mismatch before invoking GHC. + + Users on a ``cabal-install`` that predates the v2 update will continue + to build successfully on Linux/POSIX, but will lose the cross-process + ``-jsem`` coordination and fall back to ``-j<N>`` per GHC invocation. + To recover full parallelism, upgrade to a ``cabal-install`` that + supports protocol v2. + + See also: + + - the `GHC proposal amendment <https://github.com/ghc-proposals/ghc-proposals/pull/673>`_ + - the `cabal-install patch <https://github.com/haskell/cabal/pull/11628>`_ + - the `semaphore-compat library MR <https://gitlab.haskell.org/ghc/semaphore-compat/-/merge_requests/8>`_ ===================================== compiler/GHC/Driver/Errors/Ppr.hs ===================================== @@ -24,6 +24,8 @@ import GHC.Types.Hint import GHC.Types.SrcLoc import Data.Version +import System.Semaphore + ( SemaphoreError(..), getSemaphoreProtocolVersion ) import Language.Haskell.Syntax.Decls (RuleDecl(..)) import GHC.Tc.Errors.Types (TcRnMessage) import GHC.HsToCore.Errors.Types (DsMessage) @@ -90,6 +92,20 @@ instance Diagnostic GhcMessage where instance HasDefaultDiagnosticOpts DriverMessageOpts where defaultOpts = DriverMessageOpts (defaultDiagnosticOpts @PsMessage) (defaultDiagnosticOpts @IfaceMessage) +pprSemaphoreError :: SemaphoreError -> SDoc +pprSemaphoreError = \case + SemaphoreAlreadyExists nm -> + text "a semaphore named" <+> quotes (text nm) <+> text "already exists" + SemaphoreDoesNotExist nm -> + text "no semaphore named" <+> quotes (text nm) + SemaphoreIncompatibleVersion got want -> + text "protocol version mismatch (got v" + <> int (getSemaphoreProtocolVersion got) + <> text ", supported v" + <> int (getSemaphoreProtocolVersion want) <> text ")" + SemaphoreOtherError ioe -> + text (show ioe) + instance Diagnostic DriverMessage where type DiagnosticOpts DriverMessage = DriverMessageOpts diagnosticMessage opts = \case @@ -282,6 +298,15 @@ instance Diagnostic DriverMessage where -> mkSimpleDecorated $ vcat [ text "The following modules are missing a linkable which is needed for creating a library:" , nest 2 $ hcat (map ppr mods) ] + DriverSemaphoreVersionMismatch received supported + -> mkSimpleDecorated $ + text "Semaphore version mismatch (received v" <> int (getSemaphoreProtocolVersion received) <> + text ", this GHC supports v" <> int (getSemaphoreProtocolVersion supported) <> + text "); ignoring -jsem and compiling sequentially." + DriverSemaphoreOpenFailure err + -> mkSimpleDecorated $ + text "Failed to open -jsem semaphore:" <+> pprSemaphoreError err <> + text "; ignoring -jsem and compiling sequentially." diagnosticReason = \case DriverUnknownMessage m @@ -355,6 +380,10 @@ instance Diagnostic DriverMessage where -> WarningWithoutFlag DriverMissingLinkableForModule {} -> ErrorWithoutFlag + DriverSemaphoreVersionMismatch {} + -> WarningWithFlag Opt_WarnSemaphoreVersionMismatch + DriverSemaphoreOpenFailure {} + -> WarningWithFlag Opt_WarnSemaphoreOpenFailure diagnosticHints = \case DriverUnknownMessage m @@ -430,5 +459,14 @@ instance Diagnostic DriverMessage where -> noHints DriverMissingLinkableForModule {} -> noHints + DriverSemaphoreVersionMismatch received _supported + | received < _supported + -> [UnknownHint (text "The parent process (e.g. cabal-install) uses an older semaphore protocol." + $$ text "Upgrading cabal-install may resolve this." :: SDoc)] + | otherwise + -> [UnknownHint (text "The parent process (e.g. cabal-install) uses a newer semaphore protocol." + $$ text "Upgrading GHC may resolve this." :: SDoc)] + DriverSemaphoreOpenFailure {} + -> noHints diagnosticCode = constructorCode @GHC ===================================== compiler/GHC/Driver/Errors/Types.hs ===================================== @@ -37,6 +37,7 @@ import qualified GHC.LanguageExtensions as LangExt import GHC.Generics ( Generic ) +import System.Semaphore ( SemaphoreError, SemaphoreProtocolVersion ) import GHC.Tc.Errors.Types import GHC.Iface.Errors.Types @@ -419,6 +420,23 @@ data DriverMessage where DriverMissingLinkableForModule :: ![Module] -> DriverMessage + {-| DriverSemaphoreVersionMismatch is a warning that occurs when GHC + receives a @-jsem@ semaphore name whose protocol version is incompatible + with the version this GHC supports. GHC ignores @-jsem@ and compiles + sequentially. + + The first field is the received version (or 1 for unversioned names), + the second is the version this GHC supports. + -} + DriverSemaphoreVersionMismatch :: !SemaphoreProtocolVersion -> !SemaphoreProtocolVersion -> DriverMessage + + {-| DriverSemaphoreOpenFailure is a warning that occurs when GHC fails to + open the semaphore specified by @-jsem@ (e.g. the socket does not exist + or a system error occurred). GHC ignores @-jsem@ and compiles + sequentially. + -} + DriverSemaphoreOpenFailure :: !SemaphoreError -> DriverMessage + deriving instance Generic DriverMessage data DriverMessageOpts = ===================================== compiler/GHC/Driver/Flags.hs ===================================== @@ -1115,6 +1115,8 @@ data WarningFlag = | Opt_WarnUnusableUnpackPragmas -- ^ @since 9.14 | Opt_WarnPatternNamespaceSpecifier -- ^ @since 9.14 | Opt_WarnUnrecognisedModifiers -- ^ @since 10.0 + | Opt_WarnSemaphoreVersionMismatch -- Since 10.0.1 + | Opt_WarnSemaphoreOpenFailure -- Since 10.0.1 deriving (Eq, Ord, Show, Enum, Bounded) -- | Return the names of a WarningFlag @@ -1237,6 +1239,8 @@ warnFlagNames wflag = case wflag of Opt_WarnUnusableUnpackPragmas -> "unusable-unpack-pragmas" :| [] Opt_WarnPatternNamespaceSpecifier -> "pattern-namespace-specifier" :| [] Opt_WarnUnrecognisedModifiers -> "unrecognised-modifiers" :| [] + Opt_WarnSemaphoreVersionMismatch -> "semaphore-version-mismatch" :| [] + Opt_WarnSemaphoreOpenFailure -> "semaphore-open-failure" :| [] -- ----------------------------------------------------------------------------- -- Standard sets of warning options @@ -1383,7 +1387,9 @@ standardWarnings -- see Note [Documenting warning flags] Opt_WarnDeprecatedPragmas, Opt_WarnRuleLhsEqualities, Opt_WarnUnusableUnpackPragmas, - Opt_WarnUnrecognisedModifiers + Opt_WarnUnrecognisedModifiers, + Opt_WarnSemaphoreVersionMismatch, + Opt_WarnSemaphoreOpenFailure ] -- | Things you get with @-W@. ===================================== compiler/GHC/Driver/MakeAction.hs ===================================== @@ -28,6 +28,14 @@ import GHC.Driver.Errors.Types import GHC.Driver.Messager import GHC.Driver.MakeSem +import System.Semaphore + ( SemaphoreError(..) ) + +import GHC.Driver.Config.Diagnostic ( initDiagOpts, initPrintConfig ) +import GHC.Driver.Errors ( printOrThrowDiagnostics ) +import GHC.Types.Error ( singleMessage ) +import GHC.Types.SrcLoc ( noSrcSpan ) +import GHC.Utils.Error ( mkPlainMsgEnvelope ) import GHC.Utils.Logger import GHC.Utils.TmpFs @@ -49,7 +57,7 @@ mkWorkerLimit :: DynFlags -> IO WorkerLimit mkWorkerLimit dflags = case parMakeCount dflags of Nothing -> pure $ num_procs 1 - Just (ParMakeSemaphore h) -> pure (JSemLimit (SemaphoreName h)) + Just (ParMakeSemaphore h) -> pure (JSemLimit h) Just ParMakeNumProcessors -> num_procs <$> getNumProcessors Just (ParMakeThisMany n) -> pure $ num_procs n where @@ -65,8 +73,8 @@ isWorkerLimitSequential (JSemLimit {}) = False data WorkerLimit = NumProcessorsLimit Int | JSemLimit - SemaphoreName - -- ^ Semaphore name to use + String + -- ^ Raw semaphore identifier from @-jsem@ deriving Eq -- | Environment used when compiling a module @@ -122,17 +130,29 @@ runNjobsAbstractSem n_jobs action = do resetNumCapabilities = set_num_caps n_capabilities MC.bracket_ updNumCapabilities resetNumCapabilities $ action asem -runWorkerLimit :: WorkerLimit -> (AbstractSem -> IO a) -> IO a +runWorkerLimit :: Logger -> DynFlags -> WorkerLimit -> (AbstractSem -> IO a) -> IO a #if defined(wasm32_HOST_ARCH) -runWorkerLimit _ action = do +runWorkerLimit _logger _dflags _ action = do lock <- newMVar () action $ AbstractSem (takeMVar lock) (putMVar lock ()) #else -runWorkerLimit worker_limit action = case worker_limit of +runWorkerLimit logger dflags worker_limit action = case worker_limit of NumProcessorsLimit n_jobs -> runNjobsAbstractSem n_jobs action - JSemLimit sem -> - runJSemAbstractSem sem action + JSemLimit sem_ident -> do + result <- MC.try $ runJSemAbstractSem sem_ident action + case result of + Right a -> return a + Left (SemaphoreIncompatibleVersion actual expected) -> do + let diag = DriverSemaphoreVersionMismatch actual expected + msg = singleMessage $ mkPlainMsgEnvelope (initDiagOpts dflags) noSrcSpan diag + printOrThrowDiagnostics logger (initPrintConfig dflags) (initDiagOpts dflags) (GhcDriverMessage <$> msg) + runNjobsAbstractSem 1 action + Left (err :: SemaphoreError) -> do + let diag = DriverSemaphoreOpenFailure err + msg = singleMessage $ mkPlainMsgEnvelope (initDiagOpts dflags) noSrcSpan diag + printOrThrowDiagnostics logger (initPrintConfig dflags) (initDiagOpts dflags) (GhcDriverMessage <$> msg) + runNjobsAbstractSem 1 action #endif -- | Build and run a pipeline @@ -159,7 +179,7 @@ runParPipelines worker_limit plugin_hsc_env diag_wrapper mHscMessager all_pipeli thread_safe_logger <- liftIO $ makeThreadSafe (hsc_logger plugin_hsc_env) let thread_safe_hsc_env = plugin_hsc_env { hsc_logger = thread_safe_logger } - runWorkerLimit worker_limit $ \abstract_sem -> do + runWorkerLimit (hsc_logger plugin_hsc_env) (hsc_dflags plugin_hsc_env) worker_limit $ \abstract_sem -> do let env = MakeEnv { hsc_env = thread_safe_hsc_env , withLogger = withParLog log_queue_queue_var , compile_sem = abstract_sem ===================================== compiler/GHC/Driver/MakeSem.hs ===================================== @@ -9,9 +9,6 @@ module GHC.Driver.MakeSem -- by a system semaphore (Posix/Windows) runJSemAbstractSem - -- * System semaphores - , Semaphore, SemaphoreName(..) - -- * Abstract semaphores , AbstractSem(..) , withAbstractSem @@ -46,11 +43,14 @@ import Debug.Trace -- available from the semaphore. data Jobserver = Jobserver - { jSemaphore :: !Semaphore + { jSemaphore :: !ClientSemaphore -- ^ The semaphore which controls available resources , jobs :: !(TVar JobResources) -- ^ The currently pending jobs, and the resources -- obtained from the semaphore + , activeChild :: !(TVar (Maybe (ThreadId, TMVar (Maybe MC.SomeException)))) + -- ^ Handle on the current acquire thread (if any). The loop's exit + -- handler reads this to drain a still-running child on shutdown. } data JobserverOptions @@ -81,6 +81,9 @@ data JobResources , jobsWaiting :: !(OrdList (TMVar ())) -- ^ Pending jobs waiting on a token, the job will be blocked on the TMVar so putting into -- the TMVar will allow the job to continue. + , heldTokens :: [SemaphoreToken] + -- ^ Actual semaphore tokens (for release/cleanup). + -- Length should equal tokensOwned - 1 (the implicit token has no SemaphoreToken). } instance Outputable JobResources where @@ -93,9 +96,9 @@ instance Outputable JobResources where ] ) -- | Add one new token. -addToken :: JobResources -> JobResources -addToken jobs@( Jobs { tokensOwned = owned, tokensFree = free }) - = jobs { tokensOwned = owned + 1, tokensFree = free + 1 } +addToken :: SemaphoreToken -> JobResources -> JobResources +addToken tok jobs@( Jobs { tokensOwned = owned, tokensFree = free, heldTokens = toks }) + = jobs { tokensOwned = owned + 1, tokensFree = free + 1, heldTokens = tok : toks } -- | Free one token. addFreeToken :: JobResources -> JobResources @@ -111,12 +114,14 @@ removeFreeToken jobs@( Jobs { tokensFree = free }) (text "removeFreeToken:" <+> ppr free) $ jobs { tokensFree = free - 1 } --- | Return one owned token. -removeOwnedToken :: JobResources -> JobResources -removeOwnedToken jobs@( Jobs { tokensOwned = owned }) +-- | Return one owned token, extracting the 'SemaphoreToken' for release. +removeOwnedToken :: JobResources -> (SemaphoreToken, JobResources) +removeOwnedToken jobs@( Jobs { tokensOwned = owned, heldTokens = toks }) = assertPpr (owned > 1) (text "removeOwnedToken:" <+> ppr owned) - $ jobs { tokensOwned = owned - 1 } + $ case toks of + (t:rest) -> (t, jobs { tokensOwned = owned - 1, heldTokens = rest }) + [] -> panic "removeOwnedToken: no held tokens" -- | Add one new job to the end of the list of pending jobs. addJob :: TMVar () -> JobResources -> JobResources @@ -143,7 +148,7 @@ data JobserverAction = Idle -- | A thread is waiting for a token on the semaphore. | Acquiring - { activeWaitId :: WaitId + { activeThreadId :: ThreadId , threadFinished :: TMVar (Maybe MC.SomeException) } -- | Retrieve the 'TMVar' that signals if the current thread has finished, @@ -189,17 +194,26 @@ releaseJob jobs_tvar = do return ((), addFreeToken jobs) --- | Release all tokens owned from the semaphore (to clean up --- the jobserver at the end). -cleanupJobserver :: Jobserver -> IO () -cleanupJobserver (Jobserver { jSemaphore = sem - , jobs = jobs_tvar }) - = do - Jobs { tokensOwned = owned } <- readTVarIO jobs_tvar - let toks_to_release = owned - 1 - -- Subtract off the implicit token: whoever spawned the ghc process - -- in the first place is responsible for that token. - releaseSemaphore sem toks_to_release +-- | Kill the current acquire thread, if any, and wait for it to exit. +-- +-- Relies on the invariant from 'acquireThread' that a forked child always +-- fills its 'threadFinished' TMVar before it dies; this is what lets the +-- 'takeTMVar' below terminate after the 'killThread'. +drainActiveChild :: Jobserver -> IO () +drainActiveChild (Jobserver { activeChild = active_tvar }) = do + mb <- readTVarIO active_tvar + for_ mb $ \(tid, tmv) -> do + killThread tid + void $ atomically (takeTMVar tmv) + atomically $ writeTVar active_tvar Nothing + +-- | Release every token currently in 'heldTokens'. Safe to call only when +-- nothing else is mutating the 'JobResources' TVar. +releaseAllHeld :: Jobserver -> IO () +releaseAllHeld (Jobserver { jobs = jobs_tvar }) = do + Jobs { heldTokens = toks } <- readTVarIO jobs_tvar + forM_ toks $ \t -> + void $ MC.try @_ @MC.SomeException (releaseSemaphoreToken t) -- | Dispatch the available tokens acquired from the semaphore -- to the pending jobs in the job server. @@ -252,7 +266,7 @@ tracedAtomically origin act = do return a renderJobResources :: String -> JobResources -> String -renderJobResources origin (Jobs own free pending) = showSDocUnsafe $ renderJSON $ +renderJobResources origin (Jobs own free pending _heldToks) = showSDocUnsafe $ renderJSON $ JSObject [ ("name", JSString origin) , ("owned", JSInt own) , ("free", JSInt free) @@ -262,61 +276,66 @@ renderJobResources origin (Jobs own free pending) = showSDocUnsafe $ renderJSON -- | Spawn a new thread that waits on the semaphore in order to acquire -- an additional token. +-- +-- The child is forked masked so the only async-exception delivery point +-- is the interruptible 'waitOnSemaphore'; the STM commit afterwards then +-- always runs to completion, so 'threadFinished' is always filled. +-- +-- The (tid, threadFinished) pair is also published to 'activeChild' so +-- shutdown can drain the child even after the in-loop 'JobserverState' +-- is gone. acquireThread :: Jobserver -> IO JobserverAction -acquireThread (Jobserver { jSemaphore = sem, jobs = jobs_tvar }) = do +acquireThread (Jobserver { jSemaphore = sem, jobs = jobs_tvar, activeChild = active_tvar }) = do threadFinished_tmvar <- newEmptyTMVarIO - let - wait_result_action :: Either MC.SomeException Bool -> IO () - wait_result_action wait_res = + tid <- MC.mask_ $ do + tid <- forkIO $ do + wait_res <- MC.try @_ @MC.SomeException $ waitOnSemaphore sem tracedAtomically_ "acquire_thread" do (r, jb) <- case wait_res of Left (e :: MC.SomeException) -> do return $ (Just e, Nothing) - Right success -> do - if success - then do - modifyJobResources jobs_tvar \ jobs -> - return (Nothing, addToken jobs) - else - return (Nothing, Nothing) + Right tok -> do + modifyJobResources jobs_tvar \ jobs -> + return (Nothing, addToken tok jobs) putTMVar threadFinished_tmvar r return jb - wait_id <- forkWaitOnSemaphoreInterruptible sem wait_result_action - labelThread (waitingThreadId wait_id) "acquire_thread" - return $ Acquiring { activeWaitId = wait_id + atomically $ writeTVar active_tvar (Just (tid, threadFinished_tmvar)) + return tid + labelThread tid "acquire_thread" + return $ Acquiring { activeThreadId = tid , threadFinished = threadFinished_tmvar } -- | Spawn a thread to release ownership of one resource from the semaphore, -- provided we have spare resources and no pending jobs. releaseThread :: Jobserver -> IO JobserverAction -releaseThread (Jobserver { jSemaphore = sem, jobs = jobs_tvar }) = do +releaseThread (Jobserver { jobs = jobs_tvar }) = do threadFinished_tmvar <- newEmptyTMVarIO MC.mask_ do -- Pre-release the resource so that another thread doesn't take control of it -- just as we release the lock on the semaphore. - still_ok_to_release + mb_tok <- tracedAtomically "pre_release" $ modifyJobResources jobs_tvar \ jobs -> if guardRelease jobs - -- TODO: should this also debounce? - then return (True , removeOwnedToken $ removeFreeToken jobs) - else return (False, jobs) - if not still_ok_to_release - then return Idle - else do - tid <- forkIO $ do - x <- MC.try $ releaseSemaphore sem 1 - tracedAtomically_ "post-release" $ do - (r, jobs) <- case x of - Left (e :: MC.SomeException) -> do - modifyJobResources jobs_tvar \ jobs -> - return (Just e, addToken jobs) - Right _ -> do - return (Nothing, Nothing) - putTMVar threadFinished_tmvar r - return jobs - labelThread tid "release_thread" - return Idle + then let (tok, jobs') = removeOwnedToken $ removeFreeToken jobs + in return (Just tok, jobs') + else return (Nothing, jobs) + case mb_tok of + Nothing -> return Idle + Just tok -> do + tid <- forkIO $ do + x <- MC.try $ releaseSemaphoreToken tok + tracedAtomically_ "post-release" $ do + (r, jobs) <- case x of + Left (e :: MC.SomeException) -> do + modifyJobResources jobs_tvar \ jobs -> + return (Just e, addToken tok jobs) + Right _ -> do + return (Nothing, Nothing) + putTMVar threadFinished_tmvar r + return jobs + labelThread tid "release_thread" + return Idle -- | When there are pending jobs but no free tokens, -- spawn a thread to acquire a new token from the semaphore. @@ -363,13 +382,14 @@ tryRelease _ _ = retry -- | Wait for an active thread to finish. Once it finishes: -- -- - set the 'JobserverAction' to 'Idle', +-- - clear the 'activeChild' handle, -- - update the number of capabilities to reflect the number -- of owned tokens from the semaphore. tryNoticeIdle :: JobserverOptions - -> TVar JobResources + -> Jobserver -> JobserverState -> STM (IO JobserverState) -tryNoticeIdle opts jobs_tvar jobserver_state +tryNoticeIdle opts (Jobserver { jobs = jobs_tvar, activeChild = active_tvar }) jobserver_state | Just threadFinished_tmvar <- activeThread_maybe $ jobserverAction jobserver_state = sync_num_caps (canChangeNumCaps jobserver_state) threadFinished_tmvar | otherwise @@ -381,6 +401,7 @@ tryNoticeIdle opts jobs_tvar jobserver_state sync_num_caps can_change_numcaps_tvar threadFinished_tmvar = do mb_ex <- takeTMVar threadFinished_tmvar for_ mb_ex MC.throwM + writeTVar active_tvar Nothing Jobs { tokensOwned } <- readTVar jobs_tvar can_change_numcaps <- readTVar can_change_numcaps_tvar guard can_change_numcaps @@ -404,11 +425,11 @@ tryStopThread :: TVar JobResources -> STM (IO JobserverState) tryStopThread jobs_tvar jsj = do case jobserverAction jsj of - Acquiring { activeWaitId = wait_id } -> do + Acquiring { activeThreadId = tid } -> do jobs <- readTVar jobs_tvar guard $ null (jobsWaiting jobs) return do - interruptWaitOnSemaphore wait_id + killThread tid return $ jsj { jobserverAction = Idle } _ -> retry @@ -430,30 +451,38 @@ jobserverLoop opts sjs@(Jobserver { jobs = jobs_tvar }) action <- atomically $ asum $ (\x -> x s) <$> [ tryRelease sjs , tryAcquire opts sjs - , tryNoticeIdle opts jobs_tvar + , tryNoticeIdle opts sjs , tryStopThread jobs_tvar ] s <- action loop s --- | Create a new jobserver using the given semaphore handle. -makeJobserver :: SemaphoreName -> IO (AbstractSem, IO ()) -makeJobserver sem_name = do - semaphore <- openSemaphore sem_name +-- | Create a new jobserver using the given semaphore identifier. +makeJobserver :: String -> IO (AbstractSem, IO ()) +makeJobserver sem_ident = do + semaphore <- openSemaphore sem_ident >>= either MC.throwM pure let init_jobs = Jobs { tokensOwned = 1 , tokensFree = 1 , jobsWaiting = NilOL + , heldTokens = [] } jobs_tvar <- newTVarIO init_jobs + active_tvar <- newTVarIO Nothing let opts = defaultJobserverOptions -- TODO: allow this to be configured - sjs = Jobserver { jSemaphore = semaphore - , jobs = jobs_tvar } + sjs = Jobserver { jSemaphore = semaphore + , jobs = jobs_tvar + , activeChild = active_tvar } loop_finished_mvar <- newEmptyMVar loop_tid <- forkIOWithUnmask \ unmask -> do r <- try $ unmask $ jobserverLoop opts sjs + -- Always-run exit handler: any child the loop spawned is still alive + -- in its own thread, so drain it before touching jobs_tvar. No one + -- else can mutate the resources once both are dead. + drainActiveChild sjs + releaseAllHeld sjs putMVar loop_finished_mvar $ case r of Left e @@ -467,8 +496,8 @@ makeJobserver sem_name = do acquireSem = acquireJob jobs_tvar releaseSem = releaseJob jobs_tvar cleanupSem = do - -- this is interruptible - cleanupJobserver sjs + -- Trigger the loop's exit handler; it drains the active child and + -- releases all held tokens, then signals loop_finished_mvar. killThread loop_tid mb_ex <- takeMVar loop_finished_mvar for_ mb_ex MC.throwM @@ -477,12 +506,12 @@ makeJobserver sem_name = do -- | Implement an abstract semaphore using a semaphore 'Jobserver' -- which queries the system semaphore of the given name for resources. -runJSemAbstractSem :: SemaphoreName -- ^ the system semaphore to use +runJSemAbstractSem :: String -- ^ the semaphore identifier (from @-jsem@) -> (AbstractSem -> IO a) -- ^ the operation to run -- which requires a semaphore -> IO a -runJSemAbstractSem sem action = MC.mask \ unmask -> do - (abs, cleanup) <- makeJobserver sem +runJSemAbstractSem sem_ident action = MC.mask \ unmask -> do + (abs, cleanup) <- makeJobserver sem_ident r <- try $ unmask $ action abs case r of Left (e1 :: MC.SomeException) -> do @@ -517,8 +546,13 @@ increases the number of `free` jobs. If there are more pending jobs when the fre is increased, the token is immediately reused (see `modifyJobResources`). The `jobServerLoop` interacts with the system semaphore: when there are pending -jobs, `acquireThread` blocks, waiting for a token from the semaphore. Once a -token is obtained, it increases the owned count. +jobs, `acquireThread` forks a child that calls the interruptible +`waitOnSemaphore`. The child is forked in the masked state, so the only place +an async exception can be delivered is the wait itself; once the wait returns, +the child's STM commit always completes, recording either the new token in +`heldTokens` or the failure exception in `threadFinished`. The (tid, tmvar) +pair is also published in `activeChild` so the loop's exit handler can drain +the child on shutdown even after the in-loop `JobserverState` is gone. When GHC has free tokens (tokens from the semaphore that it is not using), no pending jobs, and the debounce has expired, then `releaseThread` will @@ -531,6 +565,12 @@ This second token is no longer needed, so we should cancel the wait (as it would not be used to do any work, and not be returned until the debounce). We only need to kill `acquireJob`, because `releaseJob` never blocks. +Shutdown starts with `killThread loop_tid`. The loop's exit handler then +runs `drainActiveChild` followed by `releaseAllHeld`; only then does the +loop signal `loop_finished_mvar`. This sequence makes the heldTokens +snapshot consistent because no other thread can mutate it once the loop and +its child are both dead. + Note [Eventlog Messages for jsem] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ It can be tricky to verify that the work is shared adequately across different ===================================== compiler/GHC/Driver/Session.hs ===================================== @@ -2445,6 +2445,8 @@ wWarningFlagsDeps = [minBound..maxBound] >>= \x -> case x of Opt_WarnUnusableUnpackPragmas -> warnSpec x Opt_WarnPatternNamespaceSpecifier -> warnSpec x Opt_WarnUnrecognisedModifiers -> warnSpec x + Opt_WarnSemaphoreVersionMismatch -> warnSpec x + Opt_WarnSemaphoreOpenFailure -> warnSpec x warningGroupsDeps :: [(Deprecation, FlagSpec WarningGroup)] warningGroupsDeps = map mk warningGroups ===================================== compiler/GHC/Types/Error/Codes.hs ===================================== @@ -403,6 +403,8 @@ type family GhcDiagnosticCode c = n | n -> c where GhcDiagnosticCode "DriverInstantiationNodeInDependencyGeneration" = 74284 GhcDiagnosticCode "DriverNoConfiguredLLVMToolchain" = 66599 GhcDiagnosticCode "DriverMissingLinkableForModule" = 47338 + GhcDiagnosticCode "DriverSemaphoreVersionMismatch" = 56206 + GhcDiagnosticCode "DriverSemaphoreOpenFailure" = 19877 -- Constraint solver diagnostic codes GhcDiagnosticCode "BadTelescope" = 97739 ===================================== docs/users_guide/using-warnings.rst ===================================== @@ -2721,6 +2721,37 @@ of ``-W(no-)*``. f :: a %True -> a g :: a %(k :: Int) -> a +.. ghc-flag:: -Wsemaphore-version-mismatch + :shortdesc: warn when GHC receives a ``-jsem`` semaphore whose protocol + version is incompatible with the version this GHC supports. + :type: dynamic + :reverse: -Wno-semaphore-version-mismatch + :category: + + :since: 10.0.1 + + Warn when GHC is invoked with :ghc-flag:`-jsem` but the semaphore + name indicates a protocol version that is incompatible with this GHC + (e.g. an unversioned v1 name passed to a v2 GHC, or vice versa). + When this occurs, GHC ignores ``-jsem`` and compiles modules sequentially. + + This situation typically arises when ``cabal-install`` and GHC are built + against different versions of the ``semaphore-compat`` library. Upgrading + both to versions that use the same protocol resolves the mismatch. + +.. ghc-flag:: -Wsemaphore-open-failure + :shortdesc: warn when GHC cannot open the ``-jsem`` semaphore. + :type: dynamic + :reverse: -Wno-semaphore-open-failure + :category: + + :since: 10.0.1 + + Warn when GHC is invoked with :ghc-flag:`-jsem` but the semaphore + cannot be opened (e.g. the socket does not exist or a system error + occurred). When this occurs, GHC ignores ``-jsem`` and compiles + modules sequentially. + ---- If you're feeling really paranoid, the :ghc-flag:`-dcore-lint` option is a good choice. ===================================== docs/users_guide/using.rst ===================================== @@ -797,7 +797,14 @@ There are two kinds of participants in the GHC Jobserver protocol: Perform compilation in parallel when possible, coordinating with other processes through the semaphore ⟨sem⟩ (specified as a string). - Error if the semaphore doesn't exist. + + If the semaphore's protocol version is incompatible, GHC emits a + :ghc-flag:`-Wsemaphore-version-mismatch` warning and compiles + sequentially. If the semaphore cannot be opened for other reasons + (e.g. the socket does not exist), GHC emits a + :ghc-flag:`-Wsemaphore-open-failure` warning and compiles + sequentially. In both cases GHC uses only the implicit token + inherited from the parent process. Use of ``-jsem`` will override use of :ghc-flag:`-j[⟨n⟩]`, and vice-versa. ===================================== hadrian/cabal.project ===================================== @@ -1,6 +1,7 @@ packages: ./ ../utils/ghc-toolchain/ ../libraries/ghc-platform/ + ../libraries/semaphore-compat/ -- This essentially freezes the build plan for hadrian -- It would be wise to keep this up to date with the state set in .gitlab/ci.sh. ===================================== hadrian/hadrian.cabal ===================================== @@ -172,6 +172,7 @@ executable hadrian , base16-bytestring >= 0.1.1 && < 1.1.0.0 , ghc-platform , ghc-toolchain + , semaphore-compat ghc-options: -Wall -Wincomplete-record-updates -Wredundant-constraints ===================================== hadrian/src/Flavour.hs ===================================== @@ -149,10 +149,6 @@ werror = -- unix has many unused imports , package unix ? mconcat [arg "-Wwarn=unused-imports", arg "-Wwarn=unused-top-binds"] - -- semaphore-compat relies on sem_getvalue as provided by unix, which is - -- not implemented on Darwin and therefore throws a deprecation warning - , package semaphoreCompat - ? mconcat [arg "-Wwarn=deprecations"] ] , builder Ghc ? package rts ===================================== hadrian/src/Rules/Generate.hs ===================================== @@ -25,6 +25,7 @@ import Utilities import GHC.Toolchain as Toolchain hiding (HsCpp(HsCpp)) import GHC.Platform.ArchOS import Settings.Program (ghcWithInterpreter) +import System.Semaphore (semaphoreVersion, getSemaphoreProtocolVersion) -- | Track this file to rebuild generated files whenever it changes. trackGenerateHs :: Expr () @@ -488,6 +489,7 @@ generateSettings settingsFile = do , ("RTS ways", escapeArgs . map show . Set.toList <$> getRtsWays) , ("Relative Global Package DB", pure rel_pkg_db) , ("base unit-id", pure base_unit_id) + , ("Semaphore version", pure (show (getSemaphoreProtocolVersion semaphoreVersion))) ] let showTuple (k, v) = "(" ++ show k ++ ", " ++ show v ++ ")" pure $ case settings of ===================================== hadrian/src/Settings/Packages.hs ===================================== @@ -231,6 +231,10 @@ packageArgs = do , package hpcBin ? builder (Cabal Flags) ? arg "-build-tool-depends" + ------------------------------ semaphore-compat ---------------------------- + , package semaphoreCompat + ? builder (Cabal Flags) ? arg "-build-testing" + ] ghcInternalArgs :: Args ===================================== hadrian/stack.yaml ===================================== @@ -16,6 +16,7 @@ packages: - '.' - '../utils/ghc-toolchain' - '../libraries/ghc-platform' +- '../libraries/semaphore-compat' nix: enable: false ===================================== libraries/semaphore-compat ===================================== @@ -1 +1 @@ -Subproject commit 7929702401d49bc64d809c501ed5fe80aebc3cc1 +Subproject commit 2453a03c00e25e30e321816d53c8dbdb113de08b ===================================== testsuite/tests/diagnostic-codes/codes.stdout ===================================== @@ -21,6 +21,8 @@ [GHC-29747] is untested (constructor = DriverMissingSafeHaskellMode) [GHC-74284] is untested (constructor = DriverInstantiationNodeInDependencyGeneration) [GHC-66599] is untested (constructor = DriverNoConfiguredLLVMToolchain) +[GHC-56206] is untested (constructor = DriverSemaphoreVersionMismatch) +[GHC-19877] is untested (constructor = DriverSemaphoreOpenFailure) [GHC-81325] is untested (constructor = ExpectingMoreArguments) [GHC-78125] is untested (constructor = AmbiguityPreventsSolvingCt) [GHC-84170] is untested (constructor = TcRnModMissingRealSrcSpan) View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/9d4a5f5ecd70d726d78f0f397347418… -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/9d4a5f5ecd70d726d78f0f397347418… You're receiving this email because of your account on gitlab.haskell.org.

1 0

[Git][ghc/ghc][wip/marge_bot_batch_merge_job] 2 commits: Introduce a cache of home module name providers
by Marge Bot (＠marge-bot) 13 May '26

13 May '26

Marge Bot pushed to branch wip/marge_bot_batch_merge_job at Glasgow Haskell Compiler / GHC Commits: 5fab2238 by Wolfgang Jeltsch at 2026-05-12T21:24:27+03:00 Introduce a cache of home module name providers This contribution introduces to the module graph a cache that maps home module names to sets of units providing them and changes the finder to use that cache. This is a performance optimization, especially for multi-home-unit builds. The particular changes are as follows: * In `GHC.Unit.Module.Graph`, `ModuleGraph` is extended with a new field `mg_home_module_name_providers_map`, exposed as `mgHomeModuleNameProvidersMap`. This is a cache that assigns to each home module name the set of IDs of home units that define it. Operations that construct module graphs are updated such that this cache stays synchronized. * In `GHC.Unit.Finder`, `findImportedModule` is changed to pull `mgHomeModuleNameProvidersMap` from `hsc_mod_graph` and pass it to `findImportedModuleNoHsc`, which now does not search home units in arbitrary order but prioritizes those units that the cache mentions as potential providers of the requested module. In addition, this contribution adds variants of the two multi-component compiler performance tests that use 100 units instead of 20, because with just 20 units the benefits from caching of home module name providers are still negligible. The following table shows the total time needed for running both multi-component tests before and after this contribution and with different numbers of units: | # of units | Before | After | |-----------:|-------:|------:| | 20 | 0:12 | 0:12 | | 100 | 0:47 | 0:42 | | 200 | 3:05 | 2:08 | Note that there seems to be a general overhead of 12 seconds that is not attributable to the actual tests, so that the real running times should be 12 seconds smaller than shown above. Resolves #27055. Metric Decrease: MultiComponentModules MultiComponentModulesRecomp Co-authored-by: Matthew Pickering <matthewtpickering(a)gmail.com> Co-authored-by: Fendor <fendor(a)posteo.de> - - - - - 79d9c37f by Cheng Shao at 2026-05-13T08:47:00-04:00 testsuite: mark T22159 as fragile This patch marks T22159 as fragile on Windows for issue described in #27248. Before we get to the bottom of those failures, this unblocks newer Windows runners. - - - - - 7 changed files: - + changelog.d/more-efficient-home-unit-imports-finding - compiler/GHC/Unit/Finder.hs - compiler/GHC/Unit/Module/Graph.hs - testsuite/tests/ffi/should_run/all.T - testsuite/tests/perf/compiler/Makefile - testsuite/tests/perf/compiler/all.T - testsuite/tests/perf/compiler/genMultiComp.py Changes: ===================================== changelog.d/more-efficient-home-unit-imports-finding ===================================== @@ -0,0 +1,15 @@ +section: compiler +synopsis: Introduce a cache of home module name providers +issues: #27055 +mrs: !15888 +description: { + This contribution optimizes the algorithm for finding out which home + unit provides the module that a certain import declaration refers + to. The previous approach has been to simply search all home units + in no particular order. This change introduces a cache that allows + for efficiently determining those complete home units that provide a + certain module name and changes the module-finding algorithm such + that it searches these units before the other home units. This leads + to significant performance improvements in situations where there + are lots of home units. +} ===================================== compiler/GHC/Unit/Finder.hs ===================================== @@ -44,6 +44,11 @@ import GHC.Data.OsPath import GHC.Unit.Env import GHC.Unit.Types import GHC.Unit.Module +import GHC.Unit.Module.Graph + ( + HomeModuleNameProvidersMap, + mgHomeModuleNameProvidersMap + ) import GHC.Unit.Home import GHC.Unit.Home.Graph (UnitEnvGraph) import qualified GHC.Unit.Home.Graph as HUG @@ -72,7 +77,8 @@ import GHC.Driver.Config.Finder import GHC.Types.Unique.Set import qualified Data.List as L(sort) import Data.List.NonEmpty ( NonEmpty (..) ) -import qualified Data.Set as Set (toList) +import Data.Set (Set) +import qualified Data.Set as Set (empty, intersection, difference, null, toList) import qualified System.Directory as SD import qualified System.OsPath as OsPath import qualified Data.List.NonEmpty as NE @@ -177,12 +183,13 @@ getDirHash dir = do findImportedModule :: HscEnv -> ModuleName -> PkgQual -> IO FindResult findImportedModule hsc_env mod pkg_qual = - let fc = hsc_FC hsc_env - mhome_unit = hsc_home_unit_maybe hsc_env - dflags = hsc_dflags hsc_env - fopts = initFinderOpts dflags + let fc = hsc_FC hsc_env + mb_home_unit = hsc_home_unit_maybe hsc_env + dflags = hsc_dflags hsc_env + fopts = initFinderOpts dflags in do - findImportedModuleNoHsc fc fopts (hsc_unit_env hsc_env) mhome_unit mod pkg_qual + let home_module_name_providers_map = mgHomeModuleNameProvidersMap (hsc_mod_graph hsc_env) + findImportedModuleNoHsc fc fopts (hsc_unit_env hsc_env) home_module_name_providers_map mb_home_unit mod pkg_qual findImportedModuleWithIsBoot :: HscEnv -> ModuleName -> IsBootInterface -> PkgQual -> IO FindResult findImportedModuleWithIsBoot hsc_env mod is_boot pkg_qual = do @@ -195,55 +202,126 @@ findImportedModuleNoHsc :: FinderCache -> FinderOpts -> UnitEnv + -> HomeModuleNameProvidersMap -> Maybe HomeUnit -> ModuleName -> PkgQual -> IO FindResult -findImportedModuleNoHsc fc fopts ue mhome_unit mod_name mb_pkg = +findImportedModuleNoHsc fc fopts ue home_module_name_providers_map mb_home_unit mod_name mb_pkg = case mb_pkg of NoPkgQual -> unqual_import - ThisPkg uid | (homeUnitId <$> mhome_unit) == Just uid -> home_import + ThisPkg uid | (homeUnitId <$> mb_home_unit) == Just uid -> home_import | Just os <- lookup uid other_fopts -> home_pkg_import (uid, os) - | otherwise -> pprPanic "findImportModule" (ppr mod_name $$ ppr mb_pkg $$ ppr (homeUnitId <$> mhome_unit) $$ ppr uid $$ ppr (map fst all_opts)) + | otherwise -> pprPanic "findImportModule" (ppr mod_name $$ ppr mb_pkg $$ ppr (homeUnitId <$> mb_home_unit) $$ ppr uid $$ ppr (map fst all_opts)) OtherPkg _ -> pkg_import where - all_opts = case mhome_unit of - Nothing -> other_fopts - Just home_unit -> (homeUnitId home_unit, fopts) : other_fopts + mb_home_unit_id :: Maybe UnitId + mb_home_unit_id = homeUnitId <$> mb_home_unit - home_import = case mhome_unit of - Just home_unit -> findHomeModule fc fopts home_unit mod_name - Nothing -> pure $ NoPackage (panic "findImportedModule: no home-unit") + all_opts :: [(UnitId, FinderOpts)] + all_opts = case mb_home_unit_id of + Nothing -> other_fopts + Just home_unit_id -> (home_unit_id, fopts) : other_fopts + home_import :: IO FindResult + home_import = case mb_home_unit of + Just home_unit -> findHomeModule fc fopts home_unit mod_name + Nothing -> pure $ + NoPackage (panic "findImportedModule: no home-unit") + home_pkg_import :: (UnitId, FinderOpts) -> IO FindResult home_pkg_import (uid, opts) - -- If the module is reexported, then look for it as if it was from the perspective - -- of that package which reexports it. - | Just real_mod_name <- lookupUniqMap (finder_reexportedModules opts) mod_name = - findImportedModuleNoHsc fc opts ue (Just $ DefiniteHomeUnit uid Nothing) real_mod_name NoPkgQual - | elementOfUniqSet mod_name (finder_hiddenModules opts) = - return (mkHomeHidden uid) - | otherwise = - findHomePackageModule fc opts uid mod_name - - -- Do not be smart and change this to `foldr orIfNotFound home_import hs` as - -- that is not the same!! home_import is first because we need to look within ourselves - -- first before looking at the packages in order. - any_home_import = foldr1 orIfNotFound (home_import:| map home_pkg_import other_fopts) - - pkg_import = findExposedPackageModule fc fopts units mod_name mb_pkg - - unqual_import = any_home_import - `orIfNotFound` - findExposedPackageModule fc fopts units mod_name NoPkgQual - - units = case mhome_unit of - Nothing -> ue_homeUnitState ue - Just home_unit -> HUG.homeUnitEnv_units $ ue_findHomeUnitEnv (homeUnitId home_unit) ue - hpt_deps :: [UnitId] - hpt_deps = Set.toList (homeUnitDepends units) - other_fopts = map (\uid -> (uid, initFinderOpts (homeUnitEnv_dflags (ue_findHomeUnitEnv uid ue)))) hpt_deps + -- If the module is reexported, then look for it as if it was from the + -- perspective of that package which reexports it. + | Just real_mod_name + <- lookupUniqMap (finder_reexportedModules opts) mod_name + = findImportedModuleNoHsc fc opts ue home_module_name_providers_map + (Just $ DefiniteHomeUnit uid Nothing) + real_mod_name + NoPkgQual + | elementOfUniqSet mod_name (finder_hiddenModules opts) + = return (mkHomeHidden uid) + | otherwise + = findHomePackageModule fc opts uid mod_name + + any_home_import :: IO FindResult + any_home_import = foldr1 orIfNotFound $ + home_import :| map home_pkg_import other_fopts + -- Do not try to be smart and change this to `foldr orIfNotFound home_import + -- (map home_pkg_import other_fopts)`, as that would not be the same. + -- `home_import` is first because we need to first look within the current + -- unit before looking at the other units in order. + + pkg_import :: IO FindResult + pkg_import = findExposedPackageModule fc fopts unit_state mod_name mb_pkg + + unqual_import :: IO FindResult + unqual_import + = any_home_import + `orIfNotFound` + findExposedPackageModule fc fopts unit_state mod_name NoPkgQual + + unit_state :: UnitState + unit_state = case mb_home_unit_id of + Nothing -> ue_homeUnitState ue + Just home_unit_id -> HUG.homeUnitEnv_units $ + ue_findHomeUnitEnv home_unit_id ue + + home_unit_deps :: Set UnitId + home_unit_deps = homeUnitDepends unit_state + + ranked_home_unit_deps :: [UnitId] + ranked_home_unit_deps = rankedHomeUnitDeps home_module_name_providers_map + mod_name + home_unit_deps + + other_fopts :: [(UnitId, FinderOpts)] + other_fopts + = [ + (uid, opts) | + uid <- ranked_home_unit_deps, + let opts = initFinderOpts $ + homeUnitEnv_dflags (ue_findHomeUnitEnv uid ue) + ] + +-- | Yields the unit IDs from the given set as a list with those that refer to +-- providers of the given home module name coming first. This is to prioritize +-- such providers during module finding. +rankedHomeUnitDeps :: HomeModuleNameProvidersMap + -> ModuleName + -> Set UnitId + -> [UnitId] +rankedHomeUnitDeps _ _ home_unit_deps | Set.null home_unit_deps + = [] +-- The special handling of the situation where the dependency set is empty does +-- not change the result, but it avoids triggering evaluation of the module +-- graph. This is particularly important in one-shot mode, where the module +-- graph is not needed. Computing it nevertheless would result in a, possibly +-- dramatic, increase of memory usage. Worse, GHC would erroneously look for the +-- sources of modules, which would, for example, cause test `boot1` to fail with +-- the following error message: +-- +-- B.hs:3:1: error: [GHC-87110] +-- Could not find module ‘A’. +-- Use -v to see a list of the files searched for. +-- | +-- 3 | import {-# source #-} A +-- | ^^^^^^^^^^^^^^^^^^^^^^^ +rankedHomeUnitDeps home_module_name_providers_map mod_name home_unit_deps + = Set.toList cached_deps ++ Set.toList uncached_deps + where + + cached_providers :: Set UnitId + cached_providers = lookupWithDefaultUniqMap home_module_name_providers_map + Set.empty + mod_name + + cached_deps :: Set UnitId + cached_deps = Set.intersection home_unit_deps cached_providers + + uncached_deps :: Set UnitId + uncached_deps = Set.difference home_unit_deps cached_providers -- | Locate a plugin module requested by the user, for a compiler -- plugin. This consults the same set of exposed packages as @@ -261,15 +339,15 @@ findPluginModule :: HscEnv -> ModuleName -> IO FindResult findPluginModule hsc_env mod_name = do let fc = hsc_FC hsc_env let units = hsc_units hsc_env - let mhome_unit = hsc_home_unit_maybe hsc_env - findPluginModuleNoHsc fc (initFinderOpts (hsc_dflags hsc_env)) units mhome_unit mod_name + let mb_home_unit = hsc_home_unit_maybe hsc_env + findPluginModuleNoHsc fc (initFinderOpts (hsc_dflags hsc_env)) units mb_home_unit mod_name -- | A version of findExactModule which takes the exact parts of the HscEnv it needs -- directly. findExactModuleNoHsc :: FinderCache -> FinderOpts -> UnitEnvGraph FinderOpts -> UnitState -> Maybe HomeUnit -> InstalledModule -> IsBootInterface -> IO InstalledFindResult -findExactModuleNoHsc fc fopts other_fopts unit_state mhome_unit mod is_boot = do - res <- case mhome_unit of +findExactModuleNoHsc fc fopts other_fopts unit_state mb_home_unit mod is_boot = do + res <- case mb_home_unit of Just home_unit | isHomeInstalledModule home_unit mod -> findInstalledHomeModule fc fopts (homeUnitId home_unit) (moduleName mod) ===================================== compiler/GHC/Unit/Module/Graph.hs ===================================== @@ -67,6 +67,8 @@ module GHC.Unit.Module.Graph , mgLookupModule , mgLookupModuleName , mgHasHoles + , HomeModuleNameProvidersMap + , mgHomeModuleNameProvidersMap , showModMsg -- ** Reachability queries @@ -156,10 +158,12 @@ import GHC.Unit.Module.ModIface import GHC.Utils.Misc ( partitionWith ) import System.FilePath +import Data.Set (Set) +import qualified Data.Set as Set +import Data.Map (Map) import qualified Data.Map as Map import GHC.Types.Unique.DSet -import qualified Data.Set as Set -import Data.Set (Set) +import GHC.Types.Unique.Map (UniqMap, emptyUniqMap, listToUniqMap_C) import GHC.Unit.Module import GHC.Unit.Module.ModNodeKey import GHC.Unit.Module.Stage @@ -202,14 +206,32 @@ data ModuleGraph = ModuleGraph -- Cached computation, whether any of the ModuleGraphNode are isHoleModule, -- This is only used for a hack in GHC.Iface.Load to do with backpack, please -- remove this at the earliest opportunity. + , mg_home_module_name_providers_map :: HomeModuleNameProvidersMap + -- ^ For each module name, which home units provide it. } +type HomeModuleNameProvidersMap = UniqMap ModuleName (Set UnitId) + +mkHomeModuleNameProvidersMap :: [ModuleGraphNode] -> HomeModuleNameProvidersMap +mkHomeModuleNameProvidersMap nodes + = listToUniqMap_C Set.union $ + [ + (moduleName, Set.singleton unitID) | + ModuleNode _ moduleNodeInfo <- nodes, + let moduleName = moduleNodeInfoModuleName moduleNodeInfo, + let unitID = moduleNodeInfoUnitId moduleNodeInfo + ] + +mgHomeModuleNameProvidersMap :: ModuleGraph -> HomeModuleNameProvidersMap +mgHomeModuleNameProvidersMap = mg_home_module_name_providers_map + -- | Why do we ever need to construct empty graphs? Is it because of one shot mode? emptyMG :: ModuleGraph emptyMG = ModuleGraph [] (graphReachability emptyGraph, const Nothing) (graphReachability emptyGraph, const Nothing) (graphReachability emptyGraph, const Nothing) False + emptyUniqMap -- | Construct a module graph. This function should be the only entry point for -- building a 'ModuleGraph', since it is supposed to be built once and never modified. @@ -308,7 +330,7 @@ checkModuleGraph ModuleGraph{..} = where duplicate_errs = rights (Map.elems node_types) - node_types :: Map.Map NodeKey (Either ModuleNodeType ModuleGraphInvariantError) + node_types :: Map NodeKey (Either ModuleNodeType ModuleGraphInvariantError) node_types = Map.fromListWithKey go [ (mkNodeKey n, Left (moduleNodeType n)) | n <- mg_mss ] where -- Multiple nodes with the same key are not allowed. @@ -319,7 +341,7 @@ checkModuleGraph ModuleGraph{..} = -- | Check that all dependencies in the graph are present in the node_types map. -- This is a helper function used by checkModuleGraph. -checkAllDependenciesInGraph :: Map.Map NodeKey (Either ModuleNodeType ModuleGraphInvariantError) +checkAllDependenciesInGraph :: Map NodeKey (Either ModuleNodeType ModuleGraphInvariantError) -> ModuleGraphNode -> Maybe ModuleGraphInvariantError checkAllDependenciesInGraph node_types node = @@ -334,7 +356,7 @@ checkAllDependenciesInGraph node_types node = -- | Check if for the fixed module node invariant: -- -- Fixed nodes can only depend on other fixed nodes. -checkFixedModuleInvariant :: Map.Map NodeKey (Either ModuleNodeType ModuleGraphInvariantError) +checkFixedModuleInvariant :: Map NodeKey (Either ModuleNodeType ModuleGraphInvariantError) -> ModuleGraphNode -> Maybe ModuleGraphInvariantError checkFixedModuleInvariant node_types node = case node of @@ -484,13 +506,17 @@ isEmptyMG = null . mg_mss -- To preserve invariants, 'f' can't change the isBoot status. mapMG :: (ModSummary -> ModSummary) -> ModuleGraph -> ModuleGraph mapMG f mg@ModuleGraph{..} = mg - { mg_mss = flip fmap mg_mss $ \case - InstantiationNode uid iuid -> InstantiationNode uid iuid - LinkNode uid nks -> LinkNode uid nks - ModuleNode deps (ModuleNodeFixed key loc) -> ModuleNode deps (ModuleNodeFixed key loc) - ModuleNode deps (ModuleNodeCompile ms) -> ModuleNode deps (ModuleNodeCompile (f ms)) - UnitNode deps uid -> UnitNode deps uid + { mg_mss = new_mss + , mg_home_module_name_providers_map = mkHomeModuleNameProvidersMap new_mss } + where + new_mss = + flip fmap mg_mss $ \case + InstantiationNode uid iuid -> InstantiationNode uid iuid + LinkNode uid nks -> LinkNode uid nks + ModuleNode deps (ModuleNodeFixed key loc) -> ModuleNode deps (ModuleNodeFixed key loc) + ModuleNode deps (ModuleNodeCompile ms) -> ModuleNode deps (ModuleNodeCompile (f ms)) + UnitNode deps uid -> UnitNode deps uid -- | Map a function 'f' over all the 'ModSummaries', in 'IO'. -- To preserve invariants, 'f' can't change the isBoot status. @@ -856,7 +882,7 @@ moduleNodeInfoBootString mn@(ModuleNodeFixed {}) = -- described in the export list haddocks. -------------------------------------------------------------------------------- -newtype NodeMap a = NodeMap { unNodeMap :: Map.Map NodeKey a } +newtype NodeMap a = NodeMap { unNodeMap :: Map NodeKey a } deriving (Functor, Traversable, Foldable) -- | Transitive dependencies, including SOURCE edges @@ -932,7 +958,7 @@ moduleGraphNodesZero summaries = lookup_key :: ZeroScopeKey -> Maybe Int lookup_key = fmap zeroSummaryNodeKey . lookup_node - node_map :: Map.Map ZeroScopeKey ZeroSummaryNode + node_map :: Map ZeroScopeKey ZeroSummaryNode node_map = Map.fromList [ (s, node) | node <- nodes @@ -1031,7 +1057,7 @@ moduleGraphNodesStages summaries = lookup_key :: (NodeKey, ModuleStage) -> Maybe Int lookup_key = fmap stageSummaryNodeKey . lookup_node - node_map :: Map.Map (NodeKey, ModuleStage) StageSummaryNode + node_map :: Map (NodeKey, ModuleStage) StageSummaryNode node_map = Map.fromList [ (s, node) | node <- nodes @@ -1049,10 +1075,13 @@ moduleGraphNodesStages summaries = extendMG :: ModuleGraph -> ModuleGraphNode -> ModuleGraph extendMG ModuleGraph{..} node = ModuleGraph - { mg_mss = node : mg_mss - , mg_graph = mkTransDeps (node : mg_mss) - , mg_loop_graph = mkTransLoopDeps (node : mg_mss) - , mg_zero_graph = mkTransZeroDeps (node : mg_mss) + { mg_mss = new_mss + , mg_graph = mkTransDeps new_mss + , mg_loop_graph = mkTransLoopDeps new_mss + , mg_zero_graph = mkTransZeroDeps new_mss , mg_has_holes = mg_has_holes || maybe False isHsigFile (moduleNodeInfoHscSource =<< mgNodeIsModule node) + , mg_home_module_name_providers_map = mkHomeModuleNameProvidersMap new_mss } + where + new_mss = node : mg_mss ===================================== testsuite/tests/ffi/should_run/all.T ===================================== @@ -249,6 +249,7 @@ test('T21305', [cmm_src], multi_compile_and_run, test('T22159', [unless(opsys('mingw32'), skip), + fragile(27248), extra_files(['T22159_c.c'])], makefile_test, ['T22159']) ===================================== testsuite/tests/perf/compiler/Makefile ===================================== @@ -31,7 +31,11 @@ MultiModulesDefsWithCore: ./genMultiLayerModulesCore MultiComponentModulesRecomp: - '$(PYTHON)' genMultiComp.py + '$(PYTHON)' genMultiComp.py 20 20 + TEST_HC='$(TEST_HC)' TEST_HC_OPTS='$(TEST_HC_OPTS)' ./run + +MultiComponentModulesRecomp100: + '$(PYTHON)' genMultiComp.py 100 20 TEST_HC='$(TEST_HC)' TEST_HC_OPTS='$(TEST_HC_OPTS)' ./run MultiLayerModulesTH_Make_Prep: ===================================== testsuite/tests/perf/compiler/all.T ===================================== @@ -507,13 +507,31 @@ test('MultiComponentModulesRecomp', test('MultiComponentModules', [ collect_compiler_runtime(2), - pre_cmd('$PYTHON ./genMultiComp.py'), + pre_cmd('$PYTHON ./genMultiComp.py 20 20'), extra_files(['genMultiComp.py']), compile_timeout_multiplier(5) ], multiunit_compile, [['unitp%d' % n for n in range(20)], '-fno-code -fwrite-interface -v0']) +test('MultiComponentModulesRecomp100', + [ collect_compiler_runtime(2), + pre_cmd('$MAKE -s --no-print-directory MultiComponentModulesRecomp100'), + extra_files(['genMultiComp.py']), + compile_timeout_multiplier(5) + ], + multiunit_compile, + [['unitp%d' % n for n in range(100)], '-fno-code -fwrite-interface -v0']) + +test('MultiComponentModules100', + [ collect_compiler_runtime(2), + pre_cmd('$PYTHON ./genMultiComp.py 100 20'), + extra_files(['genMultiComp.py']), + compile_timeout_multiplier(5) + ], + multiunit_compile, + [['unitp%d' % n for n in range(100)], '-fno-code -fwrite-interface -v0']) + test('ManyConstructors', [ collect_compiler_stats('bytes allocated',2), pre_cmd('./genManyConstructors'), ===================================== testsuite/tests/perf/compiler/genMultiComp.py ===================================== @@ -7,11 +7,12 @@ # * A number of modules names Mod_<pid>_<mid>, each module imports all the top # modules beneath it, and all the modules in the current unit beneath it. +import sys import os import stat -modules_per = 20 -packages = 20 +packages = int(sys.argv[1]) +modules_per = int(sys.argv[2]) total = modules_per * packages def unit_dir(p): View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/2ceed4507e06a45de307f1152c9fd1… -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/2ceed4507e06a45de307f1152c9fd1… You're receiving this email because of your account on gitlab.haskell.org.

1 0

[Git][ghc/ghc][wip/sol/dont-use-global-variables] 28 commits: Use __attribute__((dllimport)) for external RTS symbol declarations
by Simon Hengel (＠sol) 13 May '26

13 May '26

Simon Hengel pushed to branch wip/sol/dont-use-global-variables at Glasgow Haskell Compiler / GHC Commits: 9a9ae4df by Duncan Coutts at 2026-05-05T14:44:37-04:00 Use __attribute__((dllimport)) for external RTS symbol declarations This is needed to be hygenic about DLL symbol imports and exports. The attribute is ignored on platforms other than Windows. Use of the attribute however means that external data symbols do not have a compile-time constant address (they are loaded using an indirection). This means we have to adjust the rtsSyms initial linker table so that it is a local constant in a function, rather than a global constant. We now define it within a function that pre-populates the symbol table with the RTS symbols. - - - - - 2ad3e01e by Duncan Coutts at 2026-05-05T14:44:37-04:00 Fix the rts linker declarations for a few data symbols and ensure that the (windows only) rts_IOManagerIsWin32Native data symbol is marked as externally visible. - - - - - 8ff4fdb5 by David Eichmann at 2026-05-05T14:44:37-04:00 Hadrian: Disable runtime pseudo relocations for RTS on windows hosts - - - - - 96974723 by Teo Camarasu at 2026-05-05T14:45:20-04:00 ghci/TH: refactor to use IORef QState This is a pure refactor and shouldn't modify semantics at all - - - - - eff6bfaf by Teo Camarasu at 2026-05-05T14:45:20-04:00 iserv: recover/getQ/putQ should behave same as internal interpreter The internal and external interpreter should behave the same when handling `recover`, the exeception recovery method of Q. In practice, they diverge. In case of failure, the internal interpreter only restores error message state to before the computation, wheras the external interperter restores error message state *and* the state of putQ/getQ. As far as I can tell this is a simple mistake in the implementation. Note [TH recover with -fexternal-interpreter] describes the correct behaviour but the implementation doesn't mirror this. This change restores the correct behaviour by keeping the effects of putQ in the erroring computation. This is a breaking change since it modifies the behaviour of programs that rely on recover ignoring putQ from failling computations when used with the external interpreter. Although I highly doubt anyone relies on this behaviour. This divergence was first introduced in d00c308633fe7d216d31a1087e00e63532d87d6d. As far as I can tell this was unintentional and tha commit was trying to solve a different bug. Resolves #27022 - - - - - 1cb1d672 by Wen Kokke at 2026-05-06T09:53:40-04:00 rts: Add dynamic trace flags API This commit adds an API to the RTS (exposed via Rts.h) that allows users to dynamically change the trace flags. Prior to this commit, users were able to stop and start the profiling and heap profiling timers (via startProfTimer/stopProfTimer and startHeapProfTimer/stopHeapProfTimer). This extends that functionality to also cover the core event types. The getTraceFlag/setTraceFlag functions read and write the values of the trace flag cache, which is allocated by Trace.c, rather than modifying the members of RtsFlags.TraceFlags. This is done under the assumption that the members of RtsFlags should not be modified after RTS initialisation. Consequently, if the user modifies the trace flags using setTraceFlag, the object returned by getTraceFlags (from base) will not reflect these changes. The trace flags are not protected by locks of any sort. Hence, these functions are not thread-safe. However, the trace flags are not modified by the RTS after initialisation, only read, so the race conditions introduced by one user modifying them are most likely benign. This PR also puts the trace flag cache in a single global struct, as opposed to a collection of global variables, and changes the types of the individual flags from uint8_t to bool, as these have the same size on both Clang and GCC and are a better semantic match. Prior to the change to uint8_t, they had type int, see 42c47cd6. Even with its deprecation in C23, I don't think there should be any issue depending on stdbool.h. The TRACE_X macros are redefined to access the global struct, with values cast to const bool to ensure they are read-only. - - - - - 9d54dc94 by Wen Kokke at 2026-05-06T09:53:40-04:00 rts: Ensure TRACE_X values are used in place of RtsFlags.TraceFlags.X - - - - - 418d737b by Wen Kokke at 2026-05-06T09:53:40-04:00 rts: Fix nonmoving-GC tracing The current nonmoving-GC tracing functions were written in a different style from the other tracing functions. They were directly implemented as, e.g., a traceConcMarkEnd function that called postConcMarkEnd. The other tracing functions are implemented as, e.g., traceThreadLabel_, a function that posts the thread label event, and traceThreadLabel, a macro that checks whether TRACE_scheduler is set. This commit fixes that implementation, and ensures that the nonmoving-GC tracing functions only emit events if nonmoving-GC tracing is enabled. - - - - - 99f4afa4 by Wen Kokke at 2026-05-06T09:53:40-04:00 rts: Add SymI_HasProto for get/setTraceFlag - - - - - 7e9eb8b9 by Wen Kokke at 2026-05-06T09:53:40-04:00 rts: Add SymI_HasProto for start/endEventLogging - - - - - 3a3045fb by Wen Kokke at 2026-05-06T09:53:41-04:00 rts: Add changelog entry - - - - - a3b339a4 by Teo Camarasu at 2026-05-06T09:54:25-04:00 interface-stability/base: don't distinguish ws-32 The interface of base is identical when the Word size is 32bits. Therefore, there is no need to have another file for this case. So, we delete it. Step towards: #26752 - - - - - eb922183 by Duncan Coutts at 2026-05-07T14:28:50+01:00 Add a rts posix FdWakup utility module This will be used to implement wakeupIOManager for in-RTS I/O managers. It provides a notification/wakeup mechanism using FDs, suitable for situations when a thread is blocked on a set of fds anyway. It uses the classic self-pipe trick, or equivalently eventfd on supported platforms. This will initially be used to implement prompt interrupt or shutdown of the posix ticker thread. - - - - - 01b0e233 by Duncan Coutts at 2026-05-07T14:28:50+01:00 Add prompt shutdown to the pthread ticker implementation. The Linux timerfd ticker monitors a pipe which is used by exitTicker to ensure a prompt wakeup and shutdown. The pthread ticker lacked this and so would only exit at the next ticker wakeup (10ms by default). This patch adds the same mechanism to the pthread ticker. This changes the pthread ticker from waiting by using nanosleep() to waiting using either ppoll() or select(), so that it can wait on both a time and a file descriptor. On Linux at least, a test program to compare the timing jitter of these APIs shows that using nanpsleep, ppoll or select makes no statistical difference to the maximum or average jitter. This is a step towards unifying the posix ticker implementations, so that we can have just one portable one (albeit with some limited cpp). It is also a step towards using the ticker as part of a more general implementation of wakeUpRts, since this will require a method to wake the rts from a signal handler context (ctl-c handler). - - - - - bc41d646 by Duncan Coutts at 2026-05-07T14:28:50+01:00 Update ticker header commentary It was antique and didn't apply even to the previous implementation, and certainly not to the updated one. - - - - - 4ed9a386 by Duncan Coutts at 2026-05-07T14:28:50+01:00 Remove the timerfd-based ticker implementation There does not appear to be any remaining advantage on Linux to using the timerfd ticker implementation over the portable one (using ppoll on Linux for precise timing). The eventfd implementation was originally added at a time when Linux was still using a signal based implementation. So it made sense at the time. See (closed) issue #10840. - - - - - 97504fa6 by Duncan Coutts at 2026-05-07T14:28:50+01:00 Consolidate to a single posix ticker implementation Previously we had four implementations, two using signals and two using threads. Having just one should make behaviour more consistent between platforms, and should make maintenance easier. - - - - - 1e60023b by Facundo Domínguez at 2026-05-07T18:01:16-04:00 Generalize so_inline to specify which bindings should be preserved This commit generalizes the so_inline option of the simple optimizer so we can indicate with a predicate the specific bindings that should be kept. This feature is important for the LiquidHaskell plugin, which relies on the simple optimizer to make core programs easier to read, but needs to preserve bindings that are relevant for verification. See https://gitlab.haskell.org/ghc/ghc/-/issues/24386 for the full discussion. - - - - - 44cf9cd7 by Wolfgang Jeltsch at 2026-05-12T09:48:18-04:00 Move the `Text.Read` implementation into `base` - - - - - 4ac3f7d6 by Vladislav Zavialov at 2026-05-12T09:49:03-04:00 EPA: Use AnnParen for tuples and sums Summary of changes * Do not use AnnParen in XListTy, replace it with EpToken "[" and "]" * Specialise AnnParen to tuple/sums by dropping the AnnParensSquare and keeping only AnnParens and AnnParensHash * Use AnnParen in XExplicitTuple * Use AnnParen in XExplicitTupleTy * Use AnnParen in XTuplePat * Use AnnParen in XExplicitSum (via AnnExplicitSum) * Use AnnParen in XSumPat (via EpAnnSumPat) This is a refactoring with no user-facing changes. - - - - - 1bdcddec by Duncan Coutts at 2026-05-12T09:49:48-04:00 Add minimal dlltool support to ghc-toolchain The dlltool is a tool that can create dll import libraries from .def files. These .def files list the exported symbols of dlls. Its somewhat like gnu linker scripts, but more limited. We will need dlltool to build the rts and ghc-internal libraries as DLLs on Windows. The rts and ghc-internal libraries have a recursive dependency on each other. Import libraries can be used to resolve recursive dependencies between dlls. We will use an import library for the rts when linking the ghc-internal library. - - - - - f7fc3770 by Duncan Coutts at 2026-05-12T09:49:48-04:00 Add minimal dlltool support into ./configure Find dlltool, and hopefully support finding it within the bundled llvm toolchain on windows. - - - - - e4e22bfb by Duncan Coutts at 2026-05-12T09:49:48-04:00 Update the default host and target files for dlltool support - - - - - 5666c8f9 by Duncan Coutts at 2026-05-12T09:49:48-04:00 Add dlltool as a hadrian builder Optional except on windows. - - - - - 5e14fe3f by Duncan Coutts at 2026-05-12T09:49:48-04:00 Update and generate libHSghc-internal.def from .def.in file The only symbol that the rts imports from the ghc-internal package now is init_ghc_hs_iface. So the rts only needs an import lib that defines that one symbol. Also, remove the libHSghc-prim.def because it is redundant. The rts no longer imports anything from ghc-prim. Keep libHSffi.def for now. We may yet need it once it is clear how libffi is going to be built/used for ghc. - - - - - 3d91e4a6 by Duncan Coutts at 2026-05-12T09:49:48-04:00 Add rule to build libHSghc-internal.dll.a and link into the rts On windows only, with dynamic linking. This is needed because on windows, all symbols in dlls must be resolved. No dangling symbols allowed. References to external symbols must be explicit. We resolve this with an import library. We create an import library for ghc-internal, a .dll.a file. This is a static archive containing .o files that define the symbols we need, and crucially have ".idata" sections that specifies the symbols the dll imports and from where. Note that we do not install this libHSghc-internal.dll.a, and it does not need to list all the symbols exported by that package. We create a special purpose import lib and only use it when linking the rts dll, so it only has to list the symbols that the rts uses from ghc-internal (which is exactly one symbol: init_ghc_hs_iface). - - - - - c8dae539 by Alice Rixte at 2026-05-12T09:50:52-04:00 Script for downloading and copying `base-exports` file - - - - - dea6fdbf by Simon Hengel at 2026-05-13T19:15:39+07:00 Don't use global variables to address concurrency bugs! (fixes #27234) This was originally introduce with 88f38b03025386f0f1e8f5861eed67d80495168a to address #17922. In this specific case a better fix would have been to synchronize on stderr: withHandle_ "stderrSupportsAnsiColors" stderr $ \ _ -> do ... But apparently the dependency on `terminfo` was removed in 32ab07bf3d6ce45e8ea5b55e8095174a6b42a7f0, preventing #17922 in the first place. - - - - - 74 changed files: - + changelog.d/T27022 - + changelog.d/dynamic-trace-flags - + changelog.d/ghc-api-epa-parens - + changelog.d/so_inline_is_a_predicate - compiler/GHC/Core/SimpleOpt.hs - compiler/GHC/Driver/Config.hs - compiler/GHC/Driver/DynFlags.hs - compiler/GHC/Hs/Dump.hs - compiler/GHC/Hs/Expr.hs - compiler/GHC/Hs/Pat.hs - compiler/GHC/Hs/Type.hs - compiler/GHC/Parser.y - compiler/GHC/Parser/Annotation.hs - compiler/GHC/Parser/PostProcess.hs - compiler/GHC/SysTools/Terminal.hs - configure.ac - distrib/configure.ac.in - hadrian/cfg/default.host.target.in - hadrian/cfg/default.target.in - hadrian/src/Builder.hs - hadrian/src/Rules/Generate.hs - hadrian/src/Rules/Library.hs - hadrian/src/Rules/Rts.hs - hadrian/src/Settings/Packages.hs - libraries/base/src/Data/Functor/Classes.hs - libraries/base/src/Data/Functor/Compose.hs - libraries/base/src/Prelude.hs - libraries/base/src/Text/Read.hs - libraries/ghc-internal/ghc-internal.cabal.in - libraries/ghc-internal/src/GHC/Internal/IO/Encoding.hs - − libraries/ghc-internal/src/GHC/Internal/Text/Read.hs - libraries/ghci/GHCi/TH.hs - m4/find_llvm_prog.m4 - m4/fp_setup_windows_toolchain.m4 - m4/ghc_toolchain.m4 - m4/prep_target_file.m4 - rts/.gitignore - rts/IOManager.h - rts/Linker.c - rts/LinkerInternals.h - rts/RtsSymbols.c - rts/RtsSymbols.h - rts/Trace.c - rts/Trace.h - rts/include/rts/EventLogWriter.h - rts/linker/Elf.c - + rts/posix/FdWakeup.c - + rts/posix/FdWakeup.h - rts/posix/Ticker.c - − rts/posix/ticker/Pthread.c - − rts/posix/ticker/TimerFd.c - rts/rts.cabal - rts/sm/NonMoving.c - + rts/win32/libHSghc-internal.def.in - + testsuite/tests/ghc-api/T24386.hs - testsuite/tests/ghc-api/T25121_status.stdout - testsuite/tests/ghc-api/all.T - + testsuite/tests/interface-stability/.gitignore - testsuite/tests/interface-stability/README.mkd - − testsuite/tests/interface-stability/base-exports.stdout-ws-32 - + testsuite/tests/interface-stability/download-base-exports.sh - testsuite/tests/parser/should_compile/DumpParsedAst.stderr - testsuite/tests/parser/should_compile/DumpRenamedAst.stderr - testsuite/tests/parser/should_compile/KindSigs.stderr - testsuite/tests/parser/should_compile/T20452.stderr - testsuite/tests/th/T24111.stdout - + testsuite/tests/th/T27022.hs - + testsuite/tests/th/T27022.stdout - testsuite/tests/th/all.T - testsuite/tests/typecheck/should_compile/subsumption_sort_hole_fits.stderr - testsuite/tests/typecheck/should_fail/T21130.stderr - utils/check-exact/ExactPrint.hs - utils/ghc-toolchain/exe/Main.hs - utils/ghc-toolchain/src/GHC/Toolchain/Target.hs The diff was not included because it is too large. View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/a3c8f7c88ecd3814ede806604eae0e… -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/a3c8f7c88ecd3814ede806604eae0e… You're receiving this email because of your account on gitlab.haskell.org.

1 0