28 Oct '26
Hannes Siebenhandl pushed new branch wip/fendor/27700 at Glasgow Haskell Compiler / GHC
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/tree/wip/fendor/27700
You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help
1
0
[Git][ghc/ghc] Pushed new branch wip/andrea/displayExceptionWithInfo
by Andrea Vezzosi (@trac-Saizan) 28 Oct '26
by Andrea Vezzosi (@trac-Saizan) 28 Oct '26
28 Oct '26
Andrea Vezzosi pushed new branch wip/andrea/displayExceptionWithInfo at Glasgow Haskell Compiler / GHC
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/tree/wip/andrea/displayExceptionWithIn…
You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help
1
0
[Git][ghc/ghc][wip/romes/interactive-ids-refactor] refactor: Keep the interactive Ids directly rather than Name
by Rodrigo Mesquita (@alt-romes) 28 Oct '26
by Rodrigo Mesquita (@alt-romes) 28 Oct '26
28 Oct '26
Rodrigo Mesquita pushed to branch wip/romes/interactive-ids-refactor at Glasgow Haskell Compiler / GHC
Commits:
26fbcce5 by Rodrigo Mesquita at 2026-10-28T14:16:37+00:00
refactor: Keep the interactive Ids directly rather than Name
Removes things like
names <- map idName ids
map (\i -> let id = find (\x -> i==idName x) names) idso
and is more straightforward because there's less state going around (no
more updating HscEnv as an output)
It also kills `printTypeOfNames` completely, since now we just have the
Ids. No longer `lookupName` names to get back the ids...
```
printTypeOfName :: GHC.GhcMonad m => Name -> m ()
printTypeOfName n
= do maybe_tything <- GHC.lookupName n
case maybe_tything of
Nothing -> return ()
Just thing -> printTyThing thing
```
now is just `printTyThing . AnId`
Also:
```diff
- tythings <- catMaybes `liftM` mapM GHC.lookupName namesSorted
- docs <- mapM pprTypeAndContents [i | AnId i <- tythings]
+ docs <- mapM pprTypeAndContents idsSorted
```
(and yes, in ExecBreaks there are only Ids, no TyThings, bc they are
only constructed by the interpreter execution which can only bind ids.
for TyThings use runDecls which doesn't return execBreaks.)
There's some care to be had to make sure we use the "most refined" Ids,
whereas when using `Name`s we'd always fetch the most refined Id from
scratch.
Fixes #28052
- - - - -
12 changed files:
- + changelog.d/ghc-api-interactive-ids
- compiler/GHC/Runtime/Context.hs
- compiler/GHC/Runtime/Debugger.hs
- compiler/GHC/Runtime/Eval.hs
- compiler/GHC/Runtime/Eval/Types.hs
- compiler/GHC/Runtime/Heap/Inspect.hs
- ghc/GHCi/UI.hs
- ghc/GHCi/UI/Monad.hs
- + testsuite/tests/ghci.debugger/scripts/RttiSharedTyVar.hs
- + testsuite/tests/ghci.debugger/scripts/RttiSharedTyVar.script
- + testsuite/tests/ghci.debugger/scripts/RttiSharedTyVar.stdout
- testsuite/tests/ghci.debugger/scripts/all.T
Changes:
=====================================
changelog.d/ghc-api-interactive-ids
=====================================
@@ -0,0 +1,15 @@
+section: ghc-lib
+synopsis: The GHCi evaluation API now returns ``Id``\s and ``TyThing``\s rather than ``Name``\s
+issues: #28052
+mrs: !16883
+
+description: {
+The results of interactive evaluation are now returned directly, rather than as
+``Name``\s that had to be looked up again in the interactive context:
+
+- ``ExecComplete``'s ``execResult`` and ``ExecBreak``'s ``breakNames`` now hold ``[Id]``.
+- ``runDecls``, ``runDeclsWithLocation`` and ``runParsedDecls`` now return ``[TyThing]`` rather than ``[Name]``.
+- ``rttiEnvironment`` now returns a Subst with the inferred types.
+
+The returned ``Id``\s carry the types refined by runtime type inference.
+}
=====================================
compiler/GHC/Runtime/Context.hs
=====================================
@@ -6,6 +6,8 @@ module GHC.Runtime.Context
, extendInteractiveContextWithIds
, setInteractivePrintName
, substInteractiveContext
+ , substInteractiveTyThing
+ , substInteractiveId
, replaceImportEnv
, icReaderEnv
, icExtendGblRdrEnv
@@ -496,15 +498,18 @@ icExtendGblRdrEnv drop_only_qualified env tythings
substInteractiveContext :: InteractiveContext -> Subst -> InteractiveContext
substInteractiveContext ictxt@InteractiveContext{ ic_tythings = tts } subst
| isEmptyTCvSubst subst = ictxt
- | otherwise = ictxt { ic_tythings = map subst_ty tts }
- where
- subst_ty (AnId id)
- = AnId $ updateIdTypeAndMult (substTyAddInScope subst) id
- -- Variables in the interactive context *can* mention free type variables
- -- because of the runtime debugger. Otherwise you'd expect all
- -- variables bound in the interactive context to be closed.
- subst_ty tt
- = tt
+ | otherwise = ictxt { ic_tythings = map (substInteractiveTyThing subst) tts }
+
+substInteractiveTyThing :: Subst -> TyThing -> TyThing
+substInteractiveTyThing subst (AnId id) = AnId $ substInteractiveId subst id
+substInteractiveTyThing _ tt = tt
+
+substInteractiveId :: Subst -> Id -> Id
+substInteractiveId subst id =
+ -- Variables in the interactive context *can* mention free type variables
+ -- because of the runtime debugger. Otherwise you'd expect all
+ -- variables bound in the interactive context to be closed.
+ updateIdTypeAndMult (substTyAddInScope subst) id
instance Outputable InteractiveImport where
ppr (IIModule m) = char '*' <> ppr m
=====================================
compiler/GHC/Runtime/Debugger.hs
=====================================
@@ -121,8 +121,7 @@ pprintClosureCommand bindThings force str = do
-- Then, we extract a substitution,
-- mapping the old tyvars to the reconstructed types.
let reconstructed_type = termType term
- hsc_env <- getSession
- case (improveRTTIType hsc_env id_ty' reconstructed_type) of
+ case improveRTTIType id_ty' reconstructed_type of
Nothing -> return (subst, term')
Just subst' -> do { logger <- getLogger
; liftIO $
=====================================
compiler/GHC/Runtime/Eval.hs
=====================================
@@ -131,7 +131,7 @@ import GHC.ByteCode.Breakpoints
import Control.Monad
import Data.Dynamic
import Data.IntMap (IntMap)
-import Data.List (find,intercalate)
+import Data.List (intercalate)
import Data.List.NonEmpty (NonEmpty)
import Unsafe.Coerce ( unsafeCoerce )
import qualified GHC.Unit.Home.Graph as HUG
@@ -241,12 +241,12 @@ execStmt' stmt stmt_text ExecOptions{..} = do
handleRunStatus execSingleStep stmt_text bindings ids
status (emptyHistory size)
-runDecls :: GhcMonad m => String -> m [Name]
+runDecls :: GhcMonad m => String -> m [TyThing]
runDecls = runDeclsWithLocation "<interactive>" 1
-- | Run some declarations and return any user-visible names that were brought
-- into scope.
-runDeclsWithLocation :: GhcMonad m => String -> Int -> String -> m [Name]
+runDeclsWithLocation :: GhcMonad m => String -> Int -> String -> m [TyThing]
runDeclsWithLocation source line_num input = do
hsc_env <- getSession
decls <- liftIO (hscParseDeclsWithLocation hsc_env source line_num input)
@@ -255,18 +255,18 @@ runDeclsWithLocation source line_num input = do
-- | Like `runDeclsWithLocation`, but takes parsed declarations as argument.
-- Useful when doing preprocessing on the AST before execution, e.g. in GHCi
-- (see GHCi.UI.runStmt).
-runParsedDecls :: GhcMonad m => [LHsDecl GhcPs] -> m [Name]
+runParsedDecls :: GhcMonad m => [LHsDecl GhcPs] -> m [TyThing]
runParsedDecls decls = do
- hsc_env <- getSession
- (tyThings, ic) <- liftIO (hscParsedDecls hsc_env decls)
-
+ hsc_env <- getSession
+ (tyThings, ic) <- liftIO $ hscParsedDecls hsc_env decls
setSession $ hsc_env { hsc_IC = ic }
hsc_env <- getSession
- hsc_env' <- liftIO $ rttiEnvironment hsc_env
+ (hsc_env', subst) <- liftIO $ rttiEnvironment hsc_env
setSession hsc_env'
- return $ filter (not . isDerivedOccName . nameOccName)
+ return $ map (substInteractiveTyThing subst)
+ $ filter (not . isDerivedOccName . nameOccName . getName)
-- For this filter, see Note [What to show to users]
- $ map getName tyThings
+ $ tyThings
{- Note [What to show to users]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -310,9 +310,10 @@ handleRunStatus step expr bindings final_ids status history0 = do
final_ic = extendInteractiveContextWithIds (hsc_IC hsc_env) final_ids
final_names = map getName final_ids
liftIO $ Loader.extendLoadedEnv interp modifyHomePackageBytecodeState (zip final_names hvals)
- hsc_env' <- liftIO $ rttiEnvironment hsc_env{hsc_IC=final_ic}
+ (hsc_env', subst) <- liftIO $ rttiEnvironment hsc_env{hsc_IC=final_ic}
+ let improved_final_ids = map (substInteractiveId subst) final_ids
setSession hsc_env'
- return (ExecComplete (Right final_names) allocs)
+ return (ExecComplete (Right improved_final_ids) allocs)
-- Completed with an exception
EvalComplete alloc (EvalException e) ->
@@ -323,7 +324,7 @@ handleRunStatus step expr bindings final_ids status history0 = do
resume_ctxt_fhv <- liftIO $ mkFinalizedHValue interp resume_ctxt
apStack_fhv <- liftIO $ mkFinalizedHValue interp apStack_ref
let span = mkGeneralSrcSpan (fsLit "<unknown>")
- (hsc_env1, names) <- liftIO $
+ (hsc_env1, break_ids) <- liftIO $
bindLocalsAtBreakpoint hsc_env apStack_fhv span Nothing
let
resume = Resume
@@ -342,7 +343,7 @@ handleRunStatus step expr bindings final_ids status history0 = do
hsc_env2 = pushResume hsc_env1 resume
setSession hsc_env2
- return (ExecBreak names Nothing)
+ return (ExecBreak break_ids Nothing)
-- EvalBreak (Just ...) case: the interpreter stopped at a breakpoint
--
@@ -370,7 +371,7 @@ handleRunStatus step expr bindings final_ids status history0 = do
-- This function only returns control to ghci with 'ExecBreak' when it is really meant to break.
-- Specifically, for :steplocal or :stepmodule, don't return control
-- and simply resume execution from here until we hit a breakpoint we do want to stop at.
- (hsc_env1, names) <- liftIO $
+ (hsc_env1, break_ids) <- liftIO $
bindLocalsAtBreakpoint hsc_env apStack_fhv span (Just ibi)
let
resume = Resume
@@ -388,7 +389,7 @@ handleRunStatus step expr bindings final_ids status history0 = do
}
hsc_env2 = pushResume hsc_env1 resume
setSession hsc_env2
- return (ExecBreak names (Just ibi))
+ return (ExecBreak break_ids (Just ibi))
else do
-- resume with the same step type
let eval_opts = initEvalOpts dflags (enableGhcStepMode step)
@@ -491,13 +492,13 @@ getBreakArray interp InternalBreakpointId{ibi_info_mod} imbs = do
( ld_st'
, ba
)
-back :: GhcMonad m => Int -> m ([Name], Int, SrcSpan)
+back :: GhcMonad m => Int -> m ([Id], Int, SrcSpan)
back n = moveHist (+n)
-forward :: GhcMonad m => Int -> m ([Name], Int, SrcSpan)
+forward :: GhcMonad m => Int -> m ([Id], Int, SrcSpan)
forward n = moveHist (subtract n)
-moveHist :: GhcMonad m => (Int -> Int) -> m ([Name], Int, SrcSpan)
+moveHist :: GhcMonad m => (Int -> Int) -> m ([Id], Int, SrcSpan)
moveHist fn = do
hsc_env <- getSession
case ic_resume (hsc_IC hsc_env) of
@@ -521,7 +522,7 @@ moveHist fn = do
let hug = hsc_HUG hsc_env
brks <- readIModBreaks hug ibi
getBreakLoc (readIModModBreaks hug) ibi brks
- (hsc_env1, names) <-
+ (hsc_env1, final_ids) <-
liftIO $ bindLocalsAtBreakpoint hsc_env apStack span mb_info
let ic = hsc_IC hsc_env1
r' = r { resumeHistoryIx = new_ix }
@@ -529,7 +530,7 @@ moveHist fn = do
setSession hsc_env1{ hsc_IC = ic' }
- return (names, new_ix, span)
+ return (final_ids, new_ix, span)
-- careful: we want apStack to be the AP_STACK itself, not a thunk
-- around it, hence the cases are carefully constructed below to
@@ -555,7 +556,7 @@ bindLocalsAtBreakpoint
-> ForeignHValue
-> SrcSpan
-> Maybe InternalBreakpointId
- -> IO (HscEnv, [Name])
+ -> IO (HscEnv, [Id])
-- Nothing case: we stopped when an exception was raised, not at a
-- breakpoint. We have no location information or local variables to
@@ -575,7 +576,7 @@ bindLocalsAtBreakpoint hsc_env apStack span Nothing = do
interp = hscInterp hsc_env
--
Loader.extendLoadedEnv interp modifyHomePackageBytecodeState [(exn_name, apStack)]
- return (hsc_env{ hsc_IC = ictxt1 }, [exn_name])
+ return (hsc_env{ hsc_IC = ictxt1 }, [exn_id])
-- Just case: we stopped at a breakpoint, we have information about the location
-- of the breakpoint and the free variables of the expression.
@@ -635,8 +636,9 @@ bindLocalsAtBreakpoint hsc_env apStack_fhv span (Just ibi) = do
let fhvs = catMaybes mb_hValues
Loader.extendLoadedEnv interp modifyHomePackageBytecodeState (zip names fhvs)
when result_ok $ Loader.extendLoadedEnv interp modifyHomePackageBytecodeState [(result_name, apStack_fhv)]
- hsc_env1 <- rttiEnvironment hsc_env{ hsc_IC = ictxt1 }
- return (hsc_env1, if result_ok then result_name:names else names)
+ (hsc_env1, subst) <- rttiEnvironment hsc_env{ hsc_IC = ictxt1 }
+ let improved_final_ids = map (substInteractiveId subst) final_ids
+ return (hsc_env1, improved_final_ids)
where
-- We need a fresh Unique for each Id we bind, because the linker
-- state is single-threaded and otherwise we'd spam old bindings
@@ -676,34 +678,33 @@ bindLocalsAtBreakpoint hsc_env apStack_fhv span (Just ibi) = do
joinOccs = zipWithEqual joinOcc
joinOcc mbV oc = (\(a,b) c -> (a,b,c)) <$> mbV <*> pure oc
-rttiEnvironment :: HscEnv -> IO HscEnv
+rttiEnvironment :: HscEnv -> IO (HscEnv, Subst)
rttiEnvironment hsc_env@HscEnv{hsc_IC=ic} = do
let tmp_ids = [id | AnId id <- ic_tythings ic]
incompletelyTypedIds =
[id | id <- tmp_ids
, not $ noSkolems id
, (occNameFS.nameOccName.idName) id /= result_fs]
- foldM improveTypes hsc_env (map idName incompletelyTypedIds)
+ foldM improveTypes (hsc_env, emptySubst) incompletelyTypedIds
where
noSkolems = noFreeVarsOfType . idType
- improveTypes hsc_env@HscEnv{hsc_IC=ic} name = do
- let tmp_ids = [id | AnId id <- ic_tythings ic]
- id = expectJust $ find (\i -> idName i == name) tmp_ids
+ improveTypes (hsc_env@HscEnv{hsc_IC=ic}, last_subst) id0 = do
+ let id = substInteractiveId last_subst id0
if noSkolems id
- then return hsc_env
+ then return (hsc_env, last_subst)
else do
mb_new_ty <- reconstructType hsc_env 10 id
let old_ty = idType id
case mb_new_ty of
- Nothing -> return hsc_env
+ Nothing -> return (hsc_env, last_subst)
Just new_ty -> do
- case improveRTTIType hsc_env old_ty new_ty of
+ case improveRTTIType old_ty new_ty of
Nothing -> warnPprTrace True (":print failed to calculate the "
++ "improvement for a type")
(vcat [ text "id" <+> ppr id
, text "old_ty" <+> debugPprType old_ty
, text "new_ty" <+> debugPprType new_ty ]) $
- return hsc_env
+ return (hsc_env, last_subst)
Just subst -> do
let logger = hsc_logger hsc_env
putDumpFileMaybe logger Opt_D_dump_rtti "RTTI"
@@ -712,7 +713,7 @@ rttiEnvironment hsc_env@HscEnv{hsc_IC=ic} = do
ppr subst])
let ic' = substInteractiveContext ic subst
- return hsc_env{hsc_IC=ic'}
+ return (hsc_env{hsc_IC=ic'}, subst `composeTCvSubst` last_subst)
pushResume :: HscEnv -> Resume -> HscEnv
pushResume hsc_env resume = hsc_env { hsc_IC = ictxt1 }
=====================================
compiler/GHC/Runtime/Eval/Types.hs
=====================================
@@ -19,7 +19,6 @@ import GHCi.RemoteTypes
import GHCi.Message (EvalExpr, ResumeContext)
import GHC.ByteCode.Types (InternalBreakpointId(..))
import GHC.Types.Id
-import GHC.Types.Name
import GHC.Types.TyThing
import GHC.Types.Name.Reader
import GHC.Types.SrcLoc
@@ -150,7 +149,7 @@ data ExecResult
-- | Execution is complete with either an exception or the list of
-- user-visible names that were brought into scope.
= ExecComplete
- { execResult :: Either SomeException [Name]
+ { execResult :: Either SomeException [Id]
, execAllocation :: Word64
}
@@ -161,7 +160,7 @@ data ExecResult
-- subsequently deciding whether to really stop here.
-- `ExecBreak` always means GHCi breaks.
| ExecBreak
- { breakNames :: [Name]
+ { breakNames :: [Id]
, breakPointId :: Maybe InternalBreakpointId
}
=====================================
compiler/GHC/Runtime/Heap/Inspect.hs
=====================================
@@ -1163,8 +1163,8 @@ findPtrTyss i tys = foldM step (i, []) tys
-- improveType <base_type> <rtti_type>
-- The types can contain skolem type variables, which need to be treated as normal vars.
-- In particular, we want them to unify with things.
-improveRTTIType :: HscEnv -> RttiType -> RttiType -> Maybe Subst
-improveRTTIType _ base_ty new_ty = U.tcUnifyDebugger base_ty new_ty
+improveRTTIType :: RttiType -> RttiType -> Maybe Subst
+improveRTTIType base_ty new_ty = U.tcUnifyDebugger base_ty new_ty
getDataConArgTys :: DataCon -> Type -> TR [Type]
-- Given the result type ty of a constructor application (D a b c :: ty)
=====================================
ghc/GHCi/UI.hs
=====================================
@@ -1495,7 +1495,7 @@ runOneCommand eh gCmd = do
-- runStmt wrapper for temporarily overridden line-number
runStmtWithLineNum :: Int -> String -> SingleStep
- -> GHCi (Maybe GHC.ExecResult)
+ -> GHCi (Maybe RunResult)
runStmtWithLineNum lnum stmt step = do
st0 <- getGHCiState
setGHCiState st0 { line_number = lnum }
@@ -1560,7 +1560,7 @@ enqueueCommands cmds = do
-- | Entry point to execute some haskell code from user.
-- The return value True indicates success, as in `runOneCommand`.
-runStmt :: GhciMonad m => String -> SingleStep -> m (Maybe GHC.ExecResult)
+runStmt :: GhciMonad m => String -> SingleStep -> m (Maybe RunResult)
runStmt input step = do
dflags <- GHC.getInteractiveDynFlags
let pflags = initParserOpts dflags
@@ -1582,7 +1582,7 @@ runStmt input step = do
case mb_stmt of
Nothing ->
-- empty statement / comment
- return (Just exec_complete)
+ return (Just (RunDeclsResult []))
Just stmt ->
run_stmt stmt
@@ -1605,8 +1605,6 @@ runStmt input step = do
run_imports imports
run_decls decls
where
- exec_complete = GHC.ExecComplete (Right []) 0
-
run_imports imports = mapM_ (addImportToContext . unLoc) imports
set_pragmas pflags sec supported =
@@ -1615,12 +1613,12 @@ runStmt input step = do
opts = unLoc <$> loc_opts
in setOptions opts
- run_stmt :: GhciMonad m => GhciLStmt GhcPs -> m (Maybe GHC.ExecResult)
+ run_stmt :: GhciMonad m => GhciLStmt GhcPs -> m (Maybe RunResult)
run_stmt stmt = do
m_result <- GhciMonad.runStmt stmt input step
case m_result of
Nothing -> return Nothing
- Just result -> Just <$> afterRunStmt step result
+ Just result -> Just <$> afterRunStmt step (RunStmtResult result)
-- `x = y` (a declaration) should be treated as `let x = y` (a statement).
-- The reason is because GHCi wasn't designed to support `x = y`, but then
@@ -1636,7 +1634,7 @@ runStmt input step = do
--
-- Instead of dealing with all these problems individually here we fix this
-- mess by just treating `x = y` as `let x = y`.
- run_decls :: GhciMonad m => [LHsDecl GhcPs] -> m (Maybe GHC.ExecResult)
+ run_decls :: GhciMonad m => [LHsDecl GhcPs] -> m (Maybe RunResult)
-- Only turn `FunBind` and `VarBind` into statements, other bindings
-- (e.g. `PatBind`) need to stay as decls.
run_decls [L l (ValD _ bind@FunBind{})] = run_stmt (mk_stmt (locA l) bind)
@@ -1651,8 +1649,8 @@ runStmt input step = do
-- a file, otherwise the read buffer can't be flushed).
_ <- liftIO $ tryIO $ hFlushAll stdin
m_result <- GhciMonad.runDecls' decls
- forM m_result $ \result ->
- afterRunStmt step (GHC.ExecComplete (Right result) 0)
+ forM m_result $ \result -> do
+ afterRunStmt step (RunDeclsResult result)
mk_stmt :: SrcSpan -> HsBind GhcPs -> GhciLStmt GhcPs
mk_stmt loc bind =
@@ -1668,32 +1666,39 @@ runStmt input step = do
where
modStr = moduleNameString $ moduleName $ icInteractiveModule $ ic
+data RunResult
+ = RunStmtResult GHC.ExecResult
+ | RunDeclsResult [TyThing]
+
-- | Clean up the GHCi environment after a statement has run
afterRunStmt :: GhciMonad m => SingleStep {-^ Type of step we took just before -}
- -> GHC.ExecResult -> m GHC.ExecResult
+ -> RunResult -> m RunResult
afterRunStmt step run_result = do
resumes <- GHC.getResumeContext
case run_result of
- GHC.ExecComplete{..} ->
- case execResult of
- Left ex -> liftIO $ Exception.throwIO ex
- Right names -> do
- show_types <- isOptionSet ShowType
- when show_types $ printTypeOfNames names
- GHC.ExecBreak names mb_info
- | first_resume : _ <- resumes
- -> do mb_id_loc <- toBreakIdAndLocation mb_info
- let bCmd = maybe "" ( \(_,l) -> onBreakCmd l ) mb_id_loc
- if (null bCmd)
- then printStoppedAtBreakInfo first_resume names
- else enqueueCommands [bCmd]
- -- run the command set with ":set stop <cmd>"
- st <- getGHCiState
- enqueueCommands [stop st]
- return ()
-
- | otherwise -> resume step Nothing >>=
- afterRunStmt step >> return ()
+ RunDeclsResult things -> do
+ show_types <- isOptionSet ShowType
+ when show_types $ printTypeOfThings things
+ RunStmtResult GHC.ExecComplete{..} ->
+ case execResult of
+ Left ex -> liftIO $ Exception.throwIO ex
+ Right ids -> do
+ show_types <- isOptionSet ShowType
+ when show_types $ printTypeOfIds ids
+ RunStmtResult (GHC.ExecBreak ids mb_info)
+ | first_resume : _ <- resumes
+ -> do mb_id_loc <- toBreakIdAndLocation mb_info
+ let bCmd = maybe "" ( \(_,l) -> onBreakCmd l ) mb_id_loc
+ if (null bCmd)
+ then printStoppedAtBreakInfo first_resume ids
+ else enqueueCommands [bCmd]
+ -- run the command set with ":set stop <cmd>"
+ st <- getGHCiState
+ enqueueCommands [stop st]
+ return ()
+
+ | otherwise -> resume step Nothing >>=
+ afterRunStmt step . RunStmtResult >> return ()
flushInterpBuffers
withSignalHandlers $ do
@@ -1702,16 +1707,19 @@ afterRunStmt step run_result = do
return run_result
-runSuccess :: Maybe GHC.ExecResult -> Bool
+runSuccess :: Maybe RunResult -> Bool
runSuccess run_result
- | Just (GHC.ExecComplete { execResult = Right _ }) <- run_result = True
+ | Just (RunDeclsResult _) <- run_result = True
+ | Just (RunStmtResult GHC.ExecComplete { execResult = Right _ })
+ <- run_result = True
| otherwise = False
-runAllocs :: Maybe GHC.ExecResult -> Maybe Integer
+runAllocs :: Maybe RunResult -> Maybe Integer
runAllocs m = do
res <- m
case res of
- GHC.ExecComplete{..} -> Just (fromIntegral execAllocation)
+ RunDeclsResult _ -> Just 0
+ RunStmtResult GHC.ExecComplete{..} -> Just (fromIntegral execAllocation)
_ -> Nothing
toBreakIdAndLocation :: GhciMonad m
@@ -1725,30 +1733,23 @@ toBreakIdAndLocation (Just inf) = do
return $ listToMaybe [ id_loc | id_loc@(_,loc) <- IntMap.assocs (breaks st),
breakId loc == bi ]
-printStoppedAtBreakInfo :: GHC.GhcMonad m => Resume -> [Name] -> m ()
-printStoppedAtBreakInfo res names = do
+printStoppedAtBreakInfo :: GHC.GhcMonad m => Resume -> [GHC.Id] -> m ()
+printStoppedAtBreakInfo res ids = do
printForUser =<< pprStopped res
- -- printTypeOfNames session names
- let namesSorted = sortBy compareNames names
- tythings <- catMaybes `liftM` mapM GHC.lookupName namesSorted
- docs <- mapM pprTypeAndContents [i | AnId i <- tythings]
+ let idsSorted = sortBy (compareNames `on` getName) ids
+ docs <- mapM pprTypeAndContents idsSorted
printForUserPartWay $ vcat docs
-printTypeOfNames :: GHC.GhcMonad m => [Name] -> m ()
-printTypeOfNames names
- = mapM_ (printTypeOfName ) $ sortBy compareNames names
+printTypeOfIds :: GHC.GhcMonad m => [GHC.Id] -> m ()
+printTypeOfIds = printTypeOfThings . map AnId
+
+printTypeOfThings :: GHC.GhcMonad m => [GHC.TyThing] -> m ()
+printTypeOfThings things
+ = mapM_ printTyThing $ sortBy (compareNames `on` getName) things
compareNames :: Name -> Name -> Ordering
compareNames = on compare getOccString S.<> on SrcLoc.leftmost_smallest getSrcSpan
-printTypeOfName :: GHC.GhcMonad m => Name -> m ()
-printTypeOfName n
- = do maybe_tything <- GHC.lookupName n
- case maybe_tything of
- Nothing -> return ()
- Just thing -> printTyThing thing
-
-
data MaybeCommand = GotCommand Command | BadCommand | NoLastCommand
-- | Entry point for execution a ':<command>' input from user
@@ -4400,7 +4401,7 @@ doContinue step = doContinue' step Nothing
doContinue' :: GhciMonad m => SingleStep -> Maybe Int -> m ()
doContinue' step mbCnt= do
runResult <- resume step mbCnt
- _ <- afterRunStmt step runResult
+ _ <- afterRunStmt step (RunStmtResult runResult)
return ()
abandonCmd :: GhciMonad m => String -> m ()
@@ -4551,9 +4552,9 @@ backCmd arg
| otherwise = liftIO $ putStrLn "Syntax: :back [num]"
where
back num = withSandboxOnly ":back" $ do
- (names, _, pan) <- GHC.back num
+ (ids, _, pan) <- GHC.back num
printForUser $ text "Logged breakpoint at" <+> ppr pan
- printTypeOfNames names
+ printTypeOfIds ids
-- run the command set with ":set stop <cmd>"
st <- getGHCiState
enqueueCommands [stop st]
@@ -4565,11 +4566,11 @@ forwardCmd arg
| otherwise = liftIO $ putStrLn "Syntax: :forward [num]"
where
forward num = withSandboxOnly ":forward" $ do
- (names, ix, pan) <- GHC.forward num
+ (ids, ix, pan) <- GHC.forward num
printForUser $ (if (ix == 0)
then text "Stopped at"
else text "Logged breakpoint at") <+> ppr pan
- printTypeOfNames names
+ printTypeOfIds ids
-- run the command set with ":set stop <cmd>"
st <- getGHCiState
enqueueCommands [stop st]
=====================================
ghc/GHCi/UI/Monad.hs
=====================================
@@ -381,7 +381,7 @@ runStmt stmt stmt_text step = do
(EvalThis fhv) }
Just <$> GHC.execStmt' stmt stmt_text opts
-runDecls :: GhciMonad m => String -> m (Maybe [GHC.Name])
+runDecls :: GhciMonad m => String -> m (Maybe [GHC.TyThing])
runDecls decls = do
st <- getGHCiState
reifyGHCi $ \x ->
@@ -393,7 +393,7 @@ runDecls decls = do
r <- GHC.runDeclsWithLocation (progname st) (line_number st) decls
return (Just r)
-runDecls' :: GhciMonad m => [LHsDecl GhcPs] -> m (Maybe [GHC.Name])
+runDecls' :: GhciMonad m => [LHsDecl GhcPs] -> m (Maybe [GHC.TyThing])
runDecls' decls = do
st <- getGHCiState
reifyGHCi $ \x ->
=====================================
testsuite/tests/ghci.debugger/scripts/RttiSharedTyVar.hs
=====================================
@@ -0,0 +1,4 @@
+module RttiSharedTyVar where
+
+f :: a -> a -> ()
+f x y = x `seq` y `seq` ()
=====================================
testsuite/tests/ghci.debugger/scripts/RttiSharedTyVar.script
=====================================
@@ -0,0 +1,9 @@
+:l RttiSharedTyVar
+let a = Just undefined :: Maybe Int
+let b = Just 5 :: Maybe Int
+a `seq` b `seq` ()
+:break f
+-- should say x y have type Maybe Int, not Maybe a
+f a b
+:t x
+:t y
=====================================
testsuite/tests/ghci.debugger/scripts/RttiSharedTyVar.stdout
=====================================
@@ -0,0 +1,8 @@
+()
+Breakpoint 0 activated at RttiSharedTyVar.hs:4:9-26
+Stopped in RttiSharedTyVar.f, RttiSharedTyVar.hs:4:9-26
+_result :: () = _
+x :: Maybe Int = Just _
+y :: Maybe Int = Just 5
+x :: Maybe Int
+y :: Maybe Int
=====================================
testsuite/tests/ghci.debugger/scripts/all.T
=====================================
@@ -155,3 +155,5 @@ test('T26042f2', [extra_hc_opts('-O -fno-unoptimized-core-for-interpreter'), ext
test('T26042g', extra_files(['T26042g.hs']), ghci_script, ['T26042g.script'])
test('T27633c', extra_files(['T27633c_obj.hs']), ghci_script, ['T27633c.script'])
+
+test('RttiSharedTyVar', extra_files(['RttiSharedTyVar.hs']), ghci_script, ['RttiSharedTyVar.script'])
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/26fbcce5413b6b9a39c2b9714c3c80c…
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/26fbcce5413b6b9a39c2b9714c3c80c…
You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help
1
0
[Git][ghc/ghc] Pushed new branch wip/romes/interactive-ids-refactor
by Rodrigo Mesquita (@alt-romes) 28 Oct '26
by Rodrigo Mesquita (@alt-romes) 28 Oct '26
28 Oct '26
Rodrigo Mesquita pushed new branch wip/romes/interactive-ids-refactor at Glasgow Haskell Compiler / GHC
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/tree/wip/romes/interactive-ids-refactor
You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help
1
0
[Git][ghc/ghc][wip/andreask/9.10.4-batch1] 9 commits: hadrian: fix GHC.Platform.Host generation for cross stage1
by Andreas Klebinger (@AndreasK) 28 Oct '26
by Andreas Klebinger (@AndreasK) 28 Oct '26
28 Oct '26
Andreas Klebinger pushed to branch wip/andreask/9.10.4-batch1 at Glasgow Haskell Compiler / GHC
Commits:
b4dc7b1c by Cheng Shao at 2026-10-28T11:42:58+01:00
hadrian: fix GHC.Platform.Host generation for cross stage1
This patch fixes incorrectly GHC.Platform.Host generation logic for
cross stage1 in hadrian (#26449). Also adds T26449 test case to
witness the fix.
Co-authored-by: Codex <codex(a)openai.com>
(cherry picked from commit 8cbe006ad09d5a64e4a3cdf4c91a8b81ff1511be)
- - - - -
0eb7b151 by Ben Gamari at 2026-10-28T11:43:25+01:00
gitlab-ci: Make RELEASE_JOB an input
Rather than an undocumented variable.
(cherry picked from commit f9790ca81deb8b14ff2eabf701aecbcfd6501963)
- - - - -
a026b215 by Ben Gamari at 2026-10-28T11:43:38+01:00
rts/nonmoving: Fix comment spelling
(cherry picked from commit 14281a22eb27498886def8e5d17797c9ce62f3ad)
- - - - -
b30ecb58 by Ben Gamari at 2026-10-28T11:43:38+01:00
rts/nonmoving: Use atomic operations to update bd->flags
(cherry picked from commit bedd38b01d6b113cb3bd10b5d784c16b32efb5bb)
- - - - -
ce2add9b by Ben Gamari at 2026-10-28T11:43:38+01:00
nonmoving: Use get_itbl instead of explicit loads
This is cleaner and also fixes unnecessary (and unsound) use of
`volatile`.
(cherry picked from commit 215d68414020dc4ed0636508c9eecd9f44f62168)
- - - - -
374ca5c4 by Ben Gamari at 2026-10-28T11:43:38+01:00
rts/Scav: Handle WHITEHOLEs in scavenge_one
`scavenge_one`, used to scavenge mutable list entries, may encounter
`WHITEHOLE`s when the non-moving GC is in use via two paths:
1. when an MVAR is being marked concurrently
2. when the object belongs to a chain of selectors being short-cutted.
Fixes #26204.
(cherry picked from commit 2c94aa3aa87c14b1ff5c4355c9a90efedd5d10f4)
- - - - -
97a0f7a4 by Cheng Shao at 2026-10-28T11:45:58+01:00
testsuite: fix T3586 for non-SSE3 platforms
`T3586.hs` contains `-fvia-C -optc-msse3` which I think is a
best-effort basis to harvest the C compiler's auto vectorization
optimizations via the C backend back when the test was added. The
`-fvia-C` part is now a deprecated no-op because GHC can't fall back
to the C backend on a non-unregisterised build, and `-optc-msse3`
might actually cause the test to fail on non x86/x64 platforms, e.g.
recent builds of wasi-sdk would report `wasm32-wasi-clang: error:
unsupported option '-msse3' for target 'wasm32-unknown-wasi'`.
So this patch cleans up this historical cruft. `-fvia-C` is removed,
and `-optc-msse3` is only passed when cpuid contains `pni` (which
indicates support of SSE3).
(cherry picked from commit 70ee825a516bcf7aac762bfedb4a017d35f8dcf3)
- - - - -
ba4aac3d by Andreas Klebinger at 2026-10-28T11:53:47+01:00
Add a perf test for #26425
(cherry picked from commit 4038a28b8594ae91c000d5bc7fceed4815eb4826)
- - - - -
5a256308 by Andreas Klebinger at 2026-10-28T14:08:55+01:00
OccAnal: Be stricter for better compiler perf.
In particular we are now stricter:
* When combining usageDetails.
* When computing binder info.
In combineUsageDetails when combining the underlying adds we compute a
new `LocalOcc` for each entry by combining the two existing ones.
Rather than wait for those entries to be forced down the road we now
force them immediately. Speeding up T26425 by about 10% with little
effect on the common case.
We also force binders we put into the Core AST everywhere now.
Failure to do so risks leaking the occ env used to set the binders
OccInfo.
For T26425 compiler residency went down by a factor of ~10x.
Compile time also improved by a factor of ~1.6.
-------------------------
Metric Decrease:
T18698a
T26425
T9233
-------------------------
(cherry picked from commit f997618e08ce5aad8b7564b8073889b52bc18700)
- - - - -
16 changed files:
- .gitlab-ci.yml
- compiler/GHC/Core/Opt/OccurAnal.hs
- compiler/GHC/Types/Unique/FM.hs
- compiler/GHC/Types/Var/Env.hs
- hadrian/src/Rules/Generate.hs
- rts/include/rts/storage/Block.h
- rts/sm/NonMoving.c
- rts/sm/NonMovingMark.c
- rts/sm/Scav.c
- testsuite/driver/cpu_features.py
- + testsuite/tests/cross/should_run/T26449.hs
- + testsuite/tests/cross/should_run/all.T
- + testsuite/tests/perf/compiler/T26425.hs
- testsuite/tests/perf/compiler/all.T
- testsuite/tests/perf/should_run/T3586.hs
- testsuite/tests/perf/should_run/all.T
Changes:
=====================================
.gitlab-ci.yml
=====================================
@@ -1,3 +1,12 @@
+spec:
+ inputs:
+ release:
+ default: "no"
+ description: Release pipeline
+ options: [ "yes", "no" ]
+
+---
+
variables:
GIT_SSL_NO_VERIFY: "1"
@@ -25,6 +34,8 @@ variables:
# Makes ci.sh isolate CABAL_DIR
HERMETIC: "YES"
+ RELEASE_JOB: $[[ inputs.release ]]
+
# Reduce XZ compression level for regular jobs (it is bumped to 9 for releases
# and nightly jobs). In my experiments I've got the following bindist size in
# the given time for each compression level (with the quick flavour):
=====================================
compiler/GHC/Core/Opt/OccurAnal.hs
=====================================
@@ -9,6 +9,8 @@
-- many /other/ arguments the function has. Inconsistent unboxing is very
-- bad for performance, so I increased the limit to allow it to unbox
-- consistently.
+-- AK: Seems we no longer unbox OccEnv now anyway so it might be redundant.
+
{-
(c) The GRASP/AQUA Project, Glasgow University, 1992-1998
@@ -967,6 +969,12 @@ occAnalBind
-> ([CoreBind] -> r -> r) -- How to combine the scope with new binds
-> WithUsageDetails r -- Of the whole let(rec)
+-- AK: While not allocating any less inlining occAnalBind turns calls to the
+-- passed functions into known calls with all the benefits that brings.
+-- On a version of T26425 with 6k alternatives this improved compile
+-- by 10-20% with -O.
+{-# INLINE occAnalBind #-}
+
occAnalBind env lvl ire (Rec pairs) thing_inside combine
= addInScopeList env (map fst pairs) $ \env ->
let WUD body_uds body' = thing_inside env
@@ -984,7 +992,7 @@ occAnalBind !env lvl ire (NonRec bndr rhs) thing_inside combine
= -- Analyse the RHS and /then/ the body
let -- Analyse the rhs first, generating rhs_uds
!(rhs_uds_s, bndr', rhs') = occAnalNonRecRhs env ire mb_join bndr rhs
- rhs_uds = foldr1 orUDs rhs_uds_s -- NB: orUDs. See (W4) of
+ rhs_uds = foldl1' orUDs rhs_uds_s -- NB: orUDs. See (W4) of
-- Note [Occurrence analysis for join points]
-- Now analyse the body, adding the join point
@@ -1049,6 +1057,7 @@ occAnalNonRecRhs !env imp_rule_edges mb_join bndr rhs
-- Match join arity O from mb_join_arity with manifest join arity M as
-- returned by of occAnalLamTail. It's totally OK for them to mismatch;
-- hence adjust the UDs from the RHS
+
WUD adj_rhs_uds final_rhs = adjustNonRecRhs mb_join $
occAnalLamTail rhs_env rhs
final_bndr_with_rules
@@ -2053,6 +2062,18 @@ So The Plan is this:
was a loop breaker last time round
Hence the is_lb field of NodeScore
+
+Note [Strictness in the occurrence analyser]
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+By carefully making the occurrence analyser strict in some places, we can
+dramatically reduce its memory residency. Among other things we:
+* Evaluate the result of `tagLamBinder` and friends, so that the binder (or its
+ OccInfo) does not retain the entire `UsageDetails`. Also use `strictMap` in `tagLamBinders`.
+* In `combineUsageDetailsWith`, the fields of the data constructor are strict, and we use
+ `strictPlusVarEnv` on the maps that are bound to be needed later on to avoid thunks being
+ stored in the values.
+
+These measures reduced residency for test T26425 by a factor of at least 5x.
-}
{- *********************************************************************
@@ -2187,7 +2208,9 @@ occ_anal_lam_tail env expr@(Lam {})
go env rev_bndrs body
= addInScope env rev_bndrs $ \env ->
let !(WUD usage body') = occ_anal_lam_tail env body
- wrap_lam body bndr = Lam (tagLamBinder usage bndr) body
+ -- See Note [Strictness in the occurrence analyser]
+ wrap_lam !body !bndr = let !bndr' = tagLamBinder usage bndr
+ in Lam bndr' body
in WUD (usage `addLamCoVarOccs` rev_bndrs)
(foldl' wrap_lam body' rev_bndrs)
@@ -2540,7 +2563,8 @@ occAnal env (Case scrut bndr ty alts)
let alt_env = addBndrSwap scrut' bndr $
setTailCtxt env -- Kill off OccRhs
WUD alts_usage alts' = do_alts alt_env alts
- tagged_bndr = tagLamBinder alts_usage bndr
+ !tagged_bndr = tagLamBinder alts_usage bndr
+ -- See Note [Strictness in the occurrence analyser]
in WUD alts_usage (tagged_bndr, alts')
total_usage = markAllNonTail scrut_usage `andUDs` alts_usage
@@ -2558,11 +2582,13 @@ occAnal env (Case scrut bndr ty alts)
do_alt !env (Alt con bndrs rhs)
= addInScopeList env bndrs $ \ env ->
let WUD rhs_usage rhs' = occAnal env rhs
- tagged_bndrs = tagLamBinders rhs_usage bndrs
+ !tagged_bndrs = tagLamBinders rhs_usage bndrs
+ -- See Note [Strictness in the occurrence analyser]
in -- See Note [Binders in case alternatives]
WUD rhs_usage (Alt con tagged_bndrs rhs')
occAnal env (Let bind body)
+ -- TODO: Would be nice to use a strict version of mkLets here
= occAnalBind env NotTopLevel noImpRuleEdges bind
(\env -> occAnal env body) mkLets
@@ -2622,10 +2648,12 @@ occAnalApp !env (Var fun, args, ticks)
| fun `hasKey` runRWKey
, [t1, t2, arg] <- args
, WUD usage arg' <- adjustNonRecRhs (JoinPoint 1) $ occAnalLamTail env arg
- = WUD usage (mkTicks ticks $ mkApps (Var fun) [t1, t2, arg'])
+ = let app_out = mkTicks ticks $ mkApps (Var fun) [t1, t2, arg']
+ in WUD usage app_out
occAnalApp env (Var fun_id, args, ticks)
- = WUD all_uds (mkTicks ticks app')
+ = let app_out = mkTicks ticks app'
+ in WUD all_uds app_out
where
-- Lots of banged bindings: this is a very heavily bit of code,
-- so it pays not to make lots of thunks here, all of which
@@ -2670,8 +2698,9 @@ occAnalApp env (Var fun_id, args, ticks)
-- See Note [Sources of one-shot information], bullet point A']
occAnalApp env (fun, args, ticks)
- = WUD (markAllNonTail (fun_uds `andUDs` args_uds))
- (mkTicks ticks app')
+ = let app_out = mkTicks ticks app'
+ in WUD (markAllNonTail (fun_uds `andUDs` args_uds)) app_out
+
where
!(WUD args_uds app') = occAnalArgs env fun' args []
!(WUD fun_uds fun') = occAnal (addAppCtxt env args) fun
@@ -3613,8 +3642,8 @@ data WithTailUsageDetails a = WTUD !TailUsageDetails !a
-------------------
-- UsageDetails API
-andUDs, orUDs
- :: UsageDetails -> UsageDetails -> UsageDetails
+andUDs:: UsageDetails -> UsageDetails -> UsageDetails
+orUDs :: UsageDetails -> UsageDetails -> UsageDetails
andUDs = combineUsageDetailsWith andLocalOcc
orUDs = combineUsageDetailsWith orLocalOcc
@@ -3729,10 +3758,13 @@ combineUsageDetailsWith plus_occ_info
| isEmptyVarEnv env1 = uds2
| isEmptyVarEnv env2 = uds1
| otherwise
- = UD { ud_env = plusVarEnv_C plus_occ_info env1 env2
- , ud_z_many = plusVarEnv z_many1 z_many2
+ -- See Note [Strictness in the occurrence analyser]
+ -- Using strictPlusVarEnv here speeds up the test T26425 by about 10% by avoiding
+ -- intermediate thunks.
+ = UD { ud_env = strictPlusVarEnv_C plus_occ_info env1 env2
+ , ud_z_many = strictPlusVarEnv z_many1 z_many2
, ud_z_in_lam = plusVarEnv z_in_lam1 z_in_lam2
- , ud_z_tail = plusVarEnv z_tail1 z_tail2 }
+ , ud_z_tail = strictPlusVarEnv z_tail1 z_tail2 }
lookupLetOccInfo :: UsageDetails -> Id -> OccInfo
-- Don't use locally-generated occ_info for exported (visible-elsewhere)
@@ -3810,7 +3842,8 @@ tagLamBinders :: UsageDetails -- Of scope
-> [Id] -- Binders
-> [IdWithOccInfo] -- Tagged binders
tagLamBinders usage binders
- = map (tagLamBinder usage) binders
+ -- See Note [Strictness in the occurrence analyser]
+ = strictMap (tagLamBinder usage) binders
tagLamBinder :: UsageDetails -- Of scope
-> Id -- Binder
@@ -3819,6 +3852,7 @@ tagLamBinder :: UsageDetails -- Of scope
-- No-op on TyVars
-- A lambda binder never has an unfolding, so no need to look for that
tagLamBinder usage bndr
+ -- See Note [Strictness in the occurrence analyser]
= setBinderOcc (markNonTail occ) bndr
-- markNonTail: don't try to make an argument into a join point
where
=====================================
compiler/GHC/Types/Unique/FM.hs
=====================================
@@ -51,7 +51,9 @@ module GHC.Types.Unique.FM (
delListFromUFM,
delListFromUFM_Directly,
plusUFM,
+ strictPlusUFM,
plusUFM_C,
+ strictPlusUFM_C,
plusUFM_CD,
plusUFM_CD2,
mergeUFM,
@@ -249,16 +251,24 @@ delListFromUFM_Directly = foldl' delFromUFM_Directly
delFromUFM_Directly :: UniqFM key elt -> Unique -> UniqFM key elt
delFromUFM_Directly (UFM m) u = UFM (M.delete (getKey u) m)
--- Bindings in right argument shadow those in the left
+-- | Bindings in right argument shadow those in the left.
+--
+-- Unlike containers this union is right-biased for historic reasons.
plusUFM :: UniqFM key elt -> UniqFM key elt -> UniqFM key elt
--- M.union is left-biased, plusUFM should be right-biased.
plusUFM (UFM x) (UFM y) = UFM (M.union y x)
-- Note (M.union y x), with arguments flipped
-- M.union is left-biased, plusUFM should be right-biased.
+-- | Right biased
+strictPlusUFM :: UniqFM key elt -> UniqFM key elt -> UniqFM key elt
+strictPlusUFM (UFM x) (UFM y) = UFM (MS.union y x)
+
plusUFM_C :: (elt -> elt -> elt) -> UniqFM key elt -> UniqFM key elt -> UniqFM key elt
plusUFM_C f (UFM x) (UFM y) = UFM (M.unionWith f x y)
+strictPlusUFM_C :: (elt -> elt -> elt) -> UniqFM key elt -> UniqFM key elt -> UniqFM key elt
+strictPlusUFM_C f (UFM x) (UFM y) = UFM (MS.unionWith f x y)
+
-- | `plusUFM_CD f m1 d1 m2 d2` merges the maps using `f` as the
-- combinding function and `d1` resp. `d2` as the default value if
-- there is no entry in `m1` reps. `m2`. The domain is the union of
=====================================
compiler/GHC/Types/Var/Env.hs
=====================================
@@ -12,7 +12,8 @@ module GHC.Types.Var.Env (
elemVarEnv, disjointVarEnv, anyVarEnv,
extendVarEnv, extendVarEnv_C, extendVarEnv_Acc,
extendVarEnvList,
- plusVarEnv, plusVarEnv_C, plusVarEnv_CD, plusMaybeVarEnv_C,
+ strictPlusVarEnv, plusVarEnv, plusVarEnv_C, strictPlusVarEnv_C,
+ plusVarEnv_CD, plusMaybeVarEnv_C,
plusVarEnvList, alterVarEnv,
delVarEnvList, delVarEnv,
minusVarEnv,
@@ -510,6 +511,7 @@ extendVarEnv :: VarEnv a -> Var -> a -> VarEnv a
extendVarEnv_C :: (a->a->a) -> VarEnv a -> Var -> a -> VarEnv a
extendVarEnv_Acc :: (a->b->b) -> (a->b) -> VarEnv b -> Var -> a -> VarEnv b
plusVarEnv :: VarEnv a -> VarEnv a -> VarEnv a
+strictPlusVarEnv :: VarEnv a -> VarEnv a -> VarEnv a
plusVarEnvList :: [VarEnv a] -> VarEnv a
extendVarEnvList :: VarEnv a -> [(Var, a)] -> VarEnv a
varEnvDomain :: VarEnv elt -> UnVarSet
@@ -521,6 +523,7 @@ delVarEnvList :: VarEnv a -> [Var] -> VarEnv a
delVarEnv :: VarEnv a -> Var -> VarEnv a
minusVarEnv :: VarEnv a -> VarEnv b -> VarEnv a
plusVarEnv_C :: (a -> a -> a) -> VarEnv a -> VarEnv a -> VarEnv a
+strictPlusVarEnv_C :: (a -> a -> a) -> VarEnv a -> VarEnv a -> VarEnv a
plusVarEnv_CD :: (a -> a -> a) -> VarEnv a -> a -> VarEnv a -> a -> VarEnv a
plusMaybeVarEnv_C :: (a -> a -> Maybe a) -> VarEnv a -> VarEnv a -> VarEnv a
mapVarEnv :: (a -> b) -> VarEnv a -> VarEnv b
@@ -547,12 +550,14 @@ extendVarEnv_C = addToUFM_C
extendVarEnv_Acc = addToUFM_Acc
extendVarEnvList = addListToUFM
plusVarEnv_C = plusUFM_C
+strictPlusVarEnv_C = strictPlusUFM_C
plusVarEnv_CD = plusUFM_CD
plusMaybeVarEnv_C = plusMaybeUFM_C
delVarEnvList = delListFromUFM
delVarEnv = delFromUFM
minusVarEnv = minusUFM
plusVarEnv = plusUFM
+strictPlusVarEnv = strictPlusUFM
plusVarEnvList = plusUFMList
-- lookupVarEnv is very hot (in part due to being called by substTyVar),
-- if it's not inlined than the mere allocation of the Just constructor causes
=====================================
hadrian/src/Rules/Generate.hs
=====================================
@@ -570,8 +570,12 @@ generateVersionHs = do
generatePlatformHostHs :: Expr String
generatePlatformHostHs = do
trackGenerateHs
- cHostPlatformArch <- queryHost (archOS_arch . tgtArchOs)
- cHostPlatformOS <- queryHost (archOS_OS . tgtArchOs)
+ stage <- getStage
+ let chooseHostQuery = case stage of
+ Stage0 {} -> queryHost
+ _ -> queryTarget
+ cHostPlatformArch <- chooseHostQuery (archOS_arch . tgtArchOs)
+ cHostPlatformOS <- chooseHostQuery (archOS_OS . tgtArchOs)
return $ unlines
[ "module GHC.Platform.Host where"
, ""
=====================================
rts/include/rts/storage/Block.h
=====================================
@@ -300,6 +300,28 @@ dbl_link_replace(bdescr *new_, bdescr *old, bdescr **list)
}
}
+INLINE_HEADER void
+block_set_flag(bdescr *bd, uint16_t flag)
+{
+ // TODO: This ordering is stronger than necessary for many users (e.g.
+ // setting flags).
+ __atomic_or_fetch(&bd->flags, flag, __ATOMIC_SEQ_CST);
+}
+
+INLINE_HEADER void
+block_clear_flag(bdescr *bd, uint16_t flag)
+{
+ // TODO: This ordering is stronger than necessary for many users (e.g.
+ // setting flags).
+ __atomic_and_fetch(&bd->flags, ~flag, __ATOMIC_SEQ_CST);
+}
+
+INLINE_HEADER uint16_t
+block_get_flags(bdescr *bd)
+{
+ return RELAXED_LOAD(&bd->flags);
+}
+
/* Initialisation ---------------------------------------------------------- */
extern void initBlockAllocator(void);
=====================================
rts/sm/NonMoving.c
=====================================
@@ -550,7 +550,7 @@ static void nonmovingBumpEpoch(void) {
* arbitrary allocator sizes, we need to do some precomputation and make
* use of the integer division by constants optimisation.
*
- * We currenlty try to balance these considerations by adopting the following scheme.
+ * We currently try to balance these considerations by adopting the following scheme.
* We have nonmoving_alloca_dense_cnt "dense" allocators starting with size
* NONMOVING_ALLOCA0, and incrementing by NONMOVING_ALLOCA_DENSE_INCREMENT.
* These service the vast majority of allocations.
=====================================
rts/sm/NonMovingMark.c
=====================================
@@ -606,7 +606,7 @@ static
bool check_in_nonmoving_heap(StgClosure *p) {
if (HEAP_ALLOCED_GC(p)) {
// This works for both large and small objects:
- return Bdescr((P_)p)->flags & BF_NONMOVING;
+ return block_get_flags(Bdescr((P_)p)) & BF_NONMOVING;
} else {
return true; // a static object
}
@@ -619,7 +619,7 @@ inline void updateRemembSetPushThunk(Capability *cap, StgThunk *thunk)
{
const StgInfoTable *info;
do {
- info = *(StgInfoTable* volatile*) &thunk->header.info;
+ info = (StgInfoTable*) RELAXED_LOAD(&thunk->header.info);
} while (info == &stg_WHITEHOLE_info);
const StgThunkInfoTable *thunk_info = THUNK_INFO_PTR_TO_STRUCT(info);
@@ -722,13 +722,14 @@ STATIC_INLINE bool needs_upd_rem_set_mark(StgClosure *p)
{
// TODO: Deduplicate with mark_closure
bdescr *bd = Bdescr((StgPtr) p);
+ uint16_t flags = block_get_flags(bd);
if (bd->gen != oldest_gen) {
return false;
- } else if (bd->flags & BF_LARGE) {
- if (! (bd->flags & BF_NONMOVING_SWEEPING)) {
+ } else if (flags & BF_LARGE) {
+ if (! (flags & BF_NONMOVING_SWEEPING)) {
return false;
} else {
- return ! (bd->flags & BF_MARKED);
+ return ! (flags & BF_MARKED);
}
} else {
struct NonmovingSegment *seg = nonmovingGetSegment((StgPtr) p);
@@ -740,8 +741,8 @@ STATIC_INLINE bool needs_upd_rem_set_mark(StgClosure *p)
static void finish_upd_rem_set_mark_large(bdescr* bd) {
// Someone else may have already marked it.
ACQUIRE_LOCK(&nonmoving_large_objects_mutex);
- if (! (bd->flags & BF_MARKED)) {
- bd->flags |= BF_MARKED;
+ if (! (block_get_flags(bd) & BF_MARKED)) {
+ block_set_flag(bd, BF_MARKED);
dbl_link_remove(bd, &nonmoving_large_objects);
dbl_link_onto(bd, &nonmoving_marked_large_objects);
n_nonmoving_large_blocks -= bd->blocks;
@@ -754,7 +755,7 @@ static void finish_upd_rem_set_mark_large(bdescr* bd) {
STATIC_INLINE void finish_upd_rem_set_mark(StgClosure *p)
{
bdescr *bd = Bdescr((StgPtr) p);
- if (bd->flags & BF_LARGE) {
+ if (block_get_flags(bd) & BF_LARGE) {
// This function is extracted so that this function can be inline
finish_upd_rem_set_mark_large(bd);
} else {
@@ -1342,7 +1343,7 @@ mark_closure (MarkQueue *queue, const StgClosure *p0, StgClosure **origin)
goto done;
case WHITEHOLE:
- while (*(StgInfoTable* volatile*) &p->header.info == &stg_WHITEHOLE_info)
+ while (RELAXED_LOAD(&p->header.info) == &stg_WHITEHOLE_info)
#if defined(PARALLEL_GC)
busy_wait_nop()
#endif
@@ -1376,35 +1377,36 @@ mark_closure (MarkQueue *queue, const StgClosure *p0, StgClosure **origin)
// N.B. only the first block of a compact region is guaranteed to carry
// BF_NONMOVING; consequently we must separately check for BF_COMPACT.
- if (bd->flags & (BF_COMPACT | BF_NONMOVING)) {
+ const uint16_t flags = block_get_flags(bd);
+ if (flags & (BF_COMPACT | BF_NONMOVING)) {
- if (bd->flags & BF_COMPACT) {
+ if (flags & BF_COMPACT) {
StgCompactNFData *str = objectGetCompact((StgClosure*)p);
bd = Bdescr((P_)str);
- if (! (bd->flags & BF_NONMOVING_SWEEPING)) {
+ if (! (flags & BF_NONMOVING_SWEEPING)) {
// Not in the snapshot
return;
}
- if (! (bd->flags & BF_MARKED)) {
+ if (! (flags & BF_MARKED)) {
dbl_link_remove(bd, &nonmoving_compact_objects);
dbl_link_onto(bd, &nonmoving_marked_compact_objects);
StgWord blocks = str->totalW / BLOCK_SIZE_W;
n_nonmoving_compact_blocks -= blocks;
n_nonmoving_marked_compact_blocks += blocks;
- bd->flags |= BF_MARKED;
+ block_set_flag(bd, BF_MARKED);
}
// N.B. the object being marked is in a compact region so by
// definition there is no need to do any tracing here.
goto done;
- } else if (bd->flags & BF_LARGE) {
- if (! (bd->flags & BF_NONMOVING_SWEEPING)) {
+ } else if (flags & BF_LARGE) {
+ if (! (flags & BF_NONMOVING_SWEEPING)) {
// Not in the snapshot
goto done;
}
- if (bd->flags & BF_MARKED) {
+ if (flags & BF_MARKED) {
goto done;
}
} else {
@@ -1712,7 +1714,7 @@ mark_closure (MarkQueue *queue, const StgClosure *p0, StgClosure **origin)
break;
case WHITEHOLE:
- while (*(StgInfoTable* volatile*) &p->header.info == &stg_WHITEHOLE_info);
+ while ((StgInfoTable *) RELAXED_LOAD(&p->header.info) == &stg_WHITEHOLE_info);
goto try_again;
case COMPACT_NFDATA:
@@ -1736,24 +1738,25 @@ mark_closure (MarkQueue *queue, const StgClosure *p0, StgClosure **origin)
* the object's pointers since in the case of marking stacks there may be a
* mutator waiting for us to finish so it can start execution.
*/
- if (bd->flags & BF_LARGE) {
+ uint16_t bd_flags = block_get_flags(bd);
+ if (bd_flags & BF_LARGE) {
/* Marking a large object isn't idempotent since we move it to
* nonmoving_marked_large_objects; to ensure that we don't repeatedly
* mark a large object, we only set BF_MARKED on large objects in the
* nonmoving heap while holding nonmoving_large_objects_mutex
*/
ACQUIRE_LOCK(&nonmoving_large_objects_mutex);
- if (! (bd->flags & BF_MARKED)) {
+ if (! (bd_flags & BF_MARKED)) {
// Remove the object from nonmoving_large_objects and link it to
// nonmoving_marked_large_objects
dbl_link_remove(bd, &nonmoving_large_objects);
dbl_link_onto(bd, &nonmoving_marked_large_objects);
n_nonmoving_large_blocks -= bd->blocks;
n_nonmoving_marked_large_blocks += bd->blocks;
- bd->flags |= BF_MARKED;
+ block_set_flag(bd, BF_MARKED);
}
RELEASE_LOCK(&nonmoving_large_objects_mutex);
- } else if (bd->flags & BF_NONMOVING) {
+ } else if (bd_flags & BF_NONMOVING) {
// TODO: Kill repetition
struct NonmovingSegment *seg = nonmovingGetSegment((StgPtr) p);
nonmoving_block_idx block_idx = nonmovingGetBlockIdx((StgPtr) p);
@@ -1768,7 +1771,7 @@ mark_closure (MarkQueue *queue, const StgClosure *p0, StgClosure **origin)
}
done:
- if (origin != NULL && (!HEAP_ALLOCED(p) || bd->flags & BF_NONMOVING)) {
+ if (origin != NULL && (!HEAP_ALLOCED(p) || block_get_flags(bd) & BF_NONMOVING)) {
if (UNTAG_CLOSURE((StgClosure*)p0) != p && *origin == p0) {
if (cas((StgVolatilePtr)origin, (StgWord)p0, (StgWord)TAG_CLOSURE(tag, p)) == (StgWord)p0) {
// debugBelch("Thunk optimization successful\n");
@@ -1865,19 +1868,20 @@ bool nonmovingIsAlive (StgClosure *p)
}
bdescr *bd = Bdescr((P_)p);
+ uint16_t bd_flags = block_get_flags(bd);
// All non-static objects in the non-moving heap should be marked as
// BF_NONMOVING
- ASSERT(bd->flags & BF_NONMOVING);
+ ASSERT(bd_flags & BF_NONMOVING);
- if (bd->flags & (BF_COMPACT | BF_LARGE)) {
- if (bd->flags & BF_COMPACT) {
+ if (bd_flags & (BF_COMPACT | BF_LARGE)) {
+ if (bd_flags & BF_COMPACT) {
StgCompactNFData *str = objectGetCompact((StgClosure*)p);
bd = Bdescr((P_)str);
}
- return (bd->flags & BF_NONMOVING_SWEEPING) == 0
+ return (bd_flags & BF_NONMOVING_SWEEPING) == 0
// the large object wasn't in the snapshot and therefore wasn't marked
- || (bd->flags & BF_MARKED) != 0;
+ || (bd_flags & BF_MARKED) != 0;
// The object was marked
} else {
struct NonmovingSegment *seg = nonmovingGetSegment((StgPtr) p);
@@ -1931,8 +1935,8 @@ static bool nonmovingIsNowAlive (StgClosure *p)
}
bdescr *bd = Bdescr((P_)p);
+ const uint16_t flags = block_get_flags(bd);
- const uint16_t flags = bd->flags;
if (flags & BF_LARGE) {
if (flags & BF_PINNED && !(flags & BF_NONMOVING)) {
// In this case we have a pinned object living in a non-full
@@ -1942,15 +1946,15 @@ static bool nonmovingIsNowAlive (StgClosure *p)
return true;
}
- ASSERT(bd->flags & BF_NONMOVING);
- return (bd->flags & BF_NONMOVING_SWEEPING) == 0
+ ASSERT(flags & BF_NONMOVING);
+ return (flags & BF_NONMOVING_SWEEPING) == 0
// the large object wasn't in the snapshot and therefore wasn't marked
- || (bd->flags & BF_MARKED) != 0;
+ || (flags & BF_MARKED) != 0;
// The object was marked
} else {
// All non-static objects in the non-moving heap should be marked as
// BF_NONMOVING.
- ASSERT(bd->flags & BF_NONMOVING);
+ ASSERT(flags & BF_NONMOVING);
struct NonmovingSegment *seg = nonmovingGetSegment((StgPtr) p);
StgClosure *snapshot_loc =
@@ -2013,7 +2017,8 @@ bool nonmovingTidyWeaks (struct MarkQueue_ *queue)
// See Note [Weak pointer processing and the non-moving GC] in
// MarkWeak.c
- bool key_in_nonmoving = HEAP_ALLOCED_GC(w->key) && Bdescr((StgPtr) w->key)->flags & BF_NONMOVING;
+ bdescr *key_bd = Bdescr((StgPtr) w->key);
+ bool key_in_nonmoving = HEAP_ALLOCED_GC(w->key) && block_get_flags(key_bd) & BF_NONMOVING;
if (!key_in_nonmoving || nonmovingIsNowAlive(w->key)) {
nonmovingMarkLiveWeak(queue, w);
did_work = true;
=====================================
rts/sm/Scav.c
=====================================
@@ -1269,6 +1269,7 @@ scavenge_one(StgPtr p)
bool no_luck;
bool saved_eager_promotion;
+try_again:
saved_eager_promotion = gct->eager_promotion;
ASSERT(LOOKS_LIKE_CLOSURE_PTR(p));
@@ -1610,6 +1611,13 @@ scavenge_one(StgPtr p)
scavenge_continuation((StgContinuation *)p);
break;
+ case WHITEHOLE:
+ // This may happen when the nonmoving GC is in use via two paths:
+ // 1. when an MVAR is being marked concurrently
+ // 2. when the object belongs to a chain of selectors being short-cutted.
+ while (get_itbl((StgClosure*) p) == &stg_WHITEHOLE_info);
+ goto try_again;
+
default:
barf("scavenge_one: strange object %d", (int)(info->type));
}
=====================================
testsuite/driver/cpu_features.py
=====================================
@@ -8,8 +8,8 @@ SUPPORTED_CPU_FEATURES = {
# These aren't comprehensive; they are only CPU features that we care about
# x86:
- 'sse', 'sse2', 'sse3', 'ssse3', 'sse4_1', 'sse4_2',
- 'avx1', 'avx2',
+ 'sse', 'sse2', 'sse3', 'pni', 'ssse3', 'sse4_1', 'sse4_2',
+ 'avx', 'avx2', 'avx512f',
'fma',
'popcnt', 'bmi1', 'bmi2'
}
=====================================
testsuite/tests/cross/should_run/T26449.hs
=====================================
@@ -0,0 +1,16 @@
+import Control.Monad
+import GHC.Platform.ArchOS
+import GHC.Platform.Host
+import System.Info
+
+main :: IO ()
+main =
+ when ((arch, os) /= (arch', os')) $
+ fail $
+ "System.Info says host platform is "
+ <> show (arch, os)
+ <> " but GHC.Platform.Host says "
+ <> show (arch', os')
+ where
+ (arch', os') =
+ (stringEncodeArch hostPlatformArch, stringEncodeOS hostPlatformOS)
=====================================
testsuite/tests/cross/should_run/all.T
=====================================
@@ -0,0 +1 @@
+test('T26449', [], compile_and_run, [''])
=====================================
testsuite/tests/perf/compiler/T26425.hs
=====================================
@@ -0,0 +1,514 @@
+{-# LANGUAGE OverloadedStrings #-}
+
+module Reproducer (strToInt) where
+
+import qualified Data.Text as T
+
+{- This program results in a nested chain of join points and cases which tests
+ primarily OccAnal and Unfolding performance.
+-}
+
+strToInt :: T.Text -> Maybe Int
+strToInt txt = case txt of
+ "0" -> Just 0
+ "1" -> Just 1
+ "2" -> Just 2
+ "3" -> Just 3
+ "4" -> Just 4
+ "5" -> Just 5
+ "6" -> Just 6
+ "7" -> Just 7
+ "8" -> Just 8
+ "9" -> Just 9
+ "10" -> Just 10
+ "11" -> Just 11
+ "12" -> Just 12
+ "13" -> Just 13
+ "14" -> Just 14
+ "15" -> Just 15
+ "16" -> Just 16
+ "17" -> Just 17
+ "18" -> Just 18
+ "19" -> Just 19
+ "20" -> Just 20
+ "21" -> Just 21
+ "22" -> Just 22
+ "23" -> Just 23
+ "24" -> Just 24
+ "25" -> Just 25
+ "26" -> Just 26
+ "27" -> Just 27
+ "28" -> Just 28
+ "29" -> Just 29
+ "30" -> Just 30
+ "31" -> Just 31
+ "32" -> Just 32
+ "33" -> Just 33
+ "34" -> Just 34
+ "35" -> Just 35
+ "36" -> Just 36
+ "37" -> Just 37
+ "38" -> Just 38
+ "39" -> Just 39
+ "40" -> Just 40
+ "41" -> Just 41
+ "42" -> Just 42
+ "43" -> Just 43
+ "44" -> Just 44
+ "45" -> Just 45
+ "46" -> Just 46
+ "47" -> Just 47
+ "48" -> Just 48
+ "49" -> Just 49
+ "50" -> Just 50
+ "51" -> Just 51
+ "52" -> Just 52
+ "53" -> Just 53
+ "54" -> Just 54
+ "55" -> Just 55
+ "56" -> Just 56
+ "57" -> Just 57
+ "58" -> Just 58
+ "59" -> Just 59
+ "60" -> Just 60
+ "61" -> Just 61
+ "62" -> Just 62
+ "63" -> Just 63
+ "64" -> Just 64
+ "65" -> Just 65
+ "66" -> Just 66
+ "67" -> Just 67
+ "68" -> Just 68
+ "69" -> Just 69
+ "70" -> Just 70
+ "71" -> Just 71
+ "72" -> Just 72
+ "73" -> Just 73
+ "74" -> Just 74
+ "75" -> Just 75
+ "76" -> Just 76
+ "77" -> Just 77
+ "78" -> Just 78
+ "79" -> Just 79
+ "80" -> Just 80
+ "81" -> Just 81
+ "82" -> Just 82
+ "83" -> Just 83
+ "84" -> Just 84
+ "85" -> Just 85
+ "86" -> Just 86
+ "87" -> Just 87
+ "88" -> Just 88
+ "89" -> Just 89
+ "90" -> Just 90
+ "91" -> Just 91
+ "92" -> Just 92
+ "93" -> Just 93
+ "94" -> Just 94
+ "95" -> Just 95
+ "96" -> Just 96
+ "97" -> Just 97
+ "98" -> Just 98
+ "99" -> Just 99
+ "100" -> Just 100
+ "101" -> Just 101
+ "102" -> Just 102
+ "103" -> Just 103
+ "104" -> Just 104
+ "105" -> Just 105
+ "106" -> Just 106
+ "107" -> Just 107
+ "108" -> Just 108
+ "109" -> Just 109
+ "110" -> Just 110
+ "111" -> Just 111
+ "112" -> Just 112
+ "113" -> Just 113
+ "114" -> Just 114
+ "115" -> Just 115
+ "116" -> Just 116
+ "117" -> Just 117
+ "118" -> Just 118
+ "119" -> Just 119
+ "120" -> Just 120
+ "121" -> Just 121
+ "122" -> Just 122
+ "123" -> Just 123
+ "124" -> Just 124
+ "125" -> Just 125
+ "126" -> Just 126
+ "127" -> Just 127
+ "128" -> Just 128
+ "129" -> Just 129
+ "130" -> Just 130
+ "131" -> Just 131
+ "132" -> Just 132
+ "133" -> Just 133
+ "134" -> Just 134
+ "135" -> Just 135
+ "136" -> Just 136
+ "137" -> Just 137
+ "138" -> Just 138
+ "139" -> Just 139
+ "140" -> Just 140
+ "141" -> Just 141
+ "142" -> Just 142
+ "143" -> Just 143
+ "144" -> Just 144
+ "145" -> Just 145
+ "146" -> Just 146
+ "147" -> Just 147
+ "148" -> Just 148
+ "149" -> Just 149
+ "150" -> Just 150
+ "151" -> Just 151
+ "152" -> Just 152
+ "153" -> Just 153
+ "154" -> Just 154
+ "155" -> Just 155
+ "156" -> Just 156
+ "157" -> Just 157
+ "158" -> Just 158
+ "159" -> Just 159
+ "160" -> Just 160
+ "161" -> Just 161
+ "162" -> Just 162
+ "163" -> Just 163
+ "164" -> Just 164
+ "165" -> Just 165
+ "166" -> Just 166
+ "167" -> Just 167
+ "168" -> Just 168
+ "169" -> Just 169
+ "170" -> Just 170
+ "171" -> Just 171
+ "172" -> Just 172
+ "173" -> Just 173
+ "174" -> Just 174
+ "175" -> Just 175
+ "176" -> Just 176
+ "177" -> Just 177
+ "178" -> Just 178
+ "179" -> Just 179
+ "180" -> Just 180
+ "181" -> Just 181
+ "182" -> Just 182
+ "183" -> Just 183
+ "184" -> Just 184
+ "185" -> Just 185
+ "186" -> Just 186
+ "187" -> Just 187
+ "188" -> Just 188
+ "189" -> Just 189
+ "190" -> Just 190
+ "191" -> Just 191
+ "192" -> Just 192
+ "193" -> Just 193
+ "194" -> Just 194
+ "195" -> Just 195
+ "196" -> Just 196
+ "197" -> Just 197
+ "198" -> Just 198
+ "199" -> Just 199
+ "200" -> Just 200
+ "201" -> Just 201
+ "202" -> Just 202
+ "203" -> Just 203
+ "204" -> Just 204
+ "205" -> Just 205
+ "206" -> Just 206
+ "207" -> Just 207
+ "208" -> Just 208
+ "209" -> Just 209
+ "210" -> Just 210
+ "211" -> Just 211
+ "212" -> Just 212
+ "213" -> Just 213
+ "214" -> Just 214
+ "215" -> Just 215
+ "216" -> Just 216
+ "217" -> Just 217
+ "218" -> Just 218
+ "219" -> Just 219
+ "220" -> Just 220
+ "221" -> Just 221
+ "222" -> Just 222
+ "223" -> Just 223
+ "224" -> Just 224
+ "225" -> Just 225
+ "226" -> Just 226
+ "227" -> Just 227
+ "228" -> Just 228
+ "229" -> Just 229
+ "230" -> Just 230
+ "231" -> Just 231
+ "232" -> Just 232
+ "233" -> Just 233
+ "234" -> Just 234
+ "235" -> Just 235
+ "236" -> Just 236
+ "237" -> Just 237
+ "238" -> Just 238
+ "239" -> Just 239
+ "240" -> Just 240
+ "241" -> Just 241
+ "242" -> Just 242
+ "243" -> Just 243
+ "244" -> Just 244
+ "245" -> Just 245
+ "246" -> Just 246
+ "247" -> Just 247
+ "248" -> Just 248
+ "249" -> Just 249
+ "250" -> Just 250
+ "251" -> Just 251
+ "252" -> Just 252
+ "253" -> Just 253
+ "254" -> Just 254
+ "255" -> Just 255
+ "256" -> Just 256
+ "257" -> Just 257
+ "258" -> Just 258
+ "259" -> Just 259
+ "260" -> Just 260
+ "261" -> Just 261
+ "262" -> Just 262
+ "263" -> Just 263
+ "264" -> Just 264
+ "265" -> Just 265
+ "266" -> Just 266
+ "267" -> Just 267
+ "268" -> Just 268
+ "269" -> Just 269
+ "270" -> Just 270
+ "271" -> Just 271
+ "272" -> Just 272
+ "273" -> Just 273
+ "274" -> Just 274
+ "275" -> Just 275
+ "276" -> Just 276
+ "277" -> Just 277
+ "278" -> Just 278
+ "279" -> Just 279
+ "280" -> Just 280
+ "281" -> Just 281
+ "282" -> Just 282
+ "283" -> Just 283
+ "284" -> Just 284
+ "285" -> Just 285
+ "286" -> Just 286
+ "287" -> Just 287
+ "288" -> Just 288
+ "289" -> Just 289
+ "290" -> Just 290
+ "291" -> Just 291
+ "292" -> Just 292
+ "293" -> Just 293
+ "294" -> Just 294
+ "295" -> Just 295
+ "296" -> Just 296
+ "297" -> Just 297
+ "298" -> Just 298
+ "299" -> Just 299
+ "300" -> Just 300
+ "301" -> Just 301
+ "302" -> Just 302
+ "303" -> Just 303
+ "304" -> Just 304
+ "305" -> Just 305
+ "306" -> Just 306
+ "307" -> Just 307
+ "308" -> Just 308
+ "309" -> Just 309
+ "310" -> Just 310
+ "311" -> Just 311
+ "312" -> Just 312
+ "313" -> Just 313
+ "314" -> Just 314
+ "315" -> Just 315
+ "316" -> Just 316
+ "317" -> Just 317
+ "318" -> Just 318
+ "319" -> Just 319
+ "320" -> Just 320
+ "321" -> Just 321
+ "322" -> Just 322
+ "323" -> Just 323
+ "324" -> Just 324
+ "325" -> Just 325
+ "326" -> Just 326
+ "327" -> Just 327
+ "328" -> Just 328
+ "329" -> Just 329
+ "330" -> Just 330
+ "331" -> Just 331
+ "332" -> Just 332
+ "333" -> Just 333
+ "334" -> Just 334
+ "335" -> Just 335
+ "336" -> Just 336
+ "337" -> Just 337
+ "338" -> Just 338
+ "339" -> Just 339
+ "340" -> Just 340
+ "341" -> Just 341
+ "342" -> Just 342
+ "343" -> Just 343
+ "344" -> Just 344
+ "345" -> Just 345
+ "346" -> Just 346
+ "347" -> Just 347
+ "348" -> Just 348
+ "349" -> Just 349
+ "350" -> Just 350
+ "351" -> Just 351
+ "352" -> Just 352
+ "353" -> Just 353
+ "354" -> Just 354
+ "355" -> Just 355
+ "356" -> Just 356
+ "357" -> Just 357
+ "358" -> Just 358
+ "359" -> Just 359
+ "360" -> Just 360
+ "361" -> Just 361
+ "362" -> Just 362
+ "363" -> Just 363
+ "364" -> Just 364
+ "365" -> Just 365
+ "366" -> Just 366
+ "367" -> Just 367
+ "368" -> Just 368
+ "369" -> Just 369
+ "370" -> Just 370
+ "371" -> Just 371
+ "372" -> Just 372
+ "373" -> Just 373
+ "374" -> Just 374
+ "375" -> Just 375
+ "376" -> Just 376
+ "377" -> Just 377
+ "378" -> Just 378
+ "379" -> Just 379
+ "380" -> Just 380
+ "381" -> Just 381
+ "382" -> Just 382
+ "383" -> Just 383
+ "384" -> Just 384
+ "385" -> Just 385
+ "386" -> Just 386
+ "387" -> Just 387
+ "388" -> Just 388
+ "389" -> Just 389
+ "390" -> Just 390
+ "391" -> Just 391
+ "392" -> Just 392
+ "393" -> Just 393
+ "394" -> Just 394
+ "395" -> Just 395
+ "396" -> Just 396
+ "397" -> Just 397
+ "398" -> Just 398
+ "399" -> Just 399
+ "400" -> Just 400
+ "401" -> Just 401
+ "402" -> Just 402
+ "403" -> Just 403
+ "404" -> Just 404
+ "405" -> Just 405
+ "406" -> Just 406
+ "407" -> Just 407
+ "408" -> Just 408
+ "409" -> Just 409
+ "410" -> Just 410
+ "411" -> Just 411
+ "412" -> Just 412
+ "413" -> Just 413
+ "414" -> Just 414
+ "415" -> Just 415
+ "416" -> Just 416
+ "417" -> Just 417
+ "418" -> Just 418
+ "419" -> Just 419
+ "420" -> Just 420
+ "421" -> Just 421
+ "422" -> Just 422
+ "423" -> Just 423
+ "424" -> Just 424
+ "425" -> Just 425
+ "426" -> Just 426
+ "427" -> Just 427
+ "428" -> Just 428
+ "429" -> Just 429
+ "430" -> Just 430
+ "431" -> Just 431
+ "432" -> Just 432
+ "433" -> Just 433
+ "434" -> Just 434
+ "435" -> Just 435
+ "436" -> Just 436
+ "437" -> Just 437
+ "438" -> Just 438
+ "439" -> Just 439
+ "440" -> Just 440
+ "441" -> Just 441
+ "442" -> Just 442
+ "443" -> Just 443
+ "444" -> Just 444
+ "445" -> Just 445
+ "446" -> Just 446
+ "447" -> Just 447
+ "448" -> Just 448
+ "449" -> Just 449
+ "450" -> Just 450
+ "451" -> Just 451
+ "452" -> Just 452
+ "453" -> Just 453
+ "454" -> Just 454
+ "455" -> Just 455
+ "456" -> Just 456
+ "457" -> Just 457
+ "458" -> Just 458
+ "459" -> Just 459
+ "460" -> Just 460
+ "461" -> Just 461
+ "462" -> Just 462
+ "463" -> Just 463
+ "464" -> Just 464
+ "465" -> Just 465
+ "466" -> Just 466
+ "467" -> Just 467
+ "468" -> Just 468
+ "469" -> Just 469
+ "470" -> Just 470
+ "471" -> Just 471
+ "472" -> Just 472
+ "473" -> Just 473
+ "474" -> Just 474
+ "475" -> Just 475
+ "476" -> Just 476
+ "477" -> Just 477
+ "478" -> Just 478
+ "479" -> Just 479
+ "480" -> Just 480
+ "481" -> Just 481
+ "482" -> Just 482
+ "483" -> Just 483
+ "484" -> Just 484
+ "485" -> Just 485
+ "486" -> Just 486
+ "487" -> Just 487
+ "488" -> Just 488
+ "489" -> Just 489
+ "490" -> Just 490
+ "491" -> Just 491
+ "492" -> Just 492
+ "493" -> Just 493
+ "494" -> Just 494
+ "495" -> Just 495
+ "496" -> Just 496
+ "497" -> Just 497
+ "498" -> Just 498
+ "499" -> Just 499
+ "500" -> Just 500
+ _ -> Nothing
=====================================
testsuite/tests/perf/compiler/all.T
=====================================
@@ -716,3 +716,8 @@ test ('LookupFusion',
test('T24471',
[req_th, collect_compiler_stats('all', 5)],
multimod_compile, ['T24471', '-v0 -O'])
+
+test ('T26425',
+ [ collect_compiler_stats('all',5) ],
+ compile,
+ ['-O'])
\ No newline at end of file
=====================================
testsuite/tests/perf/should_run/T3586.hs
=====================================
@@ -1,5 +1,5 @@
{-# LANGUAGE BangPatterns #-}
-{-# OPTIONS -fvia-C -optc-O3 -fexcess-precision -optc-msse3 #-}
+{-# OPTIONS -optc-O3 -fexcess-precision #-}
import Control.Monad.ST
import Data.Array.ST
=====================================
testsuite/tests/perf/should_run/all.T
=====================================
@@ -43,6 +43,7 @@ test('T3586',
[collect_runtime_residency(2),
collect_stats('bytes allocated', 5),
only_ways(['normal']),
+ when(have_cpu_feature('pni') or have_cpu_feature('sse3'), extra_hc_opts('-optc-msse3')),
],
compile_and_run,
['-O'])
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/e03f58bd5cbf944049017d8dba8aab…
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/e03f58bd5cbf944049017d8dba8aab…
You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help
1
0
[Git][ghc/ghc][wip/fix-9809] 4 commits: X86 NCG: use 32-bit POPCNT for sub-word popcounts
by Erik de Castro Lopo (@erikd) 28 Oct '26
by Erik de Castro Lopo (@erikd) 28 Oct '26
28 Oct '26
Erik de Castro Lopo pushed to branch wip/fix-9809 at Glasgow Haskell Compiler / GHC
Commits:
ac870ad5 by Simon Jakobi at 2026-10-28T04:40:42-04:00
X86 NCG: use 32-bit POPCNT for sub-word popcounts
A 32-bit write zero-extends the result to the full register, so we
don't need to emit additional code for this.
We also skip emitting zero-extension instructions for the argument
when it is already visibly zero-extended.
For the byte-wise popcount loop from #24813, this shrinks the popcount
sequence from 5 instructions to 3.
Assisted-by: Claude Fable 5.1
- - - - -
53087373 by Facundo Domínguez at 2026-10-28T04:41:32-04:00
Use a ShortByteString in Serialized instead of [Word8]
This makes the in-memory representation 40x more compact for all values
stored in annotations.
A test is added to measure the representation size of annotations
specified in the compiler input.
The GHC API is modified. All functions that previously handled annotations
in [Word8] form now use ShortByteString. The following definitions have
modified types:
* GHC.findGlobalAnns
* GHC.Tc.Gen.Splice.getAnnotationsByTypeRep
* GHC.Core.Opt.Monad.getAnnotations
* GHC.Core.Opt.Monad.getFirstAnnotations
* GHC.Types.Annotations.deserializeAnns
* GHC.Types.Annotations.findAnns
* GHC.Types.Annotations.findAnnsByTypeRep
* GHC.Serialized.Serialized
* GHC.Serialized.toSerialized
* GHC.Serialized.fromSerialized
* GHC.Serialized.serializeWithData
* GHC.Serialized.deserializeWithData
* GHCi.Message.ReifyAnnotations
Fixes #28002.
- - - - -
af899dcc by Erik de Castro Lopo at 2026-10-28T22:27:57+11:00
TimerManager: publish only evaluated timeout queues (#9809)
When many threads register timeouts at once, for example 100k forked
threads each calling registerDelay or threadDelay, the timer manager
showed huge GC overhead, memory use and timer latency. With 100k forked
registrations on -N4, master took 0.59 s and 437 MB peak memory, against
0.03 s and 31 MB when the same registrations come from one thread.
The cause goes back to 2d5eccdf1c ("IO manager: Edit the timeout queue
directly, rather than using an edit list", fixing #7653), which made every
registration atomicModifyIORef' the shared timeout queue. That publishes
the new queue as a thunk and only evaluates it afterwards, so each thread
builds its O(log n) insert on top of the previous thread's unevaluated
insert, giving thunk chains, threads blocking on each other's partly
evaluated updates, and many queue copies kept alive at once.
Registering threads now use a compare-and-swap loop that evaluates the new
queue before publishing it (IntPSQ is spine strict, so WHNF is fully
evaluated). The timer manager's own update in mkTimeout keeps using
atomicModifyIORef': with compare-and-swap its long update almost never won
against a stream of short inserts on other capabilities, which starved it
and delayed expired timeouts by hundreds of milliseconds.
Forked registrations of 100k timeouts (median of 5 runs):
-N4: 0.589 s, 437 MB -> 0.077 s, 37 MB
-N12: 1.182 s, 465 MB -> 0.125 s, 74 MB
Forked registrations of 1M timeouts on -N1:
5.85 s (range 0.69-6.49 s), 1824 MB -> 0.69 s, 193 MB
Single-thread registrations are unchanged or slightly faster, and timer
latency during registration storms improves (p99 lateness of a 1 ms
threadDelay during 100k forked registrations on -N4: 342-534 ms -> 4-7 ms).
The registration code still runs in constant stack, so #7653 stays fixed;
T7653 and the concurrent, base and ghc-internal testsuites pass.
modifyTimeouts is OPAQUE so that the optimiser cannot expose the edit
function's inspection of the old queue next to the compare-and-swap and
replace the expected pointer with the evaluated queue, which would make the
compare-and-swap fail forever when the MutVar# holds a thunk or indirection.
See Note [Pointer equality in modifyTimeouts].
Regression tests included.
Co-Authored-By: Claude Opus 5.5 <noreply(a)anthropic.com>
- - - - -
bf2d932e by Moritz Angermann at 2026-10-28T22:27:57+11:00
testsuite: give pause_resume_via_safe_ffi_concurrent 60s on Darwin CI
The 10s System.Timeout watchdog was a false deadlock: five serialized
1s pauses plus rts_pause waiting on threadDelay/timer-manager traffic
overruns that budget on loaded Mac runners. Keep a watchdog so a real
hang still fails.
- - - - -
29 changed files:
- + changelog.d/T28002
- + changelog.d/T28002-api
- + changelog.d/T9809-timer-manager-contention
- + changelog.d/ncg-x86-subword-popcnt
- compiler/GHC.hs
- compiler/GHC/CmmToAsm/X86/CodeGen.hs
- compiler/GHC/Core/Opt/Monad.hs
- compiler/GHC/Tc/Gen/Splice.hs
- compiler/GHC/Types/Annotations.hs
- compiler/GHC/Utils/Outputable.hs
- libraries/base/tests/.gitignore
- + libraries/base/tests/T9809.hs
- libraries/base/tests/all.T
- libraries/ghc-boot/GHC/Serialized.hs
- libraries/ghc-internal/src/GHC/Internal/Event/TimerManager.hs
- libraries/ghci/GHCi/Message.hs
- libraries/ghci/GHCi/TH.hs
- libraries/ghci/GHCi/TH/Binary.hs
- testsuite/tests/annotations/should_run/Makefile
- + testsuite/tests/annotations/should_run/T16841.hs
- + testsuite/tests/annotations/should_run/T16841.stdout
- testsuite/tests/annotations/should_run/all.T
- + testsuite/tests/codeGen/should_gen_asm/T24813.asm
- + testsuite/tests/codeGen/should_gen_asm/T24813.cmm
- testsuite/tests/codeGen/should_gen_asm/all.T
- testsuite/tests/plugins/T26761Plugin.hs
- testsuite/tests/plugins/all.T
- testsuite/tests/rts/pause-resume/all.T
- testsuite/tests/rts/pause-resume/pause_resume_via_safe_ffi_concurrent.hs
Changes:
=====================================
changelog.d/T28002
=====================================
@@ -0,0 +1,8 @@
+section: compiler
+issues: #28002
+mrs: !16841
+synopsis:
+ Use ``ShortByteString`` in the ``Serialized`` type instead of ``[Word8]``
+description:
+ ``GHC.Serialized.Serialized`` now uses ``ByteString`` to store serialized data.
+ This makes in-memory representation more compact.
=====================================
changelog.d/T28002-api
=====================================
@@ -0,0 +1,21 @@
+section: ghc-lib
+issues: #28002
+mrs: !16841
+synopsis:
+ Use ``ShortByteString`` in the ``Serialized`` type instead of ``[Word8]``
+description:
+ Breaking change: All functions that previously handled annotations in ``[Word8]``
+ form now use ``ShortByteString``. The following definitions have modified types:
+ * ``GHC.findGlobalAnns``
+ * ``GHC.Tc.Gen.Splice.getAnnotationsByTypeRep``
+ * ``GHC.Core.Opt.Monad.getAnnotations``
+ * ``GHC.Core.Opt.Monad.getFirstAnnotations``
+ * ``GHC.Types.Annotations.deserializeAnns``
+ * ``GHC.Types.Annotations.findAnns``
+ * ``GHC.Types.Annotations.findAnnsByTypeRep``
+ * ``GHC.Serialized.Serialized``
+ * ``GHC.Serialized.toSerialized``
+ * ``GHC.Serialized.fromSerialized``
+ * ``GHC.Serialized.serializeWithData``
+ * ``GHC.Serialized.deserializeWithData``
+ * ``GHCi.Message.ReifyAnnotations``
=====================================
changelog.d/T9809-timer-manager-contention
=====================================
@@ -0,0 +1,12 @@
+section: ghc-internal
+synopsis: Fix excessive GC, memory use and timer latency when many threads
+ register timeouts at once with ``threadDelay``, ``registerDelay`` or
+ ``System.Timeout.timeout``.
+issues: #9809
+mrs: !16863
+description: {
+ Threads registering timeouts now publish only fully evaluated updates to
+ the timer manager's timeout queue, using a compare-and-swap loop, instead
+ of building chains of unevaluated updates that other threads then had to
+ evaluate or block on.
+}
=====================================
changelog.d/ncg-x86-subword-popcnt
=====================================
@@ -0,0 +1,5 @@
+section: compiler
+synopsis: The x86 native code generator now emits shorter code for 8-bit and
+ 16-bit ``popCount`` when :ghc-flag:`-msse4.2` is enabled.
+mrs: !16640
+issues: #24813
=====================================
compiler/GHC.hs
=====================================
@@ -453,12 +453,12 @@ import qualified GHC.Unit.Home.Graph as HUG
import Control.Applicative ((<|>))
import Control.Monad
import Control.Monad.Catch as MC
+import Data.ByteString.Short (ShortByteString)
import Data.Foldable
import Data.Function ((&))
import Data.IORef
import Data.List (isPrefixOf)
import Data.Typeable ( Typeable )
-import Data.Word ( Word8 )
import qualified Data.Map.Strict as Map
import Data.Set (Set)
@@ -1526,7 +1526,7 @@ lookupGlobalName :: GhcMonad m => Name -> m (Maybe TyThing)
lookupGlobalName name = withSession $ \hsc_env -> do
liftIO $ lookupType hsc_env name
-findGlobalAnns :: (GhcMonad m, Typeable a) => ([Word8] -> a) -> AnnTarget Name -> m [a]
+findGlobalAnns :: (GhcMonad m, Typeable a) => (ShortByteString -> a) -> AnnTarget Name -> m [a]
findGlobalAnns deserialize target = withSession $ \hsc_env -> do
ann_env <- liftIO $ prepareAnnotations hsc_env Nothing
return (findAnns deserialize ann_env target)
=====================================
compiler/GHC/CmmToAsm/X86/CodeGen.hs
=====================================
@@ -6739,20 +6739,23 @@ genPopCnt bid width dst src = do
True -> do
code_src <- getAnyReg src
- src_r <- getNewRegNat format
+ -- Sub-word popcounts use 32-bit POPCNT, whose write zero-extends the
+ -- result to the full register.
+ src_r <- getNewRegNat (max II32 format)
let dst_r = getRegisterReg platform (CmmLocal dst)
+ -- A sub-word argument is a full Word# whose upper bits may be
+ -- dirty, so zero-extend it unless it visibly already is.
+ zx_src
+ | CmmMachOp (MO_UU_Conv from _) [_] <- src, from <= width
+ = nilOL
+ | otherwise
+ = unitOL (MOVZxL format (OpReg src_r) (OpReg src_r))
return $ code_src src_r `appOL`
- (if width == W8 then
- -- The POPCNT instruction doesn't take a r/m8
- unitOL (MOVZxL II8 (OpReg src_r) (OpReg src_r)) `appOL`
- unitOL (POPCNT II16 (OpReg src_r) dst_r)
- else
- unitOL (POPCNT format (OpReg src_r) dst_r)) `appOL`
(if width == W8 || width == W16 then
- -- We used a 16-bit destination register above,
- -- so zero-extend
- unitOL (MOVZxL II16 (OpReg dst_r) (OpReg dst_r))
- else nilOL)
+ zx_src `appOL`
+ unitOL (POPCNT II32 (OpReg src_r) dst_r)
+ else
+ unitOL (POPCNT format (OpReg src_r) dst_r))
False ->
-- generate C call to hs_popcntN in ghc-prim
=====================================
compiler/GHC/Core/Opt/Monad.hs
=====================================
@@ -68,9 +68,9 @@ import GHC.Unit.Module.ModGuts
import GHC.Unit.External
import Data.Bifunctor ( bimap )
+import Data.ByteString.Short ( ShortByteString )
import Data.Dynamic
import Data.Maybe (listToMaybe)
-import Data.Word
import Control.Monad
import Control.Applicative ( Alternative(..) )
@@ -323,14 +323,14 @@ get_eps = do
-- annotations.
--
-- See Note [Annotations]
-getAnnotations :: Typeable a => ([Word8] -> a) -> ModGuts -> CoreM (ModuleEnv [a], NameEnv [a])
+getAnnotations :: Typeable a => (ShortByteString -> a) -> ModGuts -> CoreM (ModuleEnv [a], NameEnv [a])
getAnnotations deserialize guts = do
hsc_env <- getHscEnv
ann_env <- liftIO $ prepareAnnotations hsc_env (Just guts)
return (deserializeAnns deserialize ann_env)
-- | Get at most one annotation of a given type per annotatable item.
-getFirstAnnotations :: Typeable a => ([Word8] -> a) -> ModGuts -> CoreM (ModuleEnv a, NameEnv a)
+getFirstAnnotations :: Typeable a => (ShortByteString -> a) -> ModGuts -> CoreM (ModuleEnv a, NameEnv a)
getFirstAnnotations deserialize guts
= bimap mod name <$> getAnnotations deserialize guts
where
=====================================
compiler/GHC/Tc/Gen/Splice.hs
=====================================
@@ -155,10 +155,10 @@ import Control.DeepSeq
import Control.Monad
import Data.Binary
import Data.Binary.Get
+import Data.ByteString.Short (ShortByteString)
import Data.Containers.ListUtils ( nubOrd )
import Data.List ( sortBy )
import Data.Maybe
-import qualified Data.ByteString as B
import qualified Data.ByteString.Lazy as LB
import Data.Dynamic ( fromDynamic, toDyn )
import qualified Data.IntMap as IntMap
@@ -1947,7 +1947,7 @@ handleTHMessage msg = case msg of
ReifyInstances n ts -> wrapTHResult $ runQinTcM $ TH.reifyInstances n ts
ReifyRoles n -> wrapTHResult $ runQinTcM $ TH.reifyRoles n
ReifyAnnotations lookup tyrep ->
- wrapTHResult $ (map B.pack <$> getAnnotationsByTypeRep lookup tyrep)
+ wrapTHResult $ getAnnotationsByTypeRep lookup tyrep
ReifyModule m -> wrapTHResult $ runQinTcM $ TH.reifyModule m
ReifyConStrictness nm -> wrapTHResult $ runQinTcM $ TH.reifyConStrictness nm
GetPackageRoot -> wrapTHResult $ runQinTcM $ TH.getPackageRoot
@@ -1967,7 +1967,7 @@ handleTHMessage msg = case msg of
FailIfErrs -> wrapTHResult failIfErrsM
_ -> panic ("handleTHMessage: unexpected message " ++ show msg)
-getAnnotationsByTypeRep :: TH.AnnLookup -> TypeRep -> TcM [[Word8]]
+getAnnotationsByTypeRep :: TH.AnnLookup -> TypeRep -> TcM [ShortByteString]
getAnnotationsByTypeRep th_name tyrep
= do { name <- lookupThAnnLookup th_name
; topEnv <- getTopEnv
=====================================
compiler/GHC/Types/Annotations.hs
=====================================
@@ -27,9 +27,9 @@ import GHC.Utils.Outputable
import GHC.Serialized
import Control.Monad
+import Data.ByteString.Short ( ShortByteString )
import Data.Maybe
import Data.Typeable
-import Data.Word ( Word8 )
import Control.DeepSeq
-- | Represents an annotation after it has been sufficiently desugared from
@@ -112,14 +112,14 @@ plusAnnEnv a b =
-- | Find the annotations attached to the given target as 'Typeable'
-- values of your choice. If no deserializer is specified,
-- only transient annotations will be returned.
-findAnns :: Typeable a => ([Word8] -> a) -> AnnEnv -> CoreAnnTarget -> [a]
+findAnns :: Typeable a => (ShortByteString -> a) -> AnnEnv -> CoreAnnTarget -> [a]
findAnns deserialize env
= mapMaybe (fromSerialized deserialize) . findAnnPayloads env
-- | Find the annotations attached to the given target as 'Typeable'
-- values of your choice. If no deserializer is specified,
-- only transient annotations will be returned.
-findAnnsByTypeRep :: AnnEnv -> CoreAnnTarget -> TypeRep -> [[Word8]]
+findAnnsByTypeRep :: AnnEnv -> CoreAnnTarget -> TypeRep -> [ShortByteString]
findAnnsByTypeRep env target tyrep
= [ ws | Serialized tyrep' ws <- findAnnPayloads env target
, tyrep' == tyrep ]
@@ -134,7 +134,7 @@ findAnnPayloads env target =
-- | Deserialize all annotations of a given type. This happens lazily, that is
-- no deserialization will take place until the [a] is actually demanded and
-- the [a] can also be empty (the UniqFM is not filtered).
-deserializeAnns :: Typeable a => ([Word8] -> a) -> AnnEnv -> (ModuleEnv [a], NameEnv [a])
+deserializeAnns :: Typeable a => (ShortByteString -> a) -> AnnEnv -> (ModuleEnv [a], NameEnv [a])
deserializeAnns deserialize env
= ( mapModuleEnv deserAnns (ann_mod_env env)
, mapNameEnv deserAnns (ann_name_env env)
=====================================
compiler/GHC/Utils/Outputable.hs
=====================================
@@ -138,6 +138,7 @@ import GHC.Utils.Misc (lastMaybe, snocView)
import Data.ByteString (ByteString)
import qualified Data.ByteString as BS
+import qualified Data.ByteString.Short as Short
import Data.Char
import qualified Data.Map as M
import Data.Int
@@ -1105,7 +1106,7 @@ instance Outputable a => Outputable (SCC a) where
ppr (CyclicSCC vs) = text "REC" $$ (nest 3 (vcat (map ppr vs)))
instance Outputable Serialized where
- ppr (Serialized the_type bytes) = int (length bytes) <+> text "of type" <+> text (show the_type)
+ ppr (Serialized the_type bytes) = int (Short.length bytes) <+> text "of type" <+> text (show the_type)
instance Outputable Extension where
ppr = text . show
=====================================
libraries/base/tests/.gitignore
=====================================
@@ -275,4 +275,5 @@
/weak001
/T9395
/T9532
+/T9809
/T11555
=====================================
libraries/base/tests/T9809.hs
=====================================
@@ -0,0 +1,29 @@
+-- #9809: when many threads registered timeouts at the same time, the
+-- TimerManager published unevaluated timeout queues with atomicModifyIORef'.
+-- The threads then built long chains of thunks and blocked on each other's
+-- black holes, and the heap grew to hundreds of megabytes. all.T runs this
+-- test with a heap limit that the old code exceeded.
+
+import Control.Concurrent
+import Control.Monad
+import Data.IORef
+import GHC.Conc (TVar, atomically, newTVarIO, readTVar, registerDelay, retry, writeTVar)
+
+main :: IO ()
+main = do
+ let n = 100000
+ ref <- newIORef []
+ registered <- newTVarIO (0 :: Int)
+ replicateM_ n . forkIO $ do
+ tv <- registerDelay 10
+ atomicModifyIORef' ref (\tvs -> (tv : tvs, ()))
+ atomically $ readTVar registered >>= writeTVar registered . (+ 1)
+ atomically $ do
+ r <- readTVar registered
+ when (r /= n) retry
+ readIORef ref >>= mapM_ waitFor
+
+-- | Block until a delay has fired, so that the test also checks that the
+-- TimerManager fires every timeout.
+waitFor :: TVar Bool -> IO ()
+waitFor tv = atomically $ readTVar tv >>= \fired -> unless fired retry
=====================================
libraries/base/tests/all.T
=====================================
@@ -193,6 +193,12 @@ test('T7653', [when(opsys('mingw32'), skip),
omit_ways(prof_ways + ghci_ways)], compile_and_run, [''])
test('T7787', normal, compile_and_run, [''])
+# Before the fix for #9809 this needed more than 256MB of heap.
+test('T9809', [req_ghc_smp,
+ req_target_smp,
+ only_ways(['threaded1', 'threaded2', 'nonmoving_thr'])],
+ compile_and_run, ['-threaded -with-rtsopts="-N2 -M128m"'])
+
test('topHandler01', [when(opsys('mingw32'), skip), js_broken(22261), req_process], compile_and_run, [''])
test('topHandler02',
[when(opsys('mingw32'), skip),
=====================================
libraries/ghc-boot/GHC/Serialized.hs
=====================================
@@ -19,144 +19,68 @@ module GHC.Serialized (
) where
import Prelude -- See note [Why do we import Prelude here?]
-import Data.Bits
-import Data.Word ( Word8 )
-import Data.Data
import Control.DeepSeq
+import Data.Binary
+import Data.Binary.Get
+import Data.Binary.Put
+import Data.ByteString.Short (ShortByteString)
+import qualified Data.ByteString.Short as Short
+import qualified Data.ByteString.Lazy as ByteString.Lazy
+import Data.Data
-- | Represents a serialized value of a particular type. Attempts can be made to deserialize it at certain types
-data Serialized = Serialized TypeRep [Word8]
+data Serialized = Serialized TypeRep ShortByteString
instance NFData Serialized where
rnf (Serialized tr ws) = rnf tr `seq` rnf ws
-- | Put a Typeable value that we are able to actually turn into bytes into a 'Serialized' value ready for deserialization later
-toSerialized :: forall a. Typeable a => (a -> [Word8]) -> a -> Serialized
+toSerialized :: forall a. Typeable a => (a -> ShortByteString) -> a -> Serialized
toSerialized serialize what = Serialized (typeOf what) (serialize what)
-- | If the 'Serialized' value contains something of the given type, then use the specified deserializer to return @Just@ that.
-- Otherwise return @Nothing@.
-fromSerialized :: forall a. Typeable a => ([Word8] -> a) -> Serialized -> Maybe a
+fromSerialized :: forall a. Typeable a => (ShortByteString -> a) -> Serialized -> Maybe a
fromSerialized deserialize (Serialized the_type bytes)
| the_type == rep = Just (deserialize bytes)
| otherwise = Nothing
where rep = typeRep (Proxy :: Proxy a)
-- | Use a 'Data' instance to implement a serialization scheme dual to that of 'deserializeWithData'
-serializeWithData :: Data a => a -> [Word8]
-serializeWithData what = serializeWithData' what []
-
-serializeWithData' :: Data a => a -> [Word8] -> [Word8]
-serializeWithData' what = fst $ gfoldl (\(before, a_to_b) a -> (before . serializeWithData' a, a_to_b a))
- (\x -> (serializeConstr (constrRep (toConstr what)), x))
- what
+serializeWithData :: Data a => a -> ShortByteString
+serializeWithData = Short.toShort . ByteString.Lazy.toStrict . runPut . putData
-- | Use a 'Data' instance to implement a deserialization scheme dual to that of 'serializeWithData'
-deserializeWithData :: Data a => [Word8] -> a
-deserializeWithData = snd . deserializeWithData'
-
-deserializeWithData' :: forall a. Data a => [Word8] -> ([Word8], a)
-deserializeWithData' bytes = deserializeConstr bytes $ \constr_rep bytes ->
- gunfold (\(bytes, b_to_r) -> let (bytes', b) = deserializeWithData' bytes in (bytes', b_to_r b))
- (\x -> (bytes, x))
- (repConstr (dataTypeOf (undefined :: a)) constr_rep)
-
-
-serializeConstr :: ConstrRep -> [Word8] -> [Word8]
-serializeConstr (AlgConstr ix) = serializeWord8 1 . serializeInt ix
-serializeConstr (IntConstr i) = serializeWord8 2 . serializeInteger i
-serializeConstr (FloatConstr r) = serializeWord8 3 . serializeRational r
-serializeConstr (CharConstr c) = serializeWord8 4 . serializeChar c
-
-
-deserializeConstr :: [Word8] -> (ConstrRep -> [Word8] -> a) -> a
-deserializeConstr bytes k = deserializeWord8 bytes $ \constr_ix bytes ->
- case constr_ix of
- 1 -> deserializeInt bytes $ \ix -> k (AlgConstr ix)
- 2 -> deserializeInteger bytes $ \i -> k (IntConstr i)
- 3 -> deserializeRational bytes $ \r -> k (FloatConstr r)
- 4 -> deserializeChar bytes $ \c -> k (CharConstr c)
- x -> error $ "deserializeConstr: unrecognised serialized constructor type " ++ show x ++ " in context " ++ show bytes
-
-
-serializeFixedWidthNum :: forall a. (Integral a, FiniteBits a) => a -> [Word8] -> [Word8]
-serializeFixedWidthNum what = go (finiteBitSize what) what
- where
- go :: Int -> a -> [Word8] -> [Word8]
- go size current rest
- | size <= 0 = rest
- | otherwise = fromIntegral (current .&. 255) : go (size - 8) (current `shiftR` 8) rest
-
-deserializeFixedWidthNum :: forall a b. (Integral a, FiniteBits a) => [Word8] -> (a -> [Word8] -> b) -> b
-deserializeFixedWidthNum bytes k = go (finiteBitSize (undefined :: a)) bytes k
- where
- go :: Int -> [Word8] -> (a -> [Word8] -> b) -> b
- go size bytes k
- | size <= 0 = k 0 bytes
- | otherwise = case bytes of
- (byte:bytes) -> go (size - 8) bytes (\x -> k ((x `shiftL` 8) .|. fromIntegral byte))
- [] -> error "deserializeFixedWidthNum: unexpected end of stream"
-
-
-serializeEnum :: (Enum a) => a -> [Word8] -> [Word8]
-serializeEnum = serializeInt . fromEnum
-
-deserializeEnum :: Enum a => [Word8] -> (a -> [Word8] -> b) -> b
-deserializeEnum bytes k = deserializeInt bytes (k . toEnum)
-
-
-serializeWord8 :: Word8 -> [Word8] -> [Word8]
-serializeWord8 x = (x:)
-
-deserializeWord8 :: [Word8] -> (Word8 -> [Word8] -> a) -> a
-deserializeWord8 (byte:bytes) k = k byte bytes
-deserializeWord8 [] _ = error "deserializeWord8: unexpected end of stream"
-
-
-serializeInt :: Int -> [Word8] -> [Word8]
-serializeInt = serializeFixedWidthNum
-
-deserializeInt :: [Word8] -> (Int -> [Word8] -> a) -> a
-deserializeInt = deserializeFixedWidthNum
-
-
-serializeRational :: (Real a) => a -> [Word8] -> [Word8]
-serializeRational = serializeString . show . toRational
-
-deserializeRational :: (Fractional a) => [Word8] -> (a -> [Word8] -> b) -> b
-deserializeRational bytes k = deserializeString bytes (k . fromRational . read)
-
-
-serializeInteger :: Integer -> [Word8] -> [Word8]
-serializeInteger = serializeString . show
-
-deserializeInteger :: [Word8] -> (Integer -> [Word8] -> a) -> a
-deserializeInteger bytes k = deserializeString bytes (k . read)
-
-
-serializeChar :: Char -> [Word8] -> [Word8]
-serializeChar = serializeString . show
-
-deserializeChar :: [Word8] -> (Char -> [Word8] -> a) -> a
-deserializeChar bytes k = deserializeString bytes (k . read)
-
-
-serializeString :: String -> [Word8] -> [Word8]
-serializeString = serializeList serializeEnum
-
-deserializeString :: [Word8] -> (String -> [Word8] -> a) -> a
-deserializeString = deserializeList deserializeEnum
-
-
-serializeList :: (a -> [Word8] -> [Word8]) -> [a] -> [Word8] -> [Word8]
-serializeList serialize_element xs = serializeInt (length xs) . foldr (.) id (map serialize_element xs)
-
-deserializeList :: forall a b. (forall c. [Word8] -> (a -> [Word8] -> c) -> c)
- -> [Word8] -> ([a] -> [Word8] -> b) -> b
-deserializeList deserialize_element bytes k = deserializeInt bytes $ \len bytes -> go len bytes k
- where
- go :: Int -> [Word8] -> ([a] -> [Word8] -> b) -> b
- go len bytes k
- | len <= 0 = k [] bytes
- | otherwise = deserialize_element bytes (\elt bytes -> go (len - 1) bytes (k . (elt:)))
+deserializeWithData :: Data a => ShortByteString -> a
+deserializeWithData bs =
+ case runGetOrFail getData (ByteString.Lazy.fromStrict (Short.fromShort bs)) of
+ Right (_, _, x) -> x
+ Left (_, off, msg) ->
+ error ("deserializeWithData: " ++ msg ++ " at offset " ++ show off)
+
+putData :: Data a => a -> Put
+putData x = do
+ putConstrRep (constrRep (toConstr x))
+ sequence_ (gmapQ putData x)
+
+getData :: forall a. Data a => Get a
+getData = do
+ rep <- getConstrRep
+ fromConstrM getData (repConstr (dataTypeOf (undefined :: a)) rep)
+
+putConstrRep :: ConstrRep -> Put
+putConstrRep (AlgConstr ix) = putWord8 1 >> put ix
+putConstrRep (IntConstr i) = putWord8 2 >> put i
+putConstrRep (FloatConstr r) = putWord8 3 >> put r
+putConstrRep (CharConstr c) = putWord8 4 >> put c
+
+getConstrRep :: Get ConstrRep
+getConstrRep = do
+ tag <- getWord8
+ case tag of
+ 1 -> AlgConstr <$> get
+ 2 -> IntConstr <$> get
+ 3 -> FloatConstr <$> get
+ 4 -> CharConstr <$> get
+ _ -> fail ("unrecognised constructor tag " ++ show tag)
=====================================
libraries/ghc-internal/src/GHC/Internal/Event/TimerManager.hs
=====================================
@@ -13,8 +13,10 @@ import GHC.Internal.Types ()
{-# LANGUAGE CPP #-}
{-# LANGUAGE ExistentialQuantification #-}
{-# LANGUAGE FlexibleInstances #-}
+{-# LANGUAGE MagicHash #-}
{-# LANGUAGE NoImplicitPrelude #-}
{-# LANGUAGE Trustworthy #-}
+{-# LANGUAGE UnboxedTuples #-}
-- TODO: use the new Windows IO manager
module GHC.Internal.Event.TimerManager
@@ -53,6 +55,9 @@ import GHC.Internal.Control.Exception (finally)
import GHC.Internal.Data.Foldable (sequence_)
import GHC.Internal.Data.IORef (IORef, atomicModifyIORef', mkWeakIORef, newIORef, readIORef,
writeIORef)
+import GHC.Internal.IORef (IORef(..))
+import GHC.Internal.Prim (casMutVar#, readMutVar#)
+import GHC.Internal.STRef (STRef(..))
import GHC.Internal.Err (errorWithoutStackTrace)
import GHC.Internal.Clock (getMonotonicTimeNSec)
import GHC.Internal.Conc.Signal (runHandlers)
@@ -260,7 +265,7 @@ updateTimeout mgr (TK key) us = do
editTimeouts :: TimerManager -> TimeoutEdit -> IO ()
editTimeouts mgr g = do
- wake <- atomicModifyIORef' (emTimeouts mgr) f
+ wake <- modifyTimeouts (emTimeouts mgr) f
when wake (wakeManager mgr)
where
f q = (q', wake)
@@ -276,4 +281,75 @@ editTimeouts mgr g = do
t0 /= t1
_ -> True
+-- | Atomically replace the timeout queue with the first component of
+-- @f old@ and return the second component.
+--
+-- Unlike 'atomicModifyIORef'', the new queue is evaluated /before/ it is
+-- published, using a compare-and-swap loop. 'atomicModifyIORef'' installs
+-- the new contents as a thunk and only evaluates it afterwards, so when many
+-- threads register timeouts at once (#9809) each one builds its insert on
+-- top of the previous thread's unevaluated insert, giving long thunk chains,
+-- threads blocking on each other's half evaluated updates and lots of extra
+-- GC work. Since 'Q.IntPSQ' is spine strict, WHNF means fully evaluated, and
+-- other threads only ever see a fully evaluated queue.
+--
+-- The cost is that a thread whose compare-and-swap loses a race has to redo
+-- its O(log n) update. The update runs in constant stack (see #7653).
+--
+-- Only the short edits made by registering threads use this. The timer
+-- manager's own update in 'mkTimeout' (removing every expired timeout) can
+-- take much longer, and under a stream of registrations from other
+-- capabilities its compare-and-swap would almost never succeed, starving it
+-- so that expired timeouts don't fire. It therefore keeps using
+-- 'atomicModifyIORef'', which never retries: it publishes a single thunk and
+-- evaluates it straight away. A registering thread that reads that thunk
+-- evaluates it (or waits for the manager to) before publishing its own fully
+-- evaluated queue, so thunks still cannot pile up.
+--
+-- 'casMutVar#' compares pointers, so the expected value must be exactly the
+-- pointer read from the 'MutVar#'. See Note [Pointer equality in
+-- modifyTimeouts] for why this function is OPAQUE.
+{-# OPAQUE modifyTimeouts #-}
+modifyTimeouts :: IORef TimeoutQueue -> (TimeoutQueue -> (TimeoutQueue, b)) -> IO b
+modifyTimeouts (IORef (STRef ref)) f = IO modLoop
+ where
+ modLoop s0 = case readMutVar# ref s0 of
+ (# s1, oldVar #) -> case f oldVar of
+ (newVar, r) -> newVar `seq` case casMutVar# ref oldVar newVar s1 of
+ (# s2, 0#, _ #) -> (# s2, r #)
+ (# s2, _, _ #) -> modLoop s2
+
+{- Note [Pointer equality in modifyTimeouts]
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+'casMutVar#' succeeds only if the expected value is the very pointer stored
+in the 'MutVar#', so all the difficulties of 'reallyUnsafePtrEquality#'
+apply to it. In 'modifyTimeouts' the expected value is @oldVar@, exactly as
+'readMutVar#' returned it, and that must survive optimisation.
+
+The danger is the optimiser seeing a @case@ on @oldVar@ in the same function
+as the 'casMutVar#'. That would happen if 'modifyTimeouts' were inlined or
+specialised at a call site, bringing in the body of @f@ (in 'editTimeouts',
+@f@ inspects the queue to decide whether to wake the manager). The binder
+swap (Note [Binder swap] in GHC.Core.Opt.OccurAnal) may then replace @oldVar@
+by the case binder, which is the /evaluated/ queue. That is a different
+pointer whenever the 'MutVar#' holds a thunk or an indirection, for example
+the selector thunk that 'atomicModifyIORef'' in 'mkTimeout' leaves behind.
+Case-of-known-constructor can likewise replace it by a freshly built
+constructor. Either way the compare-and-swap would keep failing and the
+registering thread would spin.
+
+The OPAQUE pragma keeps @f@ an unknown function inside 'modifyTimeouts', so
+there is never a @case@ on @oldVar@ to rewrite. OPAQUE rather than NOINLINE:
+both stop inlining and specialisation, but NOINLINE still allows the
+worker/wrapper transformation. Today that is harmless (the worker only
+unboxes the 'IORef' and still takes @f@ as an argument), but OPAQUE states
+the requirement directly: the body must be compiled as written and every call
+must go to it. It also costs nothing, since 'editTimeouts' passes a closure
+anyway.
+
+An alternative is to hide the expected pointer in a box built by a NOINLINE
+function before @f@ runs (like the @Ticket@ type of the atomic-primops
+package), but that allocates on every attempt.
+-}
+
#endif
=====================================
libraries/ghci/GHCi/Message.hs
=====================================
@@ -297,7 +297,7 @@ data THMessage a where
ReifyInstances :: TH.Name -> [TH.Type] -> THMessage (THResult [TH.Dec])
ReifyRoles :: TH.Name -> THMessage (THResult [TH.Role])
ReifyAnnotations :: TH.AnnLookup -> TypeRep
- -> THMessage (THResult [ByteString])
+ -> THMessage (THResult [BS.ShortByteString])
ReifyModule :: TH.Module -> THMessage (THResult TH.ModuleInfo)
ReifyConStrictness :: TH.Name -> THMessage (THResult [TH.DecidedStrictness])
=====================================
libraries/ghci/GHCi/TH.hs
=====================================
@@ -100,7 +100,6 @@ import Control.Monad.IO.Class (MonadIO (..))
import Data.Binary
import Data.Binary.Put
import Data.ByteString (ByteString)
-import qualified Data.ByteString as B
import qualified Data.ByteString.Lazy as LB
import Data.Data
import Data.Dynamic
@@ -169,7 +168,7 @@ instance MonadIO GHCiQ where
-- serialize irrelevant annotations.
reifyAnnotations :: forall a . Data a => TH.AnnLookup -> GHCiQ [a]
reifyAnnotations lookup =
- map (deserializeWithData . B.unpack) <$>
+ map deserializeWithData <$>
ghcCmd (ReifyAnnotations lookup typerep)
where typerep = typeOf (undefined :: a)
=====================================
libraries/ghci/GHCi/TH/Binary.hs
=====================================
@@ -9,7 +9,6 @@ module GHCi.TH.Binary () where
import Prelude -- See note [Why do we import Prelude here?]
import Data.Binary
-import qualified Data.ByteString as B
import qualified Data.ByteString.Internal as B
import GHC.Serialized
import qualified GHC.Boot.TH.Syntax as TH
@@ -74,8 +73,8 @@ instance Binary TH.DocLoc
-- We need Binary TypeRep for serializing annotations
instance Binary Serialized where
- put (Serialized tyrep wds) = put tyrep >> put (B.pack wds)
- get = Serialized <$> get <*> (B.unpack <$> get)
+ put (Serialized tyrep wds) = put tyrep >> put wds
+ get = Serialized <$> get <*> get
instance Binary TH.Bytes where
put (TH.Bytes ptr off sz) = put bs
=====================================
testsuite/tests/annotations/should_run/Makefile
=====================================
@@ -9,3 +9,7 @@ config :
echo "module Config where" >>$(CONFIG_HS)
echo "cTop :: String" >> $(CONFIG_HS)
echo 'cTop = "$(subst \,\\,$(shell '$(TEST_HC)' --print-libdir))"' >> $(CONFIG_HS)
+
+T16841:
+ '$(TEST_HC)' $(TEST_HC_OPTS) -v0 -c T16841.hs
+ '$(TEST_HC)' $(TEST_HC_OPTS) --show-iface T16841.hi | grep 'annotated by'
=====================================
testsuite/tests/annotations/should_run/T16841.hs
=====================================
@@ -0,0 +1,15 @@
+-- | Measures the overhead of serializing annotations of a few types.
+module T16841 where
+
+foo :: Int
+foo = 42
+
+{-# ANN foo (42 :: Int) #-}
+{-# ANN foo (2^100 :: Integer) #-}
+{-# ANN foo (3.25 :: Double) #-}
+{-# ANN foo 'x' #-}
+{-# ANN foo "helloworld" #-}
+{-# ANN foo (Just True) #-}
+{-# ANN foo ([1..100] :: [Int]) #-}
+{-# ANN module (Left 7 :: Either Int String) #-}
+
=====================================
testsuite/tests/annotations/should_run/T16841.stdout
=====================================
@@ -0,0 +1,9 @@
+Module target T16841 annotated by 15 of type Either Int [Char]
+Named target foo annotated by 1509 of type [Int]
+Named target foo annotated by 18 of type Maybe Bool
+Named target foo annotated by 119 of type [Char]
+Named target foo annotated by 2 of type Char
+Named target foo annotated by 11 of type Double
+Named target foo annotated by 24 of type Integer
+Named target foo annotated by 6 of type Int
+
=====================================
testsuite/tests/annotations/should_run/all.T
=====================================
@@ -16,6 +16,10 @@ test('annrun01',
multimod_compile_and_run,
['annrun01', '-package ghc ' + config.ghc_th_way_flags])
+# Skipping on windows because of platform-specific failures. See
+# https://gitlab.haskell.org/ghc/ghc/-/merge_requests/16841#note_706770
+test('T16841', [req_th, windows_skip, extra_files(['T16841.hs'])], makefile_test, ['T16841'])
+
""""
Helpful things to C+P:
=====================================
testsuite/tests/codeGen/should_gen_asm/T24813.asm
=====================================
@@ -0,0 +1,10 @@
+popcnt8:
+ movzbl
+ popcnt
+ movq
+ jmp
+popcnt16:
+ movzwl
+ popcnt
+ movq
+ jmp
=====================================
testsuite/tests/codeGen/should_gen_asm/T24813.cmm
=====================================
@@ -0,0 +1,16 @@
+#include "Cmm.h"
+
+// Popcount of a zero-extended sub-word value should be a single
+// zero-extending load followed by one POPCNT (#24813).
+
+popcnt8 (W_ p) {
+ W_ r;
+ (r) = prim %popcnt8(%zx64(bits8[p]));
+ return (r);
+}
+
+popcnt16 (W_ p) {
+ W_ r;
+ (r) = prim %popcnt16(%zx64(bits16[p]));
+ return (r);
+}
=====================================
testsuite/tests/codeGen/should_gen_asm/all.T
=====================================
@@ -27,6 +27,7 @@ test('T25233', [unless(arch('x86_64'), skip),
when(unregisterised(), skip)], compile_grep_asm, ['hs', True, '-O'])
test('T25233b', [unless(arch('x86_64'), skip),
when(unregisterised(), skip)], compile_grep_asm, ['cmm', True, ''])
+test('T24813', is_amd64_codegen, compile_cmp_asm, ['cmm', '-msse4.2'])
is_aarch64_codegen = [
unless(arch('aarch64'), skip),
when(unregisterised(), skip),
=====================================
testsuite/tests/plugins/T26761Plugin.hs
=====================================
@@ -7,6 +7,7 @@ import GHC
import GHC.Plugins
import GHC.Tc.Types
import GHC.Tc.Utils.Monad
+import qualified Data.ByteString.Short as Short
import qualified Data.Map as Map
import Data.Word (Word8)
import System.IO (hFlush, stdout)
@@ -38,5 +39,5 @@ plugin = defaultPlugin { typeCheckResultAction = typecheckPlugin }
-- check that it is visible from other modules.
let ann = Annotation
(ModuleTarget $ tcg_mod gblEnv)
- (toSerialized id [0])
+ (toSerialized id (Short.pack [0]))
return gblEnv { tcg_anns = ann : tcg_anns gblEnv }
=====================================
testsuite/tests/plugins/all.T
=====================================
@@ -155,7 +155,7 @@ test('T14335',
-fexternal-interpreter -package simple-plugin ' + config.plugin_way_flags])
test('T26761', [extra_files(['T26761Plugin.hs', 'T26761a.hs'])],
- multimod_compile, ['T26761', '-package ghc ' + config.plugin_way_flags ])
+ multimod_compile, ['T26761', '-package ghc -package bytestring ' + config.plugin_way_flags ])
test('plugin-recomp-pure',
[extra_files(['plugin-recomp/', 'plugin-recomp-test.hs']),
=====================================
testsuite/tests/rts/pause-resume/all.T
=====================================
@@ -14,6 +14,8 @@ test('pause_resume_via_safe_ffi_concurrent',
[ only_ways(['threaded1', 'threaded2'])
, req_c
, extra_files(['pause_resume.c','pause_resume.h'])
+ # Internal watchdog is 60s; give the harness room on loaded Darwin CI.
+ , run_timeout_multiplier(2)
],
multi_compile_and_run, ['pause_resume_via_safe_ffi_concurrent', [('pause_resume.c','')], ''])
test('pause_and_use_rts_api',
=====================================
testsuite/tests/rts/pause-resume/pause_resume_via_safe_ffi_concurrent.hs
=====================================
@@ -8,6 +8,7 @@ import Foreign.Marshal.Alloc
import Foreign.Ptr
import Foreign.Storable
import System.Exit
+import System.IO (hFlush, stdout)
import System.Timeout
foreign import ccall safe "pause_resume.h pauseAndResume"
@@ -38,11 +39,17 @@ main = do
countPtr
putMVar mvar ()
- -- Wait (at least 2n seconds to be safe) for all threads to finish.
- result <- timeout (2 * n * 1000000) (mapM_ takeMVar mvars)
+ -- Each pause holds the RTS for about a second, so n concurrent
+ -- calls serialize to ~n seconds. 2n (10s) was enough on Linux but
+ -- a false deadlock on Darwin CI: rts_pause waits for every
+ -- capability, and the counter threads keep hitting the timer
+ -- manager via threadDelay. Keep a watchdog so a real hang still
+ -- fails, but give 60s of slack.
+ result <- timeout (60 * 1000000) (mapM_ takeMVar mvars)
case result of
Nothing -> do
putStrLn "Not all rts_pause/rts_resume threads have finished. Assuming deadlocked and failing test."
+ hFlush stdout
exitFailure
Just () -> do
putStrLn "All threads finished"
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/cb573bc9cae5f7b2618ac8f0f310da…
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/cb573bc9cae5f7b2618ac8f0f310da…
You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help
1
0
[Git][ghc/ghc][wip/T27737] 2 commits: Simplifier: fix in-scope-set leak from lambda RHSs; drop LetFloats synonym
by Simon Peyton Jones (@simonpj) 28 Oct '26
by Simon Peyton Jones (@simonpj) 28 Oct '26
28 Oct '26
Simon Peyton Jones pushed to branch wip/T27737 at Glasgow Haskell Compiler / GHC
Commits:
4ae4a61b by Simon Peyton Jones at 2026-10-28T10:28:12+00:00
Simplifier: fix in-scope-set leak from lambda RHSs; drop LetFloats synonym
* simpl_lam rebuilt with env', whose in-scope set contains the lambda
binders, so the (usually empty) floats it returned carried an
over-approximate in-scope set. Now it rebuilds with env.
* doFloatFromRhs returns Nothing when there is nothing to float, as it
did before the recent refactoring, so prepareBinding returns
(emptyFloats env) rather than the RHS's own empty floats. Without
this, simplLazyBind set the in-scope set for completeBind from those
floats; re-simplifying an INLINE stable unfolding such as
happyIn4 = /\t4..t104. \x -> ... (T14683, Happy-generated) then found
all its type variables already in scope, renamed them all, and did a
non-empty type substitution over the whole unfolding. Simplifier
allocation on T14683 was up 61%; the leaked binders also inflated the
in-scope set for the rest of the module, costing 1-4% on many tests
(T9233, T13056, T5642, T16577, T9630, T12227, T15703, ...). All gone.
* abstractFloats: the substitution's in-scope set must include the
type variables being abstracted (they are free in the floats' RHSs);
the returned in-scope set is env's extended with the new polymorphic
bindings only. Fixes an in-scope assertion failure building
haddock-library.
* Replace the LetFloats type synonym by OrdList OutBind; remove some
commented-out traces in SimpleOpt.
Co-Authored-By: Claude Fable 5.1 <noreply(a)anthropic.com>
- - - - -
1db13ad7 by Simon Peyton Jones at 2026-10-28T10:28:49+00:00
Fix two typos in a comment
- - - - -
4 changed files:
- compiler/GHC/Core/Opt/Simplify/Env.hs
- compiler/GHC/Core/Opt/Simplify/Iteration.hs
- compiler/GHC/Core/Opt/Simplify/Utils.hs
- compiler/GHC/Core/SimpleOpt.hs
Changes:
=====================================
compiler/GHC/Core/Opt/Simplify/Env.hs
=====================================
@@ -43,8 +43,7 @@ module GHC.Core.Opt.Simplify.Env (
extendFloats, isEmptyJoinFloats,
doFloatFromRhs, getTopFloatBinds,
- -- * LetFloats
- LetFloats, FloatEnable(..), emptyLetFloats, mapLetFloats,
+ FloatEnable(..), mapLetFloats,
-- * JoinFloats
JoinFloat, JoinFloats, emptyJoinFloats,
@@ -452,8 +451,8 @@ isEmptyFloats :: SimplFloats -> Bool
-- Precondition: used only when sfJoins is empty
isEmptyFloats (SimplFloats { sfBinds = fs
, sfJoins = js })
- = assertPpr (isNilOL js) (ppr js ) $
- isNilOL fs
+ = assertPpr (isEmptyJoinFloats js) (ppr js ) $
+ isEmptyFloatBinds fs
pprSimplEnv :: SimplEnv -> SDoc
-- Used for debugging; selective
@@ -738,7 +737,7 @@ setSubstEnv env tvs cvs ids = env { seTvSubst = tvs, seCvSubst = cvs, seIdSubst
{-
************************************************************************
* *
-\subsection{LetFloats}
+\subsection{Floats}
* *
************************************************************************
@@ -891,15 +890,13 @@ let-bindings that can, and wrap the rest back around the RHS. We tried this
difference to compile time, so we did not keep it.
-}
-type LetFloats = OrdList OutBind
- -- Let-bindings only: no case-floats or tick-floats
- -- Invariant: every binding satisfies the let-can-float invariant
-
type JoinFloat = OutBind
type JoinFloats = OrdList JoinFloat
doFloatFromRhs :: SimplEnv -> TopLevelFlag -> RecFlag
- -> SimplFloats -> LetFloats
+ -> SimplFloats
+ -> OrdList OutBind -- Let-bindings only (from prepareRhs); they
+ -- satisfy the let-can-float invariant
-> OutExpr
-> Maybe SimplFloats -- Contains let-bindings only
-- See Note [doFloatFromRhs]
@@ -908,10 +905,11 @@ doFloatFromRhs :: SimplEnv -> TopLevelFlag -> RecFlag
--
-- If you change this function look also at FloatIn.noFloatIntoRhs
doFloatFromRhs env lvl rec floats anf_lets rhs
- | not float_enabled = Nothing
- | not want_to_float = Nothing
- | not can_float = Nothing
- | otherwise = Just new_floats
+ | isNilOL fs, isNilOL anf_lets = Nothing -- Short cut only
+ | not float_enabled = Nothing
+ | not want_to_float = Nothing
+ | not can_float = Nothing
+ | otherwise = Just new_floats
where
!(SimplFloats { sfBinds = fs, sfJoins = jfs, sfInScope = in_scope }) = floats
new_floats = assertPpr (isNilOL jfs) (ppr jfs) $
@@ -952,9 +950,6 @@ doFloatFromRhs env lvl rec floats anf_lets rhs
FloatNestedOnly -> not (isTopLevel lvl)
FloatEnabled -> True
-emptyLetFloats :: LetFloats
-emptyLetFloats = nilOL
-
emptyJoinFloats :: JoinFloats
emptyJoinFloats = nilOL
@@ -1023,11 +1018,11 @@ extendFloats floats@(SimplFloats { sfJoins = jfloats, sfInScope = in_scope }) bi
| otherwise
= floats `addLetFloat` bind
-addLetFloats :: SimplFloats -> LetFloats -> SimplFloats
+addLetFloats :: SimplFloats -> OrdList OutBind -> SimplFloats
-- Add the let-bindings /inside/ the existing floats,
-- and extend the in-scope set
--- No need for addLetFloat's needsCaseBinding test: LetFloats satisfy the
--- let-can-float invariant (see the declaration of LetFloats)
+-- Precondition: the bindings satisfy the let-can-float invariant,
+-- so there is no need for addLetFloat's needsCaseBinding test
addLetFloats floats binds = foldlOL add floats binds
where
add fs bind = assertPpr (okLetFloat bind) (ppr bind) $
@@ -1091,7 +1086,7 @@ addJoinFlts = appOL
mkRecFloats :: SimplFloats -> SimplFloats
-- Flattens the floats into a single Rec group,
--- They must either all be lifted LetFloats or all JoinFloats
+-- They must either all be lifted let-floats or all JoinFloats
mkRecFloats floats@(SimplFloats { sfBinds = bs
, sfJoins = jbs
, sfInScope = in_scope })
@@ -1125,7 +1120,7 @@ getTopFloatBinds floats@(SimplFloats { sfJoins = jbs })
letFloatBinds floats
{-# INLINE mapLetFloats #-}
-mapLetFloats :: LetFloats -> ((Id,CoreExpr) -> (Id,CoreExpr)) -> LetFloats
+mapLetFloats :: OrdList OutBind -> ((Id,CoreExpr) -> (Id,CoreExpr)) -> OrdList OutBind
mapLetFloats binds fun
= binds1
where
=====================================
compiler/GHC/Core/Opt/Simplify/Iteration.hs
=====================================
@@ -64,7 +64,7 @@ import GHC.Builtin.WiredIn.Ids( seqId )
import qualified GHC.Data.List.Infinite as Inf
import GHC.Data.Maybe ( isNothing, orElse, mapMaybe )
import GHC.Data.FastString
-import GHC.Data.OrdList ( unitOL, isNilOL, appOL, snocOL )
+import GHC.Data.OrdList ( OrdList, nilOL, unitOL, isNilOL, appOL, snocOL )
import GHC.Unit.Module ( moduleName )
import GHC.Utils.Outputable
import GHC.Utils.Panic
@@ -764,7 +764,7 @@ prepareRhs :: HasDebugCallStack
=> SimplEnvIS -> TopLevelFlag
-> FastString -- Base for any new variables
-> OutExpr
- -> SimplM (LetFloats, OutExpr)
+ -> SimplM (OrdList OutBind, OutExpr)
-- Transforms a RHS into a better RHS by ANF'ing args
-- for expandable RHSs: constructors and PAPs
-- e.g x = Just e
@@ -773,7 +773,7 @@ prepareRhs :: HasDebugCallStack
-- See Note [prepareRhs]
prepareRhs env top_lvl occ rhs0
| is_expandable = anfise rhs0
- | otherwise = return (emptyLetFloats, rhs0)
+ | otherwise = return (nilOL, rhs0)
where
-- We can't use exprIsExpandable because the WHOLE POINT is that
-- we want to treat (K <big>) as expandable, because we are just
@@ -789,7 +789,7 @@ prepareRhs env top_lvl occ rhs0
go (Tick _ rhs) n_val_args = go rhs n_val_args
go _ _ = False
- anfise :: OutExpr -> SimplM (LetFloats, OutExpr)
+ anfise :: OutExpr -> SimplM (OrdList OutBind, OutExpr)
anfise (Cast rhs co)
= do { (floats, rhs') <- anfise rhs
; return (floats, Cast rhs' co) }
@@ -801,7 +801,7 @@ prepareRhs env top_lvl occ rhs0
; (floats2, arg') <- makeTrivial env top_lvl topDmd occ arg
; return (floats1 `appOL` floats2, App fun' arg') }
anfise (Var fun)
- = return (emptyLetFloats, Var fun)
+ = return (nilOL, Var fun)
anfise (Tick t rhs)
-- We want to be able to float bindings past this tick.
@@ -819,28 +819,28 @@ prepareRhs env top_lvl occ rhs0
floats' = mapLetFloats floats tickIt
; return (floats', Tick t rhs') }
- anfise other = return (emptyLetFloats, other)
+ anfise other = return (nilOL, other)
makeTrivialArg :: HasDebugCallStack => SimplEnvIS -> ArgSpec
- -> SimplM (LetFloats, ArgSpec)
+ -> SimplM (OrdList OutBind, ArgSpec)
makeTrivialArg env arg@(ValArg { as_arg = e, as_dmd = dmd })
= do { (floats, e') <- makeTrivial env NotTopLevel dmd (fsLit "arg") e
; return (floats, arg { as_arg = e' }) }
makeTrivialArg _ arg@(TyArg {})
- = return (emptyLetFloats, arg)
+ = return (nilOL, arg)
makeTrivial :: HasDebugCallStack
=> SimplEnvIS -> TopLevelFlag -> Demand
-> FastString -- ^ A "friendly name" to build the new binder from
-> OutExpr
- -> SimplM (LetFloats, OutExpr)
+ -> SimplM (OrdList OutBind, OutExpr)
-- Binds the expression to a variable, if it's not trivial, returning the variable
-- For the Demand argument, see Note [Keeping demand info in StrictArg Plan A]
makeTrivial env top_lvl dmd occ_fs expr
| exprIsTrivial expr -- Already trivial
|| not (bindingOk top_lvl expr expr_ty) -- Cannot trivialise
-- See Note [Cannot trivialise]
- = return (emptyLetFloats, expr)
+ = return (nilOL, expr)
| Cast expr' co <- expr
= do { (floats, triv_expr) <- makeTrivial env top_lvl dmd occ_fs expr'
@@ -889,7 +889,7 @@ into
and thence, by floating, into
z = f y
x = Just z
-so that x is bound to a value. It returns only let-bindings (LetFloats), so
+so that x is bound to a value. It returns only let-bindings, so
sometimes it cannot trivialise its argument, and must return it unchanged.
(CT1) Nested bindings. Suppose that (f y) had type Int#, with a different data
@@ -2015,7 +2015,13 @@ simpl_lam env bndr body cont
; (env', bndrs') <- simplLamBndrs env (bndr:inner_bndrs)
; body' <- simplExpr env' inner_body
; new_lam <- rebuildLam env' bndrs' body' cont
- ; rebuild env' new_lam cont }
+ ; rebuild env new_lam cont }
+ -- Subtle point: rebuild with env, not env': the lambda binders are not in
+ -- scope outside the lambda, and should not end up in the in-scope set of the
+ -- floats we return. Why not? Because a too-large in-scope set can lead to
+ -- gratuitous renaming, and a non-empty substitution makes `substTy` do real
+ -- work. This happens, for example, if the function has a stable unfolding
+ -- that uses the same binders, and cost 9% in compile time in T14683.
-------------
simplLamBndr :: SimplEnv -> InBndr -> SimplM (SimplEnv, OutBndr)
=====================================
compiler/GHC/Core/Opt/Simplify/Utils.hs
=====================================
@@ -74,6 +74,7 @@ import GHC.Types.InlinePragma
import GHC.Types.Tickish
import GHC.Types.Demand
import GHC.Types.Var.Set
+import GHC.Types.Var.Env ( extendInScopeSetList )
import GHC.Types.Basic
import GHC.Types.Name.Env
@@ -2518,7 +2519,8 @@ new binding is abstracted. Several points worth noting
manifestly a literal string.
-}
-abstractFloats :: SimplEnv -> TopLevelFlag -> [OutTyVar]
+abstractFloats :: SimplEnv -- Env outside the whole binding
+ -> TopLevelFlag -> [OutTyVar]
-> SimplFloats -- Let-bindings only: postcondition of `prepareBinding`
-> OutExpr -- The right-hand side of the binding
-> SimplM (SimplFloats, OutExpr)
@@ -2535,16 +2537,19 @@ abstractFloats env top_lvl main_tvs floats rhs
; return ( SimplFloats { sfBinds = foldr (consOL . FloatLet) nilOL poly_binds
, sfJoins = emptyJoinFloats
- , sfInScope = substInScopeSet subst }
+ , sfInScope = foldl' extendInScopeSetBind in_scope poly_binds }
, GHC.Core.Subst.substExpr subst rhs) }
- -- NB: the returned in-scope set includes `env` and all the
+ -- NB: the returned in-scope set is the outer `in_scope` and all the
-- new floated bindings; but not main_tvs, nor the original floats
- -- The caller will used the returned in-scope set to cover the body of the let,
- -- and main_tvs and original floats are not in scope there
+ -- The caller will use the returned in-scope set to cover the body of the let,
+ -- and `main_tvs` and original floats are not in scope there
-- That reduces unnecessary renaming (see !7777 and test T16577)
where
!(SimplFloats { sfBinds = float_binds, sfJoins = jfs }) = floats
- empty_subst = GHC.Core.Subst.mkEmptySubst (seInScope env)
+ in_scope = seInScope env
+ empty_subst = GHC.Core.Subst.mkEmptySubst (in_scope `extendInScopeSetList` main_tvs)
+ -- main_tvs: the RHSs of the floats mention the main_tvs, so they must be
+ -- in scope for substExpr (the original float binders are in the substitution)
is_top_lvl = isTopLevel top_lvl
uf_opts = seUnfoldingOpts env
=====================================
compiler/GHC/Core/SimpleOpt.hs
=====================================
@@ -1712,8 +1712,7 @@ exprIsConApp_maybe ise@(ISE in_scope id_unf) expr
subst_co (Right s) co = GHC.Core.Subst.substCo s co
subst_expr (Left {}) e = e
- subst_expr (Right s) e = -- pprTrace "subst_expr" (text "subst" <+> ppr s $$ text "expr" <+> ppr e) $
- substExpr s e
+ subst_expr (Right s) e = substExpr s e
subst_bndrs subst bs = mapAccumL subst_bndr subst bs
@@ -1723,8 +1722,7 @@ exprIsConApp_maybe ise@(ISE in_scope id_unf) expr
subst_bndr (Right subst) bndr = subst_bndr1 subst bndr
subst_bndr1 subst bndr
- = -- pprTrace "subst_bndr1" (ppr bndr <+> ppr bndr' $$ ppr subst) $
- (Right subst', bndr')
+ = (Right subst', bndr')
where
(subst', bndr') = substBndr subst bndr
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/cb78c1aa05e6edf786113b0f3e52fa…
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/cb78c1aa05e6edf786113b0f3e52fa…
You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help
1
0
[Git][ghc/ghc][wip/T27994] 35 commits: Fix a few tag inference bugs.
by Simon Peyton Jones (@simonpj) 28 Oct '26
by Simon Peyton Jones (@simonpj) 28 Oct '26
28 Oct '26
Simon Peyton Jones pushed to branch wip/T27994 at Glasgow Haskell Compiler / GHC
Commits:
6bbca569 by Andreas Klebinger at 2026-10-21T10:01:10+00:00
Fix a few tag inference bugs.
* Treat bottoming functions and nullary join points as functions.
* When generating bytecode don't rely on imported tag sigs.
Fixes #27831, #27841 and #27842.
- - - - -
71808e29 by Sylvain Henry at 2026-10-22T04:50:24-04:00
Add GHC.Internal.Unboxed.Do: do-notation for unboxed values (#27944)
QualifiedDo operators can be representation-polymorphic: a do-block is
expanded into ordinary applications before typechecking, and the only
check on a bound variable is the deferred fixed-runtime-rep check. A
representation-polymorphic (>>=) cannot be defined directly, though,
because its body would bind a variable of unknown representation.
The new module provides such operators for State#-passing code as
methods of a class indexed by RuntimeRep, with one instance per
representation. With optimisation the dictionaries disappear and a
block compiles to the usual nested case expressions:
Ubx.do { x <- readWordArray# mba 0# -- x :: Word#
; writeWordArray# mba 1# x -- unit actions are statements
; Ubx.pure x }
Also add users guide documentation and tests.
Co-Authored-By: Claude Fable 5.1 <noreply(a)anthropic.com>
- - - - -
33247780 by Sylvain Henry at 2026-10-22T04:50:24-04:00
Bignum: use Ubx.do for State#-passing code (#27944)
Rewrite the State#-passing code of the GHC.Internal.Bignum modules
(Primitives, WordArray, BigNat, Natural, Integer, Backend.Native,
Backend.GMP) with the qualified do-notation of GHC.Internal.Unboxed.Do.
Nested case chains such as
case newWordArray# sz s of { (# s, mwa #) ->
case act mwa s of { s ->
unsafeFreezeByteArray# mwa s }}
become
Ubx.do
mwa <- newWordArray# sz
act mwa
unsafeFreezeByteArray# mwa
The optimised Core is unchanged (compared per top-level binding).
BigNat now enables MonoLocalBinds: local loops written with Ubx.do
must stay monomorphic in the state type, otherwise GHC does not turn
them into join points.
Co-Authored-By: Claude Fable 5.1 <noreply(a)anthropic.com>
- - - - -
593f5f5b by Sylvain Henry at 2026-10-22T04:51:16-04:00
JS: remove archive padding after wasm objects (#25665)
On Darwin hosts, emar (llvm-ar) writes archives in the Darwin format when
the first member is not a known object file, which is the case for the
archives of the JS backend. This format pads each member to 8 bytes with
'\n' and includes the padding in the member size. Wasm objects do not
accept trailing bytes: wasm-ld failed with "section too large".
The JS linker now removes this padding when it extracts a wasm object from
an archive. In addition, Hadrian passes --format=gnu to emar for the JS
target. See Note [Wasm objects in Darwin archives].
Also document the format of Cc (wasm) objects in Note [Cc object format].
Co-Authored-By: Claude Opus 5.5 <noreply(a)anthropic.com>
- - - - -
13dff5d0 by L0neGamer at 2026-10-22T21:37:31-04:00
add options for resizing bytearrays (#27244)
add five mutable bytearray resizing operations
for various kinds of pinnedness and aligning.
- prefer unpinned when allocating new array (``resizeMutableByteArrayPreferUnpinned#``; the ``resizeMutableByteArray#`` default)
- prefer pinned when allocating new array (``resizeMutableByteArrayPreferPinned#``)
- prefer aligned and pinned when allocating new array (``resizeMutableByteArrayPreferAlignedPinned#``)
- prefer same as input bytearray when allocating new array (``resizeMutableByteArrayKeepPinnedness#``)
- ensure the output array is aligned and pinned after operation (``resizeMutableByteArrayMakeAlignedPinned#``)
- - - - -
0c17201c by Sylvain Henry at 2026-10-22T21:38:23-04:00
JS: implement performMinorGC and performBlockingMajorGC (#28012)
GHC.Internal.System.Mem imports three C entry points: performMajorGC,
performGC (for performMinorGC) and performBlockingMajorGC. The JS RTS
only defined h$performMajorGC, so calling performMinorGC or
performBlockingMajorGC threw "ReferenceError: h$performGC is not
defined" at runtime.
The JS RTS has a single non-generational collector, so both missing
entry points are now aliases of h$performMajorGC.
Add a test that calls every GC entry point of System.Mem.
Co-Authored-By: Claude Fable 5.1 <noreply(a)anthropic.com>
- - - - -
7a631ff2 by L0neGamer at 2026-10-23T09:11:39-04:00
Mon(a|oi)d of No (Return|Mappend) progress (!15795)
Make non-canonical definitions of return and mappend
non-suppressible errors, and make canonical definitions
of those methods warnings.
CLC proposal: https://github.com/haskell/core-libraries-committee/issues/328
- - - - -
14656fd4 by Simon Jakobi at 2026-10-23T09:12:27-04:00
Cmm: keep the CBE substitution idempotent
When blocks are merged over multiple rounds during common block
elimination, a block that won a merge in an earlier round can still
lose a merge in a later round. The substitution map (Subst) used to
record this as a chain: the earlier loser mapped to a block that was
itself eliminated. In PLT terminology, the substitution became
*non-idempotent*. Applying such a substitution with a single lookup
left edges to unreachable blocks in the Cmm graph, eventually leading
to a setInfoTableStackMap panic (#27368).
With this patch, each round's fresh substitutions are applied to the old
ones as they are added (extendSubst), so every eliminated block maps
directly to its final replacement and the substitution stays idempotent.
Fixes #27368.
Assisted-by: Claude Fable 5.1
- - - - -
92848e61 by Evgeny Malyshev at 2026-10-23T20:26:04-04:00
hadrian: Include transitive dependencies in Haddock builds
Haddock needs indirect dependency interfaces to resolve references to
definitions outside a package's direct dependencies. Expose Cabal's
configured dependency closure through ContextData and use it alongside
the direct list, preserving direct-interface precedence.
Bump Shake's cache version for the serialized oracle change. Add a
Hadrian selftest requiring template-haskell's indirect ghc-prim interface.
Fixes #25850.
Fixes #27786.
Assisted-By: OpenAI Codex
Metric Increase:
haddock.Cabal
- - - - -
5e8ae81b by Zubin Duggal at 2026-10-23T20:26:47-04:00
WinIO: Don't confuse Win32 errors and NTSTATUS codes
WinIO misinterpreted NTSTATUS codes as Win32 errors, and had some ad hoc logic to deal with this.
However, this logic was incomplete and resulted in the mishandling of results of certain I/O operations
(see testcase).
Now the `ErrCode` argument in `CompletionCallback` is always the Win32 error code,
and we convert the NTSTATUS using RtlNtStatusToDosError.
Also when an overlapped/async operation such as ReadFile fails immediately,
with a Win32 error such as ERROR_BROKEN_PIPE, we don't get a further completion
notification, so we never recorded the failure. This worked by coincidence
earlier, because `STATUS_PENDING` happened to have the same numerical code as
`ERROR_NO_MORE_ITEMS`, but it doesn't now that we translate `STATUS_PENDING`
into the correct Win32 error code (`ERROR_IO_PENDING`). Fix this by using
the Win32 error value we obtained when an operation returns `STATUS_PENDING`.
Fixes #27971
- - - - -
c42ed66a by Martijn Bastiaan at 2026-10-24T16:00:42+02:00
Add single-threaded test for #27526 + #27123
Issue #27526 is a duplicate of #27123, which has already been fixed in
HEAD. The proposed patch in the former carries a "nicer" regression test
in that it doesn't rely on multi-threading to trigger.
Closes #27526
- - - - -
7f73f515 by Simon Peyton Jones at 2026-10-24T17:29:19-04:00
Refactor QuickLook a bit
This MR simply refactors QuickLook a bit. In particular, it
simplifies `tcApp`, and moves shared code into `finishApp`.
It also adds `Note [QuickLook: arguments before result]`
responding to #26599.
There should be no user-visible changes, except perhaps in error messages,
which improve marginally.
For some reason T24984 compiles 8% faster. I'll take the win!
Metric Decrease:
T24984
- - - - -
a4a3a964 by Alan Zimmerman at 2026-10-24T17:30:03-04:00
EPA: Use AnnList for XClosedTypeFamily
We now have TTG annotations for FamilyInfo.
Make use of it to store the AnnList annotations for a closed type
family.
- - - - -
953e4c7f by sheaf at 2026-10-26T18:19:10-04:00
Rework LogQueue to be a proper queue
Now 'LogQueue' uses a 'TQueue' with a separate 'TVar' to indicate
whether the log queue is finished logging, instead of the previous weird
data representation using a reverse-order '[Maybe Msg]' with 'Nothing'
denoting the end of the queue.
- - - - -
ec29c5fa by Rodrigo Mesquita at 2026-10-26T18:19:10-04:00
Refactor GHC.Driver.MakeAction
Pull out of 'runParPipelines' the logic for creating a 'MakeEnv' ready
to be used by multiple threads, as that will be useful for downsweep as
well (which doesn't fit the 'runPipelines' flow), rather than being just
for upsweep.
The code is moved and re-structured to match the export list, simplify
the sequentiality checks previously both in 'runPipelines' and
'runAllPipelines', which were weirdly similar and confusing; into the
two part step where we need these checks: (1) to construct the MakeEnv,
(2) to run the MakeActions in parallel. These two steps are separate and
used to be too mixed up.
Some additional little simplifications or clean ups here and there.
- - - - -
3fb22073 by Rodrigo Mesquita at 2026-10-26T18:19:11-04:00
Refactor GHC.Driver.Downsweep in preparation for parallel downsweep
Pure refactor to improve the code to facilitate implementing parallel
downsweep in the next commit.
This commit puts a MakeEnv into the DownsweepEnv, gives the fields
proper names and uses RecordWildcards to simplify, rather than passing
around all diagnostic wrappers, driver-message-things and using 10s of
positional fields.
No behavior changes here!
- - - - -
52f385a7 by sheaf at 2026-10-26T18:19:11-04:00
Driver: enableCodeGenForTH in load', not downsweep
enableCodeGenForTH modifies the module graph for planning, creating
fresh temporary paths. So it should only be run when we are about to
execute a build plan (e.g. in load'), so that consumers that want to
know facts about the module graph don't go looking for files that don't
exist yet (in particular 'haddock --no-compilation').
- - - - -
82402d14 by sheaf at 2026-10-26T18:19:11-04:00
2-phase Cache/Search Finder monad
This commit restructures the finder abstraction by introducing the
'FinderM' monad, which splits module lookup operations into two phases:
- a cache-only phase, performing no filesystem access,
- from the first cache miss onwards, a search action which may access
the filesystem.
This allows consumers to distinguish between quick cached results versus
more expensive filesystem search operations.
- - - - -
5c302b40 by sheaf at 2026-10-26T18:19:11-04:00
Separate Home/External finder caches
The caches for home modules and modules from external units have
different lifetimes, needing to be invalidated at different points.
This commit moves the cache for modules from external units into the
'UnitState' type, which keeps the cache correct whenever package flags
or package databases change (setSessionDynFlags, :set -package ...,
Backpack's addUnit).
This also makes it easier to clear the cache of home modules, as they
are now stored separately (obviating the 'isUnitEnvInstalledModule'
check we used to perform).
- - - - -
605fbb96 by sheaf at 2026-10-26T18:19:11-04:00
Add known home modules to the finder cache
The finder looks in the file system for the on-disk source that
corresponds to a Haskell module name. There are two kinds of modules
where we need to bypass this search:
- a file passed directly in the command line, which may not be in any
search path and whose module name may differ from the file name;
- Backpack signatures and modules, which do not exist as files at all.
This commit handles these two situations by adding an immutable
'KnownHomeModules' in the finder cache, which is always consulted first
before doing any search.
This removes the ad-hoc imperative logic which relied on mutating the
finder cache at careful points.
'setModuleGraph' now also takes the known home modules, so that the
module graph and the known home modules are always set consistently.
See Note [Known home modules] in GHC.Unit.Finder.Types.
- - - - -
82718f44 by sheaf at 2026-10-26T18:19:11-04:00
Make 'UnresolvedImport' into a suitable cache key
The goal of this commit is to make 'UnresolvedImport' suitable for use
as a cache key. Specifically, during downsweep we want to be able to
cache module resolution queries. We want to do this using some kind of
map, e.g. (roughly speaking):
Map UnresolvedImport ResolvedImport
For this to work, 'UnresolvedImport' must contain **precisely** the
input data to each resolution computation. In other words:
1. If something changes that materially affects resolution, it must
be stored in the cache key.
2. If changing some input does not materially affect the outcome,
then it must not be stored in the cache key.
To satisfy (1), we change 'UnitNode' to also keep track of the home
unit in whose unit state the unit's dependencies were looked up.
For (2), 'ImportResolution' now stores a 'ModuleLookupScope' instead of an
'UnresolvedImportOrigin'. Plugin imports are expressed with a new
'LookupPlugin' scope. The source location of an import, which is
immaterial to the resolution, is stored out of band outside
'UnresolvedImport', in 'ModSummary'.
- - - - -
d4635c7c by sheaf at 2026-10-26T18:19:11-04:00
Add GHC.Data.Dependent
This commit adds a minimal implementation of dependent sums
('Some', 'DSum') and dependent maps ('DMap').
The implementation is not entirely performance optimal, as there are a
few extraneous allocations compared to the full-blown implementation in
the 'dependent-map' package. For our use case (caching downsweep
computations), this does not matter much.
See GHC.Data.Dependent.
- - - - -
6759533e by sheaf at 2026-10-26T18:19:11-04:00
Driver: structured concurrent worker abstraction
This commit introduces a structured concurrency framework in the style
of the 'ki' library: a collection of threads within a scope.
We implement two kinds of concurrent workers on top of this framework:
- Independent workers cannot wait for one another at all. The only
scheduling operation is to wait for quiescence.
- Coordinating workers declare an STM readiness condition (waiting on
other workers to complete) which gates their start.
See Note [Deterministic concurrent workers] in GHC.Driver.Concurrency.
This commit ports upsweep to this new framework, with downsweep being
left as subsequent work.
Further changes along the way:
- Refactoring of how concurrency is acquired to avoid the footgun of
trying to use a no-op 'AbstractSem' as a lock in the serial case.
- The "re-run with -j1" logic for semaphore opening errors no longer
triggers on late semaphore failures (part-way through a lengthy
computation).
- Logger threads are properly cleaned up on exception, with each
concurrent worker's log queue and local TmpFs properly bracketed.
- The 'GhcMessage -> AnyGhcDiagnostic' and 'Maybe Messager'
arguments of 'depanalE', 'depanalPartial' and 'downsweep', which
were all dead in practice, have been dropped.
- Concurrent logging has been completely overhauled. Each worker has
its own log queue. Workers whose logs are printed as we go are
spawned in the order in which their logs must be printed, now using
a TQueue instead of the old complicated LogQueueQueue mechanism.
'logThread' is replaced by 'withLogPrinter', which runs the thread
that prints the log queues within a concurrent worker scope.
These changes allow us to fix issues with the exception handling:
we now always flush remaining log queues before exiting (even when
the action throws an exception), and exceptions that arise while
printing are properly rethrown. This fixes #28009.
- - - - -
38a99006 by sheaf at 2026-10-26T18:19:11-04:00
Rule-based deterministic concurrent downsweep
This commit rewrites downsweep as a single query-answering rule
(see 'DownsweepRule') that can be executed by concurrent worker threads.
The design allows every expensive operation (preprocessing files with CPP,
parsing headers, reading interfaces) to be performed concurrently
according to the -j<N>/-jsem flags.
See Note [Rules-based downsweep] in GHC.Driver.Downsweep.
The rules are run by "GHC.Driver.Concurrency.runRules" in a demand-driven
way: each worker can demand other workers to run, but never wait on
another worker. Every piece of work is done at most once.
Fixes #27514
Memory usage can increase due to doing more work concurrently:
-------------------------
Metric Increase:
MultiComponentModulesRecomp
MultiComponentModulesRecomp100
mhu-perf
-------------------------
- - - - -
9fa7b7be by sheaf at 2026-10-26T18:19:11-04:00
Acquire/release semaphore tokens more readily
This commit reworks GHC.Driver.MakeSem to use the scoped worker
abstraction of GHC.Utils.Concurrent.Scope. This allows us to get rid of
a lot of the tricky logic in GHC.Driver.MakeSem involving manual
exception handling, thread spawning and lifetime management, etc.
The main user-visible change is that token acquisition is no longer
rate limited (due to a bug, there previously was a 1s rate limit on
acquisition of semaphore tokens), with the release debounce period
shortened from 1s to 10ms.
Other changes:
- With -jsem, the number of capabilities is now capped at the number
of processors, bringing it in line with the -jN behaviour.
- Pending jobs whose wait for a token is interrupted by an exception
are now withdrawn from the jobserver, instead of acquiring a token
that it hung onto without doing useful work.
Fixes #27763
- - - - -
c83c6733 by Alan Zimmerman at 2026-10-26T18:19:55-04:00
EPA: Update AnnClsInstDecl to contain AnnList
It was already separately tracking the '{', '}' and leading ';' values.
This aligns it with layout-introducing parser productions ('vocurly'),
so the layout processing can be handled uniformly
- - - - -
c29ad436 by Simon Peyton Jones at 2026-10-28T10:59:03+00:00
Fix a nasty bug in fundeps
This MR fixes #27994, which turned out to be very tricky.
See Note [Exploiting closed type famililes], especially (CF7). I was
ending up with TyVars (not TcTyVars) in the solver, because the
substitution computed in getRelevantBranches mixed the two.
On the way I discovered that niFixSubst could produce a substitution
that mentioned domain variables on in its range. Boo! I fixed that, but
I'm not very pleased with the result. The "re-kinding" business seems
very shady. But it's not new, so I'll leave that as a battle for another
day.
- - - - -
5d92079a by Simon Peyton Jones at 2026-10-28T10:59:03+00:00
Simplify a tc-trace which could crash the compiler
The trace formatted an error message, which prematurely
read the ExpType in a FunResCtxt.
Solution: don't put call the error message formatting
when tc-tracing.
- - - - -
85b043f7 by Simon Peyton Jones at 2026-10-28T10:59:03+00:00
Wibbles
- - - - -
9269b55a by Simon Peyton Jones at 2026-10-28T10:59:03+00:00
Wibble tests
- - - - -
781afd2f by Simon Peyton Jones at 2026-10-28T10:59:03+00:00
Wibble (align injectivity flags)
- - - - -
7280bcf4 by Simon Peyton Jones at 2026-10-28T10:59:03+00:00
Fix niFixSubst again
- - - - -
cf7562ac by Simon Peyton Jones at 2026-10-28T10:59:04+00:00
More wibbles
- - - - -
66f90780 by Simon Peyton Jones at 2026-10-28T10:59:19+00:00
Use deep substitution.
This seems much simpler and more robust
- - - - -
ab565400 by Simon Peyton Jones at 2026-10-28T10:59:19+00:00
Tiny comment change to uniqAway
- - - - -
283 changed files:
- + changelog.d/25665
- + changelog.d/25850
- + changelog.d/27368
- + changelog.d/27971
- + changelog.d/28012
- + changelog.d/T27763
- + changelog.d/T27831-bottoming-strict-field
- + changelog.d/T27842-bytecode-imported-tags
- + changelog.d/monad-of-no-return-canonical
- + changelog.d/parallel-downsweep
- + changelog.d/resizeMutableByteArray-pinnedness
- + changelog.d/unboxed-do
- compiler/GHC.hs
- compiler/GHC/Builtin.hs
- compiler/GHC/Builtin/primops.txt.pp
- compiler/GHC/ByteCode/Asm.hs
- compiler/GHC/Cmm/CommonBlockElim.hs
- compiler/GHC/CmmToAsm/CFG/Dominators.hs
- compiler/GHC/CmmToLlvm/CodeGen.hs
- compiler/GHC/Core/Coercion/Axiom.hs
- compiler/GHC/Core/Predicate.hs
- compiler/GHC/Core/TyCo/FVs.hs
- compiler/GHC/Core/TyCo/Subst.hs
- compiler/GHC/Core/Type.hs
- compiler/GHC/Core/Unify.hs
- + compiler/GHC/Data/Dependent.hs
- compiler/GHC/Data/FastString.hs
- compiler/GHC/Data/OrdList.hs
- compiler/GHC/Data/Pair.hs
- compiler/GHC/Data/Word64Map/Internal.hs
- compiler/GHC/Data/Word64Set/Internal.hs
- compiler/GHC/Driver/Backpack.hs
- compiler/GHC/Driver/CmdLine.hs
- + compiler/GHC/Driver/Concurrency.hs
- + compiler/GHC/Driver/Config/Concurrency.hs
- compiler/GHC/Driver/Downsweep.hs
- compiler/GHC/Driver/Env.hs
- compiler/GHC/Driver/Errors/Ppr.hs
- compiler/GHC/Driver/Errors/Types.hs
- compiler/GHC/Driver/Flags.hs
- compiler/GHC/Driver/Main/Passes.hs
- compiler/GHC/Driver/Make.hs
- compiler/GHC/Driver/MakeAction.hs
- compiler/GHC/Driver/MakeFile.hs
- compiler/GHC/Driver/MakeSem.hs
- compiler/GHC/Driver/Pipeline/Execute.hs
- compiler/GHC/Driver/Pipeline/LogQueue.hs
- compiler/GHC/Driver/Session.hs
- compiler/GHC/Hs/Decls.hs
- compiler/GHC/Hs/Dump.hs
- compiler/GHC/HsToCore/Pmc/Types.hs
- compiler/GHC/Iface/Load.hs
- compiler/GHC/Iface/Recomp.hs
- compiler/GHC/Iface/Type.hs
- compiler/GHC/JS/Opt/Simple.hs
- compiler/GHC/Linker/Deps.hs
- compiler/GHC/Parser.y
- compiler/GHC/Parser/Header.hs
- compiler/GHC/Rename/Env.hs
- compiler/GHC/Rename/Module.hs
- compiler/GHC/Runtime/Interpreter/JS.hs
- compiler/GHC/Runtime/Loader.hs
- compiler/GHC/Stg/EnforceEpt.hs
- compiler/GHC/Stg/EnforceEpt/Rewrite.hs
- compiler/GHC/Stg/EnforceEpt/TagSig.hs
- compiler/GHC/Stg/EnforceEpt/Types.hs
- compiler/GHC/StgToCmm/Prim.hs
- compiler/GHC/StgToJS/Linker/Linker.hs
- compiler/GHC/StgToJS/Object.hs
- compiler/GHC/StgToJS/Prim.hs
- compiler/GHC/Tc/Deriv/Generate.hs
- compiler/GHC/Tc/Errors/Ppr.hs
- compiler/GHC/Tc/Errors/Types.hs
- compiler/GHC/Tc/Gen/App.hs
- compiler/GHC/Tc/Gen/Splice.hs
- compiler/GHC/Tc/Instance/Family.hs
- compiler/GHC/Tc/Module.hs
- compiler/GHC/Tc/Plugin.hs
- compiler/GHC/Tc/Solver/FunDeps.hs
- compiler/GHC/Tc/Solver/Monad.hs
- compiler/GHC/Tc/Types/Constraint.hs
- compiler/GHC/Tc/Utils/Backpack.hs
- compiler/GHC/Tc/Utils/TcMType.hs
- compiler/GHC/Tc/Utils/Unify.hs
- compiler/GHC/ThToHs.hs
- compiler/GHC/Types/Basic.hs
- compiler/GHC/Types/Error/Codes.hs
- compiler/GHC/Types/Hint.hs
- compiler/GHC/Types/Hint/Ppr.hs
- compiler/GHC/Types/Unique/FM.hs
- compiler/GHC/Types/Unique/Map.hs
- compiler/GHC/Types/UnresolvedImport.hs
- compiler/GHC/Types/Var/Env.hs
- compiler/GHC/Unit/External/ModuleOrigin.hs
- compiler/GHC/Unit/External/Visibility.hs
- compiler/GHC/Unit/Finder.hs
- + compiler/GHC/Unit/Finder/Cache.hs
- compiler/GHC/Unit/Finder/Types.hs
- compiler/GHC/Unit/Module/Graph.hs
- compiler/GHC/Unit/Module/ModSummary.hs
- compiler/GHC/Unit/Module/WholeCoreBindings.hs
- compiler/GHC/Unit/State.hs
- compiler/GHC/Unit/Types.hs
- + compiler/GHC/Utils/Concurrent/Scope.hs
- compiler/GHC/Utils/Monad/Codensity.hs
- compiler/GHC/Utils/Ppr/Colour.hs
- compiler/GHC/Utils/TmpFs.hs
- compiler/ghc.cabal.in
- docs/users_guide/exts/qualified_do.rst
- docs/users_guide/using-warnings.rst
- ghc/GHCi/UI.hs
- ghc/Main.hs
- ghc/ghc-bin.cabal.in
- hadrian/src/Hadrian/Haskell/Cabal/Parse.hs
- hadrian/src/Hadrian/Haskell/Cabal/Type.hs
- hadrian/src/Main.hs
- hadrian/src/Rules/Documentation.hs
- hadrian/src/Rules/Selftest.hs
- hadrian/src/Settings/Builders/Ar.hs
- hadrian/src/Settings/Warnings.hs
- libraries/base/src/GHC/Base.hs
- libraries/base/src/GHC/Exts.hs
- + libraries/base/tests/IO/T27971.hs
- + libraries/base/tests/IO/T27971.stdout
- libraries/base/tests/IO/all.T
- libraries/ghc-internal/CHANGELOG.md
- libraries/ghc-internal/ghc-internal.cabal.in
- libraries/ghc-internal/src/GHC/Internal/Base.hs
- libraries/ghc-internal/src/GHC/Internal/Bignum/Backend/GMP.hs
- libraries/ghc-internal/src/GHC/Internal/Bignum/Backend/Native.hs
- libraries/ghc-internal/src/GHC/Internal/Bignum/BigNat.hs
- libraries/ghc-internal/src/GHC/Internal/Bignum/Integer.hs
- libraries/ghc-internal/src/GHC/Internal/Bignum/Natural.hs
- libraries/ghc-internal/src/GHC/Internal/Bignum/Primitives.hs
- libraries/ghc-internal/src/GHC/Internal/Bignum/WordArray.hs
- libraries/ghc-internal/src/GHC/Internal/Event/Windows.hsc
- libraries/ghc-internal/src/GHC/Internal/Event/Windows/FFI.hsc
- libraries/ghc-internal/src/GHC/Internal/IO/Windows/Handle.hsc
- + libraries/ghc-internal/src/GHC/Internal/Unboxed/Do.hs
- linters/lint-codes/LintCodes/Static.hs
- rts/PrimOps.cmm
- rts/RtsSymbols.c
- rts/include/stg/MiscClosures.h
- rts/js/environment.js
- testsuite/tests/ado/ado-optimal.hs
- testsuite/tests/ado/ado001.hs
- + testsuite/tests/bytecode/T27842.hs
- + testsuite/tests/bytecode/T27842.stdout
- + testsuite/tests/bytecode/T27842A.hs
- + testsuite/tests/bytecode/T27842N.hs
- testsuite/tests/bytecode/all.T
- + testsuite/tests/codeGen/should_compile/T27368.hs
- testsuite/tests/codeGen/should_compile/all.T
- testsuite/tests/concurrent/T13615/Parallel.hs
- testsuite/tests/concurrent/should_run/T21651.hs
- testsuite/tests/concurrent/should_run/setnumcapabilities001.hs
- testsuite/tests/count-deps/CountDepsParser.stdout
- testsuite/tests/cpranal/should_run/T20598a.hs
- testsuite/tests/deriving/should_compile/T13758.hs
- testsuite/tests/deriving/should_run/all.T
- + testsuite/tests/deriving/should_run/drvrun-canonical-methods.hs
- + testsuite/tests/deriving/should_run/drvrun-canonical-methods.stderr
- + testsuite/tests/deriving/should_run/drvrun-canonical-methods.stdout
- testsuite/tests/determinism/determ017/A.hs
- testsuite/tests/diagnostic-codes/codes.stdout
- testsuite/tests/dmdanal/should_compile/T9208.hs
- testsuite/tests/driver/Makefile
- testsuite/tests/driver/T27461/Makefile
- + testsuite/tests/driver/T27461/T27461c.stderr
- testsuite/tests/driver/T27461/all.T
- + testsuite/tests/driver/T27461/src/Wrong.hs
- + testsuite/tests/driver/T28009.hs
- testsuite/tests/driver/all.T
- testsuite/tests/generics/T27245.hs
- testsuite/tests/generics/T27245.stdout
- testsuite/tests/ghc-api/downsweep/IncrementalDownsweep.hs
- testsuite/tests/ghc-api/downsweep/OldModLocation.hs
- testsuite/tests/ghc-api/downsweep/PartialDownsweep.hs
- testsuite/tests/ghc-api/fixed-nodes/FixedNodes.hs
- testsuite/tests/ghc-api/fixed-nodes/InterfaceModuleGraph.hs
- testsuite/tests/ghc-api/fixed-nodes/ModuleGraphInvariants.hs
- testsuite/tests/ghci.debugger/scripts/T14628.stderr
- testsuite/tests/haddock/perf/Fold.hs
- testsuite/tests/haddock/should_compile_flag_haddock/T17544.stderr
- testsuite/tests/hiefile/should_compile/hie002.hs
- testsuite/tests/hpc/raytrace/Eval.hs
- testsuite/tests/indexed-types/should_compile/T3787.hs
- testsuite/tests/indexed-types/should_fail/T7729.hs
- testsuite/tests/indexed-types/should_fail/T7729a.hs
- testsuite/tests/interface-stability/ghc-experimental-exports.stdout
- testsuite/tests/interface-stability/ghc-experimental-exports.stdout-mingw32
- testsuite/tests/interface-stability/ghc-prim-exports.stdout
- testsuite/tests/interface-stability/ghc-prim-exports.stdout-mingw32
- testsuite/tests/javascript/Makefile
- + testsuite/tests/javascript/T25665.hs
- + testsuite/tests/javascript/T25665.pkg
- + testsuite/tests/javascript/T25665.stdout
- + testsuite/tests/javascript/T25665Lib.hs
- + testsuite/tests/javascript/T25665_c.c
- + testsuite/tests/javascript/T25665_js.js
- testsuite/tests/javascript/all.T
- testsuite/tests/module/mod133.hs
- testsuite/tests/parser/should_compile/DumpParsedAst.stderr
- testsuite/tests/parser/should_compile/DumpRenamedAst.stderr
- testsuite/tests/parser/should_compile/KindSigs.stderr
- testsuite/tests/perf/compiler/T13253.hs
- testsuite/tests/perf/compiler/T14683.hs
- testsuite/tests/perf/compiler/T16473.hs
- testsuite/tests/perf/compiler/T18730_A.hs
- testsuite/tests/perf/compiler/T25723.hs
- testsuite/tests/plugins/defaulting-plugin/DefaultLifted.hs
- testsuite/tests/pmcheck/should_compile/T11276.hs
- + testsuite/tests/primops/should_run/ResizeByteArrayPinnedness.hs
- + testsuite/tests/primops/should_run/ResizeByteArrayPinnedness.stdout
- testsuite/tests/primops/should_run/all.T
- testsuite/tests/printer/Test24533.stdout
- testsuite/tests/profiling/should_compile/T19894/StreamK.hs
- testsuite/tests/profiling/should_run/T3001-2.hs
- testsuite/tests/programs/galois_raytrace/Eval.hs
- testsuite/tests/qualifieddo/should_compile/all.T
- + testsuite/tests/qualifieddo/should_compile/qdocompile003.hs
- + testsuite/tests/qualifieddo/should_compile/qdocompile003.stderr
- testsuite/tests/qualifieddo/should_fail/all.T
- + testsuite/tests/qualifieddo/should_fail/qdofail006.hs
- + testsuite/tests/qualifieddo/should_fail/qdofail006.stderr
- testsuite/tests/qualifieddo/should_run/all.T
- + testsuite/tests/qualifieddo/should_run/qdorun008.hs
- + testsuite/tests/qualifieddo/should_run/qdorun008.stdout
- testsuite/tests/rebindable/rebindable2.hs
- + testsuite/tests/rename/should_fail/Noncanonical.hs
- + testsuite/tests/rename/should_fail/Noncanonical.stderr
- testsuite/tests/rename/should_fail/all.T
- testsuite/tests/rename/should_fail/rnfail026.hs
- + testsuite/tests/rts/T27526.hs
- + testsuite/tests/rts/T27526.stdout
- + testsuite/tests/rts/T28012.hs
- + testsuite/tests/rts/T28012.stdout
- testsuite/tests/rts/all.T
- testsuite/tests/rts/ipe/IpeStats/Fold.hs
- testsuite/tests/simplCore/should_compile/T14908_Deps.hs
- testsuite/tests/simplCore/should_compile/simpl017.hs
- testsuite/tests/simplStg/should_compile/all.T
- + testsuite/tests/simplStg/should_compile/inferTags005.hs
- + testsuite/tests/simplStg/should_compile/inferTags005.stderr
- + testsuite/tests/simplStg/should_compile/inferTags006.hs
- + testsuite/tests/simplStg/should_compile/inferTags006.stderr
- + testsuite/tests/simplStg/should_run/InferTags007_A.hs
- + testsuite/tests/simplStg/should_run/T27831.hs
- + testsuite/tests/simplStg/should_run/T27831.stdout
- + testsuite/tests/simplStg/should_run/T27857.hs
- + testsuite/tests/simplStg/should_run/T27857.stdout
- testsuite/tests/simplStg/should_run/all.T
- + testsuite/tests/simplStg/should_run/inferTags007.hs
- + testsuite/tests/simplStg/should_run/inferTags007.stdout
- testsuite/tests/splice-imports/SI35.hs
- testsuite/tests/typecheck/should_compile/DeepSubsumption02.hs
- + testsuite/tests/typecheck/should_compile/T27994.hs
- + testsuite/tests/typecheck/should_compile/T27994a.hs
- testsuite/tests/typecheck/should_compile/T4524.hs
- testsuite/tests/typecheck/should_compile/T4969.hs
- testsuite/tests/typecheck/should_compile/all.T
- testsuite/tests/typecheck/should_fail/T22707.hs
- testsuite/tests/typecheck/should_fail/T22707.stderr
- testsuite/tests/typecheck/should_fail/T25950.stderr
- testsuite/tests/typecheck/should_fail/T7019.hs
- testsuite/tests/typecheck/should_fail/T8450.stderr
- testsuite/tests/typecheck/should_fail/T8603.hs
- + testsuite/tests/warnings/should_compile/RedundantCanonicalMethods.hs
- + testsuite/tests/warnings/should_compile/RedundantCanonicalMethods.stderr
- testsuite/tests/warnings/should_compile/WarnNoncanonical.hs
- testsuite/tests/warnings/should_compile/WarnNoncanonical.stderr
- testsuite/tests/warnings/should_compile/all.T
- utils/check-exact/ExactPrint.hs
- utils/check-ppr/Main.hs
- utils/haddock/haddock-api/haddock-api.cabal
- utils/haddock/haddock-api/src/Haddock/Convert.hs
- utils/haddock/haddock-api/src/Haddock/Interface.hs
- utils/haddock/haddock-api/src/Haddock/Interface/ParseModuleHeader.hs
- utils/haddock/haddock-api/src/Haddock/Types.hs
- utils/haddock/haddock-api/src/Haddock/Utils/Json.hs
- utils/haddock/haddock-library/haddock-library.cabal
- utils/haddock/haddock.cabal
- utils/runghc/Main.hs
The diff was not included because it is too large.
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/f9fcb58563cd24ca3c898cd7f65c08…
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/f9fcb58563cd24ca3c898cd7f65c08…
You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help
1
0
[Git][ghc/ghc][wip/dcoutts/signals-cleanup] 5 commits: Redo the scheme for special fds in the poll and selectbis I/O managers
by Duncan Coutts (@dcoutts) 28 Oct '26
by Duncan Coutts (@dcoutts) 28 Oct '26
28 Oct '26
Duncan Coutts pushed to branch wip/dcoutts/signals-cleanup at Glasgow Haskell Compiler / GHC
Commits:
40c1fcd3 by Duncan Coutts at 2026-10-27T22:59:12+00:00
Redo the scheme for special fds in the poll and selectbis I/O managers
At the moment we only have one special fd for these I/O managers: the
pipe (or eventfd) for interrupting them when they're sleeping. We're
about to gain a second for signal handling.
The existing scheme was rather ad-hoc and hard to extend. It was
already complicated by the fact that the interrupt fd is optional. The
signal fd will also be optional in the sense that cap 0 will have it but
cap >0 will not.
So the new scheme is that we add special aiop entries into the
aiop_table, and for the most part we can treat these uniformly with
other aiops. But the action we take for them if/when they are ready is
handled specially.
This seems much cleaner and will be easy to extend for signals.
- - - - -
b67c4955 by Duncan Coutts at 2026-10-27T22:59:12+00:00
Use the signal self-pipe trick for the poll and selectbis I/O managers
So now the low level signal handler delivers signals into a pipe that is
monitored by the I/O manager (on cap 0). The I/O manager then dispatches
the handler (starting a Haskell thread).
There are two important reasons for this change:
1. This method does not have any windows of time where signals can be
missed for arbitrary periods of time. The previous method has a race
where a signal delivered just before the I/O manager calls
poll/select will not notice the signal until some unrelated event
causes poll/select to return. See issue #27540. The new method
ensures that a fd is ready and so poll/select will notice
immediately (if the signal was delivered prior to the call to
poll/select) or later.
2. The existing method relies on temporarily masking signals for
concurrency safety which cannot work reliably in a multi-threaded
process. Fixing this is a necessary pre-requisite to enable these
in-RTS I/O managers to work in the threaded RTS. The issue is that
startPendingSignalHandlers and the low level signal handler share the
pending_handler_buf data structure, and so startPendingSignalHandlers
has to mask delivery of signals to avoid the shared data being
mutated concurrently. But masking signals in a multi-threaded process
is not generally effective because the signal can still be delivered
in any thread
Document the self-pipe trick in a Note.
This fixes issue #27540.
- - - - -
4557ab5c by Duncan Coutts at 2026-10-27T22:59:12+00:00
fixup Use the signal self-pipe trick for the poll and selectbis I/O managers
- - - - -
72413061 by Duncan Coutts at 2026-10-27T22:59:12+00:00
Move the legacy user signal handler into legacy select I/O manager
We do not convert the legacy select I/O manager to use the self-pipe
trick. It keeps the existing code, but since it is now the only user of
it, then that code can move from being a shared utility to live with the
select I/O manager.
There are some minor modifications to the code moved so that it is not
entangled with other remaining shared signal utils.
- - - - -
06de1e3c by Duncan Coutts at 2026-10-27T22:59:12+00:00
Deprecate (un)blockUserSignals for external RTS users
Encourage external users to stop using these actions.
- - - - -
12 changed files:
- rts/IOManager.c
- rts/IOManagerInternals.h
- rts/RtsSymbols.c
- rts/include/rts/BlockSignals.h
- rts/include/rts/storage/Closures.h
- rts/posix/IOManSignals.c
- rts/posix/IOManSignals.h
- rts/posix/Poll.c
- rts/posix/Poll.h
- rts/posix/Select.c
- rts/posix/SelectBis.c
- rts/posix/SelectBis.h
Changes:
=====================================
rts/IOManager.c
=====================================
@@ -462,6 +462,20 @@ void startIOManager(void)
switch (iomgr_type) {
+#if defined(IOMGR_ENABLED_SELECTBIS)
+ case IO_MANAGER_SELECTBIS:
+ for (unsigned int i = 0; i < n_capabilities; i++) {
+ startCapabilityIOManagerSelectBis(getCapability(i)->iomgr);
+ }
+ break;
+#endif
+#if defined(IOMGR_ENABLED_POLL)
+ case IO_MANAGER_POLL:
+ for (unsigned int i = 0; i < n_capabilities; i++) {
+ startCapabilityIOManagerPoll(getCapability(i)->iomgr);
+ }
+ break;
+#endif
#if defined(IOMGR_ENABLED_MIO_POSIX)
case IO_MANAGER_MIO_POSIX:
/* Posix implementation in posix/Signals.c
@@ -514,6 +528,18 @@ restartIOManager(CapIOManager *iomgr, Capability **pcap)
{
switch (iomgr_type) {
+#if defined(IOMGR_ENABLED_SELECTBIS)
+ case IO_MANAGER_SELECTBIS:
+ startCapabilityIOManagerSelectBis(iomgr);
+ break;
+#endif
+
+#if defined(IOMGR_ENABLED_POLL)
+ case IO_MANAGER_POLL:
+ startCapabilityIOManagerPoll(iomgr);
+ break;
+#endif
+
#if defined(IOMGR_ENABLED_MIO_POSIX)
case IO_MANAGER_MIO_POSIX:
/* Posix implementation in posix/Signals.c
@@ -530,8 +556,6 @@ restartIOManager(CapIOManager *iomgr, Capability **pcap)
break;
#endif
/* The IO_MANAGER_SELECT needs no initialisation */
- /* The IO_MANAGER_SELECTBIS needs no initialisation */
- /* The IO_MANAGER_POLL needs no initialisation */
/* No impl for any of the Windows I/O managers, since no forking. */
default:
=====================================
rts/IOManagerInternals.h
=====================================
@@ -76,6 +76,9 @@ struct _CapIOManager {
/* AIOP and timeout collections shared by several I/O manager impls */
ClosureTable aiop_table;
StgTimeoutQueue *timeout_queue;
+
+ /* The number of special fds at the start of the aiop table. */
+ int n_special_fds;
#endif
#if defined(IOMGR_ENABLED_SELECTBIS)
@@ -83,11 +86,8 @@ struct _CapIOManager {
#endif
#if defined(IOMGR_ENABLED_POLL)
- /* Auxiliary table with size and indexes matching the aiop_table. This is
- * aliased to the tail of the full poll table, which has a head entry for
- * the wakeup_fd_r above, so we can also poll that fd.
- */
- struct pollfd *aiop_poll_table, *full_poll_table;
+ /* Auxiliary table with size and indexes matching the aiop_table. */
+ struct pollfd *aiop_poll_table;
#endif
#if defined(IOMGR_ENABLED_WIN32_LEGACY)
@@ -113,6 +113,17 @@ struct _CapIOManager {
*/
struct _CapIOManagerShared {
+#if defined(IOMGR_ENABLED_POLL) \
+ || defined(IOMGR_ENABLED_SELECTBIS)
+#if defined(RTS_USER_SIGNALS)
+ /* Pipe FDs for delivering posix signals to the I/O manager.
+ * The signal number and siginfo_t are written to the write end by the
+ * low-level signal handler, and read by the I/O manager on capability 0.
+ */
+ int signal_fd_r, signal_fd_w;
+#endif
+#endif
+
#if defined(IOMGR_ENABLED_MIO_POSIX)
/* The timer manager control pipe (write end). This is the pipe into which
* we will send our signals.
=====================================
rts/RtsSymbols.c
=====================================
@@ -1233,6 +1233,9 @@ RTS_SYMBOLS_PRIM
(void*)(&(xxx)), strength, ty },
+#pragma GCC diagnostic push
+/* Don't warn about exporting deprecated RTS functions. */
+#pragma GCC diagnostic ignored "-Wdeprecated-declarations"
/* Initialize (if not already initialized) and return an array of symbols with stuff from the RTS. */
void initLinkerRtsSyms (StrHashTable *symhash) {
@@ -1270,6 +1273,7 @@ void initLinkerRtsSyms (StrHashTable *symhash) {
IF_DEBUG(linker, debugBelch("done with built-in RTS symbols\n"));
}
+#pragma GCC diagnostic pop
// Note [Extra RTS symbols]
=====================================
rts/include/rts/BlockSignals.h
=====================================
@@ -33,13 +33,17 @@
* This is exported from the RTS but should no longer be used. It is likely to
* be removed in future.
*/
-void blockUserSignals(void);
+void blockUserSignals(void)
+ __attribute__((deprecated("blockUserSignals should no longer be used, "
+ "use pthread_sigmask or similar.")));
/*
* Function: unblockUserSignals()
*
* The inverse of blockUserSignals(); re-enable the deliver of console events.
*/
-void unblockUserSignals(void);
+void unblockUserSignals(void)
+ __attribute__((deprecated("unblockUserSignals should no longer be "
+ "used, use pthread_sigmask or similar.")));
#endif
=====================================
rts/include/rts/storage/Closures.h
=====================================
@@ -722,14 +722,16 @@ typedef struct {
------------------------------------------------------------------------- */
union NotifyCompletion {
- StgTSO *tso;
- StgMVar *mvar;
- StgTVar *tvar;
+ StgTSO *tso;
+ StgMVar *mvar;
+ StgTVar *tvar;
+ StgClosure *unused;
};
enum NotifyCompletionType {
- NotifyTSO = 0, // thread-synchronous I/O
- NotifyMVar = 1, // async I/O with MVar notification
- NotifyTVar = 2 // async I/O with TVar notification
+ NotifyTSO = 0, // thread-synchronous I/O
+ NotifyMVar = 1, // async I/O with MVar notification
+ NotifyTVar = 2, // async I/O with TVar notification
+ NotifyInternal = 3 // used for special I/O manager-internal notification
};
/* A node in the leftist heap. */
@@ -873,6 +875,10 @@ typedef struct {
// For failed outcomes, this is the error code.
uint32_t error;
+
+ // For notify_type NotifyInternal, this is available for internal
+ // use by individual I/O managers.
+ uint32_t internal;
};
// Note that because we use fixed size Ctypes here then the size in words
=====================================
rts/posix/IOManSignals.c
=====================================
@@ -92,76 +92,6 @@ more_handlers(int sig)
nHandlers = sig + 1;
}
-#if !defined(THREADED_RTS)
-
-#define N_PENDING_HANDLERS 16
-
-siginfo_t pending_handler_buf[N_PENDING_HANDLERS];
-siginfo_t *next_pending_handler = pending_handler_buf;
-
-static inline bool signals_pending(void) {
- return (next_pending_handler != pending_handler_buf);
-}
-
-#endif /* THREADED_RTS */
-
-/* -----------------------------------------------------------------------------
- * Low-level signal handler
- *
- * Places the requested handler on a stack of pending handlers to be
- * started up at the next context switch.
- * -------------------------------------------------------------------------- */
-
-#if !defined(THREADED_RTS)
-void
-generic_handler(int sig USED_IF_THREADS,
- siginfo_t *info,
- void *p STG_UNUSED)
-{
-
- /* Can't call allocate from here. Probably can't call malloc
- either. However, we have to schedule a new thread somehow.
-
- It's probably ok to request a context switch and allow the
- scheduler to start the handler thread, but how do we
- communicate this to the scheduler?
-
- We need some kind of locking, but with low overhead (i.e. no
- blocking signals every time around the scheduler).
-
- Signal Handlers are atomic (i.e. they can't be interrupted), and
- we can make use of this. We just need to make sure the
- critical section of the scheduler can't be interrupted - the
- only way to do this is to block signals. However, we can lower
- the overhead by only blocking signals when there are any
- handlers to run, i.e. the set of pending handlers is
- non-empty.
- */
-
- /* We use a stack to store the pending signals. We can't
- dynamically grow this since we can't allocate any memory from
- within a signal handler.
-
- Hence unfortunately we have to bomb out if the buffer
- overflows. It might be acceptable to carry on in certain
- circumstances, depending on the signal.
- */
-
- memcpy(next_pending_handler, info, sizeof(siginfo_t));
-
- next_pending_handler++;
-
- // stack full?
- if (next_pending_handler == &pending_handler_buf[N_PENDING_HANDLERS]) {
- errorBelch("too many pending signals");
- stg_exit(EXIT_FAILURE);
- }
-
- interruptCapability(&MainCapability);
-
-}
-#endif /* THREADED_RTS */
-
/* -----------------------------------------------------------------------------
* Blocking/Unblocking of the user signals
* -------------------------------------------------------------------------- */
@@ -267,34 +197,6 @@ installSignalHandler(int sig, int spi, sigset_t *mask, bool nocldstop,
* Creating new threads for signal handlers.
* -------------------------------------------------------------------------- */
-#if !defined(THREADED_RTS)
-void
-startPendingSignalHandlers(Capability *cap)
-{
- siginfo_t *info;
- int sig;
-
- if (!signals_pending()) return;
-
- blockUserSignals();
-
- while (next_pending_handler != pending_handler_buf) {
-
- next_pending_handler--;
-
- sig = next_pending_handler->si_signo;
- if (signal_handlers[sig] == STG_SIG_DFL) {
- continue; // handler has been changed.
- }
-
- info = next_pending_handler;
- startSignalHandler(cap, sig, info);
- }
-
- unblockUserSignals();
-}
-#endif
-
void startSignalHandler(Capability *cap, int sig, siginfo_t *info)
{
/* Allocate a copy. This is freed by runHandler. */
@@ -321,6 +223,43 @@ void startSignalHandler(Capability *cap, int sig, siginfo_t *info)
setThreadLabel(cap, t, "signal handler thread");
}
+/* Note [Non-racy signal handling using the self-pipe trick]
+ * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ *
+ * Reliably picking up unix/posix signals promptly is surprisingly tricky!
+ * The standard problem is that there is a race between signal delivery and
+ * waiting/blocking in something like select/poll/epoll_wait. The problem is
+ * that if one checks for a signal having been delivered (e.g. by looking at a
+ * shared variable set by the signal handler) just before calling select/poll
+ * then there's a time window between that point and calling select/poll when
+ * a signal could be delivered and then not noticed until select/poll returns.
+ * Once we've called select/poll then it can return EINTR upon signal arrival,
+ * but the window before remains a problem.
+ *
+ * One solution is to use pselect/ppoll: mask signals, then do a final check
+ * for signals having been delivered, then call pselect/ppoll to unmask
+ * signals (atomically) for the duration of the wait. This is fine in a single
+ * threaded process. In a multi-threaded process, masking signals doesn't work.
+ * See Note [pthread_sigmask and the ineffectiveness of masking signals].
+ *
+ * The standard solution is the "self-pipe trick" where the low level signal
+ * handler writes the signal info into a pipe, and poll/select monitors that
+ * pipe. This neatly eliminates the window where signals could be missed. If
+ * the signal arrives before calling poll/select then the pipe is readable and
+ * poll/select will return. If the signal arrives during the poll/select call
+ * then they will return with EINTR.
+ *
+ * In the RTS context, this is done by the I/O managers: the in-RTS ones use
+ * newNonBlockingPipe on startup, have their low level signal handler use
+ * writeSignalInfoToPipe, and when the I/O manager finds the other end of the
+ * pipe is readable then it calls readSignalInfoFromPipe, and can then use
+ * startSignalHandler.
+ *
+ * The in-Haskell MIO I/O manager does essentially the same.
+ *
+ * See issue #27540.
+ */
+
void newNonBlockingPipe(int *pipe_fd_r, int *pipe_fd_w)
{
int pipefd[2];
=====================================
rts/posix/IOManSignals.h
=====================================
@@ -25,16 +25,6 @@ void initUserSignals(void);
*/
void finiUserSignals(void);
-#if !defined(THREADED_RTS)
-/*
- * Function: startPendingSignalHandlers()
- *
- * If there are any queued up posix signals, run the handlers
- * associated with them.
- */
-void startPendingSignalHandlers(Capability *cap);
-#endif
-
void startSignalHandler(Capability *cap, int sig, siginfo_t *info);
typedef void (*SignalHandler)(int sig, siginfo_t *info, void *p);
@@ -47,11 +37,6 @@ void newNonBlockingPipe(int *pipe_fd_r, int *pipe_fd_w);
void writeSignalInfoToPipe (int sig, siginfo_t *info, int fd);
bool readSignalInfoFromPipe(int *sig, siginfo_t *info, int fd);
-/* Temporary */
-void generic_handler(int sig USED_IF_THREADS,
- siginfo_t *info,
- void *p STG_UNUSED);
-
#endif
#include "EndPrivate.h"
=====================================
rts/posix/Poll.c
=====================================
@@ -1,6 +1,6 @@
/* -----------------------------------------------------------------------------
*
- * (c) The GHC Team 2020-2023
+ * (c) The GHC Team 2020-2026
*
* An I/O manager based on the classic Unix poll() system call.
*
@@ -24,6 +24,7 @@
#include "Poll.h"
+#include <unistd.h>
#include <limits.h>
#include <errno.h>
#if defined(HAVE_DECL_PPOLL) && HAVE_DECL_PPOLL == 1
@@ -107,9 +108,10 @@ timeout (if any) as the poll() timeout parameter.
The CapIOManager structure for this I/O manager contains:
ClosureTable aiop_table;
- struct pollfd *aiop_poll_table, *full_poll_table;
+ struct pollfd *aiop_poll_table;
StgTimeoutQueue *timeout_queue;
int interrupt_fd_r, interrupt_fd_w;
+ int n_special_fds;
We also support the Linux-specific ppoll API which supports higher resolution
time delays -- nanoseconds rather than milliseconds as in classic poll(). It
@@ -118,14 +120,15 @@ also allows the signal mask to be adjusted, but we do not make use of this.
int ppoll(struct pollfd *fds, nfds_t nfds,
const struct timespec *tmo_p, const sigset_t *sigmask);
-We have both aiop_poll_table and full_poll_table. This is to cope with needing
-to wait on the special extra file descriptor interrupt_fd_r. This fd is used to
-support waking the I/O manager when we are blocked in a poll call. This
-requires waiting on an extra fd that has no corresponding entry in the
-aiop_table. To manage this quirk, we alias the aiop_poll_table to be the tail
-of the full_poll_table and have the first entry of the full_poll_table be the
-interrupt_fd_r. This means the aiop_poll_table indicies match up exactly with
-the aiop_table, but still allows the full_poll_table to have an extra entry.
+In addition to aiops, there are some special internal fds we have to monitor
+and respond to. In particular there is the interrupt_fd_r used to implement
+interruptIOManager. Whether this fd is used or not depends on configuration
+(use of interrupt_fd_r depends on HAVE_PREEMPTION). To enable multiple such
+fds and allow them to be optional we use the following scheme: we create
+special internal AsyncIOOps and put them into the aiop_table. These are
+treated much like other IOOpCodeWaitRead operations but with special handling
+for notification. The variable n_special_fds tracks how many such entries we
+have so that we can handle them specially but in a reasonably uniform manner.
******************************************************************************/
@@ -135,7 +138,20 @@ static void notifyIOCompletion(CapIOManager *iomgr, StgAsyncIOOp *aiop);
static void removeFromTables(CapIOManager *iomgr, int i);
static void reportPollError(int res, nfds_t nfds) STG_NORETURN;
+enum SpecialFd {
+ InterruptFd, /* iomgr->interrupt_fd_r for interruptIOManager */
+ SignalFd /* iomgr->shared->signal_fd_r */
+};
+static void insertSpecialFd(CapIOManager *iomgr,
+ enum SpecialFd special_fd_kind,
+ int special_fd);
+static void processSpecialFd(CapIOManager *iomgr,
+ StgAsyncIOOp *aiop, bool *interrupt);
+
+/* Called early in RTS startup _before_ the storage manager is initialised,
+ * so this is not allowed to allocate on the GC heap.
+ */
void initCapabilityIOManagerPoll(CapIOManager *iomgr)
{
initClosureTable(&iomgr->aiop_table, ClosureTableCompact);
@@ -145,26 +161,80 @@ void initCapabilityIOManagerPoll(CapIOManager *iomgr)
newFdWakeup(&iomgr->interrupt_fd_r, &iomgr->interrupt_fd_w);
#endif
- iomgr->full_poll_table = stgMallocBytes(sizeof(struct pollfd) /* size 1 */,
- "initCapabilityIOManagerPoll");
- iomgr->full_poll_table[0] = (struct pollfd) {
+ iomgr->n_special_fds = 0
#if defined(HAVE_PREEMPTION)
- .fd = iomgr->interrupt_fd_r,
- .events = POLLIN,
-#else
- .fd = -1, // unused
- .events = 0, // unused
+ + 1
+#endif
+#if defined(RTS_USER_SIGNALS)
+ + (iomgr->cap->no == 0 ? 1 : 0)
#endif
- .revents = 0
- };
- iomgr->aiop_poll_table = iomgr->full_poll_table+1; /* hence empty */
+ ;
#if defined(RTS_USER_SIGNALS)
/* Signal handlers are shared across all caps. Only init once. */
if (iomgr->cap->no == 0) {
initUserSignals();
+ /* See Note [Non-racy signal handling using the self-pipe trick] */
+ newNonBlockingPipe(&iomgr->shared->signal_fd_r,
+ &iomgr->shared->signal_fd_w);
+ }
+#endif
+}
+
+
+/* Called late in RtsStartup, once heap allocation is ready. */
+void startCapabilityIOManagerPoll(CapIOManager *iomgr)
+{
+ /* ClosureTable allocates on the heap, so we can do it here not in init */
+ if (iomgr->n_special_fds > 0) {
+ bool ok = enlargeClosureTable(iomgr->cap, &iomgr->aiop_table,
+ iomgr->n_special_fds);
+ CHECK(ok);
}
+ iomgr->aiop_poll_table =
+ stgMallocBytes(sizeof(struct pollfd) * iomgr->n_special_fds,
+ "initCapabilityIOManagerPoll");
+
+#if defined(HAVE_PREEMPTION)
+ insertSpecialFd(iomgr, InterruptFd, iomgr->interrupt_fd_r);
#endif
+#if defined(RTS_USER_SIGNALS)
+ if (iomgr->cap->no == 0) {
+ /* See Note [Non-racy signal handling using the self-pipe trick] */
+ insertSpecialFd(iomgr, SignalFd, iomgr->shared->signal_fd_r);
+ }
+#endif
+ ASSERT(iomgr->n_special_fds == sizeClosureTable(&iomgr->aiop_table));
+}
+
+
+/* For special fds, we add a dummy entry to the aiop_table and a
+ * corresponding real entry to the aiop_poll_table.
+ */
+static void insertSpecialFd(CapIOManager *iomgr,
+ enum SpecialFd special_fd_kind,
+ int special_fd)
+{
+ StgAsyncIOOp *aiop = (StgAsyncIOOp *)allocate(iomgr->cap,
+ sizeofW(StgAsyncIOOp));
+ SET_HDR(aiop, &stg_ASYNCIOOP_info, CCS_SYSTEM);
+ aiop->notify.unused = &stg_ASYNCIO_LIVE0_closure;
+ aiop->live = &stg_ASYNCIO_LIVE0_closure;
+ aiop->capno = iomgr->cap->no;
+ aiop->notify_type = NotifyInternal;
+ aiop->outcome = IOOpOutcomeInFlight;
+ aiop->operation = IOOpCodeWaitRead;
+ aiop->fd = special_fd;
+ aiop->internal = special_fd_kind;
+
+ int ix = insertClosureTable(iomgr->cap, &iomgr->aiop_table, aiop);
+ aiop->index = ix;
+
+ iomgr->aiop_poll_table[ix] = (struct pollfd) {
+ .fd = special_fd,
+ .events = POLLIN,
+ .revents = 0
+ };
}
@@ -173,9 +243,11 @@ void freeCapabilityIOManagerPoll(CapIOManager *iomgr)
#if defined(RTS_USER_SIGNALS)
if (iomgr->cap->no == 0) {
finiUserSignals();
+ close(iomgr->shared->signal_fd_r);
+ close(iomgr->shared->signal_fd_w);
}
#endif
- stgFree(iomgr->full_poll_table);
+ stgFree(iomgr->aiop_poll_table);
#if defined(HAVE_PREEMPTION)
closeFdWakeup(iomgr->interrupt_fd_r, iomgr->interrupt_fd_w);
#endif
@@ -314,22 +386,10 @@ static bool processIOCompletions(CapIOManager *iomgr, int ncompletions)
ASSERT(ncompletions > 0);
bool interrupt = false;
-#if defined(HAVE_PREEMPTION)
- /* If the interrupt_fd_r is ready, collect it */
- if (iomgr->full_poll_table[0].revents) {
- ASSERT(iomgr->full_poll_table[0].fd == iomgr->interrupt_fd_r);
- collectFdWakeup(iomgr->interrupt_fd_r);
- ncompletions--;
- interrupt = true;
- debugTrace(DEBUG_iomanager, "Received interrupt in poll I/O manager");
- }
-#endif
struct pollfd *aiop_poll_table = iomgr->aiop_poll_table;
int i = 0;
- while (ncompletions > 0) {
- ASSERT(i < sizeClosureTable(&iomgr->aiop_table));
-
+ while (ncompletions > 0 && i < sizeClosureTable(&iomgr->aiop_table)) {
/* Since each aiop_table entry is for a single (fd, rw) pair, we
* just need to check if the aiop_poll_table[n].revents contains
* anything. We don't need to distinguish read vs write.
@@ -337,26 +397,34 @@ static bool processIOCompletions(CapIOManager *iomgr, int ncompletions)
if (aiop_poll_table[i].revents) {
StgAsyncIOOp *aiop = indexClosureTable(&iomgr->aiop_table, i);
+ ASSERT(aiop->operation == IOOpCodeWaitRead ||
+ aiop->operation == IOOpCodeWaitWrite);
ASSERT(aiop->outcome == IOOpOutcomeInFlight);
- /* We do need to handle POLLNVAL, but we do not need to do anything
- * special for POLLERR or POLLHUP. (See man poll for details).
- * The calling thread will typically try to do I/O after waiting
- * for I/O readiness. Thus for POLLERR they'll discover the error
- * when doing the I/O. And for POLLHUP, there is typically no
- * action required anyway: it's a notification and not an error.
- */
- if (aiop_poll_table[i].revents & POLLNVAL) {
- aiop->outcome = IOOpOutcomeFailed;
- aiop->error = EBADF;
+ if (aiop->notify_type == NotifyInternal) {
+ processSpecialFd(iomgr, aiop, &interrupt);
+ ncompletions--;
+ i++;
} else {
- aiop->outcome = IOOpOutcomeSuccess;
- aiop->result = 0;
+ /* We do need to handle POLLNVAL, but we do not need to do anything
+ * special for POLLERR or POLLHUP. (See man poll for details).
+ * The calling thread will typically try to do I/O after waiting
+ * for I/O readiness. Thus for POLLERR they'll discover the error
+ * when doing the I/O. And for POLLHUP, there is typically no
+ * action required anyway: it's a notification and not an error.
+ */
+ if (aiop_poll_table[i].revents & POLLNVAL) {
+ aiop->outcome = IOOpOutcomeFailed;
+ aiop->error = EBADF;
+ } else {
+ aiop->outcome = IOOpOutcomeSuccess;
+ aiop->result = 0;
+ }
+
+ removeFromTables(iomgr, i);
+ notifyIOCompletion(iomgr, aiop);
+ ncompletions--;
}
-
- removeFromTables(iomgr, i);
- notifyIOCompletion(iomgr, aiop);
- ncompletions--;
} else {
/* You'd expect incrementing the table index to be unconditional,
* but we don't increment the index if we did process the entry,
@@ -366,14 +434,44 @@ static bool processIOCompletions(CapIOManager *iomgr, int ncompletions)
i++;
}
}
+ ASSERT(ncompletions == 0);
return interrupt;
}
-void pollCompletedTimeoutsOrIOPoll(CapIOManager *iomgr)
+static void processSpecialFd(CapIOManager *iomgr,
+ StgAsyncIOOp *aiop, bool *interrupt)
{
- ASSERT(iomgr->aiop_poll_table == iomgr->full_poll_table+1);
+ ASSERT(aiop->notify_type == NotifyInternal);
+ enum SpecialFd special_fd_kind = aiop->internal;
+ int fd = aiop->fd;
+
+ switch (special_fd_kind) {
+#if defined(HAVE_PREEMPTION)
+ case InterruptFd:
+ ASSERT(fd == iomgr->interrupt_fd_r);
+ collectFdWakeup(fd);
+ *interrupt = true;
+ debugTrace(DEBUG_iomanager, "Received interrupt in poll I/O manager");
+ break;
+#endif
+#if defined(RTS_USER_SIGNALS)
+ /* See Note [Non-racy signal handling using the self-pipe trick] */
+ case SignalFd:
+ ASSERT(fd == iomgr->shared->signal_fd_r);
+ int sig; siginfo_t info;
+ readSignalInfoFromPipe(&sig, &info, fd);
+ startSignalHandler(iomgr->cap, sig, &info);
+ debugTrace(DEBUG_iomanager,
+ "Dispatched signal %d in I/O manager", sig);
+ break;
+#endif
+ }
+}
+
+void pollCompletedTimeoutsOrIOPoll(CapIOManager *iomgr)
+{
if (!isEmptyTimeoutQueue(iomgr->timeout_queue)) {
Time now = getProcessElapsedTime();
processTimeoutCompletions(iomgr, now);
@@ -381,29 +479,20 @@ void pollCompletedTimeoutsOrIOPoll(CapIOManager *iomgr)
if (!isEmptyClosureTable(&iomgr->aiop_table)) {
- int n_aiops = sizeClosureTable(&iomgr->aiop_table);
-#if defined(HAVE_PREEMPTION)
- /* the full_poll_table includes interrupt_fd_r */
- nfds_t nfds = n_aiops + 1;
- struct pollfd *poll_table = iomgr->full_poll_table;
-#else
- /* the aiop_poll_table does not include interrupt_fd_r */
- nfds_t nfds = n_aiops + 0;
- struct pollfd *poll_table = iomgr->aiop_poll_table;
-#endif
+ nfds_t nfds = sizeClosureTable(&iomgr->aiop_table);
/* Poll for I/O readiness, without waiting. */
#if defined(HAVE_DECL_PPOLL) && HAVE_DECL_PPOLL == 1
/* We could use poll here, since we use no timeout, but for
consistency we use the same syscall as at the other call site. */
struct timespec tv = (struct timespec) { .tv_sec = 0, .tv_nsec = 0 };
- int res = ppoll(poll_table, nfds, &tv, NULL);
+ int res = ppoll(iomgr->aiop_poll_table, nfds, &tv, NULL);
debugTrace(DEBUG_iomanager,
"ppoll(nfds = %lu, timeout = 0) = %d",
(unsigned long) nfds, res);
#else
- int res = poll(poll_table, nfds, 0);
+ int res = poll(iomgr->aiop_poll_table, nfds, 0);
debugTrace(DEBUG_iomanager,
"poll(nfds = %lu, timeout = 0) = %d",
@@ -426,10 +515,6 @@ void pollCompletedTimeoutsOrIOPoll(CapIOManager *iomgr)
reportPollError(res, nfds);
}
}
-
-#if defined(RTS_USER_SIGNALS)
- startPendingSignalHandlers(iomgr->cap);
-#endif
}
@@ -437,8 +522,6 @@ bool awaitCompletedTimeoutsOrIOPoll(CapIOManager *iomgr)
{
bool interrupt = false; /* got woken up via interruptIOManager */
- ASSERT(iomgr->aiop_poll_table == iomgr->full_poll_table+1);
-
/* Loop until we've woken up some threads. This loop is needed because the
* poll() timing isn't accurate, we sometimes sleep for a while but not
* long enough to wake up a thread in a threadDelay. Or we may need to
@@ -461,22 +544,14 @@ bool awaitCompletedTimeoutsOrIOPoll(CapIOManager *iomgr)
*/
bool wait = emptyRunQueue(iomgr->cap);
- int n_aiops = sizeClosureTable(&iomgr->aiop_table);
-#if defined(HAVE_PREEMPTION)
- /* the full_poll_table includes interrupt_fd_r */
- nfds_t nfds = n_aiops + 1;
- struct pollfd *poll_table = iomgr->full_poll_table;
-#else
- /* the aiop_poll_table does not include interrupt_fd_r */
- nfds_t nfds = n_aiops + 0;
- struct pollfd *poll_table = iomgr->aiop_poll_table;
-#endif
-
/* Decide if we are going to wait if no I/O is ready, either:
* poll only, wait indefinitely, or wait until a timeout.
*/
Time timeout = timeoutWaitTime(iomgr, wait, now);
+ nfds_t nfds = sizeClosureTable(&iomgr->aiop_table);
+ int n_aiops = nfds - iomgr->n_special_fds;
+
if (RTS_UNLIKELY(timeout == -1 && n_aiops == 0)) {
/* See Note [Deadlock detection without idle GC] */
interrupt = notifyIdleGcDeadlock();
@@ -489,7 +564,7 @@ bool awaitCompletedTimeoutsOrIOPoll(CapIOManager *iomgr)
* time, and if that timeout is reached then we schedule an idle GC
* (see handleIdleGcTimeout below).
*/
- bool any_pending_io = nfds > 0;
+ bool any_pending_io = n_aiops > 0;
bool timeout_idlegc_delay;
adjustTimeoutForIdleGc(any_pending_io, &timeout, &timeout_idlegc_delay);
#endif
@@ -497,14 +572,14 @@ bool awaitCompletedTimeoutsOrIOPoll(CapIOManager *iomgr)
/* Check for I/O readiness, possibly waiting. */
#if defined(HAVE_DECL_PPOLL) && HAVE_DECL_PPOLL == 1
struct timespec ts, *timeout_ns = timeoutAsTimespec(timeout, &ts);
- int res = ppoll(poll_table, nfds, timeout_ns, NULL);
+ int res = ppoll(iomgr->aiop_poll_table, nfds, timeout_ns, NULL);
debugTrace(DEBUG_iomanager,
"ppoll(nfds = %lu, timeout = %" FMT_Int64 " ns) = %d",
(unsigned long) nfds, timeout, res);
#else
int timeout_ms = timeoutAsPollTimeout(timeout);
- int res = poll(poll_table, nfds, timeout_ms);
+ int res = poll(iomgr->aiop_poll_table, nfds, timeout_ms);
debugTrace(DEBUG_iomanager,
"poll(nfds = %lu, timeout = %d ms) = %d",
@@ -534,20 +609,23 @@ bool awaitCompletedTimeoutsOrIOPoll(CapIOManager *iomgr)
*/
} else if (errno == EINTR) {
- /* We got interrupted by a signal. */
-
-#if defined(RTS_USER_SIGNALS)
- /* Start any corresponding user signal handlers. If any, the run
- * queue will become non-empty and we will drop out of the loop.
- */
- startPendingSignalHandlers(iomgr->cap);
-#endif
-
- /* We can also be interrupted by the shutdown signal handler, which
- * will set sched_state and so cause us to drop out of the loop.
+ /* We got interrupted by a signal.
+ *
+ * Signals of note:
+ * + signals with a user handler: these will be delivered to us
+ * via the signal_fd_r pipe, which we handle by starting new
+ * threads, which will terminate the loop (non-empty run queue).
+ * + The backup ctl-c signal handler sets sched_state, and to avoid
+ * races, it also calls interruptIOManagerPoll (which causes
+ * interrupt_fd_r to become ready). We're in the EINTR case here
+ * so we know the signal arrived while we were waiting, so it is
+ * the getSchedState condition that will will terminate the loop.
+ * (If the signal arrives before the call to poll then we will
+ * notice interrupt_fd_r is ready and set interrupt=true, which
+ * will terminate the loop).
*
- * For any other interruption (e.g. timer) we will go round the
- * do-while loop again.
+ * For any other interruption we will go round the do-while loop
+ * again.
*/
} else {
@@ -585,11 +663,20 @@ void interruptIOManagerPoll(CapIOManager *iomgr USED_IF_PREEMPTION)
}
+#if defined(RTS_USER_SIGNALS)
+static void signal_handler(int sig, siginfo_t *info, void *p STG_UNUSED)
+{
+ int signal_fd_w = getCapability(0)->iomgr->shared->signal_fd_w;
+ writeSignalInfoToPipe(sig, info, signal_fd_w);
+}
+
+
int installUserSignalHandlerPoll(int sig, int spi, sigset_t *mask,
bool nocldstop)
{
- return installSignalHandler(sig, spi, mask, nocldstop, generic_handler);
+ return installSignalHandler(sig, spi, mask, nocldstop, signal_handler);
}
+#endif
/* Helper function to double the size of the aiop_table and aiop_poll_table.
@@ -602,14 +689,11 @@ static bool enlargeTables(CapIOManager *iomgr)
bool ok = enlargeClosureTable(iomgr->cap, &iomgr->aiop_table, newcapacity);
if (RTS_UNLIKELY(!ok)) return false;
- /* Update the auxiliary aiop_poll_table to match. The full_poll_table is
- * one bigger than the aiop_poll_table, since it has an extra entry at the
- * front for interrupt_fd_r, with no corresponding aiop. */
- iomgr->full_poll_table =
- stgReallocBytes(iomgr->full_poll_table,
- sizeof(struct pollfd) * (newcapacity+1),
+ /* Update the auxiliary aiop_poll_table to match. */
+ iomgr->aiop_poll_table =
+ stgReallocBytes(iomgr->aiop_poll_table,
+ sizeof(struct pollfd) * newcapacity,
"Poll.c: enlargeTables");
- iomgr->aiop_poll_table = iomgr->full_poll_table+1;
/* Initialise the new part of the aiop_poll_table */
struct pollfd *aiop_poll_table = iomgr->aiop_poll_table;
=====================================
rts/posix/Poll.h
=====================================
@@ -18,6 +18,7 @@
void initCapabilityIOManagerPoll(CapIOManager *iomgr);
void freeCapabilityIOManagerPoll(CapIOManager *iomgr);
+void startCapabilityIOManagerPoll(CapIOManager *iomgr);
/* Synchronous I/O and timer operations */
IOSubmitResult syncIOWaitReadyPoll(CapIOManager *iomgr, StgTSO *tso,
=====================================
rts/posix/Select.c
=====================================
@@ -55,6 +55,8 @@
#define TimeToLowResTimeRoundUp(t) (t)
#endif
+static void startPendingSignalHandlers(Capability *cap);
+
void initCapabilityIOManagerSelect(CapIOManager *iomgr)
{
iomgr->blocked_queue_hd = END_TSO_QUEUE;
@@ -524,10 +526,97 @@ awaitCompletedTimeoutsOrIOSelect(CapIOManager *iomgr, bool wait)
return !interrupt;
}
+
+/* -----------------------------------------------------------------------------
+ * Low-level signal handler
+ *
+ * Places the requested handler on a stack of pending handlers to be
+ * started up at the next context switch.
+ * -------------------------------------------------------------------------- */
+
+#define N_PENDING_HANDLERS 16
+
+siginfo_t pending_handler_buf[N_PENDING_HANDLERS];
+siginfo_t *next_pending_handler = pending_handler_buf;
+
+static void
+signal_handler(int sig STG_UNUSED, siginfo_t *info, void *p STG_UNUSED)
+{
+
+ /* Can't call allocate from here. Probably can't call malloc
+ either. However, we have to schedule a new thread somehow.
+
+ It's probably ok to request a context switch and allow the
+ scheduler to start the handler thread, but how do we
+ communicate this to the scheduler?
+
+ We need some kind of locking, but with low overhead (i.e. no
+ blocking signals every time around the scheduler).
+
+ Signal Handlers are atomic (i.e. they can't be interrupted), and
+ we can make use of this. We just need to make sure the
+ critical section of the scheduler can't be interrupted - the
+ only way to do this is to block signals. However, we can lower
+ the overhead by only blocking signals when there are any
+ handlers to run, i.e. the set of pending handlers is
+ non-empty.
+ */
+
+ /* We use a stack to store the pending signals. We can't
+ dynamically grow this since we can't allocate any memory from
+ within a signal handler.
+
+ Hence unfortunately we have to bomb out if the buffer
+ overflows. It might be acceptable to carry on in certain
+ circumstances, depending on the signal.
+ */
+
+ memcpy(next_pending_handler, info, sizeof(siginfo_t));
+
+ next_pending_handler++;
+
+ // stack full?
+ if (next_pending_handler == &pending_handler_buf[N_PENDING_HANDLERS]) {
+ errorBelch("too many pending signals");
+ stg_exit(EXIT_FAILURE);
+ }
+
+ interruptCapability(&MainCapability);
+
+}
+
int installUserSignalHandlerSelect(int sig, int spi, sigset_t *mask,
bool nocldstop)
{
- return installSignalHandler(sig, spi, mask, nocldstop, generic_handler);
+ return installSignalHandler(sig, spi, mask, nocldstop, signal_handler);
+}
+
+static void startPendingSignalHandlers(Capability *cap)
+{
+ if (next_pending_handler == pending_handler_buf) return;
+
+ // Temporarily mask all signals to stop the signal_handler from mutating
+ // next_pending_handler and pending_handler_buf while we are looking at
+ // and modifying them.
+ static sigset_t maskSignals, savedSignals;
+ sigfillset(&maskSignals);
+ pthread_sigmask(SIG_SETMASK, &maskSignals, &savedSignals);
+
+ while (next_pending_handler != pending_handler_buf) {
+
+ next_pending_handler--;
+
+ siginfo_t *info = next_pending_handler;
+ int sig = info->si_signo;
+ startSignalHandler(cap, sig, info);
+
+ // It's possible by the time the signal handler runs that the handler
+ // has changed: runHandlers will run whatever handler is set at the
+ // time (or none).
+ }
+
+ // Restore the signal mask.
+ pthread_sigmask(SIG_SETMASK, &savedSignals, NULL);
}
#endif /* IOMGR_ENABLED_SELECT */
=====================================
rts/posix/SelectBis.c
=====================================
@@ -26,6 +26,7 @@
#include "SelectBis.h"
+#include <unistd.h>
#include <errno.h>
#include <sys/select.h>
@@ -121,6 +122,17 @@ The CapIOManager structure for this I/O manager contains:
ClosureTable aiop_table;
StgTimeoutQueue *timeout_queue;
int interrupt_fd_r, interrupt_fd_w;
+ int n_special_fds;
+
+In addition to aiops, there are some special internal fds we have to monitor
+and respond to. In particular there is the interrupt_fd_r used to implement
+interruptIOManager. Whether this fd is used or not depends on configuration
+(use of interrupt_fd_r depends on HAVE_PREEMPTION). To enable multiple such
+fds and allow them to be optional we use the following scheme: we create
+special internal AsyncIOOps and put them into the aiop_table. These are
+treated much like other IOOpCodeWaitRead operations but with special handling
+for notification. The variable n_special_fds tracks how many such entries we
+have so that we can handle them specially but in a reasonably uniform manner.
******************************************************************************/
@@ -133,7 +145,20 @@ static int collectFdSets(CapIOManager *iomgr);
static void processBadFds(CapIOManager *iomgr);
static void reportSelectError(void) STG_NORETURN;
+enum SpecialFd {
+ InterruptFd, /* iomgr->interrupt_fd_r for interruptIOManager */
+ SignalFd /* iomgr->shared->signal_fd_r */
+};
+static void insertSpecialFd(CapIOManager *iomgr,
+ enum SpecialFd special_fd_kind,
+ int special_fd);
+static void processSpecialFd(CapIOManager *iomgr,
+ StgAsyncIOOp *aiop, bool *interrupt);
+
+/* Called early in RTS startup _before_ the storage manager is initialised,
+ * so this is not allowed to allocate on the GC heap.
+ */
void initCapabilityIOManagerSelectBis(CapIOManager *iomgr)
{
initClosureTable(&iomgr->aiop_table, ClosureTableCompact);
@@ -145,12 +170,21 @@ void initCapabilityIOManagerSelectBis(CapIOManager *iomgr)
/* Would never happen in a standalone process, but could plausibly happen
* if the RTS is used within another process that already has many open fds.
*/
- if (iomgr->interrupt_fd_r < 0 || iomgr->interrupt_fd_r >= (int)FD_SETSIZE ||
- iomgr->interrupt_fd_w < 0 || iomgr->interrupt_fd_w >= (int)FD_SETSIZE) {
+ if (iomgr->interrupt_fd_r >= (int)FD_SETSIZE ||
+ iomgr->interrupt_fd_w >= (int)FD_SETSIZE) {
barf("initCapabilityIOManagerSelectBis: fds out of select range");
}
#endif
+ iomgr->n_special_fds = 0
+#if defined(HAVE_PREEMPTION)
+ + 1
+#endif
+#if defined(RTS_USER_SIGNALS)
+ + (iomgr->cap->no == 0 ? 1 : 0)
+#endif
+ ;
+
iomgr->rfds = stgMallocBytes(sizeof (fd_set), "IOManagerSelectBis");
iomgr->wfds = stgMallocBytes(sizeof (fd_set), "IOManagerSelectBis");
@@ -158,8 +192,62 @@ void initCapabilityIOManagerSelectBis(CapIOManager *iomgr)
/* Signal handlers are shared across all caps. Only init once. */
if (iomgr->cap->no == 0) {
initUserSignals();
+ /* See Note [Non-racy signal handling using the self-pipe trick] */
+ newNonBlockingPipe(&iomgr->shared->signal_fd_r,
+ &iomgr->shared->signal_fd_w);
+ if (iomgr->shared->signal_fd_r >= (int)FD_SETSIZE ||
+ iomgr->shared->signal_fd_w >= (int)FD_SETSIZE) {
+ barf("initCapabilityIOManagerSelectBis: fds out of select range");
+ }
+ }
+#endif
+}
+
+
+/* Called late in RtsStartup, once heap allocation is ready. */
+void startCapabilityIOManagerSelectBis(CapIOManager *iomgr)
+{
+ /* ClosureTable allocates on the heap, so we can do it here not in init */
+ if (iomgr->n_special_fds > 0) {
+ bool ok = enlargeClosureTable(iomgr->cap, &iomgr->aiop_table,
+ iomgr->n_special_fds);
+ CHECK(ok);
+ }
+
+#if defined(HAVE_PREEMPTION)
+ insertSpecialFd(iomgr, InterruptFd, iomgr->interrupt_fd_r);
+#endif
+#if defined(RTS_USER_SIGNALS)
+ if (iomgr->cap->no == 0) {
+ /* See Note [Non-racy signal handling using the self-pipe trick] */
+ insertSpecialFd(iomgr, SignalFd, iomgr->shared->signal_fd_r);
}
#endif
+ ASSERT(iomgr->n_special_fds == sizeClosureTable(&iomgr->aiop_table));
+}
+
+
+/* For special fds, we add a dummy entry to the aiop_table and a
+ * corresponding real entry to the aiop_poll_table.
+ */
+static void insertSpecialFd(CapIOManager *iomgr,
+ enum SpecialFd special_fd_kind,
+ int special_fd)
+{
+ StgAsyncIOOp *aiop = (StgAsyncIOOp *)allocate(iomgr->cap,
+ sizeofW(StgAsyncIOOp));
+ SET_HDR(aiop, &stg_ASYNCIOOP_info, CCS_SYSTEM);
+ aiop->notify.unused = &stg_ASYNCIO_LIVE0_closure;
+ aiop->live = &stg_ASYNCIO_LIVE0_closure;
+ aiop->capno = iomgr->cap->no;
+ aiop->notify_type = NotifyInternal;
+ aiop->outcome = IOOpOutcomeInFlight;
+ aiop->operation = IOOpCodeWaitRead;
+ aiop->fd = special_fd;
+ aiop->internal = special_fd_kind;
+
+ int ix = insertClosureTable(iomgr->cap, &iomgr->aiop_table, aiop);
+ aiop->index = ix;
}
@@ -168,6 +256,8 @@ void freeCapabilityIOManagerSelectBis(CapIOManager *iomgr)
#if defined(RTS_USER_SIGNALS)
if (iomgr->cap->no == 0) {
finiUserSignals();
+ close(iomgr->shared->signal_fd_r);
+ close(iomgr->shared->signal_fd_w);
}
#endif
stgFree(iomgr->rfds);
@@ -316,20 +406,9 @@ static bool processIOCompletions(CapIOManager *iomgr, int ncompletions)
ASSERT(ncompletions > 0);
bool interrupt = false;
-#if defined(HAVE_PREEMPTION)
- /* If the interrupt_fd_r is ready, collect it */
- if (FD_ISSET(iomgr->interrupt_fd_r, iomgr->rfds)) {
- collectFdWakeup(iomgr->interrupt_fd_r);
- ncompletions--;
- interrupt = true;
- debugTrace(DEBUG_iomanager, "Received interrupt in poll I/O manager");
- }
-#endif
int i = 0;
- while (ncompletions > 0) {
- ASSERT(i < sizeClosureTable(&iomgr->aiop_table));
-
+ while (ncompletions > 0 && i < sizeClosureTable(&iomgr->aiop_table)) {
StgAsyncIOOp *aiop = indexClosureTable(&iomgr->aiop_table, i);
int fd = aiop->fd;
enum IOOpCode op = aiop->operation;
@@ -339,11 +418,17 @@ static bool processIOCompletions(CapIOManager *iomgr, int ncompletions)
if (op == IOOpCodeWaitRead ? FD_ISSET(fd, iomgr->rfds)
: FD_ISSET(fd, iomgr->wfds)) {
- aiop->outcome = IOOpOutcomeSuccess;
- aiop->result = 0;
- removeFromTables(iomgr, i);
- notifyIOCompletion(iomgr, aiop);
- ncompletions--;
+ if (aiop->notify_type == NotifyInternal) {
+ processSpecialFd(iomgr, aiop, &interrupt);
+ ncompletions--;
+ i++;
+ } else {
+ aiop->outcome = IOOpOutcomeSuccess;
+ aiop->result = 0;
+ removeFromTables(iomgr, i);
+ notifyIOCompletion(iomgr, aiop);
+ ncompletions--;
+ }
} else {
/* You'd expect incrementing the table index to be unconditional,
* but we don't increment the index if we did process the entry,
@@ -353,10 +438,42 @@ static bool processIOCompletions(CapIOManager *iomgr, int ncompletions)
i++;
}
}
+ ASSERT(ncompletions == 0);
return interrupt;
}
+static void processSpecialFd(CapIOManager *iomgr,
+ StgAsyncIOOp *aiop, bool *interrupt)
+{
+ ASSERT(aiop->notify_type == NotifyInternal);
+ enum SpecialFd special_fd_kind = aiop->internal;
+ int fd = aiop->fd;
+
+ switch (special_fd_kind) {
+#if defined(HAVE_PREEMPTION)
+ case InterruptFd:
+ ASSERT(fd == iomgr->interrupt_fd_r);
+ collectFdWakeup(fd);
+ *interrupt = true;
+ debugTrace(DEBUG_iomanager, "Received interrupt in poll I/O manager");
+ break;
+#endif
+#if defined(RTS_USER_SIGNALS)
+ /* See Note [Non-racy signal handling using the self-pipe trick] */
+ case SignalFd:
+ ASSERT(fd == iomgr->shared->signal_fd_r);
+ int sig; siginfo_t info;
+ readSignalInfoFromPipe(&sig, &info, fd);
+ startSignalHandler(iomgr->cap, sig, &info);
+ debugTrace(DEBUG_iomanager,
+ "Dispatched signal %d in I/O manager", sig);
+ break;
+#endif
+ }
+}
+
+
void pollCompletedTimeoutsOrIOSelectBis(CapIOManager *iomgr)
{
if (!isEmptyTimeoutQueue(iomgr->timeout_queue)) {
@@ -397,10 +514,6 @@ void pollCompletedTimeoutsOrIOSelectBis(CapIOManager *iomgr)
reportSelectError();
}
}
-
-#if defined(RTS_USER_SIGNALS)
- startPendingSignalHandlers(iomgr->cap);
-#endif
}
@@ -440,8 +553,10 @@ bool awaitCompletedTimeoutsOrIOSelectBis(CapIOManager *iomgr)
*/
Time timeout = timeoutWaitTime(iomgr, wait, now);
- if (RTS_UNLIKELY(timeout == -1 &&
- isEmptyClosureTable(&iomgr->aiop_table))) {
+ int nfds = sizeClosureTable(&iomgr->aiop_table);
+ int n_aiops = nfds - iomgr->n_special_fds;
+
+ if (RTS_UNLIKELY(timeout == -1 && n_aiops == 0)) {
/* See Note [Deadlock detection without idle GC] */
interrupt = notifyIdleGcDeadlock();
if (interrupt) break;
@@ -453,7 +568,7 @@ bool awaitCompletedTimeoutsOrIOSelectBis(CapIOManager *iomgr)
* time, and if that timeout is reached then we schedule an idle GC
* (see handleIdleGcTimeout below).
*/
- bool any_pending_io = maxfd >= 0;
+ bool any_pending_io = n_aiops > 0;
bool timeout_idlegc_delay;
adjustTimeoutForIdleGc(any_pending_io, &timeout, &timeout_idlegc_delay);
#endif
@@ -479,7 +594,7 @@ bool awaitCompletedTimeoutsOrIOSelectBis(CapIOManager *iomgr)
} else if (res > 0) {
int ncompletions = res;
- ASSERT(ncompletions <= sizeClosureTable(&iomgr->aiop_table)+1);
+ ASSERT(ncompletions <= nfds);
interrupt = processIOCompletions(iomgr, ncompletions);
/* We do _not_ need to check for timeouts again here. We know that
* select() returned due to I/O being ready before the timeout. If
@@ -489,20 +604,23 @@ bool awaitCompletedTimeoutsOrIOSelectBis(CapIOManager *iomgr)
*/
} else if (errno == EINTR) {
- /* We got interrupted by a signal. */
-
-#if defined(RTS_USER_SIGNALS)
- /* Start any corresponding user signal handlers. If any, the run
- * queue will become non-empty and we will drop out of the loop.
- */
- startPendingSignalHandlers(iomgr->cap);
-#endif
-
- /* We can also be interrupted by the shutdown signal handler, which
- * will set sched_state and so cause us to drop out of the loop.
+ /* We got interrupted by a signal.
*
- * For any other interruption (e.g. timer) we will go round the
- * do-while loop again.
+ * Signals of note:
+ * + signals with a user handler: these will be delivered to us
+ * via the signal_fd_r pipe, which we handle by starting new
+ * threads, which will terminate the loop (non-empty run queue).
+ * + The backup ctl-c signal handler sets sched_state, and to avoid
+ * races, it also calls interruptIOManagerPoll (which causes
+ * interrupt_fd_r to become ready). We're in the EINTR case here
+ * so we know the signal arrived while we were waiting, so it is
+ * the getSchedState condition that will will terminate the loop.
+ * (If the signal arrives before the call to poll then we will
+ * notice interrupt_fd_r is ready and set interrupt=true, which
+ * will terminate the loop).
+ *
+ * For any other interruption we will go round the do-while loop
+ * again.
*/
} else if (errno == EBADF) {
@@ -595,11 +713,20 @@ void interruptIOManagerSelectBis(CapIOManager *iomgr USED_IF_PREEMPTION)
}
+#if defined(RTS_USER_SIGNALS)
+static void signal_handler(int sig, siginfo_t *info, void *p STG_UNUSED)
+{
+ int signal_fd_w = getCapability(0)->iomgr->shared->signal_fd_w;
+ writeSignalInfoToPipe(sig, info, signal_fd_w);
+}
+
+
int installUserSignalHandlerSelectBis(int sig, int spi, sigset_t *mask,
bool nocldstop)
{
- return installSignalHandler(sig, spi, mask, nocldstop, generic_handler);
+ return installSignalHandler(sig, spi, mask, nocldstop, signal_handler);
}
+#endif
/* Helper function to double the size of the aiop_table.
@@ -647,15 +774,6 @@ static int collectFdSets(CapIOManager *iomgr)
FD_ZERO(iomgr->rfds);
FD_ZERO(iomgr->wfds);
-#if defined(HAVE_PREEMPTION)
- /* We're always interested in our interrupt fd */
- {
- int fd = iomgr->interrupt_fd_r;
- maxfd = (fd > maxfd) ? fd : maxfd;
- FD_SET(fd, iomgr->rfds);
- }
-#endif
-
for (int ix = 0; ix < nentries; ix++) {
StgAsyncIOOp *aiop = indexClosureTable(&iomgr->aiop_table, ix);
int fd = aiop->fd;
=====================================
rts/posix/SelectBis.h
=====================================
@@ -38,6 +38,7 @@
void initCapabilityIOManagerSelectBis(CapIOManager *iomgr);
void freeCapabilityIOManagerSelectBis(CapIOManager *iomgr);
+void startCapabilityIOManagerSelectBis(CapIOManager *iomgr);
/* Synchronous I/O and timer operations */
IOSubmitResult syncIOWaitReadySelectBis(CapIOManager *iomgr, StgTSO *tso,
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/d1925b83c8105e94d7498a4bec4829…
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/d1925b83c8105e94d7498a4bec4829…
You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help
1
0
[Git][ghc/ghc][wip/mangoiv/ci-stages] fixup! ci: build and test stage
by Magnus (@MangoIV) 28 Oct '26
by Magnus (@MangoIV) 28 Oct '26
28 Oct '26
Magnus pushed to branch wip/mangoiv/ci-stages at Glasgow Haskell Compiler / GHC
Commits:
e3098d06 by mangoiv at 2026-10-28T10:59:42+01:00
fixup! ci: build and test stage
- - - - -
3 changed files:
- .gitlab/ci.sh
- .gitlab/generate-ci/gen_ci.hs
- hadrian/src/Oracles/TestSettings.hs
Changes:
=====================================
.gitlab/ci.sh
=====================================
@@ -693,6 +693,9 @@ function test_hadrian_stage1() {
function test_hadrian() {
+ # if a bindist tar exists, the variable $bindist_tar will be set
+ # that way we can later check for its existance to find whether
+ # we are running with a bindist.
find_bindist_tar
dist_dir="_build/bindist"
@@ -777,8 +780,8 @@ function test_hadrian() {
run_hadrian \
test \
--summary-junit=./junit.xml \
- --test-compiler="${test_compiler}" \
--test-have-intree-files \
+ --test-compiler="${test_compiler}" \
${TEST_WAYS[@]/#/--test-way=} \
"runtest.opts+=${RUNTEST_ARGS:-}" \
"runtest.opts+=--unexpected-output-dir=$TOP/unexpected-test-output" \
=====================================
.gitlab/generate-ci/gen_ci.hs
=====================================
@@ -872,6 +872,7 @@ job arch opsys buildConfig =
, "bash .gitlab/ci.sh configure" -- configure needs to be run because hadrian gets its config like that
, "bash .gitlab/ci.sh test_hadrian" ]
| otherwise
+ -- see Note [Workaround for docker permissions]
= [ "sudo chown ghc:ghc -R ." | Linux {} <- [opsys]] ++
[ ".gitlab/ci.sh setup"
, ".gitlab/ci.sh configure"
@@ -886,6 +887,7 @@ job arch opsys buildConfig =
]
| otherwise
= [ "find libraries -name config.sub -exec cp config.sub {} \\;" | Darwin == opsys ] ++
+ -- see Note [Workaround for docker permissions]
[ "sudo chown ghc:ghc -R ." | Linux {} <- [opsys]] ++
[ ".gitlab/ci.sh setup"
, ".gitlab/ci.sh configure"
@@ -984,6 +986,19 @@ job arch opsys buildConfig =
jobAllowFailure = False
+{-
+Note [Workaround for docker permissions]
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+In GHC's CI, (docker) containers try to run build and tests as the user ghc
+which is unusual. Sometimes we have files in the file tree that are owned
+by root (which is the user that containers usually run as) so we have to
+chown them.
+
+Since running as user ghc in the CI yields virtually no extra protection in
+any situation, we will remove this in the future and replace it by better
+non-filesystem-based virtualisation.
+-}
+
---------------------------------------------------------------------------
-- Job Modifiers
---------------------------------------------------------------------------
=====================================
hadrian/src/Oracles/TestSettings.hs
=====================================
@@ -154,7 +154,7 @@ getTestExePath testGhc pkg = do
bindir <- getBinaryDirectory testGhc
compiler_path <- getCompilerPath testGhc
cross <- getBooleanSetting TestCrossCompiling
- tgt <- getHostTarget -- FIXME: is this right -- ask rodrigo about the targettarget and query/get versions of this function
+ tgt <- getHostTarget
let cross_prefix = if cross then dropWhileEnd ((/=) '-') (takeFileName compiler_path) else ""
-- get relative path for the given program in the given stage
liftIO $ makeAbsolute (bindir </> (cross_prefix ++ programBasename pkg) <.> exe (tgtArchOs tgt))
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/e3098d062cd2852da936e000eac7285…
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/e3098d062cd2852da936e000eac7285…
You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help
1
0