[Git][ghc/ghc][wip/semaphore-v2] Update to semaphore-compat 2.0.0 using v2 of the protocol
by Zubin (＠wz1000) 19 May '26
by Zubin (＠wz1000) 19 May '26
19 May '26
Zubin pushed to branch wip/semaphore-v2 at Glasgow Haskell Compiler / GHC
Commits:
34171a94 by Zubin Duggal at 2026-05-19T15:26:42+05:30
Update to semaphore-compat 2.0.0 using v2 of the protocol
On Linux and other POSIX platforms, GHC's -jsem jobserver client now
speaks v2 of the semaphore-compat protocol, which uses Unix domain
sockets in place of POSIX named semaphores. This avoids the libc-ABI
issues that affected the old implementation. Windows is unaffected
and continues to use the v1 protocol (Win32 named semaphores); its
reported protocol version remains v1.
When GHC receives a -jsem name whose protocol version it does not
support, it emits a -Wsemaphore-version-mismatch warning and falls
back to -j<N> rather than crashing. ghc --info exposes the supported
version in a new "Semaphore version" entry so cabal-install can detect
a mismatch before invoking GHC.
Users on a cabal-install that predates the v2 update will continue to
build successfully on Linux/POSIX, but will lose the cross-process
-jsem coordination and fall back to -j<N> per GHC invocation. Users
must upgrade to a cabal-install that supports protocol v2 to recover
full parallelism.
Also fix a leak in cleanupSem (#27253): cleanupSem used to snapshot
heldTokens and release them before killing the loop, while the loop's
in-flight acquire/release children could still be mutating it.
Cleanup now runs inside the loop's own exit handler, after draining
the active child via a new activeChild TVar, so the snapshot has no
concurrent mutator.
See also:
- GHC proposal amendment: https://github.com/ghc-proposals/ghc-proposals/pull/673
- cabal-install patch: https://github.com/haskell/cabal/pull/11628
- semaphore-compat MR: https://gitlab.haskell.org/ghc/semaphore-compat/-/merge_requests/8
Bump semaphore-compat submodule to 2.0.0
Fixes #25087 and #27253
- - - - -
19 changed files:
- + changelog.d/jobserver-leak-fix
- + changelog.d/semaphore-v2
- compiler/GHC/Driver/Errors/Ppr.hs
- compiler/GHC/Driver/Errors/Types.hs
- compiler/GHC/Driver/Flags.hs
- compiler/GHC/Driver/MakeAction.hs
- compiler/GHC/Driver/MakeSem.hs
- compiler/GHC/Driver/Session.hs
- compiler/GHC/Types/Error/Codes.hs
- docs/users_guide/using-warnings.rst
- docs/users_guide/using.rst
- hadrian/cabal.project
- hadrian/hadrian.cabal
- hadrian/src/Flavour.hs
- hadrian/src/Rules/Generate.hs
- hadrian/src/Settings/Packages.hs
- hadrian/stack.yaml
- libraries/semaphore-compat
- testsuite/tests/diagnostic-codes/codes.stdout
Changes:
=====================================
changelog.d/jobserver-leak-fix
=====================================
@@ -0,0 +1,8 @@
+section: compiler
+issues: #27253
+mrs: !15729
+synopsis:
+ Fix a token leak in the ``-jsem`` jobserver shutdown path
+description:
+ A build interrupted by Ctrl-C while a ``-jsem`` token transfer was in
+ flight could leak that token.
=====================================
changelog.d/semaphore-v2
=====================================
@@ -0,0 +1,30 @@
+section: compiler
+issues: #25087
+mrs: !15729
+synopsis:
+ Update to semaphore-compat 2.0.0 (``-jsem`` protocol v2)
+description:
+ On Linux and other POSIX platforms, GHC's ``-jsem`` jobserver client
+ now speaks v2 of the semaphore-compat protocol, which uses Unix
+ domain sockets in place of POSIX named semaphores. This avoids the
+ libc-ABI issues that affected the old implementation. Windows is
+ unaffected and continues to use the v1 protocol (Win32 named
+ semaphores); its reported protocol version remains v1.
+
+ When GHC receives a ``-jsem`` name whose protocol version it does not
+ support, it now emits a ``-Wsemaphore-version-mismatch`` warning and
+ falls back to ``-j1`` rather than crashing. ``ghc --info`` exposes the
+ supported version in a new ``"Semaphore version"`` entry so
+ cabal-install can detect a mismatch before invoking GHC.
+
+ Users on a ``cabal-install`` that predates the v2 update will continue
+ to build successfully, but on Linux/POSIX will lose the cross-process
+ ``-jsem`` coordination and fall back to ``-j1`` per GHC invocation.
+ To recover full parallelism, upgrade to a ``cabal-install`` that
+ supports protocol v2.
+
+ See also:
+
+ - the `GHC proposal amendment <https://github.com/ghc-proposals/ghc-proposals/pull/673>`_
+ - the `cabal-install patch <https://github.com/haskell/cabal/pull/11628>`_
+ - the `semaphore-compat library MR <https://gitlab.haskell.org/ghc/semaphore-compat/-/merge_requests/8>`_
=====================================
compiler/GHC/Driver/Errors/Ppr.hs
=====================================
@@ -24,6 +24,8 @@ import GHC.Types.Hint
import GHC.Types.SrcLoc
import Data.Version
+import System.Semaphore
+ ( SemaphoreError(..), getSemaphoreProtocolVersion )
import Language.Haskell.Syntax.Decls (RuleDecl(..))
import GHC.Tc.Errors.Types (TcRnMessage)
import GHC.HsToCore.Errors.Types (DsMessage)
@@ -90,6 +92,20 @@ instance Diagnostic GhcMessage where
instance HasDefaultDiagnosticOpts DriverMessageOpts where
defaultOpts = DriverMessageOpts (defaultDiagnosticOpts @PsMessage) (defaultDiagnosticOpts @IfaceMessage)
+pprSemaphoreError :: SemaphoreError -> SDoc
+pprSemaphoreError = \case
+ SemaphoreAlreadyExists nm ->
+ text "a semaphore named" <+> quotes (text nm) <+> text "already exists"
+ SemaphoreDoesNotExist nm ->
+ text "no semaphore named" <+> quotes (text nm)
+ SemaphoreIncompatibleVersion got want ->
+ text "protocol version mismatch (got v"
+ <> int (getSemaphoreProtocolVersion got)
+ <> text ", supported v"
+ <> int (getSemaphoreProtocolVersion want) <> text ")"
+ SemaphoreOtherError ioe ->
+ text (show ioe)
+
instance Diagnostic DriverMessage where
type DiagnosticOpts DriverMessage = DriverMessageOpts
diagnosticMessage opts = \case
@@ -282,6 +298,10 @@ instance Diagnostic DriverMessage where
-> mkSimpleDecorated $
vcat [ text "The following modules are missing a linkable which is needed for creating a library:"
, nest 2 $ hcat (map ppr mods) ]
+ DriverSemaphoreOpenFailure err
+ -> mkSimpleDecorated $
+ text "Failed to open -jsem semaphore:" <+> pprSemaphoreError err <>
+ text "; ignoring -jsem and compiling sequentially."
diagnosticReason = \case
DriverUnknownMessage m
@@ -355,6 +375,8 @@ instance Diagnostic DriverMessage where
-> WarningWithoutFlag
DriverMissingLinkableForModule {}
-> ErrorWithoutFlag
+ DriverSemaphoreOpenFailure {}
+ -> WarningWithFlag Opt_WarnSemaphoreOpenFailure
diagnosticHints = \case
DriverUnknownMessage m
@@ -430,5 +452,14 @@ instance Diagnostic DriverMessage where
-> noHints
DriverMissingLinkableForModule {}
-> noHints
+ DriverSemaphoreOpenFailure (SemaphoreIncompatibleVersion received supported)
+ | received < supported
+ -> [UnknownHint (text "The parent process (e.g. cabal-install) uses an older semaphore protocol."
+ $$ text "Upgrading cabal-install may resolve this.")]
+ | otherwise
+ -> [UnknownHint (text "The parent process (e.g. cabal-install) uses a newer semaphore protocol."
+ $$ text "Upgrading GHC may resolve this.")]
+ DriverSemaphoreOpenFailure {}
+ -> noHints
diagnosticCode = constructorCode @GHC
=====================================
compiler/GHC/Driver/Errors/Types.hs
=====================================
@@ -37,6 +37,7 @@ import qualified GHC.LanguageExtensions as LangExt
import GHC.Generics ( Generic )
+import System.Semaphore ( SemaphoreError )
import GHC.Tc.Errors.Types
import GHC.Iface.Errors.Types
@@ -419,6 +420,13 @@ data DriverMessage where
DriverMissingLinkableForModule :: ![Module] -> DriverMessage
+ {-| DriverSemaphoreOpenFailure is a warning that occurs when GHC fails to
+ open the semaphore specified by @-jsem@, e.g. the socket does not
+ exist, the protocol version is incompatible, or a system error
+ occurred. GHC ignores @-jsem@ and compiles sequentially.
+ -}
+ DriverSemaphoreOpenFailure :: !SemaphoreError -> DriverMessage
+
deriving instance Generic DriverMessage
data DriverMessageOpts =
=====================================
compiler/GHC/Driver/Flags.hs
=====================================
@@ -1115,6 +1115,7 @@ data WarningFlag =
| Opt_WarnUnusableUnpackPragmas -- ^ @since 9.14
| Opt_WarnPatternNamespaceSpecifier -- ^ @since 9.14
| Opt_WarnUnrecognisedModifiers -- ^ @since 10.0
+ | Opt_WarnSemaphoreOpenFailure -- Since 10.0.1
deriving (Eq, Ord, Show, Enum, Bounded)
-- | Return the names of a WarningFlag
@@ -1237,6 +1238,7 @@ warnFlagNames wflag = case wflag of
Opt_WarnUnusableUnpackPragmas -> "unusable-unpack-pragmas" :| []
Opt_WarnPatternNamespaceSpecifier -> "pattern-namespace-specifier" :| []
Opt_WarnUnrecognisedModifiers -> "unrecognised-modifiers" :| []
+ Opt_WarnSemaphoreOpenFailure -> "semaphore-open-failure" :| []
-- -----------------------------------------------------------------------------
-- Standard sets of warning options
@@ -1383,7 +1385,8 @@ standardWarnings -- see Note [Documenting warning flags]
Opt_WarnDeprecatedPragmas,
Opt_WarnRuleLhsEqualities,
Opt_WarnUnusableUnpackPragmas,
- Opt_WarnUnrecognisedModifiers
+ Opt_WarnUnrecognisedModifiers,
+ Opt_WarnSemaphoreOpenFailure
]
-- | Things you get with @-W@.
=====================================
compiler/GHC/Driver/MakeAction.hs
=====================================
@@ -28,6 +28,14 @@ import GHC.Driver.Errors.Types
import GHC.Driver.Messager
import GHC.Driver.MakeSem
+import System.Semaphore
+ ( SemaphoreError, SemaphoreIdentifier )
+
+import GHC.Driver.Config.Diagnostic ( initDiagOpts, initPrintConfig )
+import GHC.Driver.Errors ( printOrThrowDiagnostics )
+import GHC.Types.Error ( singleMessage )
+import GHC.Types.SrcLoc ( noSrcSpan )
+import GHC.Utils.Error ( mkPlainMsgEnvelope )
import GHC.Utils.Logger
import GHC.Utils.TmpFs
@@ -49,7 +57,7 @@ mkWorkerLimit :: DynFlags -> IO WorkerLimit
mkWorkerLimit dflags =
case parMakeCount dflags of
Nothing -> pure $ num_procs 1
- Just (ParMakeSemaphore h) -> pure (JSemLimit (SemaphoreName h))
+ Just (ParMakeSemaphore h) -> pure (JSemLimit h)
Just ParMakeNumProcessors -> num_procs <$> getNumProcessors
Just (ParMakeThisMany n) -> pure $ num_procs n
where
@@ -65,8 +73,8 @@ isWorkerLimitSequential (JSemLimit {}) = False
data WorkerLimit
= NumProcessorsLimit Int
| JSemLimit
- SemaphoreName
- -- ^ Semaphore name to use
+ SemaphoreIdentifier
+ -- ^ Semaphore identifier from @-jsem@
deriving Eq
-- | Environment used when compiling a module
@@ -122,17 +130,24 @@ runNjobsAbstractSem n_jobs action = do
resetNumCapabilities = set_num_caps n_capabilities
MC.bracket_ updNumCapabilities resetNumCapabilities $ action asem
-runWorkerLimit :: WorkerLimit -> (AbstractSem -> IO a) -> IO a
+runWorkerLimit :: Logger -> DynFlags -> WorkerLimit -> (AbstractSem -> IO a) -> IO a
#if defined(wasm32_HOST_ARCH)
-runWorkerLimit _ action = do
+runWorkerLimit _logger _dflags _ action = do
lock <- newMVar ()
action $ AbstractSem (takeMVar lock) (putMVar lock ())
#else
-runWorkerLimit worker_limit action = case worker_limit of
+runWorkerLimit logger dflags worker_limit action = case worker_limit of
NumProcessorsLimit n_jobs ->
runNjobsAbstractSem n_jobs action
- JSemLimit sem ->
- runJSemAbstractSem sem action
+ JSemLimit sem_ident -> do
+ result <- MC.try @SemaphoreError $ runJSemAbstractSem sem_ident action
+ case result of
+ Right a -> return a
+ Left err -> do
+ let diag = DriverSemaphoreOpenFailure err
+ msg = singleMessage $ mkPlainMsgEnvelope (initDiagOpts dflags) noSrcSpan diag
+ printOrThrowDiagnostics logger (initPrintConfig dflags) (initDiagOpts dflags) (GhcDriverMessage <$> msg)
+ runNjobsAbstractSem 1 action
#endif
-- | Build and run a pipeline
@@ -159,7 +174,7 @@ runParPipelines worker_limit plugin_hsc_env diag_wrapper mHscMessager all_pipeli
thread_safe_logger <- liftIO $ makeThreadSafe (hsc_logger plugin_hsc_env)
let thread_safe_hsc_env = plugin_hsc_env { hsc_logger = thread_safe_logger }
- runWorkerLimit worker_limit $ \abstract_sem -> do
+ runWorkerLimit (hsc_logger plugin_hsc_env) (hsc_dflags plugin_hsc_env) worker_limit $ \abstract_sem -> do
let env = MakeEnv { hsc_env = thread_safe_hsc_env
, withLogger = withParLog log_queue_queue_var
, compile_sem = abstract_sem
=====================================
compiler/GHC/Driver/MakeSem.hs
=====================================
@@ -9,9 +9,6 @@ module GHC.Driver.MakeSem
-- by a system semaphore (Posix/Windows)
runJSemAbstractSem
- -- * System semaphores
- , Semaphore, SemaphoreName(..)
-
-- * Abstract semaphores
, AbstractSem(..)
, withAbstractSem
@@ -46,11 +43,14 @@ import Debug.Trace
-- available from the semaphore.
data Jobserver
= Jobserver
- { jSemaphore :: !Semaphore
+ { jSemaphore :: !ClientSemaphore
-- ^ The semaphore which controls available resources
, jobs :: !(TVar JobResources)
-- ^ The currently pending jobs, and the resources
-- obtained from the semaphore
+ , activeChild :: !(TVar (Maybe (ThreadId, TMVar (Maybe MC.SomeException))))
+ -- ^ Handle on the current acquire thread (if any). The loop's exit
+ -- handler reads this to drain a still-running child on shutdown.
}
data JobserverOptions
@@ -81,6 +81,9 @@ data JobResources
, jobsWaiting :: !(OrdList (TMVar ()))
-- ^ Pending jobs waiting on a token, the job will be blocked on the TMVar so putting into
-- the TMVar will allow the job to continue.
+ , heldTokens :: [SemaphoreToken]
+ -- ^ Actual semaphore tokens (for release/cleanup).
+ -- Length should equal tokensOwned - 1 (the implicit token has no SemaphoreToken).
}
instance Outputable JobResources where
@@ -93,9 +96,9 @@ instance Outputable JobResources where
] )
-- | Add one new token.
-addToken :: JobResources -> JobResources
-addToken jobs@( Jobs { tokensOwned = owned, tokensFree = free })
- = jobs { tokensOwned = owned + 1, tokensFree = free + 1 }
+addToken :: SemaphoreToken -> JobResources -> JobResources
+addToken tok jobs@( Jobs { tokensOwned = owned, tokensFree = free, heldTokens = toks })
+ = jobs { tokensOwned = owned + 1, tokensFree = free + 1, heldTokens = tok : toks }
-- | Free one token.
addFreeToken :: JobResources -> JobResources
@@ -111,12 +114,14 @@ removeFreeToken jobs@( Jobs { tokensFree = free })
(text "removeFreeToken:" <+> ppr free)
$ jobs { tokensFree = free - 1 }
--- | Return one owned token.
-removeOwnedToken :: JobResources -> JobResources
-removeOwnedToken jobs@( Jobs { tokensOwned = owned })
+-- | Return one owned token, extracting the 'SemaphoreToken' for release.
+removeOwnedToken :: JobResources -> (SemaphoreToken, JobResources)
+removeOwnedToken jobs@( Jobs { tokensOwned = owned, heldTokens = toks })
= assertPpr (owned > 1)
(text "removeOwnedToken:" <+> ppr owned)
- $ jobs { tokensOwned = owned - 1 }
+ $ case toks of
+ (t:rest) -> (t, jobs { tokensOwned = owned - 1, heldTokens = rest })
+ [] -> panic "removeOwnedToken: no held tokens"
-- | Add one new job to the end of the list of pending jobs.
addJob :: TMVar () -> JobResources -> JobResources
@@ -143,7 +148,7 @@ data JobserverAction
= Idle
-- | A thread is waiting for a token on the semaphore.
| Acquiring
- { activeWaitId :: WaitId
+ { activeThreadId :: ThreadId
, threadFinished :: TMVar (Maybe MC.SomeException) }
-- | Retrieve the 'TMVar' that signals if the current thread has finished,
@@ -189,17 +194,30 @@ releaseJob jobs_tvar = do
return ((), addFreeToken jobs)
--- | Release all tokens owned from the semaphore (to clean up
--- the jobserver at the end).
-cleanupJobserver :: Jobserver -> IO ()
-cleanupJobserver (Jobserver { jSemaphore = sem
- , jobs = jobs_tvar })
- = do
- Jobs { tokensOwned = owned } <- readTVarIO jobs_tvar
- let toks_to_release = owned - 1
- -- Subtract off the implicit token: whoever spawned the ghc process
- -- in the first place is responsible for that token.
- releaseSemaphore sem toks_to_release
+-- | Kill the current acquire thread, if any, and wait for it to exit.
+--
+-- Called from the jobserver loop's exit handler, which runs masked.
+-- Relies on the invariant from 'acquireThread' that a forked child
+-- always fills its 'threadFinished' TMVar before it dies; this is what
+-- lets the 'takeTMVar' below terminate after the 'killThread'.
+drainActiveChild :: Jobserver -> IO ()
+drainActiveChild (Jobserver { activeChild = active_tvar }) = do
+ mb <- readTVarIO active_tvar
+ for_ mb $ \(tid, tmv) -> do
+ killThread tid
+ void $ atomically (takeTMVar tmv)
+ atomically $ writeTVar active_tvar Nothing
+
+-- | Release every token currently in 'heldTokens'.
+--
+-- Called from the jobserver loop's exit handler, which runs masked,
+-- after 'drainActiveChild': no other thread is mutating 'JobResources'
+-- at this point.
+releaseAllHeld :: Jobserver -> IO ()
+releaseAllHeld (Jobserver { jobs = jobs_tvar }) = do
+ Jobs { heldTokens = toks } <- readTVarIO jobs_tvar
+ forM_ toks $ \t ->
+ void $ MC.try @MC.SomeException (releaseSemaphoreToken t)
-- | Dispatch the available tokens acquired from the semaphore
-- to the pending jobs in the job server.
@@ -252,7 +270,7 @@ tracedAtomically origin act = do
return a
renderJobResources :: String -> JobResources -> String
-renderJobResources origin (Jobs own free pending) = showSDocUnsafe $ renderJSON $
+renderJobResources origin (Jobs own free pending _heldToks) = showSDocUnsafe $ renderJSON $
JSObject [ ("name", JSString origin)
, ("owned", JSInt own)
, ("free", JSInt free)
@@ -262,61 +280,68 @@ renderJobResources origin (Jobs own free pending) = showSDocUnsafe $ renderJSON
-- | Spawn a new thread that waits on the semaphore in order to acquire
-- an additional token.
+--
+-- The child is forked masked so the only async-exception delivery point
+-- is the interruptible 'waitOnSemaphore'; the STM commit afterwards then
+-- always runs to completion, so 'threadFinished' is always filled.
+--
+-- The (tid, threadFinished) pair is also published to 'activeChild' so
+-- shutdown can drain the child even after the in-loop 'JobserverState'
+-- is gone.
acquireThread :: Jobserver -> IO JobserverAction
-acquireThread (Jobserver { jSemaphore = sem, jobs = jobs_tvar }) = do
+acquireThread (Jobserver { jSemaphore = sem, jobs = jobs_tvar, activeChild = active_tvar }) = do
threadFinished_tmvar <- newEmptyTMVarIO
- let
- wait_result_action :: Either MC.SomeException Bool -> IO ()
- wait_result_action wait_res =
+ tid <- MC.mask_ $ do
+ tid <- forkIO $ do
+ wait_res <- MC.try @_ @MC.SomeException $ waitOnSemaphore sem
tracedAtomically_ "acquire_thread" do
(r, jb) <- case wait_res of
Left (e :: MC.SomeException) -> do
return $ (Just e, Nothing)
- Right success -> do
- if success
- then do
- modifyJobResources jobs_tvar \ jobs ->
- return (Nothing, addToken jobs)
- else
- return (Nothing, Nothing)
+ Right tok -> do
+ modifyJobResources jobs_tvar \ jobs ->
+ return (Nothing, addToken tok jobs)
putTMVar threadFinished_tmvar r
return jb
- wait_id <- forkWaitOnSemaphoreInterruptible sem wait_result_action
- labelThread (waitingThreadId wait_id) "acquire_thread"
- return $ Acquiring { activeWaitId = wait_id
+ atomically $ writeTVar active_tvar (Just (tid, threadFinished_tmvar))
+ return tid
+ labelThread tid "acquire_thread"
+ return $ Acquiring { activeThreadId = tid
, threadFinished = threadFinished_tmvar }
-- | Spawn a thread to release ownership of one resource from the semaphore,
-- provided we have spare resources and no pending jobs.
releaseThread :: Jobserver -> IO JobserverAction
-releaseThread (Jobserver { jSemaphore = sem, jobs = jobs_tvar }) = do
+releaseThread (Jobserver { jobs = jobs_tvar }) = do
threadFinished_tmvar <- newEmptyTMVarIO
MC.mask_ do
-- Pre-release the resource so that another thread doesn't take control of it
-- just as we release the lock on the semaphore.
- still_ok_to_release
+ mb_tok
<- tracedAtomically "pre_release" $
modifyJobResources jobs_tvar \ jobs ->
if guardRelease jobs
- -- TODO: should this also debounce?
- then return (True , removeOwnedToken $ removeFreeToken jobs)
- else return (False, jobs)
- if not still_ok_to_release
- then return Idle
- else do
- tid <- forkIO $ do
- x <- MC.try $ releaseSemaphore sem 1
- tracedAtomically_ "post-release" $ do
- (r, jobs) <- case x of
- Left (e :: MC.SomeException) -> do
- modifyJobResources jobs_tvar \ jobs ->
- return (Just e, addToken jobs)
- Right _ -> do
- return (Nothing, Nothing)
- putTMVar threadFinished_tmvar r
- return jobs
- labelThread tid "release_thread"
- return Idle
+ then let (tok, jobs') = removeOwnedToken $ removeFreeToken jobs
+ in return (Just tok, jobs')
+ else return (Nothing, jobs)
+ case mb_tok of
+ Nothing ->
+ -- Not OK to release: there are other pending jobs that could make use of the token.
+ return Idle
+ Just tok -> do
+ tid <- forkIO $ do
+ x <- MC.try @MC.SomeException $ releaseSemaphoreToken tok
+ tracedAtomically_ "post-release" $ do
+ (r, jobs) <- case x of
+ Left (e :: MC.SomeException) -> do
+ modifyJobResources jobs_tvar \ jobs ->
+ return (Just e, addToken tok jobs)
+ Right _ -> do
+ return (Nothing, Nothing)
+ putTMVar threadFinished_tmvar r
+ return jobs
+ labelThread tid "release_thread"
+ return Idle
-- | When there are pending jobs but no free tokens,
-- spawn a thread to acquire a new token from the semaphore.
@@ -363,13 +388,14 @@ tryRelease _ _ = retry
-- | Wait for an active thread to finish. Once it finishes:
--
-- - set the 'JobserverAction' to 'Idle',
+-- - clear the 'activeChild' handle,
-- - update the number of capabilities to reflect the number
-- of owned tokens from the semaphore.
tryNoticeIdle :: JobserverOptions
- -> TVar JobResources
+ -> Jobserver
-> JobserverState
-> STM (IO JobserverState)
-tryNoticeIdle opts jobs_tvar jobserver_state
+tryNoticeIdle opts (Jobserver { jobs = jobs_tvar, activeChild = active_tvar }) jobserver_state
| Just threadFinished_tmvar <- activeThread_maybe $ jobserverAction jobserver_state
= sync_num_caps (canChangeNumCaps jobserver_state) threadFinished_tmvar
| otherwise
@@ -381,6 +407,7 @@ tryNoticeIdle opts jobs_tvar jobserver_state
sync_num_caps can_change_numcaps_tvar threadFinished_tmvar = do
mb_ex <- takeTMVar threadFinished_tmvar
for_ mb_ex MC.throwM
+ writeTVar active_tvar Nothing
Jobs { tokensOwned } <- readTVar jobs_tvar
can_change_numcaps <- readTVar can_change_numcaps_tvar
guard can_change_numcaps
@@ -404,11 +431,11 @@ tryStopThread :: TVar JobResources
-> STM (IO JobserverState)
tryStopThread jobs_tvar jsj = do
case jobserverAction jsj of
- Acquiring { activeWaitId = wait_id } -> do
+ Acquiring { activeThreadId = tid } -> do
jobs <- readTVar jobs_tvar
guard $ null (jobsWaiting jobs)
return do
- interruptWaitOnSemaphore wait_id
+ killThread tid
return $ jsj { jobserverAction = Idle }
_ -> retry
@@ -430,30 +457,38 @@ jobserverLoop opts sjs@(Jobserver { jobs = jobs_tvar })
action <- atomically $ asum $ (\x -> x s) <$>
[ tryRelease sjs
, tryAcquire opts sjs
- , tryNoticeIdle opts jobs_tvar
+ , tryNoticeIdle opts sjs
, tryStopThread jobs_tvar
]
s <- action
loop s
--- | Create a new jobserver using the given semaphore handle.
-makeJobserver :: SemaphoreName -> IO (AbstractSem, IO ())
-makeJobserver sem_name = do
- semaphore <- openSemaphore sem_name
+-- | Create a new jobserver using the given semaphore identifier.
+makeJobserver :: SemaphoreIdentifier -> IO (AbstractSem, IO ())
+makeJobserver sem_ident = do
+ semaphore <- openSemaphore sem_ident >>= either MC.throwM pure
let
init_jobs =
Jobs { tokensOwned = 1
, tokensFree = 1
, jobsWaiting = NilOL
+ , heldTokens = []
}
jobs_tvar <- newTVarIO init_jobs
+ active_tvar <- newTVarIO Nothing
let
opts = defaultJobserverOptions -- TODO: allow this to be configured
- sjs = Jobserver { jSemaphore = semaphore
- , jobs = jobs_tvar }
+ sjs = Jobserver { jSemaphore = semaphore
+ , jobs = jobs_tvar
+ , activeChild = active_tvar }
loop_finished_mvar <- newEmptyMVar
loop_tid <- forkIOWithUnmask \ unmask -> do
r <- try $ unmask $ jobserverLoop opts sjs
+ -- Always-run exit handler: any child the loop spawned is still alive
+ -- in its own thread, so drain it before touching jobs_tvar. No one
+ -- else can mutate the resources once both are dead.
+ drainActiveChild sjs
+ releaseAllHeld sjs
putMVar loop_finished_mvar $
case r of
Left e
@@ -467,8 +502,8 @@ makeJobserver sem_name = do
acquireSem = acquireJob jobs_tvar
releaseSem = releaseJob jobs_tvar
cleanupSem = do
- -- this is interruptible
- cleanupJobserver sjs
+ -- Trigger the loop's exit handler; it drains the active child and
+ -- releases all held tokens, then signals loop_finished_mvar.
killThread loop_tid
mb_ex <- takeMVar loop_finished_mvar
for_ mb_ex MC.throwM
@@ -477,12 +512,12 @@ makeJobserver sem_name = do
-- | Implement an abstract semaphore using a semaphore 'Jobserver'
-- which queries the system semaphore of the given name for resources.
-runJSemAbstractSem :: SemaphoreName -- ^ the system semaphore to use
+runJSemAbstractSem :: SemaphoreIdentifier -- ^ the semaphore identifier (from @-jsem@)
-> (AbstractSem -> IO a) -- ^ the operation to run
-- which requires a semaphore
-> IO a
-runJSemAbstractSem sem action = MC.mask \ unmask -> do
- (abs, cleanup) <- makeJobserver sem
+runJSemAbstractSem sem_ident action = MC.mask \ unmask -> do
+ (abs, cleanup) <- makeJobserver sem_ident
r <- try $ unmask $ action abs
case r of
Left (e1 :: MC.SomeException) -> do
@@ -517,8 +552,13 @@ increases the number of `free` jobs. If there are more pending jobs when the fre
is increased, the token is immediately reused (see `modifyJobResources`).
The `jobServerLoop` interacts with the system semaphore: when there are pending
-jobs, `acquireThread` blocks, waiting for a token from the semaphore. Once a
-token is obtained, it increases the owned count.
+jobs, `acquireThread` forks a child that calls the interruptible
+`waitOnSemaphore`. The child is forked in the masked state, so the only place
+an async exception can be delivered is the wait itself; once the wait returns,
+the child's STM commit always completes, recording either the new token in
+`heldTokens` or the failure exception in `threadFinished`. The (tid, tmvar)
+pair is also published in `activeChild` so the loop's exit handler can drain
+the child on shutdown even after the in-loop `JobserverState` is gone.
When GHC has free tokens (tokens from the semaphore that it is not using),
no pending jobs, and the debounce has expired, then `releaseThread` will
@@ -531,6 +571,12 @@ This second token is no longer needed, so we should cancel the wait
(as it would not be used to do any work, and not be returned until the debounce).
We only need to kill `acquireJob`, because `releaseJob` never blocks.
+Shutdown starts with `killThread loop_tid`. The loop's exit handler then
+runs `drainActiveChild` followed by `releaseAllHeld`; only then does the
+loop signal `loop_finished_mvar`. This sequence makes the heldTokens
+snapshot consistent because no other thread can mutate it once the loop and
+its child are both dead.
+
Note [Eventlog Messages for jsem]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
It can be tricky to verify that the work is shared adequately across different
=====================================
compiler/GHC/Driver/Session.hs
=====================================
@@ -2445,6 +2445,7 @@ wWarningFlagsDeps = [minBound..maxBound] >>= \x -> case x of
Opt_WarnUnusableUnpackPragmas -> warnSpec x
Opt_WarnPatternNamespaceSpecifier -> warnSpec x
Opt_WarnUnrecognisedModifiers -> warnSpec x
+ Opt_WarnSemaphoreOpenFailure -> warnSpec x
warningGroupsDeps :: [(Deprecation, FlagSpec WarningGroup)]
warningGroupsDeps = map mk warningGroups
=====================================
compiler/GHC/Types/Error/Codes.hs
=====================================
@@ -403,6 +403,7 @@ type family GhcDiagnosticCode c = n | n -> c where
GhcDiagnosticCode "DriverInstantiationNodeInDependencyGeneration" = 74284
GhcDiagnosticCode "DriverNoConfiguredLLVMToolchain" = 66599
GhcDiagnosticCode "DriverMissingLinkableForModule" = 47338
+ GhcDiagnosticCode "DriverSemaphoreOpenFailure" = 19877
-- Constraint solver diagnostic codes
GhcDiagnosticCode "BadTelescope" = 97739
=====================================
docs/users_guide/using-warnings.rst
=====================================
@@ -2721,6 +2721,23 @@ of ``-W(no-)*``.
f :: a %True -> a
g :: a %(k :: Int) -> a
+.. ghc-flag:: -Wsemaphore-open-failure
+ :shortdesc: warn when GHC cannot open the ``-jsem`` semaphore.
+ :type: dynamic
+ :reverse: -Wno-semaphore-open-failure
+ :category:
+
+ :since: 10.0.1
+
+ Warn when GHC is invoked with :ghc-flag:`-jsem` but the semaphore
+ cannot be opened (e.g. the socket does not exist, the protocol
+ version is incompatible, or a system error occurred). When this
+ occurs, GHC ignores ``-jsem`` and compiles modules sequentially.
+
+ A common cause is ``cabal-install`` and GHC being built against
+ different versions of the ``semaphore-compat`` library; upgrading
+ both to versions that use the same protocol resolves the mismatch.
+
----
If you're feeling really paranoid, the :ghc-flag:`-dcore-lint` option is a good choice.
=====================================
docs/users_guide/using.rst
=====================================
@@ -797,7 +797,12 @@ There are two kinds of participants in the GHC Jobserver protocol:
Perform compilation in parallel when possible, coordinating with other
processes through the semaphore ⟨sem⟩ (specified as a string).
- Error if the semaphore doesn't exist.
+
+ If the semaphore cannot be opened (e.g. the socket does not exist
+ or its protocol version is incompatible with this GHC), GHC emits
+ a :ghc-flag:`-Wsemaphore-open-failure` warning and compiles
+ sequentially, using only the implicit token inherited from the
+ parent process.
Use of ``-jsem`` will override use of :ghc-flag:`-j[⟨n⟩]`,
and vice-versa.
=====================================
hadrian/cabal.project
=====================================
@@ -1,6 +1,7 @@
packages: ./
../utils/ghc-toolchain/
../libraries/ghc-platform/
+ ../libraries/semaphore-compat/
-- This essentially freezes the build plan for hadrian
-- It would be wise to keep this up to date with the state set in .gitlab/ci.sh.
=====================================
hadrian/hadrian.cabal
=====================================
@@ -172,6 +172,7 @@ executable hadrian
, base16-bytestring >= 0.1.1 && < 1.1.0.0
, ghc-platform
, ghc-toolchain
+ , semaphore-compat
ghc-options: -Wall
-Wincomplete-record-updates
-Wredundant-constraints
=====================================
hadrian/src/Flavour.hs
=====================================
@@ -149,10 +149,6 @@ werror =
-- unix has many unused imports
, package unix
? mconcat [arg "-Wwarn=unused-imports", arg "-Wwarn=unused-top-binds"]
- -- semaphore-compat relies on sem_getvalue as provided by unix, which is
- -- not implemented on Darwin and therefore throws a deprecation warning
- , package semaphoreCompat
- ? mconcat [arg "-Wwarn=deprecations"]
]
, builder Ghc
? package rts
=====================================
hadrian/src/Rules/Generate.hs
=====================================
@@ -25,6 +25,7 @@ import Utilities
import GHC.Toolchain as Toolchain hiding (HsCpp(HsCpp))
import GHC.Platform.ArchOS
import Settings.Program (ghcWithInterpreter)
+import System.Semaphore (semaphoreVersion, getSemaphoreProtocolVersion)
-- | Track this file to rebuild generated files whenever it changes.
trackGenerateHs :: Expr ()
@@ -488,6 +489,7 @@ generateSettings settingsFile = do
, ("RTS ways", escapeArgs . map show . Set.toList <$> getRtsWays)
, ("Relative Global Package DB", pure rel_pkg_db)
, ("base unit-id", pure base_unit_id)
+ , ("Semaphore version", pure (show (getSemaphoreProtocolVersion semaphoreVersion)))
]
let showTuple (k, v) = "(" ++ show k ++ ", " ++ show v ++ ")"
pure $ case settings of
=====================================
hadrian/src/Settings/Packages.hs
=====================================
@@ -231,6 +231,10 @@ packageArgs = do
, package hpcBin
? builder (Cabal Flags) ? arg "-build-tool-depends"
+ ------------------------------ semaphore-compat ----------------------------
+ , package semaphoreCompat
+ ? builder (Cabal Flags) ? arg "-build-testing"
+
]
ghcInternalArgs :: Args
=====================================
hadrian/stack.yaml
=====================================
@@ -16,6 +16,7 @@ packages:
- '.'
- '../utils/ghc-toolchain'
- '../libraries/ghc-platform'
+- '../libraries/semaphore-compat'
nix:
enable: false
=====================================
libraries/semaphore-compat
=====================================
@@ -1 +1 @@
-Subproject commit 7929702401d49bc64d809c501ed5fe80aebc3cc1
+Subproject commit 83b8935203e8c57b189c8a2a19c4173d6a93ea2b
=====================================
testsuite/tests/diagnostic-codes/codes.stdout
=====================================
@@ -21,6 +21,7 @@
[GHC-29747] is untested (constructor = DriverMissingSafeHaskellMode)
[GHC-74284] is untested (constructor = DriverInstantiationNodeInDependencyGeneration)
[GHC-66599] is untested (constructor = DriverNoConfiguredLLVMToolchain)
+[GHC-19877] is untested (constructor = DriverSemaphoreOpenFailure)
[GHC-81325] is untested (constructor = ExpectingMoreArguments)
[GHC-78125] is untested (constructor = AmbiguityPreventsSolvingCt)
[GHC-84170] is untested (constructor = TcRnModMissingRealSrcSpan)
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/34171a94c63917cfb63902a80d82539…
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/34171a94c63917cfb63902a80d82539…
You're receiving this email because of your account on gitlab.haskell.org.
1
0
[Git][ghc/ghc] Pushed new branch wip/fendor/ghc-pkg-faster-closure
by Hannes Siebenhandl (＠fendor) 19 May '26
by Hannes Siebenhandl (＠fendor) 19 May '26
19 May '26
Hannes Siebenhandl pushed new branch wip/fendor/ghc-pkg-faster-closure at Glasgow Haskell Compiler / GHC
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/tree/wip/fendor/ghc-pkg-faster-closure
You're receiving this email because of your account on gitlab.haskell.org.
1
0
[Git][ghc/ghc][wip/semaphore-v2] Update to semaphore-compat 2.0.0 using v2 of the protocol
by Zubin (＠wz1000) 19 May '26
by Zubin (＠wz1000) 19 May '26
19 May '26
Zubin pushed to branch wip/semaphore-v2 at Glasgow Haskell Compiler / GHC
Commits:
7479e9a0 by Zubin Duggal at 2026-05-19T14:56:43+05:30
Update to semaphore-compat 2.0.0 using v2 of the protocol
On Linux and other POSIX platforms, GHC's -jsem jobserver client now
speaks v2 of the semaphore-compat protocol, which uses Unix domain
sockets in place of POSIX named semaphores. This avoids the libc-ABI
issues that affected the old implementation. Windows is unaffected
and continues to use the v1 protocol (Win32 named semaphores); its
reported protocol version remains v1.
When GHC receives a -jsem name whose protocol version it does not
support, it emits a -Wsemaphore-version-mismatch warning and falls
back to -j<N> rather than crashing. ghc --info exposes the supported
version in a new "Semaphore version" entry so cabal-install can detect
a mismatch before invoking GHC.
Users on a cabal-install that predates the v2 update will continue to
build successfully on Linux/POSIX, but will lose the cross-process
-jsem coordination and fall back to -j<N> per GHC invocation. Users
must upgrade to a cabal-install that supports protocol v2 to recover
full parallelism.
Also fix a leak in cleanupSem (#27253): cleanupSem used to snapshot
heldTokens and release them before killing the loop, while the loop's
in-flight acquire/release children could still be mutating it.
Cleanup now runs inside the loop's own exit handler, after draining
the active child via a new activeChild TVar, so the snapshot has no
concurrent mutator.
See also:
- GHC proposal amendment: https://github.com/ghc-proposals/ghc-proposals/pull/673
- cabal-install patch: https://github.com/haskell/cabal/pull/11628
- semaphore-compat MR: https://gitlab.haskell.org/ghc/semaphore-compat/-/merge_requests/8
Bump semaphore-compat submodule to 2.0.0
Fixes #25087 and #27253
- - - - -
19 changed files:
- + changelog.d/jobserver-leak-fix
- + changelog.d/semaphore-v2
- compiler/GHC/Driver/Errors/Ppr.hs
- compiler/GHC/Driver/Errors/Types.hs
- compiler/GHC/Driver/Flags.hs
- compiler/GHC/Driver/MakeAction.hs
- compiler/GHC/Driver/MakeSem.hs
- compiler/GHC/Driver/Session.hs
- compiler/GHC/Types/Error/Codes.hs
- docs/users_guide/using-warnings.rst
- docs/users_guide/using.rst
- hadrian/cabal.project
- hadrian/hadrian.cabal
- hadrian/src/Flavour.hs
- hadrian/src/Rules/Generate.hs
- hadrian/src/Settings/Packages.hs
- hadrian/stack.yaml
- libraries/semaphore-compat
- testsuite/tests/diagnostic-codes/codes.stdout
Changes:
=====================================
changelog.d/jobserver-leak-fix
=====================================
@@ -0,0 +1,8 @@
+section: compiler
+issues: #27253
+mrs: !15729
+synopsis:
+ Fix a token leak in the ``-jsem`` jobserver shutdown path
+description:
+ A build interrupted by Ctrl-C while a ``-jsem`` token transfer was in
+ flight could leak that token.
=====================================
changelog.d/semaphore-v2
=====================================
@@ -0,0 +1,30 @@
+section: compiler
+issues: #25087
+mrs: !15729
+synopsis:
+ Update to semaphore-compat 2.0.0 (``-jsem`` protocol v2)
+description:
+ On Linux and other POSIX platforms, GHC's ``-jsem`` jobserver client
+ now speaks v2 of the semaphore-compat protocol, which uses Unix
+ domain sockets in place of POSIX named semaphores. This avoids the
+ libc-ABI issues that affected the old implementation. Windows is
+ unaffected and continues to use the v1 protocol (Win32 named
+ semaphores); its reported protocol version remains v1.
+
+ When GHC receives a ``-jsem`` name whose protocol version it does not
+ support, it now emits a ``-Wsemaphore-version-mismatch`` warning and
+ falls back to ``-j1`` rather than crashing. ``ghc --info`` exposes the
+ supported version in a new ``"Semaphore version"`` entry so
+ cabal-install can detect a mismatch before invoking GHC.
+
+ Users on a ``cabal-install`` that predates the v2 update will continue
+ to build successfully on Linux/POSIX, but will lose the cross-process
+ ``-jsem`` coordination and fall back to ``-j1`` per GHC invocation.
+ To recover full parallelism, upgrade to a ``cabal-install`` that
+ supports protocol v2.
+
+ See also:
+
+ - the `GHC proposal amendment <https://github.com/ghc-proposals/ghc-proposals/pull/673>`_
+ - the `cabal-install patch <https://github.com/haskell/cabal/pull/11628>`_
+ - the `semaphore-compat library MR <https://gitlab.haskell.org/ghc/semaphore-compat/-/merge_requests/8>`_
=====================================
compiler/GHC/Driver/Errors/Ppr.hs
=====================================
@@ -24,6 +24,8 @@ import GHC.Types.Hint
import GHC.Types.SrcLoc
import Data.Version
+import System.Semaphore
+ ( SemaphoreError(..), getSemaphoreProtocolVersion )
import Language.Haskell.Syntax.Decls (RuleDecl(..))
import GHC.Tc.Errors.Types (TcRnMessage)
import GHC.HsToCore.Errors.Types (DsMessage)
@@ -90,6 +92,20 @@ instance Diagnostic GhcMessage where
instance HasDefaultDiagnosticOpts DriverMessageOpts where
defaultOpts = DriverMessageOpts (defaultDiagnosticOpts @PsMessage) (defaultDiagnosticOpts @IfaceMessage)
+pprSemaphoreError :: SemaphoreError -> SDoc
+pprSemaphoreError = \case
+ SemaphoreAlreadyExists nm ->
+ text "a semaphore named" <+> quotes (text nm) <+> text "already exists"
+ SemaphoreDoesNotExist nm ->
+ text "no semaphore named" <+> quotes (text nm)
+ SemaphoreIncompatibleVersion got want ->
+ text "protocol version mismatch (got v"
+ <> int (getSemaphoreProtocolVersion got)
+ <> text ", supported v"
+ <> int (getSemaphoreProtocolVersion want) <> text ")"
+ SemaphoreOtherError ioe ->
+ text (show ioe)
+
instance Diagnostic DriverMessage where
type DiagnosticOpts DriverMessage = DriverMessageOpts
diagnosticMessage opts = \case
@@ -282,6 +298,10 @@ instance Diagnostic DriverMessage where
-> mkSimpleDecorated $
vcat [ text "The following modules are missing a linkable which is needed for creating a library:"
, nest 2 $ hcat (map ppr mods) ]
+ DriverSemaphoreOpenFailure err
+ -> mkSimpleDecorated $
+ text "Failed to open -jsem semaphore:" <+> pprSemaphoreError err <>
+ text "; ignoring -jsem and compiling sequentially."
diagnosticReason = \case
DriverUnknownMessage m
@@ -355,6 +375,8 @@ instance Diagnostic DriverMessage where
-> WarningWithoutFlag
DriverMissingLinkableForModule {}
-> ErrorWithoutFlag
+ DriverSemaphoreOpenFailure {}
+ -> WarningWithFlag Opt_WarnSemaphoreOpenFailure
diagnosticHints = \case
DriverUnknownMessage m
@@ -430,5 +452,14 @@ instance Diagnostic DriverMessage where
-> noHints
DriverMissingLinkableForModule {}
-> noHints
+ DriverSemaphoreOpenFailure (SemaphoreIncompatibleVersion received supported)
+ | received < supported
+ -> [UnknownHint (text "The parent process (e.g. cabal-install) uses an older semaphore protocol."
+ $$ text "Upgrading cabal-install may resolve this." :: SDoc)]
+ | otherwise
+ -> [UnknownHint (text "The parent process (e.g. cabal-install) uses a newer semaphore protocol."
+ $$ text "Upgrading GHC may resolve this." :: SDoc)]
+ DriverSemaphoreOpenFailure {}
+ -> noHints
diagnosticCode = constructorCode @GHC
=====================================
compiler/GHC/Driver/Errors/Types.hs
=====================================
@@ -37,6 +37,7 @@ import qualified GHC.LanguageExtensions as LangExt
import GHC.Generics ( Generic )
+import System.Semaphore ( SemaphoreError )
import GHC.Tc.Errors.Types
import GHC.Iface.Errors.Types
@@ -419,6 +420,13 @@ data DriverMessage where
DriverMissingLinkableForModule :: ![Module] -> DriverMessage
+ {-| DriverSemaphoreOpenFailure is a warning that occurs when GHC fails to
+ open the semaphore specified by @-jsem@, e.g. the socket does not
+ exist, the protocol version is incompatible, or a system error
+ occurred. GHC ignores @-jsem@ and compiles sequentially.
+ -}
+ DriverSemaphoreOpenFailure :: !SemaphoreError -> DriverMessage
+
deriving instance Generic DriverMessage
data DriverMessageOpts =
=====================================
compiler/GHC/Driver/Flags.hs
=====================================
@@ -1115,6 +1115,7 @@ data WarningFlag =
| Opt_WarnUnusableUnpackPragmas -- ^ @since 9.14
| Opt_WarnPatternNamespaceSpecifier -- ^ @since 9.14
| Opt_WarnUnrecognisedModifiers -- ^ @since 10.0
+ | Opt_WarnSemaphoreOpenFailure -- Since 10.0.1
deriving (Eq, Ord, Show, Enum, Bounded)
-- | Return the names of a WarningFlag
@@ -1237,6 +1238,7 @@ warnFlagNames wflag = case wflag of
Opt_WarnUnusableUnpackPragmas -> "unusable-unpack-pragmas" :| []
Opt_WarnPatternNamespaceSpecifier -> "pattern-namespace-specifier" :| []
Opt_WarnUnrecognisedModifiers -> "unrecognised-modifiers" :| []
+ Opt_WarnSemaphoreOpenFailure -> "semaphore-open-failure" :| []
-- -----------------------------------------------------------------------------
-- Standard sets of warning options
@@ -1383,7 +1385,8 @@ standardWarnings -- see Note [Documenting warning flags]
Opt_WarnDeprecatedPragmas,
Opt_WarnRuleLhsEqualities,
Opt_WarnUnusableUnpackPragmas,
- Opt_WarnUnrecognisedModifiers
+ Opt_WarnUnrecognisedModifiers,
+ Opt_WarnSemaphoreOpenFailure
]
-- | Things you get with @-W@.
=====================================
compiler/GHC/Driver/MakeAction.hs
=====================================
@@ -28,6 +28,14 @@ import GHC.Driver.Errors.Types
import GHC.Driver.Messager
import GHC.Driver.MakeSem
+import System.Semaphore
+ ( SemaphoreError, SemaphoreIdentifier )
+
+import GHC.Driver.Config.Diagnostic ( initDiagOpts, initPrintConfig )
+import GHC.Driver.Errors ( printOrThrowDiagnostics )
+import GHC.Types.Error ( singleMessage )
+import GHC.Types.SrcLoc ( noSrcSpan )
+import GHC.Utils.Error ( mkPlainMsgEnvelope )
import GHC.Utils.Logger
import GHC.Utils.TmpFs
@@ -49,7 +57,7 @@ mkWorkerLimit :: DynFlags -> IO WorkerLimit
mkWorkerLimit dflags =
case parMakeCount dflags of
Nothing -> pure $ num_procs 1
- Just (ParMakeSemaphore h) -> pure (JSemLimit (SemaphoreName h))
+ Just (ParMakeSemaphore h) -> pure (JSemLimit h)
Just ParMakeNumProcessors -> num_procs <$> getNumProcessors
Just (ParMakeThisMany n) -> pure $ num_procs n
where
@@ -65,8 +73,8 @@ isWorkerLimitSequential (JSemLimit {}) = False
data WorkerLimit
= NumProcessorsLimit Int
| JSemLimit
- SemaphoreName
- -- ^ Semaphore name to use
+ SemaphoreIdentifier
+ -- ^ Semaphore identifier from @-jsem@
deriving Eq
-- | Environment used when compiling a module
@@ -122,17 +130,24 @@ runNjobsAbstractSem n_jobs action = do
resetNumCapabilities = set_num_caps n_capabilities
MC.bracket_ updNumCapabilities resetNumCapabilities $ action asem
-runWorkerLimit :: WorkerLimit -> (AbstractSem -> IO a) -> IO a
+runWorkerLimit :: Logger -> DynFlags -> WorkerLimit -> (AbstractSem -> IO a) -> IO a
#if defined(wasm32_HOST_ARCH)
-runWorkerLimit _ action = do
+runWorkerLimit _logger _dflags _ action = do
lock <- newMVar ()
action $ AbstractSem (takeMVar lock) (putMVar lock ())
#else
-runWorkerLimit worker_limit action = case worker_limit of
+runWorkerLimit logger dflags worker_limit action = case worker_limit of
NumProcessorsLimit n_jobs ->
runNjobsAbstractSem n_jobs action
- JSemLimit sem ->
- runJSemAbstractSem sem action
+ JSemLimit sem_ident -> do
+ result <- MC.try @SemaphoreError $ runJSemAbstractSem sem_ident action
+ case result of
+ Right a -> return a
+ Left err -> do
+ let diag = DriverSemaphoreOpenFailure err
+ msg = singleMessage $ mkPlainMsgEnvelope (initDiagOpts dflags) noSrcSpan diag
+ printOrThrowDiagnostics logger (initPrintConfig dflags) (initDiagOpts dflags) (GhcDriverMessage <$> msg)
+ runNjobsAbstractSem 1 action
#endif
-- | Build and run a pipeline
@@ -159,7 +174,7 @@ runParPipelines worker_limit plugin_hsc_env diag_wrapper mHscMessager all_pipeli
thread_safe_logger <- liftIO $ makeThreadSafe (hsc_logger plugin_hsc_env)
let thread_safe_hsc_env = plugin_hsc_env { hsc_logger = thread_safe_logger }
- runWorkerLimit worker_limit $ \abstract_sem -> do
+ runWorkerLimit (hsc_logger plugin_hsc_env) (hsc_dflags plugin_hsc_env) worker_limit $ \abstract_sem -> do
let env = MakeEnv { hsc_env = thread_safe_hsc_env
, withLogger = withParLog log_queue_queue_var
, compile_sem = abstract_sem
=====================================
compiler/GHC/Driver/MakeSem.hs
=====================================
@@ -9,9 +9,6 @@ module GHC.Driver.MakeSem
-- by a system semaphore (Posix/Windows)
runJSemAbstractSem
- -- * System semaphores
- , Semaphore, SemaphoreName(..)
-
-- * Abstract semaphores
, AbstractSem(..)
, withAbstractSem
@@ -46,11 +43,14 @@ import Debug.Trace
-- available from the semaphore.
data Jobserver
= Jobserver
- { jSemaphore :: !Semaphore
+ { jSemaphore :: !ClientSemaphore
-- ^ The semaphore which controls available resources
, jobs :: !(TVar JobResources)
-- ^ The currently pending jobs, and the resources
-- obtained from the semaphore
+ , activeChild :: !(TVar (Maybe (ThreadId, TMVar (Maybe MC.SomeException))))
+ -- ^ Handle on the current acquire thread (if any). The loop's exit
+ -- handler reads this to drain a still-running child on shutdown.
}
data JobserverOptions
@@ -81,6 +81,9 @@ data JobResources
, jobsWaiting :: !(OrdList (TMVar ()))
-- ^ Pending jobs waiting on a token, the job will be blocked on the TMVar so putting into
-- the TMVar will allow the job to continue.
+ , heldTokens :: [SemaphoreToken]
+ -- ^ Actual semaphore tokens (for release/cleanup).
+ -- Length should equal tokensOwned - 1 (the implicit token has no SemaphoreToken).
}
instance Outputable JobResources where
@@ -93,9 +96,9 @@ instance Outputable JobResources where
] )
-- | Add one new token.
-addToken :: JobResources -> JobResources
-addToken jobs@( Jobs { tokensOwned = owned, tokensFree = free })
- = jobs { tokensOwned = owned + 1, tokensFree = free + 1 }
+addToken :: SemaphoreToken -> JobResources -> JobResources
+addToken tok jobs@( Jobs { tokensOwned = owned, tokensFree = free, heldTokens = toks })
+ = jobs { tokensOwned = owned + 1, tokensFree = free + 1, heldTokens = tok : toks }
-- | Free one token.
addFreeToken :: JobResources -> JobResources
@@ -111,12 +114,14 @@ removeFreeToken jobs@( Jobs { tokensFree = free })
(text "removeFreeToken:" <+> ppr free)
$ jobs { tokensFree = free - 1 }
--- | Return one owned token.
-removeOwnedToken :: JobResources -> JobResources
-removeOwnedToken jobs@( Jobs { tokensOwned = owned })
+-- | Return one owned token, extracting the 'SemaphoreToken' for release.
+removeOwnedToken :: JobResources -> (SemaphoreToken, JobResources)
+removeOwnedToken jobs@( Jobs { tokensOwned = owned, heldTokens = toks })
= assertPpr (owned > 1)
(text "removeOwnedToken:" <+> ppr owned)
- $ jobs { tokensOwned = owned - 1 }
+ $ case toks of
+ (t:rest) -> (t, jobs { tokensOwned = owned - 1, heldTokens = rest })
+ [] -> panic "removeOwnedToken: no held tokens"
-- | Add one new job to the end of the list of pending jobs.
addJob :: TMVar () -> JobResources -> JobResources
@@ -143,7 +148,7 @@ data JobserverAction
= Idle
-- | A thread is waiting for a token on the semaphore.
| Acquiring
- { activeWaitId :: WaitId
+ { activeThreadId :: ThreadId
, threadFinished :: TMVar (Maybe MC.SomeException) }
-- | Retrieve the 'TMVar' that signals if the current thread has finished,
@@ -189,17 +194,30 @@ releaseJob jobs_tvar = do
return ((), addFreeToken jobs)
--- | Release all tokens owned from the semaphore (to clean up
--- the jobserver at the end).
-cleanupJobserver :: Jobserver -> IO ()
-cleanupJobserver (Jobserver { jSemaphore = sem
- , jobs = jobs_tvar })
- = do
- Jobs { tokensOwned = owned } <- readTVarIO jobs_tvar
- let toks_to_release = owned - 1
- -- Subtract off the implicit token: whoever spawned the ghc process
- -- in the first place is responsible for that token.
- releaseSemaphore sem toks_to_release
+-- | Kill the current acquire thread, if any, and wait for it to exit.
+--
+-- Called from the jobserver loop's exit handler, which runs masked.
+-- Relies on the invariant from 'acquireThread' that a forked child
+-- always fills its 'threadFinished' TMVar before it dies; this is what
+-- lets the 'takeTMVar' below terminate after the 'killThread'.
+drainActiveChild :: Jobserver -> IO ()
+drainActiveChild (Jobserver { activeChild = active_tvar }) = do
+ mb <- readTVarIO active_tvar
+ for_ mb $ \(tid, tmv) -> do
+ killThread tid
+ void $ atomically (takeTMVar tmv)
+ atomically $ writeTVar active_tvar Nothing
+
+-- | Release every token currently in 'heldTokens'.
+--
+-- Called from the jobserver loop's exit handler, which runs masked,
+-- after 'drainActiveChild': no other thread is mutating 'JobResources'
+-- at this point.
+releaseAllHeld :: Jobserver -> IO ()
+releaseAllHeld (Jobserver { jobs = jobs_tvar }) = do
+ Jobs { heldTokens = toks } <- readTVarIO jobs_tvar
+ forM_ toks $ \t ->
+ void $ MC.try @MC.SomeException (releaseSemaphoreToken t)
-- | Dispatch the available tokens acquired from the semaphore
-- to the pending jobs in the job server.
@@ -252,7 +270,7 @@ tracedAtomically origin act = do
return a
renderJobResources :: String -> JobResources -> String
-renderJobResources origin (Jobs own free pending) = showSDocUnsafe $ renderJSON $
+renderJobResources origin (Jobs own free pending _heldToks) = showSDocUnsafe $ renderJSON $
JSObject [ ("name", JSString origin)
, ("owned", JSInt own)
, ("free", JSInt free)
@@ -262,61 +280,68 @@ renderJobResources origin (Jobs own free pending) = showSDocUnsafe $ renderJSON
-- | Spawn a new thread that waits on the semaphore in order to acquire
-- an additional token.
+--
+-- The child is forked masked so the only async-exception delivery point
+-- is the interruptible 'waitOnSemaphore'; the STM commit afterwards then
+-- always runs to completion, so 'threadFinished' is always filled.
+--
+-- The (tid, threadFinished) pair is also published to 'activeChild' so
+-- shutdown can drain the child even after the in-loop 'JobserverState'
+-- is gone.
acquireThread :: Jobserver -> IO JobserverAction
-acquireThread (Jobserver { jSemaphore = sem, jobs = jobs_tvar }) = do
+acquireThread (Jobserver { jSemaphore = sem, jobs = jobs_tvar, activeChild = active_tvar }) = do
threadFinished_tmvar <- newEmptyTMVarIO
- let
- wait_result_action :: Either MC.SomeException Bool -> IO ()
- wait_result_action wait_res =
+ tid <- MC.mask_ $ do
+ tid <- forkIO $ do
+ wait_res <- MC.try @_ @MC.SomeException $ waitOnSemaphore sem
tracedAtomically_ "acquire_thread" do
(r, jb) <- case wait_res of
Left (e :: MC.SomeException) -> do
return $ (Just e, Nothing)
- Right success -> do
- if success
- then do
- modifyJobResources jobs_tvar \ jobs ->
- return (Nothing, addToken jobs)
- else
- return (Nothing, Nothing)
+ Right tok -> do
+ modifyJobResources jobs_tvar \ jobs ->
+ return (Nothing, addToken tok jobs)
putTMVar threadFinished_tmvar r
return jb
- wait_id <- forkWaitOnSemaphoreInterruptible sem wait_result_action
- labelThread (waitingThreadId wait_id) "acquire_thread"
- return $ Acquiring { activeWaitId = wait_id
+ atomically $ writeTVar active_tvar (Just (tid, threadFinished_tmvar))
+ return tid
+ labelThread tid "acquire_thread"
+ return $ Acquiring { activeThreadId = tid
, threadFinished = threadFinished_tmvar }
-- | Spawn a thread to release ownership of one resource from the semaphore,
-- provided we have spare resources and no pending jobs.
releaseThread :: Jobserver -> IO JobserverAction
-releaseThread (Jobserver { jSemaphore = sem, jobs = jobs_tvar }) = do
+releaseThread (Jobserver { jobs = jobs_tvar }) = do
threadFinished_tmvar <- newEmptyTMVarIO
MC.mask_ do
-- Pre-release the resource so that another thread doesn't take control of it
-- just as we release the lock on the semaphore.
- still_ok_to_release
+ mb_tok
<- tracedAtomically "pre_release" $
modifyJobResources jobs_tvar \ jobs ->
if guardRelease jobs
- -- TODO: should this also debounce?
- then return (True , removeOwnedToken $ removeFreeToken jobs)
- else return (False, jobs)
- if not still_ok_to_release
- then return Idle
- else do
- tid <- forkIO $ do
- x <- MC.try $ releaseSemaphore sem 1
- tracedAtomically_ "post-release" $ do
- (r, jobs) <- case x of
- Left (e :: MC.SomeException) -> do
- modifyJobResources jobs_tvar \ jobs ->
- return (Just e, addToken jobs)
- Right _ -> do
- return (Nothing, Nothing)
- putTMVar threadFinished_tmvar r
- return jobs
- labelThread tid "release_thread"
- return Idle
+ then let (tok, jobs') = removeOwnedToken $ removeFreeToken jobs
+ in return (Just tok, jobs')
+ else return (Nothing, jobs)
+ case mb_tok of
+ Nothing ->
+ -- Not OK to release: there are other pending jobs that could make use of the token.
+ return Idle
+ Just tok -> do
+ tid <- forkIO $ do
+ x <- MC.try @MC.SomeException $ releaseSemaphoreToken tok
+ tracedAtomically_ "post-release" $ do
+ (r, jobs) <- case x of
+ Left (e :: MC.SomeException) -> do
+ modifyJobResources jobs_tvar \ jobs ->
+ return (Just e, addToken tok jobs)
+ Right _ -> do
+ return (Nothing, Nothing)
+ putTMVar threadFinished_tmvar r
+ return jobs
+ labelThread tid "release_thread"
+ return Idle
-- | When there are pending jobs but no free tokens,
-- spawn a thread to acquire a new token from the semaphore.
@@ -363,13 +388,14 @@ tryRelease _ _ = retry
-- | Wait for an active thread to finish. Once it finishes:
--
-- - set the 'JobserverAction' to 'Idle',
+-- - clear the 'activeChild' handle,
-- - update the number of capabilities to reflect the number
-- of owned tokens from the semaphore.
tryNoticeIdle :: JobserverOptions
- -> TVar JobResources
+ -> Jobserver
-> JobserverState
-> STM (IO JobserverState)
-tryNoticeIdle opts jobs_tvar jobserver_state
+tryNoticeIdle opts (Jobserver { jobs = jobs_tvar, activeChild = active_tvar }) jobserver_state
| Just threadFinished_tmvar <- activeThread_maybe $ jobserverAction jobserver_state
= sync_num_caps (canChangeNumCaps jobserver_state) threadFinished_tmvar
| otherwise
@@ -381,6 +407,7 @@ tryNoticeIdle opts jobs_tvar jobserver_state
sync_num_caps can_change_numcaps_tvar threadFinished_tmvar = do
mb_ex <- takeTMVar threadFinished_tmvar
for_ mb_ex MC.throwM
+ writeTVar active_tvar Nothing
Jobs { tokensOwned } <- readTVar jobs_tvar
can_change_numcaps <- readTVar can_change_numcaps_tvar
guard can_change_numcaps
@@ -404,11 +431,11 @@ tryStopThread :: TVar JobResources
-> STM (IO JobserverState)
tryStopThread jobs_tvar jsj = do
case jobserverAction jsj of
- Acquiring { activeWaitId = wait_id } -> do
+ Acquiring { activeThreadId = tid } -> do
jobs <- readTVar jobs_tvar
guard $ null (jobsWaiting jobs)
return do
- interruptWaitOnSemaphore wait_id
+ killThread tid
return $ jsj { jobserverAction = Idle }
_ -> retry
@@ -430,30 +457,38 @@ jobserverLoop opts sjs@(Jobserver { jobs = jobs_tvar })
action <- atomically $ asum $ (\x -> x s) <$>
[ tryRelease sjs
, tryAcquire opts sjs
- , tryNoticeIdle opts jobs_tvar
+ , tryNoticeIdle opts sjs
, tryStopThread jobs_tvar
]
s <- action
loop s
--- | Create a new jobserver using the given semaphore handle.
-makeJobserver :: SemaphoreName -> IO (AbstractSem, IO ())
-makeJobserver sem_name = do
- semaphore <- openSemaphore sem_name
+-- | Create a new jobserver using the given semaphore identifier.
+makeJobserver :: SemaphoreIdentifier -> IO (AbstractSem, IO ())
+makeJobserver sem_ident = do
+ semaphore <- openSemaphore sem_ident >>= either MC.throwM pure
let
init_jobs =
Jobs { tokensOwned = 1
, tokensFree = 1
, jobsWaiting = NilOL
+ , heldTokens = []
}
jobs_tvar <- newTVarIO init_jobs
+ active_tvar <- newTVarIO Nothing
let
opts = defaultJobserverOptions -- TODO: allow this to be configured
- sjs = Jobserver { jSemaphore = semaphore
- , jobs = jobs_tvar }
+ sjs = Jobserver { jSemaphore = semaphore
+ , jobs = jobs_tvar
+ , activeChild = active_tvar }
loop_finished_mvar <- newEmptyMVar
loop_tid <- forkIOWithUnmask \ unmask -> do
r <- try $ unmask $ jobserverLoop opts sjs
+ -- Always-run exit handler: any child the loop spawned is still alive
+ -- in its own thread, so drain it before touching jobs_tvar. No one
+ -- else can mutate the resources once both are dead.
+ drainActiveChild sjs
+ releaseAllHeld sjs
putMVar loop_finished_mvar $
case r of
Left e
@@ -467,8 +502,8 @@ makeJobserver sem_name = do
acquireSem = acquireJob jobs_tvar
releaseSem = releaseJob jobs_tvar
cleanupSem = do
- -- this is interruptible
- cleanupJobserver sjs
+ -- Trigger the loop's exit handler; it drains the active child and
+ -- releases all held tokens, then signals loop_finished_mvar.
killThread loop_tid
mb_ex <- takeMVar loop_finished_mvar
for_ mb_ex MC.throwM
@@ -477,12 +512,12 @@ makeJobserver sem_name = do
-- | Implement an abstract semaphore using a semaphore 'Jobserver'
-- which queries the system semaphore of the given name for resources.
-runJSemAbstractSem :: SemaphoreName -- ^ the system semaphore to use
+runJSemAbstractSem :: SemaphoreIdentifier -- ^ the semaphore identifier (from @-jsem@)
-> (AbstractSem -> IO a) -- ^ the operation to run
-- which requires a semaphore
-> IO a
-runJSemAbstractSem sem action = MC.mask \ unmask -> do
- (abs, cleanup) <- makeJobserver sem
+runJSemAbstractSem sem_ident action = MC.mask \ unmask -> do
+ (abs, cleanup) <- makeJobserver sem_ident
r <- try $ unmask $ action abs
case r of
Left (e1 :: MC.SomeException) -> do
@@ -517,8 +552,13 @@ increases the number of `free` jobs. If there are more pending jobs when the fre
is increased, the token is immediately reused (see `modifyJobResources`).
The `jobServerLoop` interacts with the system semaphore: when there are pending
-jobs, `acquireThread` blocks, waiting for a token from the semaphore. Once a
-token is obtained, it increases the owned count.
+jobs, `acquireThread` forks a child that calls the interruptible
+`waitOnSemaphore`. The child is forked in the masked state, so the only place
+an async exception can be delivered is the wait itself; once the wait returns,
+the child's STM commit always completes, recording either the new token in
+`heldTokens` or the failure exception in `threadFinished`. The (tid, tmvar)
+pair is also published in `activeChild` so the loop's exit handler can drain
+the child on shutdown even after the in-loop `JobserverState` is gone.
When GHC has free tokens (tokens from the semaphore that it is not using),
no pending jobs, and the debounce has expired, then `releaseThread` will
@@ -531,6 +571,12 @@ This second token is no longer needed, so we should cancel the wait
(as it would not be used to do any work, and not be returned until the debounce).
We only need to kill `acquireJob`, because `releaseJob` never blocks.
+Shutdown starts with `killThread loop_tid`. The loop's exit handler then
+runs `drainActiveChild` followed by `releaseAllHeld`; only then does the
+loop signal `loop_finished_mvar`. This sequence makes the heldTokens
+snapshot consistent because no other thread can mutate it once the loop and
+its child are both dead.
+
Note [Eventlog Messages for jsem]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
It can be tricky to verify that the work is shared adequately across different
=====================================
compiler/GHC/Driver/Session.hs
=====================================
@@ -2445,6 +2445,7 @@ wWarningFlagsDeps = [minBound..maxBound] >>= \x -> case x of
Opt_WarnUnusableUnpackPragmas -> warnSpec x
Opt_WarnPatternNamespaceSpecifier -> warnSpec x
Opt_WarnUnrecognisedModifiers -> warnSpec x
+ Opt_WarnSemaphoreOpenFailure -> warnSpec x
warningGroupsDeps :: [(Deprecation, FlagSpec WarningGroup)]
warningGroupsDeps = map mk warningGroups
=====================================
compiler/GHC/Types/Error/Codes.hs
=====================================
@@ -403,6 +403,7 @@ type family GhcDiagnosticCode c = n | n -> c where
GhcDiagnosticCode "DriverInstantiationNodeInDependencyGeneration" = 74284
GhcDiagnosticCode "DriverNoConfiguredLLVMToolchain" = 66599
GhcDiagnosticCode "DriverMissingLinkableForModule" = 47338
+ GhcDiagnosticCode "DriverSemaphoreOpenFailure" = 19877
-- Constraint solver diagnostic codes
GhcDiagnosticCode "BadTelescope" = 97739
=====================================
docs/users_guide/using-warnings.rst
=====================================
@@ -2721,6 +2721,23 @@ of ``-W(no-)*``.
f :: a %True -> a
g :: a %(k :: Int) -> a
+.. ghc-flag:: -Wsemaphore-open-failure
+ :shortdesc: warn when GHC cannot open the ``-jsem`` semaphore.
+ :type: dynamic
+ :reverse: -Wno-semaphore-open-failure
+ :category:
+
+ :since: 10.0.1
+
+ Warn when GHC is invoked with :ghc-flag:`-jsem` but the semaphore
+ cannot be opened (e.g. the socket does not exist, the protocol
+ version is incompatible, or a system error occurred). When this
+ occurs, GHC ignores ``-jsem`` and compiles modules sequentially.
+
+ A common cause is ``cabal-install`` and GHC being built against
+ different versions of the ``semaphore-compat`` library; upgrading
+ both to versions that use the same protocol resolves the mismatch.
+
----
If you're feeling really paranoid, the :ghc-flag:`-dcore-lint` option is a good choice.
=====================================
docs/users_guide/using.rst
=====================================
@@ -797,7 +797,12 @@ There are two kinds of participants in the GHC Jobserver protocol:
Perform compilation in parallel when possible, coordinating with other
processes through the semaphore ⟨sem⟩ (specified as a string).
- Error if the semaphore doesn't exist.
+
+ If the semaphore cannot be opened (e.g. the socket does not exist
+ or its protocol version is incompatible with this GHC), GHC emits
+ a :ghc-flag:`-Wsemaphore-open-failure` warning and compiles
+ sequentially, using only the implicit token inherited from the
+ parent process.
Use of ``-jsem`` will override use of :ghc-flag:`-j[⟨n⟩]`,
and vice-versa.
=====================================
hadrian/cabal.project
=====================================
@@ -1,6 +1,7 @@
packages: ./
../utils/ghc-toolchain/
../libraries/ghc-platform/
+ ../libraries/semaphore-compat/
-- This essentially freezes the build plan for hadrian
-- It would be wise to keep this up to date with the state set in .gitlab/ci.sh.
=====================================
hadrian/hadrian.cabal
=====================================
@@ -172,6 +172,7 @@ executable hadrian
, base16-bytestring >= 0.1.1 && < 1.1.0.0
, ghc-platform
, ghc-toolchain
+ , semaphore-compat
ghc-options: -Wall
-Wincomplete-record-updates
-Wredundant-constraints
=====================================
hadrian/src/Flavour.hs
=====================================
@@ -149,10 +149,6 @@ werror =
-- unix has many unused imports
, package unix
? mconcat [arg "-Wwarn=unused-imports", arg "-Wwarn=unused-top-binds"]
- -- semaphore-compat relies on sem_getvalue as provided by unix, which is
- -- not implemented on Darwin and therefore throws a deprecation warning
- , package semaphoreCompat
- ? mconcat [arg "-Wwarn=deprecations"]
]
, builder Ghc
? package rts
=====================================
hadrian/src/Rules/Generate.hs
=====================================
@@ -25,6 +25,7 @@ import Utilities
import GHC.Toolchain as Toolchain hiding (HsCpp(HsCpp))
import GHC.Platform.ArchOS
import Settings.Program (ghcWithInterpreter)
+import System.Semaphore (semaphoreVersion, getSemaphoreProtocolVersion)
-- | Track this file to rebuild generated files whenever it changes.
trackGenerateHs :: Expr ()
@@ -488,6 +489,7 @@ generateSettings settingsFile = do
, ("RTS ways", escapeArgs . map show . Set.toList <$> getRtsWays)
, ("Relative Global Package DB", pure rel_pkg_db)
, ("base unit-id", pure base_unit_id)
+ , ("Semaphore version", pure (show (getSemaphoreProtocolVersion semaphoreVersion)))
]
let showTuple (k, v) = "(" ++ show k ++ ", " ++ show v ++ ")"
pure $ case settings of
=====================================
hadrian/src/Settings/Packages.hs
=====================================
@@ -231,6 +231,10 @@ packageArgs = do
, package hpcBin
? builder (Cabal Flags) ? arg "-build-tool-depends"
+ ------------------------------ semaphore-compat ----------------------------
+ , package semaphoreCompat
+ ? builder (Cabal Flags) ? arg "-build-testing"
+
]
ghcInternalArgs :: Args
=====================================
hadrian/stack.yaml
=====================================
@@ -16,6 +16,7 @@ packages:
- '.'
- '../utils/ghc-toolchain'
- '../libraries/ghc-platform'
+- '../libraries/semaphore-compat'
nix:
enable: false
=====================================
libraries/semaphore-compat
=====================================
@@ -1 +1 @@
-Subproject commit 7929702401d49bc64d809c501ed5fe80aebc3cc1
+Subproject commit 83b8935203e8c57b189c8a2a19c4173d6a93ea2b
=====================================
testsuite/tests/diagnostic-codes/codes.stdout
=====================================
@@ -21,6 +21,7 @@
[GHC-29747] is untested (constructor = DriverMissingSafeHaskellMode)
[GHC-74284] is untested (constructor = DriverInstantiationNodeInDependencyGeneration)
[GHC-66599] is untested (constructor = DriverNoConfiguredLLVMToolchain)
+[GHC-19877] is untested (constructor = DriverSemaphoreOpenFailure)
[GHC-81325] is untested (constructor = ExpectingMoreArguments)
[GHC-78125] is untested (constructor = AmbiguityPreventsSolvingCt)
[GHC-84170] is untested (constructor = TcRnModMissingRealSrcSpan)
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/7479e9a07d51a73072eec7faabed154…
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/7479e9a07d51a73072eec7faabed154…
You're receiving this email because of your account on gitlab.haskell.org.
1
0
[Git][ghc/ghc][wip/semaphore-v2] Update to semaphore-compat 2.0.0 using v2 of the protocol
by Zubin (＠wz1000) 19 May '26
by Zubin (＠wz1000) 19 May '26
19 May '26
Zubin pushed to branch wip/semaphore-v2 at Glasgow Haskell Compiler / GHC
Commits:
c9f71cd4 by Zubin Duggal at 2026-05-19T14:31:56+05:30
Update to semaphore-compat 2.0.0 using v2 of the protocol
On Linux and other POSIX platforms, GHC's -jsem jobserver client now
speaks v2 of the semaphore-compat protocol, which uses Unix domain
sockets in place of POSIX named semaphores. This avoids the libc-ABI
issues that affected the old implementation. Windows is unaffected
and continues to use the v1 protocol (Win32 named semaphores); its
reported protocol version remains v1.
When GHC receives a -jsem name whose protocol version it does not
support, it emits a -Wsemaphore-version-mismatch warning and falls
back to -j<N> rather than crashing. ghc --info exposes the supported
version in a new "Semaphore version" entry so cabal-install can detect
a mismatch before invoking GHC.
Users on a cabal-install that predates the v2 update will continue to
build successfully on Linux/POSIX, but will lose the cross-process
-jsem coordination and fall back to -j<N> per GHC invocation. Users
must upgrade to a cabal-install that supports protocol v2 to recover
full parallelism.
Also fix a leak in cleanupSem (#27253): cleanupSem used to snapshot
heldTokens and release them before killing the loop, while the loop's
in-flight acquire/release children could still be mutating it.
Cleanup now runs inside the loop's own exit handler, after draining
the active child via a new activeChild TVar, so the snapshot has no
concurrent mutator.
See also:
- GHC proposal amendment: https://github.com/ghc-proposals/ghc-proposals/pull/673
- cabal-install patch: https://github.com/haskell/cabal/pull/11628
- semaphore-compat MR: https://gitlab.haskell.org/ghc/semaphore-compat/-/merge_requests/8
Bump semaphore-compat submodule to 2.0.0
Fixes #25087 and #27253
- - - - -
19 changed files:
- + changelog.d/jobserver-leak-fix
- + changelog.d/semaphore-v2
- compiler/GHC/Driver/Errors/Ppr.hs
- compiler/GHC/Driver/Errors/Types.hs
- compiler/GHC/Driver/Flags.hs
- compiler/GHC/Driver/MakeAction.hs
- compiler/GHC/Driver/MakeSem.hs
- compiler/GHC/Driver/Session.hs
- compiler/GHC/Types/Error/Codes.hs
- docs/users_guide/using-warnings.rst
- docs/users_guide/using.rst
- hadrian/cabal.project
- hadrian/hadrian.cabal
- hadrian/src/Flavour.hs
- hadrian/src/Rules/Generate.hs
- hadrian/src/Settings/Packages.hs
- hadrian/stack.yaml
- libraries/semaphore-compat
- testsuite/tests/diagnostic-codes/codes.stdout
Changes:
=====================================
changelog.d/jobserver-leak-fix
=====================================
@@ -0,0 +1,8 @@
+section: compiler
+issues: #27253
+mrs: !15729
+synopsis:
+ Fix a token leak in the ``-jsem`` jobserver shutdown path
+description:
+ A build interrupted by Ctrl-C while a ``-jsem`` token transfer was in
+ flight could leak that token.
=====================================
changelog.d/semaphore-v2
=====================================
@@ -0,0 +1,30 @@
+section: compiler
+issues: #25087
+mrs: !15729
+synopsis:
+ Update to semaphore-compat 2.0.0 (``-jsem`` protocol v2)
+description:
+ On Linux and other POSIX platforms, GHC's ``-jsem`` jobserver client
+ now speaks v2 of the semaphore-compat protocol, which uses Unix
+ domain sockets in place of POSIX named semaphores. This avoids the
+ libc-ABI issues that affected the old implementation. Windows is
+ unaffected and continues to use the v1 protocol (Win32 named
+ semaphores); its reported protocol version remains v1.
+
+ When GHC receives a ``-jsem`` name whose protocol version it does not
+ support, it now emits a ``-Wsemaphore-version-mismatch`` warning and
+ falls back to ``-j1`` rather than crashing. ``ghc --info`` exposes the
+ supported version in a new ``"Semaphore version"`` entry so
+ cabal-install can detect a mismatch before invoking GHC.
+
+ Users on a ``cabal-install`` that predates the v2 update will continue
+ to build successfully on Linux/POSIX, but will lose the cross-process
+ ``-jsem`` coordination and fall back to ``-j1`` per GHC invocation.
+ To recover full parallelism, upgrade to a ``cabal-install`` that
+ supports protocol v2.
+
+ See also:
+
+ - the `GHC proposal amendment <https://github.com/ghc-proposals/ghc-proposals/pull/673>`_
+ - the `cabal-install patch <https://github.com/haskell/cabal/pull/11628>`_
+ - the `semaphore-compat library MR <https://gitlab.haskell.org/ghc/semaphore-compat/-/merge_requests/8>`_
=====================================
compiler/GHC/Driver/Errors/Ppr.hs
=====================================
@@ -24,6 +24,8 @@ import GHC.Types.Hint
import GHC.Types.SrcLoc
import Data.Version
+import System.Semaphore
+ ( SemaphoreError(..), getSemaphoreProtocolVersion )
import Language.Haskell.Syntax.Decls (RuleDecl(..))
import GHC.Tc.Errors.Types (TcRnMessage)
import GHC.HsToCore.Errors.Types (DsMessage)
@@ -90,6 +92,20 @@ instance Diagnostic GhcMessage where
instance HasDefaultDiagnosticOpts DriverMessageOpts where
defaultOpts = DriverMessageOpts (defaultDiagnosticOpts @PsMessage) (defaultDiagnosticOpts @IfaceMessage)
+pprSemaphoreError :: SemaphoreError -> SDoc
+pprSemaphoreError = \case
+ SemaphoreAlreadyExists nm ->
+ text "a semaphore named" <+> quotes (text nm) <+> text "already exists"
+ SemaphoreDoesNotExist nm ->
+ text "no semaphore named" <+> quotes (text nm)
+ SemaphoreIncompatibleVersion got want ->
+ text "protocol version mismatch (got v"
+ <> int (getSemaphoreProtocolVersion got)
+ <> text ", supported v"
+ <> int (getSemaphoreProtocolVersion want) <> text ")"
+ SemaphoreOtherError ioe ->
+ text (show ioe)
+
instance Diagnostic DriverMessage where
type DiagnosticOpts DriverMessage = DriverMessageOpts
diagnosticMessage opts = \case
@@ -282,6 +298,10 @@ instance Diagnostic DriverMessage where
-> mkSimpleDecorated $
vcat [ text "The following modules are missing a linkable which is needed for creating a library:"
, nest 2 $ hcat (map ppr mods) ]
+ DriverSemaphoreOpenFailure err
+ -> mkSimpleDecorated $
+ text "Failed to open -jsem semaphore:" <+> pprSemaphoreError err <>
+ text "; ignoring -jsem and compiling sequentially."
diagnosticReason = \case
DriverUnknownMessage m
@@ -355,6 +375,8 @@ instance Diagnostic DriverMessage where
-> WarningWithoutFlag
DriverMissingLinkableForModule {}
-> ErrorWithoutFlag
+ DriverSemaphoreOpenFailure {}
+ -> WarningWithFlag Opt_WarnSemaphoreOpenFailure
diagnosticHints = \case
DriverUnknownMessage m
@@ -430,5 +452,14 @@ instance Diagnostic DriverMessage where
-> noHints
DriverMissingLinkableForModule {}
-> noHints
+ DriverSemaphoreOpenFailure (SemaphoreIncompatibleVersion received supported)
+ | received < supported
+ -> [UnknownHint (text "The parent process (e.g. cabal-install) uses an older semaphore protocol."
+ $$ text "Upgrading cabal-install may resolve this." :: SDoc)]
+ | otherwise
+ -> [UnknownHint (text "The parent process (e.g. cabal-install) uses a newer semaphore protocol."
+ $$ text "Upgrading GHC may resolve this." :: SDoc)]
+ DriverSemaphoreOpenFailure {}
+ -> noHints
diagnosticCode = constructorCode @GHC
=====================================
compiler/GHC/Driver/Errors/Types.hs
=====================================
@@ -37,6 +37,7 @@ import qualified GHC.LanguageExtensions as LangExt
import GHC.Generics ( Generic )
+import System.Semaphore ( SemaphoreError )
import GHC.Tc.Errors.Types
import GHC.Iface.Errors.Types
@@ -419,6 +420,13 @@ data DriverMessage where
DriverMissingLinkableForModule :: ![Module] -> DriverMessage
+ {-| DriverSemaphoreOpenFailure is a warning that occurs when GHC fails to
+ open the semaphore specified by @-jsem@, e.g. the socket does not
+ exist, the protocol version is incompatible, or a system error
+ occurred. GHC ignores @-jsem@ and compiles sequentially.
+ -}
+ DriverSemaphoreOpenFailure :: !SemaphoreError -> DriverMessage
+
deriving instance Generic DriverMessage
data DriverMessageOpts =
=====================================
compiler/GHC/Driver/Flags.hs
=====================================
@@ -1115,6 +1115,7 @@ data WarningFlag =
| Opt_WarnUnusableUnpackPragmas -- ^ @since 9.14
| Opt_WarnPatternNamespaceSpecifier -- ^ @since 9.14
| Opt_WarnUnrecognisedModifiers -- ^ @since 10.0
+ | Opt_WarnSemaphoreOpenFailure -- Since 10.0.1
deriving (Eq, Ord, Show, Enum, Bounded)
-- | Return the names of a WarningFlag
@@ -1237,6 +1238,7 @@ warnFlagNames wflag = case wflag of
Opt_WarnUnusableUnpackPragmas -> "unusable-unpack-pragmas" :| []
Opt_WarnPatternNamespaceSpecifier -> "pattern-namespace-specifier" :| []
Opt_WarnUnrecognisedModifiers -> "unrecognised-modifiers" :| []
+ Opt_WarnSemaphoreOpenFailure -> "semaphore-open-failure" :| []
-- -----------------------------------------------------------------------------
-- Standard sets of warning options
@@ -1383,7 +1385,8 @@ standardWarnings -- see Note [Documenting warning flags]
Opt_WarnDeprecatedPragmas,
Opt_WarnRuleLhsEqualities,
Opt_WarnUnusableUnpackPragmas,
- Opt_WarnUnrecognisedModifiers
+ Opt_WarnUnrecognisedModifiers,
+ Opt_WarnSemaphoreOpenFailure
]
-- | Things you get with @-W@.
=====================================
compiler/GHC/Driver/MakeAction.hs
=====================================
@@ -28,6 +28,14 @@ import GHC.Driver.Errors.Types
import GHC.Driver.Messager
import GHC.Driver.MakeSem
+import System.Semaphore
+ ( SemaphoreError, SemaphoreIdentifier )
+
+import GHC.Driver.Config.Diagnostic ( initDiagOpts, initPrintConfig )
+import GHC.Driver.Errors ( printOrThrowDiagnostics )
+import GHC.Types.Error ( singleMessage )
+import GHC.Types.SrcLoc ( noSrcSpan )
+import GHC.Utils.Error ( mkPlainMsgEnvelope )
import GHC.Utils.Logger
import GHC.Utils.TmpFs
@@ -49,7 +57,7 @@ mkWorkerLimit :: DynFlags -> IO WorkerLimit
mkWorkerLimit dflags =
case parMakeCount dflags of
Nothing -> pure $ num_procs 1
- Just (ParMakeSemaphore h) -> pure (JSemLimit (SemaphoreName h))
+ Just (ParMakeSemaphore h) -> pure (JSemLimit h)
Just ParMakeNumProcessors -> num_procs <$> getNumProcessors
Just (ParMakeThisMany n) -> pure $ num_procs n
where
@@ -65,8 +73,8 @@ isWorkerLimitSequential (JSemLimit {}) = False
data WorkerLimit
= NumProcessorsLimit Int
| JSemLimit
- SemaphoreName
- -- ^ Semaphore name to use
+ SemaphoreIdentifier
+ -- ^ Semaphore identifier from @-jsem@
deriving Eq
-- | Environment used when compiling a module
@@ -122,17 +130,24 @@ runNjobsAbstractSem n_jobs action = do
resetNumCapabilities = set_num_caps n_capabilities
MC.bracket_ updNumCapabilities resetNumCapabilities $ action asem
-runWorkerLimit :: WorkerLimit -> (AbstractSem -> IO a) -> IO a
+runWorkerLimit :: Logger -> DynFlags -> WorkerLimit -> (AbstractSem -> IO a) -> IO a
#if defined(wasm32_HOST_ARCH)
-runWorkerLimit _ action = do
+runWorkerLimit _logger _dflags _ action = do
lock <- newMVar ()
action $ AbstractSem (takeMVar lock) (putMVar lock ())
#else
-runWorkerLimit worker_limit action = case worker_limit of
+runWorkerLimit logger dflags worker_limit action = case worker_limit of
NumProcessorsLimit n_jobs ->
runNjobsAbstractSem n_jobs action
- JSemLimit sem ->
- runJSemAbstractSem sem action
+ JSemLimit sem_ident -> do
+ result <- MC.try @SemaphoreError $ runJSemAbstractSem sem_ident action
+ case result of
+ Right a -> return a
+ Left err -> do
+ let diag = DriverSemaphoreOpenFailure err
+ msg = singleMessage $ mkPlainMsgEnvelope (initDiagOpts dflags) noSrcSpan diag
+ printOrThrowDiagnostics logger (initPrintConfig dflags) (initDiagOpts dflags) (GhcDriverMessage <$> msg)
+ runNjobsAbstractSem 1 action
#endif
-- | Build and run a pipeline
@@ -159,7 +174,7 @@ runParPipelines worker_limit plugin_hsc_env diag_wrapper mHscMessager all_pipeli
thread_safe_logger <- liftIO $ makeThreadSafe (hsc_logger plugin_hsc_env)
let thread_safe_hsc_env = plugin_hsc_env { hsc_logger = thread_safe_logger }
- runWorkerLimit worker_limit $ \abstract_sem -> do
+ runWorkerLimit (hsc_logger plugin_hsc_env) (hsc_dflags plugin_hsc_env) worker_limit $ \abstract_sem -> do
let env = MakeEnv { hsc_env = thread_safe_hsc_env
, withLogger = withParLog log_queue_queue_var
, compile_sem = abstract_sem
=====================================
compiler/GHC/Driver/MakeSem.hs
=====================================
@@ -9,9 +9,6 @@ module GHC.Driver.MakeSem
-- by a system semaphore (Posix/Windows)
runJSemAbstractSem
- -- * System semaphores
- , Semaphore, SemaphoreName(..)
-
-- * Abstract semaphores
, AbstractSem(..)
, withAbstractSem
@@ -46,11 +43,14 @@ import Debug.Trace
-- available from the semaphore.
data Jobserver
= Jobserver
- { jSemaphore :: !Semaphore
+ { jSemaphore :: !ClientSemaphore
-- ^ The semaphore which controls available resources
, jobs :: !(TVar JobResources)
-- ^ The currently pending jobs, and the resources
-- obtained from the semaphore
+ , activeChild :: !(TVar (Maybe (ThreadId, TMVar (Maybe MC.SomeException))))
+ -- ^ Handle on the current acquire thread (if any). The loop's exit
+ -- handler reads this to drain a still-running child on shutdown.
}
data JobserverOptions
@@ -81,6 +81,9 @@ data JobResources
, jobsWaiting :: !(OrdList (TMVar ()))
-- ^ Pending jobs waiting on a token, the job will be blocked on the TMVar so putting into
-- the TMVar will allow the job to continue.
+ , heldTokens :: [SemaphoreToken]
+ -- ^ Actual semaphore tokens (for release/cleanup).
+ -- Length should equal tokensOwned - 1 (the implicit token has no SemaphoreToken).
}
instance Outputable JobResources where
@@ -93,9 +96,9 @@ instance Outputable JobResources where
] )
-- | Add one new token.
-addToken :: JobResources -> JobResources
-addToken jobs@( Jobs { tokensOwned = owned, tokensFree = free })
- = jobs { tokensOwned = owned + 1, tokensFree = free + 1 }
+addToken :: SemaphoreToken -> JobResources -> JobResources
+addToken tok jobs@( Jobs { tokensOwned = owned, tokensFree = free, heldTokens = toks })
+ = jobs { tokensOwned = owned + 1, tokensFree = free + 1, heldTokens = tok : toks }
-- | Free one token.
addFreeToken :: JobResources -> JobResources
@@ -111,12 +114,14 @@ removeFreeToken jobs@( Jobs { tokensFree = free })
(text "removeFreeToken:" <+> ppr free)
$ jobs { tokensFree = free - 1 }
--- | Return one owned token.
-removeOwnedToken :: JobResources -> JobResources
-removeOwnedToken jobs@( Jobs { tokensOwned = owned })
+-- | Return one owned token, extracting the 'SemaphoreToken' for release.
+removeOwnedToken :: JobResources -> (SemaphoreToken, JobResources)
+removeOwnedToken jobs@( Jobs { tokensOwned = owned, heldTokens = toks })
= assertPpr (owned > 1)
(text "removeOwnedToken:" <+> ppr owned)
- $ jobs { tokensOwned = owned - 1 }
+ $ case toks of
+ (t:rest) -> (t, jobs { tokensOwned = owned - 1, heldTokens = rest })
+ [] -> panic "removeOwnedToken: no held tokens"
-- | Add one new job to the end of the list of pending jobs.
addJob :: TMVar () -> JobResources -> JobResources
@@ -143,7 +148,7 @@ data JobserverAction
= Idle
-- | A thread is waiting for a token on the semaphore.
| Acquiring
- { activeWaitId :: WaitId
+ { activeThreadId :: ThreadId
, threadFinished :: TMVar (Maybe MC.SomeException) }
-- | Retrieve the 'TMVar' that signals if the current thread has finished,
@@ -189,17 +194,30 @@ releaseJob jobs_tvar = do
return ((), addFreeToken jobs)
--- | Release all tokens owned from the semaphore (to clean up
--- the jobserver at the end).
-cleanupJobserver :: Jobserver -> IO ()
-cleanupJobserver (Jobserver { jSemaphore = sem
- , jobs = jobs_tvar })
- = do
- Jobs { tokensOwned = owned } <- readTVarIO jobs_tvar
- let toks_to_release = owned - 1
- -- Subtract off the implicit token: whoever spawned the ghc process
- -- in the first place is responsible for that token.
- releaseSemaphore sem toks_to_release
+-- | Kill the current acquire thread, if any, and wait for it to exit.
+--
+-- Called from the jobserver loop's exit handler, which runs masked.
+-- Relies on the invariant from 'acquireThread' that a forked child
+-- always fills its 'threadFinished' TMVar before it dies; this is what
+-- lets the 'takeTMVar' below terminate after the 'killThread'.
+drainActiveChild :: Jobserver -> IO ()
+drainActiveChild (Jobserver { activeChild = active_tvar }) = do
+ mb <- readTVarIO active_tvar
+ for_ mb $ \(tid, tmv) -> do
+ killThread tid
+ void $ atomically (takeTMVar tmv)
+ atomically $ writeTVar active_tvar Nothing
+
+-- | Release every token currently in 'heldTokens'.
+--
+-- Called from the jobserver loop's exit handler, which runs masked,
+-- after 'drainActiveChild': no other thread is mutating 'JobResources'
+-- at this point.
+releaseAllHeld :: Jobserver -> IO ()
+releaseAllHeld (Jobserver { jobs = jobs_tvar }) = do
+ Jobs { heldTokens = toks } <- readTVarIO jobs_tvar
+ forM_ toks $ \t ->
+ void $ MC.try @MC.SomeException (releaseSemaphoreToken t)
-- | Dispatch the available tokens acquired from the semaphore
-- to the pending jobs in the job server.
@@ -252,7 +270,7 @@ tracedAtomically origin act = do
return a
renderJobResources :: String -> JobResources -> String
-renderJobResources origin (Jobs own free pending) = showSDocUnsafe $ renderJSON $
+renderJobResources origin (Jobs own free pending _heldToks) = showSDocUnsafe $ renderJSON $
JSObject [ ("name", JSString origin)
, ("owned", JSInt own)
, ("free", JSInt free)
@@ -262,61 +280,68 @@ renderJobResources origin (Jobs own free pending) = showSDocUnsafe $ renderJSON
-- | Spawn a new thread that waits on the semaphore in order to acquire
-- an additional token.
+--
+-- The child is forked masked so the only async-exception delivery point
+-- is the interruptible 'waitOnSemaphore'; the STM commit afterwards then
+-- always runs to completion, so 'threadFinished' is always filled.
+--
+-- The (tid, threadFinished) pair is also published to 'activeChild' so
+-- shutdown can drain the child even after the in-loop 'JobserverState'
+-- is gone.
acquireThread :: Jobserver -> IO JobserverAction
-acquireThread (Jobserver { jSemaphore = sem, jobs = jobs_tvar }) = do
+acquireThread (Jobserver { jSemaphore = sem, jobs = jobs_tvar, activeChild = active_tvar }) = do
threadFinished_tmvar <- newEmptyTMVarIO
- let
- wait_result_action :: Either MC.SomeException Bool -> IO ()
- wait_result_action wait_res =
+ tid <- MC.mask_ $ do
+ tid <- forkIO $ do
+ wait_res <- MC.try @_ @MC.SomeException $ waitOnSemaphore sem
tracedAtomically_ "acquire_thread" do
(r, jb) <- case wait_res of
Left (e :: MC.SomeException) -> do
return $ (Just e, Nothing)
- Right success -> do
- if success
- then do
- modifyJobResources jobs_tvar \ jobs ->
- return (Nothing, addToken jobs)
- else
- return (Nothing, Nothing)
+ Right tok -> do
+ modifyJobResources jobs_tvar \ jobs ->
+ return (Nothing, addToken tok jobs)
putTMVar threadFinished_tmvar r
return jb
- wait_id <- forkWaitOnSemaphoreInterruptible sem wait_result_action
- labelThread (waitingThreadId wait_id) "acquire_thread"
- return $ Acquiring { activeWaitId = wait_id
+ atomically $ writeTVar active_tvar (Just (tid, threadFinished_tmvar))
+ return tid
+ labelThread tid "acquire_thread"
+ return $ Acquiring { activeThreadId = tid
, threadFinished = threadFinished_tmvar }
-- | Spawn a thread to release ownership of one resource from the semaphore,
-- provided we have spare resources and no pending jobs.
releaseThread :: Jobserver -> IO JobserverAction
-releaseThread (Jobserver { jSemaphore = sem, jobs = jobs_tvar }) = do
+releaseThread (Jobserver { jobs = jobs_tvar }) = do
threadFinished_tmvar <- newEmptyTMVarIO
MC.mask_ do
-- Pre-release the resource so that another thread doesn't take control of it
-- just as we release the lock on the semaphore.
- still_ok_to_release
+ mb_tok
<- tracedAtomically "pre_release" $
modifyJobResources jobs_tvar \ jobs ->
if guardRelease jobs
- -- TODO: should this also debounce?
- then return (True , removeOwnedToken $ removeFreeToken jobs)
- else return (False, jobs)
- if not still_ok_to_release
- then return Idle
- else do
- tid <- forkIO $ do
- x <- MC.try $ releaseSemaphore sem 1
- tracedAtomically_ "post-release" $ do
- (r, jobs) <- case x of
- Left (e :: MC.SomeException) -> do
- modifyJobResources jobs_tvar \ jobs ->
- return (Just e, addToken jobs)
- Right _ -> do
- return (Nothing, Nothing)
- putTMVar threadFinished_tmvar r
- return jobs
- labelThread tid "release_thread"
- return Idle
+ then let (tok, jobs') = removeOwnedToken $ removeFreeToken jobs
+ in return (Just tok, jobs')
+ else return (Nothing, jobs)
+ case mb_tok of
+ Nothing ->
+ -- Not OK to release: there are other pending jobs that could make use of the token.
+ return Idle
+ Just tok -> do
+ tid <- forkIO $ do
+ x <- MC.try @MC.SomeException $ releaseSemaphoreToken tok
+ tracedAtomically_ "post-release" $ do
+ (r, jobs) <- case x of
+ Left (e :: MC.SomeException) -> do
+ modifyJobResources jobs_tvar \ jobs ->
+ return (Just e, addToken tok jobs)
+ Right _ -> do
+ return (Nothing, Nothing)
+ putTMVar threadFinished_tmvar r
+ return jobs
+ labelThread tid "release_thread"
+ return Idle
-- | When there are pending jobs but no free tokens,
-- spawn a thread to acquire a new token from the semaphore.
@@ -363,13 +388,14 @@ tryRelease _ _ = retry
-- | Wait for an active thread to finish. Once it finishes:
--
-- - set the 'JobserverAction' to 'Idle',
+-- - clear the 'activeChild' handle,
-- - update the number of capabilities to reflect the number
-- of owned tokens from the semaphore.
tryNoticeIdle :: JobserverOptions
- -> TVar JobResources
+ -> Jobserver
-> JobserverState
-> STM (IO JobserverState)
-tryNoticeIdle opts jobs_tvar jobserver_state
+tryNoticeIdle opts (Jobserver { jobs = jobs_tvar, activeChild = active_tvar }) jobserver_state
| Just threadFinished_tmvar <- activeThread_maybe $ jobserverAction jobserver_state
= sync_num_caps (canChangeNumCaps jobserver_state) threadFinished_tmvar
| otherwise
@@ -381,6 +407,7 @@ tryNoticeIdle opts jobs_tvar jobserver_state
sync_num_caps can_change_numcaps_tvar threadFinished_tmvar = do
mb_ex <- takeTMVar threadFinished_tmvar
for_ mb_ex MC.throwM
+ writeTVar active_tvar Nothing
Jobs { tokensOwned } <- readTVar jobs_tvar
can_change_numcaps <- readTVar can_change_numcaps_tvar
guard can_change_numcaps
@@ -404,11 +431,11 @@ tryStopThread :: TVar JobResources
-> STM (IO JobserverState)
tryStopThread jobs_tvar jsj = do
case jobserverAction jsj of
- Acquiring { activeWaitId = wait_id } -> do
+ Acquiring { activeThreadId = tid } -> do
jobs <- readTVar jobs_tvar
guard $ null (jobsWaiting jobs)
return do
- interruptWaitOnSemaphore wait_id
+ killThread tid
return $ jsj { jobserverAction = Idle }
_ -> retry
@@ -430,30 +457,38 @@ jobserverLoop opts sjs@(Jobserver { jobs = jobs_tvar })
action <- atomically $ asum $ (\x -> x s) <$>
[ tryRelease sjs
, tryAcquire opts sjs
- , tryNoticeIdle opts jobs_tvar
+ , tryNoticeIdle opts sjs
, tryStopThread jobs_tvar
]
s <- action
loop s
--- | Create a new jobserver using the given semaphore handle.
-makeJobserver :: SemaphoreName -> IO (AbstractSem, IO ())
-makeJobserver sem_name = do
- semaphore <- openSemaphore sem_name
+-- | Create a new jobserver using the given semaphore identifier.
+makeJobserver :: SemaphoreIdentifier -> IO (AbstractSem, IO ())
+makeJobserver sem_ident = do
+ semaphore <- openSemaphore sem_ident >>= either MC.throwM pure
let
init_jobs =
Jobs { tokensOwned = 1
, tokensFree = 1
, jobsWaiting = NilOL
+ , heldTokens = []
}
jobs_tvar <- newTVarIO init_jobs
+ active_tvar <- newTVarIO Nothing
let
opts = defaultJobserverOptions -- TODO: allow this to be configured
- sjs = Jobserver { jSemaphore = semaphore
- , jobs = jobs_tvar }
+ sjs = Jobserver { jSemaphore = semaphore
+ , jobs = jobs_tvar
+ , activeChild = active_tvar }
loop_finished_mvar <- newEmptyMVar
loop_tid <- forkIOWithUnmask \ unmask -> do
r <- try $ unmask $ jobserverLoop opts sjs
+ -- Always-run exit handler: any child the loop spawned is still alive
+ -- in its own thread, so drain it before touching jobs_tvar. No one
+ -- else can mutate the resources once both are dead.
+ drainActiveChild sjs
+ releaseAllHeld sjs
putMVar loop_finished_mvar $
case r of
Left e
@@ -467,8 +502,8 @@ makeJobserver sem_name = do
acquireSem = acquireJob jobs_tvar
releaseSem = releaseJob jobs_tvar
cleanupSem = do
- -- this is interruptible
- cleanupJobserver sjs
+ -- Trigger the loop's exit handler; it drains the active child and
+ -- releases all held tokens, then signals loop_finished_mvar.
killThread loop_tid
mb_ex <- takeMVar loop_finished_mvar
for_ mb_ex MC.throwM
@@ -477,12 +512,12 @@ makeJobserver sem_name = do
-- | Implement an abstract semaphore using a semaphore 'Jobserver'
-- which queries the system semaphore of the given name for resources.
-runJSemAbstractSem :: SemaphoreName -- ^ the system semaphore to use
+runJSemAbstractSem :: SemaphoreIdentifier -- ^ the semaphore identifier (from @-jsem@)
-> (AbstractSem -> IO a) -- ^ the operation to run
-- which requires a semaphore
-> IO a
-runJSemAbstractSem sem action = MC.mask \ unmask -> do
- (abs, cleanup) <- makeJobserver sem
+runJSemAbstractSem sem_ident action = MC.mask \ unmask -> do
+ (abs, cleanup) <- makeJobserver sem_ident
r <- try $ unmask $ action abs
case r of
Left (e1 :: MC.SomeException) -> do
@@ -517,8 +552,13 @@ increases the number of `free` jobs. If there are more pending jobs when the fre
is increased, the token is immediately reused (see `modifyJobResources`).
The `jobServerLoop` interacts with the system semaphore: when there are pending
-jobs, `acquireThread` blocks, waiting for a token from the semaphore. Once a
-token is obtained, it increases the owned count.
+jobs, `acquireThread` forks a child that calls the interruptible
+`waitOnSemaphore`. The child is forked in the masked state, so the only place
+an async exception can be delivered is the wait itself; once the wait returns,
+the child's STM commit always completes, recording either the new token in
+`heldTokens` or the failure exception in `threadFinished`. The (tid, tmvar)
+pair is also published in `activeChild` so the loop's exit handler can drain
+the child on shutdown even after the in-loop `JobserverState` is gone.
When GHC has free tokens (tokens from the semaphore that it is not using),
no pending jobs, and the debounce has expired, then `releaseThread` will
@@ -531,6 +571,12 @@ This second token is no longer needed, so we should cancel the wait
(as it would not be used to do any work, and not be returned until the debounce).
We only need to kill `acquireJob`, because `releaseJob` never blocks.
+Shutdown starts with `killThread loop_tid`. The loop's exit handler then
+runs `drainActiveChild` followed by `releaseAllHeld`; only then does the
+loop signal `loop_finished_mvar`. This sequence makes the heldTokens
+snapshot consistent because no other thread can mutate it once the loop and
+its child are both dead.
+
Note [Eventlog Messages for jsem]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
It can be tricky to verify that the work is shared adequately across different
=====================================
compiler/GHC/Driver/Session.hs
=====================================
@@ -2445,6 +2445,7 @@ wWarningFlagsDeps = [minBound..maxBound] >>= \x -> case x of
Opt_WarnUnusableUnpackPragmas -> warnSpec x
Opt_WarnPatternNamespaceSpecifier -> warnSpec x
Opt_WarnUnrecognisedModifiers -> warnSpec x
+ Opt_WarnSemaphoreOpenFailure -> warnSpec x
warningGroupsDeps :: [(Deprecation, FlagSpec WarningGroup)]
warningGroupsDeps = map mk warningGroups
=====================================
compiler/GHC/Types/Error/Codes.hs
=====================================
@@ -403,6 +403,7 @@ type family GhcDiagnosticCode c = n | n -> c where
GhcDiagnosticCode "DriverInstantiationNodeInDependencyGeneration" = 74284
GhcDiagnosticCode "DriverNoConfiguredLLVMToolchain" = 66599
GhcDiagnosticCode "DriverMissingLinkableForModule" = 47338
+ GhcDiagnosticCode "DriverSemaphoreOpenFailure" = 19877
-- Constraint solver diagnostic codes
GhcDiagnosticCode "BadTelescope" = 97739
=====================================
docs/users_guide/using-warnings.rst
=====================================
@@ -2721,6 +2721,23 @@ of ``-W(no-)*``.
f :: a %True -> a
g :: a %(k :: Int) -> a
+.. ghc-flag:: -Wsemaphore-open-failure
+ :shortdesc: warn when GHC cannot open the ``-jsem`` semaphore.
+ :type: dynamic
+ :reverse: -Wno-semaphore-open-failure
+ :category:
+
+ :since: 10.0.1
+
+ Warn when GHC is invoked with :ghc-flag:`-jsem` but the semaphore
+ cannot be opened (e.g. the socket does not exist, the protocol
+ version is incompatible, or a system error occurred). When this
+ occurs, GHC ignores ``-jsem`` and compiles modules sequentially.
+
+ A common cause is ``cabal-install`` and GHC being built against
+ different versions of the ``semaphore-compat`` library; upgrading
+ both to versions that use the same protocol resolves the mismatch.
+
----
If you're feeling really paranoid, the :ghc-flag:`-dcore-lint` option is a good choice.
=====================================
docs/users_guide/using.rst
=====================================
@@ -797,7 +797,12 @@ There are two kinds of participants in the GHC Jobserver protocol:
Perform compilation in parallel when possible, coordinating with other
processes through the semaphore ⟨sem⟩ (specified as a string).
- Error if the semaphore doesn't exist.
+
+ If the semaphore cannot be opened (e.g. the socket does not exist
+ or its protocol version is incompatible with this GHC), GHC emits
+ a :ghc-flag:`-Wsemaphore-open-failure` warning and compiles
+ sequentially, using only the implicit token inherited from the
+ parent process.
Use of ``-jsem`` will override use of :ghc-flag:`-j[⟨n⟩]`,
and vice-versa.
=====================================
hadrian/cabal.project
=====================================
@@ -1,6 +1,7 @@
packages: ./
../utils/ghc-toolchain/
../libraries/ghc-platform/
+ ../libraries/semaphore-compat/
-- This essentially freezes the build plan for hadrian
-- It would be wise to keep this up to date with the state set in .gitlab/ci.sh.
=====================================
hadrian/hadrian.cabal
=====================================
@@ -172,6 +172,7 @@ executable hadrian
, base16-bytestring >= 0.1.1 && < 1.1.0.0
, ghc-platform
, ghc-toolchain
+ , semaphore-compat
ghc-options: -Wall
-Wincomplete-record-updates
-Wredundant-constraints
=====================================
hadrian/src/Flavour.hs
=====================================
@@ -149,10 +149,6 @@ werror =
-- unix has many unused imports
, package unix
? mconcat [arg "-Wwarn=unused-imports", arg "-Wwarn=unused-top-binds"]
- -- semaphore-compat relies on sem_getvalue as provided by unix, which is
- -- not implemented on Darwin and therefore throws a deprecation warning
- , package semaphoreCompat
- ? mconcat [arg "-Wwarn=deprecations"]
]
, builder Ghc
? package rts
=====================================
hadrian/src/Rules/Generate.hs
=====================================
@@ -25,6 +25,7 @@ import Utilities
import GHC.Toolchain as Toolchain hiding (HsCpp(HsCpp))
import GHC.Platform.ArchOS
import Settings.Program (ghcWithInterpreter)
+import System.Semaphore (semaphoreVersion, getSemaphoreProtocolVersion)
-- | Track this file to rebuild generated files whenever it changes.
trackGenerateHs :: Expr ()
@@ -488,6 +489,7 @@ generateSettings settingsFile = do
, ("RTS ways", escapeArgs . map show . Set.toList <$> getRtsWays)
, ("Relative Global Package DB", pure rel_pkg_db)
, ("base unit-id", pure base_unit_id)
+ , ("Semaphore version", pure (show (getSemaphoreProtocolVersion semaphoreVersion)))
]
let showTuple (k, v) = "(" ++ show k ++ ", " ++ show v ++ ")"
pure $ case settings of
=====================================
hadrian/src/Settings/Packages.hs
=====================================
@@ -231,6 +231,10 @@ packageArgs = do
, package hpcBin
? builder (Cabal Flags) ? arg "-build-tool-depends"
+ ------------------------------ semaphore-compat ----------------------------
+ , package semaphoreCompat
+ ? builder (Cabal Flags) ? arg "-build-testing"
+
]
ghcInternalArgs :: Args
=====================================
hadrian/stack.yaml
=====================================
@@ -16,6 +16,7 @@ packages:
- '.'
- '../utils/ghc-toolchain'
- '../libraries/ghc-platform'
+- '../libraries/semaphore-compat'
nix:
enable: false
=====================================
libraries/semaphore-compat
=====================================
@@ -1 +1 @@
-Subproject commit 7929702401d49bc64d809c501ed5fe80aebc3cc1
+Subproject commit 6342de11ac80752e6111b1339abeba7617e195a3
=====================================
testsuite/tests/diagnostic-codes/codes.stdout
=====================================
@@ -21,6 +21,7 @@
[GHC-29747] is untested (constructor = DriverMissingSafeHaskellMode)
[GHC-74284] is untested (constructor = DriverInstantiationNodeInDependencyGeneration)
[GHC-66599] is untested (constructor = DriverNoConfiguredLLVMToolchain)
+[GHC-19877] is untested (constructor = DriverSemaphoreOpenFailure)
[GHC-81325] is untested (constructor = ExpectingMoreArguments)
[GHC-78125] is untested (constructor = AmbiguityPreventsSolvingCt)
[GHC-84170] is untested (constructor = TcRnModMissingRealSrcSpan)
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/c9f71cd4eea85c76d99d9a13ecc6b7d…
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/c9f71cd4eea85c76d99d9a13ecc6b7d…
You're receiving this email because of your account on gitlab.haskell.org.
1
0
[Git][ghc/ghc][wip/semaphore-v2] Update to semaphore-compat 2.0.0 using v2 of the protocol
by Zubin (＠wz1000) 19 May '26
by Zubin (＠wz1000) 19 May '26
19 May '26
Zubin pushed to branch wip/semaphore-v2 at Glasgow Haskell Compiler / GHC
Commits:
d6b0d41e by Zubin Duggal at 2026-05-19T14:30:32+05:30
Update to semaphore-compat 2.0.0 using v2 of the protocol
On Linux and other POSIX platforms, GHC's -jsem jobserver client now
speaks v2 of the semaphore-compat protocol, which uses Unix domain
sockets in place of POSIX named semaphores. This avoids the libc-ABI
issues that affected the old implementation. Windows is unaffected
and continues to use the v1 protocol (Win32 named semaphores); its
reported protocol version remains v1.
When GHC receives a -jsem name whose protocol version it does not
support, it emits a -Wsemaphore-version-mismatch warning and falls
back to -j<N> rather than crashing. ghc --info exposes the supported
version in a new "Semaphore version" entry so cabal-install can detect
a mismatch before invoking GHC.
Users on a cabal-install that predates the v2 update will continue to
build successfully on Linux/POSIX, but will lose the cross-process
-jsem coordination and fall back to -j<N> per GHC invocation. Users
must upgrade to a cabal-install that supports protocol v2 to recover
full parallelism.
Also fix a leak in cleanupSem (#27253): cleanupSem used to snapshot
heldTokens and release them before killing the loop, while the loop's
in-flight acquire/release children could still be mutating it.
Cleanup now runs inside the loop's own exit handler, after draining
the active child via a new activeChild TVar, so the snapshot has no
concurrent mutator.
See also:
- GHC proposal amendment: https://github.com/ghc-proposals/ghc-proposals/pull/673
- cabal-install patch: https://github.com/haskell/cabal/pull/11628
- semaphore-compat MR: https://gitlab.haskell.org/ghc/semaphore-compat/-/merge_requests/8
Bump semaphore-compat submodule to 2.0.0
Fixes #25087 and #27253
- - - - -
19 changed files:
- + changelog.d/jobserver-leak-fix
- + changelog.d/semaphore-v2
- compiler/GHC/Driver/Errors/Ppr.hs
- compiler/GHC/Driver/Errors/Types.hs
- compiler/GHC/Driver/Flags.hs
- compiler/GHC/Driver/MakeAction.hs
- compiler/GHC/Driver/MakeSem.hs
- compiler/GHC/Driver/Session.hs
- compiler/GHC/Types/Error/Codes.hs
- docs/users_guide/using-warnings.rst
- docs/users_guide/using.rst
- hadrian/cabal.project
- hadrian/hadrian.cabal
- hadrian/src/Flavour.hs
- hadrian/src/Rules/Generate.hs
- hadrian/src/Settings/Packages.hs
- hadrian/stack.yaml
- libraries/semaphore-compat
- testsuite/tests/diagnostic-codes/codes.stdout
Changes:
=====================================
changelog.d/jobserver-leak-fix
=====================================
@@ -0,0 +1,8 @@
+section: compiler
+issues: #27253
+mrs: !15729
+synopsis:
+ Fix a token leak in the ``-jsem`` jobserver shutdown path
+description:
+ A build interrupted by Ctrl-C while a ``-jsem`` token transfer was in
+ flight could leak that token.
=====================================
changelog.d/semaphore-v2
=====================================
@@ -0,0 +1,30 @@
+section: compiler
+issues: #25087
+mrs: !15729
+synopsis:
+ Update to semaphore-compat 2.0.0 (``-jsem`` protocol v2)
+description:
+ On Linux and other POSIX platforms, GHC's ``-jsem`` jobserver client
+ now speaks v2 of the semaphore-compat protocol, which uses Unix
+ domain sockets in place of POSIX named semaphores. This avoids the
+ libc-ABI issues that affected the old implementation. Windows is
+ unaffected and continues to use the v1 protocol (Win32 named
+ semaphores); its reported protocol version remains v1.
+
+ When GHC receives a ``-jsem`` name whose protocol version it does not
+ support, it now emits a ``-Wsemaphore-version-mismatch`` warning and
+ falls back to ``-j1`` rather than crashing. ``ghc --info`` exposes the
+ supported version in a new ``"Semaphore version"`` entry so
+ cabal-install can detect a mismatch before invoking GHC.
+
+ Users on a ``cabal-install`` that predates the v2 update will continue
+ to build successfully on Linux/POSIX, but will lose the cross-process
+ ``-jsem`` coordination and fall back to ``-j1`` per GHC invocation.
+ To recover full parallelism, upgrade to a ``cabal-install`` that
+ supports protocol v2.
+
+ See also:
+
+ - the `GHC proposal amendment <https://github.com/ghc-proposals/ghc-proposals/pull/673>`_
+ - the `cabal-install patch <https://github.com/haskell/cabal/pull/11628>`_
+ - the `semaphore-compat library MR <https://gitlab.haskell.org/ghc/semaphore-compat/-/merge_requests/8>`_
=====================================
compiler/GHC/Driver/Errors/Ppr.hs
=====================================
@@ -24,6 +24,8 @@ import GHC.Types.Hint
import GHC.Types.SrcLoc
import Data.Version
+import System.Semaphore
+ ( SemaphoreError(..), getSemaphoreProtocolVersion )
import Language.Haskell.Syntax.Decls (RuleDecl(..))
import GHC.Tc.Errors.Types (TcRnMessage)
import GHC.HsToCore.Errors.Types (DsMessage)
@@ -90,6 +92,20 @@ instance Diagnostic GhcMessage where
instance HasDefaultDiagnosticOpts DriverMessageOpts where
defaultOpts = DriverMessageOpts (defaultDiagnosticOpts @PsMessage) (defaultDiagnosticOpts @IfaceMessage)
+pprSemaphoreError :: SemaphoreError -> SDoc
+pprSemaphoreError = \case
+ SemaphoreAlreadyExists nm ->
+ text "a semaphore named" <+> quotes (text nm) <+> text "already exists"
+ SemaphoreDoesNotExist nm ->
+ text "no semaphore named" <+> quotes (text nm)
+ SemaphoreIncompatibleVersion got want ->
+ text "protocol version mismatch (got v"
+ <> int (getSemaphoreProtocolVersion got)
+ <> text ", supported v"
+ <> int (getSemaphoreProtocolVersion want) <> text ")"
+ SemaphoreOtherError ioe ->
+ text (show ioe)
+
instance Diagnostic DriverMessage where
type DiagnosticOpts DriverMessage = DriverMessageOpts
diagnosticMessage opts = \case
@@ -282,6 +298,10 @@ instance Diagnostic DriverMessage where
-> mkSimpleDecorated $
vcat [ text "The following modules are missing a linkable which is needed for creating a library:"
, nest 2 $ hcat (map ppr mods) ]
+ DriverSemaphoreOpenFailure err
+ -> mkSimpleDecorated $
+ text "Failed to open -jsem semaphore:" <+> pprSemaphoreError err <>
+ text "; ignoring -jsem and compiling sequentially."
diagnosticReason = \case
DriverUnknownMessage m
@@ -355,6 +375,8 @@ instance Diagnostic DriverMessage where
-> WarningWithoutFlag
DriverMissingLinkableForModule {}
-> ErrorWithoutFlag
+ DriverSemaphoreOpenFailure {}
+ -> WarningWithFlag Opt_WarnSemaphoreOpenFailure
diagnosticHints = \case
DriverUnknownMessage m
@@ -430,5 +452,14 @@ instance Diagnostic DriverMessage where
-> noHints
DriverMissingLinkableForModule {}
-> noHints
+ DriverSemaphoreOpenFailure (SemaphoreIncompatibleVersion received supported)
+ | received < supported
+ -> [UnknownHint (text "The parent process (e.g. cabal-install) uses an older semaphore protocol."
+ $$ text "Upgrading cabal-install may resolve this." :: SDoc)]
+ | otherwise
+ -> [UnknownHint (text "The parent process (e.g. cabal-install) uses a newer semaphore protocol."
+ $$ text "Upgrading GHC may resolve this." :: SDoc)]
+ DriverSemaphoreOpenFailure {}
+ -> noHints
diagnosticCode = constructorCode @GHC
=====================================
compiler/GHC/Driver/Errors/Types.hs
=====================================
@@ -37,6 +37,7 @@ import qualified GHC.LanguageExtensions as LangExt
import GHC.Generics ( Generic )
+import System.Semaphore ( SemaphoreError )
import GHC.Tc.Errors.Types
import GHC.Iface.Errors.Types
@@ -419,6 +420,13 @@ data DriverMessage where
DriverMissingLinkableForModule :: ![Module] -> DriverMessage
+ {-| DriverSemaphoreOpenFailure is a warning that occurs when GHC fails to
+ open the semaphore specified by @-jsem@, e.g. the socket does not
+ exist, the protocol version is incompatible, or a system error
+ occurred. GHC ignores @-jsem@ and compiles sequentially.
+ -}
+ DriverSemaphoreOpenFailure :: !SemaphoreError -> DriverMessage
+
deriving instance Generic DriverMessage
data DriverMessageOpts =
=====================================
compiler/GHC/Driver/Flags.hs
=====================================
@@ -1115,6 +1115,7 @@ data WarningFlag =
| Opt_WarnUnusableUnpackPragmas -- ^ @since 9.14
| Opt_WarnPatternNamespaceSpecifier -- ^ @since 9.14
| Opt_WarnUnrecognisedModifiers -- ^ @since 10.0
+ | Opt_WarnSemaphoreOpenFailure -- Since 10.0.1
deriving (Eq, Ord, Show, Enum, Bounded)
-- | Return the names of a WarningFlag
@@ -1237,6 +1238,7 @@ warnFlagNames wflag = case wflag of
Opt_WarnUnusableUnpackPragmas -> "unusable-unpack-pragmas" :| []
Opt_WarnPatternNamespaceSpecifier -> "pattern-namespace-specifier" :| []
Opt_WarnUnrecognisedModifiers -> "unrecognised-modifiers" :| []
+ Opt_WarnSemaphoreOpenFailure -> "semaphore-open-failure" :| []
-- -----------------------------------------------------------------------------
-- Standard sets of warning options
@@ -1383,7 +1385,8 @@ standardWarnings -- see Note [Documenting warning flags]
Opt_WarnDeprecatedPragmas,
Opt_WarnRuleLhsEqualities,
Opt_WarnUnusableUnpackPragmas,
- Opt_WarnUnrecognisedModifiers
+ Opt_WarnUnrecognisedModifiers,
+ Opt_WarnSemaphoreOpenFailure
]
-- | Things you get with @-W@.
=====================================
compiler/GHC/Driver/MakeAction.hs
=====================================
@@ -28,6 +28,14 @@ import GHC.Driver.Errors.Types
import GHC.Driver.Messager
import GHC.Driver.MakeSem
+import System.Semaphore
+ ( SemaphoreError, SemaphoreIdentifier )
+
+import GHC.Driver.Config.Diagnostic ( initDiagOpts, initPrintConfig )
+import GHC.Driver.Errors ( printOrThrowDiagnostics )
+import GHC.Types.Error ( singleMessage )
+import GHC.Types.SrcLoc ( noSrcSpan )
+import GHC.Utils.Error ( mkPlainMsgEnvelope )
import GHC.Utils.Logger
import GHC.Utils.TmpFs
@@ -49,7 +57,7 @@ mkWorkerLimit :: DynFlags -> IO WorkerLimit
mkWorkerLimit dflags =
case parMakeCount dflags of
Nothing -> pure $ num_procs 1
- Just (ParMakeSemaphore h) -> pure (JSemLimit (SemaphoreName h))
+ Just (ParMakeSemaphore h) -> pure (JSemLimit h)
Just ParMakeNumProcessors -> num_procs <$> getNumProcessors
Just (ParMakeThisMany n) -> pure $ num_procs n
where
@@ -65,8 +73,8 @@ isWorkerLimitSequential (JSemLimit {}) = False
data WorkerLimit
= NumProcessorsLimit Int
| JSemLimit
- SemaphoreName
- -- ^ Semaphore name to use
+ SemaphoreIdentifier
+ -- ^ Semaphore identifier from @-jsem@
deriving Eq
-- | Environment used when compiling a module
@@ -122,17 +130,24 @@ runNjobsAbstractSem n_jobs action = do
resetNumCapabilities = set_num_caps n_capabilities
MC.bracket_ updNumCapabilities resetNumCapabilities $ action asem
-runWorkerLimit :: WorkerLimit -> (AbstractSem -> IO a) -> IO a
+runWorkerLimit :: Logger -> DynFlags -> WorkerLimit -> (AbstractSem -> IO a) -> IO a
#if defined(wasm32_HOST_ARCH)
-runWorkerLimit _ action = do
+runWorkerLimit _logger _dflags _ action = do
lock <- newMVar ()
action $ AbstractSem (takeMVar lock) (putMVar lock ())
#else
-runWorkerLimit worker_limit action = case worker_limit of
+runWorkerLimit logger dflags worker_limit action = case worker_limit of
NumProcessorsLimit n_jobs ->
runNjobsAbstractSem n_jobs action
- JSemLimit sem ->
- runJSemAbstractSem sem action
+ JSemLimit sem_ident -> do
+ result <- MC.try @SemaphoreError $ runJSemAbstractSem sem_ident action
+ case result of
+ Right a -> return a
+ Left err -> do
+ let diag = DriverSemaphoreOpenFailure err
+ msg = singleMessage $ mkPlainMsgEnvelope (initDiagOpts dflags) noSrcSpan diag
+ printOrThrowDiagnostics logger (initPrintConfig dflags) (initDiagOpts dflags) (GhcDriverMessage <$> msg)
+ runNjobsAbstractSem 1 action
#endif
-- | Build and run a pipeline
@@ -159,7 +174,7 @@ runParPipelines worker_limit plugin_hsc_env diag_wrapper mHscMessager all_pipeli
thread_safe_logger <- liftIO $ makeThreadSafe (hsc_logger plugin_hsc_env)
let thread_safe_hsc_env = plugin_hsc_env { hsc_logger = thread_safe_logger }
- runWorkerLimit worker_limit $ \abstract_sem -> do
+ runWorkerLimit (hsc_logger plugin_hsc_env) (hsc_dflags plugin_hsc_env) worker_limit $ \abstract_sem -> do
let env = MakeEnv { hsc_env = thread_safe_hsc_env
, withLogger = withParLog log_queue_queue_var
, compile_sem = abstract_sem
=====================================
compiler/GHC/Driver/MakeSem.hs
=====================================
@@ -9,9 +9,6 @@ module GHC.Driver.MakeSem
-- by a system semaphore (Posix/Windows)
runJSemAbstractSem
- -- * System semaphores
- , Semaphore, SemaphoreName(..)
-
-- * Abstract semaphores
, AbstractSem(..)
, withAbstractSem
@@ -46,11 +43,14 @@ import Debug.Trace
-- available from the semaphore.
data Jobserver
= Jobserver
- { jSemaphore :: !Semaphore
+ { jSemaphore :: !ClientSemaphore
-- ^ The semaphore which controls available resources
, jobs :: !(TVar JobResources)
-- ^ The currently pending jobs, and the resources
-- obtained from the semaphore
+ , activeChild :: !(TVar (Maybe (ThreadId, TMVar (Maybe MC.SomeException))))
+ -- ^ Handle on the current acquire thread (if any). The loop's exit
+ -- handler reads this to drain a still-running child on shutdown.
}
data JobserverOptions
@@ -81,6 +81,9 @@ data JobResources
, jobsWaiting :: !(OrdList (TMVar ()))
-- ^ Pending jobs waiting on a token, the job will be blocked on the TMVar so putting into
-- the TMVar will allow the job to continue.
+ , heldTokens :: [SemaphoreToken]
+ -- ^ Actual semaphore tokens (for release/cleanup).
+ -- Length should equal tokensOwned - 1 (the implicit token has no SemaphoreToken).
}
instance Outputable JobResources where
@@ -93,9 +96,9 @@ instance Outputable JobResources where
] )
-- | Add one new token.
-addToken :: JobResources -> JobResources
-addToken jobs@( Jobs { tokensOwned = owned, tokensFree = free })
- = jobs { tokensOwned = owned + 1, tokensFree = free + 1 }
+addToken :: SemaphoreToken -> JobResources -> JobResources
+addToken tok jobs@( Jobs { tokensOwned = owned, tokensFree = free, heldTokens = toks })
+ = jobs { tokensOwned = owned + 1, tokensFree = free + 1, heldTokens = tok : toks }
-- | Free one token.
addFreeToken :: JobResources -> JobResources
@@ -111,12 +114,14 @@ removeFreeToken jobs@( Jobs { tokensFree = free })
(text "removeFreeToken:" <+> ppr free)
$ jobs { tokensFree = free - 1 }
--- | Return one owned token.
-removeOwnedToken :: JobResources -> JobResources
-removeOwnedToken jobs@( Jobs { tokensOwned = owned })
+-- | Return one owned token, extracting the 'SemaphoreToken' for release.
+removeOwnedToken :: JobResources -> (SemaphoreToken, JobResources)
+removeOwnedToken jobs@( Jobs { tokensOwned = owned, heldTokens = toks })
= assertPpr (owned > 1)
(text "removeOwnedToken:" <+> ppr owned)
- $ jobs { tokensOwned = owned - 1 }
+ $ case toks of
+ (t:rest) -> (t, jobs { tokensOwned = owned - 1, heldTokens = rest })
+ [] -> panic "removeOwnedToken: no held tokens"
-- | Add one new job to the end of the list of pending jobs.
addJob :: TMVar () -> JobResources -> JobResources
@@ -143,7 +148,7 @@ data JobserverAction
= Idle
-- | A thread is waiting for a token on the semaphore.
| Acquiring
- { activeWaitId :: WaitId
+ { activeThreadId :: ThreadId
, threadFinished :: TMVar (Maybe MC.SomeException) }
-- | Retrieve the 'TMVar' that signals if the current thread has finished,
@@ -189,17 +194,30 @@ releaseJob jobs_tvar = do
return ((), addFreeToken jobs)
--- | Release all tokens owned from the semaphore (to clean up
--- the jobserver at the end).
-cleanupJobserver :: Jobserver -> IO ()
-cleanupJobserver (Jobserver { jSemaphore = sem
- , jobs = jobs_tvar })
- = do
- Jobs { tokensOwned = owned } <- readTVarIO jobs_tvar
- let toks_to_release = owned - 1
- -- Subtract off the implicit token: whoever spawned the ghc process
- -- in the first place is responsible for that token.
- releaseSemaphore sem toks_to_release
+-- | Kill the current acquire thread, if any, and wait for it to exit.
+--
+-- Called from the jobserver loop's exit handler, which runs masked.
+-- Relies on the invariant from 'acquireThread' that a forked child
+-- always fills its 'threadFinished' TMVar before it dies; this is what
+-- lets the 'takeTMVar' below terminate after the 'killThread'.
+drainActiveChild :: Jobserver -> IO ()
+drainActiveChild (Jobserver { activeChild = active_tvar }) = do
+ mb <- readTVarIO active_tvar
+ for_ mb $ \(tid, tmv) -> do
+ killThread tid
+ void $ atomically (takeTMVar tmv)
+ atomically $ writeTVar active_tvar Nothing
+
+-- | Release every token currently in 'heldTokens'.
+--
+-- Called from the jobserver loop's exit handler, which runs masked,
+-- after 'drainActiveChild': no other thread is mutating 'JobResources'
+-- at this point.
+releaseAllHeld :: Jobserver -> IO ()
+releaseAllHeld (Jobserver { jobs = jobs_tvar }) = do
+ Jobs { heldTokens = toks } <- readTVarIO jobs_tvar
+ forM_ toks $ \t ->
+ void $ MC.try @MC.SomeException (releaseSemaphoreToken t)
-- | Dispatch the available tokens acquired from the semaphore
-- to the pending jobs in the job server.
@@ -252,7 +270,7 @@ tracedAtomically origin act = do
return a
renderJobResources :: String -> JobResources -> String
-renderJobResources origin (Jobs own free pending) = showSDocUnsafe $ renderJSON $
+renderJobResources origin (Jobs own free pending _heldToks) = showSDocUnsafe $ renderJSON $
JSObject [ ("name", JSString origin)
, ("owned", JSInt own)
, ("free", JSInt free)
@@ -262,61 +280,68 @@ renderJobResources origin (Jobs own free pending) = showSDocUnsafe $ renderJSON
-- | Spawn a new thread that waits on the semaphore in order to acquire
-- an additional token.
+--
+-- The child is forked masked so the only async-exception delivery point
+-- is the interruptible 'waitOnSemaphore'; the STM commit afterwards then
+-- always runs to completion, so 'threadFinished' is always filled.
+--
+-- The (tid, threadFinished) pair is also published to 'activeChild' so
+-- shutdown can drain the child even after the in-loop 'JobserverState'
+-- is gone.
acquireThread :: Jobserver -> IO JobserverAction
-acquireThread (Jobserver { jSemaphore = sem, jobs = jobs_tvar }) = do
+acquireThread (Jobserver { jSemaphore = sem, jobs = jobs_tvar, activeChild = active_tvar }) = do
threadFinished_tmvar <- newEmptyTMVarIO
- let
- wait_result_action :: Either MC.SomeException Bool -> IO ()
- wait_result_action wait_res =
+ tid <- MC.mask_ $ do
+ tid <- forkIO $ do
+ wait_res <- MC.try @_ @MC.SomeException $ waitOnSemaphore sem
tracedAtomically_ "acquire_thread" do
(r, jb) <- case wait_res of
Left (e :: MC.SomeException) -> do
return $ (Just e, Nothing)
- Right success -> do
- if success
- then do
- modifyJobResources jobs_tvar \ jobs ->
- return (Nothing, addToken jobs)
- else
- return (Nothing, Nothing)
+ Right tok -> do
+ modifyJobResources jobs_tvar \ jobs ->
+ return (Nothing, addToken tok jobs)
putTMVar threadFinished_tmvar r
return jb
- wait_id <- forkWaitOnSemaphoreInterruptible sem wait_result_action
- labelThread (waitingThreadId wait_id) "acquire_thread"
- return $ Acquiring { activeWaitId = wait_id
+ atomically $ writeTVar active_tvar (Just (tid, threadFinished_tmvar))
+ return tid
+ labelThread tid "acquire_thread"
+ return $ Acquiring { activeThreadId = tid
, threadFinished = threadFinished_tmvar }
-- | Spawn a thread to release ownership of one resource from the semaphore,
-- provided we have spare resources and no pending jobs.
releaseThread :: Jobserver -> IO JobserverAction
-releaseThread (Jobserver { jSemaphore = sem, jobs = jobs_tvar }) = do
+releaseThread (Jobserver { jobs = jobs_tvar }) = do
threadFinished_tmvar <- newEmptyTMVarIO
MC.mask_ do
-- Pre-release the resource so that another thread doesn't take control of it
-- just as we release the lock on the semaphore.
- still_ok_to_release
+ mb_tok
<- tracedAtomically "pre_release" $
modifyJobResources jobs_tvar \ jobs ->
if guardRelease jobs
- -- TODO: should this also debounce?
- then return (True , removeOwnedToken $ removeFreeToken jobs)
- else return (False, jobs)
- if not still_ok_to_release
- then return Idle
- else do
- tid <- forkIO $ do
- x <- MC.try $ releaseSemaphore sem 1
- tracedAtomically_ "post-release" $ do
- (r, jobs) <- case x of
- Left (e :: MC.SomeException) -> do
- modifyJobResources jobs_tvar \ jobs ->
- return (Just e, addToken jobs)
- Right _ -> do
- return (Nothing, Nothing)
- putTMVar threadFinished_tmvar r
- return jobs
- labelThread tid "release_thread"
- return Idle
+ then let (tok, jobs') = removeOwnedToken $ removeFreeToken jobs
+ in return (Just tok, jobs')
+ else return (Nothing, jobs)
+ case mb_tok of
+ Nothing ->
+ -- Not OK to release: there are other pending jobs that could make use of the token.
+ return Idle
+ Just tok -> do
+ tid <- forkIO $ do
+ x <- MC.try @MC.SomeException $ releaseSemaphoreToken tok
+ tracedAtomically_ "post-release" $ do
+ (r, jobs) <- case x of
+ Left (e :: MC.SomeException) -> do
+ modifyJobResources jobs_tvar \ jobs ->
+ return (Just e, addToken tok jobs)
+ Right _ -> do
+ return (Nothing, Nothing)
+ putTMVar threadFinished_tmvar r
+ return jobs
+ labelThread tid "release_thread"
+ return Idle
-- | When there are pending jobs but no free tokens,
-- spawn a thread to acquire a new token from the semaphore.
@@ -363,13 +388,14 @@ tryRelease _ _ = retry
-- | Wait for an active thread to finish. Once it finishes:
--
-- - set the 'JobserverAction' to 'Idle',
+-- - clear the 'activeChild' handle,
-- - update the number of capabilities to reflect the number
-- of owned tokens from the semaphore.
tryNoticeIdle :: JobserverOptions
- -> TVar JobResources
+ -> Jobserver
-> JobserverState
-> STM (IO JobserverState)
-tryNoticeIdle opts jobs_tvar jobserver_state
+tryNoticeIdle opts (Jobserver { jobs = jobs_tvar, activeChild = active_tvar }) jobserver_state
| Just threadFinished_tmvar <- activeThread_maybe $ jobserverAction jobserver_state
= sync_num_caps (canChangeNumCaps jobserver_state) threadFinished_tmvar
| otherwise
@@ -381,6 +407,7 @@ tryNoticeIdle opts jobs_tvar jobserver_state
sync_num_caps can_change_numcaps_tvar threadFinished_tmvar = do
mb_ex <- takeTMVar threadFinished_tmvar
for_ mb_ex MC.throwM
+ writeTVar active_tvar Nothing
Jobs { tokensOwned } <- readTVar jobs_tvar
can_change_numcaps <- readTVar can_change_numcaps_tvar
guard can_change_numcaps
@@ -404,11 +431,11 @@ tryStopThread :: TVar JobResources
-> STM (IO JobserverState)
tryStopThread jobs_tvar jsj = do
case jobserverAction jsj of
- Acquiring { activeWaitId = wait_id } -> do
+ Acquiring { activeThreadId = tid } -> do
jobs <- readTVar jobs_tvar
guard $ null (jobsWaiting jobs)
return do
- interruptWaitOnSemaphore wait_id
+ killThread tid
return $ jsj { jobserverAction = Idle }
_ -> retry
@@ -430,30 +457,38 @@ jobserverLoop opts sjs@(Jobserver { jobs = jobs_tvar })
action <- atomically $ asum $ (\x -> x s) <$>
[ tryRelease sjs
, tryAcquire opts sjs
- , tryNoticeIdle opts jobs_tvar
+ , tryNoticeIdle opts sjs
, tryStopThread jobs_tvar
]
s <- action
loop s
--- | Create a new jobserver using the given semaphore handle.
-makeJobserver :: SemaphoreName -> IO (AbstractSem, IO ())
-makeJobserver sem_name = do
- semaphore <- openSemaphore sem_name
+-- | Create a new jobserver using the given semaphore identifier.
+makeJobserver :: SemaphoreIdentifier -> IO (AbstractSem, IO ())
+makeJobserver sem_ident = do
+ semaphore <- openSemaphore sem_ident >>= either MC.throwM pure
let
init_jobs =
Jobs { tokensOwned = 1
, tokensFree = 1
, jobsWaiting = NilOL
+ , heldTokens = []
}
jobs_tvar <- newTVarIO init_jobs
+ active_tvar <- newTVarIO Nothing
let
opts = defaultJobserverOptions -- TODO: allow this to be configured
- sjs = Jobserver { jSemaphore = semaphore
- , jobs = jobs_tvar }
+ sjs = Jobserver { jSemaphore = semaphore
+ , jobs = jobs_tvar
+ , activeChild = active_tvar }
loop_finished_mvar <- newEmptyMVar
loop_tid <- forkIOWithUnmask \ unmask -> do
r <- try $ unmask $ jobserverLoop opts sjs
+ -- Always-run exit handler: any child the loop spawned is still alive
+ -- in its own thread, so drain it before touching jobs_tvar. No one
+ -- else can mutate the resources once both are dead.
+ drainActiveChild sjs
+ releaseAllHeld sjs
putMVar loop_finished_mvar $
case r of
Left e
@@ -467,8 +502,8 @@ makeJobserver sem_name = do
acquireSem = acquireJob jobs_tvar
releaseSem = releaseJob jobs_tvar
cleanupSem = do
- -- this is interruptible
- cleanupJobserver sjs
+ -- Trigger the loop's exit handler; it drains the active child and
+ -- releases all held tokens, then signals loop_finished_mvar.
killThread loop_tid
mb_ex <- takeMVar loop_finished_mvar
for_ mb_ex MC.throwM
@@ -477,12 +512,12 @@ makeJobserver sem_name = do
-- | Implement an abstract semaphore using a semaphore 'Jobserver'
-- which queries the system semaphore of the given name for resources.
-runJSemAbstractSem :: SemaphoreName -- ^ the system semaphore to use
+runJSemAbstractSem :: SemaphoreIdentifier -- ^ the semaphore identifier (from @-jsem@)
-> (AbstractSem -> IO a) -- ^ the operation to run
-- which requires a semaphore
-> IO a
-runJSemAbstractSem sem action = MC.mask \ unmask -> do
- (abs, cleanup) <- makeJobserver sem
+runJSemAbstractSem sem_ident action = MC.mask \ unmask -> do
+ (abs, cleanup) <- makeJobserver sem_ident
r <- try $ unmask $ action abs
case r of
Left (e1 :: MC.SomeException) -> do
@@ -517,8 +552,13 @@ increases the number of `free` jobs. If there are more pending jobs when the fre
is increased, the token is immediately reused (see `modifyJobResources`).
The `jobServerLoop` interacts with the system semaphore: when there are pending
-jobs, `acquireThread` blocks, waiting for a token from the semaphore. Once a
-token is obtained, it increases the owned count.
+jobs, `acquireThread` forks a child that calls the interruptible
+`waitOnSemaphore`. The child is forked in the masked state, so the only place
+an async exception can be delivered is the wait itself; once the wait returns,
+the child's STM commit always completes, recording either the new token in
+`heldTokens` or the failure exception in `threadFinished`. The (tid, tmvar)
+pair is also published in `activeChild` so the loop's exit handler can drain
+the child on shutdown even after the in-loop `JobserverState` is gone.
When GHC has free tokens (tokens from the semaphore that it is not using),
no pending jobs, and the debounce has expired, then `releaseThread` will
@@ -531,6 +571,12 @@ This second token is no longer needed, so we should cancel the wait
(as it would not be used to do any work, and not be returned until the debounce).
We only need to kill `acquireJob`, because `releaseJob` never blocks.
+Shutdown starts with `killThread loop_tid`. The loop's exit handler then
+runs `drainActiveChild` followed by `releaseAllHeld`; only then does the
+loop signal `loop_finished_mvar`. This sequence makes the heldTokens
+snapshot consistent because no other thread can mutate it once the loop and
+its child are both dead.
+
Note [Eventlog Messages for jsem]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
It can be tricky to verify that the work is shared adequately across different
=====================================
compiler/GHC/Driver/Session.hs
=====================================
@@ -2445,6 +2445,7 @@ wWarningFlagsDeps = [minBound..maxBound] >>= \x -> case x of
Opt_WarnUnusableUnpackPragmas -> warnSpec x
Opt_WarnPatternNamespaceSpecifier -> warnSpec x
Opt_WarnUnrecognisedModifiers -> warnSpec x
+ Opt_WarnSemaphoreOpenFailure -> warnSpec x
warningGroupsDeps :: [(Deprecation, FlagSpec WarningGroup)]
warningGroupsDeps = map mk warningGroups
=====================================
compiler/GHC/Types/Error/Codes.hs
=====================================
@@ -403,6 +403,7 @@ type family GhcDiagnosticCode c = n | n -> c where
GhcDiagnosticCode "DriverInstantiationNodeInDependencyGeneration" = 74284
GhcDiagnosticCode "DriverNoConfiguredLLVMToolchain" = 66599
GhcDiagnosticCode "DriverMissingLinkableForModule" = 47338
+ GhcDiagnosticCode "DriverSemaphoreOpenFailure" = 19877
-- Constraint solver diagnostic codes
GhcDiagnosticCode "BadTelescope" = 97739
=====================================
docs/users_guide/using-warnings.rst
=====================================
@@ -2721,6 +2721,23 @@ of ``-W(no-)*``.
f :: a %True -> a
g :: a %(k :: Int) -> a
+.. ghc-flag:: -Wsemaphore-open-failure
+ :shortdesc: warn when GHC cannot open the ``-jsem`` semaphore.
+ :type: dynamic
+ :reverse: -Wno-semaphore-open-failure
+ :category:
+
+ :since: 10.0.1
+
+ Warn when GHC is invoked with :ghc-flag:`-jsem` but the semaphore
+ cannot be opened (e.g. the socket does not exist, the protocol
+ version is incompatible, or a system error occurred). When this
+ occurs, GHC ignores ``-jsem`` and compiles modules sequentially.
+
+ A common cause is ``cabal-install`` and GHC being built against
+ different versions of the ``semaphore-compat`` library; upgrading
+ both to versions that use the same protocol resolves the mismatch.
+
----
If you're feeling really paranoid, the :ghc-flag:`-dcore-lint` option is a good choice.
=====================================
docs/users_guide/using.rst
=====================================
@@ -797,7 +797,12 @@ There are two kinds of participants in the GHC Jobserver protocol:
Perform compilation in parallel when possible, coordinating with other
processes through the semaphore ⟨sem⟩ (specified as a string).
- Error if the semaphore doesn't exist.
+
+ If the semaphore cannot be opened (e.g. the socket does not exist
+ or its protocol version is incompatible with this GHC), GHC emits
+ a :ghc-flag:`-Wsemaphore-open-failure` warning and compiles
+ sequentially, using only the implicit token inherited from the
+ parent process.
Use of ``-jsem`` will override use of :ghc-flag:`-j[⟨n⟩]`,
and vice-versa.
=====================================
hadrian/cabal.project
=====================================
@@ -1,6 +1,7 @@
packages: ./
../utils/ghc-toolchain/
../libraries/ghc-platform/
+ ../libraries/semaphore-compat/
-- This essentially freezes the build plan for hadrian
-- It would be wise to keep this up to date with the state set in .gitlab/ci.sh.
=====================================
hadrian/hadrian.cabal
=====================================
@@ -172,6 +172,7 @@ executable hadrian
, base16-bytestring >= 0.1.1 && < 1.1.0.0
, ghc-platform
, ghc-toolchain
+ , semaphore-compat
ghc-options: -Wall
-Wincomplete-record-updates
-Wredundant-constraints
=====================================
hadrian/src/Flavour.hs
=====================================
@@ -149,10 +149,6 @@ werror =
-- unix has many unused imports
, package unix
? mconcat [arg "-Wwarn=unused-imports", arg "-Wwarn=unused-top-binds"]
- -- semaphore-compat relies on sem_getvalue as provided by unix, which is
- -- not implemented on Darwin and therefore throws a deprecation warning
- , package semaphoreCompat
- ? mconcat [arg "-Wwarn=deprecations"]
]
, builder Ghc
? package rts
=====================================
hadrian/src/Rules/Generate.hs
=====================================
@@ -25,6 +25,7 @@ import Utilities
import GHC.Toolchain as Toolchain hiding (HsCpp(HsCpp))
import GHC.Platform.ArchOS
import Settings.Program (ghcWithInterpreter)
+import System.Semaphore (semaphoreVersion, getSemaphoreProtocolVersion)
-- | Track this file to rebuild generated files whenever it changes.
trackGenerateHs :: Expr ()
@@ -488,6 +489,7 @@ generateSettings settingsFile = do
, ("RTS ways", escapeArgs . map show . Set.toList <$> getRtsWays)
, ("Relative Global Package DB", pure rel_pkg_db)
, ("base unit-id", pure base_unit_id)
+ , ("Semaphore version", pure (show (getSemaphoreProtocolVersion semaphoreVersion)))
]
let showTuple (k, v) = "(" ++ show k ++ ", " ++ show v ++ ")"
pure $ case settings of
=====================================
hadrian/src/Settings/Packages.hs
=====================================
@@ -231,6 +231,10 @@ packageArgs = do
, package hpcBin
? builder (Cabal Flags) ? arg "-build-tool-depends"
+ ------------------------------ semaphore-compat ----------------------------
+ , package semaphoreCompat
+ ? builder (Cabal Flags) ? arg "-build-testing"
+
]
ghcInternalArgs :: Args
=====================================
hadrian/stack.yaml
=====================================
@@ -16,6 +16,7 @@ packages:
- '.'
- '../utils/ghc-toolchain'
- '../libraries/ghc-platform'
+- '../libraries/semaphore-compat'
nix:
enable: false
=====================================
libraries/semaphore-compat
=====================================
@@ -1 +1 @@
-Subproject commit 7929702401d49bc64d809c501ed5fe80aebc3cc1
+Subproject commit 20f7b117787fa1c11d62bc33d01cdabf9a7071bb
=====================================
testsuite/tests/diagnostic-codes/codes.stdout
=====================================
@@ -21,6 +21,7 @@
[GHC-29747] is untested (constructor = DriverMissingSafeHaskellMode)
[GHC-74284] is untested (constructor = DriverInstantiationNodeInDependencyGeneration)
[GHC-66599] is untested (constructor = DriverNoConfiguredLLVMToolchain)
+[GHC-19877] is untested (constructor = DriverSemaphoreOpenFailure)
[GHC-81325] is untested (constructor = ExpectingMoreArguments)
[GHC-78125] is untested (constructor = AmbiguityPreventsSolvingCt)
[GHC-84170] is untested (constructor = TcRnModMissingRealSrcSpan)
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/d6b0d41e1d764e614f1679837d38e3b…
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/d6b0d41e1d764e614f1679837d38e3b…
You're receiving this email because of your account on gitlab.haskell.org.
1
0
[Git][ghc/ghc][wip/semaphore-v2] Update to semaphore-compat 2.0.0 using v2 of the protocol
by Zubin (＠wz1000) 19 May '26
by Zubin (＠wz1000) 19 May '26
19 May '26
Zubin pushed to branch wip/semaphore-v2 at Glasgow Haskell Compiler / GHC
Commits:
b01201dd by Zubin Duggal at 2026-05-19T14:28:35+05:30
Update to semaphore-compat 2.0.0 using v2 of the protocol
On Linux and other POSIX platforms, GHC's -jsem jobserver client now
speaks v2 of the semaphore-compat protocol, which uses Unix domain
sockets in place of POSIX named semaphores. This avoids the libc-ABI
issues that affected the old implementation. Windows is unaffected
and continues to use the v1 protocol (Win32 named semaphores); its
reported protocol version remains v1.
When GHC receives a -jsem name whose protocol version it does not
support, it emits a -Wsemaphore-version-mismatch warning and falls
back to -j<N> rather than crashing. ghc --info exposes the supported
version in a new "Semaphore version" entry so cabal-install can detect
a mismatch before invoking GHC.
Users on a cabal-install that predates the v2 update will continue to
build successfully on Linux/POSIX, but will lose the cross-process
-jsem coordination and fall back to -j<N> per GHC invocation. Users
must upgrade to a cabal-install that supports protocol v2 to recover
full parallelism.
Also fix a leak in cleanupSem (#27253): cleanupSem used to snapshot
heldTokens and release them before killing the loop, while the loop's
in-flight acquire/release children could still be mutating it.
Cleanup now runs inside the loop's own exit handler, after draining
the active child via a new activeChild TVar, so the snapshot has no
concurrent mutator.
See also:
- GHC proposal amendment: https://github.com/ghc-proposals/ghc-proposals/pull/673
- cabal-install patch: https://github.com/haskell/cabal/pull/11628
- semaphore-compat MR: https://gitlab.haskell.org/ghc/semaphore-compat/-/merge_requests/8
Bump semaphore-compat submodule to 2.0.0
Fixes #25087 and #27253
- - - - -
19 changed files:
- + changelog.d/jobserver-leak-fix
- + changelog.d/semaphore-v2
- compiler/GHC/Driver/Errors/Ppr.hs
- compiler/GHC/Driver/Errors/Types.hs
- compiler/GHC/Driver/Flags.hs
- compiler/GHC/Driver/MakeAction.hs
- compiler/GHC/Driver/MakeSem.hs
- compiler/GHC/Driver/Session.hs
- compiler/GHC/Types/Error/Codes.hs
- docs/users_guide/using-warnings.rst
- docs/users_guide/using.rst
- hadrian/cabal.project
- hadrian/hadrian.cabal
- hadrian/src/Flavour.hs
- hadrian/src/Rules/Generate.hs
- hadrian/src/Settings/Packages.hs
- hadrian/stack.yaml
- libraries/semaphore-compat
- testsuite/tests/diagnostic-codes/codes.stdout
Changes:
=====================================
changelog.d/jobserver-leak-fix
=====================================
@@ -0,0 +1,8 @@
+section: compiler
+issues: #27253
+mrs: !15729
+synopsis:
+ Fix a token leak in the ``-jsem`` jobserver shutdown path
+description:
+ A build interrupted by Ctrl-C while a ``-jsem`` token transfer was in
+ flight could leak that token.
=====================================
changelog.d/semaphore-v2
=====================================
@@ -0,0 +1,30 @@
+section: compiler
+issues: #25087
+mrs: !15729
+synopsis:
+ Update to semaphore-compat 2.0.0 (``-jsem`` protocol v2)
+description:
+ On Linux and other POSIX platforms, GHC's ``-jsem`` jobserver client
+ now speaks v2 of the semaphore-compat protocol, which uses Unix
+ domain sockets in place of POSIX named semaphores. This avoids the
+ libc-ABI issues that affected the old implementation. Windows is
+ unaffected and continues to use the v1 protocol (Win32 named
+ semaphores); its reported protocol version remains v1.
+
+ When GHC receives a ``-jsem`` name whose protocol version it does not
+ support, it now emits a ``-Wsemaphore-version-mismatch`` warning and
+ falls back to ``-j1`` rather than crashing. ``ghc --info`` exposes the
+ supported version in a new ``"Semaphore version"`` entry so
+ cabal-install can detect a mismatch before invoking GHC.
+
+ Users on a ``cabal-install`` that predates the v2 update will continue
+ to build successfully on Linux/POSIX, but will lose the cross-process
+ ``-jsem`` coordination and fall back to ``-j1`` per GHC invocation.
+ To recover full parallelism, upgrade to a ``cabal-install`` that
+ supports protocol v2.
+
+ See also:
+
+ - the `GHC proposal amendment <https://github.com/ghc-proposals/ghc-proposals/pull/673>`_
+ - the `cabal-install patch <https://github.com/haskell/cabal/pull/11628>`_
+ - the `semaphore-compat library MR <https://gitlab.haskell.org/ghc/semaphore-compat/-/merge_requests/8>`_
=====================================
compiler/GHC/Driver/Errors/Ppr.hs
=====================================
@@ -24,6 +24,8 @@ import GHC.Types.Hint
import GHC.Types.SrcLoc
import Data.Version
+import System.Semaphore
+ ( SemaphoreError(..), getSemaphoreProtocolVersion )
import Language.Haskell.Syntax.Decls (RuleDecl(..))
import GHC.Tc.Errors.Types (TcRnMessage)
import GHC.HsToCore.Errors.Types (DsMessage)
@@ -90,6 +92,20 @@ instance Diagnostic GhcMessage where
instance HasDefaultDiagnosticOpts DriverMessageOpts where
defaultOpts = DriverMessageOpts (defaultDiagnosticOpts @PsMessage) (defaultDiagnosticOpts @IfaceMessage)
+pprSemaphoreError :: SemaphoreError -> SDoc
+pprSemaphoreError = \case
+ SemaphoreAlreadyExists nm ->
+ text "a semaphore named" <+> quotes (text nm) <+> text "already exists"
+ SemaphoreDoesNotExist nm ->
+ text "no semaphore named" <+> quotes (text nm)
+ SemaphoreIncompatibleVersion got want ->
+ text "protocol version mismatch (got v"
+ <> int (getSemaphoreProtocolVersion got)
+ <> text ", supported v"
+ <> int (getSemaphoreProtocolVersion want) <> text ")"
+ SemaphoreOtherError ioe ->
+ text (show ioe)
+
instance Diagnostic DriverMessage where
type DiagnosticOpts DriverMessage = DriverMessageOpts
diagnosticMessage opts = \case
@@ -282,6 +298,10 @@ instance Diagnostic DriverMessage where
-> mkSimpleDecorated $
vcat [ text "The following modules are missing a linkable which is needed for creating a library:"
, nest 2 $ hcat (map ppr mods) ]
+ DriverSemaphoreOpenFailure err
+ -> mkSimpleDecorated $
+ text "Failed to open -jsem semaphore:" <+> pprSemaphoreError err <>
+ text "; ignoring -jsem and compiling sequentially."
diagnosticReason = \case
DriverUnknownMessage m
@@ -355,6 +375,8 @@ instance Diagnostic DriverMessage where
-> WarningWithoutFlag
DriverMissingLinkableForModule {}
-> ErrorWithoutFlag
+ DriverSemaphoreOpenFailure {}
+ -> WarningWithFlag Opt_WarnSemaphoreOpenFailure
diagnosticHints = \case
DriverUnknownMessage m
@@ -430,5 +452,14 @@ instance Diagnostic DriverMessage where
-> noHints
DriverMissingLinkableForModule {}
-> noHints
+ DriverSemaphoreOpenFailure (SemaphoreIncompatibleVersion received supported)
+ | received < supported
+ -> [UnknownHint (text "The parent process (e.g. cabal-install) uses an older semaphore protocol."
+ $$ text "Upgrading cabal-install may resolve this." :: SDoc)]
+ | otherwise
+ -> [UnknownHint (text "The parent process (e.g. cabal-install) uses a newer semaphore protocol."
+ $$ text "Upgrading GHC may resolve this." :: SDoc)]
+ DriverSemaphoreOpenFailure {}
+ -> noHints
diagnosticCode = constructorCode @GHC
=====================================
compiler/GHC/Driver/Errors/Types.hs
=====================================
@@ -37,6 +37,7 @@ import qualified GHC.LanguageExtensions as LangExt
import GHC.Generics ( Generic )
+import System.Semaphore ( SemaphoreError )
import GHC.Tc.Errors.Types
import GHC.Iface.Errors.Types
@@ -419,6 +420,13 @@ data DriverMessage where
DriverMissingLinkableForModule :: ![Module] -> DriverMessage
+ {-| DriverSemaphoreOpenFailure is a warning that occurs when GHC fails to
+ open the semaphore specified by @-jsem@, e.g. the socket does not
+ exist, the protocol version is incompatible, or a system error
+ occurred. GHC ignores @-jsem@ and compiles sequentially.
+ -}
+ DriverSemaphoreOpenFailure :: !SemaphoreError -> DriverMessage
+
deriving instance Generic DriverMessage
data DriverMessageOpts =
=====================================
compiler/GHC/Driver/Flags.hs
=====================================
@@ -1115,6 +1115,7 @@ data WarningFlag =
| Opt_WarnUnusableUnpackPragmas -- ^ @since 9.14
| Opt_WarnPatternNamespaceSpecifier -- ^ @since 9.14
| Opt_WarnUnrecognisedModifiers -- ^ @since 10.0
+ | Opt_WarnSemaphoreOpenFailure -- Since 10.0.1
deriving (Eq, Ord, Show, Enum, Bounded)
-- | Return the names of a WarningFlag
@@ -1237,6 +1238,7 @@ warnFlagNames wflag = case wflag of
Opt_WarnUnusableUnpackPragmas -> "unusable-unpack-pragmas" :| []
Opt_WarnPatternNamespaceSpecifier -> "pattern-namespace-specifier" :| []
Opt_WarnUnrecognisedModifiers -> "unrecognised-modifiers" :| []
+ Opt_WarnSemaphoreOpenFailure -> "semaphore-open-failure" :| []
-- -----------------------------------------------------------------------------
-- Standard sets of warning options
@@ -1383,7 +1385,8 @@ standardWarnings -- see Note [Documenting warning flags]
Opt_WarnDeprecatedPragmas,
Opt_WarnRuleLhsEqualities,
Opt_WarnUnusableUnpackPragmas,
- Opt_WarnUnrecognisedModifiers
+ Opt_WarnUnrecognisedModifiers,
+ Opt_WarnSemaphoreOpenFailure
]
-- | Things you get with @-W@.
=====================================
compiler/GHC/Driver/MakeAction.hs
=====================================
@@ -28,6 +28,14 @@ import GHC.Driver.Errors.Types
import GHC.Driver.Messager
import GHC.Driver.MakeSem
+import System.Semaphore
+ ( SemaphoreError, SemaphoreIdentifier )
+
+import GHC.Driver.Config.Diagnostic ( initDiagOpts, initPrintConfig )
+import GHC.Driver.Errors ( printOrThrowDiagnostics )
+import GHC.Types.Error ( singleMessage )
+import GHC.Types.SrcLoc ( noSrcSpan )
+import GHC.Utils.Error ( mkPlainMsgEnvelope )
import GHC.Utils.Logger
import GHC.Utils.TmpFs
@@ -49,7 +57,7 @@ mkWorkerLimit :: DynFlags -> IO WorkerLimit
mkWorkerLimit dflags =
case parMakeCount dflags of
Nothing -> pure $ num_procs 1
- Just (ParMakeSemaphore h) -> pure (JSemLimit (SemaphoreName h))
+ Just (ParMakeSemaphore h) -> pure (JSemLimit h)
Just ParMakeNumProcessors -> num_procs <$> getNumProcessors
Just (ParMakeThisMany n) -> pure $ num_procs n
where
@@ -65,8 +73,8 @@ isWorkerLimitSequential (JSemLimit {}) = False
data WorkerLimit
= NumProcessorsLimit Int
| JSemLimit
- SemaphoreName
- -- ^ Semaphore name to use
+ SemaphoreIdentifier
+ -- ^ Semaphore identifier from @-jsem@
deriving Eq
-- | Environment used when compiling a module
@@ -122,17 +130,24 @@ runNjobsAbstractSem n_jobs action = do
resetNumCapabilities = set_num_caps n_capabilities
MC.bracket_ updNumCapabilities resetNumCapabilities $ action asem
-runWorkerLimit :: WorkerLimit -> (AbstractSem -> IO a) -> IO a
+runWorkerLimit :: Logger -> DynFlags -> WorkerLimit -> (AbstractSem -> IO a) -> IO a
#if defined(wasm32_HOST_ARCH)
-runWorkerLimit _ action = do
+runWorkerLimit _logger _dflags _ action = do
lock <- newMVar ()
action $ AbstractSem (takeMVar lock) (putMVar lock ())
#else
-runWorkerLimit worker_limit action = case worker_limit of
+runWorkerLimit logger dflags worker_limit action = case worker_limit of
NumProcessorsLimit n_jobs ->
runNjobsAbstractSem n_jobs action
- JSemLimit sem ->
- runJSemAbstractSem sem action
+ JSemLimit sem_ident -> do
+ result <- MC.try @SemaphoreError $ runJSemAbstractSem sem_ident action
+ case result of
+ Right a -> return a
+ Left err -> do
+ let diag = DriverSemaphoreOpenFailure err
+ msg = singleMessage $ mkPlainMsgEnvelope (initDiagOpts dflags) noSrcSpan diag
+ printOrThrowDiagnostics logger (initPrintConfig dflags) (initDiagOpts dflags) (GhcDriverMessage <$> msg)
+ runNjobsAbstractSem 1 action
#endif
-- | Build and run a pipeline
@@ -159,7 +174,7 @@ runParPipelines worker_limit plugin_hsc_env diag_wrapper mHscMessager all_pipeli
thread_safe_logger <- liftIO $ makeThreadSafe (hsc_logger plugin_hsc_env)
let thread_safe_hsc_env = plugin_hsc_env { hsc_logger = thread_safe_logger }
- runWorkerLimit worker_limit $ \abstract_sem -> do
+ runWorkerLimit (hsc_logger plugin_hsc_env) (hsc_dflags plugin_hsc_env) worker_limit $ \abstract_sem -> do
let env = MakeEnv { hsc_env = thread_safe_hsc_env
, withLogger = withParLog log_queue_queue_var
, compile_sem = abstract_sem
=====================================
compiler/GHC/Driver/MakeSem.hs
=====================================
@@ -9,9 +9,6 @@ module GHC.Driver.MakeSem
-- by a system semaphore (Posix/Windows)
runJSemAbstractSem
- -- * System semaphores
- , Semaphore, SemaphoreName(..)
-
-- * Abstract semaphores
, AbstractSem(..)
, withAbstractSem
@@ -46,11 +43,14 @@ import Debug.Trace
-- available from the semaphore.
data Jobserver
= Jobserver
- { jSemaphore :: !Semaphore
+ { jSemaphore :: !ClientSemaphore
-- ^ The semaphore which controls available resources
, jobs :: !(TVar JobResources)
-- ^ The currently pending jobs, and the resources
-- obtained from the semaphore
+ , activeChild :: !(TVar (Maybe (ThreadId, TMVar (Maybe MC.SomeException))))
+ -- ^ Handle on the current acquire thread (if any). The loop's exit
+ -- handler reads this to drain a still-running child on shutdown.
}
data JobserverOptions
@@ -81,6 +81,9 @@ data JobResources
, jobsWaiting :: !(OrdList (TMVar ()))
-- ^ Pending jobs waiting on a token, the job will be blocked on the TMVar so putting into
-- the TMVar will allow the job to continue.
+ , heldTokens :: [SemaphoreToken]
+ -- ^ Actual semaphore tokens (for release/cleanup).
+ -- Length should equal tokensOwned - 1 (the implicit token has no SemaphoreToken).
}
instance Outputable JobResources where
@@ -93,9 +96,9 @@ instance Outputable JobResources where
] )
-- | Add one new token.
-addToken :: JobResources -> JobResources
-addToken jobs@( Jobs { tokensOwned = owned, tokensFree = free })
- = jobs { tokensOwned = owned + 1, tokensFree = free + 1 }
+addToken :: SemaphoreToken -> JobResources -> JobResources
+addToken tok jobs@( Jobs { tokensOwned = owned, tokensFree = free, heldTokens = toks })
+ = jobs { tokensOwned = owned + 1, tokensFree = free + 1, heldTokens = tok : toks }
-- | Free one token.
addFreeToken :: JobResources -> JobResources
@@ -111,12 +114,14 @@ removeFreeToken jobs@( Jobs { tokensFree = free })
(text "removeFreeToken:" <+> ppr free)
$ jobs { tokensFree = free - 1 }
--- | Return one owned token.
-removeOwnedToken :: JobResources -> JobResources
-removeOwnedToken jobs@( Jobs { tokensOwned = owned })
+-- | Return one owned token, extracting the 'SemaphoreToken' for release.
+removeOwnedToken :: JobResources -> (SemaphoreToken, JobResources)
+removeOwnedToken jobs@( Jobs { tokensOwned = owned, heldTokens = toks })
= assertPpr (owned > 1)
(text "removeOwnedToken:" <+> ppr owned)
- $ jobs { tokensOwned = owned - 1 }
+ $ case toks of
+ (t:rest) -> (t, jobs { tokensOwned = owned - 1, heldTokens = rest })
+ [] -> panic "removeOwnedToken: no held tokens"
-- | Add one new job to the end of the list of pending jobs.
addJob :: TMVar () -> JobResources -> JobResources
@@ -143,7 +148,7 @@ data JobserverAction
= Idle
-- | A thread is waiting for a token on the semaphore.
| Acquiring
- { activeWaitId :: WaitId
+ { activeThreadId :: ThreadId
, threadFinished :: TMVar (Maybe MC.SomeException) }
-- | Retrieve the 'TMVar' that signals if the current thread has finished,
@@ -189,17 +194,30 @@ releaseJob jobs_tvar = do
return ((), addFreeToken jobs)
--- | Release all tokens owned from the semaphore (to clean up
--- the jobserver at the end).
-cleanupJobserver :: Jobserver -> IO ()
-cleanupJobserver (Jobserver { jSemaphore = sem
- , jobs = jobs_tvar })
- = do
- Jobs { tokensOwned = owned } <- readTVarIO jobs_tvar
- let toks_to_release = owned - 1
- -- Subtract off the implicit token: whoever spawned the ghc process
- -- in the first place is responsible for that token.
- releaseSemaphore sem toks_to_release
+-- | Kill the current acquire thread, if any, and wait for it to exit.
+--
+-- Called from the jobserver loop's exit handler, which runs masked.
+-- Relies on the invariant from 'acquireThread' that a forked child
+-- always fills its 'threadFinished' TMVar before it dies; this is what
+-- lets the 'takeTMVar' below terminate after the 'killThread'.
+drainActiveChild :: Jobserver -> IO ()
+drainActiveChild (Jobserver { activeChild = active_tvar }) = do
+ mb <- readTVarIO active_tvar
+ for_ mb $ \(tid, tmv) -> do
+ killThread tid
+ void $ atomically (takeTMVar tmv)
+ atomically $ writeTVar active_tvar Nothing
+
+-- | Release every token currently in 'heldTokens'.
+--
+-- Called from the jobserver loop's exit handler, which runs masked,
+-- after 'drainActiveChild': no other thread is mutating 'JobResources'
+-- at this point.
+releaseAllHeld :: Jobserver -> IO ()
+releaseAllHeld (Jobserver { jobs = jobs_tvar }) = do
+ Jobs { heldTokens = toks } <- readTVarIO jobs_tvar
+ forM_ toks $ \t ->
+ void $ MC.try @MC.SomeException (releaseSemaphoreToken t)
-- | Dispatch the available tokens acquired from the semaphore
-- to the pending jobs in the job server.
@@ -252,7 +270,7 @@ tracedAtomically origin act = do
return a
renderJobResources :: String -> JobResources -> String
-renderJobResources origin (Jobs own free pending) = showSDocUnsafe $ renderJSON $
+renderJobResources origin (Jobs own free pending _heldToks) = showSDocUnsafe $ renderJSON $
JSObject [ ("name", JSString origin)
, ("owned", JSInt own)
, ("free", JSInt free)
@@ -262,61 +280,68 @@ renderJobResources origin (Jobs own free pending) = showSDocUnsafe $ renderJSON
-- | Spawn a new thread that waits on the semaphore in order to acquire
-- an additional token.
+--
+-- The child is forked masked so the only async-exception delivery point
+-- is the interruptible 'waitOnSemaphore'; the STM commit afterwards then
+-- always runs to completion, so 'threadFinished' is always filled.
+--
+-- The (tid, threadFinished) pair is also published to 'activeChild' so
+-- shutdown can drain the child even after the in-loop 'JobserverState'
+-- is gone.
acquireThread :: Jobserver -> IO JobserverAction
-acquireThread (Jobserver { jSemaphore = sem, jobs = jobs_tvar }) = do
+acquireThread (Jobserver { jSemaphore = sem, jobs = jobs_tvar, activeChild = active_tvar }) = do
threadFinished_tmvar <- newEmptyTMVarIO
- let
- wait_result_action :: Either MC.SomeException Bool -> IO ()
- wait_result_action wait_res =
+ tid <- MC.mask_ $ do
+ tid <- forkIO $ do
+ wait_res <- MC.try @_ @MC.SomeException $ waitOnSemaphore sem
tracedAtomically_ "acquire_thread" do
(r, jb) <- case wait_res of
Left (e :: MC.SomeException) -> do
return $ (Just e, Nothing)
- Right success -> do
- if success
- then do
- modifyJobResources jobs_tvar \ jobs ->
- return (Nothing, addToken jobs)
- else
- return (Nothing, Nothing)
+ Right tok -> do
+ modifyJobResources jobs_tvar \ jobs ->
+ return (Nothing, addToken tok jobs)
putTMVar threadFinished_tmvar r
return jb
- wait_id <- forkWaitOnSemaphoreInterruptible sem wait_result_action
- labelThread (waitingThreadId wait_id) "acquire_thread"
- return $ Acquiring { activeWaitId = wait_id
+ atomically $ writeTVar active_tvar (Just (tid, threadFinished_tmvar))
+ return tid
+ labelThread tid "acquire_thread"
+ return $ Acquiring { activeThreadId = tid
, threadFinished = threadFinished_tmvar }
-- | Spawn a thread to release ownership of one resource from the semaphore,
-- provided we have spare resources and no pending jobs.
releaseThread :: Jobserver -> IO JobserverAction
-releaseThread (Jobserver { jSemaphore = sem, jobs = jobs_tvar }) = do
+releaseThread (Jobserver { jobs = jobs_tvar }) = do
threadFinished_tmvar <- newEmptyTMVarIO
MC.mask_ do
-- Pre-release the resource so that another thread doesn't take control of it
-- just as we release the lock on the semaphore.
- still_ok_to_release
+ mb_tok
<- tracedAtomically "pre_release" $
modifyJobResources jobs_tvar \ jobs ->
if guardRelease jobs
- -- TODO: should this also debounce?
- then return (True , removeOwnedToken $ removeFreeToken jobs)
- else return (False, jobs)
- if not still_ok_to_release
- then return Idle
- else do
- tid <- forkIO $ do
- x <- MC.try $ releaseSemaphore sem 1
- tracedAtomically_ "post-release" $ do
- (r, jobs) <- case x of
- Left (e :: MC.SomeException) -> do
- modifyJobResources jobs_tvar \ jobs ->
- return (Just e, addToken jobs)
- Right _ -> do
- return (Nothing, Nothing)
- putTMVar threadFinished_tmvar r
- return jobs
- labelThread tid "release_thread"
- return Idle
+ then let (tok, jobs') = removeOwnedToken $ removeFreeToken jobs
+ in return (Just tok, jobs')
+ else return (Nothing, jobs)
+ case mb_tok of
+ Nothing ->
+ -- Not OK to release: there are other pending jobs that could make use of the token.
+ return Idle
+ Just tok -> do
+ tid <- forkIO $ do
+ x <- MC.try @MC.SomeException $ releaseSemaphoreToken tok
+ tracedAtomically_ "post-release" $ do
+ (r, jobs) <- case x of
+ Left (e :: MC.SomeException) -> do
+ modifyJobResources jobs_tvar \ jobs ->
+ return (Just e, addToken tok jobs)
+ Right _ -> do
+ return (Nothing, Nothing)
+ putTMVar threadFinished_tmvar r
+ return jobs
+ labelThread tid "release_thread"
+ return Idle
-- | When there are pending jobs but no free tokens,
-- spawn a thread to acquire a new token from the semaphore.
@@ -363,13 +388,14 @@ tryRelease _ _ = retry
-- | Wait for an active thread to finish. Once it finishes:
--
-- - set the 'JobserverAction' to 'Idle',
+-- - clear the 'activeChild' handle,
-- - update the number of capabilities to reflect the number
-- of owned tokens from the semaphore.
tryNoticeIdle :: JobserverOptions
- -> TVar JobResources
+ -> Jobserver
-> JobserverState
-> STM (IO JobserverState)
-tryNoticeIdle opts jobs_tvar jobserver_state
+tryNoticeIdle opts (Jobserver { jobs = jobs_tvar, activeChild = active_tvar }) jobserver_state
| Just threadFinished_tmvar <- activeThread_maybe $ jobserverAction jobserver_state
= sync_num_caps (canChangeNumCaps jobserver_state) threadFinished_tmvar
| otherwise
@@ -381,6 +407,7 @@ tryNoticeIdle opts jobs_tvar jobserver_state
sync_num_caps can_change_numcaps_tvar threadFinished_tmvar = do
mb_ex <- takeTMVar threadFinished_tmvar
for_ mb_ex MC.throwM
+ writeTVar active_tvar Nothing
Jobs { tokensOwned } <- readTVar jobs_tvar
can_change_numcaps <- readTVar can_change_numcaps_tvar
guard can_change_numcaps
@@ -404,11 +431,11 @@ tryStopThread :: TVar JobResources
-> STM (IO JobserverState)
tryStopThread jobs_tvar jsj = do
case jobserverAction jsj of
- Acquiring { activeWaitId = wait_id } -> do
+ Acquiring { activeThreadId = tid } -> do
jobs <- readTVar jobs_tvar
guard $ null (jobsWaiting jobs)
return do
- interruptWaitOnSemaphore wait_id
+ killThread tid
return $ jsj { jobserverAction = Idle }
_ -> retry
@@ -430,30 +457,38 @@ jobserverLoop opts sjs@(Jobserver { jobs = jobs_tvar })
action <- atomically $ asum $ (\x -> x s) <$>
[ tryRelease sjs
, tryAcquire opts sjs
- , tryNoticeIdle opts jobs_tvar
+ , tryNoticeIdle opts sjs
, tryStopThread jobs_tvar
]
s <- action
loop s
--- | Create a new jobserver using the given semaphore handle.
-makeJobserver :: SemaphoreName -> IO (AbstractSem, IO ())
-makeJobserver sem_name = do
- semaphore <- openSemaphore sem_name
+-- | Create a new jobserver using the given semaphore identifier.
+makeJobserver :: SemaphoreIdentifier -> IO (AbstractSem, IO ())
+makeJobserver sem_ident = do
+ semaphore <- openSemaphore sem_ident >>= either MC.throwM pure
let
init_jobs =
Jobs { tokensOwned = 1
, tokensFree = 1
, jobsWaiting = NilOL
+ , heldTokens = []
}
jobs_tvar <- newTVarIO init_jobs
+ active_tvar <- newTVarIO Nothing
let
opts = defaultJobserverOptions -- TODO: allow this to be configured
- sjs = Jobserver { jSemaphore = semaphore
- , jobs = jobs_tvar }
+ sjs = Jobserver { jSemaphore = semaphore
+ , jobs = jobs_tvar
+ , activeChild = active_tvar }
loop_finished_mvar <- newEmptyMVar
loop_tid <- forkIOWithUnmask \ unmask -> do
r <- try $ unmask $ jobserverLoop opts sjs
+ -- Always-run exit handler: any child the loop spawned is still alive
+ -- in its own thread, so drain it before touching jobs_tvar. No one
+ -- else can mutate the resources once both are dead.
+ drainActiveChild sjs
+ releaseAllHeld sjs
putMVar loop_finished_mvar $
case r of
Left e
@@ -467,8 +502,8 @@ makeJobserver sem_name = do
acquireSem = acquireJob jobs_tvar
releaseSem = releaseJob jobs_tvar
cleanupSem = do
- -- this is interruptible
- cleanupJobserver sjs
+ -- Trigger the loop's exit handler; it drains the active child and
+ -- releases all held tokens, then signals loop_finished_mvar.
killThread loop_tid
mb_ex <- takeMVar loop_finished_mvar
for_ mb_ex MC.throwM
@@ -477,12 +512,12 @@ makeJobserver sem_name = do
-- | Implement an abstract semaphore using a semaphore 'Jobserver'
-- which queries the system semaphore of the given name for resources.
-runJSemAbstractSem :: SemaphoreName -- ^ the system semaphore to use
+runJSemAbstractSem :: SemaphoreIdentifier -- ^ the semaphore identifier (from @-jsem@)
-> (AbstractSem -> IO a) -- ^ the operation to run
-- which requires a semaphore
-> IO a
-runJSemAbstractSem sem action = MC.mask \ unmask -> do
- (abs, cleanup) <- makeJobserver sem
+runJSemAbstractSem sem_ident action = MC.mask \ unmask -> do
+ (abs, cleanup) <- makeJobserver sem_ident
r <- try $ unmask $ action abs
case r of
Left (e1 :: MC.SomeException) -> do
@@ -517,8 +552,13 @@ increases the number of `free` jobs. If there are more pending jobs when the fre
is increased, the token is immediately reused (see `modifyJobResources`).
The `jobServerLoop` interacts with the system semaphore: when there are pending
-jobs, `acquireThread` blocks, waiting for a token from the semaphore. Once a
-token is obtained, it increases the owned count.
+jobs, `acquireThread` forks a child that calls the interruptible
+`waitOnSemaphore`. The child is forked in the masked state, so the only place
+an async exception can be delivered is the wait itself; once the wait returns,
+the child's STM commit always completes, recording either the new token in
+`heldTokens` or the failure exception in `threadFinished`. The (tid, tmvar)
+pair is also published in `activeChild` so the loop's exit handler can drain
+the child on shutdown even after the in-loop `JobserverState` is gone.
When GHC has free tokens (tokens from the semaphore that it is not using),
no pending jobs, and the debounce has expired, then `releaseThread` will
@@ -531,6 +571,12 @@ This second token is no longer needed, so we should cancel the wait
(as it would not be used to do any work, and not be returned until the debounce).
We only need to kill `acquireJob`, because `releaseJob` never blocks.
+Shutdown starts with `killThread loop_tid`. The loop's exit handler then
+runs `drainActiveChild` followed by `releaseAllHeld`; only then does the
+loop signal `loop_finished_mvar`. This sequence makes the heldTokens
+snapshot consistent because no other thread can mutate it once the loop and
+its child are both dead.
+
Note [Eventlog Messages for jsem]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
It can be tricky to verify that the work is shared adequately across different
=====================================
compiler/GHC/Driver/Session.hs
=====================================
@@ -2445,6 +2445,7 @@ wWarningFlagsDeps = [minBound..maxBound] >>= \x -> case x of
Opt_WarnUnusableUnpackPragmas -> warnSpec x
Opt_WarnPatternNamespaceSpecifier -> warnSpec x
Opt_WarnUnrecognisedModifiers -> warnSpec x
+ Opt_WarnSemaphoreOpenFailure -> warnSpec x
warningGroupsDeps :: [(Deprecation, FlagSpec WarningGroup)]
warningGroupsDeps = map mk warningGroups
=====================================
compiler/GHC/Types/Error/Codes.hs
=====================================
@@ -403,6 +403,7 @@ type family GhcDiagnosticCode c = n | n -> c where
GhcDiagnosticCode "DriverInstantiationNodeInDependencyGeneration" = 74284
GhcDiagnosticCode "DriverNoConfiguredLLVMToolchain" = 66599
GhcDiagnosticCode "DriverMissingLinkableForModule" = 47338
+ GhcDiagnosticCode "DriverSemaphoreOpenFailure" = 19877
-- Constraint solver diagnostic codes
GhcDiagnosticCode "BadTelescope" = 97739
=====================================
docs/users_guide/using-warnings.rst
=====================================
@@ -2721,6 +2721,23 @@ of ``-W(no-)*``.
f :: a %True -> a
g :: a %(k :: Int) -> a
+.. ghc-flag:: -Wsemaphore-open-failure
+ :shortdesc: warn when GHC cannot open the ``-jsem`` semaphore.
+ :type: dynamic
+ :reverse: -Wno-semaphore-open-failure
+ :category:
+
+ :since: 10.0.1
+
+ Warn when GHC is invoked with :ghc-flag:`-jsem` but the semaphore
+ cannot be opened (e.g. the socket does not exist, the protocol
+ version is incompatible, or a system error occurred). When this
+ occurs, GHC ignores ``-jsem`` and compiles modules sequentially.
+
+ A common cause is ``cabal-install`` and GHC being built against
+ different versions of the ``semaphore-compat`` library; upgrading
+ both to versions that use the same protocol resolves the mismatch.
+
----
If you're feeling really paranoid, the :ghc-flag:`-dcore-lint` option is a good choice.
=====================================
docs/users_guide/using.rst
=====================================
@@ -797,7 +797,12 @@ There are two kinds of participants in the GHC Jobserver protocol:
Perform compilation in parallel when possible, coordinating with other
processes through the semaphore ⟨sem⟩ (specified as a string).
- Error if the semaphore doesn't exist.
+
+ If the semaphore cannot be opened (e.g. the socket does not exist
+ or its protocol version is incompatible with this GHC), GHC emits
+ a :ghc-flag:`-Wsemaphore-open-failure` warning and compiles
+ sequentially, using only the implicit token inherited from the
+ parent process.
Use of ``-jsem`` will override use of :ghc-flag:`-j[⟨n⟩]`,
and vice-versa.
=====================================
hadrian/cabal.project
=====================================
@@ -1,6 +1,7 @@
packages: ./
../utils/ghc-toolchain/
../libraries/ghc-platform/
+ ../libraries/semaphore-compat/
-- This essentially freezes the build plan for hadrian
-- It would be wise to keep this up to date with the state set in .gitlab/ci.sh.
=====================================
hadrian/hadrian.cabal
=====================================
@@ -172,6 +172,7 @@ executable hadrian
, base16-bytestring >= 0.1.1 && < 1.1.0.0
, ghc-platform
, ghc-toolchain
+ , semaphore-compat
ghc-options: -Wall
-Wincomplete-record-updates
-Wredundant-constraints
=====================================
hadrian/src/Flavour.hs
=====================================
@@ -149,10 +149,6 @@ werror =
-- unix has many unused imports
, package unix
? mconcat [arg "-Wwarn=unused-imports", arg "-Wwarn=unused-top-binds"]
- -- semaphore-compat relies on sem_getvalue as provided by unix, which is
- -- not implemented on Darwin and therefore throws a deprecation warning
- , package semaphoreCompat
- ? mconcat [arg "-Wwarn=deprecations"]
]
, builder Ghc
? package rts
=====================================
hadrian/src/Rules/Generate.hs
=====================================
@@ -25,6 +25,7 @@ import Utilities
import GHC.Toolchain as Toolchain hiding (HsCpp(HsCpp))
import GHC.Platform.ArchOS
import Settings.Program (ghcWithInterpreter)
+import System.Semaphore (semaphoreVersion, getSemaphoreProtocolVersion)
-- | Track this file to rebuild generated files whenever it changes.
trackGenerateHs :: Expr ()
@@ -488,6 +489,7 @@ generateSettings settingsFile = do
, ("RTS ways", escapeArgs . map show . Set.toList <$> getRtsWays)
, ("Relative Global Package DB", pure rel_pkg_db)
, ("base unit-id", pure base_unit_id)
+ , ("Semaphore version", pure (show (getSemaphoreProtocolVersion semaphoreVersion)))
]
let showTuple (k, v) = "(" ++ show k ++ ", " ++ show v ++ ")"
pure $ case settings of
=====================================
hadrian/src/Settings/Packages.hs
=====================================
@@ -231,6 +231,10 @@ packageArgs = do
, package hpcBin
? builder (Cabal Flags) ? arg "-build-tool-depends"
+ ------------------------------ semaphore-compat ----------------------------
+ , package semaphoreCompat
+ ? builder (Cabal Flags) ? arg "-build-testing"
+
]
ghcInternalArgs :: Args
=====================================
hadrian/stack.yaml
=====================================
@@ -16,6 +16,7 @@ packages:
- '.'
- '../utils/ghc-toolchain'
- '../libraries/ghc-platform'
+- '../libraries/semaphore-compat'
nix:
enable: false
=====================================
libraries/semaphore-compat
=====================================
@@ -1 +1 @@
-Subproject commit 7929702401d49bc64d809c501ed5fe80aebc3cc1
+Subproject commit 2453a03c00e25e30e321816d53c8dbdb113de08b
=====================================
testsuite/tests/diagnostic-codes/codes.stdout
=====================================
@@ -21,6 +21,8 @@
[GHC-29747] is untested (constructor = DriverMissingSafeHaskellMode)
[GHC-74284] is untested (constructor = DriverInstantiationNodeInDependencyGeneration)
[GHC-66599] is untested (constructor = DriverNoConfiguredLLVMToolchain)
+[GHC-56206] is untested (constructor = DriverSemaphoreVersionMismatch)
+[GHC-19877] is untested (constructor = DriverSemaphoreOpenFailure)
[GHC-81325] is untested (constructor = ExpectingMoreArguments)
[GHC-78125] is untested (constructor = AmbiguityPreventsSolvingCt)
[GHC-84170] is untested (constructor = TcRnModMissingRealSrcSpan)
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/b01201dd5798de0a008cec4d13fdb0c…
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/b01201dd5798de0a008cec4d13fdb0c…
You're receiving this email because of your account on gitlab.haskell.org.
1
0
[Git][ghc/ghc][wip/marge_bot_batch_merge_job] 11 commits: Do not use mkCast during typechecking
by Marge Bot (＠marge-bot) 19 May '26
by Marge Bot (＠marge-bot) 19 May '26
19 May '26
Marge Bot pushed to branch wip/marge_bot_batch_merge_job at Glasgow Haskell Compiler / GHC
Commits:
e8697579 by Simon Peyton Jones at 2026-05-19T03:00:46-04:00
Do not use mkCast during typechecking
This commit fixes #27219. The problem was that the typechecker was using
`mkCast`, whose assertion checks legitimately fail when applied to types
that contain unification variables.
- - - - -
749ec972 by Simon Peyton Jones at 2026-05-19T03:00:46-04:00
Major refactor of the Simplifier
The main payload of this patch is to refactor the Simplifer to avoid
repeated simplification when using Plan (AFTER) for rule rewrites.
The need for this was shown up by #26989.
See Note [Avoid repeated simplification] in GHC.Core.Opt.Simplify.Iteration.
Related refactoring:
* Refactor the two fields `sc_dup` and `sc_env` in `ApplyToVal` into one, `sc_env`.
Reason: the envt is irrelevant in the "simplified" case, so the data type describes
the possiblitiies much more accurately now.
* Some refactoring in `knownCon` to split off `wrapDataConFloats`.
* Refactor `lookupRule` and its auxiliary functions to return `RuleMatch`,
a new data type. See Note [data RuleMatch] in GHC.Core. Ditto for BuiltinRule.
This RuleMatch returns fragments of the target in rm_args and rm_floats,
leaving `rm_rhs` to be the stuff from the RULE itself.
Doing this has routine consequences in GHC.Core.Opt.ConstantFold. Many changes
there but all routine.
* When doing occurrence analysis on RULEs, make the occ-info on the rule
binders relate just to the RHS, not the LHS. See (OUR1) in
Note Note [OccInfo in unfoldings and rules]
This means that Lint must not complain about the fact that the patterns
in the RULE mentions binders that are marked dead.
See Note [Dead occurrences] in GHC.Core.Lint.
I changed the Core pretty-printer so that it didn't suppress dead binders,
else I can't see those binders in RULEs. That led to quite a lot of testsuite wibbles.
* Refactor FloatBinds, so that it is used both by
`exprIsConApp_mabye` and by `lookupRule`
* Move the definition of FloatBinds out of GHc.Core.Make, into GHC.Core.
* Add FloatTick as an extra constructor.
* Refactor `lookupRule` to use `FloatBinds` instead of `BindWrapper`.
This refactor just shares more code.
(Rename GHC.Core.Opt.FloatOut.FloatBinds to FloatLets, to avoid gratuitious
name clash with GHC.Core.FloatBinds.)
Corecion optimisation
* In simpleOpt, when composing coercions, call new function `optTransCo`.
This is much lighter weight than full blown coercion optimisation.
* Make `GHC.Core.Opt.Arity.pushCoValArg` and `pushCoTyArg` return the
coercionLKind of the coercion. This saves recomputing that coercionLKind
at the key call sites in GHC.Core.Opt.Simplify.Iteration.pushCast.
* Rename `addCoerce` in GHC.Core.Simplify.Iteration to become `pushCast`.
* In the `ApplyToVal` case of `pushCast` we had a very unsavoury call to `simplArg`.
I eliminated it by adding a field `sc_cast` to `ApplyToVal` that records any
pending casts. Much nicer now. See Note [The sc_cast field of ApplyToVal].
* Don't optimise coercions if the type-substitution is empty.
See Note [Optimising coercions] in GHC.Core.Opt.Simplify.Iteration.
The fix for #26838 is dramatic. For the test in perf/compiler/T26839 we have
Compiler allocs: Before: 7,363M
After: 688M
Compile time goes down generally. Here are compiler-alloc changes
over 0.5%:
CoOpt_Read(normal) 729,184,920 -0.7%
CoOpt_Singletons(normal) 666,916,960 -4.6% GOOD
LargeRecord(normal) 1,227,056,876 +1.1%
T12227(normal) 256,827,604 -4.6% GOOD
T12425(optasm) 76,879,410 -0.8%
T12545(normal) 787,826,918 -10.8% GOOD
T12707(normal) 775,186,464 -0.9%
T13253(normal) 318,599,596 -0.8%
T14766(normal) 685,857,320 -1.0%
T15304(normal) 1,123,333,422 -2.2%
T15630(normal) 123,142,330 -2.6%
T15630a(normal) 123,092,100 -2.6%
T15703(normal) 299,751,682 -2.9% GOOD
T17516(normal) 964,072,280 +1.0%
T18223(normal) 367,016,820 -6.2% GOOD
T18730(optasm) 130,643,770 -3.3% GOOD
T20261(normal) 535,608,584 -0.7%
T21839c(normal) 340,340,436 -0.9%
T24984(normal) 85,568,392 -1.9%
T3064(normal) 174,631,992 -1.2%
T3294(normal) 1,215,886,432 -0.7%
T5030(normal) 141,449,704 -17.2% GOOD
T5321Fun(normal) 258,484,744 -1.9%
T8095(normal) 770,532,232 -2.7%
T9630(normal) 858,423,408 -14.5% GOOD
T9872c(normal) 1,591,709,448 +0.7%
info_table_map_perf(normal) 19,700,614,458 -1.3%
geo. mean -0.7%
minimum -17.2%
maximum +1.1%
However, strangely there seems to be a 5.0% increase in CoOpt_Read in
the x86_64-linux-fedora43-validate+debug_info+ubsan job, although
there generally a /decrease/ in this test in other builds. The baseline
value looks strange. Anyway I'll just accept it.
Metric Decrease:
CoOpt_Singletons
T12227
T12545
T15703
T18223
T18730
T21839c
T5030
T9630
Metric Increase:
CoOpt_Read
- - - - -
145b7a5b by fendor at 2026-05-19T03:00:47-04:00
Fix regression T27202: `:load` and `:add` work in GHCi
To fix the regression there are conceptually two major things that we
fix:
* We don't remove the `importDirs` from `interactive-session`
* When `:add`ing a module, we don't try to find them via PackageImports
* The PackageImport is wrong as we can't know the package-name at
this stage in ghc/UI.hs
What does it mean to not remove the `importDirs` from
`interactive-session`?
It means that, given some initial `DynFlags`, we will use those
`importDirs` in `interactive-session`.
The initial `DynFlags`, however, depend on how you initialise the GHC
session.
For a simple session, initialised by
ghc -isrc -this-unit-id main
It is simple, just use the `DynFlags` given on the cli.
Thus, `main` and `interactive-session` will have the same `DynFlags`,
except for the `homeUnitId` and `interactive-session` depends on `main`
by construction of the GHCi session.
What about a multiple home unit session, though?
ghc -unit @unit1 -unit @unit2
What are the `DynFlags` in this cli invocation? It shouldn't be either
`@unti1` nor `@unit2`, as the order shouldn't matter or any other
implicit condition.
For consistency, we decide that the initial `DynFlags` are the top
`DynFlags` on the cli, ignoring `-unit` flags.
Thus, in this example, there are no `importsDirs` regardless of what we
might find in `@unit1` and `@unit2`.
But in this invocation:
ghc -isrc -unit @unit1 -unit @unit2
The `interactive-session` will have the `importsDirs` `src`.
Note, `-isrc` will be inherited in `@unit1` and `@unit2`, so you need to
explicitly use `-i` to clear the `importsDirs`, in order to avoid
accidentally adding `src` as an import directory to all other home
units.
This fix has been made possible by the improvements introduced in
!15888, which avoids ambiguity when a home unit shares the `importsDirs`
with the `interactive-session`, on top of being much faster for multiple
home units.
Adds regression tests for T27202 for `:load`ing and `:add`ing modules
that are located in import directories.
- - - - -
c6cc02bb by fendor at 2026-05-19T03:00:47-04:00
Use home unit package db stacks in GHCi prompt and session unit
In order to import modules from home unit dependencies (e.g., `Data.Map`),
the ghci prompt unit needs to populate its `UnitState`.
This is tricky to handle correctly, which `PackageDBFlag`s should we use
to populate the `UnitState`?
We decide, the most intuitive solution for users is to depend on all
`PackageDBFlag`s, so that any dependency can be imported in GHCi.
This assumes consistency in the `PackageDBFlag`s, so no two home units
specify `PackageDBFlag`s that are inconsistent with each other.
We could simply concat all the `PackageDBFlag`s of the existing home
units, but later `PackageDBFlag`s shadow earlier ones, leading to the
last processed home units' `PackageDBFlag`s to shadow the earlier ones.
This is hard to fix, we need to give users the capability to provide ghc
options for the ghci prompt home unit.
However, as this is considerably more work, we decided on an
approximation that should work out most of the time.
Package Db stacks in cabal and stack follow a certain structure:
-no-user-package-db > -package-db $cabal-store > -package-db $local-db
The first two arguments are always the same, namely the
`-no-user-package-db` and `-package-db`.
We compute the longest common prefix over all home units, and use that
as the start of the package db stack. Then, over the rest of the
`PackageDBFlag`s, we simply take the union and append them to our
initial stack.
We assume, that the rest of package dbs only defines very few, "local"
units that are usually not shadowing each other.
This allows us to get a relatively consistent package database stack for
the ghci prompt home unit.
Similar reasoning applies to the session unit in order to add modules to
the session and have dependencies available in the module.
We do something similar for `-package` flags, to make sure only the
correct units are actually visible in the ghci session.
This time, we simply take the union of all `PackageFlag`s, allowing us
to import modules from the home unit dependencies.
In the future, it would be beneficial to allow the user to provide the
exact ghc options to control the visibilities. For now, this will have
to do.
- - - - -
08ca666b by Wen Kokke at 2026-05-19T03:00:48-04:00
rts: Add IPE event class for -l
This commit adds a new IPE event class to the -l RTS flag.
Previously, IPE events were enabled unconditionally.
However, the IPE events can easily grow to hundreds or thousands of megabytes.
With the new event class you can pass, e.g., -l-I to disable IPE events.
- - - - -
b8fdb837 by Wen Kokke at 2026-05-19T03:00:48-04:00
ghc-internal: Add TraceFlags.traceIPE
- - - - -
94f7e58f by Wen Kokke at 2026-05-19T03:00:48-04:00
testsuite: Add test for TraceFlags.traceIpe
- - - - -
1408aa19 by Wen Kokke at 2026-05-19T03:00:48-04:00
ghc-internal: Add DebugFlags.ipe
- - - - -
390d4e33 by Wen Kokke at 2026-05-19T03:00:48-04:00
testsuite: Add test for DebugFlags.ipe
- - - - -
0f996c4e by Duncan Coutts at 2026-05-19T03:00:49-04:00
Document removal of the signal-based interval timer
Update mentions within the RTS section of the users guide.
Add a changelog entry.
- - - - -
ecb62fa3 by Duncan Coutts at 2026-05-19T03:00:49-04:00
Fix section for an recent changelog entry
- - - - -
142 changed files:
- + changelog.d/T27202
- changelog.d/dynamic-trace-flags
- + changelog.d/ipe-event-class
- + changelog.d/no-more-timer-signal
- compiler/GHC/Core.hs
- compiler/GHC/Core/Coercion.hs
- compiler/GHC/Core/Coercion/Opt.hs
- compiler/GHC/Core/Lint.hs
- compiler/GHC/Core/Make.hs
- compiler/GHC/Core/Opt/Arity.hs
- compiler/GHC/Core/Opt/ConstantFold.hs
- compiler/GHC/Core/Opt/FloatIn.hs
- compiler/GHC/Core/Opt/FloatOut.hs
- compiler/GHC/Core/Opt/OccurAnal.hs
- compiler/GHC/Core/Opt/Simplify/Env.hs
- compiler/GHC/Core/Opt/Simplify/Iteration.hs
- compiler/GHC/Core/Opt/Simplify/Utils.hs
- compiler/GHC/Core/Opt/SpecConstr.hs
- compiler/GHC/Core/Opt/Specialise.hs
- compiler/GHC/Core/Ppr.hs
- compiler/GHC/Core/Rules.hs
- compiler/GHC/Core/SimpleOpt.hs
- compiler/GHC/Core/TyCo/Subst.hs
- compiler/GHC/CoreToStg/Prep.hs
- compiler/GHC/Data/List/SetOps.hs
- compiler/GHC/Driver/Config/Core/Lint.hs
- compiler/GHC/Driver/DynFlags.hs
- compiler/GHC/HsToCore/Pmc/Solver.hs
- compiler/GHC/IfaceToCore.hs
- compiler/GHC/Tc/Types/Evidence.hs
- compiler/GHC/Types/Id/Make.hs
- compiler/GHC/Unit/State.hs
- docs/users_guide/profiling.rst
- docs/users_guide/runtime_control.rst
- ghc/GHCi/UI.hs
- ghc/Main.hs
- libraries/ghc-internal/src/GHC/Internal/RTS/Flags.hsc
- rts/IPE.c
- rts/RtsFlags.c
- rts/Trace.c
- rts/Trace.h
- rts/include/rts/EventLogWriter.h
- rts/include/rts/Flags.h
- testsuite/tests/codeGen/should_compile/T25177.stderr
- testsuite/tests/deSugar/should_compile/T13208.stdout
- testsuite/tests/driver/fat-iface/fat014.stdout
- + testsuite/tests/ghci/prog-mhu006/Makefile
- + testsuite/tests/ghci/prog-mhu006/a/A.hs
- + testsuite/tests/ghci/prog-mhu006/all.T
- + testsuite/tests/ghci/prog-mhu006/b/B.hs
- + testsuite/tests/ghci/prog-mhu006/prog-mhu006a.script
- + testsuite/tests/ghci/prog-mhu006/prog-mhu006a.stdout
- + testsuite/tests/ghci/prog-mhu006/unitA
- + testsuite/tests/ghci/prog-mhu006/unitB
- testsuite/tests/ghci/prog018/prog018.stdout
- testsuite/tests/ghci/prog020/Makefile
- testsuite/tests/ghci/prog020/all.T
- testsuite/tests/ghci/prog020/ghci.prog020.script → testsuite/tests/ghci/prog020/ghci.prog020a.script
- testsuite/tests/ghci/prog020/ghci.prog020.stderr → testsuite/tests/ghci/prog020/ghci.prog020a.stderr
- testsuite/tests/ghci/prog020/ghci.prog020.stdout → testsuite/tests/ghci/prog020/ghci.prog020a.stdout
- + testsuite/tests/ghci/prog020/ghci.prog020b.script
- + testsuite/tests/ghci/prog020/ghci.prog020b.stderr
- + testsuite/tests/ghci/prog020/ghci.prog020b.stdout
- + testsuite/tests/ghci/prog023/Makefile
- + testsuite/tests/ghci/prog023/all.T
- + testsuite/tests/ghci/prog023/prog023a.script
- + testsuite/tests/ghci/prog023/prog023a.stdout
- + testsuite/tests/ghci/prog023/prog023b.script
- + testsuite/tests/ghci/prog023/prog023b.stdout
- + testsuite/tests/ghci/prog023/src/A.hs
- + testsuite/tests/ghci/prog024/Makefile
- + testsuite/tests/ghci/prog024/all.T
- + testsuite/tests/ghci/prog024/prog024a.script
- + testsuite/tests/ghci/prog024/prog024a.stdout
- + testsuite/tests/ghci/prog024/prog024b.script
- + testsuite/tests/ghci/prog024/prog024b.stdout
- + testsuite/tests/ghci/prog024/prog024c.script
- + testsuite/tests/ghci/prog024/prog024c.stderr
- + testsuite/tests/ghci/prog024/prog024c.stdout
- + testsuite/tests/ghci/prog024/prog024d.script
- + testsuite/tests/ghci/prog024/prog024d.stderr
- + testsuite/tests/ghci/prog024/prog024d.stdout
- + testsuite/tests/ghci/prog024/prog024e.script
- + testsuite/tests/ghci/prog024/prog024e.stdout
- + testsuite/tests/ghci/prog024/prog024f.script
- + testsuite/tests/ghci/prog024/prog024f.stdout
- + testsuite/tests/ghci/prog024/src/A.hs
- + testsuite/tests/ghci/prog024/src/B.hs
- + testsuite/tests/ghci/prog025/Makefile
- + testsuite/tests/ghci/prog025/a/A.hs
- + testsuite/tests/ghci/prog025/all.T
- + testsuite/tests/ghci/prog025/prog025a.script
- + testsuite/tests/ghci/prog025/prog025a.stdout
- + testsuite/tests/ghci/prog025/prog025b.script
- + testsuite/tests/ghci/prog025/prog025b.stdout
- + testsuite/tests/ghci/prog025/testpkg/Test.hs
- + testsuite/tests/ghci/prog025/testpkg/testpkg-0.1.0.0.pkg
- + testsuite/tests/ghci/prog025/testpkg/testpkg-0.2.0.0.pkg
- + testsuite/tests/ghci/prog025/unitA
- testsuite/tests/ghci/scripts/T13997.stdout
- testsuite/tests/ghci/scripts/T1914.stdout
- testsuite/tests/ghci/scripts/T20217.stdout
- testsuite/tests/ghci/scripts/T8042.stdout
- testsuite/tests/ghci/scripts/T8042recomp.stdout
- testsuite/tests/ghci/should_run/T10920.stderr
- testsuite/tests/interface-stability/ghc-experimental-exports.stdout
- testsuite/tests/interface-stability/ghc-experimental-exports.stdout-mingw32
- testsuite/tests/linters/notes.stdout
- testsuite/tests/numeric/should_compile/T15547.stderr
- testsuite/tests/numeric/should_compile/T20347.stderr
- testsuite/tests/numeric/should_compile/T20374.stderr
- testsuite/tests/numeric/should_compile/T20376.stderr
- + testsuite/tests/perf/compiler/T26989.hs
- + testsuite/tests/perf/compiler/T26989a.hs
- testsuite/tests/perf/compiler/all.T
- testsuite/tests/printer/T18052a.stderr
- + testsuite/tests/rts/T25275/DebugIpe.hs
- + testsuite/tests/rts/T25275/T25275_A.stdout
- + testsuite/tests/rts/T25275/T25275_B.stdout
- + testsuite/tests/rts/T25275/T25275_C.stdout
- + testsuite/tests/rts/T25275/T25275_D.stdout
- + testsuite/tests/rts/T25275/TraceIpe.hs
- + testsuite/tests/rts/T25275/all.T
- testsuite/tests/simplCore/should_compile/DsSpecPragmas.stderr
- testsuite/tests/simplCore/should_compile/RewriteHigherOrderPatterns.stderr
- testsuite/tests/simplCore/should_compile/T15205.stderr
- testsuite/tests/simplCore/should_compile/T18668.stderr
- testsuite/tests/simplCore/should_compile/T19246.stderr
- testsuite/tests/simplCore/should_compile/T19599.stderr
- testsuite/tests/simplCore/should_compile/T19599a.stderr
- testsuite/tests/simplCore/should_compile/T21917.stderr
- testsuite/tests/simplCore/should_compile/T23074.stderr
- testsuite/tests/simplCore/should_compile/T24359a.stderr
- testsuite/tests/simplCore/should_compile/T25160.stderr
- testsuite/tests/simplCore/should_compile/T25718c.stderr-ws-32
- testsuite/tests/simplCore/should_compile/T25718c.stderr-ws-64
- testsuite/tests/simplCore/should_compile/T26051.stderr
- testsuite/tests/simplCore/should_compile/T26116.stderr
- testsuite/tests/simplCore/should_compile/T8331.stderr
- testsuite/tests/simplCore/should_compile/T8848a.stderr
- testsuite/tests/simplCore/should_compile/spec004.stderr
- testsuite/tests/typecheck/should_compile/T13032.stderr
The diff was not included because it is too large.
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/394b2187dc056a42784420ea8cb0be…
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/394b2187dc056a42784420ea8cb0be…
You're receiving this email because of your account on gitlab.haskell.org.
1
0
[Git][ghc/ghc][wip/marge_bot_batch_merge_job] 11 commits: Do not use mkCast during typechecking
by Marge Bot (＠marge-bot) 19 May '26
by Marge Bot (＠marge-bot) 19 May '26
19 May '26
Marge Bot pushed to branch wip/marge_bot_batch_merge_job at Glasgow Haskell Compiler / GHC
Commits:
9683bbbf by Simon Peyton Jones at 2026-05-18T20:36:56-04:00
Do not use mkCast during typechecking
This commit fixes #27219. The problem was that the typechecker was using
`mkCast`, whose assertion checks legitimately fail when applied to types
that contain unification variables.
- - - - -
b68e1e15 by Simon Peyton Jones at 2026-05-18T20:36:56-04:00
Major refactor of the Simplifier
The main payload of this patch is to refactor the Simplifer to avoid
repeated simplification when using Plan (AFTER) for rule rewrites.
The need for this was shown up by #26989.
See Note [Avoid repeated simplification] in GHC.Core.Opt.Simplify.Iteration.
Related refactoring:
* Refactor the two fields `sc_dup` and `sc_env` in `ApplyToVal` into one, `sc_env`.
Reason: the envt is irrelevant in the "simplified" case, so the data type describes
the possiblitiies much more accurately now.
* Some refactoring in `knownCon` to split off `wrapDataConFloats`.
* Refactor `lookupRule` and its auxiliary functions to return `RuleMatch`,
a new data type. See Note [data RuleMatch] in GHC.Core. Ditto for BuiltinRule.
This RuleMatch returns fragments of the target in rm_args and rm_floats,
leaving `rm_rhs` to be the stuff from the RULE itself.
Doing this has routine consequences in GHC.Core.Opt.ConstantFold. Many changes
there but all routine.
* When doing occurrence analysis on RULEs, make the occ-info on the rule
binders relate just to the RHS, not the LHS. See (OUR1) in
Note Note [OccInfo in unfoldings and rules]
This means that Lint must not complain about the fact that the patterns
in the RULE mentions binders that are marked dead.
See Note [Dead occurrences] in GHC.Core.Lint.
I changed the Core pretty-printer so that it didn't suppress dead binders,
else I can't see those binders in RULEs. That led to quite a lot of testsuite wibbles.
* Refactor FloatBinds, so that it is used both by
`exprIsConApp_mabye` and by `lookupRule`
* Move the definition of FloatBinds out of GHc.Core.Make, into GHC.Core.
* Add FloatTick as an extra constructor.
* Refactor `lookupRule` to use `FloatBinds` instead of `BindWrapper`.
This refactor just shares more code.
(Rename GHC.Core.Opt.FloatOut.FloatBinds to FloatLets, to avoid gratuitious
name clash with GHC.Core.FloatBinds.)
Corecion optimisation
* In simpleOpt, when composing coercions, call new function `optTransCo`.
This is much lighter weight than full blown coercion optimisation.
* Make `GHC.Core.Opt.Arity.pushCoValArg` and `pushCoTyArg` return the
coercionLKind of the coercion. This saves recomputing that coercionLKind
at the key call sites in GHC.Core.Opt.Simplify.Iteration.pushCast.
* Rename `addCoerce` in GHC.Core.Simplify.Iteration to become `pushCast`.
* In the `ApplyToVal` case of `pushCast` we had a very unsavoury call to `simplArg`.
I eliminated it by adding a field `sc_cast` to `ApplyToVal` that records any
pending casts. Much nicer now. See Note [The sc_cast field of ApplyToVal].
* Don't optimise coercions if the type-substitution is empty.
See Note [Optimising coercions] in GHC.Core.Opt.Simplify.Iteration.
The fix for #26838 is dramatic. For the test in perf/compiler/T26839 we have
Compiler allocs: Before: 7,363M
After: 688M
Compile time goes down generally. Here are compiler-alloc changes
over 0.5%:
CoOpt_Read(normal) 729,184,920 -0.7%
CoOpt_Singletons(normal) 666,916,960 -4.6% GOOD
LargeRecord(normal) 1,227,056,876 +1.1%
T12227(normal) 256,827,604 -4.6% GOOD
T12425(optasm) 76,879,410 -0.8%
T12545(normal) 787,826,918 -10.8% GOOD
T12707(normal) 775,186,464 -0.9%
T13253(normal) 318,599,596 -0.8%
T14766(normal) 685,857,320 -1.0%
T15304(normal) 1,123,333,422 -2.2%
T15630(normal) 123,142,330 -2.6%
T15630a(normal) 123,092,100 -2.6%
T15703(normal) 299,751,682 -2.9% GOOD
T17516(normal) 964,072,280 +1.0%
T18223(normal) 367,016,820 -6.2% GOOD
T18730(optasm) 130,643,770 -3.3% GOOD
T20261(normal) 535,608,584 -0.7%
T21839c(normal) 340,340,436 -0.9%
T24984(normal) 85,568,392 -1.9%
T3064(normal) 174,631,992 -1.2%
T3294(normal) 1,215,886,432 -0.7%
T5030(normal) 141,449,704 -17.2% GOOD
T5321Fun(normal) 258,484,744 -1.9%
T8095(normal) 770,532,232 -2.7%
T9630(normal) 858,423,408 -14.5% GOOD
T9872c(normal) 1,591,709,448 +0.7%
info_table_map_perf(normal) 19,700,614,458 -1.3%
geo. mean -0.7%
minimum -17.2%
maximum +1.1%
However, strangely there seems to be a 5.0% increase in CoOpt_Read in
the x86_64-linux-fedora43-validate+debug_info+ubsan job, although
there generally a /decrease/ in this test in other builds. The baseline
value looks strange. Anyway I'll just accept it.
Metric Decrease:
CoOpt_Singletons
T12227
T12545
T15703
T18223
T18730
T21839c
T5030
T9630
Metric Increase:
CoOpt_Read
- - - - -
95cb621a by fendor at 2026-05-18T20:36:57-04:00
Fix regression T27202: `:load` and `:add` work in GHCi
To fix the regression there are conceptually two major things that we
fix:
* We don't remove the `importDirs` from `interactive-session`
* When `:add`ing a module, we don't try to find them via PackageImports
* The PackageImport is wrong as we can't know the package-name at
this stage in ghc/UI.hs
What does it mean to not remove the `importDirs` from
`interactive-session`?
It means that, given some initial `DynFlags`, we will use those
`importDirs` in `interactive-session`.
The initial `DynFlags`, however, depend on how you initialise the GHC
session.
For a simple session, initialised by
ghc -isrc -this-unit-id main
It is simple, just use the `DynFlags` given on the cli.
Thus, `main` and `interactive-session` will have the same `DynFlags`,
except for the `homeUnitId` and `interactive-session` depends on `main`
by construction of the GHCi session.
What about a multiple home unit session, though?
ghc -unit @unit1 -unit @unit2
What are the `DynFlags` in this cli invocation? It shouldn't be either
`@unti1` nor `@unit2`, as the order shouldn't matter or any other
implicit condition.
For consistency, we decide that the initial `DynFlags` are the top
`DynFlags` on the cli, ignoring `-unit` flags.
Thus, in this example, there are no `importsDirs` regardless of what we
might find in `@unit1` and `@unit2`.
But in this invocation:
ghc -isrc -unit @unit1 -unit @unit2
The `interactive-session` will have the `importsDirs` `src`.
Note, `-isrc` will be inherited in `@unit1` and `@unit2`, so you need to
explicitly use `-i` to clear the `importsDirs`, in order to avoid
accidentally adding `src` as an import directory to all other home
units.
This fix has been made possible by the improvements introduced in
!15888, which avoids ambiguity when a home unit shares the `importsDirs`
with the `interactive-session`, on top of being much faster for multiple
home units.
Adds regression tests for T27202 for `:load`ing and `:add`ing modules
that are located in import directories.
- - - - -
31b708f2 by fendor at 2026-05-18T20:36:57-04:00
Use home unit package db stacks in GHCi prompt and session unit
In order to import modules from home unit dependencies (e.g., `Data.Map`),
the ghci prompt unit needs to populate its `UnitState`.
This is tricky to handle correctly, which `PackageDBFlag`s should we use
to populate the `UnitState`?
We decide, the most intuitive solution for users is to depend on all
`PackageDBFlag`s, so that any dependency can be imported in GHCi.
This assumes consistency in the `PackageDBFlag`s, so no two home units
specify `PackageDBFlag`s that are inconsistent with each other.
We could simply concat all the `PackageDBFlag`s of the existing home
units, but later `PackageDBFlag`s shadow earlier ones, leading to the
last processed home units' `PackageDBFlag`s to shadow the earlier ones.
This is hard to fix, we need to give users the capability to provide ghc
options for the ghci prompt home unit.
However, as this is considerably more work, we decided on an
approximation that should work out most of the time.
Package Db stacks in cabal and stack follow a certain structure:
-no-user-package-db > -package-db $cabal-store > -package-db $local-db
The first two arguments are always the same, namely the
`-no-user-package-db` and `-package-db`.
We compute the longest common prefix over all home units, and use that
as the start of the package db stack. Then, over the rest of the
`PackageDBFlag`s, we simply take the union and append them to our
initial stack.
We assume, that the rest of package dbs only defines very few, "local"
units that are usually not shadowing each other.
This allows us to get a relatively consistent package database stack for
the ghci prompt home unit.
Similar reasoning applies to the session unit in order to add modules to
the session and have dependencies available in the module.
We do something similar for `-package` flags, to make sure only the
correct units are actually visible in the ghci session.
This time, we simply take the union of all `PackageFlag`s, allowing us
to import modules from the home unit dependencies.
In the future, it would be beneficial to allow the user to provide the
exact ghc options to control the visibilities. For now, this will have
to do.
- - - - -
957801e6 by Wen Kokke at 2026-05-18T20:36:58-04:00
rts: Add IPE event class for -l
This commit adds a new IPE event class to the -l RTS flag.
Previously, IPE events were enabled unconditionally.
However, the IPE events can easily grow to hundreds or thousands of megabytes.
With the new event class you can pass, e.g., -l-I to disable IPE events.
- - - - -
ea7acee2 by Wen Kokke at 2026-05-18T20:36:58-04:00
ghc-internal: Add TraceFlags.traceIPE
- - - - -
3f0c18db by Wen Kokke at 2026-05-18T20:36:58-04:00
testsuite: Add test for TraceFlags.traceIpe
- - - - -
24d8fb6a by Wen Kokke at 2026-05-18T20:36:59-04:00
ghc-internal: Add DebugFlags.ipe
- - - - -
53bbeb8c by Wen Kokke at 2026-05-18T20:36:59-04:00
testsuite: Add test for DebugFlags.ipe
- - - - -
9d02df6f by Duncan Coutts at 2026-05-18T20:36:59-04:00
Document removal of the signal-based interval timer
Update mentions within the RTS section of the users guide.
Add a changelog entry.
- - - - -
394b2187 by Duncan Coutts at 2026-05-18T20:37:00-04:00
Fix section for an recent changelog entry
- - - - -
142 changed files:
- + changelog.d/T27202
- changelog.d/dynamic-trace-flags
- + changelog.d/ipe-event-class
- + changelog.d/no-more-timer-signal
- compiler/GHC/Core.hs
- compiler/GHC/Core/Coercion.hs
- compiler/GHC/Core/Coercion/Opt.hs
- compiler/GHC/Core/Lint.hs
- compiler/GHC/Core/Make.hs
- compiler/GHC/Core/Opt/Arity.hs
- compiler/GHC/Core/Opt/ConstantFold.hs
- compiler/GHC/Core/Opt/FloatIn.hs
- compiler/GHC/Core/Opt/FloatOut.hs
- compiler/GHC/Core/Opt/OccurAnal.hs
- compiler/GHC/Core/Opt/Simplify/Env.hs
- compiler/GHC/Core/Opt/Simplify/Iteration.hs
- compiler/GHC/Core/Opt/Simplify/Utils.hs
- compiler/GHC/Core/Opt/SpecConstr.hs
- compiler/GHC/Core/Opt/Specialise.hs
- compiler/GHC/Core/Ppr.hs
- compiler/GHC/Core/Rules.hs
- compiler/GHC/Core/SimpleOpt.hs
- compiler/GHC/Core/TyCo/Subst.hs
- compiler/GHC/CoreToStg/Prep.hs
- compiler/GHC/Data/List/SetOps.hs
- compiler/GHC/Driver/Config/Core/Lint.hs
- compiler/GHC/Driver/DynFlags.hs
- compiler/GHC/HsToCore/Pmc/Solver.hs
- compiler/GHC/IfaceToCore.hs
- compiler/GHC/Tc/Types/Evidence.hs
- compiler/GHC/Types/Id/Make.hs
- compiler/GHC/Unit/State.hs
- docs/users_guide/profiling.rst
- docs/users_guide/runtime_control.rst
- ghc/GHCi/UI.hs
- ghc/Main.hs
- libraries/ghc-internal/src/GHC/Internal/RTS/Flags.hsc
- rts/IPE.c
- rts/RtsFlags.c
- rts/Trace.c
- rts/Trace.h
- rts/include/rts/EventLogWriter.h
- rts/include/rts/Flags.h
- testsuite/tests/codeGen/should_compile/T25177.stderr
- testsuite/tests/deSugar/should_compile/T13208.stdout
- testsuite/tests/driver/fat-iface/fat014.stdout
- + testsuite/tests/ghci/prog-mhu006/Makefile
- + testsuite/tests/ghci/prog-mhu006/a/A.hs
- + testsuite/tests/ghci/prog-mhu006/all.T
- + testsuite/tests/ghci/prog-mhu006/b/B.hs
- + testsuite/tests/ghci/prog-mhu006/prog-mhu006a.script
- + testsuite/tests/ghci/prog-mhu006/prog-mhu006a.stdout
- + testsuite/tests/ghci/prog-mhu006/unitA
- + testsuite/tests/ghci/prog-mhu006/unitB
- testsuite/tests/ghci/prog018/prog018.stdout
- testsuite/tests/ghci/prog020/Makefile
- testsuite/tests/ghci/prog020/all.T
- testsuite/tests/ghci/prog020/ghci.prog020.script → testsuite/tests/ghci/prog020/ghci.prog020a.script
- testsuite/tests/ghci/prog020/ghci.prog020.stderr → testsuite/tests/ghci/prog020/ghci.prog020a.stderr
- testsuite/tests/ghci/prog020/ghci.prog020.stdout → testsuite/tests/ghci/prog020/ghci.prog020a.stdout
- + testsuite/tests/ghci/prog020/ghci.prog020b.script
- + testsuite/tests/ghci/prog020/ghci.prog020b.stderr
- + testsuite/tests/ghci/prog020/ghci.prog020b.stdout
- + testsuite/tests/ghci/prog023/Makefile
- + testsuite/tests/ghci/prog023/all.T
- + testsuite/tests/ghci/prog023/prog023a.script
- + testsuite/tests/ghci/prog023/prog023a.stdout
- + testsuite/tests/ghci/prog023/prog023b.script
- + testsuite/tests/ghci/prog023/prog023b.stdout
- + testsuite/tests/ghci/prog023/src/A.hs
- + testsuite/tests/ghci/prog024/Makefile
- + testsuite/tests/ghci/prog024/all.T
- + testsuite/tests/ghci/prog024/prog024a.script
- + testsuite/tests/ghci/prog024/prog024a.stdout
- + testsuite/tests/ghci/prog024/prog024b.script
- + testsuite/tests/ghci/prog024/prog024b.stdout
- + testsuite/tests/ghci/prog024/prog024c.script
- + testsuite/tests/ghci/prog024/prog024c.stderr
- + testsuite/tests/ghci/prog024/prog024c.stdout
- + testsuite/tests/ghci/prog024/prog024d.script
- + testsuite/tests/ghci/prog024/prog024d.stderr
- + testsuite/tests/ghci/prog024/prog024d.stdout
- + testsuite/tests/ghci/prog024/prog024e.script
- + testsuite/tests/ghci/prog024/prog024e.stdout
- + testsuite/tests/ghci/prog024/prog024f.script
- + testsuite/tests/ghci/prog024/prog024f.stdout
- + testsuite/tests/ghci/prog024/src/A.hs
- + testsuite/tests/ghci/prog024/src/B.hs
- + testsuite/tests/ghci/prog025/Makefile
- + testsuite/tests/ghci/prog025/a/A.hs
- + testsuite/tests/ghci/prog025/all.T
- + testsuite/tests/ghci/prog025/prog025a.script
- + testsuite/tests/ghci/prog025/prog025a.stdout
- + testsuite/tests/ghci/prog025/prog025b.script
- + testsuite/tests/ghci/prog025/prog025b.stdout
- + testsuite/tests/ghci/prog025/testpkg/Test.hs
- + testsuite/tests/ghci/prog025/testpkg/testpkg-0.1.0.0.pkg
- + testsuite/tests/ghci/prog025/testpkg/testpkg-0.2.0.0.pkg
- + testsuite/tests/ghci/prog025/unitA
- testsuite/tests/ghci/scripts/T13997.stdout
- testsuite/tests/ghci/scripts/T1914.stdout
- testsuite/tests/ghci/scripts/T20217.stdout
- testsuite/tests/ghci/scripts/T8042.stdout
- testsuite/tests/ghci/scripts/T8042recomp.stdout
- testsuite/tests/ghci/should_run/T10920.stderr
- testsuite/tests/interface-stability/ghc-experimental-exports.stdout
- testsuite/tests/interface-stability/ghc-experimental-exports.stdout-mingw32
- testsuite/tests/linters/notes.stdout
- testsuite/tests/numeric/should_compile/T15547.stderr
- testsuite/tests/numeric/should_compile/T20347.stderr
- testsuite/tests/numeric/should_compile/T20374.stderr
- testsuite/tests/numeric/should_compile/T20376.stderr
- + testsuite/tests/perf/compiler/T26989.hs
- + testsuite/tests/perf/compiler/T26989a.hs
- testsuite/tests/perf/compiler/all.T
- testsuite/tests/printer/T18052a.stderr
- + testsuite/tests/rts/T25275/DebugIpe.hs
- + testsuite/tests/rts/T25275/T25275_A.stdout
- + testsuite/tests/rts/T25275/T25275_B.stdout
- + testsuite/tests/rts/T25275/T25275_C.stdout
- + testsuite/tests/rts/T25275/T25275_D.stdout
- + testsuite/tests/rts/T25275/TraceIpe.hs
- + testsuite/tests/rts/T25275/all.T
- testsuite/tests/simplCore/should_compile/DsSpecPragmas.stderr
- testsuite/tests/simplCore/should_compile/RewriteHigherOrderPatterns.stderr
- testsuite/tests/simplCore/should_compile/T15205.stderr
- testsuite/tests/simplCore/should_compile/T18668.stderr
- testsuite/tests/simplCore/should_compile/T19246.stderr
- testsuite/tests/simplCore/should_compile/T19599.stderr
- testsuite/tests/simplCore/should_compile/T19599a.stderr
- testsuite/tests/simplCore/should_compile/T21917.stderr
- testsuite/tests/simplCore/should_compile/T23074.stderr
- testsuite/tests/simplCore/should_compile/T24359a.stderr
- testsuite/tests/simplCore/should_compile/T25160.stderr
- testsuite/tests/simplCore/should_compile/T25718c.stderr-ws-32
- testsuite/tests/simplCore/should_compile/T25718c.stderr-ws-64
- testsuite/tests/simplCore/should_compile/T26051.stderr
- testsuite/tests/simplCore/should_compile/T26116.stderr
- testsuite/tests/simplCore/should_compile/T8331.stderr
- testsuite/tests/simplCore/should_compile/T8848a.stderr
- testsuite/tests/simplCore/should_compile/spec004.stderr
- testsuite/tests/typecheck/should_compile/T13032.stderr
The diff was not included because it is too large.
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/55c4d1b36d55fcba24479a02f2f01d…
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/55c4d1b36d55fcba24479a02f2f01d…
You're receiving this email because of your account on gitlab.haskell.org.
1
0
[Git][ghc/ghc] Pushed new branch wip/jeltsch/base-buildable-with-ghc-9-14
by Wolfgang Jeltsch (＠jeltsch) 18 May '26
by Wolfgang Jeltsch (＠jeltsch) 18 May '26
18 May '26
Wolfgang Jeltsch pushed new branch wip/jeltsch/base-buildable-with-ghc-9-14 at Glasgow Haskell Compiler / GHC
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/tree/wip/jeltsch/base-buildable-with-g…
You're receiving this email because of your account on gitlab.haskell.org.
1
0
[Git][ghc/ghc][wip/marge_bot_batch_merge_job] 7 commits: Do not use mkCast during typechecking
by Marge Bot (＠marge-bot) 18 May '26
by Marge Bot (＠marge-bot) 18 May '26
18 May '26
Marge Bot pushed to branch wip/marge_bot_batch_merge_job at Glasgow Haskell Compiler / GHC
Commits:
ab9a2ef4 by Simon Peyton Jones at 2026-05-18T14:15:05-04:00
Do not use mkCast during typechecking
This commit fixes #27219. The problem was that the typechecker was using
`mkCast`, whose assertion checks legitimately fail when applied to types
that contain unification variables.
- - - - -
6d0fd5d3 by Simon Peyton Jones at 2026-05-18T14:15:05-04:00
Major refactor of the Simplifier
The main payload of this patch is to refactor the Simplifer to avoid
repeated simplification when using Plan (AFTER) for rule rewrites.
The need for this was shown up by #26989.
See Note [Avoid repeated simplification] in GHC.Core.Opt.Simplify.Iteration.
Related refactoring:
* Refactor the two fields `sc_dup` and `sc_env` in `ApplyToVal` into one, `sc_env`.
Reason: the envt is irrelevant in the "simplified" case, so the data type describes
the possiblitiies much more accurately now.
* Some refactoring in `knownCon` to split off `wrapDataConFloats`.
* Refactor `lookupRule` and its auxiliary functions to return `RuleMatch`,
a new data type. See Note [data RuleMatch] in GHC.Core. Ditto for BuiltinRule.
This RuleMatch returns fragments of the target in rm_args and rm_floats,
leaving `rm_rhs` to be the stuff from the RULE itself.
Doing this has routine consequences in GHC.Core.Opt.ConstantFold. Many changes
there but all routine.
* When doing occurrence analysis on RULEs, make the occ-info on the rule
binders relate just to the RHS, not the LHS. See (OUR1) in
Note Note [OccInfo in unfoldings and rules]
This means that Lint must not complain about the fact that the patterns
in the RULE mentions binders that are marked dead.
See Note [Dead occurrences] in GHC.Core.Lint.
I changed the Core pretty-printer so that it didn't suppress dead binders,
else I can't see those binders in RULEs. That led to quite a lot of testsuite wibbles.
* Refactor FloatBinds, so that it is used both by
`exprIsConApp_mabye` and by `lookupRule`
* Move the definition of FloatBinds out of GHc.Core.Make, into GHC.Core.
* Add FloatTick as an extra constructor.
* Refactor `lookupRule` to use `FloatBinds` instead of `BindWrapper`.
This refactor just shares more code.
(Rename GHC.Core.Opt.FloatOut.FloatBinds to FloatLets, to avoid gratuitious
name clash with GHC.Core.FloatBinds.)
Corecion optimisation
* In simpleOpt, when composing coercions, call new function `optTransCo`.
This is much lighter weight than full blown coercion optimisation.
* Make `GHC.Core.Opt.Arity.pushCoValArg` and `pushCoTyArg` return the
coercionLKind of the coercion. This saves recomputing that coercionLKind
at the key call sites in GHC.Core.Opt.Simplify.Iteration.pushCast.
* Rename `addCoerce` in GHC.Core.Simplify.Iteration to become `pushCast`.
* In the `ApplyToVal` case of `pushCast` we had a very unsavoury call to `simplArg`.
I eliminated it by adding a field `sc_cast` to `ApplyToVal` that records any
pending casts. Much nicer now. See Note [The sc_cast field of ApplyToVal].
* Don't optimise coercions if the type-substitution is empty.
See Note [Optimising coercions] in GHC.Core.Opt.Simplify.Iteration.
The fix for #26838 is dramatic. For the test in perf/compiler/T26839 we have
Compiler allocs: Before: 7,363M
After: 688M
Compile time goes down generally. Here are compiler-alloc changes
over 0.5%:
CoOpt_Read(normal) 729,184,920 -0.7%
CoOpt_Singletons(normal) 666,916,960 -4.6% GOOD
LargeRecord(normal) 1,227,056,876 +1.1%
T12227(normal) 256,827,604 -4.6% GOOD
T12425(optasm) 76,879,410 -0.8%
T12545(normal) 787,826,918 -10.8% GOOD
T12707(normal) 775,186,464 -0.9%
T13253(normal) 318,599,596 -0.8%
T14766(normal) 685,857,320 -1.0%
T15304(normal) 1,123,333,422 -2.2%
T15630(normal) 123,142,330 -2.6%
T15630a(normal) 123,092,100 -2.6%
T15703(normal) 299,751,682 -2.9% GOOD
T17516(normal) 964,072,280 +1.0%
T18223(normal) 367,016,820 -6.2% GOOD
T18730(optasm) 130,643,770 -3.3% GOOD
T20261(normal) 535,608,584 -0.7%
T21839c(normal) 340,340,436 -0.9%
T24984(normal) 85,568,392 -1.9%
T3064(normal) 174,631,992 -1.2%
T3294(normal) 1,215,886,432 -0.7%
T5030(normal) 141,449,704 -17.2% GOOD
T5321Fun(normal) 258,484,744 -1.9%
T8095(normal) 770,532,232 -2.7%
T9630(normal) 858,423,408 -14.5% GOOD
T9872c(normal) 1,591,709,448 +0.7%
info_table_map_perf(normal) 19,700,614,458 -1.3%
geo. mean -0.7%
minimum -17.2%
maximum +1.1%
However, strangely there seems to be a 5.0% increase in CoOpt_Read in
the x86_64-linux-fedora43-validate+debug_info+ubsan job, although
there generally a /decrease/ in this test in other builds. The baseline
value looks strange. Anyway I'll just accept it.
Metric Decrease:
CoOpt_Singletons
T12227
T12545
T15703
T18223
T18730
T21839c
T5030
T9630
Metric Increase:
CoOpt_Read
- - - - -
81f4862e by Vladislav Zavialov at 2026-05-18T14:15:06-04:00
Add type families: Tuple, Constraints, Tuple#, Sum# (#27179)
These type families map tuples of types to the corresponding Tuple<N>,
Tuple<N>#, CTuple<N>, and Sum<N># types. Some examples at N=2:
Tuple (Int, Bool) = Tuple2 Int Bool
Constraints (Show a, Eq a) = CTuple2 (Show a) (Eq a)
Tuple# (Int#, Float#) = Tuple2# Int# Float#
Sum# (Int#, Float#) = Sum2# Int# Float#
See GHC Proposal #145 "Non-punning list and tuple syntax".
To make the Sum# instance at N=64 possible, this patch also introduces
the Sum64# constructor declaration and bumps mAX_SUM_SIZE from 63 to 64.
Metric Increase:
ghc_experimental_dir
- - - - -
8c23b3e0 by fendor at 2026-05-18T14:15:07-04:00
Fix regression T27202: `:load` and `:add` work in GHCi
To fix the regression there are conceptually two major things that we
fix:
* We don't remove the `importDirs` from `interactive-session`
* When `:add`ing a module, we don't try to find them via PackageImports
* The PackageImport is wrong as we can't know the package-name at
this stage in ghc/UI.hs
What does it mean to not remove the `importDirs` from
`interactive-session`?
It means that, given some initial `DynFlags`, we will use those
`importDirs` in `interactive-session`.
The initial `DynFlags`, however, depend on how you initialise the GHC
session.
For a simple session, initialised by
ghc -isrc -this-unit-id main
It is simple, just use the `DynFlags` given on the cli.
Thus, `main` and `interactive-session` will have the same `DynFlags`,
except for the `homeUnitId` and `interactive-session` depends on `main`
by construction of the GHCi session.
What about a multiple home unit session, though?
ghc -unit @unit1 -unit @unit2
What are the `DynFlags` in this cli invocation? It shouldn't be either
`@unti1` nor `@unit2`, as the order shouldn't matter or any other
implicit condition.
For consistency, we decide that the initial `DynFlags` are the top
`DynFlags` on the cli, ignoring `-unit` flags.
Thus, in this example, there are no `importsDirs` regardless of what we
might find in `@unit1` and `@unit2`.
But in this invocation:
ghc -isrc -unit @unit1 -unit @unit2
The `interactive-session` will have the `importsDirs` `src`.
Note, `-isrc` will be inherited in `@unit1` and `@unit2`, so you need to
explicitly use `-i` to clear the `importsDirs`, in order to avoid
accidentally adding `src` as an import directory to all other home
units.
This fix has been made possible by the improvements introduced in
!15888, which avoids ambiguity when a home unit shares the `importsDirs`
with the `interactive-session`, on top of being much faster for multiple
home units.
Adds regression tests for T27202 for `:load`ing and `:add`ing modules
that are located in import directories.
- - - - -
17a83421 by fendor at 2026-05-18T14:15:08-04:00
Use home unit package db stacks in GHCi prompt and session unit
In order to import modules from home unit dependencies (e.g., `Data.Map`),
the ghci prompt unit needs to populate its `UnitState`.
This is tricky to handle correctly, which `PackageDBFlag`s should we use
to populate the `UnitState`?
We decide, the most intuitive solution for users is to depend on all
`PackageDBFlag`s, so that any dependency can be imported in GHCi.
This assumes consistency in the `PackageDBFlag`s, so no two home units
specify `PackageDBFlag`s that are inconsistent with each other.
We could simply concat all the `PackageDBFlag`s of the existing home
units, but later `PackageDBFlag`s shadow earlier ones, leading to the
last processed home units' `PackageDBFlag`s to shadow the earlier ones.
This is hard to fix, we need to give users the capability to provide ghc
options for the ghci prompt home unit.
However, as this is considerably more work, we decided on an
approximation that should work out most of the time.
Package Db stacks in cabal and stack follow a certain structure:
-no-user-package-db > -package-db $cabal-store > -package-db $local-db
The first two arguments are always the same, namely the
`-no-user-package-db` and `-package-db`.
We compute the longest common prefix over all home units, and use that
as the start of the package db stack. Then, over the rest of the
`PackageDBFlag`s, we simply take the union and append them to our
initial stack.
We assume, that the rest of package dbs only defines very few, "local"
units that are usually not shadowing each other.
This allows us to get a relatively consistent package database stack for
the ghci prompt home unit.
Similar reasoning applies to the session unit in order to add modules to
the session and have dependencies available in the module.
We do something similar for `-package` flags, to make sure only the
correct units are actually visible in the ghci session.
This time, we simply take the union of all `PackageFlag`s, allowing us
to import modules from the home unit dependencies.
In the future, it would be beneficial to allow the user to provide the
exact ghc options to control the visibilities. For now, this will have
to do.
- - - - -
bb6ac1c9 by Duncan Coutts at 2026-05-18T14:15:08-04:00
Document removal of the signal-based interval timer
Update mentions within the RTS section of the users guide.
Add a changelog entry.
- - - - -
55c4d1b3 by Duncan Coutts at 2026-05-18T14:15:08-04:00
Fix section for an recent changelog entry
- - - - -
149 changed files:
- + changelog.d/T27202
- changelog.d/dynamic-trace-flags
- + changelog.d/lib-add-tuple-tyfam-27179
- + changelog.d/no-more-timer-signal
- compiler/GHC/Core.hs
- compiler/GHC/Core/Coercion.hs
- compiler/GHC/Core/Coercion/Opt.hs
- compiler/GHC/Core/Lint.hs
- compiler/GHC/Core/Make.hs
- compiler/GHC/Core/Opt/Arity.hs
- compiler/GHC/Core/Opt/ConstantFold.hs
- compiler/GHC/Core/Opt/FloatIn.hs
- compiler/GHC/Core/Opt/FloatOut.hs
- compiler/GHC/Core/Opt/OccurAnal.hs
- compiler/GHC/Core/Opt/Simplify/Env.hs
- compiler/GHC/Core/Opt/Simplify/Iteration.hs
- compiler/GHC/Core/Opt/Simplify/Utils.hs
- compiler/GHC/Core/Opt/SpecConstr.hs
- compiler/GHC/Core/Opt/Specialise.hs
- compiler/GHC/Core/Ppr.hs
- compiler/GHC/Core/Rules.hs
- compiler/GHC/Core/SimpleOpt.hs
- compiler/GHC/Core/TyCo/Subst.hs
- compiler/GHC/CoreToStg/Prep.hs
- compiler/GHC/Data/List/SetOps.hs
- compiler/GHC/Driver/Config/Core/Lint.hs
- compiler/GHC/Driver/DynFlags.hs
- compiler/GHC/HsToCore/Pmc/Solver.hs
- compiler/GHC/IfaceToCore.hs
- compiler/GHC/Settings/Constants.hs
- compiler/GHC/Tc/Types/Evidence.hs
- compiler/GHC/Types/Id/Make.hs
- compiler/GHC/Unit/State.hs
- docs/users_guide/profiling.rst
- docs/users_guide/runtime_control.rst
- ghc/GHCi/UI.hs
- ghc/Main.hs
- libraries/base/src/GHC/Base.hs
- libraries/base/src/GHC/Exts.hs
- libraries/ghc-experimental/src/Data/Sum/Experimental.hs
- libraries/ghc-experimental/src/Data/Tuple/Experimental.hs
- libraries/ghc-internal/src/GHC/Internal/Base.hs
- libraries/ghc-internal/src/GHC/Internal/Exts.hs
- libraries/ghc-internal/src/GHC/Internal/Types.hs
- testsuite/tests/codeGen/should_compile/T25177.stderr
- testsuite/tests/deSugar/should_compile/T13208.stdout
- testsuite/tests/driver/fat-iface/fat014.stdout
- + testsuite/tests/ghci/prog-mhu006/Makefile
- + testsuite/tests/ghci/prog-mhu006/a/A.hs
- + testsuite/tests/ghci/prog-mhu006/all.T
- + testsuite/tests/ghci/prog-mhu006/b/B.hs
- + testsuite/tests/ghci/prog-mhu006/prog-mhu006a.script
- + testsuite/tests/ghci/prog-mhu006/prog-mhu006a.stdout
- + testsuite/tests/ghci/prog-mhu006/unitA
- + testsuite/tests/ghci/prog-mhu006/unitB
- testsuite/tests/ghci/prog018/prog018.stdout
- testsuite/tests/ghci/prog020/Makefile
- testsuite/tests/ghci/prog020/all.T
- testsuite/tests/ghci/prog020/ghci.prog020.script → testsuite/tests/ghci/prog020/ghci.prog020a.script
- testsuite/tests/ghci/prog020/ghci.prog020.stderr → testsuite/tests/ghci/prog020/ghci.prog020a.stderr
- testsuite/tests/ghci/prog020/ghci.prog020.stdout → testsuite/tests/ghci/prog020/ghci.prog020a.stdout
- + testsuite/tests/ghci/prog020/ghci.prog020b.script
- + testsuite/tests/ghci/prog020/ghci.prog020b.stderr
- + testsuite/tests/ghci/prog020/ghci.prog020b.stdout
- + testsuite/tests/ghci/prog023/Makefile
- + testsuite/tests/ghci/prog023/all.T
- + testsuite/tests/ghci/prog023/prog023a.script
- + testsuite/tests/ghci/prog023/prog023a.stdout
- + testsuite/tests/ghci/prog023/prog023b.script
- + testsuite/tests/ghci/prog023/prog023b.stdout
- + testsuite/tests/ghci/prog023/src/A.hs
- + testsuite/tests/ghci/prog024/Makefile
- + testsuite/tests/ghci/prog024/all.T
- + testsuite/tests/ghci/prog024/prog024a.script
- + testsuite/tests/ghci/prog024/prog024a.stdout
- + testsuite/tests/ghci/prog024/prog024b.script
- + testsuite/tests/ghci/prog024/prog024b.stdout
- + testsuite/tests/ghci/prog024/prog024c.script
- + testsuite/tests/ghci/prog024/prog024c.stderr
- + testsuite/tests/ghci/prog024/prog024c.stdout
- + testsuite/tests/ghci/prog024/prog024d.script
- + testsuite/tests/ghci/prog024/prog024d.stderr
- + testsuite/tests/ghci/prog024/prog024d.stdout
- + testsuite/tests/ghci/prog024/prog024e.script
- + testsuite/tests/ghci/prog024/prog024e.stdout
- + testsuite/tests/ghci/prog024/prog024f.script
- + testsuite/tests/ghci/prog024/prog024f.stdout
- + testsuite/tests/ghci/prog024/src/A.hs
- + testsuite/tests/ghci/prog024/src/B.hs
- + testsuite/tests/ghci/prog025/Makefile
- + testsuite/tests/ghci/prog025/a/A.hs
- + testsuite/tests/ghci/prog025/all.T
- + testsuite/tests/ghci/prog025/prog025a.script
- + testsuite/tests/ghci/prog025/prog025a.stdout
- + testsuite/tests/ghci/prog025/prog025b.script
- + testsuite/tests/ghci/prog025/prog025b.stdout
- + testsuite/tests/ghci/prog025/testpkg/Test.hs
- + testsuite/tests/ghci/prog025/testpkg/testpkg-0.1.0.0.pkg
- + testsuite/tests/ghci/prog025/testpkg/testpkg-0.2.0.0.pkg
- + testsuite/tests/ghci/prog025/unitA
- testsuite/tests/ghci/scripts/ListTuplePunsPprNoAbbrevTuple.stdout
- testsuite/tests/ghci/scripts/T13997.stdout
- testsuite/tests/ghci/scripts/T1914.stdout
- testsuite/tests/ghci/scripts/T20217.stdout
- testsuite/tests/ghci/scripts/T8042.stdout
- testsuite/tests/ghci/scripts/T8042recomp.stdout
- testsuite/tests/ghci/scripts/all.T
- testsuite/tests/ghci/should_run/T10920.stderr
- testsuite/tests/interface-stability/ghc-experimental-exports.stdout
- testsuite/tests/interface-stability/ghc-experimental-exports.stdout-mingw32
- testsuite/tests/interface-stability/ghc-prim-exports.stdout
- testsuite/tests/interface-stability/ghc-prim-exports.stdout-mingw32
- testsuite/tests/linters/notes.stdout
- testsuite/tests/numeric/should_compile/T15547.stderr
- testsuite/tests/numeric/should_compile/T20347.stderr
- testsuite/tests/numeric/should_compile/T20374.stderr
- testsuite/tests/numeric/should_compile/T20376.stderr
- testsuite/tests/parser/should_compile/ListTuplePunsSuccess1.hs
- testsuite/tests/parser/should_compile/all.T
- + testsuite/tests/parser/should_fail/ListTuplePunsFail6.hs
- + testsuite/tests/parser/should_fail/ListTuplePunsFail6.stderr
- testsuite/tests/parser/should_fail/all.T
- testsuite/tests/parser/should_run/ListTuplePunsConstraints.hs
- + testsuite/tests/perf/compiler/T26989.hs
- + testsuite/tests/perf/compiler/T26989a.hs
- testsuite/tests/perf/compiler/all.T
- testsuite/tests/printer/T18052a.stderr
- testsuite/tests/profiling/should_run/callstack001.stdout
- testsuite/tests/simplCore/should_compile/DsSpecPragmas.stderr
- testsuite/tests/simplCore/should_compile/RewriteHigherOrderPatterns.stderr
- testsuite/tests/simplCore/should_compile/T15205.stderr
- testsuite/tests/simplCore/should_compile/T18668.stderr
- testsuite/tests/simplCore/should_compile/T19246.stderr
- testsuite/tests/simplCore/should_compile/T19599.stderr
- testsuite/tests/simplCore/should_compile/T19599a.stderr
- testsuite/tests/simplCore/should_compile/T21917.stderr
- testsuite/tests/simplCore/should_compile/T23074.stderr
- testsuite/tests/simplCore/should_compile/T24359a.stderr
- testsuite/tests/simplCore/should_compile/T25160.stderr
- testsuite/tests/simplCore/should_compile/T25718c.stderr-ws-32
- testsuite/tests/simplCore/should_compile/T25718c.stderr-ws-64
- testsuite/tests/simplCore/should_compile/T26051.stderr
- testsuite/tests/simplCore/should_compile/T26116.stderr
- testsuite/tests/simplCore/should_compile/T8331.stderr
- testsuite/tests/simplCore/should_compile/T8848a.stderr
- testsuite/tests/simplCore/should_compile/spec004.stderr
- testsuite/tests/typecheck/should_compile/T13032.stderr
- + testsuite/tests/typecheck/should_compile/T23135.hs
- testsuite/tests/typecheck/should_compile/all.T
The diff was not included because it is too large.
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/296562c80a72152121dc0f4ad28bd9…
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/296562c80a72152121dc0f4ad28bd9…
You're receiving this email because of your account on gitlab.haskell.org.
1
0