[Git][ghc/ghc][wip/fendor/dep-stdout] Add support for streaming `ghc -M` to stdout
by Hannes Siebenhandl (@fendor) 21 Oct '26
by Hannes Siebenhandl (@fendor) 21 Oct '26
21 Oct '26
Hannes Siebenhandl pushed to branch wip/fendor/dep-stdout at Glasgow Haskell Compiler / GHC
Commits:
4ca57979 by fendor at 2026-10-21T10:19:59+02:00
Add support for streaming `ghc -M` to stdout
Adds the flags `-dep-json-stdout` and `-dep-makefile-stdout` which
streams dependency information line-by-line to stdout.
Each dependency generation can be redirected to separate outputs.
Providing any extra `-dep-{json,makefile}[-stdout]` disables the default
generation of the `Makefile` file.
Allows consumers to start processing the dependency information as it
comes in and not waiting for the dependency generation to finish.
`DepStream` abstracts over the possible output handles, which is either
a file or the stdout handle.
Writing to a file requires temporary file paths, thus `DepStream` must
provide a way to perform clean up when dependency processing finished or
when an error is thrown for any reason.
We further decouple the rendering and writing of the result from the
dependency generation, localising side effects and making the code
slightly more readable.
The testsuite now also handles `ghc -M` writing to stdout.
We add a test for each combination of writing to stdout and file,
except for writing both makefile and json.
While it probably works, is really undefined behaviour.
Closes #27937
- - - - -
15 changed files:
- + changelog.d/make-depends-streaming-output
- compiler/GHC/Driver/DynFlags.hs
- compiler/GHC/Driver/MakeFile.hs
- compiler/GHC/Driver/Session.hs
- docs/users_guide/separate_compilation.rst
- testsuite/driver/my_typing.py
- testsuite/driver/testglobals.py
- testsuite/driver/testlib.py
- testsuite/tests/driver/mkdepend/all.T
- testsuite/tests/driver/mkdepend/mkdepend_mhu01.dep-json → testsuite/tests/driver/mkdepend/mkdepend_mhu01a.dep-json
- testsuite/tests/driver/mkdepend/mkdepend_mhu01.dep-makefile → testsuite/tests/driver/mkdepend/mkdepend_mhu01a.dep-makefile
- + testsuite/tests/driver/mkdepend/mkdepend_mhu01b.stderr
- + testsuite/tests/driver/mkdepend/mkdepend_mhu01c.stderr
- + testsuite/tests/driver/mkdepend/mkdepend_mhu01d.dep-makefile
- + testsuite/tests/driver/mkdepend/mkdepend_mhu01d.stderr
Changes:
=====================================
changelog.d/make-depends-streaming-output
=====================================
@@ -0,0 +1,16 @@
+section: compiler
+synopsis: Allow streaming `ghc -M` output to stdout
+issues: #27937
+mrs: !16830
+description: {
+ Add options `-dep-makefile-stdout` and `-dep-json-stdout` which stream the dependency generation
+ output to stdout.
+
+ The Makefile and json dependency generation can stream to separate targets, e.g. `-dep-makefile target.Makefile -dep-json-stdout`
+ streams the JSON Lines output to stdout and the Makefile to `target.Makefile`.
+
+ If `-dep-makefile-stdout`, `-dep-json` or `-dep-json-stdout` is given, we no longer generate a Makefile by default, but only
+ if the user explicitly requests the Makefile dependency generation.
+
+ The output from a particular backend can be disabled via `-no-dep-json` and `-no-dep-makefile` respectively.
+}
=====================================
compiler/GHC/Driver/DynFlags.hs
=====================================
@@ -73,6 +73,9 @@ module GHC.Driver.DynFlags (
IncludeSpecs(..), addGlobalInclude, addQuoteInclude, flattenIncludes,
addImplicitQuoteInclude,
+ -- * `ghc -M` specification
+ MkDependOutput(..),
+
-- * SDoc
initSDocContext, initDefaultSDocContext,
initPromotionTickContext,
@@ -350,8 +353,8 @@ data DynFlags = DynFlags {
-- ^ External plugins loaded from shared libraries
-- For ghc -M
- depMakefile :: Maybe FilePath,
- depJson :: Maybe FilePath,
+ depMakefile :: MkDependOutput,
+ depJson :: MkDependOutput,
depIncludePkgDeps :: Bool,
depIncludeCppDeps :: Bool,
depExcludeMods :: [ModuleName],
@@ -694,8 +697,8 @@ defaultDynFlags mySettings =
llvmOptLevel = 0,
-- ghc -M values
- depMakefile = Just "Makefile",
- depJson = Nothing,
+ depMakefile = MkDepend_Default "Makefile",
+ depJson = MkDepend_None,
depIncludePkgDeps = False,
depIncludeCppDeps = False,
depExcludeMods = [],
@@ -801,6 +804,15 @@ flattenExtensionFlags ml = foldr g defaultExtensionFlags
g (Off f) flags = EnumSet.delete f flags
defaultExtensionFlags = EnumSet.fromList (languageExtensions ml)
+-- -----------------------------------------------------------------------------
+-- -M -dep-*
+
+data MkDependOutput
+ = MkDepend_None
+ | MkDepend_Default !FilePath
+ | MkDepend_Explicit !FilePath
+ | MkDepend_StdOut
+
-- -----------------------------------------------------------------------------
-- -jN
=====================================
compiler/GHC/Driver/MakeFile.hs
=====================================
@@ -132,7 +132,7 @@ doMkDepend = do
module_graph <- GHC.depanal excl_mods True {- Allow dup roots -}
doMkDependModuleGraph dflags module_graph
-doMkDependModuleGraph :: GhcMonad m => DynFlags -> ModuleGraph -> m ()
+doMkDependModuleGraph :: GhcMonad m => DynFlags -> ModuleGraph -> m ()
doMkDependModuleGraph dflags module_graph = do
hsc_env <- getSession
logger <- getLogger
@@ -143,12 +143,14 @@ doMkDependModuleGraph dflags module_graph = do
root <- liftIO Directory.getCurrentDirectory
let
+ suffixes = fmap unsafeEncodeUtf (depSuffixes dflags)
+ tmpdir = tmpDir dflags
backends = concat
- [ [ initFileDepWriter logger root tmpfs dflags makefile_output
- | Just makefile_output <- [depMakefile dflags]
+ [ [ initFileDepWriter logger root tmpfs tmpdir suffixes makefile_output
+ | Just makefile_output <- [initDepOutput (depMakefile dflags)]
]
- , [ initJsonDepWriter logger tmpfs dflags json_output
- | Just json_output <- [depJson dflags]
+ , [ initJsonDepWriter logger tmpfs tmpdir json_output
+ | Just json_output <- [initDepOutput (depJson dflags)]
]
]
@@ -158,24 +160,14 @@ doMkDependModuleGraph dflags module_graph = do
liftIO $ debugTraceMsg logger 2 (text "Module dependencies" $$ ppr sorted)
liftIO $ do
-
- -- Setup the writers
- writers <- sequenceA backends
- sinks <- traverse dw_beginWriter writers
-
- -- Do the actual work
- do
+ withWriters backends $ \ depSink -> do
+ -- Do the actual work
-- Process them one by one, dumping results and complaining about cycles
- mapM_ (processDeps hsc_env (UniqSet.mkUniqSet excl_mods) sinks) sorted
+ mapM_ (processDeps hsc_env (UniqSet.mkUniqSet excl_mods) depSink) sorted
-- If -ddump-mod-cycles, show cycles in the module graph
liftIO $ dumpModCycles logger module_graph
- -- Tidy up
- do
- traverse_ dw_endWriter writers
-
-
-- Unconditional exiting is a bad idea. If an error occurs we'll get an
--exception; if that is not caught it's fine, but at least we have a
--chance to find out exactly what went wrong. Uncomment the following
@@ -216,15 +208,119 @@ data Dep
{ dep_path :: OsPath
}
+-- | Consumer of the dependency information generated from the 'ModuleGraph'.
+--
+-- Use with 'withWriter'/'withWriters' to initialise the consumers of the dependency information
+-- generation.
data DependencyWriter = DependencyWriter
- { dw_beginWriter :: IO DepSink
+ { dw_sink :: DepSink
+ -- ^ Sink for dependency information
, dw_endWriter :: IO ()
+ -- ^ We finished consuming the dependency information, perform final
+ -- tidy up operations. This may be a no-op, but could also
+ -- copy temporary files, close open handlers, etc...
+ --
+ -- If 'dw_endWriter' was called, calling 'dw_onError' is unnecessary,
+ -- but it must be safe to call 'dw_onError' any way.
+ , dw_onError :: IO ()
+ -- ^ Something crashed! Clean up any resources opened by this 'DependencyWriter'.
}
-data DepSink = DepSink
+-- | Initialise multiple backends and combine them into one output function.
+--
+-- Combines the 'DepSink' of each 'DependencyWriter' into one single consumer.
+withWriters :: [IO DependencyWriter] -> (DepSink -> IO a) -> IO a
+withWriters [] k = k noopSink
+withWriters (dw : dws) k =
+ withWriter
+ dw
+ $ \s -> withWriters dws (\s' -> k (s `mappendSink` s'))
+
+-- | Initialise a single 'DependencyWriter', making sure any allocated resource
+-- get's tidied up again.
+--
+-- Accepts 'IO' 'DependencyWriter' since the initialisation may perform setup operations,
+-- such as opening file handles, or intermediate data types.
+withWriter :: IO DependencyWriter -> (DepSink -> IO r) -> IO r
+withWriter dwIO k =
+ bracketOnError dwIO dw_onError $ \ dw -> do
+ r <- k (dw_sink dw)
+ -- Clean up open resources
+ dw_endWriter dw
+ pure r
+
+-- | Process the dependency information for one node in the 'ModuleGraph'.
+newtype DepSink = DepSink
{ ds_writeDependency :: DepNode -> [Dep] -> IO ()
}
+noopSink :: DepSink
+noopSink = DepSink $ \ _ _ -> pure ()
+
+mappendSink :: DepSink -> DepSink -> DepSink
+mappendSink (DepSink s1) (DepSink s2) = DepSink (\ node deps -> s1 node deps >> s2 node deps)
+
+-- | Where should the dependency output end up?
+data DepOutput
+ = DepStreamHandle Handle
+ -- ^ The dependency information should be written to this 'Handle'.
+ | DepFileOutput FilePath
+ -- ^ The dependency information should be written to this 'FilePath'.
+
+-- | A 'DepStream' is an abstraction over how to output single elements
+-- of the dependency generation, e.g., a json object or one makefile rule.
+data DepStream = DepStream
+ { depStream_write :: String -> IO ()
+ , depStream_close :: IO ()
+ , depStream_onError :: IO ()
+ }
+
+initDepOutput :: MkDependOutput -> Maybe DepOutput
+initDepOutput = \ case
+ MkDepend_None -> Nothing
+ MkDepend_Default fp -> Just $ DepFileOutput fp
+ MkDepend_Explicit fp -> Just $ DepFileOutput fp
+ MkDepend_StdOut -> Just $ DepStreamHandle stdout
+
+initMakefileDepStream :: Logger -> TmpFs -> TempDir -> DepOutput -> IO DepStream
+initMakefileDepStream _ _ _ (DepStreamHandle h) =
+ pure $ depStreamHandle h
+initMakefileDepStream logger tmpfs tmpdir (DepFileOutput fp) = do
+ files <- beginMkDependHS logger tmpfs tmpdir fp
+ pure $
+ DepStream
+ { depStream_write = hPutStrLn (mkd_tmp_hdl files)
+ , depStream_onError = do
+ hClose $ mkd_tmp_hdl files
+ case mkd_make_hdl files of
+ Nothing -> return ()
+ Just hdl -> hClose hdl
+ , depStream_close = endMkDependHS logger files
+ }
+
+initJsonDepStream :: Logger -> TmpFs -> TempDir -> DepOutput -> IO DepStream
+initJsonDepStream _ _ _ (DepStreamHandle h) =
+ pure $ depStreamHandle h
+initJsonDepStream logger tmpfs tmpdir (DepFileOutput fp) = do
+ files <- initJsonDepFiles logger tmpfs tmpdir fp
+ pure $
+ DepStream
+ { depStream_write = hPutStrLn (json_tmp_hdl files)
+ , depStream_onError = hClose (json_tmp_hdl files)
+ , depStream_close = endJsonDepFiles logger files
+ }
+
+depStreamHandle :: Handle -> DepStream
+depStreamHandle h =
+ DepStream
+ { depStream_write =
+ \output -> do
+ hPutStrLn h output
+ hFlush h -- We flush to ensure the output is streamed per line.
+ , depStream_close = pure ()
+ , depStream_onError = pure ()
+ }
+
-----------------------------------------------------------------
--
-- processDeps
@@ -233,7 +329,7 @@ data DepSink = DepSink
processDeps :: HscEnv
-> UniqSet ModuleName -- ^ Excludes
- -> [DepSink]
+ -> DepSink
-> SCC ModuleGraphNode
-> IO ()
-- Write suitable dependencies to handle
@@ -267,7 +363,7 @@ processDeps _ _ _ (AcyclicSCC (ModuleNode _ (ModuleNodeFixed {})))
-- No dependencies needed for fixed modules (already compiled)
= return ()
-processDeps hsc_env0 excl_mods sinks (AcyclicSCC (ModuleNode _ (ModuleNodeCompile node))) = do
+processDeps hsc_env0 excl_mods sink (AcyclicSCC (ModuleNode _ (ModuleNodeCompile node))) = do
pp <- preprocessor
let
dep_node = mkDepNode pp
@@ -279,7 +375,7 @@ processDeps hsc_env0 excl_mods sinks (AcyclicSCC (ModuleNode _ (ModuleNodeCompil
if null missing_dep_errs
then do
- traverse_ (\ sink -> ds_writeDependency sink dep_node deps) sinks
+ ds_writeDependency sink dep_node deps
else do
let sec = initSourceErrorContext (hsc_dflags local_hsc_env)
throwErrors sec (mkMessages (listToBag missing_dep_errs))
@@ -380,17 +476,16 @@ find_preprocessor d = do
--
-----------------------------------------------------------------
-initFileDepWriter :: Logger -> OsPath -> TmpFs -> DynFlags -> FilePath -> IO DependencyWriter
-initFileDepWriter logger root tmpfs dflags makefile = do
- files <- beginMkDependHS logger tmpfs dflags makefile
+initFileDepWriter :: Logger -> OsPath -> TmpFs -> TempDir -> [OsPath] -> DepOutput -> IO DependencyWriter
+initFileDepWriter logger root tmpfs tmpdir suffixes output = do
+ stream <- initMakefileDepStream logger tmpfs tmpdir output
pure DependencyWriter
- { dw_beginWriter = do
- pure $ DepSink $ \ node deps -> do
- writeDependencies root (mkd_tmp_hdl files) suffixes node deps
- , dw_endWriter = endMkDependHS logger files
+ { dw_sink =
+ DepSink $ \ node deps -> do
+ traverse_ (depStream_write stream) (dependencyRules root suffixes node deps)
+ , dw_endWriter = depStream_close stream
+ , dw_onError = depStream_onError stream
}
- where
- suffixes = map unsafeEncodeUtf (depSuffixes dflags)
data MkDepFiles
= MkDep { mkd_make_file :: FilePath, -- Name of the makefile
@@ -398,11 +493,11 @@ data MkDepFiles
mkd_tmp_file :: FilePath, -- Name of the temporary file
mkd_tmp_hdl :: Handle } -- Handle of the open temporary file
-beginMkDependHS :: Logger -> TmpFs -> DynFlags -> FilePath -> IO MkDepFiles
-beginMkDependHS logger tmpfs dflags makefile = do
+beginMkDependHS :: Logger -> TmpFs -> TempDir -> FilePath -> IO MkDepFiles
+beginMkDependHS logger tmpfs tmpdir makefile = do
-- open a new temp file in which to stuff the dependency info
-- as we go along.
- tmp_file <- newTempName logger tmpfs (tmpDir dflags) TFL_CurrentModule "dep"
+ tmp_file <- newTempName logger tmpfs tmpdir TFL_CurrentModule "dep"
tmp_hdl <- openFile tmp_file WriteMode
-- open the makefile
@@ -443,16 +538,15 @@ beginMkDependHS logger tmpfs dflags makefile = do
return (MkDep { mkd_make_file = makefile, mkd_make_hdl = mb_make_hdl,
mkd_tmp_file = tmp_file, mkd_tmp_hdl = tmp_hdl})
-writeDependencies ::
+dependencyRules ::
OsPath ->
- Handle ->
[OsString] ->
-- ^ Suffixes
DepNode ->
[Dep] ->
- IO ()
-writeDependencies root hdl suffixes node deps =
- traverse_ write tasks
+ [String]
+dependencyRules root suffixes node deps =
+ map rule tasks
where
tasks = source_dep : boot_dep ++ concatMap import_dep deps
@@ -479,7 +573,7 @@ writeDependencies root hdl suffixes node deps =
DepCpp {dep_path} -> [(obj_files, dep_path)]
- write (from, to) = writeDependency root hdl from to
+ rule (from, to) = dependencyRule root from to
obj_files = suffixed dn_obj
@@ -488,18 +582,18 @@ writeDependencies root hdl suffixes node deps =
DepNode {dn_src, dn_obj, dn_hi, dn_mod} = node
-----------------------------
-writeDependency :: OsPath -> Handle -> [OsPath] -> OsPath -> IO ()
--- (writeDependency r h [t1,t2] dep) writes to handle h the dependency
+dependencyRule :: OsPath -> [OsPath] -> OsPath -> String
+-- (dependencyRule r [t1,t2] dep) renders the dependency
-- t1 t2 : dep
-writeDependency root hdl targets dep
- = do let -- We need to avoid making deps on
- -- c:/foo/...
- -- on Windows as make gets confused by the :
- -- Making relative deps avoids some instances of this.
- dep' = OsPath.makeRelative root dep
- forOutput = escapeSpaces . reslash Forwards . unsafeDecodeUtf . OsPath.normalise
- output = unwords (map forOutput targets) ++ " : " ++ forOutput dep'
- hPutStrLn hdl output
+dependencyRule root targets dep
+ = let -- We need to avoid making deps on
+ -- c:/foo/...
+ -- on Windows as make gets confused by the :
+ -- Making relative deps avoids some instances of this.
+ dep' = OsPath.makeRelative root dep
+ forOutput = escapeSpaces . reslash Forwards . unsafeDecodeUtf . OsPath.normalise
+ in
+ unwords (map forOutput targets) ++ " : " ++ forOutput dep'
-----------------------------
insertSuffixes
@@ -554,16 +648,27 @@ endMkDependHS logger
--
-----------------------------------------------------------------
-initJsonDepWriter :: Logger -> TmpFs -> DynFlags -> FilePath -> IO DependencyWriter
-initJsonDepWriter logger tmpfs dflags output = do
- json_var <- mkJsonOutput logger tmpfs dflags output initDepJson
- pure DependencyWriter
- { dw_beginWriter =
- pure $ DepSink $ \ node deps ->
- writeHomeUnitDeps json_var node deps
- , dw_endWriter = do
- writeJsonOutput logger json_var
- }
+initJsonDepWriter :: Logger -> TmpFs -> TempDir -> DepOutput -> IO DependencyWriter
+initJsonDepWriter logger tmpfs tmpdir output = do
+ stream <- initJsonDepStream logger tmpfs tmpdir output
+ json_var <- mkJsonOutput
+ pure
+ DependencyWriter
+ { dw_sink =
+ DepSink $ \node deps -> do
+ extUnits <- readJson json_var
+ let
+ (extUnitsDeps, payload) = homeUnitDeps node deps
+ newExtUnits = Set.difference extUnitsDeps (externalDeps extUnits)
+ -- Write new external unit deps before the module that uses them
+ for_ newExtUnits $ \extUnit -> do
+ depStream_write stream (renderJsonl extUnit)
+ depStream_write stream (renderJsonl payload)
+ -- Cache external deps we already reported in the past
+ updateJson json_var $ cacheExternalDepsJson newExtUnits
+ , dw_endWriter = depStream_close stream
+ , dw_onError = depStream_onError stream
+ }
--------------------------------------------------------------------------------
-- Output interface for json dumps
@@ -572,65 +677,60 @@ initJsonDepWriter logger tmpfs dflags output = do
-- The flag @-dep-json@ add an additional output target for dependency
-- diagnostics.
--
--- Contains temporary file locations to stream output to.
-data JsonOutput =
- JsonOutput {
- -- | This ref is updated in @processDeps@ incrementally, using a
- -- flag-specific type.
- json_ref :: IORef DepJson,
-
- -- | The output file path specified as argument to the flag.
- json_path :: FilePath,
- -- | Name of the temporary file
- json_tmp_file :: FilePath,
- -- | Handle of the open temporary file
- json_tmp_hdl :: Handle
+data JsonOutput
+ = JsonOutput
+ { json_ref :: IORef DepJson
+ -- ^ This ref is updated in @processDeps@ incrementally, using a
+ -- flag-specific type.
}
--- | Set up json generation for `ghc -M`.
---
--- Creates a temporary file that we write the dependency information to.
--- The temporary file is written to the final output file path in 'writeJsonOutput'.
-mkJsonOutput ::
- Logger ->
- TmpFs ->
- DynFlags ->
- FilePath ->
- IO (IORef DepJson) ->
- IO JsonOutput
-mkJsonOutput logger tmpfs dflags json_path mk_ref = do
- json_ref <- mk_ref
- -- open a new temp file in which to stuff the dependency info
+data JsonDepFiles = JsonDepFiles
+ { json_path :: FilePath
+ -- ^ The output file path specified as argument to the flag.
+ , json_tmp_file :: FilePath
+ -- ^ Name of the temporary file
+ , json_tmp_hdl :: Handle
+ -- ^ Handle of the open temporary file
+ }
+
+initJsonDepFiles :: Logger -> TmpFs -> TempDir -> FilePath -> IO JsonDepFiles
+initJsonDepFiles logger tmpfs tmpdir path = do
+ -- open a new temp file in which to stuff the dependency info into
-- as we go along.
- json_tmp_file <- newTempName logger tmpfs (tmpDir dflags) TFL_CurrentModule "dep"
+ json_tmp_file <- newTempName logger tmpfs tmpdir TFL_CurrentModule "dep-json"
json_tmp_hdl <- openFile json_tmp_file WriteMode
-
- pure JsonOutput {json_ref, json_path, json_tmp_file, json_tmp_hdl}
-
-updateJson :: JsonOutput -> (DepJson -> DepJson) -> IO ()
-updateJson JsonOutput {json_ref} f = modifyIORef' json_ref f
-
-readJson :: JsonOutput -> IO DepJson
-readJson JsonOutput {json_ref} = readIORef json_ref
+ pure $ JsonDepFiles
+ { json_path = path
+ , json_tmp_file = json_tmp_file
+ , json_tmp_hdl = json_tmp_hdl
+ }
-- | Finalise the json generation by copying the temporary file to the final destination.
-- Avoids partial writes in the case of an error.
-writeJsonOutput ::
- Logger ->
- JsonOutput ->
- IO ()
-writeJsonOutput logger JsonOutput {json_path, json_tmp_file, json_tmp_hdl} = do
+endJsonDepFiles :: Logger -> JsonDepFiles -> IO ()
+endJsonDepFiles logger JsonDepFiles {json_path, json_tmp_file, json_tmp_hdl} = do
hClose json_tmp_hdl
-- Copy the new jsonl in place
showPass logger "Installing new json output"
SysTools.copyFile json_tmp_file json_path
+-- | Set up json generation for `ghc -M`.
+mkJsonOutput :: IO JsonOutput
+mkJsonOutput =
+ JsonOutput <$> newIORef (DepJson Set.empty)
+
+updateJson :: JsonOutput -> (DepJson -> DepJson) -> IO ()
+updateJson JsonOutput {json_ref} f = modifyIORef' json_ref f
+
+readJson :: JsonOutput -> IO DepJson
+readJson JsonOutput {json_ref} = readIORef json_ref
+
--------------------------------------------------------------------------------
-- Output helpers
-writeJsonl :: ToJson a => JsonOutput -> a -> IO ()
-writeJsonl JsonOutput {json_tmp_hdl} output = do
- hPutStrLn json_tmp_hdl (showSDocUnsafe $ renderJSON $ json output)
+renderJsonl :: ToJson a => a -> String
+renderJsonl output =
+ showSDocUnsafe $ renderJSON $ json output
--------------------------------------------------------------------------------
-- Payload for -dep-json
@@ -788,9 +888,6 @@ instance ToJson ExtUnitDep where
where
showPackageId (DPackageId (PackageId pid)) = unpackFS pid
-initDepJson :: IO (IORef DepJson)
-initDepJson = newIORef $ DepJson Set.empty
-
cacheExternalDepsJson :: Set.Set ExtUnitDep -> DepJson -> DepJson
cacheExternalDepsJson extUnits (DepJson e0) =
DepJson
@@ -798,19 +895,10 @@ cacheExternalDepsJson extUnits (DepJson e0) =
Set.union e0 extUnits
}
-writeHomeUnitDeps :: JsonOutput -> DepNode -> [Dep] -> IO ()
-writeHomeUnitDeps deps_json DepNode {..} deps = do
- extUnits <- readJson deps_json
- let newExtUnits = Set.difference extUnitsDeps (externalDeps extUnits)
- -- Write new external unit deps before the module that uses them
- for_ newExtUnits $ \ extUnit -> do
- writeJsonl deps_json extUnit
- writeJsonl deps_json payload
- -- Cache external deps we already reported in the past
- updateJson deps_json $ cacheExternalDepsJson newExtUnits
+homeUnitDeps :: Foldable t => DepNode -> t Dep -> (Set.Set ExtUnitDep, HomeUnitDep)
+homeUnitDeps DepNode {..} deps =
+ foldl' go (Set.empty, initial_node_data) deps
where
- (extUnitsDeps, payload) = foldl' go (Set.empty, initial_node_data) deps
-
initial_node_data =
HomeUnitDep
{ unitId = dn_unitId
=====================================
compiler/GHC/Driver/Session.hs
=====================================
@@ -790,16 +790,44 @@ addOptCmmP f = alterToolSettings $ \s -> s
}
setDepMakefile :: FilePath -> DynFlags -> DynFlags
-setDepMakefile f d = d { depMakefile = Just f }
+setDepMakefile f d = d { depMakefile = MkDepend_Explicit f
+ -- depJson has no default value, we add this for symmetry with `setDepJson`
+ -- and if anything changes in the future, then the code is already here :)
+ , depJson = unsetMkDependOutputIfDefault (depJson d)
+ }
+
+setDepMakefileStdout :: DynFlags -> DynFlags
+setDepMakefileStdout d = d { depMakefile = MkDepend_StdOut
+ -- depJson has no default value, we add this for symmetry with `setDepJson`
+ -- and if anything changes in the future, then the code is already here :)
+ , depJson = unsetMkDependOutputIfDefault (depJson d)
+ }
unsetDepMakefile :: DynFlags -> DynFlags
-unsetDepMakefile d = d { depMakefile = Nothing }
+unsetDepMakefile d = d { depMakefile = MkDepend_None }
setDepJson :: FilePath -> DynFlags -> DynFlags
-setDepJson f d = d { depJson = Just f }
+setDepJson f d = d { depJson = MkDepend_Explicit f
+ -- We generate a Makefile by default, but if a user requests json
+ -- output, then they most likely don't need a Makefile.
+ -- Thus, we only generate it if the user requests it.
+ , depMakefile = unsetMkDependOutputIfDefault (depMakefile d)
+ }
+
+setDepJsonStdout :: DynFlags -> DynFlags
+setDepJsonStdout d = d { depJson = MkDepend_StdOut
+ -- We generate a Makefile by default, but if a user requests json
+ -- output, then they most likely don't need a Makefile.
+ -- Thus, we only generate it if the user requests it.
+ , depMakefile = unsetMkDependOutputIfDefault (depMakefile d)
+ }
unsetDepJson :: DynFlags -> DynFlags
-unsetDepJson d = d { depJson = Nothing }
+unsetDepJson d = d { depJson = MkDepend_None }
+
+unsetMkDependOutputIfDefault :: MkDependOutput -> MkDependOutput
+unsetMkDependOutputIfDefault (MkDepend_Default _) = MkDepend_None
+unsetMkDependOutputIfDefault mkDepend_output = mkDepend_output
setDepIncludeCppDeps :: Bool -> DynFlags -> DynFlags
setDepIncludeCppDeps b d = d { depIncludeCppDeps = b }
@@ -1272,8 +1300,10 @@ dynamic_flags_deps = [
, make_ord_flag defGhcFlag "dep-suffix" (hasArg addDepSuffix)
, make_ord_flag defGhcFlag "dep-makefile" (hasArg setDepMakefile)
, make_ord_flag defGhcFlag "no-dep-makefile" (noArg unsetDepMakefile)
+ , make_ord_flag defGhcFlag "dep-makefile-stdout" (noArg setDepMakefileStdout)
, make_ord_flag defGhcFlag "dep-json" (hasArg setDepJson)
, make_ord_flag defGhcFlag "no-dep-json" (noArg unsetDepJson)
+ , make_ord_flag defGhcFlag "dep-json-stdout" (noArg setDepJsonStdout)
, make_ord_flag defGhcFlag "include-cpp-deps"
(noArg (setDepIncludeCppDeps True))
, make_ord_flag defGhcFlag "include-pkg-deps"
=====================================
docs/users_guide/separate_compilation.rst
=====================================
@@ -1511,6 +1511,23 @@ generation are:
Introduced in GHC 10.1, ``-no-dep-makefile`` disables the generation of a Makefile altogether.
+.. ghc-flag:: -dep-makefile-stdout
+ :shortdesc: Write dependency generation to stdout using makefile format
+ :type: dynamic
+ :reverse: -no-dep-makefile
+ :category: redirect-output
+
+ :since: 10.1
+
+ Stream the dependency generation of :ghc-flag:`-dep-makefile ⟨file⟩` to stdout instead of ⟨file⟩.
+
+ Contrary to :ghc-flag:`-dep-makefile ⟨file⟩`, the markers
+
+ * "``# DO NOT DELETE: Beginning of Haskell dependencies``" and
+ * "``# DO NOT DELETE: End of Haskell dependencies``"
+
+ are not emitted.
+
.. ghc-flag:: -dep-json ⟨file⟩
:shortdesc: Write dependencies to ⟨file⟩ as a JSON Lines format
:type: dynamic
@@ -1540,8 +1557,19 @@ generation are:
``-no-dep-json`` disables the generation of the JSON file.
- By default, :ghc-flag:`-dep-makefile ⟨file⟩` is enabled even if :ghc-flag:`-dep-json ⟨file⟩` is given.
- Disable the generation of any Makefile via ``-no-dep-makefile``.
+ :ghc-flag:`-dep-json ⟨file⟩` and :ghc-flag:`-dep-json-stdout` disable the default generation of a Makefile.
+ To generate a Makefile for the dependency information, explicitly use :ghc-flag:`-dep-makefile ⟨file⟩` or
+ :ghc-flag:`-dep-makefile-stdout`.
+
+.. ghc-flag:: -dep-json-stdout
+ :shortdesc: Write dependency generation to stdout using JSON Lines format
+ :type: dynamic
+ :reverse: -no-dep-json
+ :category: redirect-output
+
+ :since: 10.1
+
+ Stream the dependency generation of :ghc-flag:`-dep-json ⟨file⟩` to stdout instead of ⟨file⟩.
.. ghc-flag:: -dep-suffix ⟨suffix⟩
:shortdesc: Make dependencies that declare that files with suffix
=====================================
testsuite/driver/my_typing.py
=====================================
@@ -10,7 +10,7 @@ The testsuite driver can be typechecked using mypy [1].
from typing import *
import typing
-
+import enum
####################################################
# Backwards compatibility shims
@@ -34,3 +34,7 @@ GitHash = NewType("GitHash", str) # a Git commit hash
GitRef = NewType("GitRef", str)
TestEnv = NewType("TestEnv", str)
MetricName = NewType("MetricName", str)
+
+# Used by `ghc -M` test mode
+DepMode = enum.Enum('DepMode', [('NONE', 0), ('FILE', 1), ('STDOUT', 2)])
+
=====================================
testsuite/driver/testglobals.py
=====================================
@@ -390,8 +390,8 @@ class TestOptions:
self.ignore_extension = False
# make_depend tests: which outputs to write and check
- self.dep_makefile = False
- self.dep_json = False
+ self.dep_makefile: DepMode = DepMode.NONE
+ self.dep_json: DepMode = DepMode.NONE
self.ignore_dep_outputs = False
# Backpack test
=====================================
testsuite/driver/testlib.py
=====================================
@@ -438,14 +438,26 @@ def dep_makefile( name, opts ):
For make_depend tests: write the -dep-makefile output and
compare it against <name>.dep-makefile.
"""
- opts.dep_makefile = True
+ opts.dep_makefile = DepMode.FILE
def dep_json( name, opts ):
"""
For make_depend tests: write the -dep-json output and
compare it against <name>.dep-json.
"""
- opts.dep_json = True
+ opts.dep_json = DepMode.FILE
+
+def dep_makefile_stdout( name, opts ):
+ """
+ For make_depend tests: add -dep-makefile-stdout flag.
+ """
+ opts.dep_makefile = DepMode.STDOUT
+
+def dep_json_stdout( name, opts ):
+ """
+ For make_depend tests: add -dep-json-stdout flag.
+ """
+ opts.dep_json = DepMode.STDOUT
def ignore_dep_outputs( name, opts ):
"""
@@ -2052,6 +2064,10 @@ async def do_make_depend(name: TestName,
against <name>.dep-makefile. Otherwise, -no-dep-makefile is passed.
* 'dep_json': write the -dep-json output and compare it
against <name>.dep-json.
+ * 'dep_json_stdout': write the -dep-json-stdout and compare it with
+ <name>.stderr
+ * 'dep_makefile_stdout': write the -dep-makefile-stdout and compare it with
+ <name>.stderr
"""
opts = getTestOpts()
@@ -2060,14 +2076,29 @@ async def do_make_depend(name: TestName,
outputs = []
output_flags = []
- if opts.dep_makefile:
+ if opts.dep_makefile == DepMode.FILE:
outputs.append(('dep-makefile', normalise_dep_makefile))
output_flags.append('-dep-makefile %s' % actual_file('dep-makefile'))
- else:
- output_flags.append('-no-dep-makefile')
- if opts.dep_json:
+ elif opts.dep_makefile == DepMode.STDOUT:
+ outputs.append(('stdout', normalise_dep_makefile))
+ output_flags.append('-dep-makefile-stdout')
+
+ if opts.dep_json == DepMode.FILE:
outputs.append(('dep-json', normalise_dep_json))
output_flags.append('-dep-json %s' % actual_file('dep-json'))
+ elif opts.dep_json == DepMode.STDOUT:
+ outputs.append(('stdout', normalise_dep_json))
+ output_flags.append('-dep-json-stdout')
+
+ # Somewhat non-sensical here, but `do_compile` outputs stdout and stderr into the same.
+ # Then, it only applies the `normalise_errmsg_fun` to the output, and compares it against
+ # <name>.stderr as the golden file.
+ #
+ # This is not really what we want in this situation, but it seems this requires more refactoring
+ # of the `do_compile` function than what fendor is ready to commit to.
+ # Thus, as a workaround, we simply add the normalisers for the stderr...
+ stdout_normalisers = [ n for (k, n) in outputs if k == 'stdout' ]
+ opts.extra_errmsg_normaliser = join_normalisers(opts.extra_errmsg_normaliser, normalise_slashes_, *stdout_normalisers)
result = await do_compile( name, way, False, None, [], units,
' '.join(output_flags + [extra_hc_opts]),
@@ -2077,9 +2108,9 @@ async def do_make_depend(name: TestName,
for (kind, normaliser) in outputs:
output_match = await compare_outputs(way, kind,
- join_normalisers(opts.extra_normaliser,
- normalise_slashes_, normaliser),
- find_expected_file(name, kind, way), actual_file(kind))
+ join_normalisers(opts.extra_normaliser, normalise_slashes_, normaliser),
+ find_expected_file(name, kind, way),
+ actual_file(kind))
if not output_match:
return failBecause(kind + ' mismatch', diff=output_match.diff)
=====================================
testsuite/tests/driver/mkdepend/all.T
=====================================
@@ -15,9 +15,20 @@ test('T24384b', [ *T24384_extras, when(opsys('mingw32'), skip), *json_normaliser
test('T24384c', [ *T24384_extras, when(opsys('mingw32'), skip), *json_normalisers ], makefile_test, [])
# Multiple home units with explicit level imports.
-test('mkdepend_mhu01',
+test('mkdepend_mhu01a',
[ extra_files(['mhu01/']), dep_makefile, dep_json ],
make_depend_multiunit, [['mhu01/unitA', 'mhu01/unitB', 'mhu01/unitC'], ''])
+# test the output streaming and writing to files
+test('mkdepend_mhu01b',
+ [ extra_files(['mhu01/']), dep_json_stdout ],
+ make_depend_multiunit, [['mhu01/unitA', 'mhu01/unitB', 'mhu01/unitC'], ''])
+test('mkdepend_mhu01c',
+ [ extra_files(['mhu01/']), dep_makefile_stdout ],
+ make_depend_multiunit, [['mhu01/unitA', 'mhu01/unitB', 'mhu01/unitC'], ''])
+test('mkdepend_mhu01d',
+ [ extra_files(['mhu01/']), dep_json_stdout, dep_makefile ],
+ make_depend_multiunit, [['mhu01/unitA', 'mhu01/unitB', 'mhu01/unitC'], ''])
+
# Preprocessor must not be inherited by default active home unit
test('mkdepend_mhu02',
[ extra_files(['mhu02/']), dep_json
=====================================
testsuite/tests/driver/mkdepend/mkdepend_mhu01.dep-json → testsuite/tests/driver/mkdepend/mkdepend_mhu01a.dep-json
=====================================
=====================================
testsuite/tests/driver/mkdepend/mkdepend_mhu01.dep-makefile → testsuite/tests/driver/mkdepend/mkdepend_mhu01a.dep-makefile
=====================================
@@ -1,23 +1,23 @@
# DO NOT DELETE: Beginning of Haskell dependencies
+mhu01/a/A/Base.o-boot : mhu01/a/A/Base.hs-boot
mhu01/a/A/Base.o : mhu01/a/A/Base.hi-boot
+mhu01/a/Util.o : mhu01/a/Util.hs
+mhu01/a/Util.o : mhu01/a/A/Base.hi-boot
mhu01/a/A/Base.o : mhu01/a/A/Base.hs
mhu01/a/A/Base.o : mhu01/a/Util.hi
-mhu01/a/A/Base.o-boot : mhu01/a/A/Base.hs-boot
-mhu01/a/A/TH.o : mhu01/a/A/Base.hi
mhu01/a/A/TH.o : mhu01/a/A/TH.hs
-mhu01/a/Util.o : mhu01/a/A/Base.hi-boot
-mhu01/a/Util.o : mhu01/a/Util.hs
-mhu01/b/B/Normal.o : mhu01/a/A/Base.hi
-mhu01/b/B/Normal.o : mhu01/b/B/Normal.hs
-mhu01/b/B/Normal.o : mhu01/b/B/Splice.hi
-mhu01/b/B/Splice.o : mhu01/a/A/Base.hi
-mhu01/b/B/Splice.o : mhu01/a/A/TH.hi
+mhu01/a/A/TH.o : mhu01/a/A/Base.hi
+mhu01/b/Util.o : mhu01/b/Util.hs
mhu01/b/B/Splice.o : mhu01/b/B/Splice.hs
mhu01/b/B/Splice.o : mhu01/b/Util.hi
-mhu01/b/Util.o : mhu01/b/Util.hs
-mhu01/c/C.o : mhu01/a/Util.hi
-mhu01/c/C.o : mhu01/b/B/Normal.hi
+mhu01/b/B/Splice.o : mhu01/a/A/Base.hi
+mhu01/b/B/Splice.o : mhu01/a/A/TH.hi
+mhu01/b/B/Normal.o : mhu01/b/B/Normal.hs
+mhu01/b/B/Normal.o : mhu01/b/B/Splice.hi
+mhu01/b/B/Normal.o : mhu01/a/A/Base.hi
+mhu01/c/C.o : mhu01/c/C.hs
mhu01/c/C.o : mhu01/b/B/Splice.hi
+mhu01/c/C.o : mhu01/b/B/Normal.hi
mhu01/c/C.o : mhu01/b/Util.hi
-mhu01/c/C.o : mhu01/c/C.hs
+mhu01/c/C.o : mhu01/a/Util.hi
# DO NOT DELETE: End of Haskell dependencies
=====================================
testsuite/tests/driver/mkdepend/mkdepend_mhu01b.stderr
=====================================
@@ -0,0 +1,8 @@
+{"module":"A.Base","unit-id":"a","boot":true,"imports":{},"package-name":"a","source":"mhu01/a/A/Base.hs-boot"}
+{"module":"Util","unit-id":"a","imports":{"a":[{"module":"A.Base","boot":true}]},"package-name":"a","source":"mhu01/a/Util.hs"}
+{"module":"A.Base","unit-id":"a","imports":{"a":["Util"]},"package-name":"a","source":"mhu01/a/A/Base.hs"}
+{"module":"A.TH","unit-id":"a","imports":{"a":["A.Base"]},"package-name":"a","source":"mhu01/a/A/TH.hs"}
+{"module":"Util","unit-id":"b","imports":{},"package-name":"b","source":"mhu01/b/Util.hs"}
+{"module":"B.Splice","unit-id":"b","imports":{"a":[{"module":"A.Base","level":"quote"},{"module":"A.TH","level":"splice"}],"b":["Util"]},"package-name":"b","source":"mhu01/b/B/Splice.hs"}
+{"module":"B.Normal","unit-id":"b","imports":{"a":["A.Base"],"b":["B.Splice"]},"package-name":"b","source":"mhu01/b/B/Normal.hs"}
+{"module":"C","unit-id":"c","imports":{"a":[{"module":"Util","level":"splice"}],"b":["B.Normal",{"module":"B.Splice","level":"splice"},{"module":"Util","level":"quote"}]},"package-name":"c","source":"mhu01/c/C.hs"}
=====================================
testsuite/tests/driver/mkdepend/mkdepend_mhu01c.stderr
=====================================
@@ -0,0 +1,21 @@
+mhu01/a/A/Base.o-boot : mhu01/a/A/Base.hs-boot
+mhu01/a/A/Base.o : mhu01/a/A/Base.hi-boot
+mhu01/a/Util.o : mhu01/a/Util.hs
+mhu01/a/Util.o : mhu01/a/A/Base.hi-boot
+mhu01/a/A/Base.o : mhu01/a/A/Base.hs
+mhu01/a/A/Base.o : mhu01/a/Util.hi
+mhu01/a/A/TH.o : mhu01/a/A/TH.hs
+mhu01/a/A/TH.o : mhu01/a/A/Base.hi
+mhu01/b/Util.o : mhu01/b/Util.hs
+mhu01/b/B/Splice.o : mhu01/b/B/Splice.hs
+mhu01/b/B/Splice.o : mhu01/b/Util.hi
+mhu01/b/B/Splice.o : mhu01/a/A/Base.hi
+mhu01/b/B/Splice.o : mhu01/a/A/TH.hi
+mhu01/b/B/Normal.o : mhu01/b/B/Normal.hs
+mhu01/b/B/Normal.o : mhu01/b/B/Splice.hi
+mhu01/b/B/Normal.o : mhu01/a/A/Base.hi
+mhu01/c/C.o : mhu01/c/C.hs
+mhu01/c/C.o : mhu01/b/B/Splice.hi
+mhu01/c/C.o : mhu01/b/B/Normal.hi
+mhu01/c/C.o : mhu01/b/Util.hi
+mhu01/c/C.o : mhu01/a/Util.hi
=====================================
testsuite/tests/driver/mkdepend/mkdepend_mhu01d.dep-makefile
=====================================
@@ -0,0 +1,23 @@
+# DO NOT DELETE: Beginning of Haskell dependencies
+mhu01/a/A/Base.o-boot : mhu01/a/A/Base.hs-boot
+mhu01/a/A/Base.o : mhu01/a/A/Base.hi-boot
+mhu01/a/Util.o : mhu01/a/Util.hs
+mhu01/a/Util.o : mhu01/a/A/Base.hi-boot
+mhu01/a/A/Base.o : mhu01/a/A/Base.hs
+mhu01/a/A/Base.o : mhu01/a/Util.hi
+mhu01/a/A/TH.o : mhu01/a/A/TH.hs
+mhu01/a/A/TH.o : mhu01/a/A/Base.hi
+mhu01/b/Util.o : mhu01/b/Util.hs
+mhu01/b/B/Splice.o : mhu01/b/B/Splice.hs
+mhu01/b/B/Splice.o : mhu01/b/Util.hi
+mhu01/b/B/Splice.o : mhu01/a/A/Base.hi
+mhu01/b/B/Splice.o : mhu01/a/A/TH.hi
+mhu01/b/B/Normal.o : mhu01/b/B/Normal.hs
+mhu01/b/B/Normal.o : mhu01/b/B/Splice.hi
+mhu01/b/B/Normal.o : mhu01/a/A/Base.hi
+mhu01/c/C.o : mhu01/c/C.hs
+mhu01/c/C.o : mhu01/b/B/Splice.hi
+mhu01/c/C.o : mhu01/b/B/Normal.hi
+mhu01/c/C.o : mhu01/b/Util.hi
+mhu01/c/C.o : mhu01/a/Util.hi
+# DO NOT DELETE: End of Haskell dependencies
=====================================
testsuite/tests/driver/mkdepend/mkdepend_mhu01d.stderr
=====================================
@@ -0,0 +1,8 @@
+{"module":"A.Base","unit-id":"a","boot":true,"imports":{},"package-name":"a","source":"mhu01/a/A/Base.hs-boot"}
+{"module":"Util","unit-id":"a","imports":{"a":[{"module":"A.Base","boot":true}]},"package-name":"a","source":"mhu01/a/Util.hs"}
+{"module":"A.Base","unit-id":"a","imports":{"a":["Util"]},"package-name":"a","source":"mhu01/a/A/Base.hs"}
+{"module":"A.TH","unit-id":"a","imports":{"a":["A.Base"]},"package-name":"a","source":"mhu01/a/A/TH.hs"}
+{"module":"Util","unit-id":"b","imports":{},"package-name":"b","source":"mhu01/b/Util.hs"}
+{"module":"B.Splice","unit-id":"b","imports":{"a":[{"module":"A.Base","level":"quote"},{"module":"A.TH","level":"splice"}],"b":["Util"]},"package-name":"b","source":"mhu01/b/B/Splice.hs"}
+{"module":"B.Normal","unit-id":"b","imports":{"a":["A.Base"],"b":["B.Splice"]},"package-name":"b","source":"mhu01/b/B/Normal.hs"}
+{"module":"C","unit-id":"c","imports":{"a":[{"module":"Util","level":"splice"}],"b":["B.Normal",{"module":"B.Splice","level":"splice"},{"module":"Util","level":"quote"}]},"package-name":"c","source":"mhu01/c/C.hs"}
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/4ca57979813af37a4f08c53c15bf655…
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/4ca57979813af37a4f08c53c15bf655…
You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help
1
0
[Git][ghc/ghc][wip/jeltsch/smaller-pit] 27 commits: Remove the Eq and Ord instances for ForAllTyFlag
by Wolfgang Jeltsch (@jeltsch) 20 Oct '26
by Wolfgang Jeltsch (@jeltsch) 20 Oct '26
20 Oct '26
Wolfgang Jeltsch pushed to branch wip/jeltsch/smaller-pit at Glasgow Haskell Compiler / GHC
Commits:
d07a5176 by Simon Peyton Jones at 2026-10-13T18:10:39-04:00
Remove the Eq and Ord instances for ForAllTyFlag
In Note [Comparing visibility] in GHC.Core.TyCo.Compare we read that
"we always compare ForAllTyFlags with `eqForAllVis` or `cmpForAllVis`"
But alas there was an Eq and Ord instance for ForAllTyFlag, which made it
impossible to check this claim. So I made these changes:
* Move the definition of ForAllTyFlag and its related functions from
L.H.S.Specificity (where it did not belong, since L.H.S didn't use it)
into GHC.Types.Var.
* Remove the Eq and Ord instances for ForAllTyFlag
* Deal with the fallout, by using `eqForAllVis` or related functions.
* In interface files we use IfaceType as a key for a de-deduplication map
See Note [Deduplication during iface binary serialisation]
in GHC.Iface.Binary. For this we need /exact/ comparison, so we use a
newtype wrapper for IfaceForAllTyFlag, and give use `cmpForAllTyFlagExactly`
to implement its Ord instance. Tiresome but easy.
* Rename `toIfaceForAllBndrs` to `toIfaceForAllTyBndrs`
- - - - -
ea5c054d by Simon Peyton Jones at 2026-10-13T18:10:39-04:00
Refactor type and coercion substitution
This MR refactors type and coercion substitution so that they use
TyCoMapper, rather than hand-rolled code. Fundamentally it's a simple
idea, but I had to do quite a bit of refactoring to make it fly.
I got Claude to help me this this refactoring, but I paid careful
attention to every line of code.
Main changes:
* New module GHC.Core.TyCo.Make which defines TyCoMapper, and a bunch of other
simple "smart constructors" for types and coercions. See
GHC.Core.TyCo.Rep Note [The Type-related module hierarchy], esp (TRM1)
* The new TyCoMapper has new fields `tcm_tcapp_ty` and `tcm_tcapp_co` to handle
type-constructor applications. These replace `tcm_tycon`.
* Rewrite GHC.Core.TyCo.Subst to use TyCoMapper
* Rewrite GHC.Core.Type.expandTypeSynonyms to use TyCoMapper.
* Implement `seqType` and `seqCo` with TyCoFolder. To to this I had to add
a `tcf_lit` field to TyCoFolder.
* A new module GHC.Utils.StrictIdentity to define a strict identity monad,
which itself is used for
substTyCoMapper :: TyCoMapper Subst StrictIdentity
* I removed an apparently-unused instance for Binary Levity
Refactoring `seq` functions
* In GHC.Core.Seq, redefine `seqOneShot`, `seqRuleInfo`, and friends to return
`SeqResult`, a new type defined in GHC.Types.Baseic, rather than `()`. We
can define a Monoid instance for `SeqResult` that does the right thing.
* This has knock-on effects across the compiler.
- - - - -
34165e23 by Simon Peyton Jones at 2026-10-13T18:10:39-04:00
Refactor TyCoMapper to take a single monad; add StrictReaderT
TyCoMapper used to be parameterised over both an environment and a
monad, with fields like
tcm_tyvar :: env -> TyVar -> m Type
Now it is parameterised over just the monad:
tcm_tyvar :: TyVar -> m Type
If the mapping needs an environment, `m` is a reader monad over that
environment. Many uses (e.g. zonkTcTypeMapper, swizzleMapper) need no
environment at all, and used to pass a dummy (). The binder field
becomes
tcm_tycobinder :: forall r. TyCoVar -> ForAllTyFlag
-> (TyCoVar -> m r) -> m r
and runs the continuation in the extended environment. mapTyCoX is gone;
only mapTyCo remains.
To make this efficient, the reader monad must use the one-shot trick and
be strict in the environment. So:
* New module GHC.Utils.Monad.StrictReader defines
newtype StrictReaderT r m a
with hand-written instances. The builder of the pattern synonym
`StrictReaderT` is a smart constructor that adds both `oneShot` and a
bang on the environment, so every StrictReaderT computation is strict
in the environment. See Note [Instances for StrictReaderT].
* ZonkT becomes type ZonkT = StrictReaderT ZonkEnv
ZonkM becomes type ZonkM = StrictReaderT ZonkGblEnv IO
replacing their own hand-written instances.
* Substitution uses
type SubstM = StrictReaderT Subst StrictIdentity
substTyCoMapper :: TyCoMapper SubstM
and similarly expandTypeSynonymMapper.
* zonk_tycomapper :: TyCoMapper (ZonkT TcM) is now much simpler.
* swizzleMapper uses StrictIdentity rather than Identity.
* Module renaming, so that the strict monads are all peers in GHC.Utils.Monad:
GHC.Utils.StrictIdentity ==> GHC.Utils.Monad.StrictIdentity
GHC.Utils.Monad.State.Strict ==> GHC.Utils.Monad.StrictState
alongside the new GHC.Utils.Monad.StrictReader.
Co-Authored-By: Claude Opus 5.5 <noreply(a)anthropic.com>
- - - - -
c0299e15 by Simon Jakobi at 2026-10-13T18:11:23-04:00
testsuite: Line-buffer the driver's stdout
Python block-buffers stdout when it is a pipe. This is the case for
example in CI. As a result there is often no indication of progress at
all until the run ends.
To fix this, we configure it explicitly to use line buffering instead.
stderr is reconfigured too, for consistency: Python line-buffers it by
default since 3.9.
The type-ignores are for mypy, which types these handles as TextIO — an
interface without reconfigure.
Fixes #12934.
Assisted-by: Claude Fable 5
- - - - -
9b5beaa2 by Florian Ragwitz at 2026-10-13T18:12:05-04:00
hpc: Improve coverage semantics for instance methods and instances
Users care about which of their explicitly written methods are or aren't
covered, not whether all of the inherited methods are covered as well. Those
should be covered where they are implemented.
We therefore skip generating top-level boxes for inherited methods which will
appear as uncovered with no clear indication as to why, and no way to fix other
than exercising code the user has not written.
Previously, top-level boxes for those inherited methods would use the instance
head as their source span. Now that those boxes are gone, we can use that
source span for another purpose:
We introduce a new top-level box for each class instance with its instance head
as the source span, and use that to indicate whether or not an instance has
been covered at all.
We do this by making any method of the instance, inherited/generated or
otherwise, tick the box for the instance head. Excluded from this are
methodless classes, which could never be covered using the current mechanism.
This could be improved upon further.
These changes apply to ticking for program coverage only, and don't affect
source notes. Additionally, we update the scope of record selector ticking to
be limited to program coverage as well, as the oversight to also extend it to
source notes has previously caused bugs and because record selector ticking is
coverage-specific.
Fixes #17155, #15932.
- - - - -
836ec50a by Simon Peyton Jones at 2026-10-13T18:12:54-04:00
Refactor fundeps again
This MR fixes #27828; there was a bad scoping error in `trim_qtvs`.
The main changes are
* Refactor `trim_qtvs` completely to eliminate the bug.
* Remove a redundant `apartnessCheck` from `mkTopOpenFamEqFDs`.
* Lots of improvements to the Notes in FunDeps.hs
I also did a couple of pure refactors:
* I realised that we were carefully caching `cab_incomps` in each
`CoAxBranch`, to make it more efficient to reduce a type family application,
but we were not doing the same thing when computing relevance for functional
dependencies; see Note [Relevance] in GHC.Tc.Solver.FunDeps. So I added
`cab_overlaps`, which is now used in `noEarlierMatch`.
* Use a proper data types, `AmIUnifying` and `OuterKindsFlag`, for two
previously-Boolean arguments to `tc_unify_tys`. This is a pure refactor.
- - - - -
889ef371 by Simon Peyton Jones at 2026-10-13T18:13:36-04:00
Refactor and simplify solver's unification code
Ticket #27844 showed that GHC's constraint solver could miss an
occurs check, and thereby build an infinite type.
This patch significantly simplifes the code in
GHC.Tc.Solver.Equality.canEqCanLHSFinish_try_unification
GHC.Tc.Solver.Equality.canEqCanLHSFinish_no_unification
It is now significantly simpler, and requires much less subtle
reasoning to understand.
Main changes
* If `canEqCanLHSFinish_try_unification` fails to unify, it now /always/ calls
`canEqCanLHSFinish_no_unification` rather than trying to finish the job itself.
* `GHC.Tc.Solver.Monad.checkTypeEq` is moved from Monad into Equality, and renamed
to `GHC.Tc.Solver.Equality.checkTypeEqNoUnification`.
* `canEqCanLHSFinish_no_unification` calls `checkTypeEqNoUnification` to test
the invariants for an inert equality. Before this MR there was a special case for
`do_not_prevent_rewriting`, but that has gone; instead it's just a consequence of
the flags passed to `checkTyEqRhs`, documented in the `PuOK` case of
`canEqCanLHSFinish_no_unification`.
* There is only one bit of "swap back" code, in `finish_no_unify` in
`canEqCanLHSFinish_try_unification. Previously this code was duplicated.
* `kickOutAfterFillingCoercionHole` should account for equalities with a
TyFamLHS. See (KOC5) in Note [Kick out after filling a coercion hole]
* No need to gather coercion holes in the `PuOK` case of `canEqCanLHSFinish_no_unification`.
See documentation with that code.
- - - - -
f9193e53 by Simon Peyton Jones at 2026-10-13T18:13:37-04:00
One line fix to fallout from
...#27731, !16564, simplifying Tc.Errors machinery
There was an over-eager assert!
- - - - -
cdd56b7a by Florian Ragwitz at 2026-10-13T18:14:19-04:00
Fix several sphinx errors and warnings from rendering the release notes
The extension references which are removed here are for extension which were
already previously undocumented, so just removing them seems fine for now.
- - - - -
4f00da25 by Florian Ragwitz at 2026-10-13T18:14:19-04:00
Link to QualifiedStrings documentation
- - - - -
97adaf89 by Simon Peyton Jones at 2026-10-14T00:07:27+01:00
Line up simpleUnifyCheck and check_ty_eq_rhs
Even after #26543 was allegedly fixed, the original repo case in the
Description continued to fail. The reason was that the QuickLook
unifier, `qlUnify`, used `simpleUnifyCheck` for checking unification
invariants; and `simpleUnifyCheck` conservatively rejected any RHS
with a coercion hole in it. The upshot was that QuickLook was not
as clever as it should be, wrongly failing to do an impredicative
instantiation.
Interestingly `check_ty_eq_rhs`, which does the same job during
constraint solving, was more liberal: it just looked at the free vars
of the coercion, and allowed coercion holes.
This MR lines them up, adding some careful notes. See
Note [simpleUnifyCheck]
Note [Occurs check and coercions]
Note [Unification preconditions] esp (COERCIONS)
Things are better than before, but I am still uncomfortable about the
possibilty that a hole might be filled with a coercion that mentions
the LHS type variable; for now I have left this discomfort documented
in Note [Occurs check and coercions].
Main changes
* `qlUnify` now only unifies /instantiation variables/ not regular unification
variables. This makes QuickLook more predictable. See (UQL6) in
Note [QuickLook unification].
* `qlUnify` no longer needs to call `simpleUnifyCheck`, so we can get rid of
`UC_QuickLook` and its code in `simpleUnifyCheck`.
* I found I could get rid of `mkOccFolders` in favour of `anyFreeVarsOfType`.
Hooray.
* Improve documentation of CycleBreakerTv, and rename it to GivenCycleBreakerTv
because they are only uesd for Givens!
* I accepted a 9% worsening in compile time allocation on T5030
because the newly-consistent story triggered #27934.
Other changes
* I refactored `atvFolder` so that it pairs the "interesting-var" function
with the in-scope set, to avoid building a closure with a free variable.
Now we get nice tight loops for `anyFreeVarsOfType` and friends.
See Note [Any-free-var folder].
Metric Increase:
T5030
- - - - -
5f7b3047 by Simon Peyton Jones at 2026-10-14T00:07:27+01:00
Fix order-dependence in QuickLook
Ticket #27917 pointed out that QuickLook was order-dependent
-- very bad!
The solution is in `qlUnify`: spot when we are unifying a type with a
definite monotype, and then monomorphise any free instantiation
variables of that type.
See (UQL8) and (UQL9) in Note [QuickLook unification].
- - - - -
c391c70c by Simon Peyton Jones at 2026-10-14T00:07:28+01:00
Add some comments about RuntimeUnk
This commit just adds comments explaining why we don't need to worry about
RuntimeUnkTv in the constraint solver.
It also tightens up docmentation of earlier (overlapping) commits in
this MR.
- - - - -
7df3ba2f by Andrea Vezzosi at 2026-10-14T06:15:48-04:00
add runGhcWithSignalHandlers and remove handling from runGhc[T]
it only ever makes sense to call `withSignalHandlers` from `main`, otherwise
signals can be too easily swallowed without notification to the rest of your
process.
This commit takes withSignalHandlers out of `runGhc/runGhcT` and provides
`runGhcWithSignalHandlers` for the old behaviour.
- - - - -
3a72b8a2 by Hannes Siebenhandl at 2026-10-14T13:48:32-04:00
Fix #27836, normalise unit database path in ExternalUnitDatabases
Add a regression test, proofing that all variations:
- local-unit-db/my-test.conf
- ./local-unit-db/my-test.conf
- ./local-unit-db/my-test.conf/
are parsed only once.
- - - - -
8ddbc6f9 by Florian Ragwitz at 2026-10-14T13:49:28-04:00
Remove obsolete HPC types
Fixes #27946.
- - - - -
f34c79ee by Alan Zimmerman at 2026-10-15T04:54:47-04:00
EPA: Start using vocurly as basis for AnnListLayout
For the existing cases that populate AnnList.
This uses AnnListBraces when braces are explicitly supplied, and
AnnListLayout when vocurly is used
The change also shows up AnnList usages that can never capture layout,
these are cleared up later.
- - - - -
033ba59c by sheaf at 2026-10-15T04:55:44-04:00
Make exprFreeVars deep
exprFreeVars used to be a shallow free-variable finder (it didn't look
into kinds), but most callers wanted it to be deep. In particular,
'exprFreeVars' is often used to seed an 'InScopeSet', and that must
contain the /deep/ free variables.
This commit makes it deep, which allows us to get rid of calls to
'closeOverKinds' or other workarounds for shallowness in:
- 'GHC.Core.Opt.Simplify.Utils.abstractFloats'
- 'GHC.HsToCore.Binds.dsSpec_help'
- 'GHC.Core.Opt.Arity.etaBodyForJoinPoint'
- 'GHC.Core.FVs.freeVars'
- 'GHC.Core.Lint.lintCoreRule'
We also remove 'deepExprsFreeVarsDSet' and 'closeOverKindsDSet'.
Other places needed a little bit more care, in particular to properly
handle coercion variables free in kinds:
- 'GHC.Core.Opt.Specialise.specHeader' quantifies over the binder of
an unspecialised argument directly, instead of computing the free Ids
of the argument.
- 'GHC.HsToCore.dsWarnRuleShadowing' filters out 'CoVar's.
- 'GHC.Core.Opt.SpecConstr.callToPat' had some ad-hoc logic to compute
argument pattern types and a workaround with 'expandTypeSynonyms'.
That logic is removed in favour of a new 'argsToPats' function which
directly computes the argument pattern types by instantiating the
type of the head of the call pattern.
Fixes #27936
- - - - -
f93d523c by Luite Stegeman at 2026-10-15T04:56:34-04:00
testsuite: Use mkWeakMVar in memo001/memo002
mkWeakPtr in the memo001/memo001 tests created weak pointers
with the MVar box as the key, which is fragile in the presence
of optimization. The fix is using mkWeakMVar, keying on the
primitive MVar# instead.
Fixes #27924 and #27396
- - - - -
8c947573 by Simon Peyton Jones at 2026-10-16T13:11:53+01:00
Improve exprOkForSpeculation
Fixes #27910, which exposed an infelicity in exprOkForSpeculation
The new story is carefully described in new Note [exprOkForSpeculation],
esp (a-c) and (EOKS1).
- - - - -
7dedcf13 by Alan Zimmerman at 2026-10-16T19:03:17-04:00
EPA: Add TTG fields to FamilyInfo
This commit is a simple add, with unused extension fields.
It lays the groundwork for using them for exact print annotations
- - - - -
26b9a8a0 by Arkissa at 2026-10-17T16:04:10+00:00
ghci: Prefer exact source spans in :type-at
Prefer expression types for identical source spans.
Fixes #27412.
- - - - -
70dc4e61 by sheaf at 2026-10-18T18:15:15-04:00
Carefully implement coercion zapping
This commit implements coercion zapping in the rewriter, controlled by
the -fzap-coercions flag (on by default). This significantly reduces
the size of coercions produced when rewriting type family applications,
resulting in much faster compilation. See Note [Coercion zapping]
in GHC.Tc.Solver.Rewrite for an explanation and further details.
The rewriter now keeps track of the `CoVar`s and `CoercionHole`s it uses
when rewriting, which allows computing the dependencies of a zapped
coercion without traversing the coercion the rewriter produced, as
explained in Note [Computing coercion dependencies] in GHC.Tc.Solver.Rewrite.
Accompanying changes:
- Rework how dependencies of 'UnivCo's are tracked, purely for
performance reasons.
We now use the 'UnivCoDeps' datatype instead of '[Coercion]', and
we now insist that 'uco_deps' also contains the CoVars that appear
in the LHS and RHS types of the UnivCo (as a caching mechanism).
- Several performance improvements to the rewriter (e.g. increased
strictness).
- Improvements to the family application cache, including:
- Better documentation.
- Only cache a whole chain of reduction steps.
- Don't put coercion holes in the cache, fixing #27518.
- Allow Core Lint to be called from within the typechecker (and from
within the coercion optimiser).
- Move out "annotation linting" from Core Lint into the general Core
pipeline, to avoid the dependency GHC.Core.Coercion.Opt->GHC.Core.Lint
causing a regression in CountDepsParser due to transitive
dependencies of GHC.Core.Opt.Monad and GHC.Unit.Module.ModGuts.
Fixes #8095 #27336 #27518
Summary of performance numbers
Compile-time byte allocated in GHC's testsuite (0 regressions)
CoOpt_Singletons(normal) ghc/alloc 634,040,332 595,448,728 -6.1% GOOD
FamAppCachePerf(normal) ghc/alloc 495,629,812 191,612,920 -61.3% GOOD
LargeRecord(normal) ghc/alloc 1,226,806,434 966,260,976 -21.2% GOOD
SimplCastPerf(normal) ghc/alloc 160,505,678 148,112,688 -7.7% GOOD
T12227(normal) ghc/alloc 244,668,982 233,037,216 -4.8% GOOD
T12545(normal) ghc/alloc 697,804,842 424,705,152 -39.1% GOOD
T13386(normal) ghc/alloc 422,871,076 35,213,656 -91.7% GOOD
T15703(normal) ghc/alloc 288,402,364 273,692,560 -5.1% GOOD
T26426(normal) ghc/alloc 86,002,052 78,152,104 -9.1% GOOD
T27336(normal) ghc/alloc 498,932,268 35,886,016 -92.8% GOOD
T5030(normal) ghc/alloc 117,057,706 88,785,648 -24.2% GOOD
T5321Fun(normal) ghc/alloc 252,552,628 243,820,176 -3.5% GOOD
T8095(normal) ghc/alloc 768,198,564 127,779,520 -83.4% GOOD
T9872a(normal) ghc/alloc 1,655,675,020 1,406,017,008 -15.1% GOOD
T9872b(normal) ghc/alloc 1,864,180,628 1,644,706,696 -11.8% GOOD
T9872b_defer(normal) ghc/alloc 2,834,585,468 1,657,729,760 -41.5% GOOD
T9872c(normal) ghc/alloc 1,572,573,116 1,354,055,424 -13.9% GOOD
T9872d(normal) ghc/alloc 351,516,004 331,071,424 -5.8% GOOD
TcPlugin_RewritePerf(normal) ghc/alloc 2,063,065,680 1,847,464,272 -10.5% GOOD
Chesskell (github.com/Proxymoron461/chesskell/issues)
Typechecking that the first N plies of kholmovVsBronsteinShort
are a valid chess game:
Plies | GHC 9.14 time/alloc | This commit time/alloc |
-----:|---------------------|------------------------|
2 | 41s/20GB | 0.2s/19MB |
4 | 95s/49GB | 0.3s/25MB |
6 | timed out | 0.6s/522MB |
8 | timed out | 7s/673MB |
10 | timed out | timed out |
Compilation of singletons-base: compile time -28%, allocations -25%
-------------------------
Metric Decrease:
CoOpt_Singletons
FamAppCachePerf
LargeRecord
SimplCastPerf
T12227
T12545
T13386
T15703
T24984
T26426
T27336
T5030
T5321Fun
T8095
T9872a
T9872b
T9872b_defer
T9872c
T9872d
TcPlugin_RewritePerf
-------------------------
Co-Authored-By: Sylvain Henry <sylvain(a)haskus.fr>
- - - - -
3d326d7d by Sylvain Henry at 2026-10-18T18:16:09-04:00
JS: fix stack corruption in h$ap_gen_fast's pushAllRegs (#27624)
The loop added in 9e46990976 ("JS: support more than 128 registers
(#26558)") pushed register N to stack[sp - N - 2] while the switch it
falls through into, and every reader of the h$ap_gen frame, uses
stack[sp - (N - 2)]. Registers >= R32 were therefore written 4 slots
too low: the slots the readers consult were left stale and the frame
below was silently corrupted.
Fix the loop to use the same slot formula as the switch, and add a
regression test: an unknown call to a thunk with 35 argument register
slots, which goes through the generic apply.
Co-Authored-By: Claude Fable 5 <noreply(a)anthropic.com>
- - - - -
960cc7c8 by Wolfgang Jeltsch at 2026-10-20T21:52:19+03:00
Make Haddock avoid the PIT
To this end, Haddock has been using `loadSysInterface` to fetch module
interfaces. The `loadSysInterface` operation is very complex, containing
many bits that do not seem to be needed by Haddock. In particular,
storing interface data in and potentially retrieving it from the PIT
seems to be superfluous. This interaction with the PIT limits how much
we can statically shrink the amount of information that is stored in the
PIT. Therefore, we drop the use of `loadSysInterface` and implement the
necessary bits of interface fetching directly in `loadHiFile`, where
`loadSysInterface` use to be invoked.
- - - - -
6eaf047f by Wolfgang Jeltsch at 2026-10-20T21:52:44+03:00
Statically enforce module interface reduction for the PIT
`GHC.Iface.Load.loadInterface` used to replace certain fields of each
fetched module interface by `panic` applications before storing it in
the PIT. With the present change, the absence of those fields in the PIT
is guaranteed statically.
See #27044 for the bigger picture.
- - - - -
c8dfce01 by Wolfgang Jeltsch at 2026-10-20T21:52:44+03:00
Make `IfaceBinHandle` a type synonym family
This brings the handling of `IfaceBinHandle` somewhat in line with how
other phase-dependent data is handled. Note that `mi_hi_bytes_` now
includes the `Strict.Maybe` layer itself, independently of the phase.
This is necessary, because with a type family instead of a GADT
`clear_mi_hi_bytes` cannot determine the phase at runtime anymore and
thus has to set `mi_hi_bytes_` to `Strict.Nothing` for every phase.
- - - - -
269 changed files:
- changelog.d/26616
- + changelog.d/27624
- + changelog.d/27936
- + changelog.d/T26543
- changelog.d/T27329
- + changelog.d/T27412
- + changelog.d/T27828
- + changelog.d/T27844
- + changelog.d/T4162
- changelog.d/boxing-family
- + changelog.d/hpc-classinsts
- + changelog.d/hpc-cleanup
- changelog.d/tcplugin_init.md
- changelog.d/unit-index
- + changelog.d/zap-coercions
- compiler/GHC.hs
- compiler/GHC/Builtin/WiredIn/Prim.hs
- compiler/GHC/Builtin/WiredIn/Types/Box.hs
- compiler/GHC/Cmm/GenericOpt.hs
- compiler/GHC/Cmm/UniqueRenamer.hs
- compiler/GHC/CmmToAsm/Dwarf/Types.hs
- compiler/GHC/CmmToAsm/Monad.hs
- compiler/GHC/CmmToAsm/Reg/Graph/Spill.hs
- compiler/GHC/CmmToAsm/Reg/Graph/SpillClean.hs
- compiler/GHC/CmmToAsm/Reg/Graph/SpillCost.hs
- compiler/GHC/CmmToAsm/Reg/Graph/Stats.hs
- compiler/GHC/CmmToAsm/Reg/Linear/State.hs
- compiler/GHC/CmmToAsm/Reg/Linear/Stats.hs
- compiler/GHC/CmmToAsm/Reg/Liveness.hs
- compiler/GHC/CmmToAsm/Wasm/Asm.hs
- compiler/GHC/CmmToAsm/Wasm/Types.hs
- compiler/GHC/CmmToC.hs
- compiler/GHC/Core.hs
- compiler/GHC/Core/Coercion.hs
- compiler/GHC/Core/Coercion.hs-boot
- compiler/GHC/Core/Coercion/Axiom.hs
- compiler/GHC/Core/Coercion/Opt.hs
- compiler/GHC/Core/DataCon.hs
- compiler/GHC/Core/FVs.hs
- compiler/GHC/Core/FamInstEnv.hs
- compiler/GHC/Core/Lint.hs
- + compiler/GHC/Core/Lint.hs-boot
- compiler/GHC/Core/Map/Type.hs
- compiler/GHC/Core/Opt/Arity.hs
- compiler/GHC/Core/Opt/CallerCC.hs
- compiler/GHC/Core/Opt/Exitify.hs
- compiler/GHC/Core/Opt/Pipeline.hs
- compiler/GHC/Core/Opt/Simplify/Utils.hs
- compiler/GHC/Core/Opt/SpecConstr.hs
- compiler/GHC/Core/Opt/Specialise.hs
- compiler/GHC/Core/Reduction.hs
- compiler/GHC/Core/Seq.hs
- compiler/GHC/Core/SimpleOpt.hs
- compiler/GHC/Core/Subst.hs
- compiler/GHC/Core/TyCo/Compare.hs
- compiler/GHC/Core/TyCo/FVs.hs
- compiler/GHC/Core/TyCo/FVs.hs-boot
- + compiler/GHC/Core/TyCo/Make.hs
- + compiler/GHC/Core/TyCo/Make.hs-boot
- compiler/GHC/Core/TyCo/Ppr.hs
- compiler/GHC/Core/TyCo/Rep.hs
- compiler/GHC/Core/TyCo/Rep.hs-boot
- compiler/GHC/Core/TyCo/Subst.hs
- compiler/GHC/Core/TyCo/Tidy.hs
- compiler/GHC/Core/Type.hs
- compiler/GHC/Core/Type.hs-boot
- compiler/GHC/Core/Unify.hs
- compiler/GHC/Core/Utils.hs
- compiler/GHC/CoreToIface.hs
- compiler/GHC/CoreToIface.hs-boot
- compiler/GHC/CoreToStg.hs
- compiler/GHC/Data/OsPath.hs
- compiler/GHC/Driver/Config.hs
- compiler/GHC/Driver/Config/Core/Lint.hs
- compiler/GHC/Driver/Downsweep.hs
- compiler/GHC/Driver/DynFlags.hs
- compiler/GHC/Driver/Env.hs
- compiler/GHC/Driver/Flags.hs
- compiler/GHC/Driver/Main/Compile.hs
- compiler/GHC/Driver/Main/Interactive.hs
- compiler/GHC/Driver/Main/Passes.hs
- compiler/GHC/Driver/Plugins.hs
- compiler/GHC/Driver/Session.hs
- compiler/GHC/Driver/Session/Inspect.hs
- compiler/GHC/Hs/Decls.hs
- compiler/GHC/Hs/Utils.hs
- compiler/GHC/HsToCore.hs
- compiler/GHC/HsToCore/Binds.hs
- compiler/GHC/HsToCore/Quote.hs
- compiler/GHC/HsToCore/Ticks.hs
- compiler/GHC/HsToCore/Usage.hs
- compiler/GHC/Iface/Binary.hs
- compiler/GHC/Iface/Decl.hs
- compiler/GHC/Iface/Ext/Ast.hs
- compiler/GHC/Iface/Ext/Types.hs
- compiler/GHC/Iface/Ext/Utils.hs
- compiler/GHC/Iface/Load.hs
- compiler/GHC/Iface/Load.hs-boot
- compiler/GHC/Iface/Make.hs
- compiler/GHC/Iface/Recomp.hs
- compiler/GHC/Iface/Rename.hs
- compiler/GHC/Iface/Syntax.hs
- compiler/GHC/Iface/Type.hs
- compiler/GHC/Iface/Type.hs-boot
- compiler/GHC/IfaceToCore.hs
- compiler/GHC/Parser.y
- compiler/GHC/Parser/PostProcess.hs
- compiler/GHC/Rename/Module.hs
- compiler/GHC/Rename/Names.hs
- compiler/GHC/Rename/Utils.hs
- compiler/GHC/Runtime/Eval/Types.hs
- compiler/GHC/Runtime/Heap/Inspect.hs
- compiler/GHC/Runtime/Loader.hs
- compiler/GHC/Stg/BcPrep.hs
- compiler/GHC/Stg/Debug.hs
- compiler/GHC/Stg/EnforceEpt/Rewrite.hs
- compiler/GHC/Stg/EnforceEpt/TagSig.hs
- compiler/GHC/Stg/Subst.hs
- compiler/GHC/StgToJS/Apply.hs
- compiler/GHC/StgToJS/ExprCtx.hs
- compiler/GHC/Tc/Deriv/Functor.hs
- compiler/GHC/Tc/Errors.hs
- compiler/GHC/Tc/Errors/Ppr.hs
- compiler/GHC/Tc/Errors/Types.hs
- compiler/GHC/Tc/Gen/App.hs
- compiler/GHC/Tc/Gen/Splice.hs
- compiler/GHC/Tc/Instance/Family.hs
- compiler/GHC/Tc/Instance/FunDeps.hs
- compiler/GHC/Tc/Module.hs
- compiler/GHC/Tc/Solver/Default.hs
- compiler/GHC/Tc/Solver/Equality.hs
- compiler/GHC/Tc/Solver/FunDeps.hs
- compiler/GHC/Tc/Solver/InertSet.hs
- compiler/GHC/Tc/Solver/Monad.hs
- compiler/GHC/Tc/Solver/Rewrite.hs
- compiler/GHC/Tc/Solver/Solve.hs
- compiler/GHC/Tc/TyCl.hs
- compiler/GHC/Tc/TyCl/Instance.hs
- compiler/GHC/Tc/Types.hs
- compiler/GHC/Tc/Types/Constraint.hs
- compiler/GHC/Tc/Utils/TcMType.hs
- compiler/GHC/Tc/Utils/TcType.hs
- compiler/GHC/Tc/Utils/Unify.hs
- compiler/GHC/Tc/Zonk/Env.hs
- compiler/GHC/Tc/Zonk/Monad.hs
- compiler/GHC/Tc/Zonk/TcType.hs
- compiler/GHC/Tc/Zonk/Type.hs
- compiler/GHC/ThToHs.hs
- compiler/GHC/Types/Arity.hs
- compiler/GHC/Types/Basic.hs
- compiler/GHC/Types/Cpr.hs
- compiler/GHC/Types/Demand.hs
- compiler/GHC/Types/Id/Info.hs
- compiler/GHC/Types/Name/Env.hs
- compiler/GHC/Types/Name/Occurrence.hs
- compiler/GHC/Types/Unique/DSM.hs
- compiler/GHC/Types/Unique/DSet.hs
- compiler/GHC/Types/Unique/FM.hs
- compiler/GHC/Types/Unique/Map.hs
- compiler/GHC/Types/Unique/Set.hs
- compiler/GHC/Types/Unique/Supply.hs
- compiler/GHC/Types/Var.hs
- compiler/GHC/Types/Var.hs-boot
- compiler/GHC/Types/Var/FV.hs
- compiler/GHC/Types/Var/Set.hs
- compiler/GHC/Unit/External.hs
- compiler/GHC/Unit/External/Database.hs
- compiler/GHC/Unit/Module/ModIface.hs
- compiler/GHC/Utils/Binary.hs
- + compiler/GHC/Utils/Monad/StrictIdentity.hs
- + compiler/GHC/Utils/Monad/StrictReader.hs
- compiler/GHC/Utils/Monad/State/Strict.hs → compiler/GHC/Utils/Monad/StrictState.hs
- compiler/GHC/Utils/Outputable.hs
- compiler/GHC/Utils/Panic.hs
- compiler/Language/Haskell/Syntax/Decls.hs
- compiler/Language/Haskell/Syntax/Extension.hs
- compiler/Language/Haskell/Syntax/Specificity.hs
- compiler/ghc.cabal.in
- docs/users_guide/extending_ghc.rst
- docs/users_guide/exts/literals.rst
- docs/users_guide/exts/type_families.rst
- ghc/GHCi/UI/Info.hs
- ghc/Main.hs
- libraries/base/tests/Memo1.lhs
- libraries/base/tests/Memo2.lhs
- libraries/base/tests/all.T
- linters/lint-codes/LintCodes/Static.hs
- testsuite/driver/runtests.py
- + testsuite/tests/codeGen/should_run/T27624.hs
- + testsuite/tests/codeGen/should_run/T27624.stdout
- testsuite/tests/codeGen/should_run/all.T
- testsuite/tests/corelint/T27374.hs
- testsuite/tests/count-deps/CountDepsParser.stdout
- testsuite/tests/default/default-fail05.stderr
- testsuite/tests/dependent/should_compile/T14729.stderr
- + testsuite/tests/driver/T27836/A.hs
- + testsuite/tests/driver/T27836/B.hs
- + testsuite/tests/driver/T27836/C.hs
- + testsuite/tests/driver/T27836/Makefile
- + testsuite/tests/driver/T27836/T27836.stdout
- + testsuite/tests/driver/T27836/all.T
- + testsuite/tests/driver/T27836/unitA
- + testsuite/tests/driver/T27836/unitB
- + testsuite/tests/driver/T27836/unitC
- + testsuite/tests/ghc-api/T4162.hs
- + testsuite/tests/ghc-api/T4162.stdout
- testsuite/tests/ghc-api/all.T
- testsuite/tests/ghci.debugger/scripts/T14628.stderr
- testsuite/tests/ghci/prog-mhu001/prog-mhu001c.stdout
- testsuite/tests/ghci/prog-mhu002/prog-mhu002d.stdout
- testsuite/tests/ghci/scripts/T16575.script
- testsuite/tests/ghci/scripts/T16575.stdout
- testsuite/tests/ghci/scripts/T16804.stdout
- testsuite/tests/ghci/scripts/T20150.stdout
- + testsuite/tests/ghci/scripts/T27412.hs
- + testsuite/tests/ghci/scripts/T27412.script
- + testsuite/tests/ghci/scripts/T27412.stdout
- testsuite/tests/ghci/scripts/all.T
- testsuite/tests/haddock/should_compile_flag_haddock/T17544.stderr
- + testsuite/tests/hpc/instmeths/instmeths.hs
- + testsuite/tests/hpc/instmeths/instmeths.stdout
- + testsuite/tests/hpc/instmeths/test.T
- + testsuite/tests/impredicative/T27917.hs
- testsuite/tests/impredicative/all.T
- + testsuite/tests/indexed-types/should_compile/T27828.hs
- + testsuite/tests/indexed-types/should_compile/T27828a.hs
- + testsuite/tests/indexed-types/should_compile/T27828b.hs
- + testsuite/tests/indexed-types/should_compile/ZapDeps.hs
- testsuite/tests/indexed-types/should_compile/all.T
- testsuite/tests/indexed-types/should_fail/T1897b.stderr
- testsuite/tests/indexed-types/should_fail/T9662.stderr
- testsuite/tests/parser/should_compile/DumpParsedAst.stderr
- testsuite/tests/parser/should_compile/DumpRenamedAst.stderr
- testsuite/tests/parser/should_compile/DumpSemis.stderr
- testsuite/tests/parser/should_compile/KindSigs.stderr
- testsuite/tests/rep-poly/RepPolyRule3.stderr
- testsuite/tests/simplCore/should_compile/T22375.stderr
- testsuite/tests/simplCore/should_compile/T22375DataFamily.stderr
- testsuite/tests/simplCore/should_compile/T26051.stderr
- + testsuite/tests/simplCore/should_compile/T27910.hs
- + testsuite/tests/simplCore/should_compile/T27910.stderr
- + testsuite/tests/simplCore/should_compile/T27936.hs
- testsuite/tests/simplCore/should_compile/all.T
- testsuite/tests/typecheck/should_compile/LocalGivenEqs.hs
- + testsuite/tests/typecheck/should_compile/T26543_orig.hs
- testsuite/tests/typecheck/should_compile/all.T
- testsuite/tests/typecheck/should_fail/T12785b.stderr
- testsuite/tests/typecheck/should_fail/T22707.stderr
- + testsuite/tests/typecheck/should_fail/T26543a.hs
- + testsuite/tests/typecheck/should_fail/T26543a.stderr
- + testsuite/tests/typecheck/should_fail/T27844.hs
- + testsuite/tests/typecheck/should_fail/T27844.stderr
- testsuite/tests/typecheck/should_fail/all.T
- utils/check-exact/ExactPrint.hs
- utils/check-exact/Parsers.hs
- utils/check-exact/Utils.hs
- utils/check-ppr/Main.hs
- utils/count-deps/Main.hs
- utils/dump-decls/Main.hs
- utils/haddock/haddock-api/src/Haddock.hs
- utils/haddock/haddock-api/src/Haddock/Backends/Hoogle.hs
- utils/haddock/haddock-api/src/Haddock/Backends/Hyperlinker/Utils.hs
- utils/haddock/haddock-api/src/Haddock/Backends/LaTeX.hs
- utils/haddock/haddock-api/src/Haddock/Backends/Xhtml/Decl.hs
- utils/haddock/haddock-api/src/Haddock/Convert.hs
- utils/haddock/haddock-api/src/Haddock/Interface.hs
- utils/haddock/haddock-api/src/Haddock/Interface/Rename.hs
- utils/haddock/haddock-api/src/Haddock/Types.hs
- utils/haddock/haddock-api/test/Haddock/Backends/Hyperlinker/ParserSpec.hs
The diff was not included because it is too large.
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/8869f526a69bc2c2a4acc4d9d60ea3…
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/8869f526a69bc2c2a4acc4d9d60ea3…
You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help
1
0
[Git][ghc/ghc][wip/romes/multi-threaded-debugger] 69 commits: Remove the call to hoistFlag from processCmdLineP
by Rodrigo Mesquita (@alt-romes) 20 Oct '26
by Rodrigo Mesquita (@alt-romes) 20 Oct '26
20 Oct '26
Rodrigo Mesquita pushed to branch wip/romes/multi-threaded-debugger at Glasgow Haskell Compiler / GHC
Commits:
098bba70 by Facundo Domínguez at 2026-09-17T14:57:41-04:00
Remove the call to hoistFlag from processCmdLineP
This call was undesirable because it was causing a reconstruction
of the list of flag processors.
Even relying on list fusion would not be sufficient because we want
to replace the list with a data structure with faster lookups to
address #25763.
The removal is done in two moves:
* We move CmdLineP to CmdLine.hs
* Then we move the effect of hoisting to processArgs via a new function
hoistEwM
- - - - -
e01799c7 by Facundo Domínguez at 2026-09-17T14:57:42-04:00
Do not create duplicate flag processors for the -W flag
This makes simpler implementing the command line parser with a trie map
holding flag processors. If we were to allow duplicate flag processors
for any flag, the trie map would need to be a multimap.
- - - - -
3875b257 by Facundo Domínguez at 2026-09-17T14:57:42-04:00
Replace the list of flag processors with a trie map in `processArgs`
Previously `GHC.Driver.CmdLine.findArg` was called for each
command-line argument, doing a linear search on the list of flag
processors each time. This was somewhat costly as the list of flag
processor has near 2000 elements.
A lookup with a trie map is faster in general, but the speed up is
only noticeable when calling `GHC.Driver.CmdLine.processArgs` sufficient
times, so the amount of lookups piles up. This could happen in large
projects with many units each having many dependencies. If only a
module needs to be recompiled, the user still needs to wait for all of
the flag parsing to complete.
The benchmark at testsuite/tests/perf/compiler/cmdline-parser shows a
10x speed up when using the trie.
The trie map is implemented in `GHC.Data.StringTrie`.
The design is explained in Note [Processing command line argument fast]
in GHC.Driver.Session.
The commit also:
* adds two property-based tests for trie map operations,
* adds a benchmark of the command line parser, and
* renames the predicate `arg_ok` to `valid_flag_suffix` to decide what
flag processor should be used when there are multiple matching
prefixes on a flag.
Fixes #25763.
- - - - -
5236634a by Andreas Klebinger at 2026-09-17T14:58:24-04:00
Compile Word64{Map,Set} with -O2 always.
This is the hottest code and even if we build GHC with -O0 otherwise -O2
pays off for these modules.
Fixes #27814.
- - - - -
545fd4fb by ARATA Mizuki at 2026-09-22T15:03:24-04:00
Change the C types of hs_{bitrev,bswap}{16,32} to match the Haskell types
bitReverse{16,32}# and byteSwap{16,32}# have the type
Word# -> Word#, but their C correspondents took and
returned StgWord{16,32}.
The mismatch on bitReverse16# led to the problem reported
in #27299. Although no similar issue is reported for
bitReverse32# and byteSwap{16,32}#, it seemed wise to
change them proactively.
This commit fixes the mismatch by changing the C type.
Fixes #27299
- - - - -
bf17f289 by fendor at 2026-09-22T15:04:05-04:00
Don't lose instances from shadowed imported modules with the same name
When we look for instances below the current module, we need to check
the unit id as well when we check for whether the module is the current
module, otherwise we ignore modules we import that have the same name as
the import module.
Adds regression test.
- - - - -
53eead15 by Wolfgang Jeltsch at 2026-09-24T20:12:03-04:00
Correct the documentation of `explicitUnits`
Resolves #27787.
- - - - -
35bf6f4b by Simon Jakobi at 2026-09-24T20:12:49-04:00
Remove stale specHeader bullet from Note [Don't make fillers for constraint types]
The Note said that Specialise.specHeader also calls mkAbsentFiller.
abfc224a27c removed that call: the specialiser no longer drops dead
arguments at all, see Note [Do not drop dead args from specialisations]
in GHC.Core.Opt.Specialise.
Assisted-by: Claude Fable 5.1
- - - - -
1abef1ef by Wolfgang Jeltsch at 2026-09-25T17:02:07-04:00
Improve the `mhu-closure` makefile
- - - - -
a79a68ea by Wolfgang Jeltsch at 2026-09-25T17:02:07-04:00
Make `ghci-multi` hide all unneeded packages
- - - - -
d6e72450 by Wolfgang Jeltsch at 2026-09-25T17:02:07-04:00
Improve the definition of the home unit closure property
- - - - -
c9a2766d by Wolfgang Jeltsch at 2026-09-25T17:02:07-04:00
Re-implement the home unit closure check
Compared to the previous implementation, the new one has the following
advantages:
* It is correct.
- It distinguishes between units that have the same unit ID but
different ABI hashes.
- When `-hide-all-packages` is not used, it considers as home unit
dependencies also units that are made implicitly available
because they are in the package database.
* It seems to be faster in usual settings.
- In particular, it does not have a preparation phase in which it
merges the dependency information from all home units, so that
its running time does not grow linearly with the size of the
package database in normal Cabal scenarios.
* It is (hopefully) clearer.
* It is better documented.
- - - - -
fa96acf8 by Wolfgang Jeltsch at 2026-09-25T17:02:07-04:00
Remove erroneous GHC flag in `hadrian-multi`
The flag was `-Werror=-Wno-error=incomplete-record-selectors`. Its
purpose was to permit incomplete record selectors, but because of the
illegal combination of `-Werror=` and `-Wno-error` it was ignored,
without any negative consequences.
- - - - -
9f48a5b9 by Alan Zimmerman at 2026-09-25T17:02:50-04:00
EPA: Add specific layout field to AnnList
Up to now AnnList has effectively served to represent lists of things
having layout, regardless of whether they do or not.
In Haskell, layout is optional in many places, as there is also an
alternative syntax using braces and semi-colons, which is free-form.
This commit introduces AnnListLayout, a data structure capturing this
distinction.
It is deliberately set up similarly to EpLayout, so that these two can
be merged in time.
The transition is done mechanically, where AnnListBraces is not used
yet, the original Nothing case maps to AnnListNoLayout, and the Just
case to AnnListLayout.
Later commits will fine tune these against the actual parser layout.
- - - - -
234bab08 by Vladislav Zavialov at 2026-09-29T17:22:54-04:00
Add regression test for #7803
Commit 2effe18ab5, first shipped with GHC 8.2.1, fixed #7803 "Superclass
methods are left unspecialized".
This change turns the example from the ticket description into a regression
test, checking that foo1 is fully specialised at Double, with no calls to the
overloaded exp5Tail.
Closes #7803
Assisted-by: Claude Opus 5
- - - - -
c406135c by Andreas Klebinger at 2026-10-01T06:57:28-04:00
RTS: Update flag parsing/handling:
Using 0 as argument to any of these flags now resets the flags setting to default:
- ``-n0`` disables nursery chunking.
- ``-K0`` (or a bare ``-K``) removes the stack size limit.
- ``-M0`` removes the heap size limit.
- ``-H0`` resets the suggested heap size, including the nursery scaling set by ``-H``.
- ``-AL0`` makes the large object allocation limit follow ``-A`` again.
GC Generations are now limited to 64.
-n larger than -A now emits a warning and disables chunking.
Fixes #27830 #27829
- - - - -
984ee82b by Andreas Klebinger at 2026-10-01T06:58:10-04:00
Fix MultiLayerModules timeout multiplier.
We accidentally used the compile multiplier for ghci commands which
use the run timeout setting to set the timeout.
Fixes #26263 and #27796.
- - - - -
50a9cf26 by Andreas Klebinger at 2026-10-01T06:58:10-04:00
testsuite: Include runtime in JUnit report.
- - - - -
e358619f by Alan Zimmerman at 2026-10-01T06:59:01-04:00
EPA: Use EpaLocation in EpVirtualBraces
Haddock processing uses EpLayout.EpVirtualBraces to record the current
layout column, for allocating haddock comments correctly.
For exact printing, we need the line as well as the col, so we know
how many lines to skip when the indented code block starts.
This commit updates EpLayout.EpVirtualBraces so that it carries an
EpaLocation instead of just the Int column, to allow it to be used for
both purposes.
It is a simple mechanical change, with no other impact, but it allows
us to use EpLayout instead of AnnListLayout in AnnList, in the next
commit in this series.
- - - - -
1d2a26ee by Andrew Lelechenko at 2026-10-01T20:22:04-04:00
Remove bitSize from MINIMAL pragma of class Bits
Cf. https://github.com/haskell/core-libraries-committee/issues/441
The function is deprecated for 12 years and has a default implementation,
yet it was never removed from the MINIMAL pragma, putting library authors
in a very awkward position. Let's finally remove bitSize from MINIMAL.
- - - - -
3cc21e4c by Hannes Siebenhandl at 2026-10-01T20:22:59-04:00
Add regression test for #27641
- - - - -
c35096f5 by Hannes Siebenhandl at 2026-10-01T20:23:00-04:00
Don't `:add` modules the loader will reject
Only home modules can be actually added to the loader context, thus we
only accept the module if it belongs to a home unit.
- - - - -
a563cb27 by Simon Jakobi at 2026-10-02T17:37:36-04:00
ci: accept modified changelog.d entries in lint-changelog
The changelog guide says to extend an existing entry rather than add a
new one when the feature is already documented. lint_changelog only
counted added files, so such MRs failed before the validator ran.
Fixes #27779.
Assisted-by: Claude Fable 5.1
- - - - -
03082cd4 by Simon Jakobi at 2026-10-02T17:38:17-04:00
changelog.d: Fix warning flag name in the semaphore-v2 entry
The entry refers to -Wsemaphore-version-mismatch, which does not exist.
The warning GHC emits when it cannot use a -jsem semaphore, including
on a protocol version mismatch, is -Wsemaphore-open-failure.
Context: !15729, #25087
Assisted-by: Claude Fable 5.1
- - - - -
a93705ee by Simon Jakobi at 2026-10-02T17:38:17-04:00
changelog.d: Fix the CLC proposal link in the T27455 entry
The entry used the Sphinx :ref: role with a URL as its target. :ref:
resolves only internal labels, so the generated release notes would
show the proposal name without a link. Use the plain hyperlink form
like the other entries.
Context: !16274, #27455
Assisted-by: Claude Fable 5.1
- - - - -
54381079 by Hannes Siebenhandl at 2026-10-02T17:39:01-04:00
Extend the `TrustOverlay` for wired-in units
The TrustOverlay only contained the `UnitId` of all on-disk units.
However, wired-in units are referred to by their wired-in unit id, which
is not the same as the on-disk unit id.
Thus, trust checks for wired-in units failed because the lookup was
using the wired-in unit, which wasn't present in the `TrustOverlay`.
We fix #27834, by copying the trust info for wired-in unit ids.
- - - - -
f8cfb1a5 by Hécate Kleidukos at 2026-10-04T19:36:39+02:00
[CLC 341] Strict variants of mutable types
This commit is the implementation of CLC Proposal 341
https://github.com/haskell/core-libraries-committee/issues/341
- - - - -
575c4bc6 by Rodrigo Mesquita at 2026-10-05T12:32:02-04:00
ttg: HsModuleName in the AST, ModuleName in GHC
This commit makes the AST fully independent from GHC by parsing a
"module name" as an `HsModuleName (= HText)` rather than constructing an
`ModuleName (= FastString)` straight away.
In the GHC side, everything remains as a `ModuleName`. We turn AST
`HsModuleName`s into fast strings with the cheap `mkFastStringShortText`
when we process the AST for manipulation within the compiler.
There are no performance regressions introduced by this approach.
This is the final piece of #21592
Alternative plans were discussed in
gitlab.haskell.org/ghc/ghc/-/merge_requests/16546#note_694647,
but the Simon suggested one won in simplicity and performance when
compared against my attempt at implementing my original plan of having
a family `ModuleNameP`.
- - - - -
b3b89d32 by mangoiv at 2026-10-05T22:03:32-04:00
hadrian: Persist the used flavour name in the build root (#20650)
This executes the idea proposed in #20650, teaching Hadrian to persist
the flavour used to build a build root. This both makes use of multiple
build roots more ergonomic (as the user no longer needs to repeat
themselves) and avoids a tiresome foot-gun (forgetting to specify a
flavour, often wholly invalidating a build root).
Closes #20650.
Co-authored-by: Sylvain Henry <sylvain(a)haskus.fr>
- - - - -
f977837b by mangoiv at 2026-10-05T22:04:21-04:00
configure: fix sed script and subst tooldir script
Fixes #27672
Adds sanity checking for SUBST_TOOLDIR
Metric Decrease:
LinkableUsage01
MultiLayerModulesTH_OneShot
T10547
T12234
T12425
T13035
T13820
T18140
T18730
T18923
T9198
TcPlugin_RewritePerf
mhu-perf
- - - - -
f1cb08be by Hannes Siebenhandl at 2026-10-05T22:05:03-04:00
Prepend in-memory unit database for backpack
The packageDbFlags are stored in reverse order. Thus, prepending will
ensure that the in-memory unit database is the last item on the unit
database stack.
Further, the in-memory unit database needs to be able to shadow units
from earlier unit databases in the unit database stack.
Moreover, a `-clear-package-db` would remove the in-memory database if
it weren't the last item on the stack.
Closes #27835
- - - - -
8d49ed42 by Kazu Yamamoto at 2026-10-06T19:37:25-04:00
Do not leave standard file descriptors free at startup (#24450)
A process may be started with stdout closed -- `prog >&-` from a shell does
exactly that. Nothing then reserves descriptor 1, and open(), pipe() and
socket() hand out the lowest free descriptor, so the next descriptor the
runtime creates for itself takes it: the ticker's pipe, or a capability's IO
manager control pipe.
'GHC.Internal.IO.StdHandles.stdout' is a CAF, built the first time the program
writes to stdout, which is long after that has happened. It builds a 'Handle'
over descriptor 1, which now belongs to the runtime, and nothing reports an
error because the descriptor is perfectly good. The program then writes its
output into the runtime's own plumbing.
What that looks like from the outside is a hang. The descriptor the Handle has
taken over is the read end of a pipe, so poll() for POLLOUT on it never says
yes and epoll_ctl() accepts a registration for it; the thread waiting to write
blocks for ever. Closing the Handle at exit closes the runtime's descriptor,
which is worse: the ticker then fails on every tick.
'hs_init_ghc' now opens /dev/null onto whichever of descriptors 0, 1 and 2 are
closed, before the runtime creates any descriptor of its own. Each is opened
for the access that descriptor would normally have, read-only for 0 and
write-only for 1 and 2, so that writing to 'stdin' or reading from 'stdout'
still fails rather than quietly succeeding against /dev/null. See
Note [Standard file descriptors must be open].
Moving only the descriptors the runtime opens would not do: the first file the
program itself opened would land on the free number instead. The fix has to
leave no free number. A program run this way therefore finds stdout connected
to /dev/null rather than failing with EBADF on the first write, which is what
such a program would get from most other language runtimes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com>
- - - - -
65bcf9e9 by Simon Jakobi at 2026-10-06T19:38:09-04:00
changelog: Repair the sentence describing #27477
The description read "programs using could crash", the object of
"using" having been lost. The synopsis already says which programs are
affected, so drop the dangling word.
Assisted-by: Claude Fable 5.1
- - - - -
15ece296 by mangoiv at 2026-10-08T15:14:56-04:00
ci: move hadrian/ghci into the lint category so it can be picked up by more runners
Fixes #27817
- - - - -
c0823135 by mangoiv at 2026-10-08T15:14:56-04:00
ci: make lint-adjacent jobs into platform independent
- move lint jobs into lint-anyarch tag
- unify lint jobs under the same platform independent container
- - - - -
f76464cf by Zubin Duggal at 2026-10-08T15:15:39-04:00
DmdAnal: Make AbsDmd neutral in maxDmd
fixes #27845
- - - - -
ffa56ae4 by sheaf at 2026-10-09T17:15:59+02:00
Preserve sign of zero when parsing -0.0#
This commit leverages the recently added GHC.Types.Literal.Floating
module in order to preserve the sign of negative zero from
source-written floating-point literals.
Fixes #27545
- - - - -
d538cac9 by Facundo Domínguez at 2026-10-09T20:26:38-04:00
Propagate annotations to simple interfaces
This allows annotations to be visible to plugins when run by
haddock. The LiquidHaskell plugin needs the annotations, in particular.
Fixes #26761
- - - - -
32c4f4ab by meooow25 at 2026-10-09T20:27:28-04:00
Bump containers submodule to 0.8.1
And fix affected tests.
* T4086a: containers depends on template-haskell-lift now instead of
template-haskell.
* T22611: Updated to be less dependent on containers internals.
- - - - -
c786065e by sheaf at 2026-10-09T20:28:20-04:00
Use an explicit pattern match in toIfaceIdDetails
The old wildcard pattern let this code bit rot: it did not properly account
for recently added 'IdDetails' such as 'RepPolyId', which resulted in
spurious 'pprTrace' messages that could even pollute HLS sessions.
Fixes #23785
- - - - -
e7cacf59 by sheaf at 2026-10-09T20:29:09-04:00
Take into account ifKind in freeNamesIfDecl
Commit 3c2f4bb4 added the 'ifKind' field to various constructors
of 'IfaceDecl', which preserves the original kind (before type synonym
expansion) for better error messages.
However, that commit forgot to update 'freeNamesIfDecl' correspondingly.
This could result in a "urk! lookup local fingerpring" crash when e.g.
a type's kind uses a type synonym defined later, as we failed to add a
dependency on the type synonym during dependency analysis.
Fixes #27916
- - - - -
41e80f41 by Simon Peyton Jones at 2026-10-11T14:38:51-04:00
Simplify the error-suppression mechanism in GHC.Tc.Errors
In `GHC.Tc.Errors.reportWanteds` we suppress some errors in favour of
others. But the mechanism had grown crufty, and #27331 exposed a bug.
This MR makes it simpler and more uniform, by doing everything via
the `ei_suppress` field of the ErrorItem.
Claude then found another closely-related bug (see the review linked
on !16564). That bug is immortalised in new test T18851d.
I'm still not happy with `ignoreConstraint` but we can worry about
that another day; I have not touched it.
Main changes
* Instead of /predicting/ when an error will be reported, we set
`cec_error_reported` when an error is /actually/ reported. (I renamed
it from `cec_suppress` to reflect this change.)
Now errors are suppressed only when a preceding error has actually been
reported. Simpler and more robust.
See `Note [cec_error_reported: suppressing less-serious errors]`
The big changes are in `reportWanteds` which looks very different.
* New `Note [ei_suppress: suppressing confusing errors]` describes the
(quite separate) `ei_suppress` mechanism for suppressing confusing
errors, mainly arising from non-empty rewriter sets
* In `tryTypeClassDefaulting`, instead of switching off type-class defaulting
if there is an insoluble /anywhere/, we just switch it off locally when
we find an insoluble sub-tree. See Note [When to do type-class defaulting]
and (W5) in Note [ApproximateWC]
* Greatly simplify `insolubleWantedCt`.
* I made type-class-defaulting be a bit more eager, only switched off when
a peer or ancestor (but not child) constraint is insoluble. See
Note [When to do type-class defaulting].
Small refactorings
* Rename `askNoErrs` to `askErrsFound` and flip its polarity
* The code for "check if ei_suppress is true of all items" code is
moved to `mkErrorItems` rather than in the large `reportWanteds`.
* Code for `has_gadt_match` moves to `mkGivenErrorReporter`, rather than
polluting the large `reportWanteds`.
* Many tests with `-fdefer-type-errors` get more warnings, especially
about ambiguous (Num a) constraints. This is by-design: see
Note [Always warn with -fdefer-type-errors]. Previously GHC was not
honouring this Note but now it is, hence the extra errors.
- - - - -
f217fc31 by Cheng Shao at 2026-10-11T14:39:33-04:00
compiler: ensure cross ghc --info output target unit ids
Previously, for cross ghcs, the base/ghc/ghc-internal unit ids in
--info output do not match the target unit ids reported by ghc-pkg,
breaking cross ghc release builds with hash-unit-ids when being used
with cabal 3.18. This patch fixes it by making --info output target
unit ids. Fixes #27919.
Co-authored-by: Codex <codex(a)openai.com>
- - - - -
b5227250 by Rodrigo Mesquita at 2026-10-15T16:38:09+01:00
Asynchronous RPC with external interpreter
There's a single pipe to connect the host compiler to the external
interpreter. On this pipe, we have historically done synchronous
request/reply communication to invoke methods on the external
interpreter.
However, with the advent of multi-threaded debugging, it has become
necessary to do asynchronous calls on the external interpreter, where we
may block on the host waiting for a reply, but the external interpreter
should handle the request asynchronously and reply when ready. For
instance, we want to start the evaluation of the debuggee but not block
the interpreter waiting for it to finish or hit a breakpoint. Or e.g. we
want to resume two threads of the debuggee simulatenously instead of
blocking waiting for the resume result of each one sequentially.
And it turns out, we were already doing some ad-hoc asynchronous
communication to evaluate Template Haskell on the external interpreter.
It used to go as an ad-hoc "send request without expecting reply",
followed by handling on the host a set of 'THMessage's sent by the
interpreter, followed by "expect a response to arrive now". It worked
because there was still a fixed order of messages, but it fits an async
model in a non-ad-hoc way.
Asynchronous communication that unblocks the external interpreter main
thread will also make it possible to resolve issues such as #18852
(exceptions leaving the external interpreter in an unrecoverrable state)
The new design
--------------
The new invariant: every single request should receive a matching reply.
The pipe is bidirectional (the ext. interpreter can receive requests,
but it can also send requests for TH evaluation; the host can send
requests, but it can also receive TH requests from the interpreter).
When a request is sent in whichever direction, it always contains a
correlation-id header. The Sender places an empty MVar on a
pending-reply map keyed by id and waits on it for a response.
See 'remoteCall' and 'remoteTHCall'.
The pipe is associated to a thread which drains the read end. Because
it's bidirectional, two things may be read off the pipe:
- A Reply, with a correlation-id header, which we match against the
pending requests map to unblock the corresponding pending-reply MVar
- A Request, which is queued to the pending-request queue.
To handle a message, we take a Request from the pending-request queue
and run a handler that produces a Reply. Currently, the handler is run
synchronously in the thread, but it is trivial to make the handler run
in a forked thread for certain ("interruptible") messages in an upcoming
patch (when it is necessary for the multi-threaded debugger).
See 'handleRemoteCall' and 'handleRemoteTHCall'.
The old interp lock now just protect the pipe writing (reads don't lock
bc they are done exclusively by drainPipe), it doesn't need nor should
lock the full remote procedure call.
Fixes #27960
- - - - -
cc116c23 by Rodrigo Mesquita at 2026-10-15T16:38:09+01:00
interp: Add separate mailboxes for TH async RPC
When the host sends a `RunTH` command to the interpreter, it expects a
few "reverse requests" from the interpreter (e.g. `NewName`) which must
be run in the same context that initiated the `RunTH`.
When the host is evaluating many splices at once, it will send many
`RunTH` commands to the interpreter and start listening for reverse
requests in the meantime. However, a received request previously had no
way to be associated to the scope of a particular `RunTH` invocation, so
the reverse commands could get replied to from incorrect contexts, or
e.g. a `RunTHDone` from the interp. could terminate early the wrong
`RunTH` computation.
In this commit we add independent mailboxes to the interpreter
asynchronous RPC infrastructure, allowing a RunTH to announce a
particular `InboxId` that all reverse requests for that RunTH should
use. On the host, we create that mailbox id and have
`handleRemoteTHCall` listen on that specific mailbox rather than for
requests without a specific mailbox address.
- - - - -
58a378d9 by Rodrigo Mesquita at 2026-10-15T16:38:10+01:00
refactor: Move Async RPC abstraction into GHCi.Message.Async
Carves out the Async RPC over the Pipe abstraction to a new module,
making the GHCi.Message more about the specific interpreter messages
rather than about the IPC protocol.
- - - - -
014202d0 by Rodrigo Mesquita at 2026-10-15T16:38:10+01:00
interp: If we have pending reads in the queue we can deliver them
We may still be able to reply even if the read end was closed.
As long as we have pending messages we should try to deliver them.
- - - - -
d707be17 by Andrea Vezzosi at 2026-10-15T16:38:10+01:00
interp: close Handles when done with Pipe
Ensures we close the pipe handles (if created using 'mkPipeFromHandles')
on `closePipe`, and do `closePipe` on `Shutdown`.
The thread draining the read end always `'finally'` closes the read end.
And `closePipe` kills the draining thread and closes the write end.
When `remoteAsyncReq`, we throw PipeReadClosedException if the pipe read is
closed, since we wouldn't be able to receive the reply.
It's fine for the drain thread not to close the write end, even if it
dies without `closePipe` (which does close the write end), because:
- requests pending a reply will get PipeClosedException (we'll not be
able to read any further "real" reply)
- new calls to `remoteAsyncReq` will throw PipeReadClosedException
- pending pipe writes, in response to an already read request, might still succeed
And the clean up code should call close pipe to ensure the write end is
closed as well.
So, if the drain thread dies naturally and the read end is closed, may
still be attempted to be used, but it will throw accordingly. The user
likely wants to catch this to close the write end as well.
- - - - -
66d87bca by Zubin Duggal at 2026-10-16T15:18:00+05:30
interp: fix external interpreter shutdown deadlock on Windows
On shutdown both GHC and iserv call `closePipe`, which stopped the drain
thread before closing the write end. On Windows the drain thread reads
with a safe foreign call to _read, which `throwTo' can't interrupt, so
the two sides were stuck in the deadlock deadlock.
Closing the write end first is not enough on its own. On Windows a read
on a pipe only unblocks once all write handles to it are closed [1].
However, all all four pipe ends were inheritable and we didn't close the
child's ends in the parent. This means a close on any one side wouldn't
unblock any reads, becuase the other side had another copy open.
* runWithPipes now uses System.Process.CommunicationHandle [2], which is
built for passing pipes to a child process: only the child's ends are
inheritable, and it simplifies our mess of CPP. Additionally, it is
important to close our copy of the endpoints passed to the child so
each side only keeps open copies it needs to allow a close to unblock
reads.
* readGhcHandle makes the handle non-inheritable so we don't have
additional references floating around that stop reads from unblocking
on close.
* In closePipe the ordering is subtle and important. We first fail any
requests still waiting for a reply, then close the write end so the
other side sees EOF. We take the write lock to close it, so we never
close under a concurrent write [3]. Only then do we stop the drain
thread, which closes the read end itself once its read has finished.
Bump bound to process >= 1.6.20 as this is the version with
System.Process.CommunicationHandle
[1] https://learn.microsoft.com/en-us/windows/win32/ipc/anonymous-pipe-operatio…
[2] https://github.com/haskell/process/pull/308
[3] https://gitlab.haskell.org/ghc/ghc/-/work_items/27801
- - - - -
6f97d0cf by Andrea Vezzosi at 2026-10-20T11:35:34+01:00
GHCi.Server: handle RunTH and RunModFinalizers in forked thread
This commit makes the external interpreter able to many interpret
template haskell splices in parallel without blocking the main server
thread. Simply: fork off a thread to handle a RunTH request. The handler
will issue the reverse-requests on the given RunTH mailbox, to keep each
RunTH isolated from the others.
`handleRemoteCall` was generalised slightly by allowing `WriteReply` to
be used linearly in the handler directly, so that the caller of
`handleRemoteCall` may `WriteReply` in the forked thread rather than in
the main thread.
The only question is how can we guarantee `WriteReply` is used linearly
if we want to use it inside a forked thread? We need two guarantees:
1. `WriteReply` is used in the forked thread regardless of whether an
exception is thrown to that thread.
2. The forked thread is always awaited for, so that the main program
doesn't exit before the thread has a chance to actually use
`WriteReply`.
We introduced a small `ThreadManager` which provides a `spawnFinally`
that provides these two guarantees:
1. is guaranteed by `forkFinally`, which always runs the clean up action
exactly once.
2. is guaranteed by `ThreadManager` being abstract and the only exposed
function that introduces `ThreadManager` being `withThreadManager`,
which always awaits for all spawned threads.
Co-authored-by: Rodrigo Mesquita <rodrigo.m.mesquita(a)gmail.com>
- - - - -
beb35d96 by Andrea Vezzosi at 2026-10-20T13:56:59+02:00
interp: spawn fillPipe thread, writePipe now safely interruptible
Previously an async exception during a call to `writePipe` could have caused a
partial write, making any further write impossible to parse from the other end.
Now `writePipe` merely pushes the message onto a queue, and the actual write
happens in a separate `fillPipe` thread. As a result of this and previous
changes, `interpCmd` no longer needs `interruptibleMask` to preserve the state
of the protocol.
`writePipe` waits for the actual write to complete, and rethrows exceptions
raised in the write attempt, if any.
- - - - -
926c35bb by Rodrigo Mesquita at 2026-10-20T14:09:50+01:00
Global break action which uses shared thread map
the status Var must actually be global across all threads, otherwise all of them will block trying to write some Status Var no one is looking at at all
have to fix these toggle untoggle things
very satisfying solution with mutable state
Matthew-certified big brain
WIP
HMmm almost there I think
pretty great progressssssssss
ExecOptions isolateThreadBreaks
ThreadBreaksIsolationMode
move to GHCi.Run.Breakpoints
- - - - -
ca7c44a5 by Rodrigo Mesquita at 2026-10-20T14:09:50+01:00
Refactor: we have the interactive Ids directly... clean up significantly
Removes things like
names <- map idName ids
map (\i -> let id = find (\x -> i==idName x) names) idso
and is more straightforward because there's less state going around (no
more updating HscEnv as an output)
It also kills `printTypeOfNames` completely, since now we just have the
Ids. No longer `lookupName` names to get back the ids...
```
printTypeOfName :: GHC.GhcMonad m => Name -> m ()
printTypeOfName n
= do maybe_tything <- GHC.lookupName n
case maybe_tything of
Nothing -> return ()
Just thing -> printTyThing thing
```
now is just `printTyThing . AnId`
Also:
```diff
- tythings <- catMaybes `liftM` mapM GHC.lookupName namesSorted
- docs <- mapM pprTypeAndContents [i | AnId i <- tythings]
+ docs <- mapM pprTypeAndContents idsSorted
```
(and yes, in ExecBreaks there are only Ids, no TyThings, bc they are
only constructed by the interpreter execution which can only bind ids.
for TyThings use runDecls which doesn't return execBreaks.)
I have a plan for the `resumes`, I'm getting there.
fixes
grrr
UNDO the removing-hsc-env-globality part of the clean up and keep just the good parts
fixes
- - - - -
b35d5c73 by Rodrigo Mesquita at 2026-10-20T14:09:50+01:00
Make the ExecBreak contain a Resume
Put the resume in ExecBreak
- - - - -
f052dd80 by Rodrigo Mesquita at 2026-10-20T14:09:50+01:00
Make resumeExec receive a Resume as an argument
- - - - -
9ff76575 by Rodrigo Mesquita at 2026-10-20T14:09:50+01:00
Build EvalOpts once per exec rather than per breakpoint stop
in interpreter_steplocal the allocations increased because now we're
updating `eval_opts` after constructing them in `handleRunStatus` (which
is run many times since in step-local most often we don't stop, just
check).
So drop the update to the eval_opts, and, in fact, pull their
construction out. the loop always uses the same eval_opts, so no point
in constructing them inside from scratch every time.
-3.6% from this branch which had the regressions in
interpreter_steplocal
- - - - -
21fee4b3 by Rodrigo Mesquita at 2026-10-20T14:09:50+01:00
Look up a thread's ResumeContext without taking a lock
modifyMVar is unnecessarily expensive and blocking when most often we
just care about reading without modifying. optimize for default case
(thread context already exists). It is called every single time a
breakpoint is hit (regardless of whether we stop there, e.g. in
interpreter_steplocal we do not stop there often)
-3.3% improvement over original regression
- - - - -
e8a5011a by Rodrigo Mesquita at 2026-10-20T14:09:51+01:00
Every thread resume-context keeps a var for its EvalStatus
Kill the global mapping from thread id to EvalStatus
Instead, keep the EvalStatus in the ResumeContext associated to a thread
in the threadContextVar (where entries are created on breakpoints being
hit and when writing an EvalComplete result)
Before, every breakpoint hit meant reading the global map TVar and
deleting a map entry which did a lot of allocations. Now, we have a
Nothing variable which we overwrite to "take" rather than deleting from
a Map and allocating the GlobalDbgMap too. Note that for every
breakpoint hit we'd have to insert the ThreadId entry in the map again
(because we deleted it last time we resumed), again allocating
unnecessarily. Now it's only a look up on the map and an update on the
variable directly.
And a LOT of breakpoints were hit on interpreter_steplocal, stress
testing the cost of hitting a breakpoint
The isolated threads move to their own top-level TVar Set.
-4.4% improvement over the regression
- - - - -
3b6556e0 by Rodrigo Mesquita at 2026-10-20T14:09:51+01:00
Move whether a thread is isolated to the same global map
Stops us from reconstructing the set every time this changed instead to
being a per thread variable just toggled on/off, by default non-isolated
like before (before it was Set.empty, Set.insert for True, Set.delete
for FALSE). This is more straightforward -- there's always an isolation
mode per thread and we can toggle it directly
reduces slightly allocations, but the main advantage is killing a second
global map and making it more uniform with just one thread map and no
membership check for toggledness.
- - - - -
6e22a30f by Rodrigo Mesquita at 2026-10-20T14:09:51+01:00
Prefer foldl' in readAnyThreadEvalBreak txn
retry 'orElse' next is just next, so we might as well use a strict fold
- - - - -
03cf2a12 by Rodrigo Mesquita at 2026-10-20T14:09:51+01:00
Add inline TODO about improving efficiency of readyAnyThreadBreak
It is currently O(N) in number of threads that have hit breakpoints
It should be number of *live* threads that have hit breaks, bc we still need to find a way to evacuate threads from the map
- - - - -
97da5f78 by Rodrigo Mesquita at 2026-10-20T14:09:51+01:00
Set global breakpoint action in sandboxIO rather than withBreakAction
- - - - -
67a6b65b by Rodrigo Mesquita at 2026-10-20T14:09:51+01:00
TODO comment on breakOnException being global rather than per-thread
- - - - -
042b5d77 by Rodrigo Mesquita at 2026-10-20T14:31:23+01:00
fixmeup
- - - - -
4501a016 by Rodrigo Mesquita at 2026-10-20T15:43:54+01:00
WIP: Asynchronous eval/resume
- - - - -
a1ec78a5 by Rodrigo Mesquita at 2026-10-20T16:11:17+01:00
fixup! WIP: Asynchronous eval/resume
- - - - -
c73a96b6 by Rodrigo Mesquita at 2026-10-20T16:36:00+01:00
Fix commentary
- - - - -
8a1f45eb by Rodrigo Mesquita at 2026-10-20T16:37:53+01:00
fix
- - - - -
2f52e43f by Rodrigo Mesquita at 2026-10-20T17:01:17+01:00
more fixes
- - - - -
314 changed files:
- .gitlab-ci.yml
- .gitlab/ci.sh
- .gitlab/hello.hs
- + changelog.d/24450
- + changelog.d/25763-cmdline-opt
- + changelog.d/26610
- + changelog.d/26761
- + changelog.d/27641
- + changelog.d/27845
- + changelog.d/T23785
- changelog.d/T27455
- + changelog.d/T27545
- + changelog.d/T27731
- + changelog.d/T27830-rts-flag-bounds
- + changelog.d/T27916
- + changelog.d/T27960
- + changelog.d/clc-341-strict-mutable-data-types
- + changelog.d/fix-cross-ghc-info-unit-id
- changelog.d/fix-parallel-gc-selector-race-27477
- + changelog.d/hadrian-persist-flavour
- changelog.d/semaphore-v2
- changelog.d/unit-index
- compiler/GHC.hs
- compiler/GHC/Builtin/Modules.hs
- compiler/GHC/Builtin/WiredIn/Ids.hs
- compiler/GHC/ByteCode/Asm.hs
- compiler/GHC/ByteCode/Show.hs
- compiler/GHC/Core/DataCon.hs
- compiler/GHC/Core/Make.hs
- compiler/GHC/Core/Opt/CallerCC/Types.hs
- compiler/GHC/Core/Opt/WorkWrap/Utils.hs
- compiler/GHC/CoreToIface.hs
- + compiler/GHC/Data/StringTrie.hs
- compiler/GHC/Data/Word64Map/Internal.hs
- compiler/GHC/Data/Word64Map/Strict/Internal.hs
- compiler/GHC/Data/Word64Set/Internal.hs
- compiler/GHC/Driver/Backpack.hs
- compiler/GHC/Driver/CmdLine.hs
- compiler/GHC/Driver/Config.hs
- compiler/GHC/Driver/Downsweep.hs
- compiler/GHC/Driver/Env.hs
- compiler/GHC/Driver/Errors/Ppr.hs
- compiler/GHC/Driver/Errors/Types.hs
- compiler/GHC/Driver/Main/Compile.hs
- compiler/GHC/Driver/Pipeline.hs-boot
- compiler/GHC/Driver/Pipeline/Execute.hs
- compiler/GHC/Driver/Pipeline/Phases.hs
- compiler/GHC/Driver/Session.hs
- compiler/GHC/Hs/Doc.hs
- compiler/GHC/Hs/Dump.hs
- compiler/GHC/Hs/Expr.hs
- compiler/GHC/Hs/ImpExp.hs
- compiler/GHC/Hs/Lit.hs
- compiler/GHC/HsToCore/Docs.hs
- compiler/GHC/HsToCore/Match/Literal.hs
- compiler/GHC/HsToCore/Quote.hs
- compiler/GHC/Iface/Decl.hs
- compiler/GHC/Iface/Errors/Types.hs
- compiler/GHC/Iface/Ext/Ast.hs
- compiler/GHC/Iface/Ext/Debug.hs
- compiler/GHC/Iface/Ext/Types.hs
- compiler/GHC/Iface/Syntax.hs
- compiler/GHC/Iface/Tidy.hs
- compiler/GHC/Parser.y
- compiler/GHC/Parser/Annotation.hs
- compiler/GHC/Parser/Errors/Types.hs
- compiler/GHC/Parser/Header.hs
- compiler/GHC/Parser/Lexer.x
- compiler/GHC/Parser/PostProcess.hs
- compiler/GHC/Parser/PostProcess/Haddock.hs
- compiler/GHC/Parser/String.hs
- compiler/GHC/Rename/Env.hs
- compiler/GHC/Rename/Module.hs
- compiler/GHC/Rename/Names.hs
- compiler/GHC/Rename/Unbound.hs
- compiler/GHC/Runtime/Debugger.hs
- compiler/GHC/Runtime/Eval.hs
- compiler/GHC/Runtime/Eval/Types.hs
- compiler/GHC/Runtime/Heap/Inspect.hs
- compiler/GHC/Runtime/Interpreter.hs
- compiler/GHC/Runtime/Interpreter/JS.hs
- compiler/GHC/Runtime/Interpreter/Process.hs
- compiler/GHC/Runtime/Interpreter/Types.hs
- compiler/GHC/Runtime/Interpreter/Wasm.hs
- compiler/GHC/Runtime/Utils.hs
- compiler/GHC/Settings.hs
- compiler/GHC/Settings/IO.hs
- compiler/GHC/StgToJS/Linker/Linker.hs
- compiler/GHC/Tc/Deriv/Generics.hs
- compiler/GHC/Tc/Errors.hs
- compiler/GHC/Tc/Errors/Ppr.hs
- compiler/GHC/Tc/Errors/Types.hs
- compiler/GHC/Tc/Gen/Export.hs
- compiler/GHC/Tc/Gen/Splice.hs
- compiler/GHC/Tc/Instance/Family.hs
- compiler/GHC/Tc/Module.hs
- compiler/GHC/Tc/Solver/Default.hs
- compiler/GHC/Tc/Solver/Equality.hs
- compiler/GHC/Tc/Types.hs
- compiler/GHC/Tc/Types/Constraint.hs
- compiler/GHC/Tc/Types/ErrCtxt.hs
- compiler/GHC/Tc/Utils/Monad.hs
- compiler/GHC/ThToHs.hs
- compiler/GHC/Types/Demand.hs
- compiler/GHC/Types/Unique.hs
- compiler/GHC/Types/UnresolvedImport.hs
- compiler/GHC/Unit/Env.hs
- compiler/GHC/Unit/External/Wired.hs
- compiler/GHC/Unit/Home.hs
- compiler/GHC/Unit/Module.hs
- compiler/GHC/Unit/Module/Env.hs
- + compiler/GHC/Unit/Module/Name.hs
- compiler/GHC/Unit/Parser.hs
- compiler/GHC/Unit/State.hs
- compiler/GHC/Unit/Types.hs
- compiler/GHC/Unit/Types.hs-boot
- compiler/GHC/Utils/Binary.hs
- compiler/GHC/Utils/Outputable.hs
- compiler/Language/Haskell/Syntax.hs
- compiler/Language/Haskell/Syntax/Expr.hs
- compiler/Language/Haskell/Syntax/ImpExp.hs
- compiler/Language/Haskell/Syntax/Lit.hs
- compiler/Language/Haskell/Syntax/Module/Name.hs
- compiler/Language/Haskell/Syntax/Text.hs
- compiler/ghc.cabal.in
- distrib/configure.ac.in
- docs/users_guide/runtime_control.rst
- ghc/GHC/Driver/Session/Mode.hs
- ghc/GHCi/UI.hs
- ghc/GHCi/UI/Info.hs
- ghc/GHCi/UI/Monad.hs
- hadrian/bindist/Makefile
- hadrian/bindist/config.mk.in
- hadrian/doc/flavours.md
- hadrian/ghci-multi-cabal.in
- hadrian/src/Rules.hs
- hadrian/src/Rules/BinaryDist.hs
- hadrian/src/Rules/Generate.hs
- hadrian/src/Settings.hs
- libraries/base/base.cabal.in
- libraries/base/changelog.md
- libraries/base/src/Control/Concurrent/Chan.hs
- + libraries/base/src/Control/Concurrent/Chan/Strict.hs
- libraries/base/src/Control/Concurrent/MVar.hs
- + libraries/base/src/Control/Concurrent/MVar/Strict.hs
- libraries/base/src/Data/IORef.hs
- + libraries/base/src/Data/IORef/Strict.hs
- libraries/containers
- libraries/ghc-internal/src/GHC/Internal/Bits.hs
- libraries/ghci/GHCi/Message.hs
- + libraries/ghci/GHCi/Message/Async.hs
- libraries/ghci/GHCi/Run.hs
- + libraries/ghci/GHCi/Run/Breakpoints.hs
- libraries/ghci/GHCi/Server.hs
- + libraries/ghci/GHCi/Server/Threads.hs
- libraries/ghci/GHCi/TH.hs
- libraries/ghci/GHCi/Utils.hsc
- libraries/ghci/ghci.cabal.in
- linters/lint-codes/LintCodes/Static.hs
- m4/subst_tooldir.m4
- rts/RtsFlags.c
- rts/RtsStartup.c
- rts/include/rts/Flags.h
- rts/include/rts/storage/GC.h
- rts/include/stg/Prim.h
- rts/prim/bitrev.c
- rts/prim/bswap.c
- rts/sm/GC.c
- rts/sm/Storage.c
- testsuite/driver/junit.py
- testsuite/driver/testglobals.py
- testsuite/driver/testlib.py
- testsuite/tests/MiniQuickCheck.hs
- + testsuite/tests/backpack/should_compile/T27835.bkp
- + testsuite/tests/backpack/should_compile/T27835.stderr
- testsuite/tests/backpack/should_compile/all.T
- testsuite/tests/count-deps/CountDepsAst.stdout
- testsuite/tests/count-deps/CountDepsParser.stdout
- + testsuite/tests/dmdanal/should_run/T27845.hs
- + testsuite/tests/dmdanal/should_run/T27845.stdout
- + testsuite/tests/dmdanal/should_run/T27845_Lib.hs
- + testsuite/tests/dmdanal/should_run/T27845b.hs
- + testsuite/tests/dmdanal/should_run/T27845b.stdout
- + testsuite/tests/dmdanal/should_run/T27845b_Lib.hs
- testsuite/tests/dmdanal/should_run/all.T
- testsuite/tests/driver/all.T
- testsuite/tests/driver/multipleHomeUnits/mhu-closure/Makefile
- testsuite/tests/driver/multipleHomeUnits/mhu-closure/mhu-closure.stderr
- testsuite/tests/driver/multipleHomeUnits/mhu-closure/mhu-closure.stdout
- + testsuite/tests/driver/stringtrie.hs
- + testsuite/tests/driver/stringtrie.stdout
- testsuite/tests/ghc-api/downsweep/IncrementalDownsweep.hs
- testsuite/tests/ghc-api/exactprint/T22919.stderr
- testsuite/tests/ghc-api/exactprint/Test20239.stderr
- testsuite/tests/ghc-api/exactprint/ZeroWidthSemi.stderr
- testsuite/tests/ghci/custom-external-interpreter-commands/Main.hs
- + testsuite/tests/ghci/prog-mhu008/Makefile
- + testsuite/tests/ghci/prog-mhu008/a/M.hs
- + testsuite/tests/ghci/prog-mhu008/all.T
- + testsuite/tests/ghci/prog-mhu008/b/M.hs
- + testsuite/tests/ghci/prog-mhu008/prog-mhu008.script
- + testsuite/tests/ghci/prog-mhu008/prog-mhu008.stdout
- + testsuite/tests/ghci/prog-mhu008/unitA
- + testsuite/tests/ghci/prog-mhu008/unitB
- testsuite/tests/ghci/prog020/ghci.prog020a.stderr
- testsuite/tests/ghci/prog020/ghci.prog020b.stderr
- testsuite/tests/ghci/prog021/prog021a.stderr
- testsuite/tests/ghci/prog021/prog021b.stderr
- testsuite/tests/ghci/prog022/ghci.prog022a.stderr
- testsuite/tests/ghci/prog022/ghci.prog022b.stderr
- testsuite/tests/ghci/prog022/ghci.prog022c.stderr
- testsuite/tests/ghci/prog022/ghci.prog022d.stderr
- testsuite/tests/ghci/prog022/ghci.prog022e.stderr
- testsuite/tests/ghci/prog022/ghci.prog022f.stderr
- testsuite/tests/ghci/scripts/Defer02.stderr
- + testsuite/tests/ghci/scripts/T27641.script
- + testsuite/tests/ghci/scripts/T27641.stderr
- + testsuite/tests/ghci/scripts/T27641.stdout
- + testsuite/tests/ghci/scripts/T27834.hs
- + testsuite/tests/ghci/scripts/T27834.script
- + testsuite/tests/ghci/scripts/T27834.stdout
- testsuite/tests/ghci/scripts/all.T
- testsuite/tests/haddock/should_compile_flag_haddock/T17544.stderr
- testsuite/tests/haddock/should_compile_flag_haddock/T17544_kw.stderr
- testsuite/tests/haddock/should_compile_flag_haddock/T24221.stderr
- testsuite/tests/hiefile/should_run/T24544.hs
- testsuite/tests/interface-stability/base-exports.stdout
- testsuite/tests/interface-stability/base-exports.stdout-javascript-unknown-ghcjs
- testsuite/tests/interface-stability/base-exports.stdout-mingw32
- testsuite/tests/module/mod130.stderr
- testsuite/tests/module/mod185.stderr
- + testsuite/tests/numeric/should_run/T27545.hs
- + testsuite/tests/numeric/should_run/T27545.stdout
- testsuite/tests/numeric/should_run/all.T
- testsuite/tests/overloadedstrings/should_fail/T25926.stderr
- testsuite/tests/overloadedstrings/should_fail/T27124.stderr
- testsuite/tests/package/T4806a.stderr
- testsuite/tests/package/all.T
- testsuite/tests/parser/should_compile/DumpParsedAst.stderr
- testsuite/tests/parser/should_compile/DumpParsedAstComments.stderr
- testsuite/tests/parser/should_compile/DumpRenamedAst.stderr
- testsuite/tests/parser/should_compile/DumpSemis.stderr
- testsuite/tests/parser/should_compile/KindSigs.stderr
- testsuite/tests/parser/should_compile/T14189.stderr
- testsuite/tests/parser/should_compile/T15279.stderr
- testsuite/tests/parser/should_compile/T15323.stderr
- testsuite/tests/parser/should_compile/T20452.stderr
- testsuite/tests/parser/should_compile/T20718.stderr
- testsuite/tests/parser/should_compile/T20718b.stderr
- testsuite/tests/parser/should_compile/T20846.stderr
- testsuite/tests/parser/should_compile/T23315/T23315.stderr
- testsuite/tests/patsyn/should_fail/records-nofieldselectors.stderr
- + testsuite/tests/perf/compiler/T25763.stderr
- testsuite/tests/perf/compiler/all.T
- + testsuite/tests/plugins/T26761.hs
- + testsuite/tests/plugins/T26761.stderr
- + testsuite/tests/plugins/T26761Plugin.hs
- + testsuite/tests/plugins/T26761a.hs
- testsuite/tests/plugins/all.T
- + testsuite/tests/primops/should_run/T27299.hs
- + testsuite/tests/primops/should_run/T27299.stdout
- testsuite/tests/primops/should_run/all.T
- testsuite/tests/printer/AnnotationNoListTuplePuns.stdout
- testsuite/tests/printer/T17697.stderr
- testsuite/tests/printer/T18791.stderr
- testsuite/tests/printer/Test20297.stdout
- testsuite/tests/printer/Test24533.stdout
- testsuite/tests/profiling/should_run/caller-cc/CallerCcParser.hs
- testsuite/tests/rename/should_fail/T15607.stderr
- testsuite/tests/rename/should_fail/T19843b.stderr
- + testsuite/tests/rts/T24450.hs
- + testsuite/tests/rts/T24450.stderr
- testsuite/tests/rts/all.T
- testsuite/tests/rts/rtsflags001.stderr-ws-32
- testsuite/tests/rts/rtsflags001.stderr-ws-64
- testsuite/tests/safeHaskell/ghci/p4.stderr
- testsuite/tests/safeHaskell/ghci/p6.stderr
- testsuite/tests/simplCore/should_compile/T22611.hs
- testsuite/tests/simplCore/should_compile/T22611.stderr
- + testsuite/tests/simplCore/should_compile/T22611a.hs
- + testsuite/tests/simplCore/should_compile/T7803.hs
- + testsuite/tests/simplCore/should_compile/T7803.stderr
- + testsuite/tests/simplCore/should_compile/T7803a.hs
- testsuite/tests/simplCore/should_compile/all.T
- testsuite/tests/typecheck/should_compile/T11254.stderr
- testsuite/tests/typecheck/should_compile/T15370.stderr
- + testsuite/tests/typecheck/should_compile/T27916.hs
- testsuite/tests/typecheck/should_compile/all.T
- testsuite/tests/typecheck/should_compile/valid_hole_fits.stderr
- testsuite/tests/typecheck/should_fail/ClassOperator.stderr
- testsuite/tests/typecheck/should_fail/DoExpansion2.stderr
- testsuite/tests/typecheck/should_fail/DoExpansion3.stderr
- testsuite/tests/typecheck/should_fail/FunDepOrigin1b.hs
- testsuite/tests/typecheck/should_fail/T18851.stderr
- + testsuite/tests/typecheck/should_fail/T18851e.hs
- + testsuite/tests/typecheck/should_fail/T18851e.stderr
- + testsuite/tests/typecheck/should_fail/T18851f.hs
- + testsuite/tests/typecheck/should_fail/T18851f.stderr
- testsuite/tests/typecheck/should_fail/T22940.stderr
- testsuite/tests/typecheck/should_fail/T26823.stderr
- + testsuite/tests/typecheck/should_fail/T27731.hs
- + testsuite/tests/typecheck/should_fail/T27731.stderr
- testsuite/tests/typecheck/should_fail/T5691.stderr
- testsuite/tests/typecheck/should_fail/T7437.stderr
- testsuite/tests/typecheck/should_fail/all.T
- testsuite/tests/typecheck/should_fail/mc24.stderr
- testsuite/tests/typecheck/should_fail/tcfail019.stderr
- testsuite/tests/typecheck/should_fail/tcfail189.stderr
- + testsuite/tests/warnings/should_fail/WExpectedNoArgError.stderr
- testsuite/tests/warnings/should_fail/all.T
- utils/check-exact/ExactPrint.hs
- utils/check-exact/Main.hs
- utils/check-exact/Transform.hs
- utils/haddock/haddock-api/src/Haddock/Backends/Hyperlinker/Renderer.hs
The diff was not included because it is too large.
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/2540e67aeedc9174b28a365af2a451…
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/2540e67aeedc9174b28a365af2a451…
You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help
1
0
[Git][ghc/ghc][wip/romes/async-rpc] interp: spawn fillPipe thread, writePipe now safely interruptible
by Andrea Vezzosi (@trac-Saizan) 20 Oct '26
by Andrea Vezzosi (@trac-Saizan) 20 Oct '26
20 Oct '26
Andrea Vezzosi pushed to branch wip/romes/async-rpc at Glasgow Haskell Compiler / GHC
Commits:
beb35d96 by Andrea Vezzosi at 2026-10-20T13:56:59+02:00
interp: spawn fillPipe thread, writePipe now safely interruptible
Previously an async exception during a call to `writePipe` could have caused a
partial write, making any further write impossible to parse from the other end.
Now `writePipe` merely pushes the message onto a queue, and the actual write
happens in a separate `fillPipe` thread. As a result of this and previous
changes, `interpCmd` no longer needs `interruptibleMask` to preserve the state
of the protocol.
`writePipe` waits for the actual write to complete, and rethrows exceptions
raised in the write attempt, if any.
- - - - -
2 changed files:
- compiler/GHC/Runtime/Interpreter.hs
- libraries/ghci/GHCi/Message/Async.hs
Changes:
=====================================
compiler/GHC/Runtime/Interpreter.hs
=====================================
@@ -197,9 +197,7 @@ interpCmd interp msg = case interpInstance interp of
InternalInterp -> run msg -- Just run it directly
#endif
ExternalInterp ext -> withExtInterp ext $ \inst ->
- uninterruptibleMask_ $ -- Note [uninterruptibleMask_ and interpCmd]
- sendMessage inst msg
-
+ sendMessage inst msg
withExtInterp :: ExceptionMonad m => ExtInterp -> (forall d. ExtInterpInstance d -> m a) -> m a
withExtInterp ext action = case ext of
@@ -213,15 +211,6 @@ withExtInterpStatus ext action = case ext of
ExtWasm i -> action $ interpStatus i
ExtIServ i -> action (interpStatus i)
--- Note [uninterruptibleMask_ and interpCmd]
--- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
--- If we receive an async exception, such as ^C, while communicating
--- with the iserv process then we will be out-of-sync and not be able
--- to recover. Thus we use uninterruptibleMask_ during
--- communication. A ^C will be delivered to the iserv process (because
--- signals get sent to the whole process group) which will interrupt
--- the running computation and return an EvalException result.
-
-- | Grab a lock on the 'IServ' and do something with it.
-- Overloaded because this is used from TcM as well as IO.
withIServ
=====================================
libraries/ghci/GHCi/Message/Async.hs
=====================================
@@ -77,6 +77,10 @@ data Pipe = Pipe
, pipeReadClosed :: !(TVar Bool)
-- ^ After the pipe is closed, messages can no longer be received from the
-- other end, but the 'AsyncManager' queues might not be empty yet.
+ , pipeWriteClosed :: !(TVar Bool)
+ -- ^ After the write end of the pipe is closed we can no longer submit
+ -- requests or respond to ones we receive, but we might still receive
+ -- replies.
}
-- | Make a 'Pipe' from a 'Handle' to read and a 'Handle' to write.
@@ -101,6 +105,7 @@ mkPipeFromContinuations getSome putAll pipeCloseRead pipeCloseWrite = mdo
pipeLeftovers <- newIORef Nothing
pipeLock <- newMVar ()
pipeReadClosed <- newTVarIO False
+ pipeWriteClosed <- newTVarIO False
mdo
pipeAsyncMngr <- newAsyncManager p
let p = Pipe { .. }
@@ -114,20 +119,34 @@ mkPipeFromContinuations getSome putAll pipeCloseRead pipeCloseWrite = mdo
-- write. Only then do we stop the drain thread, which closes the read end
-- itself once its read has finished. On Windows its read can't be
-- interrupted, so this waits for the other side to close its write end too.
+-- Finally we stop the write thread, which could be blocked waiting for more to write.
closePipe :: Pipe -> IO ()
closePipe p@Pipe{..} = do
readAlreadyClosed <- atomically $ do
closed <- readTVar pipeReadClosed
failPendingReplies p (toException PipeReadClosedException)
pure closed
- withLock p pipeCloseWrite `finally`
- unless readAlreadyClosed
- (throwTo (asyncDrainTid pipeAsyncMngr) PipeReadClosedException)
+
+ writeAlreadyClosed <- atomically $ do
+ closed <- readTVar pipeWriteClosed
+ failPendingWrites p (toException PipeWriteClosedException)
+ pure closed
+
+ unless writeAlreadyClosed (withLock p pipeCloseWrite)
+ `finally` do
+ unless readAlreadyClosed
+ (throwTo (asyncDrainTid pipeAsyncMngr) PipeReadClosedException)
+ unless writeAlreadyClosed
+ (throwTo (asyncFillTid pipeAsyncMngr) PipeWriteClosedException)
-- | We've closed the pipe read end on this process
data PipeReadClosedException = PipeReadClosedException deriving Show
instance Exception PipeReadClosedException
+-- | We've closed the pipe write end on this process
+data PipeWriteClosedException = PipeWriteClosedException deriving Show
+instance Exception PipeWriteClosedException
+
--------------------------------------------------------------------------------
-- ** Pipe thread-safe read/write
--------------------------------------------------------------------------------
@@ -135,7 +154,14 @@ instance Exception PipeReadClosedException
-- and identify whether they are requests or replies.
writePipe :: Pipe -> IddMsg -> IO ()
-writePipe p@Pipe{..} = withLock p . putAll . execPut . put
+writePipe Pipe{..} msg = do
+ var <- newEmptyTMVarIO
+ atomically $ do
+ readTVar pipeWriteClosed >>= \case
+ True -> throwSTM PipeWriteClosedException
+ False -> pure ()
+ writeTQueue (writeQueue pipeAsyncMngr) (msg, var)
+ either throwIO pure =<< atomically (takeTMVar var)
-- | Only the 'drainPipe' thread should read from the pipe directly. Other
-- functions should wait for the matching reply to be delivered asynchronously.
@@ -229,6 +255,10 @@ handleAsyncReq pipe getit mii k = mask $ \restore -> do
-- Block waiting for request
mbreq <- atomically $ do
+ readTVar (pipeWriteClosed pipe) >>= \case
+ -- We will not be able to write the reply, so no point in continuiting.
+ True -> throwSTM PipeWriteClosedException
+ False -> pure ()
isClosed <- readTVar (pipeReadClosed pipe)
taken <- case mii of
Nothing -> tryReadTQueue $ requestsQueue am
@@ -320,7 +350,10 @@ data AsyncManager = AsyncManager
, mailboxQueues :: !(TVar (IntMap (TQueue (CorrelationId, LB.ByteString))))
-- ^ Requests for a specific 'InboxId' that can be read using
-- @'handleAsyncReq' p get (Just <mailboxid>)@.
+ , writeQueue :: !(TQueue (IddMsg, TMVar (Either SomeException ())))
+ -- ^ Messages to write on the pipe.
, asyncDrainTid :: !ThreadId
+ , asyncFillTid :: !ThreadId
, asyncFreshId :: !(IORef Int)
}
@@ -363,11 +396,16 @@ newAsyncManager pipe = do
pendingReply <- newTVarIO IntMap.empty
requestsQueue <- newTQueueIO
mailboxQueues <- newTVarIO IntMap.empty
+ writeQueue <- newTQueueIO
-- Fork thread to drain read end.
-- (cleaned up on 'closePipe')
asyncDrainTid <- forkIOWithUnmask (drainPipe pipe)
+ -- Fork thread to fill write end.
+ -- (cleaned up on 'closePipe')
+ asyncFillTid <- forkIOWithUnmask (fillPipe pipe)
+
let am = AsyncManager{..}
pure am
@@ -417,6 +455,30 @@ drainPipe p unmask = catch (unmask $ loop `finally` pipeCloseRead p) replyToPend
am = pipeAsyncMngr p
+
+-- | "Fill"s write end of the Pipe with queues messages.
+--
+-- Because Pipe is written to from this thread rather than 'writePipe', calls to
+-- the latter can be interrupted without compromising the protocol.
+--
+-- Closes the Pipe write end when stopping, notifying any messages still queued to write.
+fillPipe :: Pipe -> (forall x. IO x -> IO x) -> IO ()
+fillPipe p unmask = catch (loop `finally` pipeCloseWrite p) (atomically . failPendingWrites p) where
+ loop = do
+ (msg,var) <- atomically $ readTQueue (writeQueue am)
+ unmask (withLock p (putAll p . execPut . put $ msg))
+ `catch` \ e -> atomically (putTMVar var (Left e)) >> throwIO e
+ atomically $ putTMVar var (Right ())
+ loop
+
+ am = pipeAsyncMngr p
+
+failPendingWrites :: Pipe -> SomeException -> STM ()
+failPendingWrites p e = do
+ writeTVar (pipeWriteClosed p) True
+ writes <- flushTQueue (writeQueue (pipeAsyncMngr p))
+ mapM_ (`writeTMVar` Left e) (map snd writes)
+
failPendingReplies :: Pipe -> SomeException -> STM ()
failPendingReplies p e = do
-- reqs are only added if pipe not closed;
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/beb35d96d1287ae7f15841043d63466…
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/beb35d96d1287ae7f15841043d63466…
You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help
1
0
[Git][ghc/ghc][wip/fendor/dep-stdout] Add support for streaming `ghc -M` to stdout
by Hannes Siebenhandl (@fendor) 20 Oct '26
by Hannes Siebenhandl (@fendor) 20 Oct '26
20 Oct '26
Hannes Siebenhandl pushed to branch wip/fendor/dep-stdout at Glasgow Haskell Compiler / GHC
Commits:
419cfed9 by fendor at 2026-10-20T14:01:13+02:00
Add support for streaming `ghc -M` to stdout
Adds the flags `-dep-json-stdout` and `-dep-makefile-stdout` which
streams dependency information line-by-line to stdout.
Each dependency generation can be redirected to separate outputs.
Providing any extra `-dep-{json,makefile}[-stdout]` disables the default
generation of the `Makefile` file.
Allows consumers to start processing the dependency information as it
comes in and not waiting for the dependency generation to finish.
`DepStream` abstracts over the possible output handles, which is either
a file or the stdout handle.
Writing to a file requires temporary file paths, thus `DepStream` must
provide a way to perform clean up when dependency processing finished or
when an error is thrown for any reason.
We further decouple the rendering and writing of the result from the
dependency generation, localising side effects and making the code
slightly more readable.
The testsuite now also handles `ghc -M` writing to stdout.
We add a test for each combination of writing to stdout and file,
except for writing both makefile and json.
While it probably works, is really undefined behaviour.
Closes #27937
- - - - -
15 changed files:
- + changelog.d/make-depends-streaming-output
- compiler/GHC/Driver/DynFlags.hs
- compiler/GHC/Driver/MakeFile.hs
- compiler/GHC/Driver/Session.hs
- docs/users_guide/separate_compilation.rst
- testsuite/driver/my_typing.py
- testsuite/driver/testglobals.py
- testsuite/driver/testlib.py
- testsuite/tests/driver/mkdepend/all.T
- testsuite/tests/driver/mkdepend/mkdepend_mhu01.dep-json → testsuite/tests/driver/mkdepend/mkdepend_mhu01a.dep-json
- testsuite/tests/driver/mkdepend/mkdepend_mhu01.dep-makefile → testsuite/tests/driver/mkdepend/mkdepend_mhu01a.dep-makefile
- + testsuite/tests/driver/mkdepend/mkdepend_mhu01b.stderr
- + testsuite/tests/driver/mkdepend/mkdepend_mhu01c.stderr
- + testsuite/tests/driver/mkdepend/mkdepend_mhu01d.dep-makefile
- + testsuite/tests/driver/mkdepend/mkdepend_mhu01d.stderr
Changes:
=====================================
changelog.d/make-depends-streaming-output
=====================================
@@ -0,0 +1,16 @@
+section: compiler
+synopsis: Allow streaming `ghc -M` output to stdout
+issues: #27937
+mrs: !16830
+description: {
+ Add options `-dep-makefile-stdout` and `-dep-json-stdout` which stream the dependency generation
+ output to stdout.
+
+ The Makefile and json dependency generation can stream to separate targets, e.g. `-dep-makefile target.Makefile -dep-json-stdout`
+ streams the JSON Lines output to stdout and the Makefile to `target.Makefile`.
+
+ If `-dep-makefile-stdout`, `-dep-json` or `-dep-json-stdout` is given, we no longer generate a Makefile by default, but only
+ if the user explicitly requests the Makefile dependency generation.
+
+ The output from a particular backend can be disabled via `-no-dep-json` and `-no-dep-makefile` respectively.
+}
=====================================
compiler/GHC/Driver/DynFlags.hs
=====================================
@@ -73,6 +73,9 @@ module GHC.Driver.DynFlags (
IncludeSpecs(..), addGlobalInclude, addQuoteInclude, flattenIncludes,
addImplicitQuoteInclude,
+ -- * `ghc -M` specification
+ MkDependOutput(..),
+
-- * SDoc
initSDocContext, initDefaultSDocContext,
initPromotionTickContext,
@@ -350,8 +353,8 @@ data DynFlags = DynFlags {
-- ^ External plugins loaded from shared libraries
-- For ghc -M
- depMakefile :: Maybe FilePath,
- depJson :: Maybe FilePath,
+ depMakefile :: MkDependOutput,
+ depJson :: MkDependOutput,
depIncludePkgDeps :: Bool,
depIncludeCppDeps :: Bool,
depExcludeMods :: [ModuleName],
@@ -694,8 +697,8 @@ defaultDynFlags mySettings =
llvmOptLevel = 0,
-- ghc -M values
- depMakefile = Just "Makefile",
- depJson = Nothing,
+ depMakefile = MkDepend_Default "Makefile",
+ depJson = MkDepend_None,
depIncludePkgDeps = False,
depIncludeCppDeps = False,
depExcludeMods = [],
@@ -801,6 +804,15 @@ flattenExtensionFlags ml = foldr g defaultExtensionFlags
g (Off f) flags = EnumSet.delete f flags
defaultExtensionFlags = EnumSet.fromList (languageExtensions ml)
+-- -----------------------------------------------------------------------------
+-- -M -dep-*
+
+data MkDependOutput
+ = MkDepend_None
+ | MkDepend_Default !FilePath
+ | MkDepend_Explicit !FilePath
+ | MkDepend_StdOut
+
-- -----------------------------------------------------------------------------
-- -jN
=====================================
compiler/GHC/Driver/MakeFile.hs
=====================================
@@ -132,7 +132,7 @@ doMkDepend = do
module_graph <- GHC.depanal excl_mods True {- Allow dup roots -}
doMkDependModuleGraph dflags module_graph
-doMkDependModuleGraph :: GhcMonad m => DynFlags -> ModuleGraph -> m ()
+doMkDependModuleGraph :: GhcMonad m => DynFlags -> ModuleGraph -> m ()
doMkDependModuleGraph dflags module_graph = do
hsc_env <- getSession
logger <- getLogger
@@ -143,12 +143,14 @@ doMkDependModuleGraph dflags module_graph = do
root <- liftIO Directory.getCurrentDirectory
let
+ suffixes = fmap unsafeEncodeUtf (depSuffixes dflags)
+ tmpdir = tmpDir dflags
backends = concat
- [ [ initFileDepWriter logger root tmpfs dflags makefile_output
- | Just makefile_output <- [depMakefile dflags]
+ [ [ initFileDepWriter logger root tmpfs tmpdir suffixes makefile_output
+ | Just makefile_output <- [initDepOutput (depMakefile dflags)]
]
- , [ initJsonDepWriter logger tmpfs dflags json_output
- | Just json_output <- [depJson dflags]
+ , [ initJsonDepWriter logger tmpfs tmpdir json_output
+ | Just json_output <- [initDepOutput (depJson dflags)]
]
]
@@ -158,24 +160,14 @@ doMkDependModuleGraph dflags module_graph = do
liftIO $ debugTraceMsg logger 2 (text "Module dependencies" $$ ppr sorted)
liftIO $ do
-
- -- Setup the writers
- writers <- sequenceA backends
- sinks <- traverse dw_beginWriter writers
-
- -- Do the actual work
- do
+ withWriters backends $ \ depSink -> do
+ -- Do the actual work
-- Process them one by one, dumping results and complaining about cycles
- mapM_ (processDeps hsc_env (UniqSet.mkUniqSet excl_mods) sinks) sorted
+ mapM_ (processDeps hsc_env (UniqSet.mkUniqSet excl_mods) depSink) sorted
-- If -ddump-mod-cycles, show cycles in the module graph
liftIO $ dumpModCycles logger module_graph
- -- Tidy up
- do
- traverse_ dw_endWriter writers
-
-
-- Unconditional exiting is a bad idea. If an error occurs we'll get an
--exception; if that is not caught it's fine, but at least we have a
--chance to find out exactly what went wrong. Uncomment the following
@@ -216,15 +208,119 @@ data Dep
{ dep_path :: OsPath
}
+-- | Consumer of the dependency information generated from the 'ModuleGraph'.
+--
+-- Use with 'withWriter'/'withWriters' to initialise the consumers of the dependency information
+-- generation.
data DependencyWriter = DependencyWriter
- { dw_beginWriter :: IO DepSink
+ { dw_sink :: DepSink
+ -- ^ Sink for dependency information
, dw_endWriter :: IO ()
+ -- ^ We finished consuming the dependency information, perform final
+ -- tidy up operations. This may be a no-op, but could also
+ -- copy temporary files, close open handlers, etc...
+ --
+ -- If 'dw_endWriter' was called, calling 'dw_onError' is unnecessary,
+ -- but it must be safe to call 'dw_onError' any way.
+ , dw_onError :: IO ()
+ -- ^ Something crashed! Clean up any resources opened by this 'DependencyWriter'.
}
-data DepSink = DepSink
+-- | Initialise multiple backends and combine them into one output function.
+--
+-- Combines the 'DepSink' of each 'DependencyWriter' into one single consumer.
+withWriters :: [IO DependencyWriter] -> (DepSink -> IO a) -> IO a
+withWriters [] k = k noopSink
+withWriters (dw : dws) k =
+ withWriter
+ dw
+ $ \s -> withWriters dws (\s' -> k (s `mappendSink` s'))
+
+-- | Initialise a single 'DependencyWriter', making sure any allocated resource
+-- get's tidied up again.
+--
+-- Accepts 'IO' 'DependencyWriter' since the initialisation may perform setup operations,
+-- such as opening file handles, or intermediate data types.
+withWriter :: IO DependencyWriter -> (DepSink -> IO r) -> IO r
+withWriter dwIO k =
+ bracketOnError dwIO dw_onError $ \ dw -> do
+ r <- k (dw_sink dw)
+ -- Clean up open resources
+ dw_endWriter dw
+ pure r
+
+-- | Process the dependency information for one node in the 'ModuleGraph'.
+newtype DepSink = DepSink
{ ds_writeDependency :: DepNode -> [Dep] -> IO ()
}
+noopSink :: DepSink
+noopSink = DepSink $ \ _ _ -> pure ()
+
+mappendSink :: DepSink -> DepSink -> DepSink
+mappendSink (DepSink s1) (DepSink s2) = DepSink (\ node deps -> s1 node deps >> s2 node deps)
+
+-- | Where should the dependency output end up?
+data DepOutput
+ = DepStreamHandle Handle
+ -- ^ The dependency information should be written to this 'Handle'.
+ | DepFileOutput FilePath
+ -- ^ The dependency information should be written to this 'FilePath'.
+
+-- | A 'DepStream' is an abstraction over how to output single elements
+-- of the dependency generation, e.g., a json object or one makefile rule.
+data DepStream = DepStream
+ { depStream_write :: String -> IO ()
+ , depStream_close :: IO ()
+ , depStream_onError :: IO ()
+ }
+
+initDepOutput :: MkDependOutput -> Maybe DepOutput
+initDepOutput = \ case
+ MkDepend_None -> Nothing
+ MkDepend_Default fp -> Just $ DepFileOutput fp
+ MkDepend_Explicit fp -> Just $ DepFileOutput fp
+ MkDepend_StdOut -> Just $ DepStreamHandle stdout
+
+initMakefileDepStream :: Logger -> TmpFs -> TempDir -> DepOutput -> IO DepStream
+initMakefileDepStream _ _ _ (DepStreamHandle h) =
+ pure $ depStreamHandle h
+initMakefileDepStream logger tmpfs tmpdir (DepFileOutput fp) = do
+ files <- beginMkDependHS logger tmpfs tmpdir fp
+ pure $
+ DepStream
+ { depStream_write = hPutStrLn (mkd_tmp_hdl files)
+ , depStream_onError = do
+ hClose $ mkd_tmp_hdl files
+ case mkd_make_hdl files of
+ Nothing -> return ()
+ Just hdl -> hClose hdl
+ , depStream_close = endMkDependHS logger files
+ }
+
+initJsonDepStream :: Logger -> TmpFs -> TempDir -> DepOutput -> IO DepStream
+initJsonDepStream _ _ _ (DepStreamHandle h) =
+ pure $ depStreamHandle h
+initJsonDepStream logger tmpfs tmpdir (DepFileOutput fp) = do
+ files <- initJsonDepFiles logger tmpfs tmpdir fp
+ pure $
+ DepStream
+ { depStream_write = hPutStrLn (json_tmp_hdl files)
+ , depStream_onError = hClose (json_tmp_hdl files)
+ , depStream_close = endJsonDepFiles logger files
+ }
+
+depStreamHandle :: Handle -> DepStream
+depStreamHandle h =
+ DepStream
+ { depStream_write =
+ \output -> do
+ hPutStrLn h output
+ hFlush h -- We flush to ensure the output is streamed per line.
+ , depStream_close = pure ()
+ , depStream_onError = pure ()
+ }
+
-----------------------------------------------------------------
--
-- processDeps
@@ -233,7 +329,7 @@ data DepSink = DepSink
processDeps :: HscEnv
-> UniqSet ModuleName -- ^ Excludes
- -> [DepSink]
+ -> DepSink
-> SCC ModuleGraphNode
-> IO ()
-- Write suitable dependencies to handle
@@ -267,7 +363,7 @@ processDeps _ _ _ (AcyclicSCC (ModuleNode _ (ModuleNodeFixed {})))
-- No dependencies needed for fixed modules (already compiled)
= return ()
-processDeps hsc_env0 excl_mods sinks (AcyclicSCC (ModuleNode _ (ModuleNodeCompile node))) = do
+processDeps hsc_env0 excl_mods sink (AcyclicSCC (ModuleNode _ (ModuleNodeCompile node))) = do
pp <- preprocessor
let
dep_node = mkDepNode pp
@@ -279,7 +375,7 @@ processDeps hsc_env0 excl_mods sinks (AcyclicSCC (ModuleNode _ (ModuleNodeCompil
if null missing_dep_errs
then do
- traverse_ (\ sink -> ds_writeDependency sink dep_node deps) sinks
+ ds_writeDependency sink dep_node deps
else do
let sec = initSourceErrorContext (hsc_dflags local_hsc_env)
throwErrors sec (mkMessages (listToBag missing_dep_errs))
@@ -380,17 +476,16 @@ find_preprocessor d = do
--
-----------------------------------------------------------------
-initFileDepWriter :: Logger -> OsPath -> TmpFs -> DynFlags -> FilePath -> IO DependencyWriter
-initFileDepWriter logger root tmpfs dflags makefile = do
- files <- beginMkDependHS logger tmpfs dflags makefile
+initFileDepWriter :: Logger -> OsPath -> TmpFs -> TempDir -> [OsPath] -> DepOutput -> IO DependencyWriter
+initFileDepWriter logger root tmpfs tmpdir suffixes output = do
+ stream <- initMakefileDepStream logger tmpfs tmpdir output
pure DependencyWriter
- { dw_beginWriter = do
- pure $ DepSink $ \ node deps -> do
- writeDependencies root (mkd_tmp_hdl files) suffixes node deps
- , dw_endWriter = endMkDependHS logger files
+ { dw_sink =
+ DepSink $ \ node deps -> do
+ traverse_ (depStream_write stream) (dependencyRules root suffixes node deps)
+ , dw_endWriter = depStream_close stream
+ , dw_onError = depStream_onError stream
}
- where
- suffixes = map unsafeEncodeUtf (depSuffixes dflags)
data MkDepFiles
= MkDep { mkd_make_file :: FilePath, -- Name of the makefile
@@ -398,11 +493,11 @@ data MkDepFiles
mkd_tmp_file :: FilePath, -- Name of the temporary file
mkd_tmp_hdl :: Handle } -- Handle of the open temporary file
-beginMkDependHS :: Logger -> TmpFs -> DynFlags -> FilePath -> IO MkDepFiles
-beginMkDependHS logger tmpfs dflags makefile = do
+beginMkDependHS :: Logger -> TmpFs -> TempDir -> FilePath -> IO MkDepFiles
+beginMkDependHS logger tmpfs tmpdir makefile = do
-- open a new temp file in which to stuff the dependency info
-- as we go along.
- tmp_file <- newTempName logger tmpfs (tmpDir dflags) TFL_CurrentModule "dep"
+ tmp_file <- newTempName logger tmpfs tmpdir TFL_CurrentModule "dep"
tmp_hdl <- openFile tmp_file WriteMode
-- open the makefile
@@ -443,16 +538,15 @@ beginMkDependHS logger tmpfs dflags makefile = do
return (MkDep { mkd_make_file = makefile, mkd_make_hdl = mb_make_hdl,
mkd_tmp_file = tmp_file, mkd_tmp_hdl = tmp_hdl})
-writeDependencies ::
+dependencyRules ::
OsPath ->
- Handle ->
[OsString] ->
-- ^ Suffixes
DepNode ->
[Dep] ->
- IO ()
-writeDependencies root hdl suffixes node deps =
- traverse_ write tasks
+ [String]
+dependencyRules root suffixes node deps =
+ map rule tasks
where
tasks = source_dep : boot_dep ++ concatMap import_dep deps
@@ -479,7 +573,7 @@ writeDependencies root hdl suffixes node deps =
DepCpp {dep_path} -> [(obj_files, dep_path)]
- write (from, to) = writeDependency root hdl from to
+ rule (from, to) = dependencyRule root from to
obj_files = suffixed dn_obj
@@ -488,18 +582,18 @@ writeDependencies root hdl suffixes node deps =
DepNode {dn_src, dn_obj, dn_hi, dn_mod} = node
-----------------------------
-writeDependency :: OsPath -> Handle -> [OsPath] -> OsPath -> IO ()
--- (writeDependency r h [t1,t2] dep) writes to handle h the dependency
+dependencyRule :: OsPath -> [OsPath] -> OsPath -> String
+-- (dependencyRule r [t1,t2] dep) renders the dependency
-- t1 t2 : dep
-writeDependency root hdl targets dep
- = do let -- We need to avoid making deps on
- -- c:/foo/...
- -- on Windows as make gets confused by the :
- -- Making relative deps avoids some instances of this.
- dep' = OsPath.makeRelative root dep
- forOutput = escapeSpaces . reslash Forwards . unsafeDecodeUtf . OsPath.normalise
- output = unwords (map forOutput targets) ++ " : " ++ forOutput dep'
- hPutStrLn hdl output
+dependencyRule root targets dep
+ = let -- We need to avoid making deps on
+ -- c:/foo/...
+ -- on Windows as make gets confused by the :
+ -- Making relative deps avoids some instances of this.
+ dep' = OsPath.makeRelative root dep
+ forOutput = escapeSpaces . reslash Forwards . unsafeDecodeUtf . OsPath.normalise
+ in
+ unwords (map forOutput targets) ++ " : " ++ forOutput dep'
-----------------------------
insertSuffixes
@@ -554,16 +648,27 @@ endMkDependHS logger
--
-----------------------------------------------------------------
-initJsonDepWriter :: Logger -> TmpFs -> DynFlags -> FilePath -> IO DependencyWriter
-initJsonDepWriter logger tmpfs dflags output = do
- json_var <- mkJsonOutput logger tmpfs dflags output initDepJson
- pure DependencyWriter
- { dw_beginWriter =
- pure $ DepSink $ \ node deps ->
- writeHomeUnitDeps json_var node deps
- , dw_endWriter = do
- writeJsonOutput logger json_var
- }
+initJsonDepWriter :: Logger -> TmpFs -> TempDir -> DepOutput -> IO DependencyWriter
+initJsonDepWriter logger tmpfs tmpdir output = do
+ stream <- initJsonDepStream logger tmpfs tmpdir output
+ json_var <- mkJsonOutput
+ pure
+ DependencyWriter
+ { dw_sink =
+ DepSink $ \node deps -> do
+ extUnits <- readJson json_var
+ let
+ (extUnitsDeps, payload) = homeUnitDeps node deps
+ newExtUnits = Set.difference extUnitsDeps (externalDeps extUnits)
+ -- Write new external unit deps before the module that uses them
+ for_ newExtUnits $ \extUnit -> do
+ depStream_write stream (renderJsonl extUnit)
+ depStream_write stream (renderJsonl payload)
+ -- Cache external deps we already reported in the past
+ updateJson json_var $ cacheExternalDepsJson newExtUnits
+ , dw_endWriter = depStream_close stream
+ , dw_onError = depStream_onError stream
+ }
--------------------------------------------------------------------------------
-- Output interface for json dumps
@@ -572,65 +677,60 @@ initJsonDepWriter logger tmpfs dflags output = do
-- The flag @-dep-json@ add an additional output target for dependency
-- diagnostics.
--
--- Contains temporary file locations to stream output to.
-data JsonOutput =
- JsonOutput {
- -- | This ref is updated in @processDeps@ incrementally, using a
- -- flag-specific type.
- json_ref :: IORef DepJson,
-
- -- | The output file path specified as argument to the flag.
- json_path :: FilePath,
- -- | Name of the temporary file
- json_tmp_file :: FilePath,
- -- | Handle of the open temporary file
- json_tmp_hdl :: Handle
+data JsonOutput
+ = JsonOutput
+ { json_ref :: IORef DepJson
+ -- ^ This ref is updated in @processDeps@ incrementally, using a
+ -- flag-specific type.
}
--- | Set up json generation for `ghc -M`.
---
--- Creates a temporary file that we write the dependency information to.
--- The temporary file is written to the final output file path in 'writeJsonOutput'.
-mkJsonOutput ::
- Logger ->
- TmpFs ->
- DynFlags ->
- FilePath ->
- IO (IORef DepJson) ->
- IO JsonOutput
-mkJsonOutput logger tmpfs dflags json_path mk_ref = do
- json_ref <- mk_ref
- -- open a new temp file in which to stuff the dependency info
+data JsonDepFiles = JsonDepFiles
+ { json_path :: FilePath
+ -- ^ The output file path specified as argument to the flag.
+ , json_tmp_file :: FilePath
+ -- ^ Name of the temporary file
+ , json_tmp_hdl :: Handle
+ -- ^ Handle of the open temporary file
+ }
+
+initJsonDepFiles :: Logger -> TmpFs -> TempDir -> FilePath -> IO JsonDepFiles
+initJsonDepFiles logger tmpfs tmpdir path = do
+ -- open a new temp file in which to stuff the dependency info into
-- as we go along.
- json_tmp_file <- newTempName logger tmpfs (tmpDir dflags) TFL_CurrentModule "dep"
+ json_tmp_file <- newTempName logger tmpfs tmpdir TFL_CurrentModule "dep-json"
json_tmp_hdl <- openFile json_tmp_file WriteMode
-
- pure JsonOutput {json_ref, json_path, json_tmp_file, json_tmp_hdl}
-
-updateJson :: JsonOutput -> (DepJson -> DepJson) -> IO ()
-updateJson JsonOutput {json_ref} f = modifyIORef' json_ref f
-
-readJson :: JsonOutput -> IO DepJson
-readJson JsonOutput {json_ref} = readIORef json_ref
+ pure $ JsonDepFiles
+ { json_path = path
+ , json_tmp_file = json_tmp_file
+ , json_tmp_hdl = json_tmp_hdl
+ }
-- | Finalise the json generation by copying the temporary file to the final destination.
-- Avoids partial writes in the case of an error.
-writeJsonOutput ::
- Logger ->
- JsonOutput ->
- IO ()
-writeJsonOutput logger JsonOutput {json_path, json_tmp_file, json_tmp_hdl} = do
+endJsonDepFiles :: Logger -> JsonDepFiles -> IO ()
+endJsonDepFiles logger JsonDepFiles {json_path, json_tmp_file, json_tmp_hdl} = do
hClose json_tmp_hdl
-- Copy the new jsonl in place
showPass logger "Installing new json output"
SysTools.copyFile json_tmp_file json_path
+-- | Set up json generation for `ghc -M`.
+mkJsonOutput :: IO JsonOutput
+mkJsonOutput =
+ JsonOutput <$> newIORef (DepJson Set.empty)
+
+updateJson :: JsonOutput -> (DepJson -> DepJson) -> IO ()
+updateJson JsonOutput {json_ref} f = modifyIORef' json_ref f
+
+readJson :: JsonOutput -> IO DepJson
+readJson JsonOutput {json_ref} = readIORef json_ref
+
--------------------------------------------------------------------------------
-- Output helpers
-writeJsonl :: ToJson a => JsonOutput -> a -> IO ()
-writeJsonl JsonOutput {json_tmp_hdl} output = do
- hPutStrLn json_tmp_hdl (showSDocUnsafe $ renderJSON $ json output)
+renderJsonl :: ToJson a => a -> String
+renderJsonl output =
+ showSDocUnsafe $ renderJSON $ json output
--------------------------------------------------------------------------------
-- Payload for -dep-json
@@ -788,9 +888,6 @@ instance ToJson ExtUnitDep where
where
showPackageId (DPackageId (PackageId pid)) = unpackFS pid
-initDepJson :: IO (IORef DepJson)
-initDepJson = newIORef $ DepJson Set.empty
-
cacheExternalDepsJson :: Set.Set ExtUnitDep -> DepJson -> DepJson
cacheExternalDepsJson extUnits (DepJson e0) =
DepJson
@@ -798,19 +895,10 @@ cacheExternalDepsJson extUnits (DepJson e0) =
Set.union e0 extUnits
}
-writeHomeUnitDeps :: JsonOutput -> DepNode -> [Dep] -> IO ()
-writeHomeUnitDeps deps_json DepNode {..} deps = do
- extUnits <- readJson deps_json
- let newExtUnits = Set.difference extUnitsDeps (externalDeps extUnits)
- -- Write new external unit deps before the module that uses them
- for_ newExtUnits $ \ extUnit -> do
- writeJsonl deps_json extUnit
- writeJsonl deps_json payload
- -- Cache external deps we already reported in the past
- updateJson deps_json $ cacheExternalDepsJson newExtUnits
+homeUnitDeps :: Foldable t => DepNode -> t Dep -> (Set.Set ExtUnitDep, HomeUnitDep)
+homeUnitDeps DepNode {..} deps =
+ foldl' go (Set.empty, initial_node_data) deps
where
- (extUnitsDeps, payload) = foldl' go (Set.empty, initial_node_data) deps
-
initial_node_data =
HomeUnitDep
{ unitId = dn_unitId
=====================================
compiler/GHC/Driver/Session.hs
=====================================
@@ -790,16 +790,44 @@ addOptCmmP f = alterToolSettings $ \s -> s
}
setDepMakefile :: FilePath -> DynFlags -> DynFlags
-setDepMakefile f d = d { depMakefile = Just f }
+setDepMakefile f d = d { depMakefile = MkDepend_Explicit f
+ -- depJson has no default value, we add this for symmetry with `setDepJson`
+ -- and if anything changes in the future, then the code is already here :)
+ , depJson = unsetMkDependOutputIfDefault (depJson d)
+ }
+
+setDepMakefileStdout :: DynFlags -> DynFlags
+setDepMakefileStdout d = d { depMakefile = MkDepend_StdOut
+ -- depJson has no default value, we add this for symmetry with `setDepJson`
+ -- and if anything changes in the future, then the code is already here :)
+ , depJson = unsetMkDependOutputIfDefault (depJson d)
+ }
unsetDepMakefile :: DynFlags -> DynFlags
-unsetDepMakefile d = d { depMakefile = Nothing }
+unsetDepMakefile d = d { depMakefile = MkDepend_None }
setDepJson :: FilePath -> DynFlags -> DynFlags
-setDepJson f d = d { depJson = Just f }
+setDepJson f d = d { depJson = MkDepend_Explicit f
+ -- We generate a Makefile by default, but if a user requests json
+ -- output, then they most likely don't need a Makefile.
+ -- Thus, we only generate it if the user requests it.
+ , depMakefile = unsetMkDependOutputIfDefault (depMakefile d)
+ }
+
+setDepJsonStdout :: DynFlags -> DynFlags
+setDepJsonStdout d = d { depJson = MkDepend_StdOut
+ -- We generate a Makefile by default, but if a user requests json
+ -- output, then they most likely don't need a Makefile.
+ -- Thus, we only generate it if the user requests it.
+ , depMakefile = unsetMkDependOutputIfDefault (depMakefile d)
+ }
unsetDepJson :: DynFlags -> DynFlags
-unsetDepJson d = d { depJson = Nothing }
+unsetDepJson d = d { depJson = MkDepend_None }
+
+unsetMkDependOutputIfDefault :: MkDependOutput -> MkDependOutput
+unsetMkDependOutputIfDefault (MkDepend_Default _) = MkDepend_None
+unsetMkDependOutputIfDefault mkDepend_output = mkDepend_output
setDepIncludeCppDeps :: Bool -> DynFlags -> DynFlags
setDepIncludeCppDeps b d = d { depIncludeCppDeps = b }
@@ -1272,8 +1300,10 @@ dynamic_flags_deps = [
, make_ord_flag defGhcFlag "dep-suffix" (hasArg addDepSuffix)
, make_ord_flag defGhcFlag "dep-makefile" (hasArg setDepMakefile)
, make_ord_flag defGhcFlag "no-dep-makefile" (noArg unsetDepMakefile)
+ , make_ord_flag defGhcFlag "dep-makefile-stdout" (noArg setDepMakefileStdout)
, make_ord_flag defGhcFlag "dep-json" (hasArg setDepJson)
, make_ord_flag defGhcFlag "no-dep-json" (noArg unsetDepJson)
+ , make_ord_flag defGhcFlag "dep-json-stdout" (noArg setDepJsonStdout)
, make_ord_flag defGhcFlag "include-cpp-deps"
(noArg (setDepIncludeCppDeps True))
, make_ord_flag defGhcFlag "include-pkg-deps"
=====================================
docs/users_guide/separate_compilation.rst
=====================================
@@ -1511,6 +1511,23 @@ generation are:
Introduced in GHC 10.1, ``-no-dep-makefile`` disables the generation of a Makefile altogether.
+.. ghc-flag:: -dep-makefile-stdout
+ :shortdesc: Write dependency generation to stdout using makefile format
+ :type: dynamic
+ :reverse: -no-dep-makefile
+ :category: redirect-output
+
+ :since: 10.1
+
+ Stream the dependency generation of :ghc-flag:`-dep-makefile ⟨file⟩` to stdout instead of ⟨file⟩.
+
+ Contrary to :ghc-flag:`-dep-makefile ⟨file⟩`, the markers
+
+ * "``# DO NOT DELETE: Beginning of Haskell dependencies``" and
+ * "``# DO NOT DELETE: End of Haskell dependencies``"
+
+ are not emitted.
+
.. ghc-flag:: -dep-json ⟨file⟩
:shortdesc: Write dependencies to ⟨file⟩ as a JSON Lines format
:type: dynamic
@@ -1540,8 +1557,19 @@ generation are:
``-no-dep-json`` disables the generation of the JSON file.
- By default, :ghc-flag:`-dep-makefile ⟨file⟩` is enabled even if :ghc-flag:`-dep-json ⟨file⟩` is given.
- Disable the generation of any Makefile via ``-no-dep-makefile``.
+ :ghc-flag:`-dep-json ⟨file⟩` and :ghc-flag:`-dep-json-stdout` disable the default generation of a Makefile.
+ To generate a Makefile for the dependency information, explicitly use :ghc-flag:`-dep-makefile ⟨file⟩` or
+ :ghc-flag:`-dep-makefile-stdout`.
+
+.. ghc-flag:: -dep-json-stdout
+ :shortdesc: Write dependency generation to stdout using JSON Lines format
+ :type: dynamic
+ :reverse: -no-dep-json
+ :category: redirect-output
+
+ :since: 10.1
+
+ Stream the dependency generation of :ghc-flag:`-dep-json ⟨file⟩` to stdout instead of ⟨file⟩.
.. ghc-flag:: -dep-suffix ⟨suffix⟩
:shortdesc: Make dependencies that declare that files with suffix
=====================================
testsuite/driver/my_typing.py
=====================================
@@ -10,7 +10,7 @@ The testsuite driver can be typechecked using mypy [1].
from typing import *
import typing
-
+import enum
####################################################
# Backwards compatibility shims
@@ -34,3 +34,7 @@ GitHash = NewType("GitHash", str) # a Git commit hash
GitRef = NewType("GitRef", str)
TestEnv = NewType("TestEnv", str)
MetricName = NewType("MetricName", str)
+
+# Used by `ghc -M` test mode
+DepMode = enum.Enum('DepMode', [('NONE', 0), ('FILE', 1), ('STDOUT', 2)])
+
=====================================
testsuite/driver/testglobals.py
=====================================
@@ -390,8 +390,8 @@ class TestOptions:
self.ignore_extension = False
# make_depend tests: which outputs to write and check
- self.dep_makefile = False
- self.dep_json = False
+ self.dep_makefile: DepMode = DepMode.NONE
+ self.dep_json: DepMode = DepMode.NONE
self.ignore_dep_outputs = False
# Backpack test
=====================================
testsuite/driver/testlib.py
=====================================
@@ -438,14 +438,26 @@ def dep_makefile( name, opts ):
For make_depend tests: write the -dep-makefile output and
compare it against <name>.dep-makefile.
"""
- opts.dep_makefile = True
+ opts.dep_makefile = DepMode.FILE
def dep_json( name, opts ):
"""
For make_depend tests: write the -dep-json output and
compare it against <name>.dep-json.
"""
- opts.dep_json = True
+ opts.dep_json = DepMode.FILE
+
+def dep_makefile_stdout( name, opts ):
+ """
+ For make_depend tests: write the -dep-makefile output to stdout.
+ """
+ opts.dep_makefile = DepMode.STDOUT
+
+def dep_json_stdout( name, opts ):
+ """
+ For make_depend tests: write the -dep-json output to stdout.
+ """
+ opts.dep_json = DepMode.STDOUT
def ignore_dep_outputs( name, opts ):
"""
@@ -2060,14 +2072,19 @@ async def do_make_depend(name: TestName,
outputs = []
output_flags = []
- if opts.dep_makefile:
+ if opts.dep_makefile == DepMode.FILE:
outputs.append(('dep-makefile', normalise_dep_makefile))
output_flags.append('-dep-makefile %s' % actual_file('dep-makefile'))
- else:
- output_flags.append('-no-dep-makefile')
- if opts.dep_json:
+ elif opts.dep_makefile == DepMode.STDOUT:
+ outputs.append(('stdout', normalise_dep_makefile))
+ output_flags.append('-dep-makefile-stdout')
+
+ if opts.dep_json == DepMode.FILE:
outputs.append(('dep-json', normalise_dep_json))
output_flags.append('-dep-json %s' % actual_file('dep-json'))
+ elif opts.dep_json == DepMode.STDOUT:
+ outputs.append(('stdout', normalise_dep_json))
+ output_flags.append('-dep-json-stdout')
result = await do_compile( name, way, False, None, [], units,
' '.join(output_flags + [extra_hc_opts]),
@@ -2077,9 +2094,9 @@ async def do_make_depend(name: TestName,
for (kind, normaliser) in outputs:
output_match = await compare_outputs(way, kind,
- join_normalisers(opts.extra_normaliser,
- normalise_slashes_, normaliser),
- find_expected_file(name, kind, way), actual_file(kind))
+ join_normalisers(opts.extra_normaliser, normalise_slashes_, normaliser),
+ find_expected_file(name, kind, way),
+ actual_file(kind))
if not output_match:
return failBecause(kind + ' mismatch', diff=output_match.diff)
=====================================
testsuite/tests/driver/mkdepend/all.T
=====================================
@@ -15,9 +15,20 @@ test('T24384b', [ *T24384_extras, when(opsys('mingw32'), skip), *json_normaliser
test('T24384c', [ *T24384_extras, when(opsys('mingw32'), skip), *json_normalisers ], makefile_test, [])
# Multiple home units with explicit level imports.
-test('mkdepend_mhu01',
+test('mkdepend_mhu01a',
[ extra_files(['mhu01/']), dep_makefile, dep_json ],
make_depend_multiunit, [['mhu01/unitA', 'mhu01/unitB', 'mhu01/unitC'], ''])
+# test the output streaming and writing to files
+test('mkdepend_mhu01b',
+ [ extra_files(['mhu01/']), dep_json_stdout ],
+ make_depend_multiunit, [['mhu01/unitA', 'mhu01/unitB', 'mhu01/unitC'], ''])
+test('mkdepend_mhu01c',
+ [ extra_files(['mhu01/']), dep_makefile_stdout ],
+ make_depend_multiunit, [['mhu01/unitA', 'mhu01/unitB', 'mhu01/unitC'], ''])
+test('mkdepend_mhu01d',
+ [ extra_files(['mhu01/']), dep_json_stdout, dep_makefile ],
+ make_depend_multiunit, [['mhu01/unitA', 'mhu01/unitB', 'mhu01/unitC'], ''])
+
# Preprocessor must not be inherited by default active home unit
test('mkdepend_mhu02',
[ extra_files(['mhu02/']), dep_json
=====================================
testsuite/tests/driver/mkdepend/mkdepend_mhu01.dep-json → testsuite/tests/driver/mkdepend/mkdepend_mhu01a.dep-json
=====================================
=====================================
testsuite/tests/driver/mkdepend/mkdepend_mhu01.dep-makefile → testsuite/tests/driver/mkdepend/mkdepend_mhu01a.dep-makefile
=====================================
@@ -1,23 +1,23 @@
# DO NOT DELETE: Beginning of Haskell dependencies
+mhu01/a/A/Base.o-boot : mhu01/a/A/Base.hs-boot
mhu01/a/A/Base.o : mhu01/a/A/Base.hi-boot
+mhu01/a/Util.o : mhu01/a/Util.hs
+mhu01/a/Util.o : mhu01/a/A/Base.hi-boot
mhu01/a/A/Base.o : mhu01/a/A/Base.hs
mhu01/a/A/Base.o : mhu01/a/Util.hi
-mhu01/a/A/Base.o-boot : mhu01/a/A/Base.hs-boot
-mhu01/a/A/TH.o : mhu01/a/A/Base.hi
mhu01/a/A/TH.o : mhu01/a/A/TH.hs
-mhu01/a/Util.o : mhu01/a/A/Base.hi-boot
-mhu01/a/Util.o : mhu01/a/Util.hs
-mhu01/b/B/Normal.o : mhu01/a/A/Base.hi
-mhu01/b/B/Normal.o : mhu01/b/B/Normal.hs
-mhu01/b/B/Normal.o : mhu01/b/B/Splice.hi
-mhu01/b/B/Splice.o : mhu01/a/A/Base.hi
-mhu01/b/B/Splice.o : mhu01/a/A/TH.hi
+mhu01/a/A/TH.o : mhu01/a/A/Base.hi
+mhu01/b/Util.o : mhu01/b/Util.hs
mhu01/b/B/Splice.o : mhu01/b/B/Splice.hs
mhu01/b/B/Splice.o : mhu01/b/Util.hi
-mhu01/b/Util.o : mhu01/b/Util.hs
-mhu01/c/C.o : mhu01/a/Util.hi
-mhu01/c/C.o : mhu01/b/B/Normal.hi
+mhu01/b/B/Splice.o : mhu01/a/A/Base.hi
+mhu01/b/B/Splice.o : mhu01/a/A/TH.hi
+mhu01/b/B/Normal.o : mhu01/b/B/Normal.hs
+mhu01/b/B/Normal.o : mhu01/b/B/Splice.hi
+mhu01/b/B/Normal.o : mhu01/a/A/Base.hi
+mhu01/c/C.o : mhu01/c/C.hs
mhu01/c/C.o : mhu01/b/B/Splice.hi
+mhu01/c/C.o : mhu01/b/B/Normal.hi
mhu01/c/C.o : mhu01/b/Util.hi
-mhu01/c/C.o : mhu01/c/C.hs
+mhu01/c/C.o : mhu01/a/Util.hi
# DO NOT DELETE: End of Haskell dependencies
=====================================
testsuite/tests/driver/mkdepend/mkdepend_mhu01b.stderr
=====================================
@@ -0,0 +1,8 @@
+{"module":"A.Base","unit-id":"a","boot":true,"imports":{},"package-name":"a","source":"mhu01/a/A/Base.hs-boot"}
+{"module":"Util","unit-id":"a","imports":{"a":[{"module":"A.Base","boot":true}]},"package-name":"a","source":"mhu01/a/Util.hs"}
+{"module":"A.Base","unit-id":"a","imports":{"a":["Util"]},"package-name":"a","source":"mhu01/a/A/Base.hs"}
+{"module":"A.TH","unit-id":"a","imports":{"a":["A.Base"]},"package-name":"a","source":"mhu01/a/A/TH.hs"}
+{"module":"Util","unit-id":"b","imports":{},"package-name":"b","source":"mhu01/b/Util.hs"}
+{"module":"B.Splice","unit-id":"b","imports":{"a":[{"module":"A.Base","level":"quote"},{"module":"A.TH","level":"splice"}],"b":["Util"]},"package-name":"b","source":"mhu01/b/B/Splice.hs"}
+{"module":"B.Normal","unit-id":"b","imports":{"a":["A.Base"],"b":["B.Splice"]},"package-name":"b","source":"mhu01/b/B/Normal.hs"}
+{"module":"C","unit-id":"c","imports":{"a":[{"module":"Util","level":"splice"}],"b":["B.Normal",{"module":"B.Splice","level":"splice"},{"module":"Util","level":"quote"}]},"package-name":"c","source":"mhu01/c/C.hs"}
=====================================
testsuite/tests/driver/mkdepend/mkdepend_mhu01c.stderr
=====================================
@@ -0,0 +1,21 @@
+mhu01/a/A/Base.o-boot : mhu01/a/A/Base.hs-boot
+mhu01/a/A/Base.o : mhu01/a/A/Base.hi-boot
+mhu01/a/Util.o : mhu01/a/Util.hs
+mhu01/a/Util.o : mhu01/a/A/Base.hi-boot
+mhu01/a/A/Base.o : mhu01/a/A/Base.hs
+mhu01/a/A/Base.o : mhu01/a/Util.hi
+mhu01/a/A/TH.o : mhu01/a/A/TH.hs
+mhu01/a/A/TH.o : mhu01/a/A/Base.hi
+mhu01/b/Util.o : mhu01/b/Util.hs
+mhu01/b/B/Splice.o : mhu01/b/B/Splice.hs
+mhu01/b/B/Splice.o : mhu01/b/Util.hi
+mhu01/b/B/Splice.o : mhu01/a/A/Base.hi
+mhu01/b/B/Splice.o : mhu01/a/A/TH.hi
+mhu01/b/B/Normal.o : mhu01/b/B/Normal.hs
+mhu01/b/B/Normal.o : mhu01/b/B/Splice.hi
+mhu01/b/B/Normal.o : mhu01/a/A/Base.hi
+mhu01/c/C.o : mhu01/c/C.hs
+mhu01/c/C.o : mhu01/b/B/Splice.hi
+mhu01/c/C.o : mhu01/b/B/Normal.hi
+mhu01/c/C.o : mhu01/b/Util.hi
+mhu01/c/C.o : mhu01/a/Util.hi
=====================================
testsuite/tests/driver/mkdepend/mkdepend_mhu01d.dep-makefile
=====================================
@@ -0,0 +1,23 @@
+# DO NOT DELETE: Beginning of Haskell dependencies
+mhu01/a/A/Base.o-boot : mhu01/a/A/Base.hs-boot
+mhu01/a/A/Base.o : mhu01/a/A/Base.hi-boot
+mhu01/a/Util.o : mhu01/a/Util.hs
+mhu01/a/Util.o : mhu01/a/A/Base.hi-boot
+mhu01/a/A/Base.o : mhu01/a/A/Base.hs
+mhu01/a/A/Base.o : mhu01/a/Util.hi
+mhu01/a/A/TH.o : mhu01/a/A/TH.hs
+mhu01/a/A/TH.o : mhu01/a/A/Base.hi
+mhu01/b/Util.o : mhu01/b/Util.hs
+mhu01/b/B/Splice.o : mhu01/b/B/Splice.hs
+mhu01/b/B/Splice.o : mhu01/b/Util.hi
+mhu01/b/B/Splice.o : mhu01/a/A/Base.hi
+mhu01/b/B/Splice.o : mhu01/a/A/TH.hi
+mhu01/b/B/Normal.o : mhu01/b/B/Normal.hs
+mhu01/b/B/Normal.o : mhu01/b/B/Splice.hi
+mhu01/b/B/Normal.o : mhu01/a/A/Base.hi
+mhu01/c/C.o : mhu01/c/C.hs
+mhu01/c/C.o : mhu01/b/B/Splice.hi
+mhu01/c/C.o : mhu01/b/B/Normal.hi
+mhu01/c/C.o : mhu01/b/Util.hi
+mhu01/c/C.o : mhu01/a/Util.hi
+# DO NOT DELETE: End of Haskell dependencies
=====================================
testsuite/tests/driver/mkdepend/mkdepend_mhu01d.stderr
=====================================
@@ -0,0 +1,8 @@
+{"module":"A.Base","unit-id":"a","boot":true,"imports":{},"package-name":"a","source":"mhu01/a/A/Base.hs-boot"}
+{"module":"Util","unit-id":"a","imports":{"a":[{"module":"A.Base","boot":true}]},"package-name":"a","source":"mhu01/a/Util.hs"}
+{"module":"A.Base","unit-id":"a","imports":{"a":["Util"]},"package-name":"a","source":"mhu01/a/A/Base.hs"}
+{"module":"A.TH","unit-id":"a","imports":{"a":["A.Base"]},"package-name":"a","source":"mhu01/a/A/TH.hs"}
+{"module":"Util","unit-id":"b","imports":{},"package-name":"b","source":"mhu01/b/Util.hs"}
+{"module":"B.Splice","unit-id":"b","imports":{"a":[{"module":"A.Base","level":"quote"},{"module":"A.TH","level":"splice"}],"b":["Util"]},"package-name":"b","source":"mhu01/b/B/Splice.hs"}
+{"module":"B.Normal","unit-id":"b","imports":{"a":["A.Base"],"b":["B.Splice"]},"package-name":"b","source":"mhu01/b/B/Normal.hs"}
+{"module":"C","unit-id":"c","imports":{"a":[{"module":"Util","level":"splice"}],"b":["B.Normal",{"module":"B.Splice","level":"splice"},{"module":"Util","level":"quote"}]},"package-name":"c","source":"mhu01/c/C.hs"}
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/419cfed9e5775cccf05d7217255ad0a…
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/419cfed9e5775cccf05d7217255ad0a…
You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help
1
0
[Git][ghc/ghc][wip/davide/windows-dlls_sysm-test_master] 4 commits: hadrian: when running tests, properly set have_gdb and have_readelf
by David Eichmann (@DavidEichmann) 20 Oct '26
by David Eichmann (@DavidEichmann) 20 Oct '26
20 Oct '26
David Eichmann pushed to branch wip/davide/windows-dlls_sysm-test_master at Glasgow Haskell Compiler / GHC
Commits:
d31f8acb by David Eichmann at 2026-10-20T12:38:56+01:00
hadrian: when running tests, properly set have_gdb and have_readelf
- - - - -
2809df43 by David Eichmann at 2026-10-20T12:39:42+01:00
testsuite: add have_ldd() test modifier
- - - - -
f8484532 by David Eichmann at 2026-10-20T12:39:45+01:00
Add golden test for rts/RtsSymbols.c
This test compares declared symbols against symbols observed
in the rts DSO (.so file).
- - - - -
007c5fab by David Eichmann at 2026-10-20T12:39:45+01:00
RtsSymbols stores redirect labels
Redirect labels are used by the rts-check-symbols test
- - - - -
15 changed files:
- hadrian/src/Settings/Builders/RunTest.hs
- rts/RtsSymbols.c
- rts/include/rts/RtsSymbols.h
- testsuite/driver/runtests.py
- testsuite/driver/testglobals.py
- testsuite/driver/testlib.py
- testsuite/mk/test.mk
- testsuite/tests/ghci/linking/T25155_iserv_main.c
- + testsuite/tests/rts/symbols/Main.hs
- + testsuite/tests/rts/symbols/Makefile
- + testsuite/tests/rts/symbols/all.T
- + testsuite/tests/rts/symbols/dumpDeclaredSymbols.c
- + testsuite/tests/rts/symbols/rts-check-symbols.ignore
- + testsuite/tests/rts/symbols/rts-check-symbols.sh
- + testsuite/tests/rts/symbols/rts-check-symbols.stdout
Changes:
=====================================
hadrian/src/Settings/Builders/RunTest.hs
=====================================
@@ -254,6 +254,10 @@ runTestBuilderArgs = builder Testsuite ? do
let asBool :: String -> Bool -> String
asBool s b = s ++ show b
+ have_gdb <- expr $ liftIO (isJust <$> findExecutable "gdb")
+ have_ldd <- expr $ liftIO (isJust <$> findExecutable "ldd")
+ have_readelf <- expr $ liftIO (isJust <$> findExecutable "readelf")
+
-- TODO: set CABAL_MINIMAL_BUILD/CABAL_PLUGIN_BUILD
mconcat [ arg "-Wdefault" -- see #22727
, arg $ "testsuite/driver/runtests.py"
@@ -274,6 +278,9 @@ runTestBuilderArgs = builder Testsuite ? do
, arg "-e", arg $ asBool "ghc_with_native_codegen=" withNativeCodeGen
, arg "-e", arg $ asBool "config.have_ncg=" withNativeCodeGen
, arg "-e", arg $ asBool "config.have_llvm=" have_llvm
+ , arg "-e", arg $ asBool "config.have_gdb=" have_gdb
+ , arg "-e", arg $ asBool "config.have_ldd=" have_ldd
+ , arg "-e", arg $ asBool "config.have_readelf=" have_readelf
, arg "-e", arg $ asBool "config.compiler_profiled=" profiled
=====================================
rts/RtsSymbols.c
=====================================
@@ -1220,13 +1220,13 @@ RTS_SYMBOLS_PRIM
/* See Note [Naming Scheme for Symbol Macros] */
#define SymI_HasProto(vvv) { MAYBE_LEADING_UNDERSCORE_STR(#vvv), \
- (void*)(&(vvv)), STRENGTH_NORMAL, SYM_TYPE_CODE },
+ (void*)(&(vvv)), STRENGTH_NORMAL, SYM_TYPE_CODE, NULL },
#define SymI_HasDataProto(vvv) { MAYBE_LEADING_UNDERSCORE_STR(#vvv), \
- (void*)(&(vvv)), STRENGTH_NORMAL, SYM_TYPE_DATA },
+ (void*)(&(vvv)), STRENGTH_NORMAL, SYM_TYPE_DATA, NULL },
#define SymE_HasProto(vvv) { MAYBE_LEADING_UNDERSCORE_STR(#vvv), \
- (void*)(&(vvv)), STRENGTH_NORMAL, SYM_TYPE_CODE },
+ (void*)(&(vvv)), STRENGTH_NORMAL, SYM_TYPE_CODE, NULL },
#define SymE_HasDataProto(vvv) { MAYBE_LEADING_UNDERSCORE_STR(#vvv), \
- (void*)(&(vvv)), STRENGTH_NORMAL, SYM_TYPE_DATA },
+ (void*)(&(vvv)), STRENGTH_NORMAL, SYM_TYPE_DATA, NULL },
#define SymI_NeedsProto(vvv) SymI_HasProto(vvv)
#define SymI_NeedsDataProto(vvv) SymI_HasDataProto(vvv)
@@ -1237,7 +1237,7 @@ RTS_SYMBOLS_PRIM
// another symbol. See newCAF/newRetainedCAF/newGCdCAF for an example.
#define SymI_HasProto_redirect(vvv,xxx,strength,ty) \
{ MAYBE_LEADING_UNDERSCORE_STR(#vvv), \
- (void*)(&(xxx)), strength, ty },
+ (void*)(&(xxx)), strength, ty, MAYBE_LEADING_UNDERSCORE_STR(#xxx) },
static RtsSymbolVal * rtsSyms = NULL;
@@ -1264,7 +1264,7 @@ RtsSymbolVal const * getRtsSymbols(void) {
RTS_ARM_OUTLINE_ATOMIC_SYMBOLS
RTS_SYMBOLS_PRIM
SymI_HasDataProto(nonmoving_write_barrier_enabled)
- { 0, 0, STRENGTH_NORMAL, SYM_TYPE_CODE } /* sentinel */
+ { 0, 0, STRENGTH_NORMAL, SYM_TYPE_CODE, NULL } /* sentinel */
};
uint32_t n = 1;
@@ -1362,7 +1362,7 @@ RtsSymbolVal const * getRtsSymbols(void) {
// CODE_SYM(getauxval)
// CODE_SYM(posix_spawn_file_actions_init)
// ...
-// { 0, 0, STRENGTH_NORMAL, SYM_TYPE_CODE } /* sentinel */
+// { 0, 0, STRENGTH_NORMAL, SYM_TYPE_CODE, NULL } /* sentinel */
// };
//
// RtsSymbolVal* rtsExtraSyms() {
@@ -1375,7 +1375,7 @@ RtsSymbolVal const * getRtsSymbols(void) {
//
static RtsSymbolVal default_extra_syms[] = {
- { 0, 0, STRENGTH_NORMAL, SYM_TYPE_CODE } /* sentinel */
+ { 0, 0, STRENGTH_NORMAL, SYM_TYPE_CODE, NULL } /* sentinel */
};
/* Default empty extra RTS symbols. See Note[Extra RTS symbols] */
=====================================
rts/include/rts/RtsSymbols.h
=====================================
@@ -8,6 +8,7 @@
#pragma once
+#include <stdbool.h>
typedef void SymbolAddr;
typedef char SymbolName;
@@ -36,6 +37,9 @@ typedef struct _RtsSymbolVal {
SymbolAddr* addr;
SymStrength strength;
SymType type;
+ // If this is a redirection, then this is the lable of the
+ // target symbol. Otherwise NULL.
+ const SymbolName* lbl_redirect_target;
} RtsSymbolVal;
RtsSymbolVal const * getRtsSymbols(void);
=====================================
testsuite/driver/runtests.py
=====================================
@@ -284,32 +284,36 @@ def format_path(path):
# On Windows we need to set $PATH to include the paths to all the DLLs
# in order for the dynamic library tests to work.
if windows:
+ # Add mingw to PATH
+ topdir = config.libdir
+ mingw = os.path.abspath(os.path.join(topdir, '../mingw/bin'))
+ mingw = format_path(mingw)
+ ghc_env['PATH'] = os.pathsep.join([ghc_env.get("PATH", ""), mingw])
+
+ # Add ghc packages to PATH
try:
- pkginfo = getStdout([config.ghc_pkg, 'dump'])
+ pkgs = getStdout([config.ghc_pkg, '--simple-output', 'list'])
except FileNotFoundError as err:
# This can happen when we are only running tests which don't depend on ghc (ie linters)
# In that case we probably haven't built ghc-pkg yet so this query will fail.
print (err)
print ("Failed to call ghc-pkg for windows path modification... some tests might fail")
- pkginfo = ""
- topdir = config.libdir
- mingw = os.path.abspath(os.path.join(topdir, '../mingw/bin'))
- mingw = format_path(mingw)
- ghc_env['PATH'] = os.pathsep.join([ghc_env.get("PATH", ""), mingw])
- for line in pkginfo.split('\n'):
- if line.startswith('library-dirs:'):
- path = line.rstrip()
- path = str_removeprefix(path, 'library-dirs: ')
- # Use string.replace instead of re.sub, because re.sub
- # interprets backslashes in the replacement string as
- # escape sequences.
- path = path.replace('$topdir', topdir)
+ pkgs = ""
+
+ newPaths = set()
+ pkgs = pkgs.split(' ')
+ for pkg in pkgs:
+ for field in ['library-dirs', 'dynamic-library-dirs']:
+ path = getStdout([config.ghc_pkg, '--simple-output', 'field', pkg, field])
+ path = path.strip()
if path.startswith('"'):
path = re.sub('^"(.*)"$', '\\1', path)
path = re.sub('\\\\(.)', '\\1', path)
+ path = format_path(path)
+ newPaths.add(path)
- path = format_path(path)
- ghc_env['PATH'] = os.pathsep.join([path, ghc_env.get("PATH", "")])
+ for path in newPaths:
+ ghc_env['PATH'] = os.pathsep.join([path, ghc_env.get("PATH", "")])
testopts_ctx_var.set(TestOptions())
=====================================
testsuite/driver/testglobals.py
=====================================
@@ -182,6 +182,9 @@ class TestConfig:
# Is gdb available?
self.have_gdb = False
+ # Is ldd available?
+ self.have_ldd = False
+
# Is readelf available?
self.have_readelf = False
=====================================
testsuite/driver/testlib.py
=====================================
@@ -1091,6 +1091,9 @@ def debug_rts( ) -> bool:
def have_gdb( ) -> bool:
return config.have_gdb
+def have_ldd( ) -> bool:
+ return config.have_ldd
+
def have_readelf( ) -> bool:
return config.have_readelf
=====================================
testsuite/mk/test.mk
=====================================
@@ -107,6 +107,7 @@ RUNTEST_OPTS += -e "config.tables_next_to_code=False"
endif
HAVE_GDB := $(shell if gdb --version > /dev/null 2> /dev/null; then echo YES; else echo NO; fi)
+HAVE_LDD := $(shell if ldd --version > /dev/null 2> /dev/null; then echo YES; else echo NO; fi)
HAVE_READELF := $(shell if readelf --version > /dev/null 2> /dev/null; then echo YES; else echo NO; fi)
# Detect whether the fast (GMP) bignum backend is in use. The GMP backend module
@@ -154,6 +155,12 @@ else
RUNTEST_OPTS += -e config.have_gdb=False
endif
+ifeq "$(HAVE_LDD)" "YES"
+RUNTEST_OPTS += -e config.have_ldd=True
+else
+RUNTEST_OPTS += -e config.have_ldd=False
+endif
+
ifeq "$(HAVE_READELF)" "YES"
RUNTEST_OPTS += -e config.have_readelf=True
else
=====================================
testsuite/tests/ghci/linking/T25155_iserv_main.c
=====================================
@@ -23,7 +23,7 @@ int main (int argc, char *argv[])
// can use the FFI call to foobar in a TH splice.
#define SymI_HasProto(vvv) { MAYBE_LEADING_UNDERSCORE_STR(#vvv), \
- (void*)(&(vvv)), STRENGTH_NORMAL, SYM_TYPE_CODE },
+ (void*)(&(vvv)), STRENGTH_NORMAL, SYM_TYPE_CODE, NULL },
void foobar(int x) {
=====================================
testsuite/tests/rts/symbols/Main.hs
=====================================
@@ -0,0 +1,119 @@
+import qualified Data.Map.Lazy as M
+import qualified Data.Set as S
+import Data.List (isSuffixOf, stripPrefix)
+import Data.Maybe (mapMaybe)
+import System.Environment (getArgs)
+
+main :: IO ()
+main = do
+ args <- getArgs
+ case args of
+ ["dump"] -> dumpDeclaredSymbols
+ ["diff", ignoreSymbolsFile, declaredSymbolsFile, observedSymbolsFile] -> do
+
+ -- Symbols to be ignored. These are symbols that are gated on CPP not correctly
+ -- specified in RtsSymbols.c. This means the symbol will sometimes appear in the
+ -- test output and sometimes not (depending on CPP defines). That means we can't
+ -- simply put those symbols into the expected test output. It's already the
+ -- job of RtsSymbols.c to declare symbols correctly in accordance with CPP. We
+ -- want to avoid copying that logic again in the test. So we just ignore some
+ -- symbols so that CI can pass and in the future we aim to fix RtsSymbols.c and
+ -- reduce the size of ignoredSymbols.
+ ignoreSymbols <- S.fromList
+ . filter (not . null)
+ . lines
+ <$> readFile ignoreSymbolsFile
+
+ declaredSymbolsStr <- readFile declaredSymbolsFile
+ observedSymbolsStr <- readFile observedSymbolsFile
+
+ let declaredSymbols = readCodeOrData <$> linePairs declaredSymbolsStr
+ let observedSymbols = parseReadelf observedSymbolsStr
+
+ let diffStr = diff ignoreSymbols declaredSymbols observedSymbols
+ -- writeFile "<ghc root>/testsuite/tests/rts/symbols/rts-check-symbols.stdout"
+ putStrLn diffStr
+
+ _ -> fail ("Unexpected arguments: " ++ show args)
+
+foreign import ccall "dumpDeclaredSymbols"
+ dumpDeclaredSymbols :: IO ()
+
+data CodeOrData = Code | Data
+ deriving (Show, Eq)
+
+-- Compare observedSymbols (symbols observed in the DSO) to declaredSymbols
+-- (symbols declared in RtsSymbols.c)
+diff :: S.Set String -> M.Map String CodeOrData -> M.Map String CodeOrData -> String
+diff ignoreSymbols declaredSymbolsAll observedSymbolsAll = unlines $
+ [ "Symbols declared in RtsSymbols.c but not observed in the DSO:" ]
+ ++ [ "Declared> " ++ k ++ " " ++ show v | (k, v) <- M.toList diffInDeclaredSymbols ]
+ ++ [ "" ]
+ ++ [ "Symbols observed in the DSO but not declared in RtsSymbols.c:" ]
+ ++ [ "Observed> " ++ k ++ " " ++ show v | (k, v) <- M.toList diffInObservedSymbols ]
+ ++ [ "" ]
+ ++ [ "Symbols observed in the DSO and declared in RtsSymbols.c but with conflicting type:" ]
+ ++ [ "WrongType> " ++ k ++ " (declared, observed): (" ++ show declared ++ ", " ++ show observed ++ ")"
+ | (k, (declared, observed)) <- M.toList diffOnType ]
+ where
+ -- Ignore some symbols
+ declaredSymbols = declaredSymbolsAll `M.withoutKeys` ignoreSymbols
+ observedSymbols = observedSymbolsAll `M.withoutKeys` ignoreSymbols
+
+ -- Undeclared observed symbols
+ diffInObservedSymbols :: M.Map String CodeOrData
+ diffInObservedSymbols = observedSymbols M.\\ declaredSymbols
+
+ -- Declared unobserved symbols
+ diffInDeclaredSymbols :: M.Map String CodeOrData
+ diffInDeclaredSymbols = declaredSymbols M.\\ observedSymbols
+
+ -- Declared observed symbols with conflicting type
+ diffOnType :: M.Map String (CodeOrData, CodeOrData)
+ diffOnType = M.filter (\(a, b) -> a /= b) $ M.intersectionWith (,) declaredSymbols observedSymbols
+
+linePairs :: String -> M.Map String String
+linePairs = M.fromList . map (\str -> let [a,b] = words str in (a,b)) . filter (/= "") . lines
+
+readCodeOrData :: String -> CodeOrData
+readCodeOrData s = case s of
+ "code" -> Code
+ "data" -> Data
+ _ -> error ("Unknown symbol type: " ++ show s)
+
+-- | Parse the output of @readelf -W --dyn-syms@ of an ELF shared object.
+--
+-- Symbol lines look like:
+--
+-- > Num: Value Size Type Bind Vis Ndx Name
+-- > 2: 0000000000000000 0 FUNC GLOBAL DEFAULT UND tcsetattr@GLIBC_2.2.5 (2)
+-- > 1149: 000000000006d020 190 OBJECT GLOBAL DEFAULT 12 stg_AP_info
+--
+-- Note that Vis may be followed by extra annotations on some platforms (e.g.
+-- @[<localentry>: 8]@ on ppc64le), so Ndx and Name are taken from the end of
+-- the line.
+parseReadelf :: String -> M.Map String CodeOrData
+parseReadelf = M.fromList . mapMaybe parseLine . lines
+ where
+ parseLine l = case words l of
+ (num : _value : _size : ty : bind : rest)
+ | ":" `isSuffixOf` num
+ , name : ndx : _ <- reverse (dropVersionIndex rest)
+ , bind == "GLOBAL"
+ , ndx /= "UND" -- Imports
+ -> (,) name <$> typeToCodeOrData ty
+ _ -> Nothing
+
+ -- Drop the trailing version index e.g. "(2)"
+ dropVersionIndex ws = case reverse ws of
+ ('(' : _) : ws' -> reverse ws'
+ _ -> ws
+
+ -- Returns Nothing if the symbol should be ignored
+ typeToCodeOrData :: String -> Maybe CodeOrData
+ typeToCodeOrData ty = case ty of
+ "FILE" -> Nothing
+ "NOTYPE" -> Nothing
+ "FUNC" -> Just Code
+ "OBJECT" -> Just Data
+ _ -> error ("Unknown symbol type: " ++ show ty)
\ No newline at end of file
=====================================
testsuite/tests/rts/symbols/Makefile
=====================================
@@ -0,0 +1,9 @@
+TOP=../../..
+include $(TOP)/mk/boilerplate.mk
+include $(TOP)/mk/test.mk
+
+# This makefile rule only exists because TEST_HC is set in boilerplate.mk
+# and otherwise the TEST_HC variable wouldn't be set in the context of the bash script
+rts-check-symbols:
+ '$(TEST_HC)' $(TEST_HC_OPTS) -package containers -dynamic dumpDeclaredSymbols.c Main.hs -o Main > /dev/null
+ GHC_PKG="$(GHC_PKG)" GHC="$(TEST_HC)" ./rts-check-symbols.sh
=====================================
testsuite/tests/rts/symbols/all.T
=====================================
@@ -0,0 +1,12 @@
+test('rts-check-symbols', [
+ unless(have_dynamic() and have_readelf() and have_ldd(), skip),
+ windows_skip,
+ extra_files([
+ 'dumpDeclaredSymbols.c',
+ 'Main.hs',
+ 'rts-check-symbols.ignore',
+ 'rts-check-symbols.sh'
+ ])],
+ makefile_test,
+ []
+)
=====================================
testsuite/tests/rts/symbols/dumpDeclaredSymbols.c
=====================================
@@ -0,0 +1,34 @@
+
+#include "Rts.h"
+#include "rts/RtsSymbols.h"
+
+#include <stdio.h>
+
+void dumpDeclaredSymbols(void);
+
+void dumpDeclaredSymbols(void) {
+ const RtsSymbolVal * declaredSyms = getRtsSymbols();
+ for (const RtsSymbolVal * sym = &declaredSyms[0]; sym->lbl; sym++) {
+ // Ignore hidden
+ if (sym->type & SYM_TYPE_HIDDEN) continue;
+
+ char * type = NULL;
+ if (sym->type & SYM_TYPE_CODE) {
+ type = "code";
+ } else if (sym->type & (SYM_TYPE_DATA | SYM_TYPE_INDIRECT_DATA)) {
+ type = "data";
+ }
+
+ const char * lbl = NULL;
+ if (sym->lbl_redirect_target) {
+ // In the case of redirects, the true symbol name is lbl_redirect_target
+ lbl = sym->lbl_redirect_target;
+ } else {
+ lbl = sym->lbl;
+ }
+
+ if (type) {
+ printf("%s %s\n", lbl, type);
+ }
+ }
+}
=====================================
testsuite/tests/rts/symbols/rts-check-symbols.ignore
=====================================
@@ -0,0 +1,40 @@
+complex_ccall_adjustor_template
+decodeAddendAarch64
+encodeAddendAarch64
+findStub
+flushInstructionCache
+flushInstructionCacheAarch64
+freeStubs
+hs_quotInt16X8
+hs_quotInt32X4
+hs_quotInt64X2
+hs_quotInt8X16
+hs_quotWord16X8
+hs_quotWord32X4
+hs_quotWord64X2
+hs_quotWord8X16
+hs_remInt16X8
+hs_remInt32X4
+hs_remInt64X2
+hs_remInt8X16
+hs_remWord16X8
+hs_remWord32X4
+hs_remWord64X2
+hs_remWord8X16
+instSizeAarch64
+isAddSub
+isAdrp
+isBranch
+isBranchLink
+isLoadStore
+isVectorOp
+makeStub
+makeStubAarch64
+needStubForRelAarch64
+needStubForRelaAarch64
+numberOfStubsForSection
+relocateObjectCode
+relocateObjectCodeAarch64
+simple_ccall_adjustor_template
+StgReturn
+stubSizeAarch64
=====================================
testsuite/tests/rts/symbols/rts-check-symbols.sh
=====================================
@@ -0,0 +1,30 @@
+#!/usr/bin/env bash
+
+#
+# Check the diff between symbols exported from the RTS and those listed in RtsSymbols.c
+#
+
+IGNORED_SYMBOLS=rts-check-symbols.ignore
+DECLARED_SYMBOLS=declared_symbols.txt
+OBSERVED_SYMBOLS=observed_symbols.txt
+
+# Get rts Symbols listed in RtsSymbols.c
+./Main dump | sort > $DECLARED_SYMBOLS
+
+# Find the path of the RTS shared object that ./Main is linked against
+# Match e.g.:
+# libHSrts-1.0.3-ghc9.15.20260710.so => /home/alice/s p a c e/myApp/lib/libHSrts-1.0.3-ghc9.15.20260710.so (0x7ffebc9a0000)
+RTS_DYLIB_PATH=$(ldd ./Main | awk '/^[[:space:]]*libHSrts-.* => .+$/ {
+ sub(/^[[:space:]]*libHSrts-.* => /, "")
+ sub(/ \([^(]*\)$/, "")
+ print
+}')
+
+# Dump the exported symbols. Main.hs is responsible for parsing the output.
+# The dynamic symbol table of the shared object. We use readelf rather
+# than nm as nm classifies symbols by section rather than by symbol
+# type, and with tables next to code info tables (objects) live in the
+# .text section.
+readelf -W --dyn-syms "$RTS_DYLIB_PATH" > $OBSERVED_SYMBOLS
+
+./Main diff $IGNORED_SYMBOLS $DECLARED_SYMBOLS $OBSERVED_SYMBOLS
=====================================
testsuite/tests/rts/symbols/rts-check-symbols.stdout
=====================================
@@ -0,0 +1,759 @@
+Symbols declared in RtsSymbols.c but not observed in the DSO:
+Declared> __udivti3 Code
+Declared> __umodti3 Code
+Declared> atomic_dec Code
+Declared> dirty_TVAR Code
+Declared> ffi_call Code
+Declared> ffi_prep_cif Code
+Declared> ffi_type_double Data
+Declared> ffi_type_float Data
+Declared> ffi_type_pointer Data
+Declared> ffi_type_sint16 Data
+Declared> ffi_type_sint32 Data
+Declared> ffi_type_sint64 Data
+Declared> ffi_type_sint8 Data
+Declared> ffi_type_uint16 Data
+Declared> ffi_type_uint32 Data
+Declared> ffi_type_uint64 Data
+Declared> ffi_type_uint8 Data
+Declared> ffi_type_void Data
+Declared> signal_handlers Code
+Declared> updateRemembSetPushThunk Code
+
+Symbols observed in the DSO but not declared in RtsSymbols.c:
+Observed> AP_STACK_sizeW Code
+Observed> AP_sizeW Code
+Observed> BLACKHOLE_sizeW Code
+Observed> Bdescr Code
+Observed> CHARLIKE_CLOSURE Code
+Observed> CONSTR_sizeW Code
+Observed> CONTINUATION_sizeW Code
+Observed> CON_INFO_PTR_TO_STRUCT Code
+Observed> FUN_INFO_PTR_TO_STRUCT Code
+Observed> FileEventLogWriter Data
+Observed> GET_CLOSURE_TAG Code
+Observed> GET_INFO Code
+Observed> GET_TAG Code
+Observed> INFO_PTR_TO_STRUCT Code
+Observed> INTLIKE_CLOSURE Code
+Observed> LOOKS_LIKE_CLOSURE_PTR Code
+Observed> LOOKS_LIKE_INFO_PTR Code
+Observed> LOOKS_LIKE_INFO_PTR_NOT_NULL Code
+Observed> LOOKS_LIKE_PTR Code
+Observed> NullEventLogWriter Data
+Observed> OS_TRY_ACQUIRE_LOCK Code
+Observed> PAP_sizeW Code
+Observed> RET_INFO_PTR_TO_STRUCT Code
+Observed> SET_INFO Code
+Observed> SET_INFO_RELAXED Code
+Observed> SET_INFO_RELEASE Code
+Observed> STATIC_LINK Code
+Observed> TAG_CLOSURE Code
+Observed> THUNK_INFO_PTR_TO_STRUCT Code
+Observed> THUNK_SELECTOR_sizeW Code
+Observed> UNTAG_CLOSURE Code
+Observed> UNTAG_CONST_CLOSURE Code
+Observed> UPD_BH_UPDATABLE_ctr Data
+Observed> __rts_fopen Code
+Observed> _warnFail Code
+Observed> addInitFini Code
+Observed> addProddableBlock Code
+Observed> allocAlignedGroupOnNode Code
+Observed> allocBlock Code
+Observed> allocBlockOnNode Code
+Observed> allocBlockOnNode_lock Code
+Observed> allocBlock_lock Code
+Observed> allocGroup Code
+Observed> allocGroupOnNode Code
+Observed> allocGroupOnNode_lock Code
+Observed> allocGroup_lock Code
+Observed> allocLimitKill Data
+Observed> allocLimitRunHook Data
+Observed> allocMBlockAlignedGroupOnNode Code
+Observed> allocateMightFail Code
+Observed> allocatePinned Code
+Observed> allocated_bytes_at_heapoverflow Data
+Observed> ap_sizeW Code
+Observed> ap_stack_sizeW Code
+Observed> arr_words_sizeW Code
+Observed> arr_words_words Code
+Observed> bco_sizeW Code
+Observed> broadcastCondition Code
+Observed> cas_word8 Code
+Observed> checkProddableBlock Code
+Observed> checkVectorSupport Code
+Observed> clearImportSymbol Code
+Observed> closeCondition Code
+Observed> closeMutex Code
+Observed> closure_sizeW Code
+Observed> collect_pointers Code
+Observed> compact_nfdata_full_sizeW Code
+Observed> containsSpan Code
+Observed> continuation_sizeW Code
+Observed> createAttachedOSThread Code
+Observed> createGenThread Code
+Observed> createIOThread Code
+Observed> createOSThread Code
+Observed> createStrictIOThread Code
+Observed> createThread Code
+Observed> currentWorkerCount Data
+Observed> debugMsgFn Data
+Observed> dequeElements Code
+Observed> dirty_STACK Code
+Observed> dirty_TSO Code
+Observed> discardElements Code
+Observed> era Data
+Observed> errorMsgFn Data
+Observed> eventLogStatus Code
+Observed> eventTypes Data
+Observed> exitFn Data
+Observed> exitHeapOverflow Code
+Observed> exitHpc Code
+Observed> fatalInternalErrorFn Data
+Observed> fillGot Code
+Observed> findSymbol Code
+Observed> findSymbolTable Code
+Observed> flushExec Code
+Observed> formatClosureDescIpe Code
+Observed> freeAllMBlocks Code
+Observed> freeChain Code
+Observed> freeChain_lock Code
+Observed> freeGot Code
+Observed> freeGroup Code
+Observed> freeGroup_lock Code
+Observed> freeInitFiniList Code
+Observed> freeMBlocks Code
+Observed> freeProddableBlocks Code
+Observed> freeThreadingResources Code
+Observed> freeWSDeque Code
+Observed> full_prog_argc Data
+Observed> full_prog_argv Data
+Observed> generations Data
+Observed> getFirstMBlock Code
+Observed> getMBlock Code
+Observed> getMBlockOnNode Code
+Observed> getMBlocks Code
+Observed> getMBlocksOnNode Code
+Observed> getNextMBlock Code
+Observed> getObjectLoadStatus Code
+Observed> get_con_itbl Code
+Observed> get_fun_itbl Code
+Observed> get_itbl Code
+Observed> get_itbl_acquire Code
+Observed> get_ret_itbl Code
+Observed> get_thunk_itbl Code
+Observed> heap_view_closurePtrs Code
+Observed> heap_view_closureSize Code
+Observed> heap_view_closure_ptrs_in_pap_payload Code
+Observed> highMemDynamic Data
+Observed> hs_main Code
+Observed> initAdjustors Code
+Observed> initBlockAllocator Code
+Observed> initCondition Code
+Observed> initGenerationStats Code
+Observed> initMBlocks Code
+Observed> initMutex Code
+Observed> initProddableBlockSet Code
+Observed> interp_shutdown Code
+Observed> interp_startup Code
+Observed> interruptOSThread Code
+Observed> isSymbolImport Code
+Observed> isSymbolWeak Code
+Observed> itbl_to_con_itbl Code
+Observed> itbl_to_fun_itbl Code
+Observed> itbl_to_ret_itbl Code
+Observed> itbl_to_thunk_itbl Code
+Observed> joinOSThread Code
+Observed> kernelThreadId Code
+Observed> listAllBlocks Code
+Observed> listThreads Code
+Observed> looksEmptyWSDeque Code
+Observed> lookupIPEId Code
+Observed> m32_free_page_pool Data
+Observed> m32_free_page_pool_size Data
+Observed> makeGot Code
+Observed> mblock_address_space Data
+Observed> mblocks_allocated Data
+Observed> modules Data
+Observed> mpc_misses Data
+Observed> mutArrPtrsCard Code
+Observed> mutArrPtrsCardTableSize Code
+Observed> mutArrPtrsCards Code
+Observed> mut_arr_ptrs_sizeW Code
+Observed> n_alloc_blocks_by_node Data
+Observed> needGotSlot Code
+Observed> newCAF Code
+Observed> newGCdCAF Code
+Observed> newRetainedCAF Code
+Observed> newWSDeque Code
+Observed> next_nursery Data
+Observed> no_break_on_exception Data
+Observed> nonmovingAllocatorCensus Code
+Observed> nonmovingAllocatorCensusWithWords Code
+Observed> nonmovingPrintAllocatorCensus Code
+Observed> nonmovingSweep Code
+Observed> nonmovingSweepCompactObjects Code
+Observed> nonmovingSweepLargeObjects Code
+Observed> nonmovingSweepMutLists Code
+Observed> nonmovingSweepStableNameTable Code
+Observed> nonmovingTraceAllocatorCensus Code
+Observed> oldest_gen Data
+Observed> osThreadId Code
+Observed> osThreadIsAlive Code
+Observed> pap_sizeW Code
+Observed> pauseTokenCapability Code
+Observed> pbarf Code
+Observed> peak_mblocks_allocated Data
+Observed> poolFlush Code
+Observed> poolFree Code
+Observed> poolGetDesiredSize Code
+Observed> poolGetMaxSize Code
+Observed> poolInit Code
+Observed> poolRelease Code
+Observed> poolSetDesiredSize Code
+Observed> poolSetMaxSize Code
+Observed> poolTake Code
+Observed> poolTryTake Code
+Observed> popWSDeque Code
+Observed> printLoadedObjects Code
+Observed> prog_name Data
+Observed> pushWSDeque Code
+Observed> re_invalid Data
+Observed> re_realso Data
+Observed> releaseFreeMemory Code
+Observed> releaseThreadNode Code
+Observed> reportMemoryMap Code
+Observed> rtsDebugMsgFn Code
+Observed> rtsErrorMsgFn Code
+Observed> rtsFatalInternalErrorFn Code
+Observed> rtsSysErrorMsgFn Code
+Observed> rts_argc Data
+Observed> rts_argv Data
+Observed> rts_argv_size Data
+Observed> rts_done Code
+Observed> rts_evalLazyIO_ Code
+Observed> rts_getSchedStatus Code
+Observed> rts_isPaused Code
+Observed> rts_listMiscRoots Code
+Observed> rts_listThreads Code
+Observed> rts_pause Code
+Observed> rts_pinThreadToNumaNode Code
+Observed> rts_resume Code
+Observed> runFini Code
+Observed> runInit Code
+Observed> sbarf Code
+Observed> scheduleWaitThread Code
+Observed> setHighMemDynamic Code
+Observed> setImportSymbol Code
+Observed> setSymbolInfo Code
+Observed> setTSOLink Code
+Observed> setTSOPrev Code
+Observed> setThreadAffinity Code
+Observed> setThreadNode Code
+Observed> setVectorSupport Code
+Observed> setWeakSymbol Code
+Observed> shndx_table_uninit_label Data
+Observed> shouldCompact Code
+Observed> shutdownThread Code
+Observed> signalCondition Code
+Observed> sizeW_fromITBL Code
+Observed> small_mut_arr_ptrs_sizeW Code
+Observed> ssbarf Code
+Observed> stack_frame_sizeW Code
+Observed> stack_sizeW Code
+Observed> state_change_mutex Data
+Observed> stealWSDeque Code
+Observed> stealWSDeque_ Code
+Observed> stg_AP_NOUPD_info Data
+Observed> stg_AP_STACK_NOUPD_info Data
+Observed> stg_AP_STACK_info Data
+Observed> stg_AP_info Data
+Observed> stg_ASYNCIOOP_info Data
+Observed> stg_ASYNCIO_LIVE0_closure Data
+Observed> stg_ASYNCIO_LIVE0_info Data
+Observed> stg_BCO_info Data
+Observed> stg_CLOSURE_TABLE_NULL_closure Data
+Observed> stg_CLOSURE_TABLE_NULL_info Data
+Observed> stg_COMPACT_NFDATA_CLEAN_info Data
+Observed> stg_COMPACT_NFDATA_DIRTY_info Data
+Observed> stg_CONTINUATION_apply Code
+Observed> stg_CONTINUATION_info Data
+Observed> stg_C_FINALIZER_LIST_info Data
+Observed> stg_DEAD_SLOT_closure Data
+Observed> stg_DEAD_SLOT_info Data
+Observed> stg_DEAD_WEAK_info Data
+Observed> stg_END_STM_CHUNK_LIST_closure Data
+Observed> stg_END_STM_CHUNK_LIST_info Data
+Observed> stg_END_STM_WATCH_QUEUE_closure Data
+Observed> stg_END_STM_WATCH_QUEUE_info Data
+Observed> stg_END_TSO_QUEUE_closure Data
+Observed> stg_END_TSO_QUEUE_info Data
+Observed> stg_GCD_CAF_info Data
+Observed> stg_IND_info Data
+Observed> stg_MSG_BLACKHOLE_info Data
+Observed> stg_MSG_CLONE_STACK_info Data
+Observed> stg_MSG_NULL_info Data
+Observed> stg_MSG_THROWTO_info Data
+Observed> stg_MSG_TRY_WAKEUP_info Data
+Observed> stg_MSG_UPD_TSO_FLAG_info Data
+Observed> stg_MUT_ARR_PTRS_CLEAN_info Data
+Observed> stg_MVAR_TSO_QUEUE_info Data
+Observed> stg_NO_FINALIZER_closure Data
+Observed> stg_NO_FINALIZER_info Data
+Observed> stg_NO_TREC_closure Data
+Observed> stg_NO_TREC_info Data
+Observed> stg_PAP_apply Code
+Observed> stg_PAP_info Data
+Observed> stg_PROMPT_TAG_info Data
+Observed> stg_SMALL_MUT_ARR_PTRS_CLEAN_info Data
+Observed> stg_STABLE_NAME_info Data
+Observed> stg_STACK_info Data
+Observed> stg_TIMEOUT_QUEUE_EMPTY_closure Data
+Observed> stg_TIMEOUT_QUEUE_EMPTY_info Data
+Observed> stg_TIMEOUT_QUEUE_info Data
+Observed> stg_TREC_CHUNK_info Data
+Observed> stg_TREC_HEADER_info Data
+Observed> stg_TSO_info Data
+Observed> stg_TVAR_WATCH_QUEUE_info Data
+Observed> stg_WHITEHOLE_info Data
+Observed> stg_abort Code
+Observed> stg_ann_frame_info Data
+Observed> stg_ap_0_ret_str Data
+Observed> stg_ap_stack_entries Data
+Observed> stg_ap_stk_ Code
+Observed> stg_ap_stk_d Code
+Observed> stg_ap_stk_f Code
+Observed> stg_ap_stk_l Code
+Observed> stg_ap_stk_n Code
+Observed> stg_ap_stk_nn Code
+Observed> stg_ap_stk_nnn Code
+Observed> stg_ap_stk_nnp Code
+Observed> stg_ap_stk_np Code
+Observed> stg_ap_stk_npn Code
+Observed> stg_ap_stk_npp Code
+Observed> stg_ap_stk_p Code
+Observed> stg_ap_stk_pn Code
+Observed> stg_ap_stk_pnn Code
+Observed> stg_ap_stk_pnp Code
+Observed> stg_ap_stk_pp Code
+Observed> stg_ap_stk_ppn Code
+Observed> stg_ap_stk_ppp Code
+Observed> stg_ap_stk_pppp Code
+Observed> stg_ap_stk_ppppp Code
+Observed> stg_ap_stk_pppppp Code
+Observed> stg_ap_stk_ppppppp Code
+Observed> stg_ap_stk_pppppppp Code
+Observed> stg_ap_stk_v16 Code
+Observed> stg_ap_stk_v32 Code
+Observed> stg_ap_stk_v64 Code
+Observed> stg_apply_interp_info Data
+Observed> stg_atomically_frame_info Data
+Observed> stg_atomically_info Data
+Observed> stg_atomically_waiting_frame_info Data
+Observed> stg_block_blackhole Code
+Observed> stg_block_io_int Code
+Observed> stg_block_io_int_info Data
+Observed> stg_block_io_unit Code
+Observed> stg_block_io_unit_info Data
+Observed> stg_block_putmvar_finally Code
+Observed> stg_block_putmvar_info Data
+Observed> stg_block_readmvar_finally Code
+Observed> stg_block_readmvar_info Data
+Observed> stg_block_stmwait Code
+Observed> stg_block_takemvar_finally Code
+Observed> stg_block_takemvar_info Data
+Observed> stg_block_throwto Code
+Observed> stg_block_throwto_finally Code
+Observed> stg_block_throwto_info Data
+Observed> stg_catch_frame_info Data
+Observed> stg_catch_retry_frame_info Data
+Observed> stg_catch_stm_frame_info Data
+Observed> stg_compactAddWorkerzh Code
+Observed> stg_control0zh_ll Code
+Observed> stg_ctoi_D1_info Data
+Observed> stg_ctoi_F1_info Data
+Observed> stg_ctoi_L1_info Data
+Observed> stg_ctoi_R1n_info Data
+Observed> stg_ctoi_R1p_info Data
+Observed> stg_ctoi_V_info Data
+Observed> stg_dead_thread_info Data
+Observed> stg_do_unpack_cstring Code
+Observed> stg_do_unpack_cstring_utf8 Code
+Observed> stg_dummy_ret_closure Data
+Observed> stg_dummy_ret_info Data
+Observed> stg_forceIO_info Data
+Observed> stg_gc_prim Code
+Observed> stg_gc_prim_p_ll Code
+Observed> stg_gc_prim_p_ll_info Data
+Observed> stg_gc_prim_pp_ll Code
+Observed> stg_gc_prim_pp_ll_info Data
+Observed> stg_handleIOSubmitResult Code
+Observed> stg_keepAlive_frame_info Data
+Observed> stg_killMyself Code
+Observed> stg_marked_upd_frame_info Data
+Observed> stg_maskAsyncExceptionszh_ret_info Data
+Observed> stg_maskUninterruptiblezh_ret_info Data
+Observed> stg_noDuplicate_info Data
+Observed> stg_overwritingClosure Code
+Observed> stg_overwritingClosureSize Code
+Observed> stg_prompt_frame_info Data
+Observed> stg_raise_info Data
+Observed> stg_raise_ret_info Data
+Observed> stg_restore_cccs_d_info Data
+Observed> stg_restore_cccs_eval_info Data
+Observed> stg_restore_cccs_v16_info Data
+Observed> stg_restore_cccs_v32_info Data
+Observed> stg_restore_cccs_v64_info Data
+Observed> stg_returnToSched Code
+Observed> stg_returnToSchedButFirst Code
+Observed> stg_returnToSchedNotPaused Code
+Observed> stg_returnToStackTop Code
+Observed> stg_stack_save_entries Data
+Observed> stg_stack_underflow_frame_d_info Data
+Observed> stg_stack_underflow_frame_v16_info Data
+Observed> stg_stack_underflow_frame_v32_info Data
+Observed> stg_stack_underflow_frame_v64_info Data
+Observed> stg_stk_save_ Code
+Observed> stg_stk_save_d Code
+Observed> stg_stk_save_f Code
+Observed> stg_stk_save_l Code
+Observed> stg_stk_save_n Code
+Observed> stg_stk_save_nn Code
+Observed> stg_stk_save_nnn Code
+Observed> stg_stk_save_nnp Code
+Observed> stg_stk_save_np Code
+Observed> stg_stk_save_npn Code
+Observed> stg_stk_save_npp Code
+Observed> stg_stk_save_p Code
+Observed> stg_stk_save_pn Code
+Observed> stg_stk_save_pnn Code
+Observed> stg_stk_save_pnp Code
+Observed> stg_stk_save_pp Code
+Observed> stg_stk_save_ppn Code
+Observed> stg_stk_save_ppp Code
+Observed> stg_stk_save_pppp Code
+Observed> stg_stk_save_ppppp Code
+Observed> stg_stk_save_pppppp Code
+Observed> stg_stk_save_ppppppp Code
+Observed> stg_stk_save_pppppppp Code
+Observed> stg_stk_save_v16 Code
+Observed> stg_stk_save_v32 Code
+Observed> stg_stk_save_v64 Code
+Observed> stg_stop_thread_info Data
+Observed> stg_threadFinished Code
+Observed> stg_unmaskAsyncExceptionszh_ret_info Data
+Observed> stg_writeSlopMarker Code
+Observed> sync_phase_marking_budget Data
+Observed> sysErrorMsgFn Data
+Observed> tagConstr Code
+Observed> thunk_sizeW_fromITBL Code
+Observed> timedWaitCondition Code
+Observed> totalArgumentSize Code
+Observed> total_ticks Data
+Observed> unloadNativeObj Code
+Observed> user_era Data
+Observed> validate_trec_optimistic Code
+Observed> vbarf Code
+Observed> vdebugBelch Code
+Observed> vectorSupportGlobalVar Data
+Observed> verifyGot Code
+Observed> verrorBelch Code
+Observed> vsysErrorBelch Code
+Observed> waitCondition Code
+Observed> warnMissingKBLibraryPaths Code
+Observed> yieldThread Code
+
+Symbols observed in the DSO and declared in RtsSymbols.c but with conflicting type:
+WrongType> ALLOC_BH_adm (declared, observed): (Code, Data)
+WrongType> ALLOC_BH_ctr (declared, observed): (Code, Data)
+WrongType> ALLOC_BH_gds (declared, observed): (Code, Data)
+WrongType> ALLOC_BH_slp (declared, observed): (Code, Data)
+WrongType> ALLOC_CON_adm (declared, observed): (Code, Data)
+WrongType> ALLOC_CON_ctr (declared, observed): (Code, Data)
+WrongType> ALLOC_CON_gds (declared, observed): (Code, Data)
+WrongType> ALLOC_CON_slp (declared, observed): (Code, Data)
+WrongType> ALLOC_FUN_adm (declared, observed): (Code, Data)
+WrongType> ALLOC_FUN_ctr (declared, observed): (Code, Data)
+WrongType> ALLOC_FUN_gds (declared, observed): (Code, Data)
+WrongType> ALLOC_FUN_slp (declared, observed): (Code, Data)
+WrongType> ALLOC_HEAP_ctr (declared, observed): (Code, Data)
+WrongType> ALLOC_HEAP_tot (declared, observed): (Code, Data)
+WrongType> ALLOC_PAP_adm (declared, observed): (Code, Data)
+WrongType> ALLOC_PAP_ctr (declared, observed): (Code, Data)
+WrongType> ALLOC_PAP_gds (declared, observed): (Code, Data)
+WrongType> ALLOC_PAP_slp (declared, observed): (Code, Data)
+WrongType> ALLOC_PRIM_adm (declared, observed): (Code, Data)
+WrongType> ALLOC_PRIM_ctr (declared, observed): (Code, Data)
+WrongType> ALLOC_PRIM_gds (declared, observed): (Code, Data)
+WrongType> ALLOC_PRIM_slp (declared, observed): (Code, Data)
+WrongType> ALLOC_RTS_ctr (declared, observed): (Code, Data)
+WrongType> ALLOC_RTS_tot (declared, observed): (Code, Data)
+WrongType> ALLOC_SE_THK_ctr (declared, observed): (Code, Data)
+WrongType> ALLOC_STACK_ctr (declared, observed): (Code, Data)
+WrongType> ALLOC_STACK_tot (declared, observed): (Code, Data)
+WrongType> ALLOC_THK_adm (declared, observed): (Code, Data)
+WrongType> ALLOC_THK_gds (declared, observed): (Code, Data)
+WrongType> ALLOC_THK_slp (declared, observed): (Code, Data)
+WrongType> ALLOC_TSO_ctr (declared, observed): (Code, Data)
+WrongType> ALLOC_TSO_tot (declared, observed): (Code, Data)
+WrongType> ALLOC_TUP_adm (declared, observed): (Code, Data)
+WrongType> ALLOC_TUP_ctr (declared, observed): (Code, Data)
+WrongType> ALLOC_TUP_gds (declared, observed): (Code, Data)
+WrongType> ALLOC_TUP_slp (declared, observed): (Code, Data)
+WrongType> ALLOC_UP_THK_ctr (declared, observed): (Code, Data)
+WrongType> CATCHF_PUSHED_ctr (declared, observed): (Code, Data)
+WrongType> ENT_AP_STACK_ctr (declared, observed): (Code, Data)
+WrongType> ENT_AP_ctr (declared, observed): (Code, Data)
+WrongType> ENT_BH_ctr (declared, observed): (Code, Data)
+WrongType> ENT_CONTINUATION_ctr (declared, observed): (Code, Data)
+WrongType> ENT_DYN_CON_ctr (declared, observed): (Code, Data)
+WrongType> ENT_DYN_FUN_DIRECT_ctr (declared, observed): (Code, Data)
+WrongType> ENT_DYN_IND_ctr (declared, observed): (Code, Data)
+WrongType> ENT_DYN_THK_MANY_ctr (declared, observed): (Code, Data)
+WrongType> ENT_DYN_THK_SINGLE_ctr (declared, observed): (Code, Data)
+WrongType> ENT_LNE_ctr (declared, observed): (Code, Data)
+WrongType> ENT_PAP_ctr (declared, observed): (Code, Data)
+WrongType> ENT_PERM_IND_ctr (declared, observed): (Code, Data)
+WrongType> ENT_STATIC_CON_ctr (declared, observed): (Code, Data)
+WrongType> ENT_STATIC_FUN_DIRECT_ctr (declared, observed): (Code, Data)
+WrongType> ENT_STATIC_IND_ctr (declared, observed): (Code, Data)
+WrongType> ENT_STATIC_THK_MANY_ctr (declared, observed): (Code, Data)
+WrongType> ENT_STATIC_THK_SINGLE_ctr (declared, observed): (Code, Data)
+WrongType> ENT_VIA_NODE_ctr (declared, observed): (Code, Data)
+WrongType> GC_FAILED_PROMOTION_ctr (declared, observed): (Code, Data)
+WrongType> GC_SEL_ABANDONED_ctr (declared, observed): (Code, Data)
+WrongType> GC_SEL_MAJOR_ctr (declared, observed): (Code, Data)
+WrongType> GC_SEL_MINOR_ctr (declared, observed): (Code, Data)
+WrongType> HEAP_CHK_ctr (declared, observed): (Code, Data)
+WrongType> KNOWN_CALL_EXTRA_ARGS_ctr (declared, observed): (Code, Data)
+WrongType> KNOWN_CALL_TOO_FEW_ARGS_ctr (declared, observed): (Code, Data)
+WrongType> KNOWN_CALL_ctr (declared, observed): (Code, Data)
+WrongType> MULTI_CHUNK_SLOW_CALL_CHUNKS_ctr (declared, observed): (Code, Data)
+WrongType> MULTI_CHUNK_SLOW_CALL_ctr (declared, observed): (Code, Data)
+WrongType> MainCapability (declared, observed): (Code, Data)
+WrongType> RET_NEW_ctr (declared, observed): (Code, Data)
+WrongType> RET_NEW_hst (declared, observed): (Code, Data)
+WrongType> RET_OLD_ctr (declared, observed): (Code, Data)
+WrongType> RET_OLD_hst (declared, observed): (Code, Data)
+WrongType> RET_SEMI_loads_avoided (declared, observed): (Code, Data)
+WrongType> RET_UNBOXED_TUP_ctr (declared, observed): (Code, Data)
+WrongType> RET_UNBOXED_TUP_hst (declared, observed): (Code, Data)
+WrongType> SLOW_CALL_FUN_CORRECT_ctr (declared, observed): (Code, Data)
+WrongType> SLOW_CALL_FUN_TOO_FEW_ctr (declared, observed): (Code, Data)
+WrongType> SLOW_CALL_FUN_TOO_MANY_ctr (declared, observed): (Code, Data)
+WrongType> SLOW_CALL_PAP_CORRECT_ctr (declared, observed): (Code, Data)
+WrongType> SLOW_CALL_PAP_TOO_FEW_ctr (declared, observed): (Code, Data)
+WrongType> SLOW_CALL_PAP_TOO_MANY_ctr (declared, observed): (Code, Data)
+WrongType> SLOW_CALL_UNEVALD_ctr (declared, observed): (Code, Data)
+WrongType> SLOW_CALL_ctr (declared, observed): (Code, Data)
+WrongType> SLOW_CALL_fast_d_ctr (declared, observed): (Code, Data)
+WrongType> SLOW_CALL_fast_f_ctr (declared, observed): (Code, Data)
+WrongType> SLOW_CALL_fast_l_ctr (declared, observed): (Code, Data)
+WrongType> SLOW_CALL_fast_n_ctr (declared, observed): (Code, Data)
+WrongType> SLOW_CALL_fast_p_ctr (declared, observed): (Code, Data)
+WrongType> SLOW_CALL_fast_pp_ctr (declared, observed): (Code, Data)
+WrongType> SLOW_CALL_fast_ppp_ctr (declared, observed): (Code, Data)
+WrongType> SLOW_CALL_fast_pppp_ctr (declared, observed): (Code, Data)
+WrongType> SLOW_CALL_fast_ppppp_ctr (declared, observed): (Code, Data)
+WrongType> SLOW_CALL_fast_pppppp_ctr (declared, observed): (Code, Data)
+WrongType> SLOW_CALL_fast_pppv_ctr (declared, observed): (Code, Data)
+WrongType> SLOW_CALL_fast_ppv_ctr (declared, observed): (Code, Data)
+WrongType> SLOW_CALL_fast_pv_ctr (declared, observed): (Code, Data)
+WrongType> SLOW_CALL_fast_v16_ctr (declared, observed): (Code, Data)
+WrongType> SLOW_CALL_fast_v_ctr (declared, observed): (Code, Data)
+WrongType> STK_CHK_ctr (declared, observed): (Code, Data)
+WrongType> TAG_TAGGED_miss (declared, observed): (Code, Data)
+WrongType> TAG_TAGGED_pred (declared, observed): (Code, Data)
+WrongType> TAG_UNTAGGED_miss (declared, observed): (Code, Data)
+WrongType> TAG_UNTAGGED_pred (declared, observed): (Code, Data)
+WrongType> UNKNOWN_CALL_ctr (declared, observed): (Code, Data)
+WrongType> UPDF_OMITTED_ctr (declared, observed): (Code, Data)
+WrongType> UPDF_PUSHED_ctr (declared, observed): (Code, Data)
+WrongType> UPDF_RCC_OMITTED_ctr (declared, observed): (Code, Data)
+WrongType> UPDF_RCC_PUSHED_ctr (declared, observed): (Code, Data)
+WrongType> UPD_CAF_BH_SINGLE_ENTRY_ctr (declared, observed): (Code, Data)
+WrongType> UPD_CAF_BH_UPDATABLE_ctr (declared, observed): (Code, Data)
+WrongType> UPD_CON_IN_NEW_ctr (declared, observed): (Code, Data)
+WrongType> UPD_CON_IN_PLACE_ctr (declared, observed): (Code, Data)
+WrongType> UPD_NEW_IND_ctr (declared, observed): (Code, Data)
+WrongType> UPD_NEW_PERM_IND_ctr (declared, observed): (Code, Data)
+WrongType> UPD_OLD_IND_ctr (declared, observed): (Code, Data)
+WrongType> UPD_OLD_PERM_IND_ctr (declared, observed): (Code, Data)
+WrongType> UPD_PAP_IN_NEW_ctr (declared, observed): (Code, Data)
+WrongType> UPD_PAP_IN_PLACE_ctr (declared, observed): (Code, Data)
+WrongType> UPD_SQUEEZED_ctr (declared, observed): (Code, Data)
+WrongType> VERY_SLOW_CALL_ctr (declared, observed): (Code, Data)
+WrongType> __stg_gc_enter_1 (declared, observed): (Data, Code)
+WrongType> __stg_gc_fun (declared, observed): (Data, Code)
+WrongType> closure_flags (declared, observed): (Code, Data)
+WrongType> defaultRtsConfig (declared, observed): (Code, Data)
+WrongType> g0 (declared, observed): (Code, Data)
+WrongType> ghc_unique_counter64 (declared, observed): (Code, Data)
+WrongType> ghc_unique_inc (declared, observed): (Code, Data)
+WrongType> keepCAFs (declared, observed): (Code, Data)
+WrongType> large_alloc_lim (declared, observed): (Code, Data)
+WrongType> prog_argc (declared, observed): (Code, Data)
+WrongType> prog_argv (declared, observed): (Code, Data)
+WrongType> stable_name_table (declared, observed): (Code, Data)
+WrongType> stable_ptr_table (declared, observed): (Code, Data)
+WrongType> stg_CHARLIKE_closure (declared, observed): (Code, Data)
+WrongType> stg_INTLIKE_closure (declared, observed): (Code, Data)
+WrongType> stg_absentErrorzh (declared, observed): (Data, Code)
+WrongType> stg_annotateStackzh (declared, observed): (Data, Code)
+WrongType> stg_ap_0_fast (declared, observed): (Data, Code)
+WrongType> stg_ap_d_fast (declared, observed): (Data, Code)
+WrongType> stg_ap_f_fast (declared, observed): (Data, Code)
+WrongType> stg_ap_l_fast (declared, observed): (Data, Code)
+WrongType> stg_ap_n_fast (declared, observed): (Data, Code)
+WrongType> stg_ap_p_fast (declared, observed): (Data, Code)
+WrongType> stg_ap_pp_fast (declared, observed): (Data, Code)
+WrongType> stg_ap_ppp_fast (declared, observed): (Data, Code)
+WrongType> stg_ap_pppp_fast (declared, observed): (Data, Code)
+WrongType> stg_ap_ppppp_fast (declared, observed): (Data, Code)
+WrongType> stg_ap_pppppp_fast (declared, observed): (Data, Code)
+WrongType> stg_ap_pppv_fast (declared, observed): (Data, Code)
+WrongType> stg_ap_ppv_fast (declared, observed): (Data, Code)
+WrongType> stg_ap_pv_fast (declared, observed): (Data, Code)
+WrongType> stg_ap_v16_fast (declared, observed): (Data, Code)
+WrongType> stg_ap_v32_fast (declared, observed): (Data, Code)
+WrongType> stg_ap_v64_fast (declared, observed): (Data, Code)
+WrongType> stg_ap_v_fast (declared, observed): (Data, Code)
+WrongType> stg_atomicModifyMutVar2zh (declared, observed): (Data, Code)
+WrongType> stg_atomicModifyMutVarzuzh (declared, observed): (Data, Code)
+WrongType> stg_atomicallyzh (declared, observed): (Data, Code)
+WrongType> stg_block_noregs (declared, observed): (Data, Code)
+WrongType> stg_block_putmvar (declared, observed): (Data, Code)
+WrongType> stg_block_readmvar (declared, observed): (Data, Code)
+WrongType> stg_block_takemvar (declared, observed): (Data, Code)
+WrongType> stg_casArrayzh (declared, observed): (Data, Code)
+WrongType> stg_casInt16Arrayzh (declared, observed): (Data, Code)
+WrongType> stg_casInt32Arrayzh (declared, observed): (Data, Code)
+WrongType> stg_casInt64Arrayzh (declared, observed): (Data, Code)
+WrongType> stg_casInt8Arrayzh (declared, observed): (Data, Code)
+WrongType> stg_casIntArrayzh (declared, observed): (Data, Code)
+WrongType> stg_casMutVarzh (declared, observed): (Data, Code)
+WrongType> stg_casSmallArrayzh (declared, observed): (Data, Code)
+WrongType> stg_castDoubleToWord64zh (declared, observed): (Data, Code)
+WrongType> stg_castFloatToWord32zh (declared, observed): (Data, Code)
+WrongType> stg_castWord32ToFloatzh (declared, observed): (Data, Code)
+WrongType> stg_castWord64ToDoublezh (declared, observed): (Data, Code)
+WrongType> stg_catchRetryzh (declared, observed): (Data, Code)
+WrongType> stg_catchSTMzh (declared, observed): (Data, Code)
+WrongType> stg_catchzh (declared, observed): (Data, Code)
+WrongType> stg_clearCCSzh (declared, observed): (Data, Code)
+WrongType> stg_cloneArrayzh (declared, observed): (Data, Code)
+WrongType> stg_cloneMutableArrayzh (declared, observed): (Data, Code)
+WrongType> stg_cloneSmallArrayzh (declared, observed): (Data, Code)
+WrongType> stg_cloneSmallMutableArrayzh (declared, observed): (Data, Code)
+WrongType> stg_closureSizzezh (declared, observed): (Data, Code)
+WrongType> stg_compactAddWithSharingzh (declared, observed): (Data, Code)
+WrongType> stg_compactAddzh (declared, observed): (Data, Code)
+WrongType> stg_compactAllocateBlockzh (declared, observed): (Data, Code)
+WrongType> stg_compactContainsAnyzh (declared, observed): (Data, Code)
+WrongType> stg_compactContainszh (declared, observed): (Data, Code)
+WrongType> stg_compactFixupPointerszh (declared, observed): (Data, Code)
+WrongType> stg_compactGetFirstBlockzh (declared, observed): (Data, Code)
+WrongType> stg_compactGetNextBlockzh (declared, observed): (Data, Code)
+WrongType> stg_compactNewzh (declared, observed): (Data, Code)
+WrongType> stg_compactResizzezh (declared, observed): (Data, Code)
+WrongType> stg_compactSizzezh (declared, observed): (Data, Code)
+WrongType> stg_copyArray_barrier (declared, observed): (Data, Code)
+WrongType> stg_copyArrayzh (declared, observed): (Data, Code)
+WrongType> stg_copyMutableArrayzh (declared, observed): (Data, Code)
+WrongType> stg_copySmallArrayzh (declared, observed): (Data, Code)
+WrongType> stg_copySmallMutableArrayzh (declared, observed): (Data, Code)
+WrongType> stg_deRefStablePtrzh (declared, observed): (Data, Code)
+WrongType> stg_deRefWeakzh (declared, observed): (Data, Code)
+WrongType> stg_decodeDoublezu2Intzh (declared, observed): (Data, Code)
+WrongType> stg_decodeDoublezuInt64zh (declared, observed): (Data, Code)
+WrongType> stg_decodeFloatzuIntzh (declared, observed): (Data, Code)
+WrongType> stg_delayzh (declared, observed): (Data, Code)
+WrongType> stg_exit (declared, observed): (Data, Code)
+WrongType> stg_freezzeArrayzh (declared, observed): (Data, Code)
+WrongType> stg_freezzeSmallArrayzh (declared, observed): (Data, Code)
+WrongType> stg_gc_bpp (declared, observed): (Data, Code)
+WrongType> stg_gc_d1 (declared, observed): (Data, Code)
+WrongType> stg_gc_f1 (declared, observed): (Data, Code)
+WrongType> stg_gc_ii (declared, observed): (Data, Code)
+WrongType> stg_gc_ip (declared, observed): (Data, Code)
+WrongType> stg_gc_l1 (declared, observed): (Data, Code)
+WrongType> stg_gc_noregs (declared, observed): (Data, Code)
+WrongType> stg_gc_pi (declared, observed): (Data, Code)
+WrongType> stg_gc_pp (declared, observed): (Data, Code)
+WrongType> stg_gc_ppp (declared, observed): (Data, Code)
+WrongType> stg_gc_pppp (declared, observed): (Data, Code)
+WrongType> stg_gc_ppppp (declared, observed): (Data, Code)
+WrongType> stg_gc_prim_n (declared, observed): (Data, Code)
+WrongType> stg_gc_prim_p (declared, observed): (Data, Code)
+WrongType> stg_gc_prim_pp (declared, observed): (Data, Code)
+WrongType> stg_gc_unbx_r1 (declared, observed): (Data, Code)
+WrongType> stg_gc_unpt_r1 (declared, observed): (Data, Code)
+WrongType> stg_getApStackValzh (declared, observed): (Data, Code)
+WrongType> stg_getMaskingStatezh (declared, observed): (Data, Code)
+WrongType> stg_getOtherThreadAllocationCounterzh (declared, observed): (Data, Code)
+WrongType> stg_getSparkzh (declared, observed): (Data, Code)
+WrongType> stg_getThreadAllocationCounterzh (declared, observed): (Data, Code)
+WrongType> stg_isByteArrayPinnedzh (declared, observed): (Data, Code)
+WrongType> stg_isByteArrayWeaklyPinnedzh (declared, observed): (Data, Code)
+WrongType> stg_isCurrentThreadBoundzh (declared, observed): (Data, Code)
+WrongType> stg_isEmptyMVarzh (declared, observed): (Data, Code)
+WrongType> stg_isMutableByteArrayPinnedzh (declared, observed): (Data, Code)
+WrongType> stg_isMutableByteArrayWeaklyPinnedzh (declared, observed): (Data, Code)
+WrongType> stg_keepAlivezh (declared, observed): (Data, Code)
+WrongType> stg_killThreadzh (declared, observed): (Data, Code)
+WrongType> stg_labelThreadzh (declared, observed): (Data, Code)
+WrongType> stg_listThreadszh (declared, observed): (Data, Code)
+WrongType> stg_makeStablePtrzh (declared, observed): (Data, Code)
+WrongType> stg_maskAsyncExceptionszh (declared, observed): (Data, Code)
+WrongType> stg_maskUninterruptiblezh (declared, observed): (Data, Code)
+WrongType> stg_mkApUpd0zh (declared, observed): (Data, Code)
+WrongType> stg_newAlignedPinnedByteArrayzh (declared, observed): (Data, Code)
+WrongType> stg_newArrayzh (declared, observed): (Data, Code)
+WrongType> stg_newBCOzh (declared, observed): (Data, Code)
+WrongType> stg_newByteArrayzh (declared, observed): (Data, Code)
+WrongType> stg_newConAppObjzh (declared, observed): (Data, Code)
+WrongType> stg_newMVarzh (declared, observed): (Data, Code)
+WrongType> stg_newMutVarzh (declared, observed): (Data, Code)
+WrongType> stg_newPinnedByteArrayzh (declared, observed): (Data, Code)
+WrongType> stg_newSmallArrayzh (declared, observed): (Data, Code)
+WrongType> stg_newTVarzh (declared, observed): (Data, Code)
+WrongType> stg_noDuplicatezh (declared, observed): (Data, Code)
+WrongType> stg_numSparkszh (declared, observed): (Data, Code)
+WrongType> stg_paniczh (declared, observed): (Data, Code)
+WrongType> stg_putMVarzh (declared, observed): (Data, Code)
+WrongType> stg_raiseDivZZerozh (declared, observed): (Data, Code)
+WrongType> stg_raiseIOzh (declared, observed): (Data, Code)
+WrongType> stg_raiseOverflowzh (declared, observed): (Data, Code)
+WrongType> stg_raiseUnderflowzh (declared, observed): (Data, Code)
+WrongType> stg_raisezh (declared, observed): (Data, Code)
+WrongType> stg_readMVarzh (declared, observed): (Data, Code)
+WrongType> stg_readTVarIOzh (declared, observed): (Data, Code)
+WrongType> stg_readTVarzh (declared, observed): (Data, Code)
+WrongType> stg_resizzeMutableByteArrayzh (declared, observed): (Data, Code)
+WrongType> stg_retryzh (declared, observed): (Data, Code)
+WrongType> stg_setOtherThreadAllocationCounterzh (declared, observed): (Data, Code)
+WrongType> stg_setThreadAllocationCounterzh (declared, observed): (Data, Code)
+WrongType> stg_shrinkMutableByteArrayzh (declared, observed): (Data, Code)
+WrongType> stg_shrinkSmallMutableArrayzh (declared, observed): (Data, Code)
+WrongType> stg_takeMVarzh (declared, observed): (Data, Code)
+WrongType> stg_thawArrayzh (declared, observed): (Data, Code)
+WrongType> stg_thawSmallArrayzh (declared, observed): (Data, Code)
+WrongType> stg_threadLabelzh (declared, observed): (Data, Code)
+WrongType> stg_threadStatuszh (declared, observed): (Data, Code)
+WrongType> stg_traceBinaryEventzh (declared, observed): (Data, Code)
+WrongType> stg_traceEventzh (declared, observed): (Data, Code)
+WrongType> stg_traceMarkerzh (declared, observed): (Data, Code)
+WrongType> stg_tryReadMVarzh (declared, observed): (Data, Code)
+WrongType> stg_tryTakeMVarzh (declared, observed): (Data, Code)
+WrongType> stg_unmaskAsyncExceptionszh (declared, observed): (Data, Code)
+WrongType> stg_unpackClosurezh (declared, observed): (Data, Code)
+WrongType> stg_unsafeThawArrayzh (declared, observed): (Data, Code)
+WrongType> stg_unsafeThawSmallArrayzh (declared, observed): (Data, Code)
+WrongType> stg_waitReadzh (declared, observed): (Data, Code)
+WrongType> stg_waitWritezh (declared, observed): (Data, Code)
+WrongType> stg_whereFromzh (declared, observed): (Data, Code)
+WrongType> stg_writeTVarzh (declared, observed): (Data, Code)
+WrongType> stg_yield_noregs (declared, observed): (Data, Code)
+WrongType> stg_yield_to_interpreter (declared, observed): (Data, Code)
+WrongType> stg_yieldzh (declared, observed): (Data, Code)
+WrongType> ticky_slow_call_unevald (declared, observed): (Code, Data)
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/329c8ac9477e00d7f6e33a3b91cb65…
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/329c8ac9477e00d7f6e33a3b91cb65…
You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help
1
0
[Git][ghc/ghc][wip/27971] WinIO: Don't confuse Win32 errors and NTSTATUS codes
by Zubin (@wz1000) 20 Oct '26
by Zubin (@wz1000) 20 Oct '26
20 Oct '26
Zubin pushed to branch wip/27971 at Glasgow Haskell Compiler / GHC
Commits:
4183ff2e by Zubin Duggal at 2026-10-20T16:14:57+05:30
WinIO: Don't confuse Win32 errors and NTSTATUS codes
WinIO misinterpreted NTSTATUS codes as Win32 errors, and had some ad hoc logic to deal with this.
However, this logic was incomplete and resulted in the mishandling of results of certain I/O operations
(see testcase).
Now the `ErrCode` argument in `CompletionCallback` is always the Win32 error code,
and we convert the NTSTATUS using RtlNtStatusToDosError.
Also when an overlapped/async operation such as ReadFile fails immediately,
with a Win32 error such as ERROR_BROKEN_PIPE, we don't get a further completion
notification, so we never recorded the failure. This worked by coincidence
earlier, because `STATUS_PENDING` happened to have the same numerical code as
`ERROR_NO_MORE_ITEMS`, but it doesn't now that we translate `STATUS_PENDING`
into the correct Win32 error code (`ERROR_IO_PENDING`). Fix this by using
the Win32 error value we obtained when an operation returns `STATUS_PENDING`.
Fixes #27971
- - - - -
7 changed files:
- + changelog.d/27971
- + libraries/base/tests/IO/T27971.hs
- + libraries/base/tests/IO/T27971.stdout
- libraries/base/tests/IO/all.T
- libraries/ghc-internal/src/GHC/Internal/Event/Windows.hsc
- libraries/ghc-internal/src/GHC/Internal/Event/Windows/FFI.hsc
- libraries/ghc-internal/src/GHC/Internal/IO/Windows/Handle.hsc
Changes:
=====================================
changelog.d/27971
=====================================
@@ -0,0 +1,5 @@
+section: base
+synopsis: WinIO: Fix a bug where WinIO could confuse Win32 and NTSTATUS error codes
+mrs: !16838
+issues: #27971
+description: This could lead to the result of certian I/O operations being misinterpreted, mishandled and misreported
=====================================
libraries/base/tests/IO/T27971.hs
=====================================
@@ -0,0 +1,18 @@
+import Control.Concurrent
+import qualified Data.ByteString as B
+import System.IO
+import System.Process.CommunicationHandle
+
+main :: IO ()
+main = do
+ (r, ch) <- createWeReadTheyWritePipe
+ w <- openCommunicationHandleWrite ch
+ mapM_ (`hSetBinaryMode` True) [r, w]
+ done <- newEmptyMVar
+ _ <- forkIO $ do
+ B.hPut w (B.replicate 65536 0)
+ hClose w
+ putMVar done ()
+ s <- B.hGetContents r
+ takeMVar done
+ print (B.length s)
=====================================
libraries/base/tests/IO/T27971.stdout
=====================================
@@ -0,0 +1 @@
+65536
=====================================
libraries/base/tests/IO/all.T
=====================================
@@ -198,3 +198,10 @@ test('osHandles002WindowsHandles', only_ways(['winio', 'winio_threaded']), compi
# executables, emulates part of POSIX, enough for this test. As a
# result, this test could be generalized to also supporting Windows, but
# this would likely involve creating bindings to C code.
+test('T27971',
+ [ unless(opsys('mingw32'), skip)
+ , extra_ways(['winio_threaded'])
+ , only_ways(['threaded1', 'winio_threaded'])
+ , extra_hc_opts('-package process')
+ ],
+ compile_and_run, [''])
=====================================
libraries/ghc-internal/src/GHC/Internal/Event/Windows.hsc
=====================================
@@ -591,7 +591,7 @@ withOverlappedEx mgr fname h async offset startCB completionCB = do
++ " ptr_lpol:" ++ show ptr_lpol
++ " *ptr_lpol:" ++ show cdDataCheck
- startCBResult <- startCB lpol `onException`
+ (startCBResult, startErr) <- startCB lpol `onException`
(CbError `fmap` Win32.getLastError) >>= \result -> do
-- Check to see if the operation was completed on a
-- non-overlapping handle or was completed immediately.
@@ -629,7 +629,7 @@ withOverlappedEx mgr fname h async offset startCB completionCB = do
-- I don't know what the best default here is...
| otherwise -> CbPending
_ -> result
- case result' of
+ fmap (\r -> (r, err)) $ case result' of
CbNone _ -> error "withOverlappedEx: CbNone shouldn't happen."
CbIncomplete -> do
debugIO $ "handling incomplete request synchronously " ++ show (h, lpol)
@@ -728,13 +728,13 @@ withOverlappedEx mgr fname h async offset startCB completionCB = do
debugIO $ dbgMsg $ ":: done bytes: " ++ show bytes
case bytes of
Just res -> completionCB 0 res
- Nothing -> do err <- FFI.overlappedIOStatus lpol
+ Nothing -> do status <- FFI.overlappedIOStatus lpol
+ err <- if status == #{const STATUS_PENDING}
+ then return startErr
+ else FFI.overlappedIOErrCode lpol
numBytes <- FFI.overlappedIONumBytes lpol
- -- TODO: Remap between STATUS_ and ERROR_ instead
- -- of re-interpret here. But for now, don't care.
- let err' = fromIntegral err
- debugIO $ dbgMsg $ ":: done callback: " ++ show err' ++ " - " ++ show numBytes
- completionCB err' (fromIntegral numBytes)
+ debugIO $ dbgMsg $ ":: done callback: " ++ show err ++ " - " ++ show numBytes
+ completionCB err (fromIntegral numBytes)
else readMVar signal
CbError err -> do
reqs3 <- removeRequest
@@ -1097,10 +1097,7 @@ processCompletion Manager{..} n delay = do
cb <- deRefStablePtr (cdCallback payload)
reqs <- removeRequest
debugIO $ "-1.. " ++ show reqs ++ " requests queued."
- status <- FFI.overlappedIOStatus (lpOverlapped oe)
- -- TODO: Remap between STATUS_ and ERROR_ instead
- -- of re-interpret here. But for now, don't care.
- let status' = fromIntegral status
+ status' <- FFI.overlappedIOErrCode (lpOverlapped oe)
-- We no longer explicitly free the memory, this is because we
-- now require the callback to free the memory since the
-- callback allocated it. This allows us to simplify memory
=====================================
libraries/ghc-internal/src/GHC/Internal/Event/Windows/FFI.hsc
=====================================
@@ -46,6 +46,7 @@ module GHC.Internal.Event.Windows.FFI (
zeroOverlapped,
pokeOffsetOverlapped,
overlappedIOStatus,
+ overlappedIOErrCode,
overlappedIONumBytes,
-- * Cancel pending I/O
@@ -209,11 +210,17 @@ getQueuedCompletionStatusEx iocp arr timeout =
overlappedIOStatus :: LPOVERLAPPED -> IO NTSTATUS
overlappedIOStatus lpol = do
status <- #{peek OVERLAPPED, Internal} lpol
- -- TODO: Map NTSTATUS to ErrCode?
- -- See https://github.com/libuv/libuv/blob/b12624c13693c4d29ca84b3556eadc9e9c0936a…
return status
{-# INLINE overlappedIOStatus #-}
+foreign import ccall unsafe "winternl.h RtlNtStatusToDosError"
+ c_RtlNtStatusToDosError :: NTSTATUS -> IO ULONG
+
+overlappedIOErrCode :: LPOVERLAPPED -> IO ErrCode
+overlappedIOErrCode lpol = do
+ status <- overlappedIOStatus lpol
+ c_RtlNtStatusToDosError status
+
overlappedIONumBytes :: LPOVERLAPPED -> IO ULONG_PTR
overlappedIONumBytes lpol = do
bytes <- #{peek OVERLAPPED, InternalHigh} lpol
=====================================
libraries/ghc-internal/src/GHC/Internal/IO/Windows/Handle.hsc
=====================================
@@ -463,9 +463,7 @@ hwndRead hwnd ptr offset bytes = do
completionCB err dwBytes
| err == #{const ERROR_SUCCESS} = Mgr.ioSuccess $ fromIntegral dwBytes
| err == #{const ERROR_HANDLE_EOF} = Mgr.ioSuccess 0
- | err == #{const STATUS_END_OF_FILE} = Mgr.ioSuccess 0
| err == #{const ERROR_BROKEN_PIPE} = Mgr.ioSuccess 0
- | err == #{const STATUS_PIPE_BROKEN} = Mgr.ioSuccess 0
| err == #{const ERROR_NO_MORE_ITEMS} = Mgr.ioSuccess $ fromIntegral dwBytes
| err == #{const ERROR_MORE_DATA} = Mgr.ioSuccess $ fromIntegral dwBytes
| otherwise = Mgr.ioFailed err
@@ -493,9 +491,7 @@ hwndReadNonBlocking hwnd ptr offset bytes
completionCB err dwBytes
| err == #{const ERROR_SUCCESS} = Mgr.ioSuccess $ Just $! fromIntegral dwBytes
| err == #{const ERROR_HANDLE_EOF} = Mgr.ioSuccess Nothing
- | err == #{const STATUS_END_OF_FILE} = Mgr.ioSuccess Nothing
| err == #{const ERROR_BROKEN_PIPE} = Mgr.ioSuccess Nothing
- | err == #{const STATUS_PIPE_BROKEN} = Mgr.ioSuccess Nothing
| err == #{const ERROR_NO_MORE_ITEMS} = Mgr.ioSuccess Nothing
| err == #{const ERROR_MORE_DATA} = Mgr.ioSuccess $ Just $! fromIntegral dwBytes
| otherwise = Mgr.ioFailedAny err
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/4183ff2e42555e9f974bd0428bed9ef…
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/4183ff2e42555e9f974bd0428bed9ef…
You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help
1
0
[Git][ghc/ghc][wip/romes/async-rpc] 2 commits: GHCi.Server: handle RunTH and RunModFinalizers in forked thread
by Rodrigo Mesquita (@alt-romes) 20 Oct '26
by Rodrigo Mesquita (@alt-romes) 20 Oct '26
20 Oct '26
Rodrigo Mesquita pushed to branch wip/romes/async-rpc at Glasgow Haskell Compiler / GHC
Commits:
6f97d0cf by Andrea Vezzosi at 2026-10-20T11:35:34+01:00
GHCi.Server: handle RunTH and RunModFinalizers in forked thread
This commit makes the external interpreter able to many interpret
template haskell splices in parallel without blocking the main server
thread. Simply: fork off a thread to handle a RunTH request. The handler
will issue the reverse-requests on the given RunTH mailbox, to keep each
RunTH isolated from the others.
`handleRemoteCall` was generalised slightly by allowing `WriteReply` to
be used linearly in the handler directly, so that the caller of
`handleRemoteCall` may `WriteReply` in the forked thread rather than in
the main thread.
The only question is how can we guarantee `WriteReply` is used linearly
if we want to use it inside a forked thread? We need two guarantees:
1. `WriteReply` is used in the forked thread regardless of whether an
exception is thrown to that thread.
2. The forked thread is always awaited for, so that the main program
doesn't exit before the thread has a chance to actually use
`WriteReply`.
We introduced a small `ThreadManager` which provides a `spawnFinally`
that provides these two guarantees:
1. is guaranteed by `forkFinally`, which always runs the clean up action
exactly once.
2. is guaranteed by `ThreadManager` being abstract and the only exposed
function that introduces `ThreadManager` being `withThreadManager`,
which always awaits for all spawned threads.
Co-authored-by: Rodrigo Mesquita <rodrigo.m.mesquita(a)gmail.com>
- - - - -
42a3626a by Andrea Vezzosi at 2026-10-20T11:43:55+01:00
interp: spawn fillPipe thread, writePipe now safely interruptible
TODO
- - - - -
5 changed files:
- libraries/ghci/GHCi/Message.hs
- libraries/ghci/GHCi/Message/Async.hs
- libraries/ghci/GHCi/Server.hs
- + libraries/ghci/GHCi/Server/Threads.hs
- libraries/ghci/ghci.cabal.in
Changes:
=====================================
libraries/ghci/GHCi/Message.hs
=====================================
@@ -26,7 +26,7 @@ module GHCi.Message
, Pipe(..), mkPipeFromHandles, mkPipeFromContinuations, closePipe
, remoteCall, remoteTHCall
- , handleRemoteCall, handleRemoteTHCall
+ , handleRemoteCall, handleRemoteTHCall, writeReply
, RemoteProcessException(..), PipeReadClosedException(..)
, InboxId(..)
@@ -666,13 +666,12 @@ remoteCall pipe msg = either throwIO (pure . runGet get)
-- | Handle a single 'Message' request on the interpreter and reply to the host
handleRemoteCall :: Pipe
-> (Msg -> IO Msg) {-^ Msg hook -}
- -> (forall a. (Show a, Binary a) => Message a -> IO a)
- -> IO ()
+ -> (forall a. (Show a, Binary a) => WriteReply a %1 -> Message a -> IO r)
+ -> IO r
handleRemoteCall pipe hook k =
- handleAsyncReq pipe getMessage Nothing \writer msg0 -> IO \s0 ->
+ handleAsyncReq pipe getMessage Nothing \(WriteReply writer) msg0 -> IO \s0 ->
case hook msg0 |#> s0 of -- unIO for linearity checking
- (# s1, Msg msg #) -> case k msg |#> s1 of
- (# s2, a #) -> writeReply writer (put a) |#> s2
+ (# s1, Msg msg #) -> k (WriteReply \x -> writer (fmap put x)) msg |#> s1
-- | Send a 'THMessage' request to the host and wait for a reply from the host
-- to the interpreter.
=====================================
libraries/ghci/GHCi/Message/Async.hs
=====================================
@@ -23,8 +23,8 @@ module GHCi.Message.Async
, IddMsg(..), newAsyncManager
-- ** Consume `reply` exactly once
- , WriteReply(..)
- , writeReply, (|#>)
+ , WriteReply(..), writeReply
+ , runWriteReply, (|#>), (%>>=)
-- * Serializable exceptions
, SerializableException(..)
@@ -77,6 +77,10 @@ data Pipe = Pipe
, pipeReadClosed :: !(TVar Bool)
-- ^ After the pipe is closed, messages can no longer be received from the
-- other end, but the 'AsyncManager' queues might not be empty yet.
+ , pipeWriteClosed :: !(TVar Bool)
+ -- ^ After the write end of the pipe is closed we can no longer submit
+ -- requests or respond to ones we receive, but we might still receive
+ -- replies.
}
-- | Make a 'Pipe' from a 'Handle' to read and a 'Handle' to write.
@@ -101,6 +105,7 @@ mkPipeFromContinuations getSome putAll pipeCloseRead pipeCloseWrite = mdo
pipeLeftovers <- newIORef Nothing
pipeLock <- newMVar ()
pipeReadClosed <- newTVarIO False
+ pipeWriteClosed <- newTVarIO False
mdo
pipeAsyncMngr <- newAsyncManager p
let p = Pipe { .. }
@@ -114,20 +119,34 @@ mkPipeFromContinuations getSome putAll pipeCloseRead pipeCloseWrite = mdo
-- write. Only then do we stop the drain thread, which closes the read end
-- itself once its read has finished. On Windows its read can't be
-- interrupted, so this waits for the other side to close its write end too.
+-- Finally we stop the write thread, which could be blocked waiting for more to write.
closePipe :: Pipe -> IO ()
closePipe p@Pipe{..} = do
readAlreadyClosed <- atomically $ do
closed <- readTVar pipeReadClosed
failPendingReplies p (toException PipeReadClosedException)
pure closed
- withLock p pipeCloseWrite `finally`
- unless readAlreadyClosed
- (throwTo (asyncDrainTid pipeAsyncMngr) PipeReadClosedException)
+
+ writeAlreadyClosed <- atomically $ do
+ closed <- readTVar pipeWriteClosed
+ failPendingWrites p (toException PipeWriteClosedException)
+ pure closed
+
+ unless writeAlreadyClosed (withLock p pipeCloseWrite)
+ `finally` do
+ unless readAlreadyClosed
+ (throwTo (asyncDrainTid pipeAsyncMngr) PipeReadClosedException)
+ unless writeAlreadyClosed
+ (throwTo (asyncFillTid pipeAsyncMngr) PipeWriteClosedException)
-- | We've closed the pipe read end on this process
data PipeReadClosedException = PipeReadClosedException deriving Show
instance Exception PipeReadClosedException
+-- | We've closed the pipe write end on this process
+data PipeWriteClosedException = PipeWriteClosedException deriving Show
+instance Exception PipeWriteClosedException
+
--------------------------------------------------------------------------------
-- ** Pipe thread-safe read/write
--------------------------------------------------------------------------------
@@ -135,7 +154,14 @@ instance Exception PipeReadClosedException
-- and identify whether they are requests or replies.
writePipe :: Pipe -> IddMsg -> IO ()
-writePipe p@Pipe{..} = withLock p . putAll . execPut . put
+writePipe Pipe{..} msg = do
+ var <- newEmptyTMVarIO
+ atomically $ do
+ readTVar pipeWriteClosed >>= \case
+ True -> throwSTM PipeWriteClosedException
+ False -> pure ()
+ writeTQueue (writeQueue pipeAsyncMngr) (msg, var)
+ either throwIO pure =<< atomically (takeTMVar var)
-- | Only the 'drainPipe' thread should read from the pipe directly. Other
-- functions should wait for the matching reply to be delivered asynchronously.
@@ -229,6 +255,10 @@ handleAsyncReq pipe getit mii k = mask $ \restore -> do
-- Block waiting for request
mbreq <- atomically $ do
+ readTVar (pipeWriteClosed pipe) >>= \case
+ -- We will not be able to write the reply, so no point in continuiting.
+ True -> throwSTM PipeWriteClosedException
+ False -> pure ()
isClosed <- readTVar (pipeReadClosed pipe)
taken <- case mii of
Nothing -> tryReadTQueue $ requestsQueue am
@@ -320,7 +350,10 @@ data AsyncManager = AsyncManager
, mailboxQueues :: !(TVar (IntMap (TQueue (CorrelationId, LB.ByteString))))
-- ^ Requests for a specific 'InboxId' that can be read using
-- @'handleAsyncReq' p get (Just <mailboxid>)@.
+ , writeQueue :: !(TQueue (IddMsg, TMVar (Either SomeException ())))
+ -- ^ Messages to write on the pipe.
, asyncDrainTid :: !ThreadId
+ , asyncFillTid :: !ThreadId
, asyncFreshId :: !(IORef Int)
}
@@ -363,11 +396,16 @@ newAsyncManager pipe = do
pendingReply <- newTVarIO IntMap.empty
requestsQueue <- newTQueueIO
mailboxQueues <- newTVarIO IntMap.empty
+ writeQueue <- newTQueueIO
-- Fork thread to drain read end.
-- (cleaned up on 'closePipe')
asyncDrainTid <- forkIOWithUnmask (drainPipe pipe)
+ -- Fork thread to fill write end.
+ -- (cleaned up on 'closePipe')
+ asyncFillTid <- forkIOWithUnmask (fillPipe pipe)
+
let am = AsyncManager{..}
pure am
@@ -417,6 +455,30 @@ drainPipe p unmask = catch (unmask $ loop `finally` pipeCloseRead p) replyToPend
am = pipeAsyncMngr p
+
+-- | "Fill"s write end of the Pipe with queues messages.
+--
+-- Because Pipe is written to from this thread rather than 'writePipe', calls to
+-- the latter can be interrupted without compromising the protocol.
+--
+-- Closes the Pipe write end when stopping, notifying any messages still queued to write.
+fillPipe :: Pipe -> (forall x. IO x -> IO x) -> IO ()
+fillPipe p unmask = catch (loop `finally` pipeCloseWrite p) (atomically . failPendingWrites p) where
+ loop = do
+ (msg,var) <- atomically $ readTQueue (writeQueue am)
+ unmask (withLock p (putAll p . execPut . put $ msg))
+ `catch` \ e -> atomically (putTMVar var (Left e)) >> throwIO e
+ atomically $ putTMVar var (Right ())
+ loop
+
+ am = pipeAsyncMngr p
+
+failPendingWrites :: Pipe -> SomeException -> STM ()
+failPendingWrites p e = do
+ writeTVar (pipeWriteClosed p) True
+ writes <- flushTQueue (writeQueue (pipeAsyncMngr p))
+ mapM_ (`writeTMVar` Left e) (map snd writes)
+
failPendingReplies :: Pipe -> SomeException -> STM ()
failPendingReplies p e = do
-- reqs are only added if pipe not closed;
@@ -447,10 +509,20 @@ data WriteReply a where
writeReply :: WriteReply a %1 -> a -> IO ()
writeReply (WriteReply r) = \x -> r (Right x)
+-- | Write the reply with a value or an exception
+runWriteReply :: WriteReply a %1 -> Either SomeException a -> IO ()
+runWriteReply (WriteReply r) = r
+
-- | Infix, linear, 'unIO'
(|#>) :: IO a %1 -> State# RealWorld -> (# State# RealWorld, a #)
(|#>) (IO fs) s = fs s
+(%>>=) :: (IO a) -> (a -> IO b) %1 -> IO b
+(%>>=) = \ m f -> IO (\s0 ->
+ case m |#> s0 of -- unIO for linearity checking
+ (# s1, x #) -> f x |#> s1)
+infixl 1 %>>=
+
--------------------------------------------------------------------------------
-- * Serializable Exceptions
--------------------------------------------------------------------------------
=====================================
libraries/ghci/GHCi/Server.hs
=====================================
@@ -1,4 +1,4 @@
-{-# LANGUAGE LambdaCase, CPP, RankNTypes, RecordWildCards, GADTs, ScopedTypeVariables #-}
+{-# LANGUAGE LambdaCase, CPP, RankNTypes, RecordWildCards, GADTs, ScopedTypeVariables, LinearTypes #-}
module GHCi.Server
( MessageHook
, CustomMessageHandler
@@ -31,7 +31,6 @@ import GHCi.Utils
import Control.DeepSeq
import Control.Exception
import Control.Monad
-import Control.Concurrent (threadDelay)
import Data.Binary
import Text.Printf
@@ -39,6 +38,10 @@ import System.Environment (getProgName, getArgs)
import System.Exit
import System.IO
+import Control.Concurrent
+import GHCi.Message.Async (runWriteReply, (%>>=))
+import GHCi.Server.Threads
+
type MessageHook = Msg -> IO Msg
-- | How to interpret the 'CustomCommand'.
@@ -58,29 +61,35 @@ servWithCustom
:: Bool
-> MessageHook
-> Pipe
- -> (forall a .IO a -> IO a)
+ -> (forall a. IO a -> IO a)
-> CustomMessageHandler
-> IO ()
-servWithCustom verbose hook pipe restore customHandler = newIORef False >>= loop
+servWithCustom verbose hook pipe restore customHandler = do
+ ref <- newIORef False
+ withThreadManager $ \mngr ->
+ loop ref mngr `finally` closePipe pipe
where
- loop doneRef = do
+ loop doneRef mngr = do
when verbose $ trace "reading pipe..."
- handleRemoteCall pipe hook $ \(msg :: Message a) -> do
- discardCtrlC
+ handleRemoteCall pipe hook $ \wr (msg :: Message a) ->
+
+ discardCtrlC %>>= \_ ->
+ when verbose (trace ("msg: " ++ (show msg))) %>>= \_ ->
- when verbose $ trace ("msg: " ++ (show msg))
case msg of
- CustomMessage tag payload -> handleCustom tag payload
- Shutdown -> writeIORef doneRef True >> reply ()
- -- TODO: These two should probably be done off the main thread
- RunTH ii st q ty loc -> wrapRunTH ii $ runTH pipe ii st q ty loc
- RunModFinalizers ii st qrefs -> wrapRunTH ii $ runModFinalizerRefs pipe ii st qrefs
- _other -> run msg >>= reply
+ CustomMessage tag payload -> handleCustom tag payload %>>= writeReply wr
+ Shutdown -> writeIORef doneRef True >>
+ reply () %>>= writeReply wr
+ RunTH ii st q ty loc -> spawnFinally mngr (runWriteReply wr)
+ (wrapRunTH ii (runTH pipe ii st q ty loc))
+ RunModFinalizers ii st qs -> spawnFinally mngr (runWriteReply wr)
+ (wrapRunTH ii (runModFinalizerRefs pipe ii st qs))
+ _other -> run msg >>= reply %>>= writeReply wr
isDone <- readIORef doneRef
- if isDone then closePipe pipe
- else loop doneRef
+ if isDone then pure ()
+ else loop doneRef mngr
handleCustom tag payload = do
mresp <- customHandler tag payload
@@ -92,7 +101,7 @@ servWithCustom verbose hook pipe restore customHandler = newIORef False >>= loop
reply :: forall a. Show a => a -> IO a
reply r = do
when verbose $ trace ("writing pipe: " ++ show r)
- return r -- the result of the handler gets sent
+ return r -- the result of the handler is passed to writeReply
-- Run some TH code, which may interact with GHC by sending
-- THMessage requests, and then finally send RunTHDone followed by a
=====================================
libraries/ghci/GHCi/Server/Threads.hs
=====================================
@@ -0,0 +1,62 @@
+{-# LANGUAGE LambdaCase, CPP, RankNTypes, RecordWildCards, GADTs, ScopedTypeVariables, LinearTypes, BlockArguments #-}
+module GHCi.Server.Threads
+ ( ThreadManager -- abstract for linearity
+ , withThreadManager -- scoped, so that we're guaranteed to wait for threads
+ , spawnFinally
+ ) where
+
+import Prelude
+import Control.Exception
+import Control.Monad
+
+import Control.Concurrent
+import Control.Concurrent.STM
+import qualified Data.Set as Set
+import qualified Unsafe.Coerce as Unsafe
+
+data ThreadManager = ThreadManager
+ { managedThreads :: !(TVar (Set.Set ThreadId))
+ , managerClosed :: !(TVar Bool)
+ }
+
+-- | Cancels and Waits for all threads spawned by 'ThreadManager' to terminate
+-- before returning.
+--
+-- This thread manager which cancels then waits for all pending threads allows
+-- us to provide a resource-safe linearly typed 'spawnFinally', because
+-- 'withThreadManager' guarantees neither the threads nor their exceptions
+-- leak.
+withThreadManager :: (ThreadManager -> IO a) -> IO a
+withThreadManager k = do
+ managedThreads <- newTVarIO Set.empty
+ managerClosed <- newTVarIO False
+ let tm = ThreadManager{..}
+ k tm `finally` shutdown tm
+ where
+ shutdown ThreadManager{..} = do
+ ts <- atomically $ do
+ writeTVar managerClosed True
+ Set.elems <$> readTVar managedThreads
+ mapM_ killThread ts
+
+-- | Linear and resource-safe because 'ThreadManager' is guaranteed to wait or
+-- cancel all threads.
+spawnFinally :: ThreadManager -> (Either SomeException a -> IO ()) %1 -> IO a -> IO ()
+spawnFinally ThreadManager{..} = unsafe_linear \fin -> \action -> void $
+ forkFinally
+ ( mask $ \unmask -> do
+ tid <- myThreadId
+ registered <- atomically $ do
+ s <- readTVar managerClosed
+ unless s $ modifyTVar' managedThreads (Set.insert tid)
+ pure (not s)
+ if registered
+ then (unmask action)
+ `finally`
+ atomically (modifyTVar' managedThreads (Set.delete tid))
+ else throwIO ThreadKilled
+ )
+ ( \ea -> fin ea )
+
+unsafe_linear :: (a -> b) %1 -> (a %1 -> b)
+unsafe_linear = Unsafe.unsafeCoerce (Unsafe.unsafeCoerce :: (a -> b) -> (a %1 -> b))
=====================================
libraries/ghci/ghci.cabal.in
=====================================
@@ -68,6 +68,7 @@ library
GHCi.StaticPtrTable
GHCi.TH
GHCi.Server
+ GHCi.Server.Threads
if !arch(javascript)
exposed-modules:
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/436ac6992397cba842edefec57737f…
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/436ac6992397cba842edefec57737f…
You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help
1
0
Zubin pushed new branch wip/27971 at Glasgow Haskell Compiler / GHC
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/tree/wip/27971
You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help
1
0
[Git][ghc/ghc][wip/romes/async-rpc] Revert "ignore PipeReadClosedException on Shutdown."
by Andrea Vezzosi (@trac-Saizan) 20 Oct '26
by Andrea Vezzosi (@trac-Saizan) 20 Oct '26
20 Oct '26
Andrea Vezzosi pushed to branch wip/romes/async-rpc at Glasgow Haskell Compiler / GHC
Commits:
436ac699 by Andrea Vezzosi at 2026-10-20T12:13:24+02:00
Revert "ignore PipeReadClosedException on Shutdown."
This reverts commit 5a897150dceaac9a502bf3a095bc9741cc5cee05.
- - - - -
2 changed files:
- compiler/GHC/Runtime/Interpreter.hs
- libraries/ghci/GHCi/Server.hs
Changes:
=====================================
compiler/GHC/Runtime/Interpreter.hs
=====================================
@@ -653,7 +653,7 @@ stopInterp interp = case interpInstance interp of
ex <- getProcessExitCode (interpHandle (instProcess i))
if isJust ex
then pure ()
- else sendMessage i Shutdown `catch` \ PipeReadClosedException -> pure ()
+ else sendMessage i Shutdown
-- We asked the server to shutdown, so we will also shutdown any communication.
closePipe (interpPipe (instProcess i))
pure InterpPending
=====================================
libraries/ghci/GHCi/Server.hs
=====================================
@@ -209,6 +209,8 @@ defaultServerWithCustom customHandler = do
uninterruptibleMask $ \restore -> servWithCustom verbose hook pipe restore customHandler
where hook = return -- empty hook
+ -- we cannot allow any async exceptions while communicating, because
+ -- we will lose sync in the protocol, hence uninterruptibleMask.
dieWithUsage :: IO a
dieWithUsage = do
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/436ac6992397cba842edefec57737f6…
--
View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/436ac6992397cba842edefec57737f6…
You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help
1
0