Duncan Coutts pushed to branch wip/io-manager-deadlock-detection at Glasgow Haskell Compiler / GHC Commits: 700a1dd1 by Simon Jakobi at 2026-07-23T11:21:20-04:00 ci: Reduce lint job setup costs Avoid fetching unnecessary history and submodules for lightweight lint jobs. Run changelog validation without Hadrian. Because the lint-author job is now based on the .lint template directly, we enhance it to allow Git to read from the runner-owned checkouts, In the previously used .lint-params template, this permissions issue was addressed via `chown`. Closes #27521. Assisted-by: gpt-5.6-sol via Codex CLI - - - - - 26a44fb0 by ARATA Mizuki at 2026-07-23T11:22:10-04:00 testsuite: Fix memory issues of doublex2_* and simd010 doublex2_* had reads from uninitialized memory. simd010 had out-of-bounds array access. Fixes #27544 - - - - - 4d798b17 by Duncan Coutts at 2026-07-23T17:26:18-04:00 Eliminate STM_AWOKEN It was used as nullary closure for the block_info.closure in the case of a thread being awoken after an STM transaction. However, while it was written, it was never read, so contributed nothing to the behaviour. Furthermore, in the only place it was set (in tryWakeupThread) the why_blocked was immediately overwritten by the NotBlocked status, and the block_info was updated accordingly (by appendToRunQueue). So it didn't even serve a purpose of clarifying an intermediate state, there really was no such intermediate state. Cleaning this up will allow the BlockedOnSTM case to follow the same pattern as the other why_blocked cases that do not use the block_info, and in turn this reduces the number of different categories. - - - - - e1cece79 by Duncan Coutts at 2026-07-23T17:26:18-04:00 Document that eventlog thread stop code ThreadBlocked is no longer used It has not been used since GHC 7.0.x (2011). In 7.2 all the BlockedOn* codes were added, and these were and are used instead of ThreadBlocked. - - - - - 795db115 by Duncan Coutts at 2026-07-23T17:26:18-04:00 Add a proper mapping to eventlog external thread stop status That is the mapping from rts-internal codes, to the coes used in the status field in the eventlog EVENT_STOP_THREAD event. See issue #9003 for what goes wrong when we mess this up. In that ticket, people note that we should really not require the internal tso->why_blocked codes to leak into the external eventlog thread stop codes. The same principle applies to the StgThreadReturnCode. This change properly separates them, and explicitly maps between them using a pair of (compact, constant) tables. These tables are pretty small (with no alignment constraints) and will soon shrink so it seems a sensible tradeoff. We also introduce and use proper EVENT_STOP_THREAD constants in the event log format header. Previously there was not specification in the code for these (only in the docs): the values were encoded into the conversion code. This will allow us to renumber the internal why_blockd codes without breaking the eventlog output. - - - - - 6f1c8efa by Duncan Coutts at 2026-07-23T17:26:18-04:00 Remove unused tso->block_info.wakeup member Presumably it was used once, but not now. - - - - - 740b88a9 by Duncan Coutts at 2026-07-23T17:26:18-04:00 Document StgTSOBlockInfo to say what cases use what members In principle, tso->why_blocked is the tag for the StgTSOBlockInfo union, so we should be able to say for each union member the why_blocked cases that use that member. - - - - - 5b92eae2 by Duncan Coutts at 2026-07-23T17:26:18-04:00 Add a tso->block_info.mvar member and use it in preference to the generic block_info.closure union member, with casts. The plan is that when we know what case we're in (via tso->why_blocked) then we can always access the correct union member, and so we will only need to access block_info.closure for generic cases where we don't know or don't care. - - - - - d931715f by Duncan Coutts at 2026-07-23T17:26:18-04:00 Add a tso->block_info.unused member and use it in preference to the generic block_info.closure union member, with casts. The plan is that when we know what case we're in (via tso->why_blocked) then we can always access the correct union member, and so we will only need to access block_info.closure for generic cases where we don't know or don't care. - - - - - 47e28ebb by Duncan Coutts at 2026-07-23T17:26:18-04:00 Avoid storing to tso->block_info.closure In one case we can use a specific union member (.prev) instead. In several cases the stores were in fact redundant because of subsequent overwrites. In scavengeTSO we replace setting tso->block_info.closure to a valid closure, with an assertion that the block_info.unused is already set to END_TSO_QUEUE which is a valid (static) closure. - - - - - 96e4749d by Duncan Coutts at 2026-07-23T17:26:18-04:00 Renumber the tso->why_blocked constants We can do this now because we have separated the internal values from the external ones used in the eventlog. This lets us put them back into a deliberate order and consolodate some gaps. More importantly, it is a prepation for a slightly more sophisticated encoding. - - - - - 8f62661c by Duncan Coutts at 2026-07-23T17:26:18-04:00 Define constants for the existing stg_threadStatuszh return codes The stg_threadStatuszh reuses the internal tso->why_blocked codes but also extends them with a couple previously magic values. This is awkward since we need to know what those magic values are so we don't accidentally use those values to mean something else. By pulling a definition up to where the why_blocked codes are defined we will be able to avoid mistakenly assining those codes some meaning (or just changing the BlockedThreadComplete, BlockedThreadKilled code if necessary). - - - - - 42c69ae2 by Duncan Coutts at 2026-07-23T17:26:18-04:00 Extend the tso->why_blocked encoding to indicate block_info closures We use some bit tricks to cheaply and generically test if a tso->why_blocked tag implies that the corresponding tso->block_info will contain a non-trivial valid closure (i.e. not just block_info.unused set to END_TSO_QUEUE). In particular we arrange for most why_blocked values to naturally have a distinguishing bit, but for the BlockedOn{Read,Write,Delay} cases, they can come in either non-closure or closure forms. We allow an additional bit to distinguish these cases. The non-closure forms are only from legacy I/O managers: select and win32-legacy. So this extra bit mechanism will be able to be retired once the legacy I/O managers are themselves retired. This means in a few places we need to untag the why_blocked value before inspecting it, but in most places we do not. - - - - - 7c64632b by Duncan Coutts at 2026-07-23T17:26:18-04:00 Use BlockInfoForceNonClosure in the select I/O manager - - - - - 8fd7104a by Duncan Coutts at 2026-07-23T17:26:18-04:00 Use BlockInfoForceNonClosure in the win32-legacy I/O manager for the BlockedOn{Read,Write} since these use the non-heap allocated StgAsyncIOResult. - - - - - e0da603b by Duncan Coutts at 2026-07-23T17:26:18-04:00 Enforce the why_blocked and block_info rules in checkTSO We now check the cases wher IsBlockInfoClosure should hold, the cases that are supposed to use block_info.unused == END_TSO_QUEUE, and which cases are allowed to use BlockInfoForceNonClosure. This partially enforces the use of why_blocked as a tag for the block_info union. We could be stricter and check for the correct expected info table for the closure cases. - - - - - 1dd0f381 by Duncan Coutts at 2026-07-23T17:26:18-04:00 Use IsBlockInfoClosure to simplify several tests In GC and generic traversal we need to know if we should look at the block_info.closure or not. Now we can do just that using a cheap bit test on the why_blocked tag. This fixes issue 26717, where the problem was that some GC modes did not know when to look at block_info.closure, because the poll I/O manager uses a closure for BlockedOn{Read,Write} while the select I/O manager uses a non-closure. Now this information is in the why_blocked tag itself. - - - - - 7a00ffbc by Duncan Coutts at 2026-07-23T17:26:18-04:00 Remove the now-unused scavengeTSOIOManager The GC no longer has to delegate to the I/O manager, since it can use IsBlockInfoClosure to decide things itself. - - - - - 522a481f by Duncan Coutts at 2026-07-23T17:26:18-04:00 Remove duplicate assertion - - - - - 0874d965 by Duncan Coutts at 2026-07-23T17:26:18-04:00 Follow atomic access rules more consistently for tso->why_blocked The rule is this: store block_info *before* why_blocked store why_blocked using store release load why_blocked using load acquire load block_info *after* why_blocked This is a an atomic store release / load acquire pair and (if the reads are in a separate thread to the writes, and the read receives the value stored) then this guarantees a full "happens before" relationship of these stores and loads. In some cases, we do not need a full load acquire, because we don't read the block_info at all and so do not need any ordering. In this case we just need an atomic relaxed load. This was being followed in most places, but not all. If there's good reason in any case that we don't need atomic access, then we should document that in a comment. In the absence of that I think it's easier to follow the rule everywhere. - - - - - 8f0bdbe1 by Duncan Coutts at 2026-07-23T17:26:19-04:00 Add a changelog entry - - - - - 4fdfe757 by Alan Zimmerman at 2026-07-23T17:27:06-04:00 EPA: Keep decls together in ClassDecl Similar to 1718230f4d3d19d8c49c0e5d496cb0fb6f399528 for HsValBindsLR, this commit updates ClassDecl so that it no longer splits out the assorted `LHsDecl GhcPs` until the renamer. It does this by inserting a type family (separate from the classic TTG one) for this. So data TyClDecl ... | ClassDecl { ... tcdDecls :: XClassDecls pass with type instance XClassDecls GhcPs = [LHsDecl GhcPs] type instance XClassDecls GhcRn = ClassDeclX GhcRn type instance XClassDecls GhcTc = ClassDeclX GhcTc data ClassDeclX pass = ClassDeclX { tcdSigs :: [LSig pass], -- ^ Methods' signatures tcdMeths :: LHsBinds pass, -- ^ Default methods tcdATs :: [LFamilyDecl pass], -- ^ Associated types; tcdATDefs :: [LTyFamDefltDecl pass], -- ^ Associated type defaults tcdDocs :: [LDocDecl pass] -- ^ Haddock docs } - - - - - aa05b51e by Duncan Coutts at 2026-07-23T23:05:12+01:00 Make signal handling be a respondibility of the I/O manager(s) Previously it was scattered between I/O managers and the scheduler, and especially the scheduler's deadlock detection. Previously the scheduler would poll for pending signals each iteration of the scheduler loop. The scheduler also had some hairy signal functionality in the deadlock detection: in the non-threaded RTS (only) if there were still no threads running after deadlock detection then it would block waiting for signals. But signals can and (in my opinion) should be thought of as just a funny kind of I/O, and thus should be a responsibility of the I/O manager. So now we have the I/O managers poll for signals when they are polling for I/O completion (and removing the separate poll in the scheduler). And when I/O managers block waiting for I/O then they now also start signal handlers if they get interrupted by a signal. Crucially, if there is no pending I/O or timers, the awaitCompletedTimeoutsOrIO will still block waiting for signals. This patch puts us into an intermediate state: it temporarily breaks deadlock detection in the non-threaded RTS. The waiting on I/O currently happens before deadlock detection. This means we'll now wait forever on signals before doing deadlock detection. We need to move waiting after deadlock detection. We'll do that in a later patch. - - - - - 2974b7ff by Duncan Coutts at 2026-07-23T23:05:12+01:00 Clean up the RTS internal signal handling API Now that the I/O manager is responsible for signals, we can simplify the API we present for signal handling. We now just need startPendingSignalHandlers, which is called from the I/O managers. We can get rid of awaitUserSignals. We also don't need RtsSignals.h to re-export the platform-specific posix/Signals.h or win32/ConsoleHandler.h We can also hide more of the implementation of signals. Less has to be exposed in posix/Signals.h or win32/ConsoleHandler.h. Indeed, posix/Signals.h becomes empty and we remove it. Partly this is because we don't need inline functions (or macros) in the interface. Also remove signal_handlers from RTS ABI exported symbols list. It does not appear to have any users in the core libs, and its really an internal implementation detail. It should not be exposed unless it's really necessary. - - - - - 940b0b85 by Duncan Coutts at 2026-07-23T23:05:12+01:00 In the scheduler, move I/O blocking after deadlock detection To make deadlock detection effective in the non-threaded RTS when there are deadlocked threads and other unrelated threads waiting on I/O, we need to arrange to do deadlock detection before we block in scheduler to wait on I/O. The solution is to: 1. adjust scheduleFindWork, which runs before deadlock detection, to only poll for I/O and not block; and 2. add a step after deadlock detection to wait on I/O if there are still no threads to run (and there's any I/O or timeouts outstanding) The scheduleCheckBlockedThreads is now so simple that it made more sense to inline it into scheduleFindWork. - - - - - 37746eac by Duncan Coutts at 2026-07-23T23:05:12+01:00 Remove bogus anyPendingTimeoutsOrIO guard from scheduleDetectDeadlock The deadlock detection was only invoked if both of these conditions hold: 1. the run queue is empty 2. there is no pending I/O or timeouts The second condition is unnecessary. The deadlock detection mechanism can find deadlocks even if there are other threads waiting on I/O or timers. Having this extra condition means that we fail to detect blocked threads if there are any threads waiting on I/O or timers. Part of fixing issue #26408 - - - - - 16266802 by Duncan Coutts at 2026-07-23T23:05:12+01:00 Don't consider pending I/O for early context switch optimisation Context switches are normally initiated by the timer signal. If however the user specifies "context switch as often as possible", with +RTS -C0 then the scheduler arranges for an early context switch (when it's just about to run a Haskell thread). Context switching very often is expensive, so as an optimisation there cases where we do not arrange an early context switch: 1. if there's no other threads to run 2. if there is no pending I/O or timers This patch eliminates case 2, leaving only case 1. The rationale is as follows. The use of this was inconsistent across platforms and threaded/non-threaded RTS ways. It only worked on the non-threaded RTS and on Windows only worked for the win32-legacy I/O manager. On all other combinations anyPendingTimeoutsOrIO would always return false. The fact that nobody noticed and complained about this inconsistency suggests that the feature is not relied upon. If however it turns out that applications do rely on this, then the proper thing to do is not to restore this check, but to add a new I/O manager hint function that returns if there is any pending events that are likely to happen *soon*: for example timeouts expiring within one timeslice, or I/O waits on things likely to complete soon like disk I/O, but not for example socket/pipe I/O. The motivation to avoid this use of anyPendingTimeoutsOrIO is to allow us to eliminate anyPendingTimeoutsOrIO entirely. All other uses of this are just guards on {await,poll}CompletedTimeoutsOrIO and the guards can safely be folded into those functions. This will better cope with some I/O managers having no proper implementation of anyPendingTimeoutsOrIO. Ultimately this will let us simplify the scheduler which currently has to have special #ifdef mingw32_HOST_OS cases to cope with the lack of a working anyPendingTimeoutsOrIO for some Windows I/O managers - - - - - 1d825f35 by Duncan Coutts at 2026-07-23T23:05:12+01:00 Remove anyPendingTimeoutsOrIO guarding {poll,await}CompletedTimeoutsOrIO Previously the API of the I/O manager used a two step process: check anyPendingTimeoutsOrIO and then call {poll,await}CompletedTimeoutsOrIO. This was primarily there as a performance thing, to cheaply check if we need to do anything. And then because anyPendingTimeoutsOrIO existed, it was used for other things too. We have now eliminated the other uses, and are just left with the performance pattern. But this was problematic because not all I/O managers correctly implement anyPendingTimeoutsOrIO (specifically the win32 ones), and now that we also make I/O managers responsible for signals then we need to poll/await even if there is no pending I/O or timeouts. If there is no pending I/O or timeouts then poll/await needs to degenerate to just waiting forever for any signals. - - - - - caf43422 by Duncan Coutts at 2026-07-23T23:05:12+01:00 Remove anyPendingTimeoutsOrIO, it is no longer used And this avoids the problems arising from the win32 I/O managers having had a bogus implementation. - - - - - 74d324e5 by Duncan Coutts at 2026-07-23T23:05:12+01:00 Remove second scheduler call to awaitCompletedTimeoutsOrIO Previously awaitCompletedTimeoutsOrIO was called both before and after deadlock detection in the scheduler. The reason for that was that the win32 I/O managers had a bogus implementation of anyPendingTimeoutsOrIO and this was used to guard the call of awaitCompletedTimeoutsOrIO prior to deadlock detection. This meant the first call site was never actually called when using the win32 I/O managers. This was the reason for the second call: the first one was never used. What a mess. So now we have a simple design in the scheduler: 1. poll for completed I/O, timers or signals 2. if no runnable threads: do deadlock detection 3. if still no runnable threads: block waiting for I/O, timers or signals. - - - - - 2ee6ab72 by Duncan Coutts at 2026-07-23T23:05:12+01:00 Lift emptyRunQueue guard out of scheduleDetectDeadlock this improved the clarity of the logic when reading the scheduler code. - - - - - d159ab67 by Duncan Coutts at 2026-07-23T23:05:12+01:00 Make non-threaded deadlock detection also rely on idle GC Only do deadlock detection GC when idle GC kicks in. This also relies on using wakeUpRts, so now do this unconditionally. Previously wakeUpRts was for the threaded rts only. - - - - - b733dda2 by Duncan Coutts at 2026-07-23T23:05:12+01:00 Enable idle GC by default on non-threaded RTS The behaviour is now uniform between the threaded and non-threaded RTS ways. The deadlock detection now relies on idle GC for both threaded and non-threaded ways. Previously deadlock detection did not rely on idle GC for the non-threaded way. Also tweak test T7275 to account for idle GC. This test's output is sensitive to the number of major GCs run. Since this commit enables idle GC for the non-threaded RTS, for this test that increases the number of major GCs, since the test program is frequently idle for more than 300ms. - - - - - 2c4aaafa by Duncan Coutts at 2026-07-23T23:05:12+01:00 Fix state of idle GC control vars with +RTS -V0 Currently when the user uses +RTS -I0, then doIdleGC is set to false. But if the master tick interval -V is set to 0 then the idleGCDelayTime was being set to 0 but doIdleGC was not being set to false, which is inconsistent, and almost certainly buggy. - - - - - 41d9b6e5 by Duncan Coutts at 2026-07-23T23:05:52+01:00 Add a long Note [Deadlock detection] It describes the historical and modern designs and their trade-offs. The point is we've now unified the code for deadlock detection between the threaded and non-threaded ways, by changing the non-threaded to follow the same design as the threaded. - - - - - 9ce4f146 by Duncan Coutts at 2026-07-23T23:05:53+01:00 Add a test for deadlock detection, issue #26408 - - - - - 566c4b06 by Duncan Coutts at 2026-07-23T23:05:53+01:00 Update the user guide with the revised idle GC behaviour i.e. it's now not just for the threaded RTS, but general. Also document the fact that disabling idle GC also disables deadlock detection. And add a changelog entry. - - - - - 92 changed files: - .gitlab-ci.yml - .gitlab/ci.sh - + changelog.d/T26716 - + changelog.d/idle-gc-and-deadlock-detection - compiler/GHC/Hs/Decls.hs - compiler/GHC/Hs/Instances.hs - compiler/GHC/Hs/Stats.hs - compiler/GHC/Hs/Utils.hs - compiler/GHC/HsToCore/Docs.hs - compiler/GHC/HsToCore/Quote.hs - compiler/GHC/Iface/Ext/Ast.hs - compiler/GHC/Parser/PostProcess.hs - compiler/GHC/Parser/PostProcess/Haddock.hs - compiler/GHC/Rename/Module.hs - compiler/GHC/Tc/TyCl.hs - compiler/GHC/Tc/TyCl/Class.hs - compiler/GHC/ThToHs.hs - compiler/Language/Haskell/Syntax/Binds.hs - compiler/Language/Haskell/Syntax/Decls.hs - docs/users_guide/eventlog-formats.rst - docs/users_guide/runtime_control.rst - hadrian/src/Rules/Test.hs - libraries/ghc-heap/GHC/Exts/Heap/FFIClosures_ProfilingDisabled.hsc - libraries/ghc-heap/GHC/Exts/Heap/FFIClosures_ProfilingEnabled.hsc - libraries/ghc-internal/src/GHC/Internal/Conc/Sync.hs - rts/IOManager.c - rts/IOManager.h - rts/Linker.c - rts/Messages.c - rts/PrimOps.cmm - rts/RaiseAsync.c - rts/RaiseAsync.h - rts/RtsFlags.c - rts/RtsSignals.h - rts/RtsStartup.c - rts/RtsSymbols.c - rts/STM.c - rts/Schedule.c - rts/Schedule.h - rts/StgMiscClosures.cmm - rts/Threads.c - rts/Timer.c - rts/Trace.c - rts/Trace.h - rts/TraverseHeap.c - rts/include/Cmm.h - rts/include/rts/Constants.h - rts/include/rts/EventLogFormat.h - rts/include/rts/storage/TSO.h - rts/include/stg/MiscClosures.h - rts/posix/Poll.c - rts/posix/Poll.h - rts/posix/Select.c - rts/posix/Signals.c - − rts/posix/Signals.h - rts/posix/Timeout.c - rts/sm/Compact.c - rts/sm/NonMovingMark.c - rts/sm/Sanity.c - rts/sm/Scav.c - rts/win32/AsyncMIO.c - rts/win32/AwaitEvent.c - rts/win32/ConsoleHandler.c - rts/win32/ConsoleHandler.h - testsuite/tests/haddock/haddock_examples/haddock.Test.stderr - testsuite/tests/haddock/should_compile_flag_haddock/T17544.stderr - testsuite/tests/haddock/should_compile_flag_haddock/T17544_kw.stderr - testsuite/tests/parser/should_compile/DumpRenamedAst.stderr - testsuite/tests/parser/should_compile/DumpSemis.stderr - testsuite/tests/parser/should_compile/T20452.stderr - testsuite/tests/printer/Test24533.stdout - testsuite/tests/profiling/should_run/Makefile - + testsuite/tests/rts/T26408.hs - + testsuite/tests/rts/T26408.stderr - testsuite/tests/rts/all.T - testsuite/tests/simd/should_run/doublex2_arith.hs - testsuite/tests/simd/should_run/doublex2_arith.stdout - testsuite/tests/simd/should_run/doublex2_arith_baseline.hs - testsuite/tests/simd/should_run/doublex2_arith_baseline.stdout - testsuite/tests/simd/should_run/doublex2_fma.hs - testsuite/tests/simd/should_run/doublex2_fma.stdout - testsuite/tests/simd/should_run/simd010.hs - utils/check-exact/ExactPrint.hs - utils/check-exact/Utils.hs - utils/haddock/haddock-api/src/Haddock/Backends/Hoogle.hs - utils/haddock/haddock-api/src/Haddock/Backends/LaTeX.hs - utils/haddock/haddock-api/src/Haddock/Backends/Xhtml/Decl.hs - utils/haddock/haddock-api/src/Haddock/Convert.hs - utils/haddock/haddock-api/src/Haddock/GhcUtils.hs - utils/haddock/haddock-api/src/Haddock/Interface/Create.hs - utils/haddock/haddock-api/src/Haddock/Interface/Rename.hs - utils/haddock/haddock-api/src/Haddock/Types.hs The diff was not included because it is too large. View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/db5f26581932580102a62c276f25ea9... -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/db5f26581932580102a62c276f25ea9... You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help