Duncan Coutts pushed to branch wip/io-manager-deadlock-detection at Glasgow Haskell Compiler / GHC Commits: 234bab08 by Vladislav Zavialov at 2026-09-29T17:22:54-04:00 Add regression test for #7803 Commit 2effe18ab5, first shipped with GHC 8.2.1, fixed #7803 "Superclass methods are left unspecialized". This change turns the example from the ticket description into a regression test, checking that foo1 is fully specialised at Double, with no calls to the overloaded exp5Tail. Closes #7803 Assisted-by: Claude Opus 5 - - - - - c406135c by Andreas Klebinger at 2026-10-01T06:57:28-04:00 RTS: Update flag parsing/handling: Using 0 as argument to any of these flags now resets the flags setting to default: - ``-n0`` disables nursery chunking. - ``-K0`` (or a bare ``-K``) removes the stack size limit. - ``-M0`` removes the heap size limit. - ``-H0`` resets the suggested heap size, including the nursery scaling set by ``-H``. - ``-AL0`` makes the large object allocation limit follow ``-A`` again. GC Generations are now limited to 64. -n larger than -A now emits a warning and disables chunking. Fixes #27830 #27829 - - - - - 984ee82b by Andreas Klebinger at 2026-10-01T06:58:10-04:00 Fix MultiLayerModules timeout multiplier. We accidentally used the compile multiplier for ghci commands which use the run timeout setting to set the timeout. Fixes #26263 and #27796. - - - - - 50a9cf26 by Andreas Klebinger at 2026-10-01T06:58:10-04:00 testsuite: Include runtime in JUnit report. - - - - - e358619f by Alan Zimmerman at 2026-10-01T06:59:01-04:00 EPA: Use EpaLocation in EpVirtualBraces Haddock processing uses EpLayout.EpVirtualBraces to record the current layout column, for allocating haddock comments correctly. For exact printing, we need the line as well as the col, so we know how many lines to skip when the indented code block starts. This commit updates EpLayout.EpVirtualBraces so that it carries an EpaLocation instead of just the Int column, to allow it to be used for both purposes. It is a simple mechanical change, with no other impact, but it allows us to use EpLayout instead of AnnListLayout in AnnList, in the next commit in this series. - - - - - 1d2a26ee by Andrew Lelechenko at 2026-10-01T20:22:04-04:00 Remove bitSize from MINIMAL pragma of class Bits Cf. https://github.com/haskell/core-libraries-committee/issues/441 The function is deprecated for 12 years and has a default implementation, yet it was never removed from the MINIMAL pragma, putting library authors in a very awkward position. Let's finally remove bitSize from MINIMAL. - - - - - 3cc21e4c by Hannes Siebenhandl at 2026-10-01T20:22:59-04:00 Add regression test for #27641 - - - - - c35096f5 by Hannes Siebenhandl at 2026-10-01T20:23:00-04:00 Don't `:add` modules the loader will reject Only home modules can be actually added to the loader context, thus we only accept the module if it belongs to a home unit. - - - - - a563cb27 by Simon Jakobi at 2026-10-02T17:37:36-04:00 ci: accept modified changelog.d entries in lint-changelog The changelog guide says to extend an existing entry rather than add a new one when the feature is already documented. lint_changelog only counted added files, so such MRs failed before the validator ran. Fixes #27779. Assisted-by: Claude Fable 5.1 - - - - - 03082cd4 by Simon Jakobi at 2026-10-02T17:38:17-04:00 changelog.d: Fix warning flag name in the semaphore-v2 entry The entry refers to -Wsemaphore-version-mismatch, which does not exist. The warning GHC emits when it cannot use a -jsem semaphore, including on a protocol version mismatch, is -Wsemaphore-open-failure. Context: !15729, #25087 Assisted-by: Claude Fable 5.1 - - - - - a93705ee by Simon Jakobi at 2026-10-02T17:38:17-04:00 changelog.d: Fix the CLC proposal link in the T27455 entry The entry used the Sphinx :ref: role with a URL as its target. :ref: resolves only internal labels, so the generated release notes would show the proposal name without a link. Use the plain hyperlink form like the other entries. Context: !16274, #27455 Assisted-by: Claude Fable 5.1 - - - - - 54381079 by Hannes Siebenhandl at 2026-10-02T17:39:01-04:00 Extend the `TrustOverlay` for wired-in units The TrustOverlay only contained the `UnitId` of all on-disk units. However, wired-in units are referred to by their wired-in unit id, which is not the same as the on-disk unit id. Thus, trust checks for wired-in units failed because the lookup was using the wired-in unit, which wasn't present in the `TrustOverlay`. We fix #27834, by copying the trust info for wired-in unit ids. - - - - - 11ad6b2f by Duncan Coutts at 2026-10-28T10:46:27+00:00 Move idle GC tracking to its own file, step 1 of 2 Add a new IdleGC.{c,h} module. Move the RecentActivity type, recent_activity variable and {get,set}RecentActivity wrappers. - - - - - 91d4faaa by Duncan Coutts at 2026-10-28T10:46:27+00:00 Move idle GC functionality to its own file, step 2 of 2 We can now fully encapsulate the recent_activity state, exposing just three hooks, and one query used in the scheduler and timer tick. This is a prelude to making changes to the implementation of tracking when to perform an idle gc. - - - - - dc891fa9 by Duncan Coutts at 2026-10-28T10:46:27+00:00 Correct comments in idle GC code The comments still referred to stopTimer/startTimer. These were replaced recently by pauseTimer/unpauseTimer, but the comment was not updated. - - - - - 9d2b0a84 by Duncan Coutts at 2026-10-28T10:46:27+00:00 Make signal handling be a responsibility of the I/O manager(s) Previously it was scattered between I/O managers and the scheduler, and especially the scheduler's deadlock detection. Previously the scheduler would poll for pending signals each iteration of the scheduler loop. The scheduler also had some hairy signal functionality in the deadlock detection: in the non-threaded RTS (only) if there were still no threads running after deadlock detection then it would block waiting for signals. But signals can and (in my opinion) should be thought of as just a funny kind of I/O, and thus should be a responsibility of the I/O manager. So now we have the I/O managers poll for signals when they are polling for I/O completion (and removing the separate poll in the scheduler). And when I/O managers block waiting for I/O then they now also start signal handlers if they get interrupted by a signal. Crucially, if there is no pending I/O or timers, the awaitCompletedTimeoutsOrIO will still block waiting for signals. This patch puts us into an intermediate state: it temporarily breaks deadlock detection in the non-threaded RTS. The waiting on I/O currently happens before deadlock detection. This means we'll now wait forever on signals before doing deadlock detection. We need to move waiting after deadlock detection. We'll do that in a later patch. - - - - - b51b13e2 by Duncan Coutts at 2026-10-28T10:46:27+00:00 Clean up the RTS internal signal handling API Now that the I/O manager is responsible for signals, we can simplify the API we present for signal handling. We now just need startPendingSignalHandlers, which is called from the I/O managers. We can get rid of awaitUserSignals. We also don't need RtsSignals.h to re-export the platform-specific posix/Signals.h or win32/ConsoleHandler.h We can also hide more of the implementation of signals. Less has to be exposed in posix/Signals.h or win32/ConsoleHandler.h. Indeed, posix/Signals.h becomes empty and we remove it. Partly this is because we don't need inline functions (or macros) in the interface. Also remove signal_handlers from RTS ABI exported symbols list. It does not appear to have any users in the core libs, and its really an internal implementation detail. It should not be exposed unless it's really necessary. - - - - - dbea795c by Duncan Coutts at 2026-10-28T12:32:34+00:00 Narrow the window in which signals might be missed before poll/select Any signals delivered just before poll/select block will be missed until they are woken later by unrelated events. This arbitrarily delays the delivery of signals. This is not a complete solution. It is a cheap way to narrow but not close the window. A full solution will have to come later. This issue existed already, and is documented in issue #27540. - - - - - 95520fed by Duncan Coutts at 2026-10-28T12:35:14+00:00 In the scheduler, move I/O blocking after deadlock detection To make deadlock detection effective in the non-threaded RTS when there are deadlocked threads and other unrelated threads waiting on I/O, we need to arrange to do deadlock detection before we block in scheduler to wait on I/O. The solution is to: 1. adjust scheduleFindWork, which runs before deadlock detection, to only poll for I/O and not block; and 2. add a step after deadlock detection to wait on I/O if there are still no threads to run (and there's any I/O or timeouts outstanding) The scheduleCheckBlockedThreads is now so simple that it made more sense to inline it into scheduleFindWork. - - - - - 7e8c0432 by Duncan Coutts at 2026-10-28T12:35:14+00:00 Remove bogus anyPendingTimeoutsOrIO guard from scheduleDetectDeadlock The deadlock detection was only invoked if both of these conditions hold: 1. the run queue is empty 2. there is no pending I/O or timeouts The second condition is unnecessary. The deadlock detection mechanism can find deadlocks even if there are other threads waiting on I/O or timers. Having this extra condition means that we fail to detect blocked threads if there are any threads waiting on I/O or timers. Part of fixing issue #26408 - - - - - ce1f7e26 by Duncan Coutts at 2026-10-28T12:35:14+00:00 Don't consider pending I/O for early context switch optimisation Context switches are normally initiated by the timer signal. If however the user specifies "context switch as often as possible", with +RTS -C0 then the scheduler arranges for an early context switch (when it's just about to run a Haskell thread). Context switching very often is expensive, so as an optimisation there cases where we do not arrange an early context switch: 1. if there's no other threads to run 2. if there is no pending I/O or timers This patch eliminates case 2, leaving only case 1. The rationale is as follows. The use of this was inconsistent across platforms and threaded/non-threaded RTS ways. It only worked on the non-threaded RTS and on Windows only worked for the win32-legacy I/O manager. On all other combinations anyPendingTimeoutsOrIO would always return false. The fact that nobody noticed and complained about this inconsistency suggests that the feature is not relied upon. If however it turns out that applications do rely on this, then the proper thing to do is not to restore this check, but to add a new I/O manager hint function that returns if there is any pending events that are likely to happen *soon*: for example timeouts expiring within one timeslice, or I/O waits on things likely to complete soon like disk I/O, but not for example socket/pipe I/O. The motivation to avoid this use of anyPendingTimeoutsOrIO is to allow us to eliminate anyPendingTimeoutsOrIO entirely. All other uses of this are just guards on {await,poll}CompletedTimeoutsOrIO and the guards can safely be folded into those functions. This will better cope with some I/O managers having no proper implementation of anyPendingTimeoutsOrIO. Ultimately this will let us simplify the scheduler which currently has to have special #ifdef mingw32_HOST_OS cases to cope with the lack of a working anyPendingTimeoutsOrIO for some Windows I/O managers - - - - - e7c7595e by Duncan Coutts at 2026-10-28T12:35:14+00:00 Remove anyPendingTimeoutsOrIO guarding {poll,await}CompletedTimeoutsOrIO Previously the API of the I/O manager used a two step process: check anyPendingTimeoutsOrIO and then call {poll,await}CompletedTimeoutsOrIO. This was primarily there as a performance thing, to cheaply check if we need to do anything. And then because anyPendingTimeoutsOrIO existed, it was used for other things too. We have now eliminated the other uses, and are just left with the performance pattern. But this was problematic because not all I/O managers correctly implement anyPendingTimeoutsOrIO (specifically the win32 ones), and now that we also make I/O managers responsible for signals then we need to poll/await even if there is no pending I/O or timeouts. If there is no pending I/O or timeouts then await needs to degenerate to just waiting forever for any signals. - - - - - a8675b65 by Duncan Coutts at 2026-10-28T12:35:14+00:00 Remove anyPendingTimeoutsOrIO, it is no longer used And this avoids the problems arising from the win32 I/O managers having had a bogus implementation. - - - - - 6665b6fa by Duncan Coutts at 2026-10-28T12:35:15+00:00 Remove second scheduler call to awaitCompletedTimeoutsOrIO Previously awaitCompletedTimeoutsOrIO was called both before and after deadlock detection in the scheduler. The reason for that was that the win32 I/O managers had a bogus implementation of anyPendingTimeoutsOrIO and this was used to guard the call of awaitCompletedTimeoutsOrIO prior to deadlock detection. This meant the first call site was never actually called when using the win32 I/O managers. This was the reason for the second call: the first one was never used. What a mess. So now we have a simple design in the scheduler: 1. poll for completed I/O, timers or signals 2. if no runnable threads: do deadlock detection 3. if still no runnable threads: block waiting for I/O, timers or signals. - - - - - 5b2ba4f3 by Duncan Coutts at 2026-10-28T12:35:15+00:00 Lift emptyRunQueue guard out of scheduleDetectDeadlock this improved the clarity of the logic when reading the scheduler code. And improve the comment about scheduleDoGC that was rather unclear. - - - - - b6158704 by Duncan Coutts at 2026-10-28T12:35:15+00:00 Make interruptIOManager work for WinIO (non-threaded) Fixes #27403 The idle GC scheme now fundamentally relies on interruptIOManager working, so it's no longer acceptable for it to be missing for WinIO. Not having it would cause lock ups. - - - - - fc133aab by Duncan Coutts at 2026-10-28T12:35:15+00:00 Make non-threaded deadlock detection also rely on idle GC Only do deadlock detection GC when idle GC kicks in. This also relies on using wakeUpRts, so now do this unconditionally. Previously wakeUpRts was for the threaded rts only. - - - - - 713f3594 by Duncan Coutts at 2026-10-28T12:35:15+00:00 Enable idle GC by default on non-threaded RTS The behaviour is now uniform between the threaded and non-threaded RTS ways. The deadlock detection now relies on idle GC for both threaded and non-threaded ways. Previously deadlock detection did not rely on idle GC for the non-threaded way. Also tweak test T7275 to account for idle GC. This test's output is sensitive to the number of major GCs run. Since this commit enables idle GC for the non-threaded RTS, for this test that increases the number of major GCs, since the test program is frequently idle for more than 300ms. - - - - - da537110 by Duncan Coutts at 2026-10-28T12:35:15+00:00 Fix state of idle GC control vars with +RTS -V0 Currently when the user uses +RTS -I0, then doIdleGC is set to false. But if the master tick interval -V is set to 0 then the idleGCDelayTime was being set to 0 but doIdleGC was not being set to false, which is inconsistent, and almost certainly buggy. - - - - - 656ea7e0 by Duncan Coutts at 2026-10-28T12:35:15+00:00 Add a long Note [Deadlock detection] It describes the historical and modern designs and their trade-offs. The point is we've now unified the code for deadlock detection between the threaded and non-threaded ways, by changing the non-threaded to follow the same design as the threaded. - - - - - 93c6c1de by Duncan Coutts at 2026-10-28T12:35:15+00:00 Refactor the utils providing the I/O timeout for different system APIs We have to provide a timeout in three different time formats (int milliseconds, struct timeval *, and struct timespec *). These are for platform APIs including poll, select, ppoll, pselect, epoll_wait, epoll_wait2 and in future kevent and io_uring. Previously we had one util for each format that both worked out what the timeout should be and converted it into the particular format. This duplicates some logic and it also makes it awkward to modify the timeout value (since there's no common format). Modifying the timeout value is exactly what we want to do, in a subsequent commit, where we want to adjust the timeout to take account of idle GC. - - - - - bc5206b0 by Duncan Coutts at 2026-10-28T12:35:15+00:00 Update the old select I/O manager to use the new timeout utils This will make it much easier to adjust the timeout to take account of idle GC. - - - - - 07d7ce9c by Duncan Coutts at 2026-10-29T10:51:00+00:00 Add opportunistic deadlock detection without idle GC Our general design is now that comprehensive deadlock detection relies on idle GC. This allows us to detect partial deadlocks that are not system deadlocks. But it's not prompt. It's a nice to have feature to be able to promptly detect the cases that are system deadlocks. There's no hope to do this in the threaded RTS with -N > 1, but it can be done for the non-threaded RTS and _could_ be done for the threaded RTS with -N1. We can integrate this feature into the general design without too many contortions. We just need the I/O managers to check before sleeping if they would be about to sleep indefinately with no I/O and no registered signal handlers. If so, they can notify the idle GC that we're deadlocked and we schedule and idle GC. - - - - - f47f7c22 by Duncan Coutts at 2026-10-29T10:51:00+00:00 Add idle GC mode for !HAVE_PREEMPTION We have to do idle GC (and thus deadlock detection) differently when we do not have the ticker and the ability to interrupt the I/O manager when it's blocking. - - - - - 7a40abb2 by Duncan Coutts at 2026-10-29T10:51:00+00:00 Add and use USED_IF_PREEMPTION to avoid warnings - - - - - a33b2aae by Duncan Coutts at 2026-10-29T10:51:00+00:00 Add performDeadlockDetection and export via ghc-experimental When documenting the idle GC and deadlock detection behaviour in the user guide, it becomes obiovus that in some cases one would want manual control over deadlock detection. In particular, for applications that disable idle GC, for debugging or robustness, they may want to invoke deadlock detecting GC manually. Bind it in ghc-internal GHC.Internal.System.Mem and re-export via ghc-experimental in System.Mem.Experimental. It is clearly implementation specific behaviour, especially since in GHC the deadlock detection is tied to doing a major GC. - - - - - 8e89ffbc by Duncan Coutts at 2026-10-29T10:51:00+00:00 Add tests for deadlock detection, issue #26408 We have three tests: T26408a: make sure that we can detect a /partial deadlock/ that is not also a /system deadlock/ T26408b: make sure that we can detect a /system deadlock/ promptly without relying on idle GC T26408c: make sure that we do not falsely detect a deadlock when there is nothing to do except wait on signals T26408d: make sure that we can detect a system deadlock with idle GC disabled, after the idle GC timeout has elapsed. - - - - - 5baa4651 by Duncan Coutts at 2026-10-29T10:51:00+00:00 Update the user guide with the revised idle GC behaviour It's now not just for the threaded RTS, but general. Add a new section in the RTS chapter: Idle GC and deadlock detection. Provide general documentation on both topics. In particular, document the behaviour of deadlock detection and how it depends on idle GC. Move the existing entries for -I and -Iw to this section. Add a changelog entry. - - - - - fed3354f by Duncan Coutts at 2026-10-29T10:51:00+00:00 Remove variable n_haskell_handlers which has become write-only - - - - - 108 changed files: - .gitlab/ci.sh - + changelog.d/27641 - changelog.d/T27455 - + changelog.d/T27830-rts-flag-bounds - + changelog.d/idle-gc-and-deadlock-detection - changelog.d/semaphore-v2 - changelog.d/unit-index - compiler/GHC/Parser.y - compiler/GHC/Parser/Annotation.hs - compiler/GHC/Parser/PostProcess/Haddock.hs - compiler/GHC/Unit/External/Wired.hs - compiler/GHC/Unit/State.hs - docs/users_guide/profiling.rst - docs/users_guide/runtime_control.rst - ghc/GHCi/UI.hs - libraries/base/changelog.md - libraries/ghc-experimental/src/System/Mem/Experimental.hs - libraries/ghc-internal/src/GHC/Internal/Bits.hs - libraries/ghc-internal/src/GHC/Internal/System/Mem.hs - rts/Capability.c - rts/IOManager.c - rts/IOManager.h - + rts/IdleGC.c - + rts/IdleGC.h - rts/Linker.c - rts/RtsFlags.c - rts/RtsSignals.h - rts/RtsStartup.c - rts/RtsSymbols.c - rts/Schedule.c - rts/Schedule.h - rts/Timer.c - rts/include/rts/Flags.h - rts/include/rts/OSThreads.h - rts/include/rts/storage/GC.h - rts/posix/Poll.c - rts/posix/Poll.h - rts/posix/Select.c - rts/posix/SelectBis.c - rts/posix/SelectBis.h - rts/posix/Signals.c - − rts/posix/Signals.h - rts/posix/Timeout.c - rts/posix/Timeout.h - rts/rts.cabal - rts/sm/GC.c - rts/sm/Storage.c - rts/win32/AsyncMIO.c - rts/win32/AsyncMIO.h - rts/win32/AsyncWinIO.c - rts/win32/AsyncWinIO.h - rts/win32/AwaitEvent.c - rts/win32/ConsoleHandler.c - rts/win32/ConsoleHandler.h - testsuite/driver/junit.py - testsuite/driver/testglobals.py - testsuite/driver/testlib.py - testsuite/tests/ghc-api/exactprint/T22919.stderr - testsuite/tests/ghc-api/exactprint/Test20239.stderr - testsuite/tests/ghc-api/exactprint/ZeroWidthSemi.stderr - + testsuite/tests/ghci/scripts/T27641.script - + testsuite/tests/ghci/scripts/T27641.stderr - + testsuite/tests/ghci/scripts/T27641.stdout - + testsuite/tests/ghci/scripts/T27834.hs - + testsuite/tests/ghci/scripts/T27834.script - + testsuite/tests/ghci/scripts/T27834.stdout - testsuite/tests/ghci/scripts/all.T - testsuite/tests/haddock/should_compile_flag_haddock/T17544.stderr - testsuite/tests/haddock/should_compile_flag_haddock/T17544_kw.stderr - testsuite/tests/haddock/should_compile_flag_haddock/T24221.stderr - testsuite/tests/interface-stability/base-exports.stdout - testsuite/tests/interface-stability/base-exports.stdout-javascript-unknown-ghcjs - testsuite/tests/interface-stability/base-exports.stdout-mingw32 - testsuite/tests/interface-stability/ghc-experimental-exports.stdout - testsuite/tests/interface-stability/ghc-experimental-exports.stdout-mingw32 - testsuite/tests/module/mod185.stderr - testsuite/tests/parser/should_compile/DumpParsedAst.stderr - testsuite/tests/parser/should_compile/DumpParsedAstComments.stderr - testsuite/tests/parser/should_compile/DumpSemis.stderr - testsuite/tests/parser/should_compile/KindSigs.stderr - testsuite/tests/parser/should_compile/T15323.stderr - testsuite/tests/parser/should_compile/T20452.stderr - testsuite/tests/parser/should_compile/T20718.stderr - testsuite/tests/parser/should_compile/T20718b.stderr - testsuite/tests/parser/should_compile/T20846.stderr - testsuite/tests/parser/should_compile/T23315/T23315.stderr - testsuite/tests/perf/compiler/all.T - testsuite/tests/printer/AnnotationNoListTuplePuns.stdout - testsuite/tests/printer/T18791.stderr - testsuite/tests/printer/Test20297.stdout - testsuite/tests/printer/Test24533.stdout - testsuite/tests/profiling/should_run/Makefile - testsuite/tests/rts/Makefile - + testsuite/tests/rts/T26408a.hs - + testsuite/tests/rts/T26408a.stderr - + testsuite/tests/rts/T26408b.hs - + testsuite/tests/rts/T26408b.stderr - + testsuite/tests/rts/T26408c.hs - + testsuite/tests/rts/T26408c.stdout - + testsuite/tests/rts/T26408d.hs - + testsuite/tests/rts/T26408d.stderr - testsuite/tests/rts/all.T - testsuite/tests/rts/rtsflags001.stderr-ws-32 - testsuite/tests/rts/rtsflags001.stderr-ws-64 - + testsuite/tests/simplCore/should_compile/T7803.hs - + testsuite/tests/simplCore/should_compile/T7803.stderr - + testsuite/tests/simplCore/should_compile/T7803a.hs - testsuite/tests/simplCore/should_compile/all.T The diff was not included because it is too large. View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/04c391127b3843dbc5f33550b96264d... -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/04c391127b3843dbc5f33550b96264d... You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help