Duncan Coutts pushed to branch wip/io-manager-deadlock-detection at Glasgow Haskell Compiler / GHC Commits: 4e9297ea by Simon Jakobi at 2026-09-11T08:32:45-04:00 ci: Actually pass --ignore-perf-failures to the testsuite driver 684c0018d9 changed the IGNORE_PERF_FAILURES handling to append to RUNTEST_ARGS with bash array syntax, but the three use sites still expand RUNTEST_ARGS as a scalar, which yields only the first array element. The appended flag was therefore dropped in every job, so perf improvements kept failing Marge Bot batches and master pipelines, contrary to #19562. Append as a string instead, matching the existing --force-colors append in test_hadrian. Fixes #27563. Assisted-by: Claude Fable 5.1 - - - - - 4f78dfdf by Simon Jakobi at 2026-09-11T08:33:24-04:00 testsuite: Give T26537 a 1.5x compile timeout Its -fregs-graph compile step tends to time out in unoptimized builds (#27188, #27776). Assisted-by: Claude Fable 5.1 - - - - - 130286e3 by Alan Zimmerman at 2026-09-11T08:34:08-04:00 EPA: Remove ListBanana / ListParens from AnnListBrackets This is a step towards making AnnList only represent lists of items which can have either braces or layout. - - - - - a9827d13 by Simon Jakobi at 2026-09-11T18:53:32-04:00 Reject ~, UNPACK and multiplicity annotations on type data fields Only `!` was rejected so far. `~`, UNPACK and multiplicity annotations were silently accepted. None of them make sense at the type level, so restriction (R3) of Note [Type data declarations] now covers all of them. Fixes #27732. Assisted-by: Claude Fable 5.1 - - - - - c71d5e23 by Alan Zimmerman at 2026-09-11T18:54:11-04:00 EPA: Add a Note for the exact print main loop - - - - - 7a108e43 by Simon Jakobi at 2026-09-12T18:35:59-04:00 FastString: Drop mkFastStringWith's constructor callback All three callers passed the same callback, a partial application of mkNewFastStringShortByteString to the string being interned. That partial application is allocated as a closure before the table lookup, on the common hit path too, although the callback is needed only after a miss. Drop the parameter and call mkNewFastStringShortByteString directly after a miss. Since nothing is passed "with" anymore, rename the function to internSB. Suggested by Simon PJ in #27528: https://gitlab.haskell.org/ghc/ghc/-/work_items/27528#note_687031 Assisted-by: Claude Fable 5 - - - - - 82c73b22 by Alan Zimmerman at 2026-09-12T18:36:38-04:00 EPA: More targeted HsDo exact print annotation HsDo is multi-purpose, as encoded in its HsDoFlavour field. Some of these are in a layout context (DoExpr, MDoExpr), others are not (ListComp, MonadComp). We are moving towards using AnnList only in layout contexts, so we switch the HsDo TTG annotation from holding an AnnList for this, to holding Either (EpToken "[", EpToken "]") AnnList This also allows us to trim down AnnListBrackets to only have braces or None, thereby opening the door for unification with the existing layout context data type EpLayout. - - - - - 749d2448 by Luite Stegeman at 2026-09-15T14:07:01-04:00 rts: handle large CONSTR closures in compacting GC The function update_fwd_large in the compacting GC could run into an unexpected object with the following error: internal error: update_fwd_large: unknown/strange object 1 Closure type 1 is CONSTR. This patch adds cases for CONSTR and CONSTR_NOCAF. This is the same problem as #27434 (large AP closures, fixed by cca0d58963), which missed this case. Fixes #27649 - - - - - 177d6155 by Luite Stegeman at 2026-09-15T14:07:02-04:00 testsuite: fix T27434 test with compacting GC Ensure that the interpreter is actually run with the compacting GC - - - - - 5796aa63 by Luite Stegeman at 2026-09-15T14:07:49-04:00 rts: initialise the stack frame header for mask_frame and apply_mask_frame We must leave the stack in consistent state before jumping to mask_frame or apply_mask_frame because they may result. Failing to do so could lead to a crash if there were waiting exceptions. Fixes #27651 - - - - - 2486cc92 by Simon Peyton Jones at 2026-09-15T14:08:36-04:00 Re-introduce the "weird SpecDict" case In this commit commit f80375dd4945350a1d784e436975b48b9c736eaf Author: Simon Peyton Jones <simon.peytonjones@gmail.com> Date: Sun Jun 29 15:26:58 2025 +0100 Refactor of Specialise.hs I removed a test for a dictionary with unbound type variables. This turned out to be wrong; see #27629. So this MR re-introduces it. - - - - - 8e09a46f by Duncan Coutts at 2026-09-15T20:11:35-04:00 Refactor (and rename) removeFromQueues, to simplify I/O managers Rename it to unblockAndAppendToRunQueue which better reflects what it is intended to do. The post-condition for unblockAndAppendToRunQueue is that the TSO is on the run queue or it is in the process of migrating to another cap. Previously it achieved that by always directly adding the TSO to the run queue itself. But this actually made things more complicated for the I/O managers, because it meant they needed a separate code path for notifying for cancellation compared to notifying for completion. The general notification code would always add the TSO to the run queue itself. So the improvement is to allow different cases in unblockAndAppendToRunQueue to achieve the same outcome in different ways: either directly adding to the run queue or calling helper functions that do so themselves. This then allows the new I/O managers to share code between the sync and async cancellation, and to reuse their notifyIOCompletion helpers for cancellation. This avoids a source of bugs where the completion path may be updated but the cancellation path may be forgotten, or similarly in future for sync/async operations. Update all the existing in-RTS I/O managers, and the posix timeout code. - - - - - d668782c by Duncan Coutts at 2026-09-15T20:11:35-04:00 Document that awaitCompletedTimeoutsOrIO expects an empty run queue This was true before but implicit and not relied on much. It's better to be explicit, and allow things to depend on it. - - - - - 11d938bb by Duncan Coutts at 2026-09-15T20:11:35-04:00 Store the I/O opcode and fd in the StgAsyncIOOp This will be useful in several I/O managers and it is handy for logging and debugging. It also doesn't increase the size of the StgAsyncIOOp structure. There was enough spare padding space already. Update the poll I/O manager to set the new fields. Add a helper function to convert the enum IOReadOrWrite into the enum IOOpCode. Also change IOReadOrWrite to be an enum without a typedef, for consistency with other enumerations in IOManager.h - - - - - bfa87882 by Duncan Coutts at 2026-09-15T20:11:36-04:00 Add a new I/O manager based on select() Yes, this is the second such I/O manager, but it is a modern re-implementation based on the new in-RTS I/O manager infrastructure. So it is cleaner and faster than the old select I/O manager. Why do we need another I/O manager based on select? Why isn't the poll() one good enough as a baseline portable unix I/O manager? Because macOS. Apple Inc. is why we cannot have nice things. The man page for poll on macOS documents the fact that it does not work. At least, it does not work for all files. Specifically, it does not work for device files. Whereas macOS select() does work for device files. Aaaaarg! We _do_ want to deprecate and remove the old select I/O manager, but due to macOS we cannot do that until we have a replacement. This is that replacement. Until of course a nice new k-queue I/O manager arrives, which could become the new default for macOS and FreeBSD. Interestingly, this select I/O manager is actually faster than the poll one, on Linix, in some circumstances: specifically when many Haskell threads are waiting on the same fd. The poll I/O manager does O(n) work for n threads waiting on I/O, whereas the select one does O(fds) work for the number of fds that threads are waiting on. Usually this is 1:1, so it's not noticable, but one can concoct extreme benchmarks to show the difference. - - - - - 8e9de15a by Duncan Coutts at 2026-09-15T20:11:36-04:00 Minor updates in the poll I/O manager to keep in sync with select This keeps it in sync with select one. The changes are based on code review while implementing the new select I/O manager. The two I/O managers are so similar in structure that it makes sense to try to minimise the diff between them. This should aid understanding, and fixes to both in future. - - - - - 222668bb by Duncan Coutts at 2026-09-15T20:11:36-04:00 Document the new select I/O manager in the user guide in the RTS section about I/O managers. And add a changelog entry. - - - - - 0f48735b by Cheng Shao at 2026-09-15T20:12:23-04:00 Revert "wasm: workaround WebKit bug in dyld" This reverts commit c9b8465c2c338176fcab9d197e9d31f4aee11f68. The underlying webkit bug has been fixed in https://commits.webkit.org/314219@main. Closes #27564. - - - - - 809294f1 by Luite Stegeman at 2026-09-15T20:13:08-04:00 rts: make stg_threadLabelzh return a valid pointer for unlabeled threads. This fixes a segfault in the GC caused by stg_threadLabelzh returning a 0 pointer in a GC pointer field. stg_threadLabelzh returns a tuple of type (# Int#, ByteArray# #). If a thread has no label, the second field is unused. We must still return a valid heap object pointer. Instead of returning 0, we now return stg_DEAD_SLOT_closure. fixes #27618 - - - - - e2d57026 by Luite Stegeman at 2026-09-15T20:13:54-04:00 JS: fix selector thunk handling fix a bug where entering a selector thunk could lead to a crash if the selected field was still a thunk. fixes #27788 - - - - - 82ca0dc9 by Duncan Coutts at 2026-09-16T06:52:02+01:00 Make signal handling be a responsibility of the I/O manager(s) Previously it was scattered between I/O managers and the scheduler, and especially the scheduler's deadlock detection. Previously the scheduler would poll for pending signals each iteration of the scheduler loop. The scheduler also had some hairy signal functionality in the deadlock detection: in the non-threaded RTS (only) if there were still no threads running after deadlock detection then it would block waiting for signals. But signals can and (in my opinion) should be thought of as just a funny kind of I/O, and thus should be a responsibility of the I/O manager. So now we have the I/O managers poll for signals when they are polling for I/O completion (and removing the separate poll in the scheduler). And when I/O managers block waiting for I/O then they now also start signal handlers if they get interrupted by a signal. Crucially, if there is no pending I/O or timers, the awaitCompletedTimeoutsOrIO will still block waiting for signals. This patch puts us into an intermediate state: it temporarily breaks deadlock detection in the non-threaded RTS. The waiting on I/O currently happens before deadlock detection. This means we'll now wait forever on signals before doing deadlock detection. We need to move waiting after deadlock detection. We'll do that in a later patch. - - - - - ddfe4e39 by Duncan Coutts at 2026-09-16T06:52:02+01:00 Clean up the RTS internal signal handling API Now that the I/O manager is responsible for signals, we can simplify the API we present for signal handling. We now just need startPendingSignalHandlers, which is called from the I/O managers. We can get rid of awaitUserSignals. We also don't need RtsSignals.h to re-export the platform-specific posix/Signals.h or win32/ConsoleHandler.h We can also hide more of the implementation of signals. Less has to be exposed in posix/Signals.h or win32/ConsoleHandler.h. Indeed, posix/Signals.h becomes empty and we remove it. Partly this is because we don't need inline functions (or macros) in the interface. Also remove signal_handlers from RTS ABI exported symbols list. It does not appear to have any users in the core libs, and its really an internal implementation detail. It should not be exposed unless it's really necessary. - - - - - 33029f9b by Duncan Coutts at 2026-09-16T06:52:02+01:00 In the scheduler, move I/O blocking after deadlock detection To make deadlock detection effective in the non-threaded RTS when there are deadlocked threads and other unrelated threads waiting on I/O, we need to arrange to do deadlock detection before we block in scheduler to wait on I/O. The solution is to: 1. adjust scheduleFindWork, which runs before deadlock detection, to only poll for I/O and not block; and 2. add a step after deadlock detection to wait on I/O if there are still no threads to run (and there's any I/O or timeouts outstanding) The scheduleCheckBlockedThreads is now so simple that it made more sense to inline it into scheduleFindWork. - - - - - 94401957 by Duncan Coutts at 2026-09-16T06:52:02+01:00 Remove bogus anyPendingTimeoutsOrIO guard from scheduleDetectDeadlock The deadlock detection was only invoked if both of these conditions hold: 1. the run queue is empty 2. there is no pending I/O or timeouts The second condition is unnecessary. The deadlock detection mechanism can find deadlocks even if there are other threads waiting on I/O or timers. Having this extra condition means that we fail to detect blocked threads if there are any threads waiting on I/O or timers. Part of fixing issue #26408 - - - - - 575632ad by Duncan Coutts at 2026-09-16T06:52:02+01:00 Don't consider pending I/O for early context switch optimisation Context switches are normally initiated by the timer signal. If however the user specifies "context switch as often as possible", with +RTS -C0 then the scheduler arranges for an early context switch (when it's just about to run a Haskell thread). Context switching very often is expensive, so as an optimisation there cases where we do not arrange an early context switch: 1. if there's no other threads to run 2. if there is no pending I/O or timers This patch eliminates case 2, leaving only case 1. The rationale is as follows. The use of this was inconsistent across platforms and threaded/non-threaded RTS ways. It only worked on the non-threaded RTS and on Windows only worked for the win32-legacy I/O manager. On all other combinations anyPendingTimeoutsOrIO would always return false. The fact that nobody noticed and complained about this inconsistency suggests that the feature is not relied upon. If however it turns out that applications do rely on this, then the proper thing to do is not to restore this check, but to add a new I/O manager hint function that returns if there is any pending events that are likely to happen *soon*: for example timeouts expiring within one timeslice, or I/O waits on things likely to complete soon like disk I/O, but not for example socket/pipe I/O. The motivation to avoid this use of anyPendingTimeoutsOrIO is to allow us to eliminate anyPendingTimeoutsOrIO entirely. All other uses of this are just guards on {await,poll}CompletedTimeoutsOrIO and the guards can safely be folded into those functions. This will better cope with some I/O managers having no proper implementation of anyPendingTimeoutsOrIO. Ultimately this will let us simplify the scheduler which currently has to have special #ifdef mingw32_HOST_OS cases to cope with the lack of a working anyPendingTimeoutsOrIO for some Windows I/O managers - - - - - babfba3d by Duncan Coutts at 2026-09-16T06:52:02+01:00 Remove anyPendingTimeoutsOrIO guarding {poll,await}CompletedTimeoutsOrIO Previously the API of the I/O manager used a two step process: check anyPendingTimeoutsOrIO and then call {poll,await}CompletedTimeoutsOrIO. This was primarily there as a performance thing, to cheaply check if we need to do anything. And then because anyPendingTimeoutsOrIO existed, it was used for other things too. We have now eliminated the other uses, and are just left with the performance pattern. But this was problematic because not all I/O managers correctly implement anyPendingTimeoutsOrIO (specifically the win32 ones), and now that we also make I/O managers responsible for signals then we need to poll/await even if there is no pending I/O or timeouts. If there is no pending I/O or timeouts then await needs to degenerate to just waiting forever for any signals. - - - - - cb4fefc8 by Duncan Coutts at 2026-09-16T06:52:02+01:00 Remove anyPendingTimeoutsOrIO, it is no longer used And this avoids the problems arising from the win32 I/O managers having had a bogus implementation. - - - - - dcb8cc08 by Duncan Coutts at 2026-09-16T06:52:02+01:00 Remove second scheduler call to awaitCompletedTimeoutsOrIO Previously awaitCompletedTimeoutsOrIO was called both before and after deadlock detection in the scheduler. The reason for that was that the win32 I/O managers had a bogus implementation of anyPendingTimeoutsOrIO and this was used to guard the call of awaitCompletedTimeoutsOrIO prior to deadlock detection. This meant the first call site was never actually called when using the win32 I/O managers. This was the reason for the second call: the first one was never used. What a mess. So now we have a simple design in the scheduler: 1. poll for completed I/O, timers or signals 2. if no runnable threads: do deadlock detection 3. if still no runnable threads: block waiting for I/O, timers or signals. - - - - - 9115d88f by Duncan Coutts at 2026-09-16T06:52:02+01:00 Lift emptyRunQueue guard out of scheduleDetectDeadlock this improved the clarity of the logic when reading the scheduler code. - - - - - fddbaf83 by Duncan Coutts at 2026-09-16T06:52:02+01:00 Make non-threaded deadlock detection also rely on idle GC Only do deadlock detection GC when idle GC kicks in. This also relies on using wakeUpRts, so now do this unconditionally. Previously wakeUpRts was for the threaded rts only. - - - - - fafa5d3e by Duncan Coutts at 2026-09-16T06:52:02+01:00 Enable idle GC by default on non-threaded RTS The behaviour is now uniform between the threaded and non-threaded RTS ways. The deadlock detection now relies on idle GC for both threaded and non-threaded ways. Previously deadlock detection did not rely on idle GC for the non-threaded way. Also tweak test T7275 to account for idle GC. This test's output is sensitive to the number of major GCs run. Since this commit enables idle GC for the non-threaded RTS, for this test that increases the number of major GCs, since the test program is frequently idle for more than 300ms. - - - - - 2c92ab6e by Duncan Coutts at 2026-09-16T06:52:02+01:00 Fix state of idle GC control vars with +RTS -V0 Currently when the user uses +RTS -I0, then doIdleGC is set to false. But if the master tick interval -V is set to 0 then the idleGCDelayTime was being set to 0 but doIdleGC was not being set to false, which is inconsistent, and almost certainly buggy. - - - - - 39c5246b by Duncan Coutts at 2026-09-16T06:52:02+01:00 Add a long Note [Deadlock detection] It describes the historical and modern designs and their trade-offs. The point is we've now unified the code for deadlock detection between the threaded and non-threaded ways, by changing the non-threaded to follow the same design as the threaded. - - - - - 4f6abab9 by Duncan Coutts at 2026-09-16T06:52:02+01:00 Add a test for deadlock detection, issue #26408 - - - - - 88d6a2f6 by Duncan Coutts at 2026-09-16T06:52:02+01:00 Update the user guide with the revised idle GC behaviour i.e. it's now not just for the threaded RTS, but general. Also document the fact that disabling idle GC also disables deadlock detection. And add a changelog entry. - - - - - 8402a41c by Duncan Coutts at 2026-09-16T06:52:02+01:00 Move idle GC tracking to its own file, step 1 of 2 Add a new IdleGC.{c,h} module. Move the RecentActivity type, recent_activity variable and {get,set}RecentActivity wrappers. - - - - - e729131f by Duncan Coutts at 2026-09-16T06:52:02+01:00 Move idle GC functionality to its own file, step 2 of 2 We can now fully encapsulate the recent_activity state, exposing just three hooks, and one query used in the scheduler and timer tick. This is a prelude to making changes to the implementation of tracking when to perform an idle gc. - - - - - bd233355 by Duncan Coutts at 2026-09-16T06:52:02+01:00 Refactor the utils providing the I/O timeout for different system APIs We have to provide a timeout in three different time formats (int milliseconds, struct timeval *, and struct timespec *). These are for platform APIs including poll, select, ppoll, pselect, epoll_wait, epoll_wait2 and in future kevent and io_uring. Previously we had one util for each format that both worked out what the timeout should be and converted it into the particular format. This duplicates some logic and it also makes it awkward to modify the timeout value (since there's no common format). Modifying the timeout value is exactly what we want to do, in the next commit, where we want to adjust the timeout to take account of idle GC. - - - - - cd4aaf05 by Duncan Coutts at 2026-09-16T06:52:02+01:00 Add idle GC mode for !HAVE_PREEMPTION We have to do idle GC (and thus deadlock detection) differently when we do not have the ticker and the ability to interrupt the I/O manager when it's blocking. - - - - - 105 changed files: - .gitlab/ci.sh - + changelog.d/27732 - + changelog.d/T27629 - + changelog.d/T27788-js-selector - + changelog.d/fix-compacting-gc-constr-27649 - + changelog.d/fix-control0-mask-trampoline - + changelog.d/fix-threadlabel-segfault-27618 - + changelog.d/idle-gc-and-deadlock-detection - + changelog.d/select-io-manager - compiler/GHC/Core/Opt/Specialise.hs - compiler/GHC/Data/FastString.hs - compiler/GHC/Hs/Expr.hs - compiler/GHC/Hs/Pat.hs - compiler/GHC/Hs/Utils.hs - compiler/GHC/Parser.y - compiler/GHC/Parser/Annotation.hs - compiler/GHC/Parser/PostProcess.hs - compiler/GHC/Rename/Module.hs - compiler/GHC/StgToJS/Apply.hs - compiler/GHC/StgToJS/Symbols.hs - compiler/GHC/Tc/Errors/Types.hs - docs/users_guide/exts/type_data.rst - docs/users_guide/runtime_control.rst - libraries/base/src/GHC/RTS/Flags.hs - libraries/ghc-internal/src/GHC/Internal/RTS/Flags.hsc - rts/Capability.c - rts/ContinuationOps.cmm - rts/IOManager.c - rts/IOManager.h - rts/IOManagerInternals.h - + rts/IdleGC.c - + rts/IdleGC.h - rts/Linker.c - rts/PrimOps.cmm - rts/RaiseAsync.c - rts/RtsFlags.c - rts/RtsSignals.h - rts/RtsStartup.c - rts/RtsSymbols.c - rts/Schedule.c - rts/Schedule.h - rts/StgMiscClosures.cmm - rts/Timer.c - rts/configure.ac - rts/include/rts/Flags.h - rts/include/rts/storage/Closures.h - rts/include/stg/MiscClosures.h - rts/posix/Poll.c - rts/posix/Poll.h - rts/posix/Select.c - + rts/posix/SelectBis.c - + rts/posix/SelectBis.h - rts/posix/Signals.c - − rts/posix/Signals.h - rts/posix/Timeout.c - rts/posix/Timeout.h - rts/rts.cabal - rts/sm/Compact.c - rts/win32/AwaitEvent.c - rts/win32/ConsoleHandler.c - rts/win32/ConsoleHandler.h - testsuite/tests/codeGen/should_run/all.T - testsuite/tests/ghc-api/T25121_status.stdout - testsuite/tests/interface-stability/ghc-experimental-exports.stdout - testsuite/tests/interface-stability/ghc-experimental-exports.stdout-mingw32 - + testsuite/tests/javascript/T27788.hs - + testsuite/tests/javascript/T27788.stdout - testsuite/tests/javascript/all.T - testsuite/tests/parser/should_compile/DumpParsedAstComments.stderr - testsuite/tests/parser/should_compile/DumpSemis.stderr - testsuite/tests/printer/Test20297.stdout - testsuite/tests/profiling/should_run/Makefile - + testsuite/tests/rts/T26408.hs - + testsuite/tests/rts/T26408.stderr - + testsuite/tests/rts/T27618.hs - + testsuite/tests/rts/T27618.stdout - + testsuite/tests/rts/T27649.hs - + testsuite/tests/rts/T27649.stdout - testsuite/tests/rts/all.T - + testsuite/tests/rts/continuations/T27651.hs - + testsuite/tests/rts/continuations/T27651.stdout - testsuite/tests/rts/continuations/all.T - testsuite/tests/simplCore/should_compile/Makefile - + testsuite/tests/simplCore/should_compile/T27629.hs - + testsuite/tests/simplCore/should_compile/T27629.stderr - + testsuite/tests/simplCore/should_compile/T27629Plugin.hs - testsuite/tests/simplCore/should_compile/all.T - + testsuite/tests/type-data/should_fail/T27732a.hs - + testsuite/tests/type-data/should_fail/T27732a.stderr - + testsuite/tests/type-data/should_fail/T27732b.hs - + testsuite/tests/type-data/should_fail/T27732b.stderr - + testsuite/tests/type-data/should_fail/T27732c.hs - + testsuite/tests/type-data/should_fail/T27732c.stderr - + testsuite/tests/type-data/should_fail/T27732d.hs - + testsuite/tests/type-data/should_fail/T27732d.stderr - + testsuite/tests/type-data/should_fail/T27732e.hs - + testsuite/tests/type-data/should_fail/T27732e.stderr - + testsuite/tests/type-data/should_fail/T27732f.hs - + testsuite/tests/type-data/should_fail/T27732f.stderr - testsuite/tests/type-data/should_fail/TDStrictnessGADT.stderr - testsuite/tests/type-data/should_fail/TDStrictnessH98.stderr - testsuite/tests/type-data/should_fail/all.T - utils/check-exact/ExactPrint.hs - utils/check-exact/Utils.hs - utils/jsffi/dyld.mjs The diff was not included because it is too large. View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/74c8aefcea73c1f18e6bfa530a783c8... -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/74c8aefcea73c1f18e6bfa530a783c8... You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help