Duncan Coutts pushed to branch wip/dcoutts/io-manager-selectbis at Glasgow Haskell Compiler / GHC Commits: 722236dd by sheaf at 2026-07-18T08:48:31-04:00 Coercion optimisation: avoid double-Sym for InstCo Ticket #27374 pointed out an issue with GHC.Core.Coercion.Opt.optCoercion's handling of InstCo: it contravened (LC2) in Note [The LiftingContext in optCoercion] because it applied the ambient 'sym' to a coercion that was then added to the lifting context substitution. Fixes #27374 Co-authored-by: Simon Jakobi <simon.jakobi@gmail.com> - - - - - ff70fc75 by sheaf at 2026-07-18T08:48:31-04:00 Coercion optimisation: avoid exponential behaviour The change to coercion optimisation of 'InstCo' in the previous commit introduces exponential behaviour to the coercion optimiser. To avoid this, this commit provides a way to push in 'Sym' of an already-optimised coercion: GHC.Core.Coercion.Opt.mkDeepSymCo. See Note [Pushing Sym without re-optimising] in GHC.Core.Coercion.Opt. - - - - - dfef27f0 by Duncan Coutts at 2026-07-18T08:49:12-04:00 Move THREADED_RTS-conditional struct members to end of Capability Accessing members of the Capability struct from CMM code rely on accessor macros. (The macros are generated by deriveConstants). These macros have a single definition. This means that the offsets of all struct members must *not* vary based on THREADED_RTS vs !THREADED_RTS. This requires that any struct members that are conditional on THREADED_RTS must occur after the unconditional struct members. Hence we move all the ones that are conditional on THREADED_RTS to the end. Add a deriveConstants entry for the iomgr member of the Capability struct, which was the motivation for this change. Add warning messages to help our future selves. Debugging this took me a couple hours in gdb! - - - - - c254e022 by Duncan Coutts at 2026-07-18T08:49:12-04:00 Make the IOManager API use CapIOManager rather than Capability This makes the API somewhat more self-contained and more consistent. Now the IOManager API and each of the backends takes just the I/O manager structure. Previously we had a bit of a mixture, depending on whether the function needed access to the Capability or just the CapIOManager. We still need access to the cap, so we introduce a back reference to reach the capability, via iomgr->cap. Convert all uses in select and poll backends, but not win32 ones. Convert callers in the scheduler and elsewhere. Also convert the three CMM primops that call IOManager APIs. They just need to use Capability_iomgr(MyCapability()). - - - - - 4f3d8f31 by Duncan Coutts at 2026-07-18T08:49:12-04:00 Split posix/MIO.c out of posix/Signals.c The MIO I/O manager was secretly living inside the Signals file. Now it gets its own file, like any other self-respecting I/O manager. - - - - - 52ce04a9 by Duncan Coutts at 2026-07-18T08:49:12-04:00 Rationalise some scheduler run queue utilities Move them all to the same place in the file. Make some static that were used only internally. Also remove a redundant assignment after calling truncateRunQueue that is already done within truncateRunQueue. - - - - - 75bbdebc by Duncan Coutts at 2026-07-18T08:49:12-04:00 Rename initIOManager{AfterFork} to {re}startIOManager These are more accurate names, since these actions happen after initialisation and are really about starting (or restarting) background threads. - - - - - 724c0517 by Duncan Coutts at 2026-07-18T08:49:12-04:00 Free per-cap I/O managers during shutdown and forkProcess Historically this was not strictly necessary. The select and win32 legacy I/O managers did not maintain any dynamically allocated resources. The new poll one does (an auxillary table), and so this should be freed. After forkProcess, all threads get deleted. This includes threads waiting on I/O or timers. So as of this patch, resetting the I/O manager is just about tidying things up. For example, for the poll I/O manager this will reset the size of the AIOP table (which otherwise grows but never shrinks). In future however the re-initialising will become neeecessary for functionality, since some I/O managers will need to re-initialise wakeup fds that are set CLOEXEC. - - - - - c007d122 by Duncan Coutts at 2026-07-18T08:49:12-04:00 Add a TODO to the MIO I/O manager The direction of travel is to make I/O managers per-capability and have all their state live in the struct CapIOManager. The MIO I/O manager however still has a number of global variables. It's not obvious how handle these globals however. - - - - - b65ab7b3 by Duncan Coutts at 2026-07-18T08:49:12-04:00 Add a FIXME note in the Poll I/O manager - - - - - daf2bd6f by Duncan Coutts at 2026-07-18T08:49:12-04:00 Add missing updateRemembSetPushClosure in poll I/O manager For the non-moving GC. - - - - - e33ca830 by Duncan Coutts at 2026-07-18T08:49:12-04:00 Minor doc improvement to struct StgAsyncIOOp member outcome Mention the enumeration names, as well as their numeric values. The rest of the code uses the enum names. - - - - - 4edd2579 by Duncan Coutts at 2026-07-18T08:49:12-04:00 Minor doc improvements for StgTSOBlockInfo Clarify that certain union members are used only by certain legacy I/O managers. Hopefully we will be able to remove these at some point. - - - - - 536bedbb by Duncan Coutts at 2026-07-18T08:49:12-04:00 Avoid exporting various win32-specific rts symbols The BeginPrivate.h / EndPrivate.h scheme works perfectly well on Windows, but all of the rts/win32/*.h files were not using it. - - - - - 8139b5ac by Duncan Coutts at 2026-07-18T08:49:12-04:00 Remove wakeupIOManager, ioManagerWakeup and setIOManagerWakeupFd We no longer need wakeupIOManager for the threaded RTS case, so we can remove it and the bits only needed to support it. This includes the pipe/eventfd fd shared between the RTS and the in-library I/O manager used for waking up the I/O manager thread. The pipe/eventfd still exists, but it no longer has to be communicated to the RTS, since the RTS no longer needs to use it. So we remove the RTS API export setIOManagerWakeupFd, and remove uses of it within the I/O managers in ghc-internal. - - - - - 74fe7c66 by Duncan Coutts at 2026-07-18T08:49:12-04:00 Add a new interruptIOManager API for the I/O managers It will be used to interrupt awaitCompletedTimeoutsOrIO. Also update the return type and docs for awaitCompletedTimeoutsOrIO to have it return false when it gets interrupted, and have no useful post condition in that case. - - - - - 38792843 by Duncan Coutts at 2026-07-18T08:49:13-04:00 Add interruptIOManager support for select I/O manager Uses the FdWakup mechanism. - - - - - 2f3b00aa by Duncan Coutts at 2026-07-18T08:49:13-04:00 Add interruptIOManager support for poll I/O manager Uses the FdWakup mechanism. A quirk we have to cope with is that we now need to poll one more fd -- the wakeup_fd_r -- but this fd has no corresponding entry in the aiop_table. This is awkward since we have set up our aiop_poll_table to be an auxilliary table with matching indicies. The solution this patch uses (and described in the comments) is to have two tables: struct pollfd *aiop_poll_table, *full_poll_table; and to have the aiop_poll_table alias the tail of the full_poll_table. The head entry in the full_poll_table is the extra fd. So we poll the full_poll_table, while the aiop_poll_table still has matching indicies with the aiop_table. Hurrah for C aliasing rules. - - - - - cee50131 by Duncan Coutts at 2026-07-18T08:49:13-04:00 Add interruptIOManager support for win32 legacy I/O manager And remove unused related helper resetAbandonRequestWait. It is not called because the event is created in auto-reset mode, so never needs to be reset manually. - - - - - cf453143 by Duncan Coutts at 2026-07-18T08:49:13-04:00 Note lack of interruptIOManager support for WinIO I/O manager Though there's a plausible design, we can't sanely test it at the moment due to related WinIO bugs. Filed as issue #27403. - - - - - 1b74a0ad by Duncan Coutts at 2026-07-18T08:49:13-04:00 Be more explicit about enum IOReadOrWrite values, and type within cmm Belt and braces. - - - - - af228f98 by Duncan Coutts at 2026-07-18T21:16:32+01:00 Add a new I/O manager based on select() Yes, this is the second such I/O manager, but it is a modern re-implementation based on the new in-RTS I/O manager infrastructure. So it is cleaner and faster than the old select I/O manager. Why do we need another I/O manager based on select? Why isn't the poll() one good enough as a baseline portable unix I/O manager? Because macOS. Apple Inc. is why we cannot have nice things. The man page for poll on macOS documents the fact that it does not work. At least, it does not work for all files. Specifically, it does not work for device files. Whereas macOS select() does work for device files. Aaaaarg! We _do_ want to deprecate and remove the old select I/O manager, but due to macOS we cannot do that until we have a replacement. This is that replacement. Until of course a nice new k-queue I/O manager arrives, which could become the new default for macOS and FreeBSD. Interestingly, this select I/O manager is actually faster than the poll one, on Linix, in some circumstances: specifically when many Haskell threads are waiting on the same fd. The poll I/O manager does O(n) work for n threads waiting on I/O, whereas the select one does O(fds) work for the number of fds that threads are waiting on. Usually this is 1:1, so it's not noticable, but one can concoct extreme benchmarks to show the difference. - - - - - 9f34b5fa by Duncan Coutts at 2026-07-18T21:16:32+01:00 Document the new select I/O manager in the user guide in the RTS section about I/O managers. And add a changelog entry. - - - - - 1423e247 by Duncan Coutts at 2026-07-18T21:16:32+01:00 Update the "location" for blockedOnBadFD exception This exception is thrown by the select, selectbis and poll I/O managers in rare circumstances. The existing location for the error was "awaitEvent" which is an old name that was internal to the RTS. The Haskell functions which this gets thrown from are the functions threadWaitRead / threadWaitWrite. So this is a more appropriate loction string. - - - - - 3222607b by Duncan Coutts at 2026-07-18T21:16:32+01:00 Minor updates in the poll I/O manager to keep in sync This keeps it in sync with select one. The changes are based on code review while implementing the new select I/O manager. The two I/O managers are so similar in structure that it makes sense to try to minimise the diff between them. This should aid understanding, and fixes to both in future. - - - - - 7860dd93 by Duncan Coutts at 2026-07-18T21:16:32+01:00 Use selectbis I/O manager by default for CI coverage This should not be committed to master. It would be nice however to get better CI coverage of non-default I/O managers. - - - - - 55 changed files: - + changelog.d/T27374 - + changelog.d/select-io-manager - compiler/GHC/Core/Coercion/Opt.hs - docs/users_guide/runtime_control.rst - libraries/base/src/GHC/RTS/Flags.hs - libraries/ghc-internal/src/GHC/Internal/Event/Control.hs - libraries/ghc-internal/src/GHC/Internal/Event/Manager.hs - libraries/ghc-internal/src/GHC/Internal/Event/Thread.hs - libraries/ghc-internal/src/GHC/Internal/Event/TimerManager.hs - libraries/ghc-internal/src/GHC/Internal/RTS/Flags.hsc - rts/Capability.c - rts/Capability.h - rts/IOManager.c - rts/IOManager.h - rts/IOManagerInternals.h - rts/PrimOps.cmm - rts/RaiseAsync.c - rts/RtsStartup.c - rts/RtsSymbols.c - rts/Schedule.c - rts/Schedule.h - rts/configure.ac - rts/include/rts/Flags.h - rts/include/rts/IOInterface.h - rts/include/rts/storage/Closures.h - rts/include/rts/storage/TSO.h - rts/posix/FdWakeup.h - + rts/posix/MIO.c - + rts/posix/MIO.h - rts/posix/Poll.c - rts/posix/Poll.h - rts/posix/Select.c - rts/posix/Select.h - + rts/posix/SelectBis.c - + rts/posix/SelectBis.h - rts/posix/Signals.c - rts/posix/Signals.h - rts/posix/Timeout.c - rts/posix/Timeout.h - rts/rts.cabal - rts/win32/AsyncMIO.c - rts/win32/AsyncMIO.h - rts/win32/AsyncWinIO.h - rts/win32/AwaitEvent.c - rts/win32/AwaitEvent.h - rts/win32/ConsoleHandler.h - rts/win32/MIOManager.h - rts/win32/ThrIOManager.h - rts/win32/WorkQueue.h - rts/win32/veh_excn.h - + testsuite/tests/corelint/T27374.hs - testsuite/tests/corelint/all.T - testsuite/tests/interface-stability/ghc-experimental-exports.stdout - testsuite/tests/interface-stability/ghc-experimental-exports.stdout-mingw32 - utils/deriveConstants/Main.hs The diff was not included because it is too large. View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/c3af2a9db4a4c9f841988dc449de41f... -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/c3af2a9db4a4c9f841988dc449de41f... You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help