Duncan Coutts pushed to branch wip/dcoutts/io-manager-selectbis at Glasgow Haskell Compiler / GHC Commits: 0f64f348 by Cheng Shao at 2026-07-16T15:41:08+00:00 ci: add missing docker permission workaround in abi-test job - - - - - 660cb239 by Cheng Shao at 2026-07-16T19:37:48+00:00 bindist: Fix make install -j race condition on macos/freebsd This patch fixes make install -j race condition on macos/freebsd. BSD install fails with EEXIST when multiple install processes concurrently create the same prefix directory. So we add an `install_dirs` prerequisite job that sequentially creates the directories for subsequent jobs to work with. Fixes #27499. Co-authored-by: Codex <codex@openai.com> - - - - - 08130257 by Cheng Shao at 2026-07-16T19:37:48+00:00 ci: run bindist make install with -j This patch makes the ci scripts run `make install` with `-j` to reduce wall clock time when installing the bindist, see related issue for benchmark numbers. This only affects ghc ci logic, the user-facing default is up to distributors and is still `-j1`. Closes #27029. - - - - - d5ae6906 by Adam Gundry at 2026-07-17T04:57:43-04:00 Mark various language extension flags as deprecated (see #27329) The following language extensions are now deprecated: - AlternativeLayoutRule - AlternativeLayoutRuleTransitional - ParallelArrays - PolymorphicComponents - Rank2Types In addition, the warning `-Walternative-layout-rule-transitional` has been marked as deprecated, as it is emitted only under the deprecated extension `XAlternativeLayoutRuleTransitional`. - - - - - fe3b059c by Andrew Lelechenko at 2026-07-17T04:58:26-04:00 base: re-export GHC.Environment.getFullArgs from System.Environment CLC proposal https://github.com/haskell/core-libraries-committee/issues/431 - - - - - bb368efa by Duncan Coutts at 2026-07-17T10:43:19+01:00 Move THREADED_RTS-conditional struct members to end of Capability Accessing members of the Capability struct from CMM code rely on accessor macros. (The macros are generated by deriveConstants). These macros have a single definition. This means that the offsets of all struct members must *not* vary based on THREADED_RTS vs !THREADED_RTS. This requires that any struct members that are conditional on THREADED_RTS must occur after the unconditional struct members. Hence we move all the ones that are conditional on THREADED_RTS to the end. Add a deriveConstants entry for the iomgr member of the Capability struct, which was the motivation for this change. Add warning messages to help our future selves. Debugging this took me a couple hours in gdb! - - - - - 35c97fb0 by Duncan Coutts at 2026-07-17T10:43:19+01:00 Make the IOManager API use CapIOManager rather than Capability This makes the API somewhat more self-contained and more consistent. Now the IOManager API and each of the backends takes just the I/O manager structure. Previously we had a bit of a mixture, depending on whether the function needed access to the Capability or just the CapIOManager. We still need access to the cap, so we introduce a back reference to reach the capability, via iomgr->cap. Convert all uses in select and poll backends, but not win32 ones. Convert callers in the scheduler and elsewhere. Also convert the three CMM primops that call IOManager APIs. They just need to use Capability_iomgr(MyCapability()). - - - - - 4a89aee4 by Duncan Coutts at 2026-07-17T10:43:19+01:00 Split posix/MIO.c out of posix/Signals.c The MIO I/O manager was secretly living inside the Signals file. Now it gets its own file, like any other self-respecting I/O manager. - - - - - 3c4536c9 by Duncan Coutts at 2026-07-17T10:43:19+01:00 Rationalise some scheduler run queue utilities Move them all to the same place in the file. Make some static that were used only internally. Also remove a redundant assignment after calling truncateRunQueue that is already done within truncateRunQueue. - - - - - 1e02b5a8 by Duncan Coutts at 2026-07-17T10:43:19+01:00 Rename initIOManager{AfterFork} to {re}startIOManager These are more accurate names, since these actions happen after initialisation and are really about starting (or restarting) background threads. - - - - - 27e0aa4f by Duncan Coutts at 2026-07-17T10:43:19+01:00 Free per-cap I/O managers during shutdown and forkProcess Historically this was not strictly necessary. The select and win32 legacy I/O managers did not maintain any dynamically allocated resources. The new poll one does (an auxillary table), and so this should be freed. After forkProcess, all threads get deleted. This includes threads waiting on I/O or timers. So as of this patch, resetting the I/O manager is just about tidying things up. For example, for the poll I/O manager this will reset the size of the AIOP table (which otherwise grows but never shrinks). In future however the re-initialising will become neeecessary for functionality, since some I/O managers will need to re-initialise wakeup fds that are set CLOEXEC. - - - - - 4a70715e by Duncan Coutts at 2026-07-17T10:43:19+01:00 Add a TODO to the MIO I/O manager The direction of travel is to make I/O managers per-capability and have all their state live in the struct CapIOManager. The MIO I/O manager however still has a number of global variables. It's not obvious how handle these globals however. - - - - - 36232b96 by Duncan Coutts at 2026-07-17T10:43:19+01:00 Add a FIXME note in the Poll I/O manager - - - - - 9e17d6f5 by Duncan Coutts at 2026-07-17T10:43:19+01:00 Add missing updateRemembSetPushClosure in poll I/O manager For the non-moving GC. - - - - - acd2b67c by Duncan Coutts at 2026-07-17T10:43:19+01:00 Minor doc improvement to struct StgAsyncIOOp member outcome Mention the enumeration names, as well as their numeric values. The rest of the code uses the enum names. - - - - - 0cc3a3f0 by Duncan Coutts at 2026-07-17T10:43:19+01:00 Minor doc improvements for StgTSOBlockInfo Clarify that certain union members are used only by certain legacy I/O managers. Hopefully we will be able to remove these at some point. - - - - - f912bc5b by Duncan Coutts at 2026-07-17T10:43:19+01:00 Avoid exporting various win32-specific rts symbols The BeginPrivate.h / EndPrivate.h scheme works perfectly well on Windows, but all of the rts/win32/*.h files were not using it. - - - - - a6b000ba by Duncan Coutts at 2026-07-17T10:43:19+01:00 Remove wakeupIOManager, ioManagerWakeup and setIOManagerWakeupFd We no longer need wakeupIOManager for the threaded RTS case, so we can remove it and the bits only needed to support it. This includes the pipe/eventfd fd shared between the RTS and the in-library I/O manager used for waking up the I/O manager thread. The pipe/eventfd still exists, but it no longer has to be communicated to the RTS, since the RTS no longer needs to use it. So we remove the RTS API export setIOManagerWakeupFd, and remove uses of it within the I/O managers in ghc-internal. - - - - - 737d71cc by Duncan Coutts at 2026-07-17T10:43:19+01:00 Add a new interruptIOManager API for the I/O managers It will be used to interrupt awaitCompletedTimeoutsOrIO. Also update the return type and docs for awaitCompletedTimeoutsOrIO to have it return false when it gets interrupted, and have no useful post condition in that case. - - - - - 0f2fb3fd by Duncan Coutts at 2026-07-17T10:43:19+01:00 Add interruptIOManager support for select I/O manager Uses the FdWakup mechanism. - - - - - f7e46e9c by Duncan Coutts at 2026-07-17T10:43:19+01:00 Add interruptIOManager support for poll I/O manager Uses the FdWakup mechanism. A quirk we have to cope with is that we now need to poll one more fd -- the wakeup_fd_r -- but this fd has no corresponding entry in the aiop_table. This is awkward since we have set up our aiop_poll_table to be an auxilliary table with matching indicies. The solution this patch uses (and described in the comments) is to have two tables: struct pollfd *aiop_poll_table, *full_poll_table; and to have the aiop_poll_table alias the tail of the full_poll_table. The head entry in the full_poll_table is the extra fd. So we poll the full_poll_table, while the aiop_poll_table still has matching indicies with the aiop_table. Hurrah for C aliasing rules. - - - - - 81f5bc3a by Duncan Coutts at 2026-07-17T10:43:19+01:00 Add interruptIOManager support for win32 legacy I/O manager And remove unused related helper resetAbandonRequestWait. It is not called because the event is created in auto-reset mode, so never needs to be reset manually. - - - - - a357b29b by Duncan Coutts at 2026-07-17T10:43:19+01:00 Note lack of interruptIOManager support for WinIO I/O manager Though there's a plausible design, we can't sanely test it at the moment due to related WinIO bugs. Filed as issue #27403. - - - - - 04703995 by Duncan Coutts at 2026-07-17T10:43:19+01:00 Be more explicit about enum IOReadOrWrite values, and type within cmm Belt and braces. - - - - - e218c111 by Duncan Coutts at 2026-07-17T21:29:11+01:00 Add a new I/O manager based on select() Yes, this is the second such I/O manager, but it is a modern re-implementation based on the new in-RTS I/O manager infrastructure. So it is cleaner and faster than the old select I/O manager. Why do we need another I/O manager based on select? Why isn't the poll() one good enough as a baseline portable unix I/O manager? Because macOS. Apple Inc. is why we cannot have nice things. The man page for poll on macOS documents the fact that it does not work. At least, it does not work for all files. Specifically, it does not work for device files. Whereas macOS select() does work for device files. Aaaaarg! We _do_ want to deprecate and remove the old select I/O manager, but due to macOS we cannot do that until we have a replacement. This is that replacement. Until of course a nice new k-queue I/O manager arrives, which could become the new default for macOS and FreeBSD. Interestingly, this select I/O manager is actually faster than the poll one, on Linix, in some circumstances: specifically when many Haskell threads are waiting on the same fd. The poll I/O manager does O(n) work for n threads waiting on I/O, whereas the select one does O(fds) work for the number of fds that threads are waiting on. Usually this is 1:1, so it's not noticable, but one can concoct extreme benchmarks to show the difference. - - - - - a0909191 by Duncan Coutts at 2026-07-17T21:29:11+01:00 Add the new select I/O manager to the user guide in the RTS section about I/O managers. And add a changelog entry. - - - - - 9dfad42b by Duncan Coutts at 2026-07-17T21:29:11+01:00 fixup Add the new select I/O manager to the user guide - - - - - f14cbd54 by Duncan Coutts at 2026-07-17T21:29:11+01:00 Update the "location" for blockedOnBadFD exception This exception is thrown by the select, selectbis and poll I/O managers in rare circumstances. The existing location for the error was "awaitEvent" which is an old name that was internal to the RTS. The Haskell functions which this gets thrown from are the functions threadWaitRead / threadWaitWrite. So this is a more appropriate loction string. - - - - - c1603ba5 by Duncan Coutts at 2026-07-17T21:29:11+01:00 Minor updates in the poll I/O manager to keep in sync with select one, based on code review when implementing the new select I/O manager. The two are so similar that it makes sense to try to minimise the diff between them. - - - - - c3af2a9d by Duncan Coutts at 2026-07-17T21:29:11+01:00 Use selectbis I/O manager by default for CI coverage This should not be committed to master. It would be nice however to get better CI coverage of non-default I/O managers. - - - - - 108 changed files: - .gitlab-ci.yml - .gitlab/ci.sh - + changelog.d/T27329 - + changelog.d/fix-make-install-j - + changelog.d/select-io-manager - compiler/GHC/Driver/Flags.hs - compiler/GHC/Driver/Session.hs - compiler/GHC/Tc/Types/Rank.hs - docs/users_guide/expected-undocumented-flags.txt - docs/users_guide/exts/rank_polymorphism.rst - docs/users_guide/exts/static_pointers.rst - docs/users_guide/runtime_control.rst - hadrian/bindist/Makefile - libraries/base/changelog.md - libraries/base/src/GHC/RTS/Flags.hs - libraries/base/src/System/Environment.hs - libraries/ghc-internal/ghc-internal.cabal.in - libraries/ghc-internal/src/GHC/Internal/Event/Control.hs - libraries/ghc-internal/src/GHC/Internal/Event/Manager.hs - libraries/ghc-internal/src/GHC/Internal/Event/Thread.hs - libraries/ghc-internal/src/GHC/Internal/Event/TimerManager.hs - libraries/ghc-internal/src/GHC/Internal/RTS/Flags.hsc - rts/Capability.c - rts/Capability.h - rts/IOManager.c - rts/IOManager.h - rts/IOManagerInternals.h - rts/PrimOps.cmm - rts/RaiseAsync.c - rts/RtsStartup.c - rts/RtsSymbols.c - rts/Schedule.c - rts/Schedule.h - rts/configure.ac - rts/include/rts/Flags.h - rts/include/rts/IOInterface.h - rts/include/rts/storage/Closures.h - rts/include/rts/storage/TSO.h - rts/posix/FdWakeup.h - + rts/posix/MIO.c - + rts/posix/MIO.h - rts/posix/Poll.c - rts/posix/Poll.h - rts/posix/Select.c - rts/posix/Select.h - + rts/posix/SelectBis.c - + rts/posix/SelectBis.h - rts/posix/Signals.c - rts/posix/Signals.h - rts/posix/Timeout.c - rts/posix/Timeout.h - rts/rts.cabal - rts/win32/AsyncMIO.c - rts/win32/AsyncMIO.h - rts/win32/AsyncWinIO.h - rts/win32/AwaitEvent.c - rts/win32/AwaitEvent.h - rts/win32/ConsoleHandler.h - rts/win32/MIOManager.h - rts/win32/ThrIOManager.h - rts/win32/WorkQueue.h - rts/win32/veh_excn.h - testsuite/tests/backpack/should_compile/T13149.bkp - testsuite/tests/determinism/determ017/A.hs - testsuite/tests/ghci/scripts/T12005.script - testsuite/tests/haddock/perf/Fold.hs - testsuite/tests/indexed-types/should_fail/T7354.hs - testsuite/tests/interface-stability/base-exports.stdout - testsuite/tests/interface-stability/base-exports.stdout-javascript-unknown-ghcjs - testsuite/tests/interface-stability/base-exports.stdout-mingw32 - testsuite/tests/interface-stability/ghc-experimental-exports.stdout - testsuite/tests/layout/layout001.stdout - testsuite/tests/layout/layout002.stdout - testsuite/tests/layout/layout003.stdout - testsuite/tests/layout/layout004.stdout - testsuite/tests/layout/layout005.stdout - testsuite/tests/layout/layout006.stdout - testsuite/tests/layout/layout007.stdout - testsuite/tests/layout/layout008.stdout - testsuite/tests/layout/layout009.stdout - testsuite/tests/linear/should_compile/T1735Min.hs - + testsuite/tests/parser/should_compile/T13087.stderr - testsuite/tests/parser/should_fail/T8431.stderr - testsuite/tests/parser/should_fail/readFail038.stderr - testsuite/tests/perf/compiler/T3064.hs - testsuite/tests/polykinds/T7594.hs - testsuite/tests/programs/thurston-modular-arith/Main.hs - testsuite/tests/rts/ipe/IpeStats/Fold.hs - testsuite/tests/simplCore/should_compile/T11562.hs - testsuite/tests/simplCore/should_run/T3591.hs - testsuite/tests/typecheck/should_compile/DeepSubsumption02.hs - testsuite/tests/typecheck/should_compile/T12507.hs - testsuite/tests/typecheck/should_compile/T13951.hs - testsuite/tests/typecheck/should_compile/T18920.hs - testsuite/tests/typecheck/should_compile/T2595.hs - testsuite/tests/typecheck/should_compile/T7541.hs - testsuite/tests/typecheck/should_fail/T6069.stderr - testsuite/tests/typecheck/should_fail/T7368a.hs - testsuite/tests/typecheck/should_run/T1735_Help/Basics.hs - testsuite/tests/typecheck/should_run/T3731-short.hs - testsuite/tests/typecheck/should_run/T3731.hs - testsuite/tests/typecheck/should_run/church.hs - testsuite/tests/typecheck/should_run/tcrun008.hs - testsuite/tests/typecheck/should_run/tcrun017.hs - testsuite/tests/typecheck/should_run/tcrun026.hs - testsuite/tests/typecheck/should_run/tcrun035.hs - testsuite/tests/typecheck/should_run/tcrun036.hs - utils/deriveConstants/Main.hs The diff was not included because it is too large. View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/ddaede066c617886433795cc5c14b84... -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/ddaede066c617886433795cc5c14b84... You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help