Duncan Coutts pushed to branch wip/dcoutts/io-manager-selectbis at Glasgow Haskell Compiler / GHC Commits: 27711d99 by Duncan Coutts at 2026-07-17T15:04:59+01:00 Add a new I/O manager based on select() Yes, this is the second such I/O manager, but it is a modern re-implementation based on the new in-RTS I/O manager infrastructure. So it is cleaner and faster than the old select I/O manager. Why do we need another I/O manager based on select? Why isn't the poll() one good enough as a baseline portable unix I/O manager? Because macOS. Apple Inc. is why we cannot have nice things. The man page for poll on macOS documents the fact that it does not work. At least, it does not work for all files. Specifically, it does not work for device files. Whereas macOS select() does work for device files. Aaaaarg! We _do_ want to deprecate and remove the old select I/O manager, but due to macOS we cannot do that until we have a replacement. This is that replacement. Until of course a nice new k-queue I/O manager arrives, which could become the new default for macOS and FreeBSD. Interestingly, this select I/O manager is actually faster than the poll one, on Linix, in some circumstances: specifically when many Haskell threads are waiting on the same fd. The poll I/O manager does O(n) work for n threads waiting on I/O, whereas the select one does O(fds) work for the number of fds that threads are waiting on. Usually this is 1:1, so it's not noticable, but one can concoct extreme benchmarks to show the difference. - - - - - 53ae2a48 by Duncan Coutts at 2026-07-17T15:04:59+01:00 Add the new select I/O manager to the user guide in the RTS section about I/O managers. And add a changelog entry. - - - - - e5766371 by Duncan Coutts at 2026-07-17T15:04:59+01:00 Update the "location" for blockedOnBadFD exception This exception is thrown by the select, selectbis and poll I/O managers in rare circumstances. The existing location for the error was "awaitEvent" which is an old name that was internal to the RTS. The Haskell functions which this gets thrown from are the functions threadWaitRead / threadWaitWrite. So this is a more appropriate loction string. - - - - - db1ba766 by Duncan Coutts at 2026-07-17T15:04:59+01:00 Minor updates in the poll I/O manager to keep in sync with select one, based on code review when implementing the new select I/O manager. The two are so similar that it makes sense to try to minimise the diff between them. - - - - - ddaede06 by Duncan Coutts at 2026-07-17T15:06:15+01:00 Use selectbis I/O manager by default for CI coverage This should not be committed to master. It would be nice however to get better CI coverage of non-default I/O managers. - - - - - 17 changed files: - + changelog.d/select-io-manager - docs/users_guide/runtime_control.rst - libraries/base/src/GHC/RTS/Flags.hs - libraries/ghc-internal/src/GHC/Internal/Event/Thread.hs - libraries/ghc-internal/src/GHC/Internal/RTS/Flags.hsc - rts/IOManager.c - rts/IOManager.h - rts/IOManagerInternals.h - rts/configure.ac - rts/include/rts/Flags.h - rts/posix/Poll.c - + rts/posix/SelectBis.c - + rts/posix/SelectBis.h - rts/posix/Timeout.c - rts/posix/Timeout.h - rts/rts.cabal - testsuite/tests/interface-stability/ghc-experimental-exports.stdout Changes: ===================================== changelog.d/select-io-manager ===================================== @@ -0,0 +1,19 @@ +section: rts +issues: +mrs: !16359 +synopsis: + New I/O manager based on select() +description: + There is a new I/O manager on Posix systems based on select(). This exists + primarily to support macOS, where the poll() API does not work correctly + (specifically it is documented not to work for device files). It is the new + default I/O manager for the non-threaded RTS for the macOS platform. + + This is intended to allow the legacy select I/O manager to be retired. It is + also a stop-gap measure until a kqueue I/O manager is added. + + The new implementation is marginally faster in some cases. It scales better + for timers, O(log n) rather than O(m). For threads waiting on I/O it is + necessarily still O(n). If used to wait on fds > 1024 it will throw an IO + exception rather than terminating the RTS, as was the behaviour of the old + select I/O manager. ===================================== docs/users_guide/runtime_control.rst ===================================== @@ -1441,15 +1441,30 @@ limited. Currently the available I/O managers are: ================ ========= ============ - Name Platforms RTS way +I/O manager name Platforms RTS way ================ ========= ============ ``select`` Posix Non-threaded -``poll`` Posix Non-threaded +``selectbis`` Posix Non-threaded +``poll`` Posix(*) Non-threaded ``mio`` All Threaded ``win32-legacy`` Windows Non-threaded ``winio`` Windows Both ================ ========= ============ +(*) The ``poll`` I/O manager is not available on macOS due to platform +limitations. + +Currently the default I/O manager on each platform is: + +========= ============ =================== +Platform RTS way default I/O manager +========= ============ =================== +macOS Non-threaded ``selectbis`` +Posix Non-threaded ``poll`` +Windows Non-threaded ``win32-legacy`` +all Threaded ``mio`` +========= ============ =================== + .. rts-flag:: --io-manager=(name) Select the I/O manager to use. On some combinations of platform and @@ -1474,7 +1489,8 @@ This is because it uses a linked list for timers. This I/O manager is highly portable and its code is very mature: it is the I/O manager that has been used by GHC in the single-threaded RTS on Posix platforms -since time immemorial. +since time immemorial. It is likely to be retired, once the ``poll`` and +``selectbis`` I/O managers are mature enough to cover all use cases. Timer resolution: on 64bit platforms it supports microsecond precision timers while on 32bit platforms it only supports millisecond precision. Timer accuracy @@ -1485,6 +1501,29 @@ support 1024 open files. More specifically it supports file descriptors with numerical value up to 1024 but no higher. It will terminate the RTS (and thus typically the process) if this limit is exceeded. +The ``selectbis`` I/O manager +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +This I/O manager based on the classic Posix ``select()`` API. It supports +waiting on I/O readiness on non-blocking file descriptors (i.e. not disk files). +It is implemented within the RTS and is currently available only in the +non-threaded RTS. + +It scales poorly for I/O readiness notification: costing O(n) in the number of +threads that are waiting on I/O simultaneously. It scales well for timers: +most timer operations cost O(log n) in the number of simultaneous timers. This +is because it uses a heap data structure for timers. + +Timer resolution: this I/O manager supports microsecond precision timers. + +Limitation: on most platforms where it is available this I/O manager can only +support 1024 open files. More specifically it supports file descriptors with +numerical value up to 1024 but no higher. It will throw an IO exception if this +limit is exceeded. + +This I/O manager exists primarily to support macOS, due to ``poll()`` not +working properly on macOS, while ``select()`` does work. It's name reflects +the fact that it is the second I/O manager to be based on ``select()``. + The ``poll`` I/O manager ~~~~~~~~~~~~~~~~~~~~~~~~ @@ -1512,6 +1551,10 @@ limit can be adjusted using OS facilities (e.g. the ``ulimit`` command). Exceeding this limit will cause the RTS (and thus typically the process) to terminate. +This I/O manager is not available on macOS due to the ``poll()`` API not +working for all file types on macOS. Specifically the macOS man page for +``poll`` documents that it does not work for device files. + The ``mio`` I/O manager ~~~~~~~~~~~~~~~~~~~~~~~ This I/O manager is based on several platform-specific APIs. It supports ===================================== libraries/base/src/GHC/RTS/Flags.hs ===================================== @@ -390,10 +390,11 @@ internal_to_base_MiscFlags i@Internal.MiscFlags{..} = internal_to_base_ioManager Internal.IoManagerFlagAuto = IoManagerFlagAuto internal_to_base_ioManager Internal.IoManagerFlagSelect = IoManagerFlagSelect #if __GLASGOW_HASKELL__ >= 1000 + internal_to_base_ioManager Internal.IoManagerFlagSelectBis = IoManagerFlagAuto internal_to_base_ioManager Internal.IoManagerFlagPoll = IoManagerFlagAuto - -- This is a lie, we cannot translate poll. We cannot translate - -- accurately because want to freeze the API of the the compat RTS flags - -- here. Using "auto" is the least bad translation. + -- This is a lie, we cannot translate these new I/O managers. We cannot + -- translate accurately because want to freeze the API of the the compat + -- RTS flags here. Using "auto" is the least bad translation. -- https://github.com/haskell/core-libraries-committee/issues/362 #endif internal_to_base_ioManager Internal.IoManagerFlagMIO = IoManagerFlagMIO ===================================== libraries/ghc-internal/src/GHC/Internal/Event/Thread.hs ===================================== @@ -188,7 +188,7 @@ threadWait evt fd = mask_ $ do -- used at least by RTS in 'select()' IO manager backend blockedOnBadFD :: SomeException -blockedOnBadFD = toException $ errnoToIOError "awaitEvent" eBADF Nothing Nothing +blockedOnBadFD = toException $ errnoToIOError "threadWaitRead/Write" eBADF Nothing Nothing threadWaitSTM :: Event -> Fd -> IO (STM (), IO ()) threadWaitSTM evt fd = mask_ $ do ===================================== libraries/ghc-internal/src/GHC/Internal/RTS/Flags.hsc ===================================== @@ -184,6 +184,7 @@ data MiscFlags = MiscFlags data IoManagerFlag = IoManagerFlagAuto | IoManagerFlagSelect -- ^ Unix only, non-threaded RTS only + | IoManagerFlagSelectBis -- ^ Unix only, non-threaded RTS only | IoManagerFlagPoll -- ^ Unix only, non-threaded RTS only | IoManagerFlagMIO -- ^ cross-platform, threaded RTS only | IoManagerFlagWinIO -- ^ Windows only ===================================== rts/IOManager.c ===================================== @@ -33,6 +33,10 @@ #include "posix/Signals.h" #endif +#if defined(IOMGR_ENABLED_SELECTBIS) +#include "posix/SelectBis.h" +#endif + #if defined(IOMGR_ENABLED_POLL) #include "posix/Poll.h" #include "posix/Timeout.h" @@ -117,6 +121,14 @@ parseIOManagerFlag(const char *iomgrstr, IO_MANAGER_FLAG *flag) return IOManagerAvailable; #else return IOManagerUnavailable; +#endif + } + else if (strcmp("selectbis", iomgrstr) == 0) { +#if defined(IOMGR_ENABLED_SELECTBIS) + *flag = IO_MNGR_FLAG_SELECTBIS; + return IOManagerAvailable; +#else + return IOManagerUnavailable; #endif } else if (strcmp("poll", iomgrstr) == 0) { @@ -226,6 +238,8 @@ void selectIOManager(void) #else // !defined(THREADED_RTS) #if defined(IOMGR_DEFAULT_NON_THREADED_SELECT) iomgr_type = IO_MANAGER_SELECT; +#elif defined(IOMGR_DEFAULT_NON_THREADED_SELECTBIS) + iomgr_type = IO_MANAGER_SELECTBIS; #elif defined(IOMGR_DEFAULT_NON_THREADED_POLL) iomgr_type = IO_MANAGER_POLL; #elif defined(IOMGR_DEFAULT_NON_THREADED_WINIO) @@ -244,6 +258,12 @@ void selectIOManager(void) break; #endif +#if defined(IOMGR_ENABLED_SELECTBIS) + case IO_MNGR_FLAG_SELECTBIS: + iomgr_type = IO_MANAGER_SELECTBIS; + break; +#endif + #if defined(IOMGR_ENABLED_POLL) case IO_MNGR_FLAG_POLL: iomgr_type = IO_MANAGER_POLL; @@ -291,6 +311,10 @@ char * showIOManager(void) case IO_MANAGER_SELECT: return "select"; #endif +#if defined(IOMGR_ENABLED_SELECTBIS) + case IO_MANAGER_SELECTBIS: + return "selectbis"; +#endif #if defined(IOMGR_ENABLED_POLL) case IO_MANAGER_POLL: return "poll"; @@ -347,6 +371,12 @@ void initCapabilityIOManager(CapIOManager *iomgr) break; #endif +#if defined(IOMGR_ENABLED_SELECTBIS) + case IO_MANAGER_SELECTBIS: + initCapabilityIOManagerSelectBis(iomgr); + break; +#endif + #if defined(IOMGR_ENABLED_POLL) case IO_MANAGER_POLL: initCapabilityIOManagerPoll(iomgr); @@ -380,6 +410,12 @@ void freeCapabilityIOManager(CapIOManager *iomgr) break; #endif +#if defined(IOMGR_ENABLED_SELECTBIS) + case IO_MANAGER_SELECTBIS: + freeCapabilityIOManagerSelectBis(iomgr); + break; +#endif + #if defined(IOMGR_ENABLED_POLL) case IO_MANAGER_POLL: freeCapabilityIOManagerPoll(iomgr); @@ -399,10 +435,14 @@ void startIOManager(void) switch (iomgr_type) { -#if defined(IOMGR_ENABLED_SELECT) || defined(IOMGR_ENABLED_POLL) +#if defined(IOMGR_ENABLED_SELECT) || defined(IOMGR_ENABLED_SELECTBIS) \ + || defined(IOMGR_ENABLED_POLL) #if defined(IOMGR_ENABLED_SELECT) case IO_MANAGER_SELECT: #endif +#if defined(IOMGR_ENABLED_SELECTBIS) + case IO_MANAGER_SELECTBIS: +#endif #if defined(IOMGR_ENABLED_POLL) case IO_MANAGER_POLL: #endif @@ -479,6 +519,7 @@ restartIOManager(CapIOManager *iomgr, Capability **pcap) break; #endif /* The IO_MANAGER_SELECT needs no initialisation */ + /* The IO_MANAGER_SELECTBIS needs no initialisation */ /* The IO_MANAGER_POLL needs no initialisation */ /* No impl for any of the Windows I/O managers, since no forking. */ @@ -570,8 +611,13 @@ void markCapabilityIOManager(evac_fn evac, void *user, CapIOManager *iomgr) break; #endif +#if defined(IOMGR_ENABLED_SELECTBIS) || defined(IOMGR_ENABLED_POLL) +#if defined(IOMGR_ENABLED_SELECTBIS) + case IO_MANAGER_SELECTBIS: +#endif #if defined(IOMGR_ENABLED_POLL) case IO_MANAGER_POLL: +#endif markClosureTable(evac, user, &iomgr->aiop_table); evac(user, (StgClosure **)(void *)&iomgr->timeout_queue); break; @@ -599,8 +645,13 @@ void scavengeTSOIOManager(StgTSO *tso) * both of these are not GC pointers, so there is nothing to do. */ +#if defined(IOMGR_ENABLED_SELECTBIS) || defined(IOMGR_ENABLED_POLL) +#if defined(IOMGR_ENABLED_SELECTBIS) + case IO_MANAGER_SELECTBIS: +#endif #if defined(IOMGR_ENABLED_POLL) case IO_MANAGER_POLL: +#endif /* BlockedOn{Read,Write} uses block_info.aiop * BlockedOnDelay uses block_info.timeout * both of these are heap allocated, so we can do the same in all @@ -650,6 +701,11 @@ bool anyPendingTimeoutsOrIO(CapIOManager *iomgr) || (iomgr->sleeping_queue != END_TSO_QUEUE); #endif +#if defined(IOMGR_ENABLED_SELECTBIS) + case IO_MANAGER_SELECTBIS: + return anyPendingTimeoutsOrIOSelectBis(iomgr); +#endif + #if defined(IOMGR_ENABLED_POLL) case IO_MANAGER_POLL: return anyPendingTimeoutsOrIOPoll(iomgr); @@ -709,6 +765,12 @@ void pollCompletedTimeoutsOrIO(CapIOManager *iomgr) break; #endif +#if defined(IOMGR_ENABLED_SELECTBIS) + case IO_MANAGER_SELECTBIS: + pollCompletedTimeoutsOrIOSelectBis(iomgr); + break; +#endif + #if defined(IOMGR_ENABLED_POLL) case IO_MANAGER_POLL: pollCompletedTimeoutsOrIOPoll(iomgr); @@ -743,6 +805,12 @@ bool awaitCompletedTimeoutsOrIO(CapIOManager *iomgr) break; #endif +#if defined(IOMGR_ENABLED_SELECTBIS) + case IO_MANAGER_SELECTBIS: + completed = awaitCompletedTimeoutsOrIOSelectBis(iomgr); + break; +#endif + #if defined(IOMGR_ENABLED_POLL) case IO_MANAGER_POLL: completed = awaitCompletedTimeoutsOrIOPoll(iomgr); @@ -784,6 +852,12 @@ void interruptIOManager(CapIOManager *iomgr) break; #endif +#if defined(IOMGR_ENABLED_SELECTBIS) + case IO_MANAGER_SELECTBIS: + interruptIOManagerSelectBis(iomgr); + break; +#endif + #if defined(IOMGR_ENABLED_POLL) case IO_MANAGER_POLL: interruptIOManagerPoll(iomgr); @@ -831,9 +905,12 @@ bool syncIOWaitReady(CapIOManager *iomgr, return true; } #endif +#if defined(IOMGR_ENABLED_SELECTBIS) + case IO_MANAGER_SELECTBIS: + return syncIOWaitReadySelectBis(iomgr, tso, rw, fd); +#endif #if defined(IOMGR_ENABLED_POLL) case IO_MANAGER_POLL: - ASSERT(tso->why_blocked == NotBlocked); return syncIOWaitReadyPoll(iomgr, tso, rw, fd); #endif default: @@ -854,6 +931,11 @@ void syncIOCancel(CapIOManager *iomgr, StgTSO *tso) tso); break; #endif +#if defined(IOMGR_ENABLED_SELECTBIS) + case IO_MANAGER_SELECTBIS: + syncIOCancelSelectBis(iomgr, tso); + break; +#endif #if defined(IOMGR_ENABLED_POLL) case IO_MANAGER_POLL: syncIOCancelPoll(iomgr, tso); @@ -895,8 +977,13 @@ bool syncDelay(CapIOManager *iomgr, StgTSO *tso, HsInt us_delay) return true; } #endif +#if defined(IOMGR_ENABLED_SELECTBIS) || defined(IOMGR_ENABLED_POLL) +#if defined(IOMGR_ENABLED_SELECTBIS) + case IO_MANAGER_SELECTBIS: +#endif #if defined(IOMGR_ENABLED_POLL) case IO_MANAGER_POLL: +#endif return syncDelayTimeout(iomgr, tso, us_delay); #endif #if defined(IOMGR_ENABLED_WIN32_LEGACY) @@ -931,8 +1018,13 @@ void syncDelayCancel(CapIOManager *iomgr, StgTSO *tso) removeThreadFromQueue(iomgr->cap, &iomgr->sleeping_queue, tso); break; #endif +#if defined(IOMGR_ENABLED_SELECTBIS) || defined(IOMGR_ENABLED_POLL) +#if defined(IOMGR_ENABLED_SELECTBIS) + case IO_MANAGER_SELECTBIS: +#endif #if defined(IOMGR_ENABLED_POLL) case IO_MANAGER_POLL: +#endif syncDelayCancelTimeout(iomgr, tso); break; #endif ===================================== rts/IOManager.h ===================================== @@ -53,6 +53,9 @@ extern bool rts_IOManagerIsWin32Native; #if defined(IOMGR_BUILD_SELECT) && !defined(THREADED_RTS) #define IOMGR_ENABLED_SELECT #endif +#if defined(IOMGR_BUILD_SELECTBIS) && !defined(THREADED_RTS) + #define IOMGR_ENABLED_SELECTBIS +#endif #if defined(IOMGR_BUILD_POLL) && !defined(THREADED_RTS) #define IOMGR_ENABLED_POLL #endif @@ -95,6 +98,8 @@ extern bool rts_IOManagerIsWin32Native; #else // !defined(THREADED_RTS) #if defined(IOMGR_DEFAULT_NON_THREADED_SELECT) #define IOMGR_DEFAULT_STR "select" +#elif defined(IOMGR_DEFAULT_NON_THREADED_SELECTBIS) + #define IOMGR_DEFAULT_STR "selectbis" #elif defined(IOMGR_DEFAULT_NON_THREADED_POLL) #define IOMGR_DEFAULT_STR "poll" #elif defined(IOMGR_DEFAULT_NON_THREADED_WINIO) @@ -115,6 +120,11 @@ extern bool rts_IOManagerIsWin32Native; #else #define IOMGR_ENABLED_STR_SELECT "" #endif +#if defined(IOMGR_ENABLED_SELECTBIS) + #define IOMGR_ENABLED_STR_SELECTBIS " selectbis" +#else + #define IOMGR_ENABLED_STR_SELECTBIS "" +#endif #if defined(IOMGR_ENABLED_POLL) #define IOMGR_ENABLED_STR_POLL " poll" #else @@ -137,6 +147,7 @@ extern bool rts_IOManagerIsWin32Native; #endif #define IOMGRS_ENABLED_STR \ IOMGR_ENABLED_STR_SELECT \ + IOMGR_ENABLED_STR_SELECTBIS \ IOMGR_ENABLED_STR_POLL \ IOMGR_ENABLED_STR_MIO \ IOMGR_ENABLED_STR_WINIO \ @@ -150,6 +161,9 @@ typedef enum { #if defined(IOMGR_ENABLED_SELECT) IO_MANAGER_SELECT, #endif +#if defined(IOMGR_ENABLED_SELECTBIS) + IO_MANAGER_SELECTBIS, +#endif #if defined(IOMGR_ENABLED_POLL) IO_MANAGER_POLL, #endif ===================================== rts/IOManagerInternals.h ===================================== @@ -14,12 +14,19 @@ #include "IOManager.h" -#if defined(IOMGR_ENABLED_POLL) -#include <poll.h> /* for struct pollfd */ +#if defined(IOMGR_ENABLED_SELECTBIS) || defined(IOMGR_ENABLED_POLL) #include "ClosureTable.h" #include "TimeoutQueue.h" #endif +#if defined(IOMGR_ENABLED_SELECTBIS) +#include <sys/select.h> /* for fd_set */ +#endif + +#if defined(IOMGR_ENABLED_POLL) +#include <poll.h> /* for struct pollfd */ +#endif + #include "BeginPrivate.h" /* The per-capability data structures belonging to the I/O manager. @@ -46,19 +53,27 @@ struct _CapIOManager { StgTSO *sleeping_queue; #endif -#if defined(IOMGR_ENABLED_SELECT) || defined(IOMGR_ENABLED_POLL) +#if defined(IOMGR_ENABLED_SELECT) \ + || defined(IOMGR_ENABLED_SELECTBIS) \ + || defined(IOMGR_ENABLED_POLL) #if defined(HAVE_PREEMPTION) /* FDs for waking up the I/O manager when it is blocked waiting */ int interrupt_fd_r, interrupt_fd_w; #endif #endif -#if defined(IOMGR_ENABLED_POLL) +#if defined(IOMGR_ENABLED_POLL) || defined(IOMGR_ENABLED_SELECTBIS) /* AIOP and timeout collections shared by several I/O manager impls */ ClosureTable aiop_table; StgTimeoutQueue *timeout_queue; #endif +#if defined(IOMGR_ENABLED_SELECTBIS) + struct fd_table_entry { int fd; IOReadOrWrite rw; } *fd_table; + fd_set *rfds, *wfds; + int ncompletions_extra; /* extra completions for synchronous failures */ +#endif + #if defined(IOMGR_ENABLED_POLL) /* Auxiliary table with size and indexes matching the aiop_table. This is * aliased to the tail of the full poll table, which has a head entry for ===================================== rts/configure.ac ===================================== @@ -368,6 +368,15 @@ GHC_IOMANAGER_ENABLE([select], [EnableIOManagerSelect], [IOMGR_BUILD_SELECT], [AC_MSG_ERROR([sys/select.h required by select I/O manager])],[]) fi]) +GHC_IOMANAGER_ENABLE([selectbis], [EnableIOManagerSelectBis], [IOMGR_BUILD_SELECTBIS], + [if test "$HostOS" = "mingw32"; then + EnableIOManagerSelectBis=NO + else + AC_CHECK_HEADER([sys/select.h], + [EnableIOManagerSelectBis=YES], + [AC_MSG_ERROR([sys/select.h required by selectbis I/O manager])],[]) + fi]) + GHC_IOMANAGER_ENABLE([poll], [EnableIOManagerPoll], [IOMGR_BUILD_POLL], [if test "$HostOS" = "mingw32"; then EnableIOManagerPoll=NO @@ -407,6 +416,7 @@ if test "$HostOS" = "mingw32"; then else GHC_IOMANAGER_DEFAULT_SELECT([IOManagerNonThreadedDefault], [select], [EnableIOManagerSelect]) GHC_IOMANAGER_DEFAULT_SELECT([IOManagerNonThreadedDefault], [poll], [EnableIOManagerPoll]) + GHC_IOMANAGER_DEFAULT_SELECT([IOManagerNonThreadedDefault], [selectbis], [EnableIOManagerSelectBis]) GHC_IOMANAGER_DEFAULT_SELECT([IOManagerThreadedDefault], [mio], [EnableIOManagerMIO]) fi GHC_IOMANAGER_DEFAULT_CHECK_NOT_EMPTY([IOManagerNonThreadedDefault],[non-threaded]) @@ -419,6 +429,9 @@ dnl Now define CPP vars for the default ones (threaded and non-threaded) GHC_IOMANAGER_DEFAULT_AC_DEFINE([IOManagerNonThreadedDefault], [non-threaded], [select], [IOMGR_DEFAULT_NON_THREADED_SELECT]) +GHC_IOMANAGER_DEFAULT_AC_DEFINE([IOManagerNonThreadedDefault], [non-threaded], + [selectbis], [IOMGR_DEFAULT_NON_THREADED_SELECTBIS]) + GHC_IOMANAGER_DEFAULT_AC_DEFINE([IOManagerNonThreadedDefault], [non-threaded], [poll], [IOMGR_DEFAULT_NON_THREADED_POLL]) ===================================== rts/include/rts/Flags.h ===================================== @@ -258,6 +258,7 @@ typedef enum _IO_MANAGER_FLAG { /* All other choices pick only the requested one, with no fallback. */ IO_MNGR_FLAG_SELECT, /* Unix only, non-threaded RTS only */ + IO_MNGR_FLAG_SELECTBIS, /* Unix only, non-threaded RTS only */ IO_MNGR_FLAG_POLL, /* Unix only, non-threaded RTS only */ IO_MNGR_FLAG_MIO, /* cross-platform, threaded RTS only */ IO_MNGR_FLAG_WINIO, /* Windows only */ ===================================== rts/posix/Poll.c ===================================== @@ -133,7 +133,7 @@ the aiop_table, but still allows the full_poll_table to have an extra entry. /* Forward declarations */ static bool enlargeTables(CapIOManager *iomgr); static void notifyIOCompletion(CapIOManager *iomgr, StgAsyncIOOp *aiop); -static void ioCancel(CapIOManager *iomgr, StgAsyncIOOp *aiop); +static void removeFromTables(CapIOManager *iomgr, int i); static void reportPollError(int res, nfds_t nfds) STG_NORETURN; @@ -224,7 +224,8 @@ void syncIOCancelPoll(CapIOManager *iomgr, StgTSO *tso) StgAsyncIOOp *aiop = tso->block_info.aiop; ASSERT(aiop->notify_type == NotifyTSO); ASSERT(indexClosureTable(&iomgr->aiop_table, aiop->index) == aiop); - ioCancel(iomgr, aiop); + removeFromTables(iomgr, aiop->index); + aiop->outcome = IOOpOutcomeCancelled; /* We cannot use the normal notifyIOCompletion here. We are in the context * of throwTo, interrupting a thread blocked on IO via an async exception. * We don't put the TSO back on the run queue or change the why_blocked @@ -250,27 +251,13 @@ void asyncIOCancelPoll(CapIOManager *iomgr, StgAsyncIOOp *aiop) */ ASSERT(aiop->notify_type != NotifyTSO); if (indexClosureTable(&iomgr->aiop_table, aiop->index) == aiop) { - ioCancel(iomgr, aiop); + removeFromTables(iomgr, aiop->index); + aiop->outcome = IOOpOutcomeCancelled; notifyIOCompletion(iomgr, aiop); } } -static void ioCancel(CapIOManager *iomgr, StgAsyncIOOp *aiop) -{ - int ix = aiop->index; - int ix_from; int ix_to; - removeCompactClosureTable(iomgr->cap, &iomgr->aiop_table, ix, - &ix_from, &ix_to); - if (ix_to != ix_from) { - StgAsyncIOOp *aiop_to = indexClosureTable(&iomgr->aiop_table, ix_to); - aiop_to->index = ix_to; - iomgr->aiop_poll_table[ix_to] = iomgr->aiop_poll_table[ix_from]; - } - aiop->outcome = IOOpOutcomeCancelled; -} - - bool anyPendingTimeoutsOrIOPoll(CapIOManager *iomgr) { return !isEmptyTimeoutQueue(iomgr->timeout_queue) @@ -284,11 +271,16 @@ static void notifyIOCompletion(CapIOManager *iomgr, StgAsyncIOOp *aiop) switch (aiop->notify_type) { case NotifyTSO: { + /* We should be guaranteed that the tso is still on the same + * cap because the tso was not on the run queue of any cap and + * so is not subject to thread migration. + */ + StgTSO *tso = aiop->notify.tso; + ASSERT(tso->cap == iomgr->cap); if (aiop->outcome == IOOpOutcomeFailed && aiop->error == EBADF) { /* The fd is invalid: raise an IOError exception in the blocked * thread. (See bug #4934 for what happens without this.) */ - StgTSO *tso = aiop->notify.tso; debugTrace(DEBUG_iomanager, "Raising exception in thread %" FMT_StgThreadID " blocked on an invalid fd", tso->id); @@ -296,11 +288,6 @@ static void notifyIOCompletion(CapIOManager *iomgr, StgAsyncIOOp *aiop) (StgClosure *)blockedOnBadFD_closure, false, NULL); } else { - /* We should be guaranteed that the tso is still on the same - * cap because the tso was not on the run queue of any cap and - * so is not subject to thread migration. - */ - StgTSO *tso = aiop->notify.tso; tso->why_blocked = NotBlocked; tso->_link = END_TSO_QUEUE; pushOnRunQueue(iomgr->cap, tso); @@ -375,19 +362,7 @@ static bool processIOCompletions(CapIOManager *iomgr, int ncompletions) aiop->result = 0; } - /* Remove from the completion table, preserving compactness, and - * apply the same compacting to the aiop_poll_table. - */ - int ix_from; int ix_to; - removeCompactClosureTable(iomgr->cap, &iomgr->aiop_table, i, - &ix_from, &ix_to); - if (ix_to != ix_from) { - StgAsyncIOOp *aiop_to; - aiop_to = indexClosureTable(&iomgr->aiop_table, ix_to); - aiop_to->index = ix_to; - aiop_poll_table[ix_to] = aiop_poll_table[ix_from]; - } - + removeFromTables(iomgr, i); notifyIOCompletion(iomgr, aiop); n--; } else { @@ -458,6 +433,10 @@ void pollCompletedTimeoutsOrIOPoll(CapIOManager *iomgr) reportPollError(res, nfds); } } + +#if defined(RTS_USER_SIGNALS) + startPendingSignalHandlers(iomgr->cap); +#endif } @@ -546,14 +525,13 @@ bool awaitCompletedTimeoutsOrIOPoll(CapIOManager *iomgr) // on and so we sould check for timeouts. } else if (errno == EINTR) { - /* We got interrupted by a signal. In the non-threaded RTS, if the - * signal is one of ours we need to return to the scheduler to let - * it handle it. Otherwise we would loop and keep waiting for I/O - * or timeouts, meaning we would block for a long time before the - * signal is serviced. - */ + /* We got interrupted by a signal. */ + #if defined(RTS_USER_SIGNALS) - if (startPendingSignalHandlers(iomgr->cap)) break; + /* Start any corresponding user signal handlers. If any, the run + * queue will become non-empty and we will drop out of the loop. + */ + startPendingSignalHandlers(iomgr->cap); #endif /* We can also be interrupted by the shutdown signal handler, which @@ -628,4 +606,20 @@ static bool enlargeTables(CapIOManager *iomgr) return true; } + +/* Remove from the completion table, preserving compactness, and apply the same + * compacting to the aiop_poll_table. + */ +static void removeFromTables(CapIOManager *iomgr, int ix) +{ + int ix_from; int ix_to; + removeCompactClosureTable(iomgr->cap, &iomgr->aiop_table, ix, + &ix_from, &ix_to); + if (ix_to != ix_from) { + StgAsyncIOOp *aiop_to = indexClosureTable(&iomgr->aiop_table, ix_to); + aiop_to->index = ix_to; + iomgr->aiop_poll_table[ix_to] = iomgr->aiop_poll_table[ix_from]; + } +} + #endif /* IOMGR_ENABLED_POLL */ ===================================== rts/posix/SelectBis.c ===================================== @@ -0,0 +1,716 @@ +/* ----------------------------------------------------------------------------- + * + * (c) The GHC Team 2020-2026 + * + * A second I/O manager based on the classic Unix select() system call. + * + * See SelectBis.h for the sad story of why this exists. + * + * ---------------------------------------------------------------------------*/ + +#include "rts/PosixSource.h" +#include "Rts.h" +#include "RtsFlags.h" // needed by SET_HDR macro + +#include "IOManager.h" // defines IOMGR_ENABLED_SELECTBIS + +#if defined(IOMGR_ENABLED_SELECTBIS) + +#include "Capability.h" +#include "Threads.h" +#include "Schedule.h" +#include "Prelude.h" +#include "RtsUtils.h" +#include "rts/Time.h" +#include "RaiseAsync.h" +#include "Trace.h" + +#include "SelectBis.h" +#include "RtsSignals.h" + +#include <sys/select.h> +#include <errno.h> + +#include "IOManagerInternals.h" +#include "Timeout.h" +#include "FdWakeup.h" + +/****************************************************************************** + +This I/O manager is based on the classic Unix select() system call. + + int select(int nfds, fd_set *readfds, fd_set *writefds, + fd_set *exceptfds, struct timeval *timeout); + +The select() call has various limits, quirks and slight differences between +historical Unix variants. + +The basic idea is to collect a set of fds (represented as a bitset) that we are +interested in: one for reads, one for writes. The call then queries for I/O +readiness on all the fds in the read and write sets. The result is a set of fds +that are ready to read from, and a set that are ready to write to. The same +bitset representation is used for the output. Indeed a "fun" quirk of select() +is that it mutates the fd sets it is passed, which means they either need to be +built up each time, or copied. There is also an optional timeout if no fds are +ready immediately. There is also an fd bitset for "exceptional conditions" +which we do not use. + +There is of course no incremental behaviour here; this is a bulk one-off call +with no persistent state. This has obvious scaling problems. The cost each time +is O(n) in the maximum of the integer value of the fds of interest. There is +also a maximum bitset size. On Linux this is 1024. This means select() cannot +be used if the process uses more than that many open files, even if we're only +interested in a few. On OSX the default limit is also 1024 but this can be +raised or even managed dynamically, at the cost of more memory (and some +non-standard code). + +That particular problem is solved by the later Unix poll() system call, which +uses an array of the fds we are interested in, which means it not limited by +the absolute value of the fds numbers (but it is still O(n) in how many fds we +are interested in). + +We have some choice in how we process results. We want to find the intersection +between the requests for notification of I/O readiness (coming from the Haskell +threads) and the read and write bit sets. There's not much clever we can do to +compute this intersection efficiently: we can either iterate over the bit sets +or over the readiness requests. There is no obvious answer here. Typically +there will be few results compared to the number of requests and a bitset scan +could be fast. In practice we cannot portably scan the bitset efficiently (e.g. +word at a time). Portably, we can only probe each bit at a time using FD_ISSET. +Portability is the main reason to use select() rather than a more modern +interface, so we have to take it seriously here. Furthermore, if we iterated +over the bit sets we would have to maintain a mapping from fd to requests. + +In principle we also have the choice to maintain the read and write fd bit sets +incrementally, or create them afresh each time we call select(). There is no +asymptotic bonus to maintaining them incrementally since the whole thing is +O(n) anyway. There could plausibly be some constant factor benefit. To maintain +the fd bit sets incrementally we would need to maintain a mapping between +requests and fds. This would also be an extra cost that would have to be +outweighed by any saving. + +In the end we take the simple approach to constructing the bitset inputs and to +results processing. We create the bit sets afresh each time from the collection +of requests. For processing results we iterate over the requests and look up +each one to see if it is in the appropriate result bitset. Along with each +operation, we store the fd and whether we were interested in reading or writing. +We iterate over the operations and use the fd and r/w information to construct +the read and write bit sets. + +A particularly frustrating feature of select() is that if any single fd in any +fd bitset is invalid (e.g. because the file was already closed) then select() +fails and tells us there is a bad fd somewhere, but it has no way to indicate +which fd was bad. This is really quite annoying as we then have to do a search +through the fds to find which one was bad. + +The primary data structure for this I/O manager is a aiop_table which is a +ClosureTable of AsyncIOOps. This table tracks the active I/O operations, with +one entry per operation (corresponding to threads calling waitRead#/waitWrite#). +We also track the fd for each operation and whether the operation is waiting on +read or write readiness. This additional information is stored in the fd_table. +The fd_table is maintained as an auxiliary table to the aiop_table, with table +indexes matching the ClosureTable. So there is an entry in the aiop_table for +each operation, and a corresponding entry in the fd_table at the same table +index. The aiop_table and the fd_table are maintained incrementally, and with +dense indexes. + +We also use a StgTimeoutQueue to track timeouts, and use the delay to the next +timeout (if any) as the poll() timeout parameter. + +The CapIOManager structure for this I/O manager contains: + + ClosureTable aiop_table; + struct fd_table_entry { int fd; IOReadOrWrite rw } *fd_table; + StgTimeoutQueue *timeout_queue; + int interrupt_fd_r, interrupt_fd_w; + +******************************************************************************/ + +/* Forward declarations */ +static bool enlargeTables(CapIOManager *iomgr); +static void removeFromTables(CapIOManager *iomgr, int i); +static void notifyIOCompletion(CapIOManager *iomgr, StgAsyncIOOp *aiop); +static void reportSelectError(void) STG_NORETURN; +static bool checkFdRange(int fd); +static int collectFdSets(CapIOManager *iomgr); +static void processBadFds(CapIOManager *iomgr); + + +void initCapabilityIOManagerSelectBis(CapIOManager *iomgr) +{ + initClosureTable(&iomgr->aiop_table, ClosureTableCompact); + iomgr->timeout_queue = emptyTimeoutQueue(); + +#if defined(HAVE_PREEMPTION) + newFdWakeup(&iomgr->interrupt_fd_r, &iomgr->interrupt_fd_w); + + /* Would never happen in a standalone process, but could plausibly happen + * if the RTS is used within another process that already has many open fds. + */ + if (iomgr->interrupt_fd_r < 0 || iomgr->interrupt_fd_r >= (int)FD_SETSIZE || + iomgr->interrupt_fd_w < 0 || iomgr->interrupt_fd_w >= (int)FD_SETSIZE) { + barf("initCapabilityIOManagerSelectBis: fds out of select range"); + } +#endif + + iomgr->fd_table = NULL; + iomgr->rfds = stgMallocBytes(sizeof (fd_set), "IOManagerSelectBis"); + iomgr->wfds = stgMallocBytes(sizeof (fd_set), "IOManagerSelectBis"); + iomgr->ncompletions_extra = 0; +} + + +void freeCapabilityIOManagerSelectBis(CapIOManager *iomgr) +{ + if (iomgr->fd_table) stgFree(iomgr->fd_table); + stgFree(iomgr->rfds); + stgFree(iomgr->wfds); +#if defined(HAVE_PREEMPTION) + closeFdWakeup(iomgr->interrupt_fd_r, iomgr->interrupt_fd_w); +#endif +} + + +/* Result is true on success, or false on allocation failure. */ +bool syncIOWaitReadySelectBis(CapIOManager *iomgr, StgTSO *tso, + IOReadOrWrite rw, HsInt fd) +{ + StgAsyncIOOp *aiop; + aiop = (StgAsyncIOOp *)allocateMightFail(iomgr->cap, sizeofW(StgAsyncIOOp)); + if (RTS_UNLIKELY(aiop == NULL)) return false; + SET_HDR(aiop, &stg_ASYNCIOOP_info, iomgr->cap->r.rCCCS); + aiop->notify.tso = tso; + aiop->notify_type = NotifyTSO; + aiop->live = &stg_ASYNCIO_LIVE0_closure; + tso->why_blocked = rw == IORead ? BlockedOnRead : BlockedOnWrite; + tso->block_info.aiop = aiop; + return asyncIOWaitReadySelectBis(iomgr, aiop, rw, fd); +} + +/* Result is true on success, or false on allocation failure. */ +bool asyncIOWaitReadySelectBis(CapIOManager *iomgr, StgAsyncIOOp *aiop, + IOReadOrWrite rw, int fd) +{ + if (RTS_UNLIKELY(isFullClosureTable(&iomgr->aiop_table))) { + bool ok = enlargeTables(iomgr); + if (RTS_UNLIKELY(!ok)) return false; + } + + int ix = insertClosureTable(iomgr->cap, &iomgr->aiop_table, aiop); + + /* We use the aiop_table and fd_table densely. */ + ASSERT(ix == sizeClosureTable(&iomgr->aiop_table) - 1); + + /* The syncIO wrapper or CMM primop filled in the notify and live fields, + * we fill the rest. + */ + aiop->capno = iomgr->cap->no; + aiop->index = ix; + aiop->outcome = IOOpOutcomeInFlight; + + /* Fill in the corresponding entry in the fd_table */ + iomgr->fd_table[ix] = (struct fd_table_entry) { + .fd = fd, + .rw = rw + }; + + if (!checkFdRange(fd)) { + /* We have a synchronous failure, but the primop is not set up to report + * exceptions. We cannot report async exceptions to the caller here + * since the thread stack is not in the right state (so we cannot use + * notifyIOCompletion). So instead we mark the aiop as failed now, but + * we report the failure later when we poll for completed I/O. + */ + aiop->outcome = IOOpOutcomeFailed; + aiop->error = EBADF; + /* completions for synchronous failures to report asynchronously */ + iomgr->ncompletions_extra++; + }; + + return true; +} + + +void syncIOCancelSelectBis(CapIOManager *iomgr, StgTSO *tso) +{ + StgAsyncIOOp *aiop = tso->block_info.aiop; + ASSERT(aiop->notify_type == NotifyTSO); + ASSERT(indexClosureTable(&iomgr->aiop_table, aiop->index) == aiop); + removeFromTables(iomgr, aiop->index); + aiop->outcome = IOOpOutcomeCancelled; + /* We cannot use the normal notifyIOCompletion here. We are in the context + * of throwTo, interrupting a thread blocked on IO via an async exception. + * We don't put the TSO back on the run queue or change the why_blocked + * status, as that is done by removeFromQueues (in the throwTo* functions). + */ + tso->block_info.closure = (StgClosure *)END_TSO_QUEUE; + + /* We are in the TSO case, where the aiop was only reachable from the TSO + * itself, and thus it is now no longer be reachable at all. + */ + IF_NONMOVING_WRITE_BARRIER_ENABLED { + updateRemembSetPushClosure(iomgr->cap, (StgClosure *)aiop); + } +} + + +void asyncIOCancelSelectBis(CapIOManager *iomgr, StgAsyncIOOp *aiop) +{ + /* We can reliably determine if the aiop is still in progress by checking + * if the aiop_table still points to this aiop object. This is reliable + * because each aiop is GC heap allocated, so cannot be recycled until it + * is no longer retained by the application. + */ + ASSERT(aiop->notify_type != NotifyTSO); + if (indexClosureTable(&iomgr->aiop_table, aiop->index) == aiop) { + removeFromTables(iomgr, aiop->index); + aiop->outcome = IOOpOutcomeCancelled; + notifyIOCompletion(iomgr, aiop); + } +} + + +bool anyPendingTimeoutsOrIOSelectBis(CapIOManager *iomgr) +{ + return !isEmptyTimeoutQueue(iomgr->timeout_queue) + || !isEmptyClosureTable(&iomgr->aiop_table); +} + + +static void notifyIOCompletion(CapIOManager *iomgr, StgAsyncIOOp *aiop) +{ + ASSERT(aiop->outcome != IOOpOutcomeInFlight); + switch (aiop->notify_type) { + case NotifyTSO: + { + /* We should be guaranteed that the tso is still on the same + * cap because the tso was not on the run queue of any cap and + * so is not subject to thread migration. + */ + StgTSO *tso = aiop->notify.tso; + ASSERT(tso->cap == iomgr->cap); + if (aiop->outcome == IOOpOutcomeFailed && aiop->error == EBADF) { + /* The fd is invalid: raise an IOError exception in the blocked + * thread. (See bug #4934 for what happens without this.) + */ + debugTrace(DEBUG_iomanager, + "Raising exception in thread %" FMT_StgThreadID + " blocked on an invalid fd", tso->id); + raiseAsync(iomgr->cap, tso, + (StgClosure *)blockedOnBadFD_closure, + false, NULL); + } else { + tso->why_blocked = NotBlocked; + tso->_link = END_TSO_QUEUE; + pushOnRunQueue(iomgr->cap, tso); + } + /* For the TSO case, the aiop was only reachable from the TSO + * itself, and thus it is now no longer be reachable at all. + */ + IF_NONMOVING_WRITE_BARRIER_ENABLED { + updateRemembSetPushClosure(iomgr->cap, (StgClosure *)aiop); + } + break; + } + case NotifyMVar: + barf("selectbis iomgr: MVar notification not yet supported"); + break; + + case NotifyTVar: + barf("selectbis iomgr: TVar notification not yet supported"); + break; + } +} + + +static bool processIOCompletions(CapIOManager *iomgr, int ncompletions) +{ + /* We want to find the intersection between the sets of ready fds returned + * by select() and the aiop_table. Given how select() represents + * things there's no particularly efficient way to do it. + * + * We just go through the whole aiop_table and look up each one in + * the read or write fd_set to see if they completed. Note that here is + * where we rely on the aiop_table being dense so we can iterate + * over the entries. We can short-cut if we hit the ncompletions before + * getting to the end of the table. + */ + debugTrace(DEBUG_iomanager, "processIOCompletions(ncompletions = %d)", + ncompletions); + + bool interrupt = false; +#if defined(HAVE_PREEMPTION) + /* If the interrupt_fd_r is ready, collect it */ + if (FD_ISSET(iomgr->interrupt_fd_r, iomgr->rfds)) { + ASSERT(iomgr->full_poll_table[0].fd == iomgr->interrupt_fd_r); + collectFdWakeup(iomgr->interrupt_fd_r); + ncompletions--; + interrupt = true; + debugTrace(DEBUG_iomanager, "Received interrupt in poll I/O manager"); + } +#endif + + struct fd_table_entry *fd_table = iomgr->fd_table; + int n = ncompletions; + int i = 0; + while (n > 0) { + ASSERT(i < sizeClosureTable(&iomgr->aiop_table)); + + StgAsyncIOOp *aiop = indexClosureTable(&iomgr->aiop_table, i); + int fd = fd_table[i].fd; + IOReadOrWrite rw = fd_table[i].rw; + + if (RTS_UNLIKELY(aiop->outcome == IOOpOutcomeFailed)) { + /* The synchronous failure case, see ncompletions_extra. */ + } else if (rw == IORead ? FD_ISSET(fd, iomgr->rfds) + : FD_ISSET(fd, iomgr->wfds)) { + aiop->outcome = IOOpOutcomeSuccess; + aiop->result = 0; + } else { + /* You'd expect incrementing the table index to be unconditional, + * but we don't increment the index if we did process the entry, + * because using removeFromTables means we'll move an entry from + * the end of the table into the index i. + */ + i++; + continue; /* skip the steps below */ + } + removeFromTables(iomgr, i); + notifyIOCompletion(iomgr, aiop); + n--; + } + return interrupt; +} + + +void pollCompletedTimeoutsOrIOSelectBis(CapIOManager *iomgr) +{ + if (!isEmptyTimeoutQueue(iomgr->timeout_queue)) { + Time now = getProcessElapsedTime(); + processTimeoutCompletions(iomgr, now); + } + + if (!isEmptyClosureTable(&iomgr->aiop_table)) { + /* Prepare to poll for I/O readiness: collect all of the fd's that + * we're interested in. + */ + int maxfd = collectFdSets(iomgr); + + /* Poll for I/O readiness, without waiting. */ + struct timeval tv = (struct timeval) { .tv_sec = 0, .tv_usec = 0 }; + int res = select(maxfd+1, iomgr->rfds, iomgr->wfds, NULL, &tv); + if (res == 0 && iomgr->ncompletions_extra == 0) { + /* There is no I/O ready. We'll return to the scheduler. */ + + } else if (res > 0 || iomgr->ncompletions_extra > 0) { + /* Extra completions for synchronous failures to report */ + int ncompletions = res + iomgr->ncompletions_extra; + iomgr->ncompletions_extra = 0; + + ASSERT(ncompletions <= sizeClosureTable(&iomgr->aiop_table)); + processIOCompletions(iomgr, ncompletions); + + } else if (errno == EBADF) { + processBadFds(iomgr); + + } else if (errno == EINTR) { + /* We got interrupted by a signal. This is unlikely since we asked + * select() not to wait, but if so we'll return to the scheduler. + */ + + } else { + reportSelectError(); + } + } + +#if defined(RTS_USER_SIGNALS) + startPendingSignalHandlers(iomgr->cap); +#endif +} + + +bool awaitCompletedTimeoutsOrIOSelectBis(CapIOManager *iomgr) +{ + bool interrupt = false; /* got woken up via interruptIOManager */ + + /* Loop until we've woken up some threads. This loop is needed because the + * select() timing isn't accurate, we sometimes sleep for a while but not + * long enough to wake up a thread in a threadDelay. Or we may need to + * sleep multiple times if we need to sleep longer than the maximum timeout + * that select() supports. + */ + do { + /* We do /not/ require that there be pending I/O or pending timers. + * If there is neither, it's because the scheduler wants us to wait + * on signals only. + */ + + Time now = getProcessElapsedTime(); + processTimeoutCompletions(iomgr, now); + + /* If we didn't wake any threads due to expiring timeouts, then we need + * to wait on I/O. Or to put it another way, even if we did wake some + * threads, we'll still poll (but not wait) for I/O. This is to ensure + * we avoid starving threads blocked on I/O. + */ + bool wait = emptyRunQueue(iomgr->cap); + + /* If we have failures to report, we must not block. */ + if (iomgr->ncompletions_extra > 0) { + wait = false; + } + + /* Prepare to poll for I/O readiness: collect all of the fd's that + * we're interested in. + */ + int maxfd = collectFdSets(iomgr); + + /* Decide if we are going to wait if no I/O is ready, either: + * poll only, wait indefinitely, or wait until a timeout. + */ + struct timeval tv, *timeout_us; + timeout_us = timeoutInMicroseconds(iomgr, wait, now, &tv); + + /* Check for I/O readiness, possibly waiting. */ + int res = select(maxfd+1, iomgr->rfds, iomgr->wfds, NULL, timeout_us); + + if (res == 0 && iomgr->ncompletions_extra == 0) { + /* Success but there is no I/O ready. This can happen either if we + * were not blocking or were in a timed wait and the timeout + * occurred before any I/O became ready. Either way, the do-while + * loop condition will handle it. + */ + ASSERT(timeout_us != NULL); + + } else if (res > 0 || iomgr->ncompletions_extra > 0) { + /* Extra completions for synchronous failures to report */ + int ncompletions = res + iomgr->ncompletions_extra; + iomgr->ncompletions_extra = 0; + + ASSERT(ncompletions <= sizeClosureTable(&iomgr->aiop_table)); + interrupt = processIOCompletions(iomgr, ncompletions); + // FIXME: do we also need to check for timeout completions now? + // we have a non-empty queue, but if !wait then we have also moved + // on and so we sould check for timeouts. + + } else if (errno == EINTR) { + /* We got interrupted by a signal. */ + +#if defined(RTS_USER_SIGNALS) + /* Start any corresponding user signal handlers. If any, the run + * queue will become non-empty and we will drop out of the loop. + */ + startPendingSignalHandlers(iomgr->cap); +#endif + + /* We can also be interrupted by the shutdown signal handler, which + * will set sched_state and so cause us to drop out of the loop. + * + * For any other interruption (e.g. timer) we will go round the + * do-while loop again. + */ + + } else if (errno == EBADF) { + processBadFds(iomgr); + + } else { + reportSelectError(); + } + + } while (emptyRunQueue(iomgr->cap) + && !interrupt + && (getSchedState() == SCHED_RUNNING)); + return !interrupt; +} + + +static void reportSelectError() +{ + sysErrorBelch("select() failed"); + stg_exit(EXIT_FAILURE); +} + + +static void processBadFds(CapIOManager *iomgr) +{ + /* This is extremely tiresome. The select() call fails with EBADF if any + * fd is invalid (usually closed), but it does not tell us which one. + * So we have to loop through them to find the offending fd. + * + * This will only find the first bad fd, so the caller must cope with + * there still being bad fds after this. + */ + + fd_set rfds, wfds; + FD_ZERO(&rfds); + FD_ZERO(&wfds); + + struct fd_table_entry *fd_table = iomgr->fd_table; + int nentries = sizeClosureTable(&iomgr->aiop_table); + for (int n = 0; n < nentries; n++) { + int fd = fd_table[n].fd; + IOReadOrWrite rw = fd_table[n].rw; + + struct timeval tv = { .tv_sec = 0, .tv_usec = 0 }; + int res; + if (rw == IORead) { + FD_SET(fd, &rfds); + res = select(fd+1, &rfds, NULL, NULL, &tv); + FD_CLR(fd, &rfds); + } else { + FD_SET(fd, &wfds); + res = select(fd+1, NULL, &wfds, NULL, &tv); + FD_CLR(fd, &wfds); + } + if (res == 0) { + continue; + + } else if (errno == EBADF) { + StgAsyncIOOp *aiop = indexClosureTable(&iomgr->aiop_table, n); + aiop->outcome = IOOpOutcomeFailed; + aiop->error = EBADF; + removeFromTables(iomgr, n); + notifyIOCompletion(iomgr, aiop); + /* There is /probably/ only one bad fd at once, so we abort the + * search here. If we are unlucky and there are several bad fds + * then the caller will just loop round again. + */ + + return; + + } else if (errno == EINTR) { + /* Unlikely, since we did a non-blocking select(). Try again. */ + n--; + continue; + + } else { + reportSelectError(); + } + } +} + + +void interruptIOManagerSelectBis(CapIOManager *iomgr) +{ +#if defined(HAVE_PREEMPTION) + sendFdWakeup(iomgr->interrupt_fd_w); +#endif +} + + +/* Helper function to double the size of the aiop_table and fd_table. + */ +static bool enlargeTables(CapIOManager *iomgr) +{ + int oldcapacity = capacityClosureTable(&iomgr->aiop_table); + int newcapacity = (oldcapacity == 0) ? 1 : (oldcapacity * 2); + + bool ok = enlargeClosureTable(iomgr->cap, &iomgr->aiop_table, newcapacity); + if (RTS_UNLIKELY(!ok)) return false; + + /* Update the auxiliary fd_table to match */ + iomgr->fd_table = + stgReallocBytes(iomgr->fd_table, + sizeof(struct fd_table_entry) * newcapacity, + "SelectBis.c: enlargeTables"); + + /* Initialise the new part of the fd_table */ + struct fd_table_entry *fd_table = iomgr->fd_table; + for (int i = oldcapacity; i < newcapacity; i++) { + fd_table[i] = (struct fd_table_entry) { + .fd = -1, + .rw = 0 + }; + } + return true; +} + + +/* Remove from the completion table, preserving compactness, and apply the same + * compacting to the fd_table. + */ +static void removeFromTables(CapIOManager *iomgr, int ix) +{ + int ix_from; int ix_to; + removeCompactClosureTable(iomgr->cap, &iomgr->aiop_table, ix, + &ix_from, &ix_to); + if (ix_to != ix_from) { + StgAsyncIOOp *aiop_to = indexClosureTable(&iomgr->aiop_table, ix_to); + aiop_to->index = ix_to; + iomgr->fd_table[ix_to] = iomgr->fd_table[ix_from]; + iomgr->fd_table[ix_from] = (struct fd_table_entry) { + .fd = -1, + .rw = 0 + }; + } +} + + +static int collectFdSets(CapIOManager *iomgr) +{ + int maxfd = -1; + int nentries = sizeClosureTable(&iomgr->aiop_table); + struct fd_table_entry *fd_table = iomgr->fd_table; + + /* In principle we could optimise this slightly by not resetting the + * whole of each fdset, by assuming that select() does not modify + * entries above maxfd. This is probably not worth doing however, since + * this I/O manager is supposed to be portable and is expected to be slow. + */ + FD_ZERO(iomgr->rfds); + FD_ZERO(iomgr->wfds); + +#if defined(HAVE_PREEMPTION) + /* We're always interested in our interrupt fd */ + { + int fd = iomgr->interrupt_fd_r; + maxfd = (fd > maxfd) ? fd : maxfd; + FD_SET(fd, iomgr->rfds); + } +#endif + + for (int i = 0; i < nentries; i++) { + int fd = fd_table[i].fd; + IOReadOrWrite rw = fd_table[i].rw; + ASSERT(fd != -1); // uninitialised + + // Skip aiops that we already know are failed + StgAsyncIOOp *aiop = indexClosureTable(&iomgr->aiop_table, i); + if (RTS_UNLIKELY(aiop->outcome == IOOpOutcomeFailed)) continue; + + if (rw == IORead) { + FD_SET(fd, iomgr->rfds); + } else { + FD_SET(fd, iomgr->wfds); + } + maxfd = (fd > maxfd) ? fd : maxfd; + } + return maxfd; +} + + +/* Helper function to check if the fd is out of range for select(). + */ +static bool checkFdRange(int fd) +{ + /* On older FreeBSDs, FD_SETSIZE is unsigned. Cast it to signed int + * in order to switch off the 'comparison between signed and + * unsigned error message + * Newer versions of FreeBSD have switched to unsigned int: + * https://github.com/freebsd/freebsd/commit/12ae7f74a071f0439763986026525094a7... + * http://fa.freebsd.cvs-all.narkive.com/bCWNHbaC/svn-commit-r265051-head-sys-s... + * So the (int) cast should be removed across the code base once + * GHC requires a version of FreeBSD that has that change in it. + */ + return ((fd >= 0) && (fd < (int)FD_SETSIZE)); + /* TODO: on several platforms, it is possible to use a larger fd set size. + For example on OSX: + https://code.saghul.net/2016/05/libuv-internals-the-osx-select2-trick/ + And probably similar on other platforms. It basically amounts to looking + through the representation abstraction of fd_set and to know that it is + indeed a bit set, and then we can simply allocate it and manipulte it + ourselves. We could do this, dynamically (re-)allocate the size. + */ +} + +#endif /* IOMGR_ENABLED_SELECTBIS */ ===================================== rts/posix/SelectBis.h ===================================== @@ -0,0 +1,61 @@ +/* ----------------------------------------------------------------------------- + * + * (c) The GHC Team 2020-2026 + * + * A second I/O manager based on the classic Unix select() system call. + * + * This I/O manager is called "selectbis", because it is the second such I/O + * manager based on select(). The historic implementation is named "select" + * and lives in Select.{c,h}. This I/O manager exists for the benefit of users + * of Apple products. + * + * The poll I/O manger _should_ be the portable baseline posix I/O manager. + * Unfortunately Mac OSX has a buggy implementation of poll(). The OSX man + * page documents this as: + * + * > BUGS The poll() system call currently does not support devices. + * + * This is quite incredible, given that poll and select should be relatively + * thin interfaces to the the same underlying kernel infrastructure. + * Furthermore, OSX is supposedly certified as POSIX compliant! Due to this + * (incompetence) we need a new I/O manager implementation based on the + * antique select() API, with all of its known limitations. + * + * Please direct all complaints to: + * Apple Inc., One Apple Park Way, Cupertino, CA 95014, USA. + * + * Prototypes for functions in SelectBis.c + * + * -------------------------------------------------------------------------*/ + +#pragma once + +#include "IOManager.h" + +#include "BeginPrivate.h" + +#if defined(IOMGR_ENABLED_SELECTBIS) + +void initCapabilityIOManagerSelectBis(CapIOManager *iomgr); +void freeCapabilityIOManagerSelectBis(CapIOManager *iomgr); + +/* Synchronous I/O and timer operations */ +bool syncIOWaitReadySelectBis(CapIOManager *iomgr, StgTSO *tso, + IOReadOrWrite rw, HsInt fd); +void syncIOCancelSelectBis(CapIOManager *iomgr, StgTSO *tso); + +/* Asynchronous operations */ +bool asyncIOWaitReadySelectBis(CapIOManager *iomgr, StgAsyncIOOp *aiop, + IOReadOrWrite rw, int fd); +void asyncIOCancelSelectBis(CapIOManager *iomgr, StgAsyncIOOp *aiop); + +/* Scheduler operations */ +bool anyPendingTimeoutsOrIOSelectBis(CapIOManager *iomgr); +void pollCompletedTimeoutsOrIOSelectBis(CapIOManager *iomgr); +bool awaitCompletedTimeoutsOrIOSelectBis(CapIOManager *iomgr); +void interruptIOManagerSelectBis(CapIOManager *iomgr); + +#endif /* IOMGR_ENABLED_SELECTBIS */ + +#include "EndPrivate.h" + ===================================== rts/posix/Timeout.c ===================================== @@ -14,8 +14,9 @@ #include "Schedule.h" #include "Prelude.h" -#include "Timeout.h" +#include "IOManager.h" #include "IOManagerInternals.h" +#include "Timeout.h" #include "TimeoutQueue.h" #include <limits.h> @@ -24,7 +25,7 @@ /* Currently only used by the poll I/O manager, but in future may be used by several in-RTS I/O managers. */ -#if defined(IOMGR_ENABLED_POLL) +#if defined(IOMGR_ENABLED_POLL) || defined(IOMGR_ENABLED_SELECTBIS) bool syncDelayTimeout(CapIOManager *iomgr, StgTSO *tso, HsInt us_delay) { @@ -223,5 +224,58 @@ struct timespec *timeoutInNanoseconds(CapIOManager *iomgr, bool wait, } #endif -#endif // defined(IOMGR_ENABLED_POLL) +/* select() expect a timeout in microseconds, using struct timeval * with + * special values of NULL for indefinite wait, and 0 for no waiting. + */ +#if defined(IOMGR_ENABLED_SELECTBIS) +struct timeval *timeoutInMicroseconds(CapIOManager *iomgr, bool wait, + Time now, struct timeval *tv) +{ + if (!wait) { + /* Don't wait, just poll. */ + *tv = (struct timeval) { .tv_sec = 0, .tv_usec = 0 }; + return tv; + + } else if (!isEmptyTimeoutQueue(iomgr->timeout_queue)) { + /* SUSv2 allows implementations to have an implementation defined + * maximum timeout for select(2). The standard requires + * implementations to silently truncate values exceeding this maximum + * to the maximum. Unfortunately, OSX and the BSD don't comply with + * SUSv2, instead opting to return EINVAL for values exceeding a + * timeout of 1e8. + * + * Select returning an error crashes the runtime in a bad way. To + * play it safe we truncate any timeout to 31 days, as SUSv2 requires + * any implementations maximum timeout to be larger than this. + * + * Truncating the timeout is not an issue, because if nothing + * interesting happens when the timeout expires, we'll see that the + * thread still wants to be blocked longer and simply block on a new + * iteration of select(2). + */ + const time_t max_seconds = 2678400; // 31 * 24 * 60 * 60 + + Time waketime = findMinWaketimeTimeoutQueue(iomgr->timeout_queue); + Time waittime = waketime - now; + + /* Any expired timeouts should have been cleared, so we must be waiting + * for a timeout in the future. */ + ASSERT(waittime > 0); + + tv->tv_sec = TimeToSeconds(waittime); + if (tv->tv_sec < max_seconds) { + tv->tv_usec = TimeToUS(waittime) % 1000000; + } else { + tv->tv_sec = max_seconds; + tv->tv_usec = 0; + } + return tv; + + } else { + return NULL; + } +} +#endif + +#endif // defined(IOMGR_ENABLED_POLL) || defined(IOMGR_ENABLED_SELECTBIS) ===================================== rts/posix/Timeout.h ===================================== @@ -46,5 +46,15 @@ struct timespec *timeoutInNanoseconds(CapIOManager *iomgr, bool wait, Time now, struct timespec *tv); #endif +/* As above, but a timeout in microseconds. This is intended to be used with + * select() which expect struct timespec *, with special values of NULL for + * indefinite wait, and 0 for no waiting. + */ +#if defined(IOMGR_ENABLED_SELECTBIS) +struct timeval *timeoutInMicroseconds(CapIOManager *iomgr, bool wait, + Time now, struct timeval *tv); + +#endif + #include "EndPrivate.h" ===================================== rts/rts.cabal ===================================== @@ -571,6 +571,7 @@ library wasm/JSFFI.c wasm/JSFFIGlobals.c posix/Select.c + posix/SelectBis.c posix/Poll.c posix/Timeout.c cmm-sources: wasm/jsval.cmm @@ -586,6 +587,7 @@ library posix/MIO.c posix/Poll.c posix/Select.c + posix/SelectBis.c posix/Signals.c posix/Timeout.c posix/TTY.c ===================================== testsuite/tests/interface-stability/ghc-experimental-exports.stdout ===================================== @@ -7854,7 +7854,7 @@ module GHC.RTS.Flags.Experimental where type HpcFlags :: * data HpcFlags = HpcFlags {readTixFile :: GHC.Internal.Types.Bool, writeTixFile :: GHC.Internal.Types.Bool} type IoManagerFlag :: * - data IoManagerFlag = IoManagerFlagAuto | IoManagerFlagSelect | IoManagerFlagPoll | IoManagerFlagMIO | IoManagerFlagWinIO | IoManagerFlagWin32Legacy + data IoManagerFlag = IoManagerFlagAuto | IoManagerFlagSelect | IoManagerFlagSelectBis | IoManagerFlagPoll | IoManagerFlagMIO | IoManagerFlagWinIO | IoManagerFlagWin32Legacy type IoSubSystem :: * data IoSubSystem = IoPOSIX | IoNative type MiscFlags :: * View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/a31182d9f52f006838501fde99f62e8... -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/a31182d9f52f006838501fde99f62e8... You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help