Duncan Coutts pushed to branch wip/io-manager-deadlock-detection at Glasgow Haskell Compiler / GHC Commits: d1c0af25 by Duncan Coutts at 2026-03-10T22:56:32+00:00 Split posix/MIO.c out of posix/Signals.c The MIO I/O manager was secretly living inside the Signals file. Now it gets its own file, like any other self-respecting I/O manager. - - - - - 21dc121a by Duncan Coutts at 2026-03-10T22:56:32+00:00 Rationalise some scheduler run queue utilities Move them all to the same place in the file. Make some static that were used only internally. Also remove a redundant assignment after calling truncateRunQueue that is already done within truncateRunQueue. - - - - - 92e1c233 by Duncan Coutts at 2026-03-10T22:56:32+00:00 Rename initIOManager{AfterFork} to {re}startIOManager These are more accurate names, since these actions happen after initialisation and are really about starting (or restarting) background threads. - - - - - f84cc4bd by Duncan Coutts at 2026-03-10T22:56:32+00:00 Add a TODO to the MIO I/O manager The direction of travel is to make I/O managers per-capability and have all their state live in the struct CapIOManager. The MIO I/O manager however still has a number of global variables. It's not obvious how handle these globals however. - - - - - 4aa7cd1c by Duncan Coutts at 2026-03-10T22:56:32+00:00 Free per-cap I/O managers during shutdown and forkProcess Historically this was not strictly necessary. The select and win32 legacy I/O managers did not maintain any dynamically allocated resources. The new poll one does (an auxillary table), and so this should be freed. After forkProcess, all threads get deleted. This includes threads waiting on I/O or timers. So as of this patch, resetting the I/O manager is just about tidying things up. For example, for the poll I/O manager this will reset the size of the AIOP table (which otherwise grows but never shrinks). In future however the re-initialising will become neeecessary for functionality, since some I/O managers will need to re-initialise wakeup fds that are set CLOEXEC. - - - - - 8f33ef3a by Duncan Coutts at 2026-03-10T22:56:32+00:00 Add an FdWakup module for posix I/O managers This will be used to implement wakeupIOManager for in-RTS I/O managers. It provides a notification/wakeup mechanism using FDs, suitable for situations when I/O managers are blocked on a set of fds anyway. - - - - - 7c193c28 by Duncan Coutts at 2026-03-10T22:56:32+00:00 Add wakeupIOManager support for select I/O manager Uses the FdWakup mechanism. - - - - - 1028c963 by Duncan Coutts at 2026-03-10T22:56:33+00:00 Add wakeupIOManager support for poll I/O manager Uses the FdWakup mechanism. A quirk we have to cope with is that we now need to poll one more fd -- the wakeup_fd_r -- but this fd has no corresponding entry in the aiop_table. This is awkward since we have set up our aiop_poll_table to be an auxilliary table with matching indicies. The solution this patch uses (and described in the comments) is to have two tables: struct pollfd *aiop_poll_table, *full_poll_table; and to have the aiop_poll_table alias the tail of the full_poll_table. The head entry in the full_poll_table is the extra fd. So we poll the full_poll_table, while the aiop_poll_table still has matching indicies with the aiop_table. Hurrah for C aliasing rules. - - - - - 0d1aeb0c by Duncan Coutts at 2026-03-10T22:56:33+00:00 Add wakeupIOManager support for win32 legacy I/O manager - - - - - 83e9ccc5 by Duncan Coutts at 2026-03-10T22:56:33+00:00 wakeupIOManager is now required for all I/O managers We are going to rely on it. Previously it could be a no-op. Update the docs in the header file. Also, temporarily disable awaitCompletedTimeoutsOrIO post-condition assertion. It will become more complicated due to wakeupIOManager, and it's not yet clear how to express it. We will re-introduce a post condition after a few more changes. - - - - - b8a005b8 by Duncan Coutts at 2026-03-10T22:56:33+00:00 Make signal handling be a respondibility of the I/O manager(s) Previously it was scattered between I/O managers and the scheduler, and especially the scheduler's deadlock detection. Previously the scheduler would poll for pending signals each iteration of the scheduler loop. The scheduler also had some hairy signal functionality in the deadlock detection: in the non-threaded RTS (only) if there were still no threads running after deadlock detection then it would block waiting for signals. But signals can and (in my opinion) should be thought of as just a funny kind of I/O, and thus should be a responsibility of the I/O manager. So now we have the I/O managers poll for signals when they are polling for I/O completion (and removing the separate poll in the scheduler). And when I/O managers block waiting for I/O then they now also start signal handlers if they get interrupted by a signal. Crucially, if there is no pending I/O or timers, the awaitCompletedTimeoutsOrIO will still block waiting for signals. This patch puts us into an intermediate state: it temporarily breaks deadlock detection in the non-threaded RTS. The waiting on I/O currently happens before deadlock detection. This means we'll now wait forever on signals before doing deadlock detection. We need to move waiting after deadlock detection. We'll do that in a later patch. - - - - - c5d0190e by Duncan Coutts at 2026-03-10T22:56:33+00:00 Clean up signal handling internal API Now that the I/O manager is responsible for signals, we can simplify the API we present for signal handling. We now just need startPendingSignalHandlers, which is called from the I/O managers. We can get rid of awaitUserSignals. We also don't need RtsSignals.h to re-export the platform-specific posix/Signals.h or win32/ConsoleHandler.h We can also hide more of the implementation of signals. Less has to be exposed in posix/Signals.h or win32/ConsoleHandler.h. Partly this is because we don't need inline functions (or macros) in the interface. Also remove signal_handlers from RTS ABI exported symbols list. It does not appear to have any users in the core libs, and its really an internal implementation detail. It should not be exposed unless its really necessary. - - - - - b2d5a452 by Duncan Coutts at 2026-03-10T22:56:33+00:00 In the scheduler, move I/O blocking after deadlock detection To make deadlock detection effective in the non-threaded RTS when there are deadlocked threads and other unrelated threads waiting on I/O, we need to arrange to do deadlock detection before we block in scheduler to wait on I/O. The solution is to: 1. adjust scheduleFindWork, which runs before deadlock detection, to only poll for I/O and not block; and 2. add a step after deadlock detection to wait on I/O if there are still no threads to run (and there's any I/O or timeouts outstanding) The scheduleCheckBlockedThreads is now so simple that it made more sense to inline it into scheduleFindWork. - - - - - 198a9d50 by Duncan Coutts at 2026-03-10T22:56:33+00:00 Remove bogus anyPendingTimeoutsOrIO guard from scheduleDetectDeadlock The deadlock detection was only invoked if both of these conditions hold: 1. the run queue is empty 2. there is no pending I/O or timeouts The second condition is unnecessary. The deadlock detection mechanism can find deadlocks even if there are other threads waiting on I/O or timers. Having this extra condition means that we fail to detect blocked threads if there are any threads waiting on I/O or timers. Part of fixing issue #26408 - - - - - 67ebecf0 by Duncan Coutts at 2026-03-10T22:56:33+00:00 Don't consider pending I/O for early context switch optimisation Context switches are normally initiated by the timer signal. If however the user specifies "context switch as often as possible", with +RTS -C0 then the scheduler arranges for an early context switch (when it's just about to run a Haskell thread). Context switching very often is expensive, so as an optimisation there cases where we do not arrange an early context switch: 1. if there's no other threads to run 2. if there is no pending I/O or timers This patch eliminates case 2, leaving only case 1. The rationale is as follows. The use of this was inconsistent across platforms and threaded/non-threaded RTS ways. It only worked on the non-threaded RTS and on Windows only worked for the win32-legacy I/O manager. On all other combinations anyPendingTimeoutsOrIO would always return false. The fact that nobody noticed and complained about this inconsistency suggests that the feature is not relied upon. If however it turns out that applications do rely on this, then the proper thing to do is not to restore this check, but to add a new I/O manager hint function that returns if there is any pending events that are likely to happen *soon*: for example timeouts expiring within one timeslice, or I/O waits on things likely to complete soon like disk I/O, but not for example socket/pipe I/O. The motivation to avoid this use of anyPendingTimeoutsOrIO is to allow us to eliminate anyPendingTimeoutsOrIO entirely. All other uses of this are just guards on {await,poll}CompletedTimeoutsOrIO and the guards can safely be folded into those functions. This will better cope with some I/O managers having no proper implementation of anyPendingTimeoutsOrIO. Ultimately this will let us simplify the scheduler which currently has to have special #ifdef mingw32_HOST_OS cases to cope with the lack of a working anyPendingTimeoutsOrIO for some Windows I/O managers - - - - - bbb1cdba by Duncan Coutts at 2026-03-10T22:56:33+00:00 Remove anyPendingTimeoutsOrIO guarding {poll,await}CompletedTimeoutsOrIO Previously the API of the I/O manager used a two step process: check anyPendingTimeoutsOrIO and then call {poll,await}CompletedTimeoutsOrIO. This was primarily there as a performance thing, to cheaply check if we need to do anything. And then because anyPendingTimeoutsOrIO existed, it was used for other things too. We have now eliminated the other uses, and are just left with the performance pattern. But this was problematic because not all I/O managers correctly implement anyPendingTimeoutsOrIO (specifically the win32 ones), and now that we also make I/O managers responsible for signals then we need to poll/await even if there is no pending I/O or timeouts. If there is no pending I/O or timeouts then poll/await needs to degenerate to just waiting forever for any signals. - - - - - a3c422f3 by Duncan Coutts at 2026-03-10T22:56:33+00:00 Remove anyPendingTimeoutsOrIO, it is no longer used And this avoids the problems arising from the win32 I/O managers having had a bogus implementation. - - - - - 298694d9 by Duncan Coutts at 2026-03-10T22:59:04+00:00 Remove second scheduler call to awaitCompletedTimeoutsOrIO Previously awaitCompletedTimeoutsOrIO was called both before and after deadlock detection in the scheduler. The reason for that was that the win32 I/O managers had a bogus implementation of anyPendingTimeoutsOrIO and this was used to guard the call of awaitCompletedTimeoutsOrIO prior to deadlock detection. This meant the first call site was never actually called when using the win32 I/O managers. This was the reason for the second call: the first one was never used. What a mess. So now we have a simple design in the scheduler: 1. poll for completed I/O, timers or signals 2. if no runnable threads: do deadlock detection 3. if still no runnable threads: block waiting for I/O, timers or signals. - - - - - f3684344 by Duncan Coutts at 2026-03-10T22:59:07+00:00 Lift emptyRunQueue guard out of scheduleDetectDeadlock this improved the clarity of the logic when reading the scheduler code. - - - - - 0b1fab72 by Duncan Coutts at 2026-03-10T22:59:07+00:00 Make non-threaded deadlock detection also rely on idle GC Only do deadlock detection GC when idle GC kicks in. This also relies on using wakeUpRts, so now do this unconditionally. Previously wakeUpRts was for the threaded rts only. - - - - - 5a5c128e by Duncan Coutts at 2026-03-10T22:59:07+00:00 Enable idle GC by default on non-threaded RTS. The behaviour is now uniform between threaded and non-threaded. The deadlock detection now relies on idle GC for both threaded and non-threaded ways. Previously deadlock detection did not rely on idle GC for the non-threaded way. - - - - - b29a5ae2 by Duncan Coutts at 2026-03-10T22:59:07+00:00 Add a long Note [Deadlock detection] It describes the historical and modern designs and their trade-offs. The point is we've now unified the code for deadlock detection between the threaded and non-threaded ways, by changing the non-threaded to follow the same design as the threaded. - - - - - f7a336ee by Duncan Coutts at 2026-03-10T22:59:07+00:00 Add a test for deadlock detection, issue #26408 - - - - - 9e15419e by Duncan Coutts at 2026-03-10T22:59:07+00:00 Update the user guide with the revised idle GC behaviour i.e. it's now not just for the threaded RTS, but general. Also document the fact that disabling idle GC also disables deadlock detection. - - - - - 29 changed files: - docs/users_guide/runtime_control.rst - rts/Capability.c - rts/IOManager.c - rts/IOManager.h - rts/IOManagerInternals.h - rts/RtsFlags.c - rts/RtsSignals.h - rts/RtsStartup.c - rts/RtsSymbols.c - rts/Schedule.c - rts/Schedule.h - rts/Timer.c - + rts/posix/FdWakeup.c - + rts/posix/FdWakeup.h - + rts/posix/MIO.c - + rts/posix/MIO.h - rts/posix/Poll.c - rts/posix/Poll.h - rts/posix/Select.c - rts/posix/Select.h - rts/posix/Signals.c - rts/posix/Signals.h - rts/rts.cabal - rts/win32/AwaitEvent.c - rts/win32/ConsoleHandler.c - rts/win32/ConsoleHandler.h - + testsuite/tests/rts/T26408.hs - + testsuite/tests/rts/T26408.stderr - testsuite/tests/rts/all.T Changes: ===================================== docs/users_guide/runtime_control.rst ===================================== @@ -739,18 +739,18 @@ performance. .. rts-flag:: -I ⟨seconds⟩ - :default: 0.3 seconds in the threaded runtime, 0 in the non-threaded runtime + :default: 0.3 seconds .. index:: single: idle GC - Set the amount of idle time which must pass before a idle GC is - performed. Setting ``-I0`` disables the idle GC. + A major GC is automatically performed if the runtime has been idle (no + Haskell computation has been running) for a period of time. Set the amount + of idle time which must pass before a idle GC is performed. - In the threaded and SMP versions of the RTS (see :ghc-flag:`-threaded`, - :ref:`options-linker`), a major GC is automatically performed if the - runtime has been idle (no Haskell computation has been running) for a - period of time. + Setting ``-I0`` disables the idle GC. This also has the unfortunate side + effect of disabling thread deadlock detection (the implementation of which + uses the idle GC). For an interactive application, it is probably a good idea to use the idle GC, because this will allow finalizers to run and @@ -767,8 +767,8 @@ performance. after the first idle collection is triggered then no more future collections will be scheduled until more work is performed. - This is an experimental feature, please let us know if it causes - problems and/or could benefit from further tuning. + Please let us know if it causes problems and/or could benefit from further + tuning. .. rts-flag:: -Iw ⟨seconds⟩ @@ -779,7 +779,7 @@ performance. Set the minimum wait time between runs of the idle GC. - By default, if idle GC is enabled in the threaded runtime, a major + By default (and if idle GC is not disabled) a major GC will be performed every time the process goes idle for a sufficiently long duration (see :rts-flag:`-I ⟨seconds⟩`). For large server processes accepting regular but infrequent requests ===================================== rts/Capability.c ===================================== @@ -1280,6 +1280,7 @@ shutdownCapabilities(Task *task, bool safe) static void freeCapability (Capability *cap) { + freeCapabilityIOManager(cap->iomgr); stgFree(cap->mut_lists); stgFree(cap->saved_mut_lists); if (cap->current_segments) { ===================================== rts/IOManager.c ===================================== @@ -39,7 +39,7 @@ #endif #if defined(IOMGR_ENABLED_MIO_POSIX) -#include "posix/Signals.h" +#include "posix/MIO.h" #include "Prelude.h" #endif @@ -343,9 +343,7 @@ void initCapabilityIOManager(CapIOManager *iomgr) switch (iomgr_type) { #if defined(IOMGR_ENABLED_SELECT) case IO_MANAGER_SELECT: - iomgr->blocked_queue_hd = END_TSO_QUEUE; - iomgr->blocked_queue_tl = END_TSO_QUEUE; - iomgr->sleeping_queue = END_TSO_QUEUE; + initCapabilityIOManagerSelect(iomgr); break; #endif @@ -373,11 +371,31 @@ void initCapabilityIOManager(CapIOManager *iomgr) } +void freeCapabilityIOManager(CapIOManager *iomgr) +{ + switch (iomgr_type) { +#if defined(IOMGR_ENABLED_SELECT) + case IO_MANAGER_SELECT: + freeCapabilityIOManagerSelect(iomgr); + break; +#endif + +#if defined(IOMGR_ENABLED_POLL) + case IO_MANAGER_POLL: + freeCapabilityIOManagerPoll(iomgr); + break; +#endif + default: + break; + } +} + + /* Called late in the RTS initialisation */ -void initIOManager(void) +void startIOManager(void) { - debugTrace(DEBUG_iomanager, "initialising %s I/O manager", showIOManager()); + debugTrace(DEBUG_iomanager, "starting %s I/O manager", showIOManager()); switch (iomgr_type) { @@ -441,7 +459,7 @@ void initIOManager(void) /* Called from forkProcess in the child process on the surviving capability. */ void -initIOManagerAfterFork(CapIOManager *iomgr, Capability **pcap) +restartIOManager(CapIOManager *iomgr, Capability **pcap) { switch (iomgr_type) { @@ -541,13 +559,25 @@ exitIOManager(bool wait_threads) } } -/* Wakeup hook: called from the scheduler's wakeUpRts (currently only in - * threaded mode). +/* Wakeup hook: called from the scheduler's wakeUpRts */ void wakeupIOManager(void) { + debugTrace(DEBUG_iomanager, "Sending wakeup to I/O manager..."); switch (iomgr_type) { +#if defined(IOMGR_ENABLED_SELECT) + case IO_MANAGER_SELECT: + wakeupIOManagerSelect(MainCapability.iomgr); + break; +#endif + +#if defined(IOMGR_ENABLED_POLL) + case IO_MANAGER_POLL: + wakeupIOManagerPoll(MainCapability.iomgr); + break; +#endif + #if defined(IOMGR_ENABLED_MIO_POSIX) case IO_MANAGER_MIO_POSIX: /* MIO Posix implementation in posix/Signals.c */ @@ -572,8 +602,13 @@ void wakeupIOManager(void) #endif break; #endif - default: +#if defined(IOMGR_ENABLED_WIN32_LEGACY) + case IO_MANAGER_WIN32_LEGACY: + abandonRequestWait(); break; +#endif + default: + barf("wakeupIOManager not implemented"); } } @@ -661,64 +696,6 @@ setIOManagerControlFd(uint32_t cap_no, int fd) { #endif -bool anyPendingTimeoutsOrIO(CapIOManager *iomgr) -{ - switch (iomgr_type) { -#if defined(IOMGR_ENABLED_SELECT) - case IO_MANAGER_SELECT: - return (iomgr->blocked_queue_hd != END_TSO_QUEUE) - || (iomgr->sleeping_queue != END_TSO_QUEUE); -#endif - -#if defined(IOMGR_ENABLED_POLL) - case IO_MANAGER_POLL: - return anyPendingTimeoutsOrIOPoll(iomgr); -#endif - -#if defined(IOMGR_ENABLED_WIN32_LEGACY) - case IO_MANAGER_WIN32_LEGACY: - return (iomgr->blocked_queue_hd != END_TSO_QUEUE); -#endif - - /* For the purpose of the scheduler, the threaded I/O managers never have - pending I/O or timers. Of course in reality they do, but they're - managed via other primitives that the scheduler can see into (threads, - MVars and foreign blocking calls). - */ -#if defined(IOMGR_ENABLED_MIO_POSIX) - case IO_MANAGER_MIO_POSIX: - return false; -#endif - -#if defined(IOMGR_ENABLED_MIO_WIN32) - case IO_MANAGER_MIO_WIN32: - return false; -#endif - -#if defined(IOMGR_ENABLED_WINIO) -#if defined(THREADED_RTS) - /* As above, the threaded variants never have pending I/O or timers */ - case IO_MANAGER_WINIO: - return false; -#else - case IO_MANAGER_WINIO: - return false; - /* FIXME: But what is this? The WinIO I/O manager *also* returns false - in the non-threaded case! This is *totally bogus*! In the - non-threaded RTS the scheduler expects to be able to poll for IO. - The fact that this gives a wrong and useless answer for WinIO is - probably the cause of the complication in the scheduler with having - to call awaitCompletedTimeoutsOrIO() in multiple places (on Windows, - non-threaded). - */ -#endif -#endif - default: - barf("anyPendingTimeoutsOrIO not implemented"); - } -} - - void pollCompletedTimeoutsOrIO(CapIOManager *iomgr) { debugTrace(DEBUG_iomanager, "polling for completed IO or timeouts"); @@ -782,7 +759,9 @@ void awaitCompletedTimeoutsOrIO(CapIOManager *iomgr) default: barf("pollCompletedTimeoutsOrIO not implemented"); } - ASSERT(!emptyRunQueue(iomgr->cap) || getSchedState() != SCHED_RUNNING); + // FIXME: the post condition is now more complicated. Await can now simply + // be interrupted by wakeupIOManager. + // ASSERT(!emptyRunQueue(iomgr->cap) || getSchedState() != SCHED_RUNNING); } ===================================== rts/IOManager.h ===================================== @@ -15,6 +15,11 @@ * subsystem implementations are centralised here. Not all implementations use * all hooks. * + * I/O manager are responsible for: + * - threads waiting on I/O + * - threads waiting on timeouts + * - signals (unix, and win32 console) starting handlers + * * -------------------------------------------------------------------------*/ #pragma once @@ -242,11 +247,33 @@ CapIOManager *allocCapabilityIOManager(Capability *cap); */ void initCapabilityIOManager(CapIOManager *iomgr); +/* When shutting down a capability, or after forkProcess, free the resources + * held by a CapIOManager to put it back into a state in which either it can be + * re-initialised using initCapabilityIOManager, or the whole structure freed. + * + * Note that this does not free the CapIOManager structure itself, just the + * contents. + * + * This is used during capability shutdown, during RTS shutdown. It is not used + * when reducing the number of capabilities. Capabilities are disabled rather + * than freed entirely: the I/O manager keeps running but threads that become + * runnable are migrated away. + * + * It is also used after forkProcess. + */ +void freeCapabilityIOManager(CapIOManager *iomgr); + +/* CapIOManager life cycle: + * + * alloc -> init -> free -> free struct + * ^ | + * +--------+ + */ /* Init hook: called from hs_init_ghc, very late in the startup after almost * everything else is done. */ -void initIOManager(void); +void startIOManager(void); /* Init hook: called from forkProcess in the child process on the surviving @@ -255,8 +282,8 @@ void initIOManager(void); * This is synchronous and can run Haskell code, so can change the given cap. * TODO: it would make for a cleaner API here if this were made asynchronous. */ -void initIOManagerAfterFork(CapIOManager *iomgr, - /* inout */ Capability **pcap); +void restartIOManager(CapIOManager *iomgr, + /* inout */ Capability **pcap); /* TODO: rationalise initIOManager and initIOManagerAfterFork into a single per-capability init function. @@ -283,19 +310,13 @@ void stopIOManager(void); void exitIOManager(bool wait_threads); -/* Wakeup hook: called from the scheduler's wakeUpRts (currently only in - * threaded mode). +/* Wakeup hook: called from the scheduler's wakeUpRts(). * * The I/O manager can be blocked waiting on I/O or timers. Sometimes there are * other external events where we need to wake up the I/O manager and return - * to the schedulr. - * - * At the moment, all the non-threaded I/O managers will do this automagically - * since a signal will interrupt any waiting system calls, so at the moment - * the implementation for the non-threaded I/O managers does nothing. + * to the scheduler. * - * For the I/O managers in threaded mode, this arranges to unblock the I/O - * manager if it waa blocked waiting. + * This arranges to unblock the I/O manager if it was blocked waiting. */ void wakeupIOManager(void); @@ -343,35 +364,27 @@ void syncDelayCancel(CapIOManager *iomgr, StgTSO *tso); void appendToIOBlockedQueue(CapIOManager *iomgr, StgTSO *tso); #endif -/* Check to see if there are any pending timeouts or I/O operations - * in progress with the I/O manager. +/* Poll for any completed I/O operations, expired timers or pending signals + * with handlers. If there are any, process the completions as appropriate + * (which will typically unblock some waiting threads). * - * This is used by the scheduler as part of deadlock-detection, and the - * "context switch as often as possible" test. - */ -bool anyPendingTimeoutsOrIO(CapIOManager *iomgr); - -/* If there are any completed I/O operations or expired timers, process the - * completions as appropriate (which will typically unblock some waiting - * threads, but no guarantee). If there are none, return without waiting. + * This polls, but does not block. + * + * No post-condition. It does not guarantee anything such as there being + * runnable threads, since this does not wait. * - * Called from schedule() both *before* and *after* scheduleDetectDeadlock(). + * Called from schedule() before scheduleDetectDeadlock(). */ void pollCompletedTimeoutsOrIO(CapIOManager *iomgr); - /* If there are any completed I/O operations or expired timers, process the - * completions as appropriate. If there are none, wait until I/O or a timer - * does complete (or we get a signal with a handler) and process the - * completions as appropriate. +/* Wait for completed I/O operations, expired timers or signals and process + * the completions as appropriate. * * Upon return this guarantees that the scheduler run queue is non-empty or * that the scheduler is no longer in the running state. Succinctly, the * post-condition is (!emptyRunQueue(cap) || getSchedState() != SCHED_RUNNING). * - * This is only expected to be called if anyPendingTimeoutsOrIO() returns true, - * i.e. there actually is something to wait for. - * - * Called from schedule() both *before* and *after* scheduleDetectDeadlock(). + * Called from schedule() after scheduleDetectDeadlock(). */ void awaitCompletedTimeoutsOrIO(CapIOManager *iomgr); ===================================== rts/IOManagerInternals.h ===================================== @@ -46,6 +46,11 @@ struct _CapIOManager { StgTSO *sleeping_queue; #endif +#if defined(IOMGR_ENABLED_SELECT) || defined(IOMGR_ENABLED_POLL) + /* FDs for waking up the I/O manager when it is blocked waiting */ + int wakeup_fd_r, wakeup_fd_w; +#endif + #if defined(IOMGR_ENABLED_POLL) /* AIOP and timeout collections shared by several I/O manager impls */ ClosureTable aiop_table; @@ -53,8 +58,11 @@ struct _CapIOManager { #endif #if defined(IOMGR_ENABLED_POLL) - /* Auxiliary table with size and indexes matching the aiop_table */ - struct pollfd *aiop_poll_table; + /* Auxiliary table with size and indexes matching the aiop_table. This is + * aliased to the tail of the full poll table, which has a head entry for + * the wakeup_fd_r above, so we can also poll that fd. + */ + struct pollfd *aiop_poll_table, *full_poll_table; #endif #if defined(IOMGR_ENABLED_WIN32_LEGACY) ===================================== rts/RtsFlags.c ===================================== @@ -173,11 +173,7 @@ void initRtsFlagsDefaults(void) RtsFlags.GcFlags.sweep = false; RtsFlags.GcFlags.idleGCDelayTime = USToTime(300000); // 300ms RtsFlags.GcFlags.interIdleGCWait = 0; -#if defined(THREADED_RTS) RtsFlags.GcFlags.doIdleGC = true; -#else - RtsFlags.GcFlags.doIdleGC = false; -#endif RtsFlags.GcFlags.heapBase = 0; /* means don't care */ RtsFlags.GcFlags.allocLimitGrace = (100*1024) / BLOCK_SIZE; RtsFlags.GcFlags.numa = false; ===================================== rts/RtsSignals.h ===================================== @@ -2,26 +2,15 @@ * * (c) The GHC Team, 1998-2005 * - * Signal processing / handling. + * Signal processing / handling. This is the shared API to the subsystems for + * POSIX signals and Win32 console events. + * + * Platform specific APIs live in posix/Signals.h and win32/ConsoleHandler.h * * ---------------------------------------------------------------------------*/ #pragma once -#if !defined(mingw32_HOST_OS) && defined(HAVE_SIGNAL_H) - -#include "posix/Signals.h" - -#elif defined(mingw32_HOST_OS) - -#include "win32/ConsoleHandler.h" - -#else - -#define signals_pending() (false) - -#endif - #if defined(RTS_USER_SIGNALS) #include "BeginPrivate.h" @@ -44,39 +33,20 @@ void resetDefaultHandlers(void); void freeSignalHandlers(void); -/* - * Function: awaitUserSignals() - * - * Wait for the next console event. Currently a NOP (returns immediately.) +/* Tear down and shut down user signal processing. + * This is called *after* freeSignalHandlers, but unconditionally! + * TODO: unify this and freeSignalHandlers together, and make them make sense! */ -void awaitUserSignals(void); +void finiUserSignals(void); /* * Function: startPendingSignalHandlers() * - * Start any pending signal handlers. This is used by the scheduler and some - * in-RTS I/O managers. It does nothing (returns false) in the threaded RTS. - * - * Returns true if any signal handlers were pending and thus started. + * If there are any queued up posix signals or win32 console events, run the + * handlers associated with them. This is used by some in-RTS I/O managers. */ -INLINE_HEADER bool startPendingSignalHandlers(Capability *cap); - #if !defined(THREADED_RTS) -INLINE_HEADER bool startPendingSignalHandlers(Capability *cap) -{ - if (RtsFlags.MiscFlags.install_signal_handlers && signals_pending()) { - // safe outside the lock - startSignalHandlers(cap); - return true; - } else { - return false; - } -} -#else -INLINE_HEADER bool startPendingSignalHandlers(Capability *cap STG_UNUSED) -{ - return false; -} +void startPendingSignalHandlers(Capability *cap); #endif #include "EndPrivate.h" ===================================== rts/RtsStartup.c ===================================== @@ -70,6 +70,10 @@ #include #endif +#if !defined(mingw32_HOST_OS) && defined(HAVE_SIGNAL_H) +#include +#endif + // Count of how many outstanding hs_init()s there have been. static StgWord hs_init_count = 0; static bool rts_shutdown = false; @@ -427,7 +431,7 @@ hs_init_ghc(int *argc, char **argv[], RtsConfig rts_config) } #endif - initIOManager(); + startIOManager(); x86_init_fpu(); @@ -619,9 +623,10 @@ hs_exit_(bool wait_foreign) #if defined(mingw32_HOST_OS) if (is_io_mng_native_p()) hs_restoreConsoleCP(); +#endif - /* Disable console signal handlers, we're going down!. */ - finiUserSignals (); +#if defined(RTS_USER_SIGNALS) + finiUserSignals(); #endif /* tear down statistics subsystem */ ===================================== rts/RtsSymbols.c ===================================== @@ -71,7 +71,6 @@ extern char **environ; SymI_HasProto(__hscore_get_saved_termios) \ SymI_HasProto(__hscore_set_saved_termios) \ SymI_HasProto(shutdownHaskellAndSignal) \ - SymI_HasProto(signal_handlers) \ SymI_HasProto(stg_sig_install) \ SymI_HasProto(rtsTimerSignal) \ SymI_NeedsDataProto(nocldstop) ===================================== rts/Schedule.c ===================================== @@ -146,7 +146,6 @@ static void acquireAllCapabilities(Capability *cap, Task *task); static void startWorkerTasks (uint32_t from USED_IF_THREADS, uint32_t to USED_IF_THREADS); #endif -static void scheduleCheckBlockedThreads (Capability *cap); static void scheduleProcessInbox(Capability **cap); static void scheduleDetectDeadlock (Capability **pcap, Task *task); static void schedulePushWork(Capability *cap, Task *task); @@ -174,6 +173,11 @@ static void deleteAllThreads (void); static void deleteThread_(StgTSO *tso); #endif +#if defined(FORKPROCESS_PRIMOP_SUPPORTED) +static void truncateRunQueue(Capability *cap); +#endif +static StgTSO *popRunQueue (Capability *cap); + /* --------------------------------------------------------------------------- Main scheduling loop. @@ -295,21 +299,21 @@ schedule (Capability *initialCapability, Task *task) (pushes threads, wakes up idle capabilities for stealing) */ schedulePushWork(cap,task); - scheduleDetectDeadlock(&cap,task); + if (emptyRunQueue(cap)) { + /* When we have no threads to run, we *might* have a deadlock. */ + scheduleDetectDeadlock(&cap,task); + } - // Normally, the only way we can get here with no threads to - // run is if a keyboard interrupt received during - // scheduleCheckBlockedThreads() or scheduleDetectDeadlock(). - // Additionally, it is not fatal for the - // threaded RTS to reach here with no threads to run. - // - // Since IOPorts have no deadlock avoidance guarantees you may also reach - // this point when blocked on an IO Port. If this is the case the only - // thing that could unblock it is an I/O event. - // - // win32: might be here due to awaitCompletedTimeoutsOrIO() being abandoned - // as a result of a console event having been delivered or as a result of - // waiting on an async I/O to complete with WinIO. +#if !defined(THREADED_RTS) + /* scheduleFindWork checks for completed I/O but does not block. If there + * is nothing to do now, we block and wait for I/O, timeouts or signals. + * Importantly, we only block /after/ checking for deadlocks. See #26408. + */ + if (emptyRunQueue(cap)) { + awaitCompletedTimeoutsOrIO(cap->iomgr); + if (emptyRunQueue(cap)) continue; // look for work again + } +#endif #if defined(THREADED_RTS) scheduleYield(&cap,task); @@ -317,22 +321,6 @@ schedule (Capability *initialCapability, Task *task) if (emptyRunQueue(cap)) continue; // look for work again #endif -#if !defined(THREADED_RTS) - if ( emptyRunQueue(cap) ) { -#if defined(mingw32_HOST_OS) - /* Notify the I/O manager that we have nothing to do. If there are - any outstanding I/O requests we'll block here. If there are not - then this is a user error and we will abort soon. */ - /* TODO: see if we can rationalise these two awaitCompletedTimeoutsOrIO - * calls before and after scheduleDetectDeadlock(). - */ - awaitCompletedTimeoutsOrIO(cap->iomgr); -#else - ASSERT(getSchedState() >= SCHED_INTERRUPTING); -#endif - } -#endif - // // Get a thread to run // @@ -403,13 +391,12 @@ schedule (Capability *initialCapability, Task *task) } #endif - /* context switches are initiated by the timer signal, unless - * the user specified "context switch as often as possible", with - * +RTS -C0 - */ - if (RtsFlags.ConcFlags.ctxtSwitchTicks == 0 && - (!emptyRunQueue(cap) || - anyPendingTimeoutsOrIO(cap->iomgr))) { + // Context switches are normally initiated by the timer signal. If however + // the user specified "context switch as often as possible", with +RTS -C0 + // then we now arrange for an early context switch. Context switching very + // often is expensive, so as an optimisation if there's no other threads + // to run then we don't arrange a context switch. + if (RtsFlags.ConcFlags.ctxtSwitchTicks == 0 && !emptyRunQueue(cap)) { RELAXED_STORE(&cap->context_switch, 1); } @@ -591,42 +578,12 @@ run_thread: } /* end of while() */ } -/* ----------------------------------------------------------------------------- - * Run queue operations - * -------------------------------------------------------------------------- */ - -static void -removeFromRunQueue (Capability *cap, StgTSO *tso) -{ - if (tso->block_info.prev == END_TSO_QUEUE) { - ASSERT(cap->run_queue_hd == tso); - cap->run_queue_hd = tso->_link; - } else { - setTSOLink(cap, tso->block_info.prev, tso->_link); - } - if (tso->_link == END_TSO_QUEUE) { - ASSERT(cap->run_queue_tl == tso); - cap->run_queue_tl = tso->block_info.prev; - } else { - setTSOPrev(cap, tso->_link, tso->block_info.prev); - } - tso->_link = tso->block_info.prev = END_TSO_QUEUE; - cap->n_run_queue--; - - IF_DEBUG(sanity, checkRunQueue(cap)); -} - -void -promoteInRunQueue (Capability *cap, StgTSO *tso) -{ - removeFromRunQueue(cap, tso); - pushOnRunQueue(cap, tso); -} - /* ----------------------------------------------------------------------------- * scheduleFindWork() * * Search for work to do, and handle messages from elsewhere. + * + * This does *not* block/wait, even in the non-threaded case. * -------------------------------------------------------------------------- */ static void @@ -635,16 +592,17 @@ scheduleFindWork (Capability **pcap) #if defined(mingw32_HOST_OS) && !defined(THREADED_RTS) queueIOThread(); #endif -#if defined(RTS_USER_SIGNALS) - startPendingSignalHandlers(*pcap); -#endif - scheduleProcessInbox(pcap); - scheduleCheckBlockedThreads(*pcap); + /* From here on, the cap can't change. */ + Capability *cap = *pcap; + +#if !defined(THREADED_RTS) + pollCompletedTimeoutsOrIO(cap->iomgr); +#endif #if defined(THREADED_RTS) - if (emptyRunQueue(*pcap)) { scheduleActivateSpark(*pcap); } + if (emptyRunQueue(cap)) { scheduleActivateSpark(cap); } #endif } @@ -889,115 +847,158 @@ schedulePushWork(Capability *cap USED_IF_THREADS, } -/* ---------------------------------------------------------------------------- - * Check for blocked threads that can be woken up. - * ------------------------------------------------------------------------- */ - -static void -scheduleCheckBlockedThreads(Capability *cap USED_IF_NOT_THREADS) -{ -#if !defined(THREADED_RTS) - /* Check whether there is any completed I/O or expired timers. If so, - * process the competions as appropriate, which will typically cause some - * waiting threads to be woken up. - * - * If the run queue is empty, and there are no other threads running, we - * can wait indefinitely for something to happen. - * - * TODO: see if we can rationalise these two awaitCompletedTimeoutsOrIO - * calls before and after scheduleDetectDeadlock() - * - * TODO: this test anyPendingTimeoutsOrIO does not have a proper - * implementation the WinIO I/O manager! - * - * The select() I/O manager uses the sleeping_queue and the blocked_queue, - * and the test checks both. The legacy win32 I/O manager only consults - * the blocked_queue, but then it puts threads waiting on delay# on the - * blocked_queue too, so that's ok. - * - * The WinIO I/O manager does not use either the sleeping_queue or the - * blocked_queue, but it's implementation of anyPendingTimeoutsOrIO still - * checks both! Since both queues will _always_ be empty then it will - * _always_ return false and so awaitCompletedTimeoutsOrIO will _never_ be - * called here for WinIO. This may explain why there is a second call to - * awaitCompletedTimeoutsOrIO below for the case of !defined(THREADED_RTS) - * && defined(mingw32_HOST_OS). - */ - if (anyPendingTimeoutsOrIO(cap->iomgr)) - { - if (emptyRunQueue(cap)) { - // block and wait - awaitCompletedTimeoutsOrIO(cap->iomgr); - } else { - // poll but do not wait - pollCompletedTimeoutsOrIO(cap->iomgr); - } - } -#endif -} - /* ---------------------------------------------------------------------------- * Detect deadlock conditions and attempt to resolve them. * ------------------------------------------------------------------------- */ +/* Note [Deadlock detection] +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +For the purpose of this explanation we define: + * a /partial deadlock/ to be a set of threads that are deadlocked; and + * a /system deadlock/ is when all threads are deadlocked. + +Obviously, we can have a partial deadlock without having a system +deadlock. The design goal of deadlock detection is to guarantee to +detect (and resolve) system deadlock, but to also try to detect (and +resolve) partial deadlocks. + +There are two designs that the RTS has used for deadlock detection: a +simple historical design originally used in the non-threaded RTS and a +modern design for the threaded RTS. These days we use the modern design +in both the threaded and non-threaded RTS. + +A high level way to think about the two designs is as follows: + 1. the historical design looks for situations in which there *must* be + a system deadlock; whereas + 2. the modern design looks for partial deadlocks opportunistically, + with the guarantee that if the overall system is deadlocked that we + will *eventually* detect this. + +An advantage of the historical design is that it will detect system +deadlock promptly. A disadvantage is that it will never detect a +partial deadlock (that isn't also a system deadlock). + +The modern design can detect partial deadlock, but it is not guaranteed +to detect system deadlock promptly, just eventually. + +The mechanism for deadlock detection is garbage collection. GC can be +instructed to look for deadlocked threads and if it finds them to throw +exceptions to one or more threads involved in the deadlock. This +mechanism can find partial deadlocks. It is however expensive -- more +expensive than a normal major GC. So the difference in the historical +and modern designs is in when we do this expensive GC check. + +The historical design +--------------------- + +When there was just one capability, as in the single threaded RTS, it +is possible to follow a very simple design. When there are no runnable +threads, and no threads blocked on pending I/O or on timers then there +*must* be a deadlock. And thus running deadlock detection promptly in +this situation is guaranteed to find the deadlock and wake up one or +more threads. Thus we can guarantee afterwards that there are runnable +threads. + +There are a couple problems with this design, but the biggest problem +is that it cannot be extended to multiple capabilities. When there are +multiple capabilities then the fact that there are no runnable threads +on the current capability says nothing about runnable threads on other +capabilities. Runnable threads elsewhere might wake up threads on this +capability, and so there is no implication that there is a deadlock. + +The other problems with this design are: + 1. it cannot find genuine deadlocks when there are any unrelated + threads blocked on I/O or timers (see issue #26408); and + 2. it requires treating signals specially. + +The problem with signals is that they're a weird kind of I/O. Threads +do not block waiting on signals. Rather signals can have handlers such +that when a signal arrives, a new thread is started to execute the +handler. This means it doesn't neatly fit into the condition "no +threads blocked on pending I/O or on timers". And if we did shoehorn it +into that definition then we would not look for deadlocks if there were +any signal handlers registered, and we would still end up with no +runnable threads after skipping deadlock detection, which violates the +post-condition that there be runnable threads. So the solution was that +after deadlock detection, if there are still no runnable threads and +there are registered signal handlers then we conclude we must wait for +a signal to be received -- which will start a thread and thus we will +end up with runnable threads. But of course this is horrible: we have +entangled two features far too tightly: deadlock detection with a weird +-- and platform specific -- kind of I/O. + +The modern design +----------------- + +A change of perspective is required. Instead of thinking of conditions +in which there must be a deadlock, we simply look for deadlocks in such +a way in which we will eventually find deadlocks if they exist. A +benefit of this approach is that we can find deadlocks that the simple +approach cannot. For example we can find deadlocks when there unrelated +threads blocked on I/O or timers (see issue #26408). + +The question is when to run GC it its more expensive deadlock detection +mode. We obviously do not want to do it too frequently. The design +choice is to do it during idle GC, at least sometimes. Idle GC is only +run some time after a capability goes idle. This is a good opportunity. +We know there are no runnable threads on the capability, so there +*might* be a deadlock, and when there's nothing else to do is also a +good moment to do a more expensive GC. + +The idle GC is controlled by the RecentActivity status, which +progresses through 4 stages: yes, maybe_no, inactive, done_gc. We only +invoke a deadlock-detecting major GC in the inactive state. We get into +the inactive state when: + * the timer tick goes off + * we were already in the maybe_no state (which itself requires no + activity on any capability for a whole timer tick) + * idle GC is enabled + * it's been long enough since the most recent idle GC. +This timer tick also wakes up the I/O manager to ensue we get back to +the scheduler, and thus to scheduleDetectDeadlock. + +Note that this means that deadlock detection is disabled if users +disable idle GC (by setting +RTS -I0). Historically, idle GC was not +used by default in the non-threaded RTS, but the modern design relies +on it, so it is enabled by default in all cases. + +But if idle GC is enabled, then if there is a full system deadlock then +eventually we will run a major GC with deadlock detection and detect +and resolve the deadlock. It is not prompt. It must wait at least for +an idle GC, which by default is 0.3s after all capabilities go idle. + +Furthermore, there is no post-condition for scheduleDetectDeadlock, +because of the non-prompt "eventually" nature of the deadlock detection +design. In particular there can still be no runnable threads. In the +threaded RTS if there's no runnable threads after this we will yield the +capability, while in the non-threaded we will ask the I/O manager to +block and wait for I/O, timers or signals. +*/ + static void scheduleDetectDeadlock (Capability **pcap, Task *task) { - Capability *cap = *pcap; - /* - * Detect deadlock: when we have no threads to run, there are no - * threads blocked, waiting for I/O, or sleeping, and all the - * other tasks are waiting for work, we must have a deadlock of - * some description. - */ - if ( emptyRunQueue(cap) && !anyPendingTimeoutsOrIO(cap->iomgr) ) - { -#if defined(THREADED_RTS) - /* - * In the threaded RTS, we only check for deadlock if there - * has been no activity in a complete timeslice. This means - * we won't eagerly start a full GC just because we don't have - * any threads to run currently. - */ - if (getRecentActivity() != ACTIVITY_INACTIVE) return; -#endif - - debugTrace(DEBUG_sched, "deadlocked, forcing major GC..."); - - // Garbage collection can release some new threads due to - // either (a) finalizers or (b) threads resurrected because - // they are unreachable and will therefore be sent an - // exception. Any threads thus released will be immediately - // runnable. - scheduleDoGC (pcap, task, true/*force major GC*/, false /* Whether it is an overflow GC */, true/*deadlock detection*/, false/*nonconcurrent*/); - cap = *pcap; - // when force_major == true. scheduleDoGC sets - // recent_activity to ACTIVITY_DONE_GC and turns off the timer - // signal. + /* See Note [Deadlock detection] */ + if (getRecentActivity() == ACTIVITY_INACTIVE) { - if ( !emptyRunQueue(cap) ) return; + debugTrace(DEBUG_sched, "maybe deadlocked, forcing major GC..."); -#if defined(RTS_USER_SIGNALS) && !defined(THREADED_RTS) - /* If we have user-installed signal handlers, then wait - * for signals to arrive rather then bombing out with a - * deadlock. + /* Garbage collection can release some new threads due to + * either (a) finalizers or (b) threads resurrected because + * they are unreachable and will therefore be sent an + * exception. Any threads thus released will be immediately + * runnable. + */ + scheduleDoGC (pcap, task, + true /* force major GC */, + false /* Whether it is an overflow GC */, + true /* deadlock detection */, + false /* nonconcurrent */); + /* When force_major == true, scheduleDoGC sets recent activity to + * getRecentActivity() == ACTIVITY_DONE_GC and turns off the timer + * signal. */ - if ( RtsFlags.MiscFlags.install_signal_handlers && anyUserHandlers() ) { - debugTrace(DEBUG_sched, - "still deadlocked, waiting for signals..."); - - awaitUserSignals(); - - if (signals_pending()) { - startSignalHandlers(cap); - } - - // either we have threads to run, or we were interrupted: - ASSERT(!emptyRunQueue(cap) || getSchedState() >= SCHED_INTERRUPTING); - - return; - } -#endif } } @@ -2191,7 +2192,15 @@ forkProcess(HsStablePtr *entry // bound threads for which the corresponding Task does not // exist. truncateRunQueue(cap); - cap->n_run_queue = 0; + + // Reset and re-initialise the capability's I/O manager, + // to get the I/O manager ready again. + // + // Any threads waiting on I/O or timers should have been + // removed from I/O manager queues by deleteThread_ above. + // TODO: but we could assert that here. + freeCapabilityIOManager(cap->iomgr); + initCapabilityIOManager(cap->iomgr); // Any suspended C-calling Tasks are no more, their OS threads // don't exist now: @@ -2208,7 +2217,7 @@ forkProcess(HsStablePtr *entry cap->n_returning_tasks = 0; #endif - // Release all caps except 0, we'll use that for starting + // Release all caps except 0, we'll use that for restarting // the IO manager and running the client action below. if (cap->no != 0) { task->cap = cap; @@ -2232,7 +2241,7 @@ forkProcess(HsStablePtr *entry // like startup event, capabilities, process info etc traceTaskCreate(task, cap); - initIOManagerAfterFork(cap->iomgr, &cap); + restartIOManager(cap->iomgr, &cap); // start timer after the IOManager is initialized // (the idle GC may wake up the IOManager) @@ -2337,6 +2346,10 @@ setNumCapabilities (uint32_t new_n_capabilities USED_IF_THREADS) // the capability; we don't have to worry about GC data // structures, the nursery, etc. // + // This approach also handles threads blocked on I/O. Such threads + // remain blocked, and when I/O completes and threads become runnable + // then they are migrated away. + // for (n = new_n_capabilities; n < enabled_capabilities; n++) { getCapability(n)->disabled = true; traceCapDisable(getCapability(n)); @@ -2897,9 +2910,7 @@ interruptStgRts(void) ASSERT(getSchedState() != SCHED_SHUTTING_DOWN); setSchedState(SCHED_INTERRUPTING); interruptAllCapabilities(); -#if defined(THREADED_RTS) wakeUpRts(); -#endif } /* ----------------------------------------------------------------------------- @@ -2915,15 +2926,13 @@ interruptStgRts(void) will have interrupted any blocking system call in progress anyway. -------------------------------------------------------------------------- */ -#if defined(THREADED_RTS) void wakeUpRts(void) { - // This forces the IO Manager thread to wakeup, which will + // This forces the IO Manager to wakeup, which will // in turn ensure that some OS thread wakes up and runs the // scheduler loop, which will cause a GC and deadlock check. wakeupIOManager(); } -#endif /* ----------------------------------------------------------------------------- Deleting threads @@ -2997,7 +3006,7 @@ pushOnRunQueue (Capability *cap, StgTSO *tso) cap->n_run_queue++; } -StgTSO *popRunQueue (Capability *cap) +static StgTSO *popRunQueue (Capability *cap) { ASSERT(cap->n_run_queue > 0); StgTSO *t = cap->run_queue_hd; @@ -3017,6 +3026,45 @@ StgTSO *popRunQueue (Capability *cap) return t; } +#if defined(FORKPROCESS_PRIMOP_SUPPORTED) +static void truncateRunQueue(Capability *cap) +{ + // Can only be called by the task owning the capability. + TSAN_ANNOTATE_BENIGN_RACE(&cap->run_queue_hd, "truncateRunQueue"); + TSAN_ANNOTATE_BENIGN_RACE(&cap->run_queue_tl, "truncateRunQueue"); + TSAN_ANNOTATE_BENIGN_RACE(&cap->n_run_queue, "truncateRunQueue"); + cap->run_queue_hd = END_TSO_QUEUE; + cap->run_queue_tl = END_TSO_QUEUE; + cap->n_run_queue = 0; +} +#endif + +static void removeFromRunQueue (Capability *cap, StgTSO *tso) +{ + if (tso->block_info.prev == END_TSO_QUEUE) { + ASSERT(cap->run_queue_hd == tso); + cap->run_queue_hd = tso->_link; + } else { + setTSOLink(cap, tso->block_info.prev, tso->_link); + } + if (tso->_link == END_TSO_QUEUE) { + ASSERT(cap->run_queue_tl == tso); + cap->run_queue_tl = tso->block_info.prev; + } else { + setTSOPrev(cap, tso->_link, tso->block_info.prev); + } + tso->_link = tso->block_info.prev = END_TSO_QUEUE; + cap->n_run_queue--; + + IF_DEBUG(sanity, checkRunQueue(cap)); +} + +void promoteInRunQueue (Capability *cap, StgTSO *tso) +{ + removeFromRunQueue(cap, tso); + pushOnRunQueue(cap, tso); +} + /* ----------------------------------------------------------------------------- raiseExceptionHelper ===================================== rts/Schedule.h ===================================== @@ -39,9 +39,7 @@ void scheduleThreadOn(Capability *cap, StgWord cpu, StgTSO *tso); * * Causes an OS thread to wake up and run the scheduler, if necessary. */ -#if defined(THREADED_RTS) void wakeUpRts(void); -#endif /* raiseExceptionHelper */ StgWord raiseExceptionHelper (StgRegTable *reg, StgTSO *tso, StgClosure *exception); @@ -164,10 +162,6 @@ void appendToRunQueue (Capability *cap, StgTSO *tso); */ void pushOnRunQueue (Capability *cap, StgTSO *tso); -/* Pop the first thread off the runnable queue. - */ -StgTSO *popRunQueue (Capability *cap); - INLINE_HEADER StgTSO * peekRunQueue (Capability *cap) { @@ -184,18 +178,6 @@ emptyRunQueue(Capability *cap) return cap->n_run_queue == 0; } -INLINE_HEADER void -truncateRunQueue(Capability *cap) -{ - // Can only be called by the task owning the capability. - TSAN_ANNOTATE_BENIGN_RACE(&cap->run_queue_hd, "truncateRunQueue"); - TSAN_ANNOTATE_BENIGN_RACE(&cap->run_queue_tl, "truncateRunQueue"); - TSAN_ANNOTATE_BENIGN_RACE(&cap->n_run_queue, "truncateRunQueue"); - cap->run_queue_hd = END_TSO_QUEUE; - cap->run_queue_tl = END_TSO_QUEUE; - cap->n_run_queue = 0; -} - #endif /* !IN_STG_CODE */ #include "EndPrivate.h" ===================================== rts/Timer.c ===================================== @@ -149,11 +149,9 @@ handle_tick(int unused STG_UNUSED) setRecentActivity(ACTIVITY_INACTIVE); inter_gc_ticks_to_gc = RtsFlags.GcFlags.interIdleGCWait / RtsFlags.MiscFlags.tickInterval; -#if defined(THREADED_RTS) wakeUpRts(); // The scheduler will call stopTimer() when it has done // the GC. -#endif } else { setRecentActivity(ACTIVITY_DONE_GC); // disable timer signals (see #1623, #5991, #9105) ===================================== rts/posix/FdWakeup.c ===================================== @@ -0,0 +1,132 @@ +/* ----------------------------------------------------------------------------- + * + * (c) The GHC Team 2025 + * + * Utilities for a simple fd-based cross-thread wakeup mechanism. + * + * This is used in I/O managers, to provide a mechanism to wake them when they + * are blocked waiting on fds and timeouts. The mechanism works by including + * the read end fd into the set of fds the I/O manager waits on, and when a + * wake up is needed, the write end fd is used. + * + * This is implemented using either eventfd() or pipe(). + * + * Linux 2.6.22+ and FreeBSD 13+ support eventfd. It is a single fd with a + * 64bit counter. It uses less resources than a pipe, and is probably a tad + * faster. Using write() adds to the counter, while read() reads and resets + * it. This gives us event combining. + * + * Otherwise we use a classic unix pipe. + * + * -------------------------------------------------------------------------*/ + +#include "rts/PosixSource.h" +#include "Rts.h" + +#include "FdWakeup.h" + +#include +#include + +#ifdef HAVE_SYS_EVENTFD_H +#include +#endif + +#if !defined(HAVE_EVENTFD) \ + || (defined(HAVE_EVENTFD) && !(defined(EFD_CLOEXEC) && defined(EFD_NONBLOCK))) +static void fcntl_CLOEXEC_NONBLOCK(int fd) +{ + int res1 = fcntl(fd, F_SETFD, FD_CLOEXEC); + int res2 = fcntl(fd, F_SETFL, O_NONBLOCK); + if (RTS_UNLIKELY(res1 < 0 || res2 < 0)) { + sysErrorBelch("newFdWakeup fcntl()"); + stg_exit(EXIT_FAILURE); + } +} +#endif + +void newFdWakeup(int *wakeup_fd_r, int *wakeup_fd_w) +{ +#if defined(HAVE_EVENTFD) + int wakeup_fd; +#if defined(EFD_CLOEXEC) && defined(EFD_NONBLOCK) + wakeup_fd = eventfd(0, EFD_CLOEXEC | EFD_NONBLOCK); +#else + wakeup_fd = eventfd(0, 0); + if (wakeup_fd >= 0) fcntl_CLOEXEC_NONBLOCK(wakeup_fd); +#endif + if (RTS_UNLIKELY(wakeup_fd < 0)) { + sysErrorBelch("newFdWakeup eventfd()"); + stg_exit(EXIT_FAILURE); + } + /* eventfd uses the same fd for each end */ + *wakeup_fd_r = wakeup_fd; + *wakeup_fd_w = wakeup_fd; +#else + int pipefd[2]; + int res; + res = pipe(pipefd); + if (RTS_UNLIKELY(res < 0)) { + sysErrorBelch("newFdWakeup pipe"); + stg_exit(EXIT_FAILURE); + } + fcntl_CLOEXEC_NONBLOCK(pipefd[0]); + fcntl_CLOEXEC_NONBLOCK(pipefd[1]); + *wakeup_fd_r = pipefd[0]; /* read end */ + *wakeup_fd_w = pipefd[1]; /* write end */ +#endif +} + +void closeFdWakeup(int wakeup_fd_r, int wakeup_fd_w) +{ +#if defined(HAVE_EVENTFD) + ASSERT(wakeup_fd_r == wakeup_fd_w); + close(wakeup_fd_r); +#else + ASSERT(wakeup_fd_r != wakeup_fd_w); + close(wakeup_fd_r); + close(wakeup_fd_w); +#endif +} + +void sendFdWakeup(int wakeup_fd_w) +{ + int res; +#if defined(HAVE_EVENTFD) + uint64_t val = 1; + res = write(wakeup_fd_w, &val, 8); +#else + unsigned char buf = 1; + res = write(wakeup_fd_w, &buf, 1); +#endif + if (RTS_UNLIKELY(res < 0)) { + /* Unlikely the pipe buffer will fill, but it would not be an error. */ + if (errno == EAGAIN) return; + sysErrorBelch("sendFdWakeup write"); + stg_exit(EXIT_FAILURE); + } +} + +void collectFdWakeup(int wakeup_fd_r) +{ + int res; +#if defined(HAVE_EVENTFD) + uint64_t buf; + /* eventfd combines events into one counter, so a single read is enough */ + res = read(wakeup_fd_r, &buf, 8); +#else + /* Drain the pipe buffer. Multiple wakeup notifications could + * have been sent before we have a chance to collect them. + */ + uint64_t buf; + do { + res = read(wakeup_fd_r, &buf, 8); + } while (res == 8); +#endif + if (RTS_UNLIKELY(res < 0)) { + /* After the first pipe read, it could block */ + if (errno == EAGAIN) return; + sysErrorBelch("collectFdWakeup read"); + stg_exit(EXIT_FAILURE); + } +} ===================================== rts/posix/FdWakeup.h ===================================== @@ -0,0 +1,27 @@ +/* ----------------------------------------------------------------------------- + * + * (c) The GHC Team 2025 + * + * Utilities for a simple fd-based cross-thread wakeup mechanism. + * + * This is used in I/O managers, to provide a mechanism to wake them when they + * are blocked waiting on fds and timeouts. The mechanism works by including + * the read end fd into the set of fds the I/O manager waits on, and when a + * wake up is needed, the write end fd is used. + * + * Prototypes for functions in FdWakeup.c + * + * -------------------------------------------------------------------------*/ + +#pragma once + +#include "BeginPrivate.h" + +void newFdWakeup(int *fd_r, int *fd_w); +void closeFdWakeup(int fd_r, int fd_w); + +void sendFdWakeup(int fd_w); +void collectFdWakeup(int fd_r); + +#include "EndPrivate.h" + ===================================== rts/posix/MIO.c ===================================== @@ -0,0 +1,163 @@ +/* ----------------------------------------------------------------------------- + * + * (c) The GHC Team, 1998-2005 + * + * Signal processing / handling. + * + * ---------------------------------------------------------------------------*/ + +#include "rts/PosixSource.h" +#include "Rts.h" + +#include "Schedule.h" +#include "RtsUtils.h" +#include "Prelude.h" +#include "ThreadLabels.h" + +#include "MIO.h" +#include "IOManager.h" +#include "IOManagerInternals.h" + +#if defined(HAVE_ERRNO_H) +# include +#endif + +#include +#include + +// Here's the pipe into which we will send our signals +static int io_manager_wakeup_fd = -1; +static int timer_manager_control_wr_fd = -1; +// TODO: Eliminate these globals. Put then into the CapIOManager, but the +// problem is these are shared across all caps, not per cap. + +#define IO_MANAGER_WAKEUP 0xff +#define IO_MANAGER_DIE 0xfe +#define IO_MANAGER_SYNC 0xfd + +void setTimerManagerControlFd(int fd) { + RELAXED_STORE(&timer_manager_control_wr_fd, fd); +} + +void +setIOManagerWakeupFd (int fd) +{ + // only called when THREADED_RTS, but unconditionally + // compiled here because GHC.Event.Control depends on it. + SEQ_CST_STORE(&io_manager_wakeup_fd, fd); +} + +#if defined(THREADED_RTS) +void timerManagerNotifySignal(int sig, siginfo_t *info) +{ + StgWord8 buf[sizeof(siginfo_t) + 1]; + int r; + + buf[0] = sig; + if (info == NULL) { + // info may be NULL on Solaris (see #3790) + memset(buf+1, 0, sizeof(siginfo_t)); + } else { + memcpy(buf+1, info, sizeof(siginfo_t)); + } + + int timer_control_fd = RELAXED_LOAD(&timer_manager_control_wr_fd); + if (0 <= timer_control_fd) + { + r = write(timer_control_fd, buf, sizeof(siginfo_t)+1); + if (r == -1 && errno == EAGAIN) { + errorBelch("lost signal due to full pipe: %d\n", sig); + } + } + + // If the IO manager hasn't told us what the FD of the write end + // of its pipe is, there's not much we can do here, so just ignore + // the signal.. +} +#endif + + +/* ----------------------------------------------------------------------------- + * Wake up at least one IO or timer manager HS thread. + * -------------------------------------------------------------------------- */ +void +ioManagerWakeup (void) +{ + int r; + const int wakeup_fd = SEQ_CST_LOAD(&io_manager_wakeup_fd); + // Wake up the IO Manager thread by sending a byte down its pipe + if (wakeup_fd >= 0) { +#if defined(HAVE_EVENTFD) + StgWord64 n = (StgWord64)IO_MANAGER_WAKEUP; + r = write(wakeup_fd, (char *) &n, 8); +#else + StgWord8 byte = (StgWord8)IO_MANAGER_WAKEUP; + r = write(wakeup_fd, &byte, 1); +#endif + /* N.B. If the TimerManager is shutting down as we run this + * then there is a possibility that our first read of + * io_manager_wakeup_fd is non-negative, but before we get to the + * write the file is closed. If this occurs, io_manager_wakeup_fd + * will be written into with -1 (GHC.Event.Control does this prior + * to closing), so checking this allows us to distinguish this case. + * To ensure we observe the correct ordering, we declare the + * io_manager_wakeup_fd as volatile. + * Since this is not an error condition, we do not print the error + * message in this case. + */ + if (r == -1 && SEQ_CST_LOAD(&io_manager_wakeup_fd) >= 0) { + sysErrorBelch("ioManagerWakeup: write"); + } + } +} + +#if defined(THREADED_RTS) +void +ioManagerDie (void) +{ + StgWord8 byte = (StgWord8)IO_MANAGER_DIE; + uint32_t i; + int r; + + { + // Shut down timer manager + const int fd = RELAXED_LOAD(&timer_manager_control_wr_fd); + if (0 <= fd) { + r = write(fd, &byte, 1); + if (r == -1) { sysErrorBelch("ioManagerDie: write"); } + RELAXED_STORE(&timer_manager_control_wr_fd, -1); + } + } + + { + // Shut down IO managers + for (i=0; i < getNumCapabilities(); i++) { + const int fd = RELAXED_LOAD(&getCapability(i)->iomgr->control_fd); + if (0 <= fd) { + r = write(fd, &byte, 1); + if (r == -1) { sysErrorBelch("ioManagerDie: write"); } + RELAXED_STORE(&getCapability(i)->iomgr->control_fd, -1); + } + } + } +} + +void +ioManagerStartCap (Capability **cap) +{ + rts_evalIO(cap,ensureIOManagerIsRunning_closure,NULL); +} + +void +ioManagerStart (void) +{ + // Make sure the IO manager thread is running + Capability *cap; + if (SEQ_CST_LOAD(&timer_manager_control_wr_fd) < 0 || SEQ_CST_LOAD(&io_manager_wakeup_fd) < 0) { + cap = rts_lock(); + ioManagerStartCap(&cap); + rts_unlock(cap); + } +} +#endif + ===================================== rts/posix/MIO.h ===================================== @@ -0,0 +1,30 @@ +/* ----------------------------------------------------------------------------- + * + * (c) The GHC Team, 1998-2005 + * + * Signal processing / handling. + * + * ---------------------------------------------------------------------------*/ + +#pragma once + +#include "IOManager.h" + +#if defined(HAVE_SIGNAL_H) +# include +#endif + +#include "BeginPrivate.h" + +/* Communicating with the IO manager thread (see GHC.Conc). + */ +void ioManagerWakeup (void); +#if defined(THREADED_RTS) +void ioManagerDie (void); +void ioManagerStart (void); +void ioManagerStartCap (/* inout */ Capability **cap); + +void timerManagerNotifySignal(int sig, siginfo_t *info); +#endif + +#include "EndPrivate.h" ===================================== rts/posix/Poll.c ===================================== @@ -41,6 +41,7 @@ #include "IOManagerInternals.h" #include "Timeout.h" +#include "FdWakeup.h" /****************************************************************************** @@ -107,8 +108,9 @@ timeout (if any) as the poll() timeout parameter. The CapIOManager structure for this I/O manager contains: ClosureTable aiop_table; - struct pollfd *aiop_poll_table; + struct pollfd *aiop_poll_table, *full_poll_table; StgTimeoutQueue *timeout_queue; + int wakeup_fd_r, wakeup_fd_w; We also support the Linux-specific ppoll API which supports higher resolution time delays -- nanoseconds rather than milliseconds as in classic poll(). It @@ -117,6 +119,15 @@ also allows the signal mask to be adjusted, but we do not make use of this. int ppoll(struct pollfd *fds, nfds_t nfds, const struct timespec *tmo_p, const sigset_t *sigmask); +We have both aiop_poll_table and full_poll_table. This is to cope with needing +to wait on the special extra file descriptor wakeup_fd_r. This fd is used to +support waking the I/O manager when we are blocked in a poll call. This +requires waiting on an extra fd that has no corresponding entry in the +aiop_table. To manage this quirk, we alias the aiop_poll_table to be the tail +of the full_poll_table and have the first entry of the full_poll_table be the +wakeup_fd_r. This means the aiop_poll_table indicies match up exactly with the +aiop_table, but still allows the full_poll_table to have an extra entry. + ******************************************************************************/ /* Forward declarations */ @@ -129,8 +140,31 @@ static void reportPollError(int res, nfds_t nfds) STG_NORETURN; void initCapabilityIOManagerPoll(CapIOManager *iomgr) { initClosureTable(&iomgr->aiop_table, ClosureTableCompact); - iomgr->aiop_poll_table = NULL; iomgr->timeout_queue = emptyTimeoutQueue(); + + newFdWakeup(&iomgr->wakeup_fd_r, &iomgr->wakeup_fd_w); + + iomgr->full_poll_table = stgMallocBytes(sizeof(struct pollfd) /* size 1 */, + "initCapabilityIOManagerPoll"); + iomgr->full_poll_table[0] = (struct pollfd) { + .fd = iomgr->wakeup_fd_r, + .events = POLLIN, + .revents = 0 + }; + iomgr->aiop_poll_table = iomgr->full_poll_table+1; /* hence empty */ +} + + +void freeCapabilityIOManagerPoll(CapIOManager *iomgr) +{ + stgFree(iomgr->full_poll_table); + closeFdWakeup(iomgr->wakeup_fd_r, iomgr->wakeup_fd_w); +} + + +void wakeupIOManagerPoll(CapIOManager *iomgr) +{ + sendFdWakeup(iomgr->wakeup_fd_w); } @@ -227,13 +261,6 @@ static void ioCancel(CapIOManager *iomgr, StgAsyncIOOp *aiop) } -bool anyPendingTimeoutsOrIOPoll(CapIOManager *iomgr) -{ - return !isEmptyTimeoutQueue(iomgr->timeout_queue) - || !isEmptyClosureTable(&iomgr->aiop_table); -} - - static void notifyIOCompletion(CapIOManager *iomgr, StgAsyncIOOp *aiop) { ASSERT(aiop->outcome != IOOpOutcomeInFlight); @@ -275,7 +302,7 @@ static void notifyIOCompletion(CapIOManager *iomgr, StgAsyncIOOp *aiop) } -static void processIOCompletions(CapIOManager *iomgr, int ncompletions) +static bool processIOCompletions(CapIOManager *iomgr, int ncompletions) { /* The scheme we use with poll is that we have a dense poll table, and a * corresponding table that maps to the closure table index. The poll @@ -285,6 +312,19 @@ static void processIOCompletions(CapIOManager *iomgr, int ncompletions) */ debugTrace(DEBUG_iomanager, "processIOCompletions(ncompletions = %d)", ncompletions); + + bool wakeup; + /* If the wakeup_fd_r is ready, collect it */ + if (iomgr->full_poll_table[0].revents) { + ASSERT(iomgr->full_poll_table[0].fd == iomgr->wakeup_fd_r); + collectFdWakeup(iomgr->wakeup_fd_r); + ncompletions--; + wakeup = true; + debugTrace(DEBUG_iomanager, "Received wakeup in poll I/O manager."); + } else { + wakeup = false; + } + struct pollfd *aiop_poll_table = iomgr->aiop_poll_table; int n = ncompletions; int i = 0; @@ -337,11 +377,14 @@ static void processIOCompletions(CapIOManager *iomgr, int ncompletions) i++; } } + return wakeup; } void pollCompletedTimeoutsOrIOPoll(CapIOManager *iomgr) { + ASSERT(iomgr->aiop_poll_table == iomgr->full_poll_table+1); + if (!isEmptyTimeoutQueue(iomgr->timeout_queue)) { Time now = getProcessElapsedTime(); processTimeoutCompletions(iomgr, now); @@ -349,20 +392,20 @@ void pollCompletedTimeoutsOrIOPoll(CapIOManager *iomgr) if (!isEmptyClosureTable(&iomgr->aiop_table)) { - nfds_t nfds = sizeClosureTable(&iomgr->aiop_table); + nfds_t nfds = sizeClosureTable(&iomgr->aiop_table) + 1; /* Poll for I/O readiness, without waiting. */ #if defined(HAVE_DECL_PPOLL) && HAVE_DECL_PPOLL == 1 /* We could use poll here, since we use no timeout, but for consistency we use the same syscall as at the other call site. */ struct timespec tv = (struct timespec) { .tv_sec = 0, .tv_nsec = 0 }; - int res = ppoll(iomgr->aiop_poll_table, nfds, &tv, NULL); + int res = ppoll(iomgr->full_poll_table, nfds, &tv, NULL); debugTrace(DEBUG_iomanager, "ppoll(nfds = %d, timeout.sec = 0, timeout.nsec = 0) = %d", nfds, res); #else - int res = poll(iomgr->aiop_poll_table, nfds, 0); + int res = poll(iomgr->full_poll_table, nfds, 0); debugTrace(DEBUG_iomanager, "poll(nfds = %d, timeout_ms = 0) = %d", @@ -385,11 +428,19 @@ void pollCompletedTimeoutsOrIOPoll(CapIOManager *iomgr) reportPollError(res, nfds); } } + +#if defined(RTS_USER_SIGNALS) + startPendingSignalHandlers(iomgr->cap); +#endif } void awaitCompletedTimeoutsOrIOPoll(CapIOManager *iomgr) { + bool wakeup = false; /* got woken up via wakeupIOManager */ + + ASSERT(iomgr->aiop_poll_table == iomgr->full_poll_table+1); + /* Loop until we've woken up some threads. This loop is needed because the * poll() timing isn't accurate, we sometimes sleep for a while but not * long enough to wake up a thread in a threadDelay. Or we may need to @@ -397,9 +448,10 @@ void awaitCompletedTimeoutsOrIOPoll(CapIOManager *iomgr) * that select() supports. */ do { - /* There is either pending I/O or pending timers. */ - ASSERT(!isEmptyTimeoutQueue(iomgr->timeout_queue) || - !isEmptyClosureTable(&iomgr->aiop_table)); + /* We do /not/ require that there be pending I/O or pending timers. + * If there is neither, it's because the scheduler wants us to wait + * on signals only. + */ Time now = getProcessElapsedTime(); processTimeoutCompletions(iomgr, now); @@ -422,9 +474,9 @@ void awaitCompletedTimeoutsOrIOPoll(CapIOManager *iomgr) #endif /* Check for I/O readiness, possibly waiting. */ - nfds_t nfds = sizeClosureTable(&iomgr->aiop_table); + nfds_t nfds = sizeClosureTable(&iomgr->aiop_table) + 1; #if defined(HAVE_DECL_PPOLL) && HAVE_DECL_PPOLL == 1 - int res = ppoll(iomgr->aiop_poll_table, nfds, timeout_ns, NULL); + int res = ppoll(iomgr->full_poll_table, nfds, timeout_ns, NULL); debugTrace(DEBUG_iomanager, "ppoll(nfds = %d, timeout.sec = %d, timeout.nsec = %d) = %d", @@ -432,7 +484,7 @@ void awaitCompletedTimeoutsOrIOPoll(CapIOManager *iomgr) timeout_ns == NULL ? 0 : timeout_ns->tv_nsec, res); #else - int res = poll(iomgr->aiop_poll_table, nfds, timeout_ms); + int res = poll(iomgr->full_poll_table, nfds, timeout_ms); debugTrace(DEBUG_iomanager, "poll(nfds = %d, timeout_ms = %d) = %d", @@ -454,17 +506,16 @@ void awaitCompletedTimeoutsOrIOPoll(CapIOManager *iomgr) } else if (res > 0) { int ncompletions = res; ASSERT(ncompletions <= (int)nfds); - processIOCompletions(iomgr, ncompletions); + wakeup = processIOCompletions(iomgr, ncompletions); } else if (errno == EINTR) { - /* We got interrupted by a signal. In the non-threaded RTS, if the - * signal is one of ours we need to return to the scheduler to let - * it handle it. Otherwise we would loop and keep waiting for I/O - * or timeouts, meaning we would block for a long time before the - * signal is serviced. - */ + /* We got interrupted by a signal. */ + #if defined(RTS_USER_SIGNALS) - if (startPendingSignalHandlers(iomgr->cap)) break; + /* Start any corresponding user signal handlers. If any, the run + * queue will become non-empty and we will drop out of the loop. + */ + startPendingSignalHandlers(iomgr->cap); #endif /* We can also be interrupted by the shutdown signal handler, which @@ -479,6 +530,7 @@ void awaitCompletedTimeoutsOrIOPoll(CapIOManager *iomgr) } } while (emptyRunQueue(iomgr->cap) + && !wakeup && (getSchedState() == SCHED_RUNNING)); } @@ -508,13 +560,17 @@ static bool enlargeTables(CapIOManager *iomgr) bool ok = enlargeClosureTable(iomgr->cap, &iomgr->aiop_table, newcapacity); if (RTS_UNLIKELY(!ok)) return false; - /* Update the auxiliary aiop_poll_table to match */ - struct pollfd *aiop_poll_table; - aiop_poll_table = stgReallocBytes(iomgr->aiop_poll_table, - sizeof(struct pollfd) * newcapacity, - "Poll.c: enlargeTables"); - iomgr->aiop_poll_table = aiop_poll_table; + /* Update the auxiliary aiop_poll_table to match. The full_poll_table is + * one bigger than the aiop_poll_table, since it has an extra entry at the + * front for wakeup_fd_r, with no corresponding aiop. */ + iomgr->full_poll_table = + stgReallocBytes(iomgr->full_poll_table, + sizeof(struct pollfd) * (newcapacity+1), + "Poll.c: enlargeTables"); + iomgr->aiop_poll_table = iomgr->full_poll_table+1; + /* Initialise the new part of the aiop_poll_table */ + struct pollfd *aiop_poll_table = iomgr->aiop_poll_table; for (int i = oldcapacity; i < newcapacity; i++) { aiop_poll_table[i] = (struct pollfd) { .fd = -1, ===================================== rts/posix/Poll.h ===================================== @@ -17,6 +17,8 @@ #if defined(IOMGR_ENABLED_POLL) void initCapabilityIOManagerPoll(CapIOManager *iomgr); +void freeCapabilityIOManagerPoll(CapIOManager *iomgr); +void wakeupIOManagerPoll(CapIOManager *iomgr); /* Synchronous I/O and timer operations */ bool syncIOWaitReadyPoll(CapIOManager *iomgr, StgTSO *tso, @@ -29,7 +31,6 @@ bool asyncIOWaitReadyPoll(CapIOManager *iomgr, StgAsyncIOOp *aiop, void asyncIOCancelPoll(CapIOManager *iomgr, StgAsyncIOOp *aiop); /* Scheduler operations */ -bool anyPendingTimeoutsOrIOPoll(CapIOManager *iomgr); void pollCompletedTimeoutsOrIOPoll(CapIOManager *iomgr); void awaitCompletedTimeoutsOrIOPoll(CapIOManager *iomgr); ===================================== rts/posix/Select.c ===================================== @@ -12,7 +12,7 @@ #include "rts/PosixSource.h" #include "Rts.h" -#include "Signals.h" +#include "RtsSignals.h" #include "Schedule.h" #include "Prelude.h" #include "RaiseAsync.h" @@ -22,6 +22,7 @@ #include "IOManagerInternals.h" #include "Stats.h" #include "GetTime.h" +#include "FdWakeup.h" # if defined(HAVE_SYS_SELECT_H) # include @@ -54,6 +55,25 @@ #define TimeToLowResTimeRoundUp(t) (t) #endif +void initCapabilityIOManagerSelect(CapIOManager *iomgr) +{ + iomgr->blocked_queue_hd = END_TSO_QUEUE; + iomgr->blocked_queue_tl = END_TSO_QUEUE; + iomgr->sleeping_queue = END_TSO_QUEUE; + + newFdWakeup(&iomgr->wakeup_fd_r, &iomgr->wakeup_fd_w); +} + +void freeCapabilityIOManagerSelect(CapIOManager *iomgr) +{ + closeFdWakeup(iomgr->wakeup_fd_r, iomgr->wakeup_fd_w); +} + +void wakeupIOManagerSelect(CapIOManager *iomgr) +{ + sendFdWakeup(iomgr->wakeup_fd_w); +} + /* * Return the time since the program started, in LowResTime, * rounded down. @@ -225,6 +245,7 @@ awaitCompletedTimeoutsOrIOSelect(CapIOManager *iomgr, bool wait) bool seen_bad_fd = false; struct timeval tv, *ptv; LowResTime now; + bool wakeup = false; /* got woken up via wakeupIOManager */ IF_DEBUG(scheduler, debugBelch("scheduler: checking for threads blocked on I/O"); @@ -252,6 +273,13 @@ awaitCompletedTimeoutsOrIOSelect(CapIOManager *iomgr, bool wait) FD_ZERO(&rfd); FD_ZERO(&wfd); + /* We're always interested in our wakeup fd */ + { + int fd = iomgr->wakeup_fd_r; + maxfd = (fd > maxfd) ? fd : maxfd; + FD_SET(fd, &rfd); + } + for(tso = iomgr->blocked_queue_hd; tso != END_TSO_QUEUE; tso = next) { @@ -346,16 +374,11 @@ awaitCompletedTimeoutsOrIOSelect(CapIOManager *iomgr, bool wait) } } - /* We got a signal; could be one of ours. If so, we need - * to start up the signal handler straight away, otherwise - * we could block for a long time before the signal is - * serviced. - */ #if defined(RTS_USER_SIGNALS) - if (RtsFlags.MiscFlags.install_signal_handlers && signals_pending()) { - startSignalHandlers(iomgr->cap); - return; /* still hold the lock */ - } + /* Start any corresponding user signal handlers. If any, the run + * queue will become non-empty and we will drop out of the loop. + */ + startPendingSignalHandlers(iomgr->cap); #endif /* we were interrupted, return to the scheduler immediately. @@ -376,6 +399,13 @@ awaitCompletedTimeoutsOrIOSelect(CapIOManager *iomgr, bool wait) } } + /* If the wakeup_fd_r is ready, collect it */ + if (FD_ISSET(iomgr->wakeup_fd_r, &rfd)) { + collectFdWakeup(iomgr->wakeup_fd_r); + wakeup = true; + debugTrace(DEBUG_iomanager, "Received wakeup in select I/O manager."); + } + /* Step through the waiting queue, unblocking every thread that now has * a file descriptor in a ready state. */ @@ -458,7 +488,8 @@ awaitCompletedTimeoutsOrIOSelect(CapIOManager *iomgr, bool wait) } } while (wait && getSchedState() == SCHED_RUNNING - && emptyRunQueue(iomgr->cap)); + && emptyRunQueue(iomgr->cap) + && !wakeup); } #endif /* IOMGR_ENABLED_SELECT */ ===================================== rts/posix/Select.h ===================================== @@ -15,6 +15,10 @@ typedef StgWord LowResTime; LowResTime getDelayTarget (HsInt us); +void initCapabilityIOManagerSelect(CapIOManager *iomgr); +void freeCapabilityIOManagerSelect(CapIOManager *iomgr); +void wakeupIOManagerSelect(CapIOManager *iomgr); + void awaitCompletedTimeoutsOrIOSelect(CapIOManager *iomgr, bool wait); #include "EndPrivate.h" ===================================== rts/posix/Signals.c ===================================== @@ -9,21 +9,13 @@ #include "rts/PosixSource.h" #include "Rts.h" -#include "Schedule.h" #include "RtsSignals.h" -#include "Signals.h" -#include "IOManager.h" +#include "posix/Signals.h" #include "RtsUtils.h" +#include "Schedule.h" #include "Prelude.h" -#include "Ticker.h" #include "ThreadLabels.h" -#include "Libdw.h" - -/* TODO: eliminate this include. This file should be about signals, not be - * part of an I/O manager implementation. The code here that are really part - * of an I/O manager should be moved into an appropriate I/O manager impl. - */ -#include "IOManagerInternals.h" +#include "MIO.h" #if defined(alpha_HOST_ARCH) # if defined(linux_HOST_OS) @@ -45,10 +37,6 @@ # include #endif -#if defined(HAVE_EVENTFD_H) -# include -#endif - #if defined(HAVE_TERMIOS_H) #include #endif @@ -107,6 +95,11 @@ freeSignalHandlers(void) { #endif } +void finiUserSignals(void) +{ + /* nothing */ +}; + /* ----------------------------------------------------------------------------- * Allocate/resize the table of signal handlers. * -------------------------------------------------------------------------- */ @@ -134,110 +127,6 @@ more_handlers(int sig) nHandlers = sig + 1; } -// Here's the pipe into which we will send our signals -static int io_manager_wakeup_fd = -1; -static int timer_manager_control_wr_fd = -1; - -#define IO_MANAGER_WAKEUP 0xff -#define IO_MANAGER_DIE 0xfe -#define IO_MANAGER_SYNC 0xfd - -void setTimerManagerControlFd(int fd) { - RELAXED_STORE(&timer_manager_control_wr_fd, fd); -} - -void -setIOManagerWakeupFd (int fd) -{ - // only called when THREADED_RTS, but unconditionally - // compiled here because GHC.Event.Control depends on it. - SEQ_CST_STORE(&io_manager_wakeup_fd, fd); -} - -/* ----------------------------------------------------------------------------- - * Wake up at least one IO or timer manager HS thread. - * -------------------------------------------------------------------------- */ -void -ioManagerWakeup (void) -{ - int r; - const int wakeup_fd = SEQ_CST_LOAD(&io_manager_wakeup_fd); - // Wake up the IO Manager thread by sending a byte down its pipe - if (wakeup_fd >= 0) { -#if defined(HAVE_EVENTFD) - StgWord64 n = (StgWord64)IO_MANAGER_WAKEUP; - r = write(wakeup_fd, (char *) &n, 8); -#else - StgWord8 byte = (StgWord8)IO_MANAGER_WAKEUP; - r = write(wakeup_fd, &byte, 1); -#endif - /* N.B. If the TimerManager is shutting down as we run this - * then there is a possibility that our first read of - * io_manager_wakeup_fd is non-negative, but before we get to the - * write the file is closed. If this occurs, io_manager_wakeup_fd - * will be written into with -1 (GHC.Event.Control does this prior - * to closing), so checking this allows us to distinguish this case. - * To ensure we observe the correct ordering, we declare the - * io_manager_wakeup_fd as volatile. - * Since this is not an error condition, we do not print the error - * message in this case. - */ - if (r == -1 && SEQ_CST_LOAD(&io_manager_wakeup_fd) >= 0) { - sysErrorBelch("ioManagerWakeup: write"); - } - } -} - -#if defined(THREADED_RTS) -void -ioManagerDie (void) -{ - StgWord8 byte = (StgWord8)IO_MANAGER_DIE; - uint32_t i; - int r; - - { - // Shut down timer manager - const int fd = RELAXED_LOAD(&timer_manager_control_wr_fd); - if (0 <= fd) { - r = write(fd, &byte, 1); - if (r == -1) { sysErrorBelch("ioManagerDie: write"); } - RELAXED_STORE(&timer_manager_control_wr_fd, -1); - } - } - - { - // Shut down IO managers - for (i=0; i < getNumCapabilities(); i++) { - const int fd = RELAXED_LOAD(&getCapability(i)->iomgr->control_fd); - if (0 <= fd) { - r = write(fd, &byte, 1); - if (r == -1) { sysErrorBelch("ioManagerDie: write"); } - RELAXED_STORE(&getCapability(i)->iomgr->control_fd, -1); - } - } - } -} - -void -ioManagerStartCap (Capability **cap) -{ - rts_evalIO(cap,ensureIOManagerIsRunning_closure,NULL); -} - -void -ioManagerStart (void) -{ - // Make sure the IO manager thread is running - Capability *cap; - if (SEQ_CST_LOAD(&timer_manager_control_wr_fd) < 0 || SEQ_CST_LOAD(&io_manager_wakeup_fd) < 0) { - cap = rts_lock(); - ioManagerStartCap(&cap); - rts_unlock(cap); - } -} -#endif - #if !defined(THREADED_RTS) #define N_PENDING_HANDLERS 16 @@ -245,6 +134,10 @@ ioManagerStart (void) siginfo_t pending_handler_buf[N_PENDING_HANDLERS]; siginfo_t *next_pending_handler = pending_handler_buf; +static inline bool signals_pending(void) { + return (next_pending_handler != pending_handler_buf); +} + #endif /* THREADED_RTS */ /* ----------------------------------------------------------------------------- @@ -260,31 +153,9 @@ generic_handler(int sig USED_IF_THREADS, void *p STG_UNUSED) { #if defined(THREADED_RTS) - - StgWord8 buf[sizeof(siginfo_t) + 1]; - int r; - - buf[0] = sig; - if (info == NULL) { - // info may be NULL on Solaris (see #3790) - memset(buf+1, 0, sizeof(siginfo_t)); - } else { - memcpy(buf+1, info, sizeof(siginfo_t)); - } - - int timer_control_fd = RELAXED_LOAD(&timer_manager_control_wr_fd); - if (0 <= timer_control_fd) - { - r = write(timer_control_fd, buf, sizeof(siginfo_t)+1); - if (r == -1 && errno == EAGAIN) { - errorBelch("lost signal due to full pipe: %d\n", sig); - } - } - - // If the IO manager hasn't told us what the FD of the write end - // of its pipe is, there's not much we can do here, so just ignore - // the signal.. - + //TODO: This calls MIO directly. We should go via IOManager API. + // The IOManager API should be extended to cover signals. + timerManagerNotifySignal(sig, info); #else /* not THREADED_RTS */ /* Can't call allocate from here. Probably can't call malloc @@ -346,22 +217,6 @@ unblockUserSignals(void) sigprocmask(SIG_SETMASK, &savedSignals, NULL); } -bool -anyUserHandlers(void) -{ - return n_haskell_handlers != 0; -} - -#if !defined(THREADED_RTS) -void -awaitUserSignals(void) -{ - while (!signals_pending() && getSchedState() == SCHED_RUNNING) { - pause(); - } -} -#endif - /* ----------------------------------------------------------------------------- * Install a Haskell signal handler. * @@ -468,11 +323,13 @@ stg_sig_install(int sig, int spi, void *mask) #if !defined(THREADED_RTS) void -startSignalHandlers(Capability *cap) +startPendingSignalHandlers(Capability *cap) { siginfo_t *info; int sig; + if (!signals_pending()) return; + blockUserSignals(); while (next_pending_handler != pending_handler_buf) { @@ -484,7 +341,7 @@ startSignalHandlers(Capability *cap) continue; // handler has been changed. } - info = stgMallocBytes(sizeof(siginfo_t), "startSignalHandlers"); + info = stgMallocBytes(sizeof(siginfo_t), "startPendingSignalHandlers"); // freed by runHandler memcpy(info, next_pending_handler, sizeof(siginfo_t)); ===================================== rts/posix/Signals.h ===================================== @@ -2,43 +2,19 @@ * * (c) The GHC Team, 1998-2005 * - * Signal processing / handling. + * POSIX signal processing / handling. + * + * Most of the API for this is common between POSIX and Win32 console events. + * The common part of the API lives in RtsSignals.h. * * ---------------------------------------------------------------------------*/ #pragma once -#if defined(HAVE_SIGNAL_H) -# include -#endif - #include "Ticker.h" #include "BeginPrivate.h" -bool anyUserHandlers(void); - -#if !defined(THREADED_RTS) && defined(RTS_USER_SIGNALS) -extern siginfo_t pending_handler_buf[]; -extern siginfo_t *next_pending_handler; -#define signals_pending() (next_pending_handler != pending_handler_buf) -void startSignalHandlers(Capability *cap); -#endif - void install_vtalrm_handler(int sig, TickProc handle_tick); -/* Communicating with the IO manager thread (see GHC.Conc). - * - * TODO: these I/O manager things are not related to signals and ought to live - * elsewhere, e.g. in a module specifically for the I/O manager. - */ -void ioManagerWakeup (void); -#if defined(THREADED_RTS) -void ioManagerDie (void); -void ioManagerStart (void); -void ioManagerStartCap (/* inout */ Capability **cap); -#endif - -extern StgInt *signal_handlers; - #include "EndPrivate.h" ===================================== rts/rts.cabal ===================================== @@ -569,6 +569,7 @@ library wasm/OSThreads.c wasm/JSFFI.c wasm/JSFFIGlobals.c + posix/FdWakeup.c posix/Select.c posix/Poll.c posix/Timeout.c @@ -581,6 +582,8 @@ library posix/Ticker.c posix/OSMem.c posix/OSThreads.c + posix/FdWakeup.c + posix/MIO.c posix/Poll.c posix/Select.c posix/Signals.c ===================================== rts/win32/AwaitEvent.c ===================================== @@ -14,6 +14,7 @@ * */ #include "Rts.h" +#include "RtsSignals.h" #include "RtsFlags.h" #include "Schedule.h" #include "IOManager.h" @@ -41,14 +42,9 @@ awaitCompletedTimeoutsOrIOWin32(Capability *cap, bool wait) awaitRequests(wait); workerWaitingForRequests = false; - // If a signal was raised, we need to service it - // XXX the scheduler loop really should be calling - // startSignalHandlers(), but this is the way that posix/Select.c - // does it and I'm feeling too paranoid to refactor it today --SDM - if (stg_pending_events != 0) { - startSignalHandlers(cap); - return; - } + // If a signal was raised, we need to service it. This will typically + // start a thread, which will cause us to drop out of the loop. + startPendingSignalHandlers(cap); // The return value from awaitRequests() is a red herring: ignore // it. Return to the scheduler if !wait, or ===================================== rts/win32/ConsoleHandler.c ===================================== @@ -154,29 +154,18 @@ unblockUserSignals(void) } -/* - * Function: awaitUserSignals() - * - * Wait for the next console event. Currently a NOP (returns immediately.) - */ -void awaitUserSignals(void) -{ - return; -} - - #if !defined(THREADED_RTS) /* - * Function: startSignalHandlers() + * Function: startPendingSignalHandlers() * - * Run the handlers associated with the stacked up console events. Console - * event delivery is blocked for the duration of this call. + * If there are any queued up console events, run the handlers associated with + * them. Console event delivery is blocked for the duration of this call. */ -void startSignalHandlers(Capability *cap) +void startPendingSignalHandlers(Capability *cap) { StgStablePtr handler; - if (console_handler < 0) { + if (stg_pending_events <= 0 || console_handler < 0) { return; } ===================================== rts/win32/ConsoleHandler.h ===================================== @@ -23,36 +23,6 @@ * thread, which starts up the handler. See ThrIOManager.c. */ -/* - * Function: signals_pending() - * - * Used by the RTS to check whether new signals have been 'recently' reported. - * If so, the RTS arranges for the delivered signals to be handled by - * de-queueing them from their table, running the associated Haskell - * signal handler. - */ -extern StgInt stg_pending_events; - -#define signals_pending() ( stg_pending_events > 0) - -/* - * Function: anyUserHandlers() - * - * Used by the Scheduler to decide whether its worth its while to stick - * around waiting for an external signal when there are no threads - * runnable. A console handler is used to handle termination events (Ctrl+C) - * and isn't considered a 'user handler'. - */ -#define anyUserHandlers() (false) - -/* - * Function: startSignalHandlers() - * - * Run the handlers associated with the queued up console events. Console - * event delivery is blocked for the duration of this call. - */ -extern void startSignalHandlers(Capability *cap); - /* * Function: rts_waitConsoleHandlerCompletion() * @@ -62,10 +32,3 @@ extern void startSignalHandlers(Capability *cap); extern int rts_waitConsoleHandlerCompletion(void); #endif /* THREADED_RTS */ - -/* - * Function: finiUserSignals() - * - * Tear down and shut down user signal processing. - */ -extern void finiUserSignals(void); ===================================== testsuite/tests/rts/T26408.hs ===================================== @@ -0,0 +1,43 @@ +import Control.Concurrent +import Control.Concurrent.STM +import Control.Exception +import Control.Monad + +-- | Test to make sure that deadlock detection works even when there are other +-- unrelated threads that are blocked on I\/O or timeouts. +-- Historically however this did affect things in the non-threaded RTS which +-- would only do deadlock detection if there were no runnable threads /and/ +-- no pending I\/O. See https://gitlab.haskell.org/ghc/ghc/-/issues/26408 +main :: IO () +main = do + + -- Set up two threads that are deadlocked on each other + aDone <- newTVarIO False + bDone <- newTVarIO False + let blockingThread theirDone ourDone = + atomically $ do + done <- readTVar theirDone + guard done + writeTVar ourDone True + _ <- forkIO (blockingThread bDone aDone) + _ <- forkIO (blockingThread aDone bDone) + + -- Set up another thread that is blocked on a long timeout. + -- + -- We use a timeout rather than I/O as it's more portable, whereas I/O waits + -- are different between posix and windows I/O managers. + -- + -- One gotcha is that when the timeout completes then the deadlock will be + -- detected again (since the bug is about I/O or timeouts masking deadlock + -- detection). So for a reliable test the timeout used here must be longer + -- than the test framework's own timeout. So we use maxBound, and we adjust + -- the test framework's timeout to be short (see run_timeout_multiplier). + _ <- forkIO (threadDelay maxBound) + + -- Wait on the deadlocked threads to terminate. We now expect that the threads + -- that are deadlocked are detected as such and an exception is raised. + -- Note that if this fails, the test itself will effectively deadlock and + -- will rely on the test framework's timeout. + atomically $ do + status <- mapM readTVar [aDone, bDone] + guard (or status) ===================================== testsuite/tests/rts/T26408.stderr ===================================== @@ -0,0 +1,3 @@ +T26408: Uncaught exception ghc-internal:GHC.Internal.IO.Exception.BlockedIndefinitelyOnSTM: + +thread blocked indefinitely in an STM transaction ===================================== testsuite/tests/rts/all.T ===================================== @@ -657,6 +657,8 @@ test('T22859', omit_ways(llvm_ways)], compile_and_run, ['-with-rtsopts -A8K']) +test('T26408', [exit_code(1), run_timeout_multiplier(0.1)], compile_and_run, ['']) + # These tests need access to the internal RTS headers. # TODO: there is probably some cleaner way to do this, and it should probably # be guarded for in-tree tests, since it cannot work against an arbitrary View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/97cb8f22c15ee24b8486d59382a1a01... -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/97cb8f22c15ee24b8486d59382a1a01... You're receiving this email because of your account on gitlab.haskell.org.