Duncan Coutts pushed to branch wip/io-manager-deadlock-detection at Glasgow Haskell Compiler / GHC

Commits:

• c2f6dcd4
  by Sasha Bogicevic at 2026-07-20T10:31:56+02:00

  Improve error messages for invalid record wildcards
  
  Record wildcard hints are now shown in more contexts and include
  constructor arity; matching with `..` on a fieldless constructor
  now produces a dedicated error message.
  
  Fixes #21101
  

• 4c02e76b
  by Duncan Coutts at 2026-07-21T10:37:21-04:00

  Mark test T27105 as fragile, citing issue #27522
  
  Scheduler fairness is fine, except when it isn't. And it isn't on CI
  machines surprisingly often! See the issue for details.
  

• 43dd2b15
  by Recursion Ninja at 2026-07-21T17:09:53-04:00

  Resolving many TTG related orphan type-class instances
  
  This is part a technical debt removal effort made possible now
  that separating out the AST via TTG has come to a close.
  
  As the AST in 'L.H.S' has been incrementally separated from the GHC internals,
  there are many accumulated orphan instance of 'Binary', 'NFData', 'Outputable',
  and 'Uniquable'. The orphan instance of data-types from within 'L.H.S' have had
  their orphan instances moved to either:
    1. The module which defines the data-type
    2. The module which defines the type-class;
       i.e. moving an orphan 'Binary' instance to 'GHC.Utils.Binary'
  
  Orphan instances resolved (37):
  
    | Data-type            | Resolved instance(s)       | Former orphan module(s)   |
    | -------------------- | -------------------------- | ------------------------- |
    | Role                 | Binary, NFData, Outputable | GHC.Core.Coercion.Axiom   |
    | SrcStrictness        | Binary, NFData, Outputable | GHC.Core.DataCon          |
    | SrcUnpackedness      | Binary, NFData, Outputable | GHC.Core.DataCon          |
    | Fixity               | Binary, Outputable         | GHC.Hs.Basic              |
    | FixityDirection      | Binary, Outputable         | GHC.Hs.Basic              |
    | LexicalFixity        | Outputable                 | GHC.Hs.Basic              |
    | CCallTarget          | NFData                     | GHC.Hs.Decls.Foreign      |
    | CType                | NFData                     | GHC.Hs.Decls.Foreign      |
    | Header               | NFData                     | GHC.Hs.Decls.Foreign      |
    | OverlapMode          | Binary, NFData             | GHC.Hs.Decls.Overlap      |
    | WithHsDocIdentifiers | NFData, Outputable         | GHC.Hs.Doc                |
    | HsDocString          | NFData                     | GHC.Hs.DocString          |
    | HsDocStringChunk     | Binary, Outputable         | GHC.Hs.DocString          |
    | HsDocStringDecorator | Binary, Outputable         | GHC.Hs.DocString          |
    | NamespaceSpecifier   | Outputable                 | GHC.Hs.ImpExp             |
    | ForAllTyFlag         | Binary, NFData, Outputable | GHC.Hs.Specificity        |
    | Specificity          | Binary, NFData             | GHC.Hs.Specificity        |
    | PromotionFlag        | Binary, Outputable         | GHC.Types.Basic           |
    | FieldLabelString     | Outputable, Uniquable      | GHC.Types.FieldLabel      |
    | InlinePragma         | Binary                     | GHC.Types.InlinePragma    |
  
  -------------------------
  Metric Decrease:
      hard_hole_fits
  -------------------------
  
  Closes #21262, #27469
  

• ab9ab895
  by Cheng Shao at 2026-07-21T17:10:53-04:00

  rts: always use StgInt to represent cost center id
  
  Currently cost center id is modeled as `Int` and it should be `StgInt`
  uniformly in the RTS, hence this patch. Fixes #27524.
  

• 94d8f83b
  by Cheng Shao at 2026-07-22T11:30:40-04:00

  hadrian: clean up stale cabal package flags in the tree
  
  This patch cleans up stale cabal package flags in the tree and related
  hadrian/autoconf logic. Closes #27474.
  
  Co-authored-by: Codex <codex@openai.com>
  

• 0bf1d8c9
  by Sasha Bogicevic at 2026-07-22T11:31:21-04:00

  parser: don't suggest ImportQualifiedPost when it is already enabled
  
  -Wprepositive-qualified-module unconditionally attached a hint to
  enable ImportQualifiedPost, even when the extension was already on
  (as it is by default under GHC2021). Record the extension's state in
  the PsWarnImportPreQualified diagnostic and drop the hint when it is
  already enabled.
  
  Fixes #27380
  

• 4adc95d8
  by Duncan Coutts at 2026-07-22T22:25:40+01:00

  Make signal handling be a respondibility of the I/O manager(s)
  
  Previously it was scattered between I/O managers and the scheduler, and
  especially the scheduler's deadlock detection.
  
  Previously the scheduler would poll for pending signals each iteration
  of the scheduler loop. The scheduler also had some hairy signal
  functionality in the deadlock detection: in the non-threaded RTS (only)
  if there were still no threads running after deadlock detection then it
  would block waiting for signals.
  
  But signals can and (in my opinion) should be thought of as just a funny
  kind of I/O, and thus should be a responsibility of the I/O manager.
  
  So now we have the I/O managers poll for signals when they are polling
  for I/O completion (and removing the separate poll in the scheduler).
  And when I/O managers block waiting for I/O then they now also start
  signal handlers if they get interrupted by a signal. Crucially, if there
  is no pending I/O or timers, the awaitCompletedTimeoutsOrIO will still
  block waiting for signals.
  
  This patch puts us into an intermediate state: it temporarily breaks
  deadlock detection in the non-threaded RTS. The waiting on I/O currently
  happens before deadlock detection. This means we'll now wait forever on
  signals before doing deadlock detection. We need to move waiting after
  deadlock detection. We'll do that in a later patch.
  

• 2d9e8f4c
  by Duncan Coutts at 2026-07-22T22:25:40+01:00

  Clean up the RTS internal signal handling API
  
  Now that the I/O manager is responsible for signals, we can simplify the
  API we present for signal handling.
  
  We now just need startPendingSignalHandlers, which is called from the
  I/O managers. We can get rid of awaitUserSignals. We also don't need
  RtsSignals.h to re-export the platform-specific posix/Signals.h or
  win32/ConsoleHandler.h
  
  We can also hide more of the implementation of signals. Less has to be
  exposed in posix/Signals.h or win32/ConsoleHandler.h. Indeed,
  posix/Signals.h becomes empty and we remove it. Partly this is because
  we don't need inline functions (or macros) in the interface.
  
  Also remove signal_handlers from RTS ABI exported symbols list. It does
  not appear to have any users in the core libs, and its really an
  internal implementation detail. It should not be exposed unless it's
  really necessary.
  

• 8ca2b02f
  by Duncan Coutts at 2026-07-22T22:25:40+01:00

  In the scheduler, move I/O blocking after deadlock detection
  
  To make deadlock detection effective in the non-threaded RTS when there
  are deadlocked threads and other unrelated threads waiting on I/O, we
  need to arrange to do deadlock detection before we block in scheduler
  to wait on I/O.
  
  The solution is to:
  
  1. adjust scheduleFindWork, which runs before deadlock detection, to
     only poll for I/O and not block; and
  2. add a step after deadlock detection to wait on I/O if there are
     still no threads to run (and there's any I/O or timeouts outstanding)
  
  The scheduleCheckBlockedThreads is now so simple that it made more sense
  to inline it into scheduleFindWork.
  

• 6ce87e56
  by Duncan Coutts at 2026-07-22T22:25:40+01:00

  Remove bogus anyPendingTimeoutsOrIO guard from scheduleDetectDeadlock
  
  The deadlock detection was only invoked if both of these conditions
  hold:
  1. the run queue is empty
  2. there is no pending I/O or timeouts
  
  The second condition is unnecessary. The deadlock detection mechanism
  can find deadlocks even if there are other threads waiting on I/O or
  timers. Having this extra condition means that we fail to detect
  blocked threads if there are any threads waiting on I/O or timers.
  
  Part of fixing issue #26408
  

• 493186c8
  by Duncan Coutts at 2026-07-22T22:25:40+01:00

  Don't consider pending I/O for early context switch optimisation
  
  Context switches are normally initiated by the timer signal. If however
  the user specifies "context switch as often as possible", with +RTS -C0
  then the scheduler arranges for an early context switch (when it's just
  about to run a Haskell thread).
  
  Context switching very often is expensive, so as an optimisation there
  cases where we do not arrange an early context switch:
  1. if there's no other threads to run
  2. if there is no pending I/O or timers
  
  This patch eliminates case 2, leaving only case 1.
  
  The rationale is as follows. The use of this was inconsistent across
  platforms and threaded/non-threaded RTS ways. It only worked on the
  non-threaded RTS and on Windows only worked for the win32-legacy I/O
  manager. On all other combinations anyPendingTimeoutsOrIO would always
  return false. The fact that nobody noticed and complained about this
  inconsistency suggests that the feature is not relied upon.
  
  If however it turns out that applications do rely on this, then the
  proper thing to do is not to restore this check, but to add a new I/O
  manager hint function that returns if there is any pending events that
  are likely to happen *soon*: for example timeouts expiring within one
  timeslice, or I/O waits on things likely to complete soon like disk I/O,
  but not for example socket/pipe I/O.
  
  The motivation to avoid this use of anyPendingTimeoutsOrIO is to
  allow us to eliminate anyPendingTimeoutsOrIO entirely. All other uses
  of this are just guards on {await,poll}CompletedTimeoutsOrIO and
  the guards can safely be folded into those functions. This will better
  cope with some I/O managers having no proper implementation of
  anyPendingTimeoutsOrIO.
  
  Ultimately this will let us simplify the scheduler which currently has
  to have special #ifdef mingw32_HOST_OS cases to cope with the lack of a
  working anyPendingTimeoutsOrIO for some Windows I/O managers
  

• 8f14388f
  by Duncan Coutts at 2026-07-22T22:25:40+01:00

  Remove anyPendingTimeoutsOrIO guarding {poll,await}CompletedTimeoutsOrIO
  
  Previously the API of the I/O manager used a two step process: check
  anyPendingTimeoutsOrIO and then call {poll,await}CompletedTimeoutsOrIO.
  This was primarily there as a performance thing, to cheaply check if we
  need to do anything.
  
  And then because anyPendingTimeoutsOrIO existed, it was used for other
  things too. We have now eliminated the other uses, and are just left
  with the performance pattern.
  
  But this was problematic because not all I/O managers correctly
  implement anyPendingTimeoutsOrIO (specifically the win32 ones), and now
  that we also make I/O managers responsible for signals then we need to
  poll/await even if there is no pending I/O or timeouts. If there is no
  pending I/O or timeouts then poll/await needs to degenerate to just
  waiting forever for any signals.
  

• e7215008
  by Duncan Coutts at 2026-07-22T22:25:40+01:00

  Remove anyPendingTimeoutsOrIO, it is no longer used
  
  And this avoids the problems arising from the win32 I/O managers having
  had a bogus implementation.
  

• 91de2781
  by Duncan Coutts at 2026-07-22T22:25:40+01:00

  Remove second scheduler call to awaitCompletedTimeoutsOrIO
  
  Previously awaitCompletedTimeoutsOrIO was called both before and after
  deadlock detection in the scheduler. The reason for that was that the
  win32 I/O managers had a bogus implementation of anyPendingTimeoutsOrIO
  and this was used to guard the call of awaitCompletedTimeoutsOrIO prior
  to deadlock detection. This meant the first call site was never actually
  called when using the win32 I/O managers. This was the reason for the
  second call: the first one was never used. What a mess.
  
  So now we have a simple design in the scheduler:
  1. poll for completed I/O, timers or signals
  2. if no runnable threads: do deadlock detection
  3. if still no runnable threads: block waiting for I/O, timers or
     signals.
  

• b010b8cf
  by Duncan Coutts at 2026-07-22T22:25:41+01:00

  Lift emptyRunQueue guard out of scheduleDetectDeadlock
  
  this improved the clarity of the logic when reading the scheduler code.
  

• ec67de61
  by Duncan Coutts at 2026-07-22T22:25:41+01:00

  Make non-threaded deadlock detection also rely on idle GC
  
  Only do deadlock detection GC when idle GC kicks in. This also relies on
  using wakeUpRts, so now do this unconditionally. Previously wakeUpRts
  was for the threaded rts only.
  

• a0b28ada
  by Duncan Coutts at 2026-07-22T22:25:41+01:00

  Enable idle GC by default on non-threaded RTS
  
  The behaviour is now uniform between the threaded and non-threaded RTS
  ways. The deadlock detection now relies on idle GC for both threaded
  and non-threaded ways. Previously deadlock detection did not rely on
  idle GC for the non-threaded way.
  
  Also tweak test T7275 to account for idle GC. This test's output is
  sensitive to the number of major GCs run. Since this commit enables idle
  GC for the non-threaded RTS, for this test that increases the number of
  major GCs, since the test program is frequently idle for more than 300ms.
  

• fed6f5fc
  by Duncan Coutts at 2026-07-22T22:25:41+01:00

  Fix state of idle GC control vars with +RTS -V0
  
  Currently when the user uses +RTS -I0, then doIdleGC is set to false.
  But if the master tick interval -V is set to 0 then the idleGCDelayTime
  was being set to 0 but doIdleGC was not being set to false, which is
  inconsistent, and almost certainly buggy.
  

• 3215f347
  by Duncan Coutts at 2026-07-22T22:25:41+01:00

  Add a long Note [Deadlock detection]
  
  It describes the historical and modern designs and their trade-offs.
  
  The point is we've now unified the code for deadlock detection between
  the threaded and non-threaded ways, by changing the non-threaded to
  follow the same design as the threaded.
  

• 56ef9726
  by Duncan Coutts at 2026-07-22T22:25:41+01:00

  Add a test for deadlock detection, issue #26408
  

• db5f2658
  by Duncan Coutts at 2026-07-22T22:25:41+01:00

  Update the user guide with the revised idle GC behaviour
  
  i.e. it's now not just for the threaded RTS, but general.
  
  Also document the fact that disabling idle GC also disables deadlock
  detection.
  
  And add a changelog entry.
  

90 changed files:

• + changelog.d/21101
• + changelog.d/27380
• + changelog.d/idle-gc-and-deadlock-detection
• compiler/GHC/Core/Coercion/Axiom.hs
• compiler/GHC/Core/DataCon.hs
• compiler/GHC/Hs/Basic.hs
• compiler/GHC/Hs/Decls/Overlap.hs
• compiler/GHC/Hs/Doc.hs
• compiler/GHC/Hs/DocString.hs
• compiler/GHC/Hs/ImpExp.hs
• − compiler/GHC/Hs/Specificity.hs
• compiler/GHC/Hs/Utils.hs
• compiler/GHC/Parser/Errors/Ppr.hs
• compiler/GHC/Parser/Errors/Types.hs
• compiler/GHC/Parser/PostProcess.hs
• compiler/GHC/Rename/Env.hs
• compiler/GHC/Rename/Names.hs
• compiler/GHC/Rename/Pat.hs
• compiler/GHC/Tc/Errors/Ppr.hs
• compiler/GHC/Tc/Errors/Types.hs
• compiler/GHC/Types/Basic.hs
• compiler/GHC/Types/FieldLabel.hs
• compiler/GHC/Types/Fixity.hs
• compiler/GHC/Types/ForeignCall.hs
• compiler/GHC/Types/GREInfo.hs
• compiler/GHC/Types/Hint.hs
• compiler/GHC/Types/Hint/Ppr.hs
• compiler/GHC/Types/InlinePragma.hs
• compiler/GHC/Types/Unique.hs
• compiler/GHC/Types/Var.hs
• compiler/GHC/Utils/Binary.hs
• compiler/GHC/Utils/Outputable.hs
• compiler/Language/Haskell/Syntax/Basic.hs
• compiler/Language/Haskell/Syntax/Decls/Foreign.hs
• compiler/Language/Haskell/Syntax/Doc.hs
• compiler/Language/Haskell/Syntax/Extension.hs
• compiler/Language/Haskell/Syntax/ImpExp.hs
• compiler/Language/Haskell/Syntax/Specificity.hs
• compiler/ghc.cabal.in
• configure.ac
• distrib/configure.ac.in
• docs/users_guide/runtime_control.rst
• hadrian/cfg/system.config.host.in
• hadrian/cfg/system.config.target.in
• hadrian/src/Oracles/Flag.hs
• hadrian/src/Rules/Generate.hs
• hadrian/src/Settings/Packages.hs
• m4/fp_check_pthreads.m4
• rts/IOManager.c
• rts/IOManager.h
• rts/Linker.c
• rts/Profiling.c
• rts/RtsFlags.c
• rts/RtsSignals.h
• rts/RtsStartup.c
• rts/RtsSymbols.c
• rts/Schedule.c
• rts/Schedule.h
• rts/Timer.c
• rts/Trace.c
• rts/Trace.h
• rts/eventlog/EventLog.c
• rts/eventlog/EventLog.h
• rts/posix/Poll.c
• rts/posix/Poll.h
• rts/posix/Select.c
• rts/posix/Signals.c
• − rts/posix/Signals.h
• rts/rts.cabal
• rts/win32/AwaitEvent.c
• rts/win32/ConsoleHandler.c
• rts/win32/ConsoleHandler.h
• testsuite/tests/concurrent/should_run/T27105.hs
• testsuite/tests/concurrent/should_run/all.T
• testsuite/tests/count-deps/CountDepsParser.stdout
• + testsuite/tests/module/T27380.hs
• + testsuite/tests/module/T27380.stderr
• testsuite/tests/module/all.T
• testsuite/tests/module/mod184.stderr
• testsuite/tests/profiling/should_run/Makefile
• + testsuite/tests/rename/should_fail/T21101.hs
• + testsuite/tests/rename/should_fail/T21101.stderr
• testsuite/tests/rename/should_fail/T9815.stderr
• testsuite/tests/rename/should_fail/T9815b.stderr
• testsuite/tests/rename/should_fail/T9815bghci.stderr
• testsuite/tests/rename/should_fail/T9815ghci.stderr
• testsuite/tests/rename/should_fail/all.T
• + testsuite/tests/rts/T26408.hs
• + testsuite/tests/rts/T26408.stderr
• testsuite/tests/rts/all.T

The diff was not included because it is too large.